Spark Group By

Lecture 23: Group By In Spark

Sample Data

Questions

Salary per department using groupBy()

Where do we use window functions?

Suppose we need to find out the percentage of total salary from a particular dept that the person is earning. we can use window function to specify the total salary per department in the particular record itself like I've shown below.

This way we dont need to perform a join.

Grouping by two columns