Spark Transformations
Lecture 17: Dataframe Transformations in Spark Part 1🔗
Data gets stored in Row() format in the form of bytes
Columns are expressions. Expressions are set of transformations on more than one value in a record.
Ways to select values / columns🔗
Column Manipulations
Other methods
selectExpr
Aliasing Columns
Lecutre 18 : Dataframe Transformations in Spark Part II🔗
filter() / where() no difference🔗
Multiple filter conditions🔗
Literals in spark🔗
Used to pass same value in all the columns
Adding Columns🔗
If the column already exists then it gets overwritten.