Skip to content

Spark Writing Data Disk

Lecture 15 : How to write data on the disk?🔗

image

image

Modes to write data🔗

image

Create three files image

  write_df = read_df.repartition(3).write.format("csv")\
    .option("header", "True")\
    .mode("overwrite")\  # Using .mode() instead of .option() for overwrite mode
    .option("path", "/FileStore/tables/Write_Data/")\
    .save()