Save Dataframe to CSV, Parquet, Avro, TXT

pyspark

#1

Hi,

How in this lab can I save files as follows:

  • From dataframe to compressed parquet file.
  • From dataframe to csv

I have tried many ways of writing files and it just worked using:

  • saveAsTextFile
  • write.json
  • write.format().save()

Thanks


#2

@juan_zapata

watch this video about different file formats