Spark csv store

Storing rdd as CSV files

Usually people using to store data
dataframe.write.format(“com.databricks.spark.csv”).option(“header”, “true”).option(“codec”, “org.apache.hadoop.io.compress.GzipCodec”).save(“file.csv”);

I got below code in learing spark pdf. In real time is this preferable to do coding like below.

pandaLovers.map(person => List(person.name, person.favoriteAnimal).toArray)
.mapPartitions{people =>
val stringWriter = new StringWriter();
val csvWriter = new CSVWriter(stringWriter);
csvWriter.writeAll(people.toList)
Iterator(stringWriter.toString)
}.saveAsTextFile(outFile);

1 Like