Can anyone confirm if the spark-csv package is available on cca175 exam? I have read the spark-avro package is so the spark-csv should be too, but wanted to confirm from anyone.
Main reason is if it is not available what is the best way to read/write (specifically write) dataframes as textFile/csv format via pyspark if they specify they need the header in the output?
For read i can get around without it with sc.textFile then import Row and map each value and specify data type but how would one write as textFile with column name without using the spark-csv package?
I guess one could vim into the first output file and edit in a header however please let me know the most efficient method if anyone knows.