How to save as textfile with '\t' delimiter?

Hi all,
I got a question to save output in textformat using ‘\t’ delimiter. I am using pyspark2 and Data Frames. how to do this? In udemy practice test course durga sir answer was df.write.csv(‘filepath’,sep=’\t’). can we do like this? In certification will they consider is they get .csv extension in output file?

I used following code
cust.select(concat(‘customer_id’,lit("\t"),‘customer_fname’,lit("\t"),‘customer_lname’)).show()

and got this output:

                                   9327  Donna   Smith|

| 9328 Mary Perez|
| 9329 Eugene Powell|
| 9330 Mary Conley|
| 9331 Donna Smith|
| 9332 Mary Jordan|
| 9333 Angela Mills|
| 9334 Mary Johnston|
| 9335 Joseph Smith|

is this correct?

Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster

hi, to my knowledge if they ask text format then it should be in .txt file not .csv.

  1. for csv format what you have written is correct only.

  2. for text format write like this. say your columns in dataframe are customer_id, customer_fname, customer_lname:

df.rdd.map(lambda x: ( str(x[0] ) + ‘\t’ + x[1] + ‘\t’ + x[2] )).saveAsTextFile(“filepath”)

1 Like