Error: value saveAsTextFile is not a member of org.apache.spark.sql.Dataset[String]

apache-spark
scala

#1

I was trying to save a dataframe into a file with gzip compression using the below command:
dataFile.map(x=> x(0)+"\t"+x(1)+"\t"+x(2)+"\t"+x(3)).saveAsTextFile("/user/cloudera/problem5/text-gzip-compress",classOf[org.apache.hadoop.io.compress.GzipCodec]);

Although it is syntactically correct - it throws me an error saying :
error: value saveAsTextFile is not a member of org.apache.spark.sql.Dataset[String]

Can someone help me by letting me know the issue?


#2

@Rakesh_Ram dataFile is a type of DataSet. Create file as RDD and try the query.