Facing problem while saving file without compression (spark-scala)


Dear All I got the answer…:smile:


Dear All,
I have saved dataframe into textfile with GzipCodec compression while solving first problem
code is given below
above code works fine .When I tried to solve next problem in which I need to save data using hiveQL

val result=hql.sql(“select * from product_replica where product_price>100”)
then my file shows with gz.parquet extension .later I used sqlContext.setConf(“spark.sql.parquet.compression.codec”,“uncompressed”) again I saved the file but still spark force fully adding gz compression…Please help…How to remove effect of compression after using in saveASTextFile() and saveAsSequenceFile()



That’s good can you share how you resolved your issue too? It would be helpful for others who may come across the same issue.

Sunil Abhishek



Awaiting for your answer :slight_smile:



I have set the configuration over hiveQL reference