I am trying to save the data as avro file. I am using spark2-shell. Below is the code.
spark2-shell --packages com.databricks:spark-avro_2.11:3.2.0
val words = sc.textFile("/public/randomtextwriter")
val word = words.flatMap(x => x.split(" “)).map(x => (x,1)).reduceByKey((total,value) => total+value , 8).toDF()
i am getting Error
org.apache.spark.sql.AnalysisException: Failed to find data source: avro. Please find an Avro package at http://spark.apache.org/third-party-projects.html;
Can anyone help me in this. Thanks!