CCA 175 LAB trying to open the avro datafile format

cca175

#1

Is there no way to practise with Avro file format in Hadoop Lab, when i used pyspark 1.6 i get the below error.

java.lang.ClassNotFoundException: Failed to find data source: avro. Please use Spark package http://spark-packages.org/package/databricks/spark-avro

Command i used is

sqlContext.read.format(“avro”).load("/user/prashantpr/samplefiles/userdata1.avro")


#2

@Prashantpr Launch pyspark as below command:

pyspark --packages com.databricks:spark-avro_2.10:2.0.1 --conf spark.ui.port=12891

Try the below code to read avro file in a data frame

df = sqlContext.read.format(“com.databricks.spark.avro”).load(“file.avro”)
df.show()