@PrakashP - Congratulations!! And the materials you provided in github are fantastic. Really appreciate it. This will help a lot. Also, could you please mention “where/how” did you find(should we memorize it?) the avro package to use during your exam? I did try the following in Itversity labs and not working:
pyspark2 --master yarn
I found the above pavro ackage details here and used it for initiating it as above:
x = spark.read.csv(“some data”)
pyspark.sql.utils.AnalysisException: u’Failed to find data source: com.databricks.spark.avro. Please find an Avro package at http://spark.apache.org/third-party-projects.html;’
I am confused if this is an issue with the lab setup that i am using or this wont work in exam also. Hope you throw some light on this. Thanks in advance!!