URGENT! - Read/Write Avro Files in Pyspark Using Python DURING CCA-175

cca-175

#1

Hi,
I have an urgent concern regarding the examination (attempting with python) which is due for me in next 2 days.
Question- Importing avro packages in pyspark during the examination?
I have been getting mixed answers from people, some say we have to load it manually and some say it is pre-loaded in the environment.
I have been using following script all this while on itversity labs-
pyspark --master yarn –packages com.databricks:spark-avro_2.10:2.0.1 --conf spark.ui.port=xxxxxx

But unfortunately, few of the folks couldn’t make it work during the examination and wasted their attempt & money. I’m really getting nervous as many of the people recently have received multiple questions around avro data packages. I will really appreciate an early reply from anyone who has an absolute solution to this problem.

Thanks in advance!


#2

Team - Im also looking for pyspark packages for avro and csv… please provide some insights… thanks