Spark-scala data loading issue


#1

Hi ,

I am getting the below issue while trying to import the package “” import com.databricks.spark.avro._;" using spark scala to load the data as dataframes.

image

Can anyone please help me in this issue.

Regards,’
Vivek


#2

@vivek_kumar2

Use the below commands

These 2 commands will solve your problem:

  1. launch spark by using below command:–
    spark-shell --master yarn --packages com.databricks:spark-avro_2.10:2.0.1 --conf spark.ui.port=12567

  2. once you are in spark shell, import avro package like below:–
    import com.databricks.spark.avro._;