Spark-shell version 1.6.2

Hello Durga garu,

The Spark-shell version is 1.6.2 on itversity labs wherein the cloudera VM for exam is 1.2.1.

Is it ok to practice on 1.6.2 for the exam or do we need to install 1.2.1 like how you have shown in your tutorials “Hadoop Certification - CCA - Setup Spark 1.2.1 on Quickstart VM” ?

Please let me know.

Hi @Seetham_Allu,

Please try below:
Login to Web Console

Set the environment variables
So that spark can discover hadoop

export YARN_CONF_DIR=/etc/hadoop/conf/
export HADOOP_CONF_DIR=/etc/hadoop/conf/

Launch scala spark shell

/usr/spark1.2.1/bin/spark-shell

Launch PySpark

/usr/spark1.2.1/bin/pyspark

@Seetham_Allu - Refer below link

1 Like