CCA-175 : Setup Cloudera quick start VM


This question is based on the video “Hadoop Certification - CCA - Setup Spark 1.2.1 on Quickstart VM”.

I am setting up Spark 1.4.0 version on my Cloudera VM. One of the steps is to copy configuration files from the location /etc/spark/conf/ to Spark location as Spark is not recognizing the HDFS files.Below is the list of files i see in my VM:

[cloudera@quickstart conf]$ ls -ltr /etc/spark/conf/
total 52
-rwxr-xr-x 1 root root 4209 Jun 16 2016
-rw-r–r-- 1 root root 1292 Jun 16 2016 spark-defaults.conf.template
-rw-r–r-- 1 root root 865 Jun 16 2016 slaves.template
-rw-r–r-- 1 root root 6671 Jun 16 2016
-rw-r–r-- 1 root root 2025 Jun 16 2016
-rw-r–r-- 1 root root 1105 Jun 16 2016 fairscheduler.xml.template
-rw-r–r-- 1 root root 987 Jun 16 2016
-rwxr-xr-x 1 root root 6029 Jun 16 2016
-rw-r–r-- 1 root root 1292 Jun 16 2016 spark-defaults.conf
-rw-rw-r-- 1 root root 92 Aug 10 2016 slaves

But, Durga has completely different files under the directory /etc/spark/conf/. His VM file list is attached to this post.

Appreciate all your help.


Once you start services, the folder will be refreshed.

Also versions are different and you will see some differences even after starting Spark based services.

Itversity, Thank you for the reply. What services do i need ti start? Spark based services?

Yes, you need to start spark based services along with HDFS and YARN.