Do i need to use /bin/pyspark mentioned in youtube videos

certifications
#1

Do i need to use /bin/pyspark to initiate pyspark shell when i use bigdata labs as mentioned in youtube video for Cloudera VM

#2

You just need to type pyspark

#3

so no need to make spark 1.2.1 ?

#4

@Tarun_Das that depends on which version of Cloudera Quickstart VM you are using.

#5

actually what confused me that video says now we can ignore set up of spark 1.2.1 as data frames no longer needed in certification

#6

Yes, if you are preparing for CCA175 certification then you need to practice and provide exam on Spark 1.2.1. In the videos Durga sir has demonstrated videos in Cloudera Quickstart VM 5.4 in which it has been built with Spark 1.3.0. The technical features differences between these 2 versions are DataFrames added in Spark 1.3.0. So if you don’t want to configure for 1.2.1 in cloudera, then you can use Spark 1.3.0 as long as you are not using DataFrames.

P.S: DataFrames is out of scope for certification.

1 Like
#7

Thank you for the information