Spark latest versions

Spark latest versions
5.0 1


Hello, Can we use versions other than 1.6.2 (2.0+) here? If yes, how?

Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster



The default version of spark in labs is 1.6.2 if you want to use spark2 then you need to use the below command before launching the spark-shell

spark-shell \
--master yarn \
--deploy-mode client \
--conf spark.ui.port=12335 \
--num-executors 2 \
--executor-memory 512M