How to launch spark2 in big data labs?



Itversity labs have two versions of Spark. The default version of spark is 1.6.2 and if you want use spark2 you need to use below command before launching spark-shell

spark-shell \
--master yarn \
--deploy-mode client \
--conf spark.ui.port=12335 \
--num-executors 2 \
--executor-memory 512M

Common issues we encounter on our state of the art Big Data Cluster with Hadoop, Spark and many others -

This is to simplify our support process so that we can answer technical issues as well.

How to start spark2 shell
Spark2-shell not working
Spark 2 access issue in labs

Working with Spark 2.0 or higher in the labs