SPARK2 - Does not launch 2 executors by default

apache-spark

#1

Hi,

I launched spark2-shell using the below command.
spark-shell --master yarn --conf spark.ui.port=15973

and the spark context UI to verify whether 2 Executors are created but there were no spark executors created and only the driver was created.

Can you help me understand why is this behaviour?

Thanks
Kaarthick


#2

Spark2 default executors in the lab is 0 you can launch with below command to view the excecutors

spark-shell --master yarn-client --num-executors 2