Tracking URL in Spark



I am studying lecture 63 in CCA175.

The lecture shows there is a tracking URL if I enter spark-shell, my finding is there is no tracking url if I run:

But if I run something like this:
spark-shell --master yarn --conf spark.ui.port=12654 --num-executors 1 --executor-memory 1024M

Then there is a tracking URL,

Can anyone explain to me why spark-shell doesn’t generate a tracking url while the second command does.

Much appreciated, thanks in advance.


Hi @paslechoix,

To my understanding tracking URL is available in yarn mode.


Thank you, are you implicating that running “spark-shell” with no parameters will NOT start Spark on Yarn?


Will not use YARN resources but local resources. Thanks


Thank you for the explanation.