Tracking URL in Spark


#1

Hello,

I am studying lecture 63 in CCA175.

The lecture shows there is a tracking URL if I enter spark-shell, my finding is there is no tracking url if I run:
spark-shell

But if I run something like this:
spark-shell --master yarn --conf spark.ui.port=12654 --num-executors 1 --executor-memory 1024M

Then there is a tracking URL,

Can anyone explain to me why spark-shell doesn’t generate a tracking url while the second command does.

Much appreciated, thanks in advance.


#2

Hi @paslechoix,

To my understanding tracking URL is available in yarn mode.


#3

Thank you, are you implicating that running “spark-shell” with no parameters will NOT start Spark on Yarn?


#4

@paslechoix:
Will not use YARN resources but local resources. Thanks


#5

Thank you for the explanation.