Unable to Access Spark UI

I am unable to see web UI for Pyspark.

I tried below command,
pyspark --master yarn --conf spark.ui.port=12888

I didn’t see any message saying service ‘SparkUI’ started at port 12888.

Seems like loglevel INFO is not set, that’s why it is not displaying any info.
At the same time, I have one more query how to set loglevel to INFO before logging to pyspark.

I know how to do it when we log into pyspark.

Kindly assist on this.


Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster

Your (itversity) Hadoop and spark cluster is a 5 node cluster. not 13. MISLEADING.