Accessing Spark UI on the Cluster

Hi @itversity, I have started spark shell and the spark UI has been created at http://172.16.1.100:4045, but I am not access the UI when I paste the URL in the browser. Please let me know how to access the Spark UI.

Sorry for asking as reply to this question. I am just looking here how to post a new question. I know how to reply for a question. I am not seeing anything like ‘ask a question or post a question’.

@jeevakrishnarg - Please use + New Topic link which is present in right side corner. it will allow the update the subject, category, details and click + Create Topic.

Thanks, Got it…

@pramodvspk You are trying to access Spark UI with private(internal) ip (172.16.1.100) which is accessible from only that VM. For public access you need to use either public ip (149.56.24.210) or host-name (gw01.itversity.com). So your Spark UI can be accessible from anyone of below URLs(if your spark-shell still running on 4045 port).

  1. Using Public IP: http://149.56.24.210:4045
  2. Using Host Name: http://gw01.itversity.com:4045

Hope this helps!

1 Like

@venkatreddy-amalla, thank you

None of above are working. Anyone knows ?

@Rajib

You will get Spark UI some times referred as tracking url when you executed spark-shell or pyspark command in your CLI as spark-context is created automatically when you execute one of these commands.
Our admin have disabled extensive logging due to which the tracking url is not generated.
You can go to resource manager and track from there.

Here is the URL for resource manager - http://rm01.itversity.com:19088/cluster