Spark Web UI is not loading correctly

bigdata-labs
#1

image
image1206×508 16.7 KB

Hi I am not able to load Spark Web UI page based on the link which comes when we run Spark-shell command
This is the link which is shown in my terminal and on clicking it.
It redirects to web UI but It doesn’t load properly.
Spark context Web UI available at http://g02.itversity.com:36121

@itversity1 @itversity team can you please let me know what should I do?
Is there a resolution to this.

I have tried below things.

  1. Ran below commands
    spark2-shell --master yarn --conf spark.ui.port=0
    spark.sparkContext.uiWebUrl
  2. refreshed page several times
  3. Removed all the cookies from the browser and tried refreshing the web page.
#2

Hi, @ishant_kaushik
The way we can see it is:

If you are running a yarn job we can get these details from Resource Manager.

Or else you can get details from the respective version of Spark History UI.

Spark2History - http://m01.itversity.com:18081/

Spark3History - http://m02.itversity.com:18080/

Eg:

1383×748

As it is spark 2.4 click on Spark2History - http://m01.itversity.com:18081/

If that is completed job using spark user identify your application id. Click on it.

If the job is still running scroll down the page click on show incomplete applications.

1548×987

Check for user name & click on the spark job application ID you want to see.

1920×871

1920×805

I hope it helps.

1 Like
#3

I am aware of these things.
Thing is it appears in Spark2History only when you stop your spark job and quit spark.
Then only it comes in history. What if I want to see how many executor or job or task at run time not after quitting spark.

Thanks man. I appreciate your effort