PySpark2 - Where to See Spark Job Logs?

I am executing my spark code on .ipynb notebook. So now want to see the Spark Job logs. Please let me know where to navigate as I am unable to see logs on ‘http://rm01.itversity.com:18080/’ history server.

Hi @Sourabh_Mahajan,

On jupyter notebook spark is integrated with Spark2.x and this link which you provide ( ‘http://rm01.itversity.com:18080/) gives you the history of spark1.x jobs that’s why you are not able to see your jobs history here.

To see the jobs history of spark2.x please go through below link-

http://nn02.itversity.com:18081/