Unable to run Pyspark


#1

I am not able to get into the Pyspark. I used the below command:
pyspark --master yarn --conf spark.ui.port=12888

But i keep getting below message continuously.
18/09/30 01:28:09 INFO Client: Application report for application_1533622723243_29372 (state: ACCEPTED)
18/09/30 01:28:10 INFO Client: Application report for application_1533622723243_29372 (state: ACCEPTED)
18/09/30 01:28:11 INFO Client: Application report for application_1533622723243_29372 (state: ACCEPTED)
18/09/30 01:28:12 INFO Client: Application report for application_1533622723243_29372 (state: ACCEPTED)
18/09/30 01:28:13 INFO Client: Application report for application_1533622723243_29372 (state: ACCEPTED)

Please help me in resolving this issue. Thank you


#2

@RITESH_PRABHAKAR_SAN,
Ignore them . You can run spark commands.
To stop those messages, use sc.setLogLevel("ERROR")


#3

When I tried using that command I am still getting the same issue:
18/09/30 02:17:55 INFO Client: Application report for application_1538287994192_0003 (state: ACCEPTED)
sc.setLogLevel(“ERROR”)
18/09/30 02:17:56 INFO Client: Application report for application_1538287994192_0003 (state: ACCEPTED)
18/09/30 02:17:57 INFO Client: Application report for application_1538287994192_0003 (state: ACCEPTED)

the spark context is not get initiated. Thank you


#4

Can you please try now?


#5

I am still having the issue. Thank you


#6

Hello,

The issue has been resolved. Thank you.


#7