Taking forever to login to spark-shell


#1

Below is the ongoing loop after I enter the spark-shell command:-

17/12/20 13:52:42 INFO Client: Application report for application_1507687444776_27229 (state: ACCEPTED)
17/12/20 13:52:43 INFO Client: Application report for application_1507687444776_27229 (state: ACCEPTED)
17/12/20 13:52:44 INFO Client: Application report for application_1507687444776_27229 (state: ACCEPTED)
17/12/20 13:52:45 INFO Client: Application report for application_1507687444776_27229 (state: ACCEPTED)
17/12/20 13:52:46 INFO Client: Application report for application_1507687444776_27229 (state: ACCEPTED)
17/12/20 13:52:47 INFO Client: Application report for application_1507687444776_27229 (state: ACCEPTED)


#2

You have to share the command you are running. Also cluster might be under heavy usage. You can go to resource manager UI or ambari and check the cluster status.

Also you can run yarn application --list to see number of jobs currently running when the issue happened.


#3

yarn application --list
17/12/21 10:51:31 INFO impl.TimelineClientImpl: Timeline service address: http:/ /rm01.itversity.com:8188/ws/v1/timeline/
17/12/21 10:51:31 INFO client.RMProxy: Connecting to ResourceManager at rm01.itv ersity.com/172.16.1.106:8050
17/12/21 10:51:32 INFO client.AHSProxy: Connecting to Application History server at rm01.itversity.com/172.16.1.106:10200
Total number of applications (application-types: [] and states: [SUBMITTED, ACCEPTED, RUNNING]):3
Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL
application_1507687444776_27577 Spark shell SPARK varunu28 default RUNNING UNDEFINED 10% http://172.16.1.100:12437
application_1507687444776_27574 Spark shell SPARK varunu28 default ACCEPTED UNDEFINED 0% N/A
application_1507687444776_23947 Spark shell SPARK training default RUNNING UNDEFINED 10% http://172.16.1.109:12456