Application report message when trying to start the spark-shell

apache-spark

#1

I am encountering an intermittant issue when trying to start the spark-shell in the bigdata-lab environment. I am using the following statement:

spark-shell --master yarn
–conf spark.ui.port=19191
–num-executors 1
–executor-memory 512M

The shell starts up with no errors in the logs, but then it starts spitting out the following message:

INFO Client: Application report for application_1528528985359_0491 (state: ACCEPTED)

over and over and over, and does not stop. The only way to stop it is to kill the shell(ctrl-C). I have even tried with multiple different port #s. This is basically rendering the spark-shell unusable, so I can’t perform my course work. Googling is telling me that it is a YARN issue, not enough resources. Is this something that I should be messing with? Killing other people jobs so that I can run my own shell? Is there any type of workaround for this? or at least some way to tell what times of the day this issue is least likely to occur? and when I pay $$ to extend my lab access, will I still have to deal with this issue on a paid account?

Thanks for any help that you can give me!