Linux and spark shell


#1

Hi All,

I am facing an issue with spark-shell --master yarn i am getting “addresses already in user error”.I have tried logging off and logging in again.But the issue is persitant.

WARN AbstractLifeCycle: FAILED org.spark-project.jetty.server.Server@67e4928c: java.net.BindException: Address already in use
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)

Thanks,
shaikfaiz.


#2

@shaik_faiz Use below command and launch spark. Use different port number between 10000 to 60000 explicitly.

spark-shell --conf spark.ui.port=22345 spark.port.maxRetries=100 --master yarn-client