Address already in use



I was using the lab’s cluster and I didn’t use it for sometime and became inactive and logged out. Now, when I login and launch pyspark I get the following error:

18/08/02 15:35:54 WARN AbstractLifeCycle: FAILED org.spark-project.jetty.server.Server@4cbf4a2a: Address already in use

I know this error is because the particular port which was assigned to me is still in use and will be in use till the default TIMEOUT of the socket. Is there a way to use the cluster immediately? It eats up lot of valuable time. Please help.




Launch pyspark, using below command:
pyspark --master yarn --conf spark.ui.port=12990