Issue with Pyspark

pyspark
#1

Hi Team,

Is there any issue with pyspark…
I was trying to invoke pyspark using below:
pyspark --master yarn

But getting below error:
return self._jvm.JavaSparkContext(jconf)
File “/usr/hdp/current/spark-client/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py”, line 1062, in call
File “/usr/hdp/current/spark-client/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py”, line 631, in send_command
File “/usr/hdp/current/spark-client/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py”, line 624, in send_command
File “/usr/hdp/current/spark-client/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py”, line 579, in _get_connection
File “/usr/hdp/current/spark-client/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py”, line 585, in _create_connection
File “/usr/hdp/current/spark-client/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py”, line 697, in start
py4j.protocol.Py4JNetworkError: An error occurred while trying to connect to the Java server

0 Likes

#2

Launch pyspark as below command and let us know:

pyspark --master yarn --conf spark.ui.port=12987

0 Likes