PYSPARK : Unable to access

pyspark
apache-spark

#1

Unable to access the pyspark environment. Please help. Following is the error:

Exception in thread “main” java.lang.IllegalArgumentException: pyspark does not support any application options.
at org.apache.spark.launcher.CommandBuilderUtils.checkArgument(CommandBuilderUtils.java:254)
at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildPySparkShellCommand(SparkSubmitComm andBuilder.java:241)
at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildCommand(SparkSubmitCommandBuilder.j ava:117)
at org.apache.spark.launcher.Main.main(Main.java:87)


#2

@Rajat_Singh_Saluja,

What is the command that you are using to launch pyspark ?


#3

I’m just trying to initiate a session.
pyspark --master yarn -conf spark.ui.port=36500


#4

@Rajat_Singh_Saluja,
Have to give
pyspark - - master yarn - - conf spark.ui.port=36500


#5

Uhh sorry for this silly one.Thanks!