Spark demo error

Spark demo error
0.0 0

#1

I am getting the below error when i am trying to run spark demo program in pycharm. Could you please help me in resolving this

C:\Users\Owner\PycharmProjects\gettingstarted\venv\Scripts\python.exe C:/Users/Owner/PycharmProjects/gettingstarted/sparkdemo.py
2018-05-15 12:36:58 WARN NativeCodeLoader:62 - Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
Setting default log level to “WARN”.
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Traceback (most recent call last):
File “C:/Users/Owner/PycharmProjects/gettingstarted/sparkdemo.py”, line 2, in
sc = SparkContext(master=“local”, appName=“SparkDemo”)
File “C:\spark-1.6.3-bin-hadoop2.6\python\pyspark\context.py”, line 113, in init
SparkContext._ensure_initialized(self, gateway=gateway)
File “C:\spark-1.6.3-bin-hadoop2.6\python\pyspark\context.py”, line 246, in _ensure_initialized
SparkContext._gateway = gateway or launch_gateway()
File “C:\spark-1.6.3-bin-hadoop2.6\python\pyspark\java_gateway.py”, line 116, in launch_gateway
java_import(gateway.jvm, “org.apache.spark.SparkConf”)
File “C:\spark-1.6.3-bin-hadoop2.6\python\lib\py4j-0.9-src.zip\py4j\java_gateway.py”, line 80, in java_import
File “C:\spark-1.6.3-bin-hadoop2.6\python\lib\py4j-0.9-src.zip\py4j\protocol.py”, line 304, in get_return_value
KeyError: u’y’

Process finished with exit code 1


#2

Hi @pujitha_mupparaju

Can you please check whether pyspark is running at your command prompt or not? If it is not running please set the path in the environment variables correctly(go through this blog to set the path).If it is running can you share the screenshot of your project structure?

Thanks & Regards,
Sunil Abhishek


#3

Hi Sunil… Pyspark is running from command prompt. Please find the screenshots of project structure and pyspark from cmd prompt


#4