Cannot import SparkSession in Spark-sumbit

I have tried to read a json file from hdfs i created a code in pycharm and tried to submit it into labs but the following error appears.

Please help me with this.

Regards,
Renganathan


Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster

@Ramesh1 ji please help me with this as i said before preparing for interview and i cannot access the labs for past few days

Regadrs,
S

@shashank1
First of all, this is not a lab issue. And if you feel it like lab issue, please post it under bigdata labs category in future. If no one responded, please send a mail to support@itversity.com in this case.

I have copied your code to a file test.py & it worked. As this is not a lab issue you have to take it from here.



@Ramesh1 Thank you ramesh it is a great help, even i can run the same copied folder from outside the directory but im unable to run the pyspark from renga/git/spark_programs/ directory.

Regareds,
S

@shashank1

You can’t give names like json to a python script. Please delete json.pyc & give json.py some other meaningful name.
Run spark-summit full command, now it will definitely work in this directory itself.

@Ramesh1 i tried for poc.py also the result is same bro, please help me out in this

You have not removed .pyc files as I suggested previously. That is why you are facing this issue.


I have removed all .pyc & renamed the scripts which are creating .pyc files.
Now we are getting different error.
As this is your own program you have to take it from here.