Issue pandas with pyspark shell

Hi All,
I’m testing some coding parts in pyspark shell in AWS EMR. but not able to import pandas and scikit, but its installed in EMR, when I submit the spark-submit command is working.

the only problem in pyspark shell. Pleeeaaase help me how to solve that.

I tried below steps, don’t know that is correct or not, this time getting different error

Suresh Selvaraj

Hi all,
I did a mistake, I installed python3 version, also installed pandas like sudo pip-3.4 install pandas==0.21.

So we need to set pyspark path export export PYSPARK_PYTHON=/usr/in/python3.4

Actually I forgot to set above path.

Suresh Selvaraj