Pyspark issues on windows

pyspark

#1

Hi Team, I am having issues with pyspark command on windows. seems like spark not setup correctly. Please help.

Note : I downloaded python 3.7.1

Bellow are errors.

C:\Users\Digvijay>pyspark
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:57:15) [MSC v.1915 64 bit (AMD64)] on win32
Type “help”, “copyright”, “credits” or “license” for more information.
Traceback (most recent call last):
File “C:\spark-1.6.3-bin-hadoop2.6\bin…\python\pyspark\shell.py”, line 30, in
import pyspark
File “C:\spark-1.6.3-bin-hadoop2.6\python\pyspark_init_.py”, line 41, in
from pyspark.context import SparkContext
File “C:\spark-1.6.3-bin-hadoop2.6\python\pyspark\context.py”, line 33, in
from pyspark.java_gateway import launch_gateway
File “C:\spark-1.6.3-bin-hadoop2.6\python\pyspark\java_gateway.py”, line 31, in
from py4j.java_gateway import java_import, JavaGateway, GatewayClient
File “”, line 983, in _find_and_load
File “”, line 967, in _find_and_load_unlocked
File “”, line 668, in _load_unlocked
File “”, line 638, in _load_backward_compatible
File “C:\spark-1.6.3-bin-hadoop2.6\python\lib\py4j-0.9-src.zip\py4j\java_gateway.py”, line 18, in
File “C:\Users\Digvijay\AppData\Local\Programs\Python\Python37\lib\pydoc.py”, line 62, in
import inspect
File “C:\Users\Digvijay\AppData\Local\Programs\Python\Python37\lib\inspect.py”, line 360, in
Attribute = namedtuple(‘Attribute’, ‘name kind defining_class object’)
File “C:\spark-1.6.3-bin-hadoop2.6\python\pyspark\serializers.py”, line 381, in namedtuple
cls = _old_namedtuple(*args, **kwargs)
TypeError: namedtuple() missing 3 required keyword-only arguments: ‘rename’, ‘defaults’, and ‘module’


#2

@digvijay_pawar

python 3.7 is not compatible with spark version 1.6.3 so downgrade to the python 2.7 and re-run pyspark


#3

Thanks , Now its working fine.


#4