Not able to install pyspark

#1

I am trying to install pyspark using python3.6 -m pip install pyspark and it gives me the below error:-

Exception:
Traceback (most recent call last):
File “/usr/lib/python3.6/site-packages/pip/basecommand.py”, line 215, in main
status = self.run(options, args)
File “/usr/lib/python3.6/site-packages/pip/commands/install.py”, line 326, in run
strip_file_prefix=options.strip_file_prefix,
File “/usr/lib/python3.6/site-packages/pip/req/req_set.py”, line 742, in install
**kwargs
File “/usr/lib/python3.6/site-packages/pip/req/req_install.py”, line 834, in install
strip_file_prefix=strip_file_prefix
File “/usr/lib/python3.6/site-packages/pip/req/req_install.py”, line 1037, in move_wheel_files
strip_file_prefix=strip_file_prefix,
File “/usr/lib/python3.6/site-packages/pip/wheel.py”, line 346, in move_wheel_files
clobber(source, lib_dir, True)
File “/usr/lib/python3.6/site-packages/pip/wheel.py”, line 317, in clobber
ensure_dir(destdir)
File “/usr/lib/python3.6/site-packages/pip/utils/init.py”, line 83, in ensure_dir
os.makedirs(path)
File “/usr/lib64/python3.6/os.py”, line 220, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: ‘/usr/lib/python3.6/site-packages/py4j-0.10.7.dist-info’

0 Likes

#2

@Sebastian Check your environmental variables whether you setup the paths correctly or not. You can follow this blog for clear documentation.

0 Likes

#3

Hi

I am trying to install pyspark using python3.6 on your environment (gw03.itversity.com). I think the error is because I do not have admin privileges. Is there a way to run python3.6 on gw03.itversity.com?

Thanks
Sebastian

0 Likes

#4

@Sebastian Run below command before launching pyspark with python3.6.

export PYSPARK_PYTHON=python3.6

0 Likes