Unable to copy data from local to hdfs file system

Permission denied

suggest me, how to read the local data from pyspark2

Hi @Ravi_sankar1,

for reading local data from pyspark2 api you have to use file:/// extention befor giving file_path like below code-

data = spark.read.csv("file:///home/username/.../")

and if you want to copy data from local file system to hdfs you can do that by using copyFromLocal command like below-

hadoop fs -copyFromLocal /home/username/file_location /user/shubham/.../