Input path does not exist in spark

I was trying to read a csv file that I have uploaded into a folder and I’m getting “input path does not exist error”. Tried going through many similar posts but did not find the answer.

initial_rdd = sc.textFile("/pyspark_bysumit/bigdatacampaigndata-201014-183159.csv")

I have right-clicked my file selected copy path.

If I try to see the first 10 records… I’m getting the following error.

for i in initial_rdd.take(10): print(i)

Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster

I have solved this issue by placing the file in hdfs using hadoop fs -put command