Why master yarn or local is needed. Just pyspark makes local file read - filenotfound exception

pyspark

#1

why master yarn or local is needed. Just pyspark makes local file read - filenotfound exception

pyspark
sc.textFile(“file:///user/myDocuments/test/one.txt”)

filenotfound exception

Learn Spark 1.6.x or Spark 2.x on our state of the art big data labs

  • Click here for access to state of the art 13 node Hadoop and Spark Cluster


#2

@Rakz_Roamerz

I think you need to mention the directory name as well like below

sc.textFile(“file:///C:/Users/sunil/Desktop/file.csv”).first()

…if not can you please share the screenshot of your file location?


#3

yes, I have left out directory of file. It works, thanks


#4