File name in Spark context

Big Data Apache Spark
#1

org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://nn01.itversity.com:8020/

how to mention file name in sc.Textfile ???

#2

@sreejesh1701 - Could you please provide your full query which tried to execute.

#3

It should be

someRdd = sc.textFile(" some hadoop file path")

#4

I m trying to execute below command in itversity lab
val lines=sc.textFile(“hdfs://user/sreejesh1701/sparkinput/moovie-rating-100k/u.data”)
lines.count()
getting error :
cala> lines.count()
java.lang.IllegalArgumentException: java.net.UnknownHostException: user
at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:411)
at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:311)
at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:688)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:629)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:159)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2761)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)

#5

@sreejesh1701 - You can remove hdfs:/ from that or
give hdfs://nn01.itversity.com:8020/user/…