Unable to copy datasets from public folder to my working location

Hello,

I am trying to copy public -> cards -> largedeck.txt file to hdfs using below command

“hadoop fs -copyFromLocal /public/cards/ /user/username/”

or

“hadoop fs -put /public/cards/ /user/username/”

Output:
copyFromLocal: `/public/cards/’: No such file or directory

Workaround: I used Ambari to copy file manually but want to know why is the o/p as above?

You are in the home directory of the lab and trying to copy from the folder /public/cards to your hdfs account directory /user/username.
The path /public/cards isn’t correct. Instead you can do something like this:

goto /public/user folder --> cd /public/cards
Assuming your “largedecjofcards.txt” is in this folder.

Now you can use -copyFromLocal command to copy the text file to hdfs account. Something like this -
hadoop fs -copyFromLocal largedeckofcards.txt /user/username. --> This should work.
or
give the full directory path where your file is located. something like this -

hadoop fs -copyFromLocal /home/username/public/cards/largedeck.txt /user/username. --> This should also work.

The basic idea here is , we are copying from the local operating system to the hdfs platform.
Our local files will be on /home/username.
hdfs files will be on /user/username.
Hence in order to copy from local file system to hdfs we use “-copyFromLocal” command. which states copying from local file system to hdfs.

you are getting error because you have given the command “hadoop fs -copyFromLocal /public/cards/ /user/username”.
The hdfs path is correct but the local path is wrong. local path should be something like /home/username…/…/…txt etc.

Hope this helps. I am also learning and correct me if i am wrong :slight_smile:

@chaitanyapolipalli Here /public/cards folder present in HDFS, NOT in local.

You can check that - hadoop fs -ls /public/cards

To copy from HDFS to HDFS you have to use -cp command.
e.g
hadoop fs -cp /public/cards /user/gnanaprakasam/cards

Note:- Public folder having read only access, you can directly use as well.

1 Like