Introduction to Hadoop eco system - Overview of HDFS - Copying files from HDFS to local

We can copy files from HDFS to the local file system using the copyToLocal or get command. These commands allow us to transfer files or directories from HDFS to the local filesystem seamlessly.

Key Concept 1

When using the get command, it reads all the blocks using an index in sequence and constructs the file in the local file system. However, if the target file or directory already exists in the local file system, the get command will fail with an “already exists” error message.

Key Concept 2

Alternatively, we can directly copy folders and their contents from HDFS to the local file system by specifying the folder path without using the wildcard character.

Hands-On Tasks

  1. Copy the entire folder /user/${USER}/retail_db to the local home directory.
  2. Copy specific files or folders using patterns from HDFS to the local file system.

Conclusion

In this article, we have explored how to copy files from HDFS to the local file system using the get command. By following the step-by-step instructions and examples provided, readers can effectively transfer data between HDFS and their local environment.

Check out the video below for a visual guide on how to copy files from HDFS to the local file system:


Watch the video tutorial here