Introduction to Hadoop eco system - Overview of HDFS - Listing HDFS Files

This article will guide you through different options available with the hdfs ls command to list files in HDFS. We will cover various sorting options, help commands, and practical examples to help you navigate the Hadoop file system efficiently.

Key Concepts Explanation

hdfs dfs -usage ls

To get the usage information for the ls command in HDFS, you can run hdfs dfs -usage ls.

hdfs dfs -usage ls

hdfs dfs -help ls

For detailed help on the ls command in HDFS, you can use hdfs dfs -help ls.

hdfs dfs -help ls

Listing Files in a Directory

You can list all files in a specific directory in HDFS by running the following command:

hdfs dfs -ls /public/nyse_all/nyse_data

Hands-On Tasks

Here are some hands-on tasks for you to practice listing files in HDFS with different options:

  1. Run hdfs dfs -ls /public/nyse_all/nyse_data to list files in a directory.
  2. Try sorting files by time using -t option with hdfs dfs -ls -t /public/nyse_all/nyse_data.

Conclusion

In this article, we explored various options available with the hdfs ls command to list files in HDFS. By following the hands-on tasks mentioned, you can gain practical experience in working with Hadoop file systems. Keep practicing and feel free to engage with the community for further learning.

Watch the video tutorial here