Introduction to Hadoop eco system - Overview of HDFS - Getting help or usage

Description: This article provides a beginner-friendly guide on understanding and using HDFS commands. It explains key concepts, provides code examples, and offers hands-on tasks for practical learning.

Explanation for the video

[Embed the video here]

Key Concepts Explanation

Usage of Gateway Node

To run HDFS commands, it is recommended to use the designated Gateway server set as the Gateway for the HDFS service. Ensure that the Namenode process is running on port 8020 for access to HDFS.

head -20 /etc/hadoop/conf/core-site.xml

Available Commands

  • hadoop fs or hdfs dfs: List all available commands.
  • hadoop fs -usage: Provides basic usage information for the given command.
  • hadoop fs -help: Offers additional information for all commands.
  • Individual command help can be accessed by appending -help to the command.
hdfs dfs -help
hdfs dfs -usage ls
hdfs dfs -help ls

Hands-On Tasks

Perform the following tasks to practice using HDFS commands:

  1. Make sure the designated Gateway server is set correctly.
  2. List all the available commands using hadoop fs.
  3. Retrieve basic usage information using hadoop fs -usage.
  4. Explore additional information for all commands with hadoop fs -help.

Conclusion

In this article, we covered the fundamentals of HDFS commands, including how to access them through the Gateway node and the various command options available. Practicing these commands will enhance your understanding of HDFS operations and help you navigate data in the Hadoop ecosystem effectively.

Getting help or usage

Let us explore details about how to list the commands and get the help or usage for a given command.

Watch the video tutorial here