Sqoop Import - Managing HDFS Directories - append or overwrite or fail

#1

Let us see how we can manage target locations while performing the import.

  • Sqoop import by default will fail if the HDFS location into which data has to be imported already exists.
  • We can append to the existing directory by using –append.
  • We can overwrite the existing directory by using **–

Here is the example for sqoop import where data will be appended in the HDFS location specified as part of target-dir or warhouse-dir.

sqoop import \
  --connect "jdbc:mysql://ms.itversity.com:3306/retail_db" \
  --username retail_user \
  --password itversity \
  --table order_items \
  --warehouse-dir /user/training/sqoop_import/retail_db \
  --append

Here is the example for sqoop import where data will be overwritten in the HDFS location specified as part of target-dir or warhouse-dir.

sqoop import \
  --connect "jdbc:mysql://ms.itversity.com:3306/retail_db" \
  --username retail_user \
  --password itversity \
  --table order_items \
  --warehouse-dir /user/training/sqoop_import/retail_db \
  --delete-target-dir
0 Likes