Sqoop error importing into existing Hive database

I have been struggling with importing data from mySQL into an existing Hive database…

Step 1 - At the Hive prompt, created a Hive database
hive> create database 0_ganesh1146;

Step 2 - At the Hive prompt, validated creation of Hive database in HDFS
hive> dfs -ls /apps/hive/warehouse/0_ganesh1146.db;
drwxrwxrwx - ganesh1146 hdfs 0 2017-03-21 13:40 /apps/hive/warehouse/0_ganesh1146.db

Step 3 - At the Hive prompt, set the active database to 0_ganesh1146
hive> use 0_ganesh1146;

Step 4 - At the Hive prompt, verified that database 0_ganesh1146 has no tables
hive> show tables;

Step 5 - Sqoop command’s import tool to import ‘departments’ table into the existing 0_ganesh1146 Hive database, using the ‘–hive-database’ argument
sqoop import --table departments --username retail_dba --password itversity --connect jdbc:mysql://nn01.itversity.com:3306/retail_db --hive-database 0_ganesh1146 --hive-import --hive-overwrite --create-hive-table --verbose

Error message on running Sqoop -->
17/03/21 13:53:45 ERROR tool.ImportTool: Encountered IOException running import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://nn01.itversity.com:8020**/user/ganesh1146/departments already exists**

Why is Sqoop looking at the default /user/ganesh1146 path… when I have specified the --hive-database argument… should it not be looking to create a folder ‘departments’ for the Hive table,under /apps/hive/warehouse/0_ganesh1146.db/… is my understanding not correct?

Thank you

Sqoop first imports the data to HDFS location and then loads the data into Hive table.

2 Likes

Awesome, thank you! You just saved me a lot of time

This worked!

hadoop fs -rm -R /user/ganesh1146/departments
And then
sqoop import --table departments --username retail_dba --password itversity --connect jdbc:mysql://nn01.itversity.com:3306/retail_db --hive-database 0_ganesh1146 --hive-import --hive-overwrite --create-hive-table