Hive sqoop command


#1

I have a question for below command

sqoop import
–connect jdbc:mysql://ms.itversity.com:3306/retail_db
–username retail_user
–password itversity
–table order_items
–hive-import
–hive-database rajeshvaasudevan_sqoop_hive_import
–hive-table H_order_items \

. when i execute this it get below error

Encountered IOException running import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://nn01.itversity.com:8020/user/rajeshvaasudevan/order_items already exists

I understand above error, it states that :file already exist". But in above sqoop import command I have mentioned DB name as " rajeshvaasudevan_sqoop_hive_import". My question is,

  1. why it is importing table to this location "/user/rajeshvaasudevan/order_items"
    I have specified my own created DB name "–hive-database rajeshvaasudevan_sqoop_hive_import "
    If table to imported to this newly created DB in which no table exist then why this error occurs?
    where I have specified this location in my code and

2)"–hive-table H_order_items " , what does this line of code mean?

I understand that table which we are importing to Hive should be named as "H_order_items ".
if it is of this new then error should not occur right?

Please correct my understanding.

Thank you


#2

–username retail_user might be wrong user;
I know, we had username for retail_db as retail_dba.


#3

Your command:
sqoop import
–connect jdbc:mysql://ms.itversity.com:3306/retail_db
–username retail_user
–password itversity
–table order_items
–hive-import
–hive-database rajeshvaasudevan_sqoop_hive_import
–hive-table H_order_items
I think:

–connect jdbc:mysql://nn01.itversity.com:3306/retail_db
instead of
–connect jdbc:mysql://ms.itversity.com:3306/retail_db

–username retail_dba
instead of
–username retail_user

– (prefix with 2 -)
instead of
– (you are prefixing 1only -)

I can able to load data using the following sqoop command:

sqoop import
–connect jdbc:mysql://nn01.itversity.com:3306/retail_db
–username retail_dba
–password itversity
–table order_items
–hive-import
–create-hive-table
–hive-table cca159.order_items ;

You questionsL
2)"–hive-table H_order_items " , what does this line of code mean?

data will load into rajeshvaasudevan_sqoop.H_order_items table.


#4

If you are using ms.itversity.com then the User name must be hr_user database hr_db and password is itversity


#6

if I use hr_db and hr_user and itversity as password then following error occurs
image

17/11/30 14:03:30 ERROR manager.SqlManager: Error executing statement: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table ‘hr_db.order_items’ doesn’t exist
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table ‘hr_db.order_items’ doesn’t exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

rajeshvaasudevan_sqoop_import database exist
image


#7

image

I am using 2 prefix only --, while copy pasting it changed as 1

using mysql://nn01.itversity.com:3306/retail_db also have same issue
17/11/30 14:10:25 ERROR tool.ImportTool: Encountered IOException running import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://nn01.itversity.com:8020/user/rajeshvaasudevan/order_items already exists

  1. I do not understand why it hive table is importing to /user/rajeshvaasudevan/ this location
    while I specify to use rajeshvaasudevan_sqoop_import

  2. also I am trying to import with new name as order_items1 instead of order_items
    still it is using order_items


#8

then delete the data at location:
/user/rajeshvaasudevan/order_items & rerun.
It will be successful.


#9

image

I tried with your sqoop command and I am able to load 172198 records into cca159.h_order_items table.