Sqoop - Hive-overwrite error


#1

**Team, **

I have the below-listed code with which I am trying to overwrite the data in an already existing hive table order_items.

sqoop import --connect jdbc:mysql://nn01.itversity.com:3306/retail_db --username retail_dba --password itversity --table order_items --hive-import --hive-overwrite --hive-database megastar --hive-table order_items ;

Below is the error log:

Warning: /usr/hdp/2.5.0.0-1245/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
18/05/01 22:53:01 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.5.0.0-1245
18/05/01 22:53:01 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/05/01 22:53:01 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
18/05/01 22:53:01 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
18/05/01 22:53:01 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
18/05/01 22:53:01 INFO tool.CodeGenTool: Beginning code generation
18/05/01 22:53:01 INFO manager.SqlManager: Executing SQL statement: SELECT t. FROM order_items AS t LIMIT 1*
18/05/01 22:53:01 INFO manager.SqlManager: Executing SQL statement: SELECT t. FROM order_items AS t LIMIT 1*
18/05/01 22:53:01 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.5.0.0-1245/hadoop-mapreduce
Note: /tmp/sqoop-sbhupathiraju86/compile/219186d3d73c5c7490175ce0132ec821/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/05/01 22:53:03 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-sbhupathiraju86/compile/219186d3d73c5c7490175ce0132ec821/order_items.jar
18/05/01 22:53:03 WARN manager.MySQLManager: It looks like you are importing from mysql.
18/05/01 22:53:03 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
18/05/01 22:53:03 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
18/05/01 22:53:03 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
18/05/01 22:53:03 INFO mapreduce.ImportJobBase: Beginning import of order_items
18/05/01 22:53:04 INFO impl.TimelineClientImpl: Timeline service address: http://rm01.itversity.com:8188/ws/v1/timeline/
18/05/01 22:53:04 INFO client.RMProxy: Connecting to ResourceManager at rm01.itversity.com/172.16.1.106:8050
18/05/01 22:53:04 INFO client.AHSProxy: Connecting to Application History server at rm01.itversity.com/172.16.1.106:10200
18/05/01 22:53:05 ERROR tool.ImportTool: Encountered IOException running import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://nn01.itversity.com:8020/user/sbhupathiraju86/order_items already exists
**** at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:146)****
** at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:266)**
** at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:139)**
** at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)**
** at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)**
** at java.security.AccessController.doPrivileged(Native Method)**
** at javax.security.auth.Subject.doAs(Subject.java:422)**
** at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)**
** at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)**
** at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)**
** at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:200)**
** at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:173)**
** at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:270)**
** at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)**
** at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127)**
** at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:507)**
** at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)**
** at org.apache.sqoop.Sqoop.run(Sqoop.java:147)**
** at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)**
** at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)**
** at org.apache.sqoop.Sqoop.runTool(Sqoop.java:225)**
** at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)**
** at org.apache.sqoop.Sqoop.main(Sqoop.java:243)**
====================================================================

**My table definition is as follows where the location of data file is hdfs://nn01.itversity.com:8020/apps/hive/warehouse/megastar.db/order_items. Based on above error, why is sqoop referring to “hdfs://nn01.itversity.com:8020/user/sbhupathiraju86/order_items” ? Please advise. **

describe formatted order_items;
OK
col_name data_type comment
**# col_name data_type comment **


**order_item_id int **
**order_item_order_id int **
**order_item_product_id int **
**order_item_quantity tinyint **
**order_item_subtotal double **
**order_item_product_price double **


**# Detailed Table Information **
**Database: megastar **
**Owner: sbhupathiraju86 **
**CreateTime: Tue May 01 22:47:50 EDT 2018 **
**LastAccessTime: UNKNOWN **
**Protect Mode: None **
**Retention: 0 **
**Location: hdfs://nn01.itversity.com:8020/apps/hive/warehouse/megastar.db/order_items **
**Table Type: MANAGED_TABLE **
Table Parameters: **
** comment Imported by sqoop on 2018/05/01 22:47:39

** numFiles 4 **
** numRows 0 **
** rawDataSize 0 **
** totalSize 5408880 **
** transient_lastDdlTime 1525229271 **


**# Storage Information **
**SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe **
**InputFormat: org.apache.hadoop.mapred.TextInputFormat **
**OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat **
**Compressed: No **
**Num Buckets: -1 **
**Bucket Columns: [] **
**Sort Columns: [] **
Storage Desc Params: **
** field.delim | **
** line.delim \n **
** serialization.format |