Error during Sqoop export -- Issue RESOLVED


#1

Hello,

I tried to import retail_db’s order_items table to HDFS and re-export it to MySql. Export is failing; can you please help me find out the issue.

Here are the commands and stacktrace for your reference.

Sqoop Import:

sqoop import --connect jdbc:mysql://localhost/retail_db --username cloudera --table order_items --warehouse-dir /user/cloudra/order_items --optionally-enclosed-by ‘"’ --fields-terminated-by ‘|’ --lines-terminated-by ‘:’ --num-mappers 1;

Sqoop Export:

sqoop export --connect jdbc:mysql://localhost/retail_db_export --username cloudera --table order_items --export-dir /user/cloudera/order_items --optionally-enclosed-by ‘"’ --fields-terminated-by ‘|’ --lines-terminated-by ‘:’ --num-mappers 1;

Error stacktrace:

2018-02-03 00:39:56,466 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper:
2018-02-03 00:39:56,466 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: Exception raised during data export
2018-02-03 00:39:56,467 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper:
2018-02-03 00:39:56,467 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: Exception:
java.lang.RuntimeException: Can’t parse input data: '1’
at order_items.__loadFromFields(order_items.java:463)
at order_items.parse(order_items.java:386)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:89)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:834)
at order_items.__loadFromFields(order_items.java:438)
… 12 more
2018-02-03 00:39:56,499 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: Dumping data is not allowed by default, please run the job with -Dorg.apache.sqoop.export.text.dump_data_on_error=true to get corrupted line.
2018-02-03 00:39:56,499 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: On input file: hdfs://quickstart.cloudera:8020/user/cloudera/order_items/part-m-00000
2018-02-03 00:39:56,499 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: At position 0
2018-02-03 00:39:56,499 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper:
2018-02-03 00:39:56,499 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: Currently processing split:
2018-02-03 00:39:56,499 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: Paths:/user/cloudera/order_items/part-m-00000:0+1303818,/user/cloudera/order_items/part-m-00001:0+1343222
2018-02-03 00:39:56,499 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper:
2018-02-03 00:39:56,499 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: This issue might not necessarily be caused by current input
2018-02-03 00:39:56,499 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper: due to the batching nature of export.
2018-02-03 00:39:56,499 ERROR [main] org.apache.sqoop.mapreduce.TextExportMapper:
2018-02-03 00:39:56,499 INFO [Thread-12] org.apache.sqoop.mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
2018-02-03 00:39:56,512 WARN [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:cloudera (auth:SIMPLE) cause:java.io.IOException: Can’t export data, please check failed map task logs
2018-02-03 00:39:56,512 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.io.IOException: Can’t export data, please check failed map task logs
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:122)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.RuntimeException: Can’t parse input data: '1’
at order_items.__loadFromFields(order_items.java:463)
at order_items.parse(order_items.java:386)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:89)
… 10 more
Caused by: java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:834)
at order_items.__loadFromFields(order_items.java:438)
… 12 more

Thanks in advance,
Siva