Issues with sqoop import

#1

Hello All,
I created my own db in hive and imported all tables and there was no issue with it. But while importing data by joining using below command I am getting below error. But the table is created and data also there in hive.

16/11/30 15:42:32 WARN hive.TableDefWriter: Column order_date had to be cast to a less precise type in Hive
16/11/30 15:42:32 INFO hive.HiveImport: Loading uploaded data into Hive

Logging initialized using configuration in jar:file:/usr/hdp/2.5.0.0-1245/hive/lib/hive-common-1.2.1000.2.5.0.0-1245.jar!/hive-log4j.properties
OK
Time taken: 1.597 seconds
Loading data to table raghavendra_db.ord_join_items
Failed with exception org.apache.hadoop.security.AccessControlException: User does not belong to hdfs
at org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:88)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1708)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:821)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:472)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask

and the 2nd question is select count(*) is working for other tables but not for newly created table.

[raghavendrakumars@gw01 ~]$ hive -e “use raghavendra_db;select count(*) from ord_join_items;”

Logging initialized using configuration in file:/etc/hive/2.5.0.0-1245/0/hive-log4j.properties
OK
Time taken: 1.038 seconds
OK
0
Time taken: 0.963 seconds, Fetched: 1 row(s)
[raghavendrakumars@gw01 ~]$

Thanks
Raghavendra

0 Likes

#2

Which environment you are running this job?

0 Likes

#3

I am running in bigdata-labs

0 Likes

#4

Can you paste the sqoop import command you are trying to run?

0 Likes

#5

Please find the command below.

sqoop import
–connect “jdbc:mysql://nn01.itversity.com:3306/retail_db”
–username retail_dba --password itversity
–query “select * from orders join order_items on orders.order_id = order_items.order_item_order_id where $CONDITIONS”
–target-dir /user/raghavendrakumars/sqoop_import
–hive-import
–hive-database raghavendra_db
–hive-table ord_join_items
–create-hive-table
–split-by order_id
–num-mappers 12

0 Likes

#6

@RaghavendraKumars - Try with --target-dir=/apps/hive/warehouse/raghavendra_db.db/ord_join_items \

0 Likes

#7

Target dir is to specify the path of HDFS I think. @itversity please correct me.

0 Likes

#8

@gnanaprakasam It solved the error. And as well as select count(*) also working on that table. In this case --target-dir is to provide the path of HIVE not HDFS. @itversity please confirm.

0 Likes

#9

@RaghavendraKumars… target-dir is used to provide the path where you want to import data. Path can be of both HDFS/Hive.

0 Likes