Sqoop import failing in AvroFormat

@itversity I am trying to import from retail database as an avro datafile, using this command

sqoop import --connect=“jdbc:mysql://nn01.itversity.com:3306/retail_db” --username=retail_dba --password=itversity --table=departmen
ts --warehouse-dir="/user/pramodvspk/sqoop_import/" --as-avrodatafile

But the error being thrown is

16/11/23 12:42:20 INFO mapreduce.Job: Task Id : attempt_1479888791849_0052_m_000002_0, Status : FAILED
Error: org.apache.avro.reflect.ReflectData.addLogicalTypeConversion(Lorg/apache/avro/Conversion;)V
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

@itversity, what might be the issue being caused.

@itversity, I have found one fix on the Horton works forum here : https://community.hortonworks.com/questions/60890/sqoop-import-to-avro-failing-which-jars-to-be-used.html

Did it resolve the issue?

@itversity, Yes sir it resolves the issue. but there is a -Dmapreduce.job.user.classpath.first=true needs to be added in the sqoop-import statement. I guess there should be a fix in the backend.

Final query @gnanaprakasam @itversity

sqoop import -Dmapreduce.job.user.classpath.first=true
–connect=“jdbc:mysql://nn01.itversity.com:3306/retail_db” \
–username=retail_dba
–password=itversity
–table=departments
–warehouse-dir="/user/pramodvspk/sqoop_import/"
–as-avrodatafile

Ok, probably we need to update properties file.

@pramodvspk Thanks Pramod. It works :slight_smile:

while sqoop importing its showing error as a access denied for user ‘retail_dba’ “%”’ to database ‘retail_db’

Can you paste the complete sqoop import command?

Thank you
Issue cleared

Which properties files are we talking about here ?

Will the total count be reduced once we import the file in avro format?

Below is the two results:

[shubhaprasadsamal@gw01 ~]$ sqoop eval --connect “jdbc:mysql://nn01.itversity.com:3306/retail_db” --username retail_dba --password itversity --query "select count(1) from products"
Warning: /usr/hdp/2.5.0.0-1245/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
17/01/04 19:33:03 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.5.0.0-1245
17/01/04 19:33:03 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
17/01/04 19:33:03 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.

count(1)
1345

[shubhaprasadsamal@gw01 ~]$ hadoop fs -ls /user/shubhaprasadsamal/training/sqoop_import/product-avro1
Found 13 items
-rw-r–r-- 3 shubhaprasadsamal hdfs 0 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1/_SUCCESS
-rw-r–r-- 3 shubhaprasadsamal hdfs 15160 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1/part-m-00000.avro
-rw-r–r-- 3 shubhaprasadsamal hdfs 14630 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1/part-m-00001.avro
-rw-r–r-- 3 shubhaprasadsamal hdfs 14506 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1/part-m-00002.avro
-rw-r–r-- 3 shubhaprasadsamal hdfs 14907 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1/part-m-00003.avro
-rw-r–r-- 3 shubhaprasadsamal hdfs 15701 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1/part-m-00004.avro
-rw-r–r-- 3 shubhaprasadsamal hdfs 15778 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1/part-m-00005.avro
-rw-r–r-- 3 shubhaprasadsamal hdfs 14021 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1/part-m-00006.avro
-rw-r–r-- 3 shubhaprasadsamal hdfs 15782 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1/part-m-00007.avro
-rw-r–r-- 3 shubhaprasadsamal hdfs 14986 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1/part-m-00008.avro
-rw-r–r-- 3 shubhaprasadsamal hdfs 14865 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1/part-m-00009.avro
-rw-r–r-- 3 shubhaprasadsamal hdfs 16983 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1/part-m-00010.avro
-rw-r–r-- 3 shubhaprasadsamal hdfs 17325 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1/part-m-00011.avro
[shubhaprasadsamal@gw01 ~]$ hadoop fs -cat /user/shubhaprasadsamal/training/sqoop_import/product-avro1/part-m-*|wc -l
118
[shubhaprasadsamal@gw01 ~]$

-Dmapreduce.job.user.classpath.first=true
this actually worked but still have to fix the issue ?
thanks