Unable to save as avrodatafile

Hello Friends,

i am trying to do a sqoopimport and save the file as avrodatafile , but its throwing an error(below).
can anyone look into it and let me know what is the issue.
it works if i try to save as sequence file , but for avrofile and parquet file it gives error.

i am using a VM provided by edureka , as i did my training there .
is the avro-tools missing? how do i install it then?

thanks in advance

here is my command:
sqoop import-all-tables -m 1 --connect “jdbc:mysql://localhost/edurekadb” --username edureka --password edureka --as-avrodatafile --warehouse-dir /user/edureka/avro

here is the log

17/01/07 13:45:49 INFO mapreduce.Job: Running job: job_1457497162945_0153
17/01/07 13:46:09 INFO mapreduce.Job: Job job_1457497162945_0153 running in uber mode : false
17/01/07 13:46:10 INFO mapreduce.Job: map 0% reduce 0%
17/01/07 13:46:21 INFO mapreduce.Job: Task Id : attempt_1457497162945_0153_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:721)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
… 7 more
Caused by: java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper
at org.apache.sqoop.mapreduce.AvroImportMapper.(AvroImportMapper.java:51)
… 12 more
Caused by: java.lang.ClassNotFoundException: org.apache.avro.mapred.AvroWrapper
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
… 13 more

@itversity : Durga sir can you please look into it and help.
thank you .

Hi Avinash, please refer the link below:

1 Like

Thanks @santy,
, i tried the above and got error.
i think i donot have avro installed

[edureka@localhost ~]$ avro-tools
-bash: avro-tools: command not found
[edureka@localhost ~]$

any idea how to install it.

thanks.

Can you use gw01 server? It is installed there. avro-tools is a wrapper for avro-tools-1.7.5.jar. You can, to get by, download latest avro-tools from apache site and sFTP to edureka and use it locally.
MK

@mak21008 hello mike, thanks for your response .
can you please tell me in detail , i dint actually get you.
thanks :slight_smile:

One of many links:
http://www.michael-noll.com/blog/2013/03/17/reading-and-writing-avro-files-from-the-command-line/#getting-avro-tools

Link: http://mirror.symnds.com/software/Apache/avro/