.avsc file not found after sqoop import using --as-avrodatafile

After successful sqoop import using --as-avrodatafile, the .avsc file is not found under the home directory /home/cloudera, though the avro files got created in the target-dir. Appreciate the help! - vsu

avsc file will be there in the directory from where you executed sqoop import command. Not necessarily in your home directory unless you ran sqoop import from there.

Thanks! In fact I executed it from the home directory but still its not there. On checking the import result log, saw the .avsc file got created in a temporary directory “/tmp/sqoop-cloudera/compile/fe18e47d969e12365b846c4e20ffe03/”. I used the same for creating hive table.

1 Like

Interesting, can you paste your complete sqoop import code?

Here’s the import code:

sqoop import
–connect “jdbc:mysql://quickstart.cloudera:3306/retail_db”
–username retail_dba
–password cloudera
–table departments
–target-dir /user/cloudera/departments
–as-avrodatafile
–outdir java_files

Which version of cloudera VM you are using? Also please review configuration files under /etc/sqoop/conf, probably there could be some parameter which is controlling about where files should be stored.

Using cloudera-quickstart-vm-5.8.0-0-vmware. Under /etc/sqoop/conf, the following is found:
drwxr-xr-x 2 root root 4096 Dec 11 08:33 managers.d
-rw-r–r-- 1 root root 288 Dec 11 08:33 sqoop-site.xml
-rw-r–r-- 1 root root 26 Dec 11 08:33 cloudera_metadata
-rw-r–r-- 1 root root 20 Dec 11 08:33 cloudera_generation
-rw-r–r-- 1 root root 139 Dec 11 08:33 sqoop-env.sh

Of which sqoop-env.sh has:
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH::

Check both .sh as well as .xml files to see if any parameter have value starting with /tmp

Content of sqoop-env.sh:
"!/usr/bin/env bash
Generated by Cloudera Manager and should not be modified directly
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:: "

Content of sqoop-site.xml:
Just has the name tags - “sqoop.connection.factories” and “sqoop.tool.plugins”

Don’t find any parameter having value starting with /tmp.

Hmm, not sure. It is tough to troubleshoot the issue as I do not have Cloudera Quickstart VM.

No Problem… Thanks Sir!