After successful sqoop import using --as-avrodatafile, the .avsc file is not found under the home directory /home/cloudera, though the avro files got created in the target-dir. Appreciate the help! - vsu
avsc file will be there in the directory from where you executed sqoop import command. Not necessarily in your home directory unless you ran sqoop import from there.
Thanks! In fact I executed it from the home directory but still its not there. On checking the import result log, saw the .avsc file got created in a temporary directory “/tmp/sqoop-cloudera/compile/fe18e47d969e12365b846c4e20ffe03/”. I used the same for creating hive table.
Interesting, can you paste your complete sqoop import code?
Here’s the import code:
Which version of cloudera VM you are using? Also please review configuration files under /etc/sqoop/conf, probably there could be some parameter which is controlling about where files should be stored.
Using cloudera-quickstart-vm-5.8.0-0-vmware. Under /etc/sqoop/conf, the following is found:
drwxr-xr-x 2 root root 4096 Dec 11 08:33 managers.d
-rw-r–r-- 1 root root 288 Dec 11 08:33 sqoop-site.xml
-rw-r–r-- 1 root root 26 Dec 11 08:33 cloudera_metadata
-rw-r–r-- 1 root root 20 Dec 11 08:33 cloudera_generation
-rw-r–r-- 1 root root 139 Dec 11 08:33 sqoop-env.sh
Of which sqoop-env.sh has:
Check both .sh as well as .xml files to see if any parameter have value starting with /tmp
Content of sqoop-env.sh:
Generated by Cloudera Manager and should not be modified directly
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:: "
Content of sqoop-site.xml:
Just has the name tags - “sqoop.connection.factories” and “sqoop.tool.plugins”
Don’t find any parameter having value starting with /tmp.
Hmm, not sure. It is tough to troubleshoot the issue as I do not have Cloudera Quickstart VM.
Avro data file extension is not avsc
No Problem… Thanks Sir!