Need demo video for Sqoop Oozie workflow action


#1

Could you please provide the ITversity demo video for Sqoop Oozie workflow action ? Urgent.


#2

Hi Team,
Any updates on this so far ?


#3

@Lakshmi_Priya Currently, there is no video for oozie sqoop action. Can you try below simple Oozie sqoop action.

step1: Create a job.properties file as below:

nameNode=hdfs://nn01.itversity.com:8020 
jobTracker=rm01.itversity.com:8050 
queueName=default oozie.use.system.libpath=true 
oozie.libpath=${nameNode}/user/oozie/share/lib/lib_20180803155221/sqoop/ 
oozie.wf.application.path=${nameNode}/user/${user.name}/oozie_demo/simple_sqoop_demo


step2: Create a workflow.xml file as below:

<action name="sqoop-node">
    <sqoop xmlns="uri:oozie:sqoop-action:0.2">
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <prepare>
            <delete path="${nameNode}/user/${wf:user()}/sqoop_script/"/>
        </prepare>
        <configuration>
            <property>
                <name>mapred.job.queue.name</name>
                <value>${queueName}</value>
            </property>
        </configuration>
        <command>import --connect "jdbc:mysql://ms.itversity.com/retail_db" --username retail_user --password itversity --table orders --driver com.mysql.jdbc.Driver --warehouse-dir /user/username/sqoop_script/ -m 1</command>
    </sqoop>
    <ok to="end"/>
    <error to="fail"/>
</action>

<kill name="fail">
    <message>Sqoop failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>

step3: Create a directory and copy a workflow.xml, job.properties files in your HDFS path:

hdfs dfs -mkdir /user/username/oozie_demo/simple_sqoop_demo
hdfs dfs -copyFromLocal job.properties workflow.xml /user/username/oozie_demo/simple_sqoop_demo

step4: Run oozie job.properties from local path:

oozie job -oozie http://nn01.itversity.com:11000/oozie -config /home/username/oozie_demo/pig_demo/job.properties

step5: verify output from HDFS path:

hdfs dfs -ls /user/username/sqoop_script/


#4

Hi Annapurna,

ITversity uses yarn - resource manager and application master. So will this code work with Jobtracker parameters ?


#5

@Lakshmi_Priya Yes, I tested it. It’s working.