Sqoop import (update on source tables )


#1

i have importing single table from mysql to hdfs .

intially i have imported entire table.

Then for future add on record my sqoop import is working for new records.
but when update happens on existing records it is not getting capture in hdfs when i run sqoop import with (incremental / check column /last value) options. the script is creating new file for update,it is not overwriting existing file in hdfs. sqoop import script for capturing updated records.

in my source table i have maintain date column (sorce_date) . i.e for new record and for old updated record, this date column i am referring while importing in hdfs.

script is working fine but instead of overwritning existing file , it is creating new file

sqoop import
–connect “jdbc:mysql://quickstart.cloudera:3306/retail_db”
–username=retail_dba
–password=cloudera
–table departments
–target-dir /user/hive/warehouse/departments
–append
–check-column “department_id”
–incremental lastmodified
–last-value source_Date \