i am a follower of your youtube channel and recently i have covered the topic of creating an enterprises data ware house in hadoop. i have a question around that. since we were using sqoop to get the data from any other relational system into hadoop and in order to get the data incrementally we can use the --last_modified or where clause or query in the sqoop command to get the incremental data.
Now my question is that how can i provide the new last value every time i want to get the updated records i.e first day i have taken the 10 records and the second day i have to set the last_modified to something like 10 or a new date so how can i achieve that
some people suggest to use oozie for the same purpose and some are suggesting to use sqoop metastore and created the sqoop job for the same. how can i use the metastore can you please throw some light on this