How to use sqoop on big table with no integer column as key


#1

Sqoop uses split to evenly split/import the data from source, sometimes there is no integer column in the source, and the table is too big to add a new key, we definitely don’t want to use only one mapper, what would be the good solution to use sqoop in this case?

Thank you. Any clue is appreciated.


#2

You can use a text field to split data. Here is an example:

sqoop import
-Dorg.apache.sqoop.splitter.allow_text_splitter=true
–connect jdbc:mysql://ms.itversity.com:3306/retail_db
–username retail_user
-P
–table orders
–target-dir /user/sqoop_import_practice
–split-by order_status
–delete-target-dir