Tried to import all tables to my folder, process went success But the folder didn’t get created. When i tried executing the same code, it errors out saying the folder is exist.
Would you please help me with where I am going wrong?:
[shubhaprasadsamal@gw01 ~]$ sqoop import-all-tables --connect “jdbc:mysql://nn01.itversity.com:3306/retail_db” --username=retail_dba --password=itversity --warehouse-dir=user/shubhaprasadsamal/training/sqoop_import/import_all1
Warning: /usr/hdp/2.5.0.0-1245/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
17/01/05 14:30:11 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.5.0.0-1245
17/01/05 14:30:11 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
17/01/05 14:30:11 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
17/01/05 14:30:11 INFO tool.CodeGenTool: Beginning code generation
17/01/05 14:30:11 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM categories
AS t LIMIT 1
17/01/05 14:30:11 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM categories
AS t LIMIT 1
17/01/05 14:30:11 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.5.0.0-1245/hadoop-mapreduce
Note: /tmp/sqoop-shubhaprasadsamal/compile/5fb47ac3a40e8db9e97de036b2181929/categories.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/01/05 14:30:13 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-shubhaprasadsamal/compile/5fb47ac3a40e8db9e97de036b2181929/categories.jar
17/01/05 14:30:13 WARN manager.MySQLManager: It looks like you are importing from mysql.
17/01/05 14:30:13 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
17/01/05 14:30:13 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
17/01/05 14:30:13 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
17/01/05 14:30:13 INFO mapreduce.ImportJobBase: Beginning import of categories
17/01/05 14:30:14 INFO impl.TimelineClientImpl: Timeline service address: http://rm01.itversity.com:8188/ws/v1/timeline/
17/01/05 14:30:14 INFO client.RMProxy: Connecting to ResourceManager at rm01.itversity.com/172.16.1.106:8050
17/01/05 14:30:14 INFO client.AHSProxy: Connecting to Application History server at rm01.itversity.com/172.16.1.106:10200
17/01/05 14:30:20 INFO db.DBInputFormat: Using read commited transaction isolation
17/01/05 14:30:20 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(category_id
), MAX(category_id
) FROM categories
17/01/05 14:30:20 INFO db.IntegerSplitter: Split size: 14; Num splits: 4 from: 1 to: 58
17/01/05 14:30:21 INFO mapreduce.JobSubmitter: number of splits:4
17/01/05 14:30:21 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1480307771710_5628
17/01/05 14:30:21 INFO impl.YarnClientImpl: Submitted application application_1480307771710_5628
17/01/05 14:30:21 INFO mapreduce.Job: The url to track the job: http://rm01.itversity.com:8088/proxy/application_1480307771710_5628/
17/01/05 14:30:21 INFO mapreduce.Job: Running job: job_1480307771710_5628
17/01/05 14:30:27 INFO mapreduce.Job: Job job_1480307771710_5628 running in uber mode : false
17/01/05 14:30:27 INFO mapreduce.Job: map 0% reduce 0%
17/01/05 14:30:32 INFO mapreduce.Job: map 75% reduce 0%
17/01/05 14:30:33 INFO mapreduce.Job: map 100% reduce 0%
17/01/05 14:30:34 INFO mapreduce.Job: Job job_1480307771710_5628 completed successfully
17/01/05 14:30:35 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=642068
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=472
HDFS: Number of bytes written=1029
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=12368
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=12368
Total vcore-milliseconds taken by all map tasks=12368
Total megabyte-milliseconds taken by all map tasks=12664832
Map-Reduce Framework
Map input records=58
Map output records=58
Input split bytes=472
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=206
CPU time spent (ms)=4860
Physical memory (bytes) snapshot=893149184
Virtual memory (bytes) snapshot=11276701696
Total committed heap usage (bytes)=646971392
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=1029
17/01/05 14:30:35 INFO mapreduce.ImportJobBase: Transferred 1.0049 KB in 20.8329 seconds (49.3931 bytes/sec)
17/01/05 14:30:35 INFO mapreduce.ImportJobBase: Retrieved 58 records.
17/01/05 14:30:35 INFO tool.CodeGenTool: Beginning code generation
17/01/05 14:30:35 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM customers
AS t LIMIT 1
17/01/05 14:30:35 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.5.0.0-1245/hadoop-mapreduce
Note: /tmp/sqoop-shubhaprasadsamal/compile/5fb47ac3a40e8db9e97de036b2181929/customers.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/01/05 14:30:35 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-shubhaprasadsamal/compile/5fb47ac3a40e8db9e97de036b2181929/customers.jar
17/01/05 14:30:35 INFO mapreduce.ImportJobBase: Beginning import of customers
17/01/05 14:30:35 INFO impl.TimelineClientImpl: Timeline service address: http://rm01.itversity.com:8188/ws/v1/timeline/
17/01/05 14:30:35 INFO client.RMProxy: Connecting to ResourceManager at rm01.itversity.com/172.16.1.106:8050
17/01/05 14:30:35 INFO client.AHSProxy: Connecting to Application History server at rm01.itversity.com/172.16.1.106:10200
17/01/05 14:30:40 INFO db.DBInputFormat: Using read commited transaction isolation
17/01/05 14:30:40 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(customer_id
), MAX(customer_id
) FROM customers
17/01/05 14:30:40 INFO db.IntegerSplitter: Split size: 3108; Num splits: 4 from: 1 to: 12435
17/01/05 14:30:41 INFO mapreduce.JobSubmitter: number of splits:4
17/01/05 14:30:41 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1480307771710_5629
17/01/05 14:30:41 INFO impl.YarnClientImpl: Submitted application application_1480307771710_5629
17/01/05 14:30:41 INFO mapreduce.Job: The url to track the job: http://rm01.itversity.com:8088/proxy/application_1480307771710_5629/
17/01/05 14:30:41 INFO mapreduce.Job: Running job: job_1480307771710_5629
17/01/05 14:30:47 INFO mapreduce.Job: Job job_1480307771710_5629 running in uber mode : false
17/01/05 14:30:47 INFO mapreduce.Job: map 0% reduce 0%
17/01/05 14:30:52 INFO mapreduce.Job: map 50% reduce 0%
17/01/05 14:30:53 INFO mapreduce.Job: map 100% reduce 0%
17/01/05 14:30:54 INFO mapreduce.Job: Job job_1480307771710_5629 completed successfully
17/01/05 14:30:54 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=642452
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=487
HDFS: Number of bytes written=953525
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=12226
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=12226
Total vcore-milliseconds taken by all map tasks=12226
Total megabyte-milliseconds taken by all map tasks=12519424
Map-Reduce Framework
Map input records=12435
Map output records=12435
Input split bytes=487
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=234
CPU time spent (ms)=6610
Physical memory (bytes) snapshot=893149184
Virtual memory (bytes) snapshot=11283017728
Total committed heap usage (bytes)=640679936
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=953525
17/01/05 14:30:54 INFO mapreduce.ImportJobBase: Transferred 931.1768 KB in 19.4063 seconds (47.9832 KB/sec)
17/01/05 14:30:54 INFO mapreduce.ImportJobBase: Retrieved 12435 records.
17/01/05 14:30:54 INFO tool.CodeGenTool: Beginning code generation
17/01/05 14:30:54 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM departments
AS t LIMIT 1
17/01/05 14:30:54 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.5.0.0-1245/hadoop-mapreduce
Note: /tmp/sqoop-shubhaprasadsamal/compile/5fb47ac3a40e8db9e97de036b2181929/departments.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/01/05 14:30:55 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-shubhaprasadsamal/compile/5fb47ac3a40e8db9e97de036b2181929/departments.jar
17/01/05 14:30:55 INFO mapreduce.ImportJobBase: Beginning import of departments
17/01/05 14:30:55 INFO impl.TimelineClientImpl: Timeline service address: http://rm01.itversity.com:8188/ws/v1/timeline/
17/01/05 14:30:55 INFO client.RMProxy: Connecting to ResourceManager at rm01.itversity.com/172.16.1.106:8050
17/01/05 14:30:55 INFO client.AHSProxy: Connecting to Application History server at rm01.itversity.com/172.16.1.106:10200
17/01/05 14:31:00 INFO db.DBInputFormat: Using read commited transaction isolation
17/01/05 14:31:00 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(department_id
), MAX(department_id
) FROM departments
17/01/05 14:31:00 INFO db.IntegerSplitter: Split size: 24; Num splits: 4 from: 2 to: 100
17/01/05 14:31:01 INFO mapreduce.JobSubmitter: number of splits:4
17/01/05 14:31:01 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1480307771710_5630
17/01/05 14:31:01 INFO impl.YarnClientImpl: Submitted application application_1480307771710_5630
17/01/05 14:31:01 INFO mapreduce.Job: The url to track the job: http://rm01.itversity.com:8088/proxy/application_1480307771710_5630/
17/01/05 14:31:01 INFO mapreduce.Job: Running job: job_1480307771710_5630
17/01/05 14:31:07 INFO mapreduce.Job: Job job_1480307771710_5630 running in uber mode : false
17/01/05 14:31:07 INFO mapreduce.Job: map 0% reduce 0%
17/01/05 14:31:11 INFO mapreduce.Job: map 25% reduce 0%
17/01/05 14:31:12 INFO mapreduce.Job: map 75% reduce 0%
17/01/05 14:31:13 INFO mapreduce.Job: map 100% reduce 0%
17/01/05 14:31:14 INFO mapreduce.Job: Job job_1480307771710_5630 completed successfully
17/01/05 14:31:14 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=642008
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=489
HDFS: Number of bytes written=84
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=11658
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=11658
Total vcore-milliseconds taken by all map tasks=11658
Total megabyte-milliseconds taken by all map tasks=11937792
Map-Reduce Framework
Map input records=7
Map output records=7
Input split bytes=489
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=205
CPU time spent (ms)=3690
Physical memory (bytes) snapshot=851218432
Virtual memory (bytes) snapshot=11266453504
Total committed heap usage (bytes)=643301376
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=84
17/01/05 14:31:14 INFO mapreduce.ImportJobBase: Transferred 84 bytes in 19.6204 seconds (4.2813 bytes/sec)
17/01/05 14:31:14 INFO mapreduce.ImportJobBase: Retrieved 7 records.
17/01/05 14:31:14 INFO tool.CodeGenTool: Beginning code generation
17/01/05 14:31:14 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM order_items
AS t LIMIT 1
17/01/05 14:31:14 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.5.0.0-1245/hadoop-mapreduce
Note: /tmp/sqoop-shubhaprasadsamal/compile/5fb47ac3a40e8db9e97de036b2181929/order_items.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/01/05 14:31:14 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-shubhaprasadsamal/compile/5fb47ac3a40e8db9e97de036b2181929/order_items.jar
17/01/05 14:31:14 INFO mapreduce.ImportJobBase: Beginning import of order_items
17/01/05 14:31:15 INFO impl.TimelineClientImpl: Timeline service address: http://rm01.itversity.com:8188/ws/v1/timeline/
17/01/05 14:31:15 INFO client.RMProxy: Connecting to ResourceManager at rm01.itversity.com/172.16.1.106:8050
17/01/05 14:31:15 INFO client.AHSProxy: Connecting to Application History server at rm01.itversity.com/172.16.1.106:10200
17/01/05 14:31:20 INFO db.DBInputFormat: Using read commited transaction isolation
17/01/05 14:31:20 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(order_item_id
), MAX(order_item_id
) FROM order_items
17/01/05 14:31:20 INFO db.IntegerSplitter: Split size: 43049; Num splits: 4 from: 1 to: 172198
17/01/05 14:31:20 INFO mapreduce.JobSubmitter: number of splits:4
17/01/05 14:31:20 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1480307771710_5631
17/01/05 14:31:21 INFO impl.YarnClientImpl: Submitted application application_1480307771710_5631
17/01/05 14:31:21 INFO mapreduce.Job: The url to track the job: http://rm01.itversity.com:8088/proxy/application_1480307771710_5631/
17/01/05 14:31:21 INFO mapreduce.Job: Running job: job_1480307771710_5631
17/01/05 14:31:27 INFO mapreduce.Job: Job job_1480307771710_5631 running in uber mode : false
17/01/05 14:31:27 INFO mapreduce.Job: map 0% reduce 0%
17/01/05 14:31:32 INFO mapreduce.Job: map 75% reduce 0%
17/01/05 14:31:33 INFO mapreduce.Job: map 100% reduce 0%
17/01/05 14:31:34 INFO mapreduce.Job: Job job_1480307771710_5631 completed successfully
17/01/05 14:31:34 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=642404
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=512
HDFS: Number of bytes written=5408880
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=13027
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=13027
Total vcore-milliseconds taken by all map tasks=13027
Total megabyte-milliseconds taken by all map tasks=13339648
Map-Reduce Framework
Map input records=172198
Map output records=172198
Input split bytes=512
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=237
CPU time spent (ms)=10790
Physical memory (bytes) snapshot=986324992
Virtual memory (bytes) snapshot=11308228608
Total committed heap usage (bytes)=620232704
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=5408880
17/01/05 14:31:34 INFO mapreduce.ImportJobBase: Transferred 5.1583 MB in 19.4125 seconds (272.0988 KB/sec)
17/01/05 14:31:34 INFO mapreduce.ImportJobBase: Retrieved 172198 records.
17/01/05 14:31:34 INFO tool.CodeGenTool: Beginning code generation
17/01/05 14:31:34 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM orders
AS t LIMIT 1
17/01/05 14:31:34 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.5.0.0-1245/hadoop-mapreduce
Note: /tmp/sqoop-shubhaprasadsamal/compile/5fb47ac3a40e8db9e97de036b2181929/orders.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/01/05 14:31:34 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-shubhaprasadsamal/compile/5fb47ac3a40e8db9e97de036b2181929/orders.jar
17/01/05 14:31:34 INFO mapreduce.ImportJobBase: Beginning import of orders
17/01/05 14:31:34 INFO impl.TimelineClientImpl: Timeline service address: http://rm01.itversity.com:8188/ws/v1/timeline/
17/01/05 14:31:34 INFO client.RMProxy: Connecting to ResourceManager at rm01.itversity.com/172.16.1.106:8050
17/01/05 14:31:34 INFO client.AHSProxy: Connecting to Application History server at rm01.itversity.com/172.16.1.106:10200
17/01/05 14:31:39 INFO db.DBInputFormat: Using read commited transaction isolation
17/01/05 14:31:39 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(order_id
), MAX(order_id
) FROM orders
17/01/05 14:31:39 INFO db.IntegerSplitter: Split size: 17220; Num splits: 4 from: 1 to: 68883
17/01/05 14:31:40 INFO mapreduce.JobSubmitter: number of splits:4
17/01/05 14:31:40 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1480307771710_5632
17/01/05 14:31:40 INFO impl.YarnClientImpl: Submitted application application_1480307771710_5632
17/01/05 14:31:40 INFO mapreduce.Job: The url to track the job: http://rm01.itversity.com:8088/proxy/application_1480307771710_5632/
17/01/05 14:31:40 INFO mapreduce.Job: Running job: job_1480307771710_5632
17/01/05 14:31:46 INFO mapreduce.Job: Job job_1480307771710_5632 running in uber mode : false
17/01/05 14:31:46 INFO mapreduce.Job: map 0% reduce 0%
17/01/05 14:31:51 INFO mapreduce.Job: map 50% reduce 0%
17/01/05 14:31:52 INFO mapreduce.Job: map 100% reduce 0%
17/01/05 14:31:53 INFO mapreduce.Job: Job job_1480307771710_5632 completed successfully
17/01/05 14:31:53 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=642008
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=469
HDFS: Number of bytes written=2999944
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=13323
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=13323
Total vcore-milliseconds taken by all map tasks=13323
Total megabyte-milliseconds taken by all map tasks=13642752
Map-Reduce Framework
Map input records=68883
Map output records=68883
Input split bytes=469
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=244
CPU time spent (ms)=9850
Physical memory (bytes) snapshot=916381696
Virtual memory (bytes) snapshot=11289374720
Total committed heap usage (bytes)=653262848
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=2999944
17/01/05 14:31:54 INFO mapreduce.ImportJobBase: Transferred 2.861 MB in 19.3822 seconds (151.1506 KB/sec)
17/01/05 14:31:54 INFO mapreduce.ImportJobBase: Retrieved 68883 records.
17/01/05 14:31:54 INFO tool.CodeGenTool: Beginning code generation
17/01/05 14:31:54 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM products
AS t LIMIT 1
17/01/05 14:31:54 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.5.0.0-1245/hadoop-mapreduce
Note: /tmp/sqoop-shubhaprasadsamal/compile/5fb47ac3a40e8db9e97de036b2181929/products.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/01/05 14:31:54 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-shubhaprasadsamal/compile/5fb47ac3a40e8db9e97de036b2181929/products.jar
17/01/05 14:31:54 INFO mapreduce.ImportJobBase: Beginning import of products
17/01/05 14:31:54 INFO impl.TimelineClientImpl: Timeline service address: http://rm01.itversity.com:8188/ws/v1/timeline/
17/01/05 14:31:54 INFO client.RMProxy: Connecting to ResourceManager at rm01.itversity.com/172.16.1.106:8050
17/01/05 14:31:54 INFO client.AHSProxy: Connecting to Application History server at rm01.itversity.com/172.16.1.106:10200
17/01/05 14:31:59 INFO db.DBInputFormat: Using read commited transaction isolation
17/01/05 14:31:59 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(product_id
), MAX(product_id
) FROM products
17/01/05 14:31:59 INFO db.IntegerSplitter: Split size: 336; Num splits: 4 from: 1 to: 1345
17/01/05 14:31:59 INFO mapreduce.JobSubmitter: number of splits:4
17/01/05 14:31:59 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1480307771710_5633
17/01/05 14:32:00 INFO impl.YarnClientImpl: Submitted application application_1480307771710_5633
17/01/05 14:32:00 INFO mapreduce.Job: The url to track the job: http://rm01.itversity.com:8088/proxy/application_1480307771710_5633/
17/01/05 14:32:00 INFO mapreduce.Job: Running job: job_1480307771710_5633
17/01/05 14:32:06 INFO mapreduce.Job: Job job_1480307771710_5633 running in uber mode : false
17/01/05 14:32:06 INFO mapreduce.Job: map 0% reduce 0%
17/01/05 14:32:11 INFO mapreduce.Job: map 50% reduce 0%
17/01/05 14:32:12 INFO mapreduce.Job: map 100% reduce 0%
17/01/05 14:32:13 INFO mapreduce.Job: Job job_1480307771710_5633 completed successfully
17/01/05 14:32:13 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=642228
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=474
HDFS: Number of bytes written=173993
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=12434
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=12434
Total vcore-milliseconds taken by all map tasks=12434
Total megabyte-milliseconds taken by all map tasks=12732416
Map-Reduce Framework
Map input records=1345
Map output records=1345
Input split bytes=474
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=240
CPU time spent (ms)=5600
Physical memory (bytes) snapshot=903610368
Virtual memory (bytes) snapshot=11285983232
Total committed heap usage (bytes)=644874240
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=173993
17/01/05 14:32:13 INFO mapreduce.ImportJobBase: Transferred 169.915 KB in 19.1079 seconds (8.8924 KB/sec)
17/01/05 14:32:13 INFO mapreduce.ImportJobBase: Retrieved 1345 records.
[shubhaprasadsamal@gw01 ~]$ hadoop fs -ls /user/shubhaprasadsamal/training/sqoop_import
Found 5 items
drwxr-xr-x - shubhaprasadsamal hdfs 0 2017-01-04 19:21 /user/shubhaprasadsamal/training/sqoop_import/product-avro
drwxr-xr-x - shubhaprasadsamal hdfs 0 2017-01-04 19:26 /user/shubhaprasadsamal/training/sqoop_import/product-avro1
drwxr-xr-x - shubhaprasadsamal hdfs 0 2017-01-04 20:02 /user/shubhaprasadsamal/training/sqoop_import/product_seq
drwxr-xr-x - shubhaprasadsamal hdfs 0 2017-01-04 20:08 /user/shubhaprasadsamal/training/sqoop_import/product_text
drwxr-xr-x - shubhaprasadsamal hdfs 0 2017-01-04 19:09 /user/shubhaprasadsamal/training/sqoop_import/products
[shubhaprasadsamal@gw01 ~]$ sqoop import-all-tables --connect “jdbc:mysql://nn01.itversity.com:3306/retail_db” --username=retail_dba --password=itversity --warehouse-dir=user/shubhaprasadsamal/training/sqoop_import/import_all1
Warning: /usr/hdp/2.5.0.0-1245/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
17/01/05 14:34:22 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.5.0.0-1245
17/01/05 14:34:22 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
17/01/05 14:34:23 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
17/01/05 14:34:23 INFO tool.CodeGenTool: Beginning code generation
17/01/05 14:34:23 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM categories
AS t LIMIT 1
17/01/05 14:34:23 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM categories
AS t LIMIT 1
17/01/05 14:34:23 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.5.0.0-1245/hadoop-mapreduce
Note: /tmp/sqoop-shubhaprasadsamal/compile/e647904e2b8fbcf0b058b327ee4fd675/categories.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/01/05 14:34:24 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-shubhaprasadsamal/compile/e647904e2b8fbcf0b058b327ee4fd675/categories.jar
17/01/05 14:34:24 WARN manager.MySQLManager: It looks like you are importing from mysql.
17/01/05 14:34:24 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
17/01/05 14:34:24 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
17/01/05 14:34:24 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
17/01/05 14:34:24 INFO mapreduce.ImportJobBase: Beginning import of categories
17/01/05 14:34:25 INFO impl.TimelineClientImpl: Timeline service address: http://rm01.itversity.com:8188/ws/v1/timeline/
17/01/05 14:34:26 INFO client.RMProxy: Connecting to ResourceManager at rm01.itversity.com/172.16.1.106:8050
17/01/05 14:34:26 INFO client.AHSProxy: Connecting to Application History server at rm01.itversity.com/172.16.1.106:10200
17/01/05 14:34:26 ERROR tool.ImportAllTablesTool: Encountered IOException running import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://nn01.itversity.com:8020/user/shubhaprasadsamal/user/shubhaprasadsamal/training/sqoop_import/import_all1/categories already exists