SQOOP Import Failing - Communication Links Failure Error


#1

Hi, please help…with this error. I have sqoop client, mysql on one machine i.e slave3 (on a multi node cluster setup).

slave3# sqoop import --connect “jdbc:mysql://localhost:3306/newdb” --username root -P -m 1 --target-dir /user/root/q4 --table employee --driver com.mysql.jdbc.Driver

After giving the password, jotb id is also coming up, but getting ‘Communication Links Failure Error’ as below…

Warning: /usr/lib/sqoop/…/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
17/12/21 15:04:55 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
Enter password:
17/12/21 15:04:58 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
17/12/21 15:04:58 INFO manager.SqlManager: Using default fetchSize of 1000
17/12/21 15:04:58 INFO tool.CodeGenTool: Beginning code generation
17/12/21 15:04:59 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM employee AS t WHERE 1=0
17/12/21 15:04:59 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM employee AS t WHERE 1=0
17/12/21 15:04:59 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-root/compile/3eca0a902ce7bf4b5549258fca6c166d/employee.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/12/21 15:05:04 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/3eca0a902ce7bf4b5549258fca6c166d/employee.jar
17/12/21 15:05:04 INFO mapreduce.ImportJobBase: Beginning import of employee
17/12/21 15:05:04 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
17/12/21 15:05:04 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM employee AS t WHERE 1=0
17/12/21 15:05:06 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
17/12/21 15:05:12 INFO db.DBInputFormat: Using read commited transaction isolation
17/12/21 15:05:12 INFO mapreduce.JobSubmitter: number of splits:1
17/12/21 15:05:12 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1513837478571_0003
17/12/21 15:05:13 INFO impl.YarnClientImpl: Submitted application application_1513837478571_0003
17/12/21 15:05:13 INFO mapreduce.Job: The url to track the job: http://aslave2.bigdata.com:8088/proxy/application_1513837478571_0003/
17/12/21 15:05:13 INFO mapreduce.Job: Running job: job_1513837478571_0003
17/12/21 15:05:26 INFO mapreduce.Job: Job job_1513837478571_0003 running in uber mode : false
17/12/21 15:05:26 INFO mapreduce.Job: map 0% reduce 0%
17/12/21 15:05:34 INFO mapreduce.Job: Task Id : attempt_1513837478571_0003_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at org.apache.sqoop.mapreduce.db.DBInputFormat.setDbConf(DBInputFormat.java:170)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:161)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:755)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure


#2

Now I tried to Grant permissions to the user to access the database over the network also as
slave3# mysql -u root -p
mysql> grant all privileges on newdb.* to ‘root’@’%’ identified by ‘password’;

Now Communications Link Failure Error has gone. But now getting some generic error as below…

[root@aslave3 ~]# sqoop import --connect “jdbc:mysql://aslave3.bigdata.com:3306/newdb” --username root -P -m 1 --target-dir /user/root/q7 --table employee --driver com.mysql.jdbc.Driver
Warning: /usr/lib/sqoop/…/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
17/12/21 15:41:22 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
Enter password:
17/12/21 15:41:26 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
17/12/21 15:41:26 INFO manager.SqlManager: Using default fetchSize of 1000
17/12/21 15:41:26 INFO tool.CodeGenTool: Beginning code generation
17/12/21 15:41:27 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM employee AS t WHERE 1=0
17/12/21 15:41:27 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM employee AS t WHERE 1=0
17/12/21 15:41:27 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-root/compile/d4447ca2a6bdcf3b3231bf73e9e3a603/employee.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/12/21 15:41:31 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/d4447ca2a6bdcf3b3231bf73e9e3a603/employee.jar
17/12/21 15:41:31 INFO mapreduce.ImportJobBase: Beginning import of employee
17/12/21 15:41:32 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
17/12/21 15:41:32 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM employee AS t WHERE 1=0
17/12/21 15:41:34 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
17/12/21 15:41:39 INFO db.DBInputFormat: Using read commited transaction isolation
17/12/21 15:41:39 INFO mapreduce.JobSubmitter: number of splits:1
17/12/21 15:41:39 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1513837478571_0006
17/12/21 15:41:40 INFO impl.YarnClientImpl: Submitted application application_1513837478571_0006
17/12/21 15:41:40 INFO mapreduce.Job: The url to track the job: http://aslave2.bigdata.com:8088/proxy/application_1513837478571_0006/
17/12/21 15:41:40 INFO mapreduce.Job: Running job: job_1513837478571_0006
17/12/21 15:41:52 INFO mapreduce.Job: Job job_1513837478571_0006 running in uber mode : false
17/12/21 15:41:52 INFO mapreduce.Job: map 0% reduce 0%
17/12/21 15:42:01 INFO mapreduce.Job: Task Id : attempt_1513837478571_0006_m_000000_0, Status : FAILED
Error: employee : Unsupported major.minor version 52.0
17/12/21 15:42:10 INFO mapreduce.Job: Task Id : attempt_1513837478571_0006_m_000000_1, Status : FAILED
Error: employee : Unsupported major.minor version 52.0
17/12/21 15:42:18 INFO mapreduce.Job: Task Id : attempt_1513837478571_0006_m_000000_2, Status : FAILED
Error: employee : Unsupported major.minor version 52.0
17/12/21 15:42:26 INFO mapreduce.Job: map 100% reduce 0%
17/12/21 15:42:27 INFO mapreduce.Job: Job job_1513837478571_0006 failed with state FAILED due to: Task failed task_1513837478571_0006_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

17/12/21 15:42:28 INFO mapreduce.Job: Counters: 11
Job Counters
Failed map tasks=4
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=28461
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=28461
Total vcore-milliseconds taken by all map tasks=28461
Total megabyte-milliseconds taken by all map tasks=29144064
Map-Reduce Framework
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
17/12/21 15:42:28 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
17/12/21 15:42:28 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 53.8783 seconds (0 bytes/sec)
17/12/21 15:42:28 INFO mapreduce.ImportJobBase: Retrieved 0 records.
17/12/21 15:42:28 ERROR tool.ImportTool: Import failed: Import job failed!
[root@aslave3 ~]# java -version
openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)
[root@aslave3 ~]# sqoop-version
Warning: /usr/lib/sqoop/…/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
17/12/21 15:42:47 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.0
Sqoop 1.4.6-cdh5.13.0
git commit id
Compiled by jenkins on Wed Oct 4 11:04:44 PDT 2017

Not sure what has to be done now…