Talend Open Studio for BigData(5.6.2)的tSqoopImport组件在连接到MapR集群上的MySQL数据库时引发错误

Question

>

用例：需要将Talend的bigdata组件即tSqoopImport连接到MapR集群上的MySQL DB。

Talend Open studio for Big-Data(5.6.2)位于我的工作站上。MySQL(5.5)数据库安装在5节点MapR(M3-edition)集群上。集群是物理设置的，可以通过像putty和MobaX这样的远程客户端访问

-->已经附加了代表我的用例的talend作业的快照。最初，我试图建立从Talend上的Sqoop组件到MapR集群上的MySQL数据库的连接，以便从数据库表中提取数据。

Starting job X_TSqoopImportClusterTest at 11:15 30/05/2017.

[statistics] connecting to socket on port 3548
[statistics] connected
[INFO ]: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
[WARN ]: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[WARN ]: org.apache.sqoop.ConnFactory - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
[INFO ]: org.apache.sqoop.manager.MySQLManager - Preparing to use a MySQL streaming resultset.
[INFO ]: org.apache.sqoop.tool.CodeGenTool - Beginning code generation
[INFO ]: org.apache.sqoop.manager.SqlManager - Executing SQL statement: SELECT t.* FROM `Persons` AS t LIMIT 1
[INFO ]: org.apache.sqoop.manager.SqlManager - Executing SQL statement: SELECT t.* FROM `Persons` AS t LIMIT 1
[INFO ]: org.apache.sqoop.orm.CompilationManager - $HADOOP_MAPRED_HOME is not set
Note: \tmp\sqoop-Mohan.Jay\compile\a091fbc8c79e3335a228dd95aabe8dbc\Persons.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
[INFO ]: org.apache.sqoop.orm.CompilationManager - Writing jar file: \tmp\sqoop-Mohan.Jay\compile\a091fbc8c79e3335a228dd95aabe8dbc\Persons.jar
[WARN ]: org.apache.sqoop.manager.MySQLManager - It looks like you are importing from mysql.
[WARN ]: org.apache.sqoop.manager.MySQLManager - This transfer can be faster! Use the --direct
[WARN ]: org.apache.sqoop.manager.MySQLManager - option to exercise a MySQL-specific fast path.
[INFO ]: org.apache.sqoop.manager.MySQLManager - Setting zero DATETIME behavior to convertToNull (mysql)
[INFO ]: org.apache.sqoop.mapreduce.ImportJobBase - Beginning import of Persons
[INFO ]: org.apache.hadoop.conf.Configuration.deprecation - mapred.jar is deprecated. Instead, use mapreduce.job.jar
[INFO ]: org.apache.sqoop.mapreduce.JobBase - Setting default value for hadoop.job.history.user.location=none
[INFO ]: org.apache.hadoop.conf.Configuration.deprecation - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
[WARN ]: org.apache.sqoop.mapreduce.JobBase - SQOOP_HOME is unset. May not be able to find all job dependencies.
[INFO ]: org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /172.25.1.26:8032
Some error on socket 1092
Some error on socket 1112
Some error on socket 1124
2017-05-30 11:15:26,0625 ERROR Cidcache fs/client/fileclient/cc/cidcache.cc:1288 Thread: 6612 Lookup of volume users failed, error Cannot send after transport endpoint shutdown(108), CLDB: 172.25.1.27:7222 backing off ...
Some error on socket 1124
2017-05-30 11:15:28,0737 ERROR Cidcache fs/client/fileclient/cc/cidcache.cc:1288 Thread: 6612 Lookup of volume users failed, error Cannot send after transport endpoint shutdown(108), CLDB: 172.25.1.28:7222 backing off ...
[INFO ]: org.apache.hadoop.mapreduce.JobSubmitter - number of splits:1
[INFO ]: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
[INFO ]: org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job: job_1495790825330_0005
[INFO ]: org.apache.hadoop.yarn.security.ExternalTokenManagerFactory - Initialized external token manager class - com.mapr.hadoop.yarn.security.MapRTicketManager
[INFO ]: org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1495790825330_0005
[INFO ]: org.apache.hadoop.mapreduce.Job - The url to track the job: http://terminal1:8088/proxy/application_1495790825330_0005/
[INFO ]: org.apache.hadoop.mapreduce.Job - Running job: job_1495790825330_0005

Answer 1

ConnectException通常意味着您试图访问的服务器无法在该端口上访问。

如果SSH工作正常，那么在从Talend作业到MySQL服务器的路径中，对端口3306的访问将被阻止--或者该端口是另一个端口。