IBM DataStage将数据加载到Apache cassandra抛出TransportException

Question

我使用DataStage 11.7 cassandra connector连接配置了cassandra VMs的2个节点(192.168.3.240&192.168.3.241)。

我的apache cassandra版本是3.11.3，我使用的是datastax驱动程序:dse-java-driver-core-1.8.1.jar和dse-java-driver-extras-1.8.1.jar。

Cassandra_Connector_0,1: com.datastax.driver.core.exceptions.TransportException: [/192.168.3.241:9042] Connection has been closed
at com.datastax.driver.core.exceptions.TransportException.copy (TransportException.java: 29)
at com.datastax.driver.core.exceptions.TransportException.copy (TransportException.java: 15)
at com.datastax.driver.core.DriverThrowables.propagateCause (DriverThrowables.java: 26)
at com.datastax.driver.core.AbstractSession.prepare (AbstractSession.java: 77)
at com.ibm.is.cc.cassandra.engine.CassandraSession.prepareInsertStatement (CassandraSession.java: 161)
at com.ibm.is.cc.cassandra.engine.InsertMutator.<init> (InsertMutator.java: 25)
at com.ibm.is.cc.cassandra.config.MutationType$1.newMutator (MutationType.java: 21)
at com.ibm.is.cc.cassandra.engine.CassandraSession.getMutator (CassandraSession.java: 156)
at com.ibm.is.cc.cassandra.runtime.CassandraConsumer.prepareColumnExtractor (CassandraConsumer.java: 92)
at com.ibm.is.cc.cassandra.runtime.CassandraConsumer.prepare (CassandraConsumer.java: 62)

更新1：现在我正在使用OSS驱动程序，“插入数据”工作现在很好。但是，这一次，如果我使用OSS驱动程序，“读取数据”作业将抛出NoSuchMethodError：

Cassandra_Connector_0,1: com.ascential.e2.common.CC_Exception: "Connector encountered an exception: java.lang.NoSuchMethodError: com/datastax/driver/core/SimpleStatement.setRoutingToken(Lcom/datastax/driver/core/Token;)Lcom/datastax/driver/core/RegularStatement; (loaded from file:/home/dsadm/cassandra-client-write/cassandra-driver-core-3.7.1.jar by com.ibm.is.cc.cassandra.wrapper.ChildFirstURLClassLoader@493caf75) called from class com.ibm.is.cc.cassandra.engine.CassandraSession (loaded from file:/opt/IBM/InformationServer/Server/DSEngine/../DSComponents/bin/cassandra/CassandraConnectorCore.jar by com.ibm.is.cc.cassandra.wrapper.ChildFirstURLClassLoader@493caf75).
    at com.ibm.is.cc.cassandra.engine.CassandraSession.executeSelect(CassandraSession.java:126)
    at com.ibm.is.cc.cassandra.runtime.CassandraAdapter.runProducers(CassandraAdapter.java:106)
    at com.ibm.is.cc.cassandra.runtime.CassandraAdapter.run(CassandraAdapter.java:94)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
    at java.lang.reflect.Method.invoke(Method.java:508)
    at com.ibm.is.cc.cassandra.wrapper.Delegator.invokeMethod(Delegator.java:40)
    at com.ibm.is.cc.cassandra.wrapper.Delegator.access$100(Delegator.java:16)
    at com.ibm.is.cc.cassandra.wrapper.Delegator$MethodWrapper.invoke(Delegator.java:56)
    at com.ibm.is.cc.cassandra.wrapper.CassandraAdapterWrapper.run(CassandraAdapterWrapper.java:169)
"
at com.ibm.is.cc.cassandra.resources.CC_CassandraErrorCode.createException (CC_CassandraErrorCode.java: 124)
at com.ibm.is.cc.cassandra.resources.CC_CassandraErrorCode.newWrapperFailureException (CC_CassandraErrorCode.java: 42)
at com.ibm.is.cc.cassandra.wrapper.Delegator.invokeMethod (Delegator.java: 44)
at com.ibm.is.cc.cassandra.wrapper.Delegator.access$100 (Delegator.java: 16)
at com.ibm.is.cc.cassandra.wrapper.Delegator$MethodWrapper.invoke (Delegator.java: 56)
at com.ibm.is.cc.cassandra.wrapper.CassandraAdapterWrapper.run (CassandraAdapterWrapper.java: 169)

Answer 1

问题很可能出在您正在使用的驱动程序上。对于DSE使用驱动程序，而对于开源的Cassandra则需要使用OSS驱动程序，其Maven坐标如下：

<dependency>
  <groupId>com.datastax.cassandra</groupId>
  <artifactId>cassandra-driver-core</artifactId>
  <version>3.7.1</version>
</dependency>
<dependency>
  <groupId>com.datastax.cassandra</groupId>
  <artifactId>cassandra-driver-extras</artifactId>
  <version>3.7.1</version>
</dependency>