强制Storm使用fat jar依赖项而不是类路径定义的依赖项

Question

如何告诉Storm使用包含在fat jar中的依赖项而不是Storm类路径中的依赖项？

以下是一些背景/细节：

• joda-time是Storm类路径中包含的2.0和fat JAR中包含的2.7的依赖项。
• 使用mvn compile exec：Java-dstorm.topology=classname运行拓扑似乎可以使用2.7。
• 使用Storm jar target/filename-jar-with-dependencies.jar classname提交拓扑似乎使用2.0而不是2.7。
• filename-jar-with-dependencies.jar是使用MVN Package创建的：
  • Joda-Time 2.7在pom.xml的依赖项部分下
  • 执行jar tvf target/filename-jar-with-dependencies.jar grep joda
  时，会发现2.7的正确joda时间戳

  我之所以注意到这一点，是因为我在通过storm命令提交拓扑时看到以下警告/错误：

  WARN  com.amazonaws.services.s3.internal.S3MetadataResponseHandler - Unable to parse last modified date: Mon, 25 May 2015 13:23:29 GMT
  java.lang.IllegalStateException: Joda-time 2.2 or later version is required, but found version: 2.0
      at com.amazonaws.util.DateUtils.handleException(DateUtils.java:156) ~[filename-jar-with-dependencies.jar:na]
      at com.amazonaws.util.DateUtils.parseRFC822Date(DateUtils.java:204) ~[filename-jar-with-dependencies.jar:na]
      at com.amazonaws.services.s3.internal.ServiceUtils.parseRfc822Date(ServiceUtils.java:78) ~[filename-jar-with-dependencies.jar:na]
      at com.amazonaws.services.s3.internal.AbstractS3ResponseHandler.populateObjectMetadata(AbstractS3ResponseHandler.java:115) ~[filename-jar-with-dependencies.jar:na]
      at com.amazonaws.services.s3.internal.S3ObjectResponseHandler.handle(S3ObjectResponseHandler.java:52) [filename-jar-with-dependencies.jar:na]
      at com.amazonaws.services.s3.internal.S3ObjectResponseHandler.handle(S3ObjectResponseHandler.java:30) [filename-jar-with-dependencies.jar:na]
      at com.amazonaws.http.AmazonHttpClient.handleResponse(AmazonHttpClient.java:1050) [filename-jar-with-dependencies.jar:na]
      at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:724) [filename-jar-with-dependencies.jar:na]
      at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:467) [filename-jar-with-dependencies.jar:na]
      at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:302) [filename-jar-with-dependencies.jar:na]
      at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3672) [filename-jar-with-dependencies.jar:na]
      at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1160) [filename-jar-with-dependencies.jar:na]
  Caused by: java.lang.IllegalArgumentException: Invalid format: "Mon, 25 May 2015 13:23:29 GMT" is malformed at "GMT"
      at org.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:747) ~[joda-time-2.0.jar:2.0]
      at com.amazonaws.util.DateUtils.parseRFC822Date(DateUtils.java:202) ~[filename-jar-with-dependencies.jar:na]
      ... 15 common frames omitted
  

Answer 1

在我使用Maven 3.3.3和“WildFly 9.0.0-RC2”的情况下，我需要在pom.xml中添加

<dependency>
   <groupId>joda-time</groupId>
   <artifactId>joda-time</artifactId>
   <version>2.8.1</version>
</dependency>

这解决了我的问题。

如果您查看'AWS-SDK-Java‘的源代码，您将看到以下验证。https://github.com/aws/aws-sdk-java/blob/1.10.1/aws-Java-sdk-core/src/main/Java/com/amazonaws/util/dateutils.Java

/**
 * Returns the original runtime exception iff the joda-time being used
 * at runtime behaves as expected.
 *
 * @throws IllegalStateException if the joda-time being used at runtime
 * doens't appear to be of the right version.
 */
private static <E extends RuntimeException> E handleException(E ex) {
    if (JodaTime.hasExpectedBehavior())
        return ex;
    throw new IllegalStateException("Joda-time 2.2 or later version is required, but found version: " + JodaTime.getVersion(), ex);
}

Answer 2

在阅读了这篇文章之后，我发现了一个解决办法，就是用新版本替换Storm的/lib目录中的库（在我的例子中为joda-time)，因为默认情况下，该目录中的所有.jar文件都显示在Storm classpath中。

我希望有一个更好的答案，以防库的两个版本由于这样或那样的原因需要共存。