StormKafka：爪哇。lang.NoClassDefFoundError:com/google/common/base/Strings

Question

我试图运行一个StormLocalCluster使用Storm核心1.0.2和kafka-spout 1.0.2。

我的pom文件如下所示

  <dependencies>
  <!-- Storm Dependency -->
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-core</artifactId>
        <version>1.0.2</version>
        <scope>provided</scope>
    </dependency>

    <!-- Storm Kafka Dependency -->
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-kafka</artifactId>
        <version>1.0.2</version>
        <exclusions>
            <exclusion>
                <groupId>org.apache.zookeeper</groupId>
                <artifactId>zookeeper</artifactId>
            </exclusion>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
 .....
 </dependencies>
<build>
    <resources>
        <resource>
            <directory>src/main/resources</directory>
        </resource>
    </resources>
    <plugins>

        <plugin>
            <artifactId>maven-assembly-plugin</artifactId>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>single</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <descriptorRefs>
                    <descriptorRef>jar-with-dependencies</descriptorRef>
                </descriptorRefs>
                <archive>
                    <manifest>
                        <mainClass>com.walmartlabs.midas.storm.Topology</mainClass>
                    </manifest>
                </archive>
            </configuration>
        </plugin>

        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>exec-maven-plugin</artifactId>
            <version>1.2.1</version>
            <executions>
                <execution>
                    <goals>
                        <goal>exec</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <executable>java</executable>
                <includeProjectDependencies>true</includeProjectDependencies>
                <includePluginDependencies>false</includePluginDependencies>
                <classpathScope>compile</classpathScope>
                <mainClass>${storm.topology}</mainClass>
            </configuration>
        </plugin>
    </plugins>
</build>

我使用以下命令在本地运行Storm集群。

mvn package
storm  jar target/myproject-0.1-SNAPSHOT-jar-with-dependencies.jar com.company.project.storm.Topology

我得到以下例外

Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/base/Strings
    at org.apache.storm.kafka.KafkaSpout.declareOutputFields(KafkaSpout.java:184)
    at org.apache.storm.topology.TopologyBuilder.getComponentCommon(TopologyBuilder.java:431)
    at org.apache.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:135)
    at com.walmartlabs.midas.storm.Topology.submitLocalTopology(Topology.java:50)
    at com.walmartlabs.midas.storm.Topology.main(Topology.java:75)
Caused by: java.lang.ClassNotFoundException: com.google.common.base.Strings
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) 

谷歌的共同依赖性是storm kafka工件。我已经将storm kafka的作用域设置为默认值，所以我相信它的所有依赖项都应该构建在我的jar中。

如果我用一个简单的喷口代替KafkaSpout，只是读取一些文件，一切正常。我相信在包装罐子时，我没有包装StormKafka神器的所有依赖项。这里怎么了？

Answer 1

我得到了这个问题，并通过添加如下依赖项（ps:guava.jar版本是storm1.0.3使用的16.0.1）解决了这个问题。顺便说一下，我们可以参考storm starter使用的番石榴版本。

        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>${guava.version}</version>
            <!--<version>16.0.1</version>-->
        </dependency>

Answer 2

https://github.com/apache/storm/tree/v1.0.2/external/storm-kafka#using-storm-kafka-with-different-versions-of-kafka

Storm 1.0.2将工件“kafka_[scala_version]”的范围更改为“提供”，这意味着您需要将其添加为依赖项。这是为了在scala版本和/或kafka版本上具有组合的灵活性。