apache storm喷口不发射元组

Question

我使用三台机器和本指南在aws集群上设置了一个测试环境。

我在本地模式下测试了我的代码，并使用wirbelsturm创建了一个本地vagrant集群，这两种方法都能产生预期的结果。

当我现在向Web服务器提交代码时，我的喷口和所有的螺栓都是静默的。我的嘴读的是csv，我把它抄送给了灵气和我的主管。storm UI显示我的拓扑为活动的，并显示所有螺栓和我的喷口，但计数器不可见。主管没有已使用得员工.防火墙配置为允许nimbus和supervisor接受来自supervisor和nimbus的端口6700-6703。动物园管理员在那些港口说话吗？

我似乎在我的机器上也找不到我的输出日志。我在/usr/local/storm/logs中找到了ui和nimbus日志，但除此之外，我似乎没有得到一个错误，甚至没有得到spouts/bolts的日志。Valrant机器显示一个worker-xxxx.log，但我的aws服务器没有。

这是因为我的代码在某个错误时崩溃还是因为我的配置错误？

更新：我用storm-starter示例验证了我的拓扑结构，那些似乎也不起作用。我使用mvn包构建了一个UberJar。

 2015-12-08 13:42:55.168 b.s.u.Utils [INFO] Using defaults.yaml from resources
2015-12-08 13:42:55.297 b.s.u.Utils [INFO] Using storm.yaml from resources
2015-12-08 13:42:57.434 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2015-12-08 13:42:57.435 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:host.name=ip-172-31-26-239.us-west-2.compute.internal
2015-12-08 13:42:57.435 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:java.version=1.7.0_91
2015-12-08 13:42:57.435 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:java.vendor=Oracle Corporation
2015-12-08 13:42:57.435 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.91.x86_64/jre
2015-12-08 13:42:57.435 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:java.class.path=/usr/local/apache-storm-0.10.0/lib/clojure-1.6.0.jar:/usr/local/apache-storm-0.10.0/lib/log4j-core-2.1.jar:/usr/local/apache-storm-0.10.0/lib/log4j-api-2.1.jar:/usr/local/apache-sto$
2015-12-08 13:42:57.435 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
2015-12-08 13:42:57.435 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:java.io.tmpdir=/tmp
2015-12-08 13:42:57.435 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:java.compiler=<NA>
2015-12-08 13:42:57.436 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:os.name=Linux
2015-12-08 13:42:57.436 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:os.arch=amd64
2015-12-08 13:42:57.436 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:os.version=2.6.32-504.8.1.el6.x86_64
2015-12-08 13:42:57.436 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:user.name=storm
2015-12-08 13:42:57.436 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:user.home=/app/home/storm
2015-12-08 13:42:57.436 o.a.s.s.o.a.z.ZooKeeper [INFO] Client environment:user.dir=/
2015-12-08 13:42:57.459 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2015-12-08 13:42:57.459 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:host.name=ip-172-31-26-239.us-west-2.compute.internal
2015-12-08 13:42:57.459 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:java.version=1.7.0_91
2015-12-08 13:42:57.459 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:java.vendor=Oracle Corporation
2015-12-08 13:42:57.459 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.91.x86_64/jre
2015-12-08 13:42:57.460 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:java.class.path=/usr/local/apache-storm-0.10.0/lib/clojure-1.6.0.jar:/usr/local/apache-storm-0.10.0/lib/log4j-core-2.1.jar:/usr/local/apache-storm-0.10.0/lib/log4j-api-2.1.jar:/usr/local/ap$
2015-12-08 13:42:57.460 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib
2015-12-08 13:42:57.460 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:java.io.tmpdir=/tmp
2015-12-08 13:42:57.460 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:java.compiler=<NA>
2015-12-08 13:42:57.460 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:os.name=Linux
2015-12-08 13:42:57.460 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:os.arch=amd64
2015-12-08 13:42:57.460 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:os.version=2.6.32-504.8.1.el6.x86_64
2015-12-08 13:42:57.460 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:user.name=storm
2015-12-08 13:42:57.460 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:user.home=/app/home/storm
2015-12-08 13:42:57.460 o.a.s.s.o.a.z.s.ZooKeeperServer [INFO] Server environment:user.dir=/
2015-12-08 13:42:57.774 b.s.u.Utils [INFO] Using defaults.yaml from resources
2015-12-08 13:42:57.803 b.s.u.Utils [INFO] Using storm.yaml from resources
2015-12-08 13:42:57.939 b.s.d.supervisor [INFO] Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" 60, "nimbus.childopts" "-Xmx1024m -Djava.net.preferIPv4Stack=true", "ui.filter.params" nil, "storm.cluster.mode" "distributed", "storm.messaging.net$
2015-12-08 13:42:57.963 b.s.u.StormBoundedExponentialBackoffRetry [INFO] The baseSleepTimeMs [1000] the maxSleepTimeMs [30000] the maxRetries [5]
2015-12-08 13:42:58.063 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2015-12-08 13:42:58.066 o.a.s.s.o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=zkserver1:2181 sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@502016b8
2015-12-08 13:42:58.081 o.a.s.s.o.a.z.ClientCnxn [INFO] Opening socket connection to server zkServer1/xx.xx.xx.xx:2181. Will not attempt to authenticate using SASL (unknown error)
2015-12-08 13:42:58.089 o.a.s.s.o.a.z.ClientCnxn [INFO] Socket connection established to zkServer1/xx.xx.xx.xx:2181, initiating session
2015-12-08 13:42:58.094 o.a.s.s.o.a.z.ClientCnxn [INFO] Session establishment complete on server zkServer1/xx.xx.xx.xx:2181, sessionid = 0x15182c7ba25000d, negotiated timeout = 20000
2015-12-08 13:42:58.096 o.a.s.s.o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2015-12-08 13:42:58.097 b.s.zookeeper [INFO] Zookeeper state update: :connected:none
2015-12-08 13:42:59.109 o.a.s.s.o.a.z.ClientCnxn [INFO] EventThread shut down
2015-12-08 13:42:59.110 o.a.s.s.o.a.z.ZooKeeper [INFO] Session: 0x15182c7ba25000d closed
2015-12-08 13:42:59.111 b.s.u.StormBoundedExponentialBackoffRetry [INFO] The baseSleepTimeMs [1000] the maxSleepTimeMs [30000] the maxRetries [5]
2015-12-08 13:42:59.116 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2015-12-08 13:42:59.116 o.a.s.s.o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=zkserver1:2181/storm sessionTimeout=20000 watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@5edfa0aa
2015-12-08 13:42:59.121 o.a.s.s.o.a.z.ClientCnxn [INFO] Opening socket connection to server zkServer1/xx.xx.xx.xx:2181. Will not attempt to authenticate using SASL (unknown error)
2015-12-08 13:42:59.122 o.a.s.s.o.a.z.ClientCnxn [INFO] Socket connection established to zkServer1/xx.xx.xx.xx:2181, initiating session
2015-12-08 13:42:59.124 o.a.s.s.o.a.z.ClientCnxn [INFO] Session establishment complete on server zkServer1/xx.xx.xx.xx:2181, sessionid = 0x15182c7ba25000e, negotiated timeout = 20000
2015-12-08 13:42:59.124 o.a.s.s.o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2015-12-08 13:42:59.169 b.s.d.supervisor [INFO] Starting supervisor with id cc5e1723-cc06-4bc1-a1bf-192a1d7f5bf6 at host xxxxxxx.us-west-2.compute.internal
2015-12-08 13:43:06.059 b.s.d.supervisor [INFO] Downloading code for storm id production-topology-4-1449599549 from /app/storm/nimbus/stormdist/production-topology-4-1449599549
2015-12-08 13:43:06.075 b.s.u.StormBoundedExponentialBackoffRetry [INFO] The baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [5]

所以我找到了这个：

java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection timed out
        at backtype.storm.security.auth.TBackoffConnect.retryNext(TBackoffConnect.java:59) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:51) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:103) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.security.auth.ThriftClient.<init>(ThriftClient.java:72) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.utils.NimbusClient.<init>(NimbusClient.java:74) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.utils.NimbusClient.getConfiguredClient(NimbusClient.java:37) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.utils.Utils.downloadFromMaster(Utils.java:361) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.daemon.supervisor$fn__7720.invoke(supervisor.clj:581) ~[storm-core-0.10.0.jar:0.10.0]
        at clojure.lang.MultiFn.invoke(MultiFn.java:241) ~[clojure-1.6.0.jar:?]
        at backtype.storm.daemon.supervisor$mk_synchronize_supervisor$this__7638.invoke(supervisor.clj:465) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.event$event_manager$fn__7258.invoke(event.clj:40) [storm-core-0.10.0.jar:0.10.0]
        at clojure.lang.AFn.run(AFn.java:22) [clojure-1.6.0.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.7.0_91]
Caused by: org.apache.thrift7.transport.TTransportException: java.net.ConnectException: Connection timed out
        at org.apache.thrift7.transport.TSocket.open(TSocket.java:187) ~[storm-core-0.10.0.jar:0.10.0]
        at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:103) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48) ~[storm-core-0.10.0.jar:0.10.0]
        ... 11 more
Caused by: java.net.ConnectException: Connection timed out
        at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.7.0_91]
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) ~[?:1.7.0_91]
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) ~[?:1.7.0_91]
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) ~[?:1.7.0_91]
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.7.0_91]
        at java.net.Socket.connect(Socket.java:579) ~[?:1.7.0_91]
        at org.apache.thrift7.transport.TSocket.open(TSocket.java:182) ~[storm-core-0.10.0.jar:0.10.0]
        at org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:103) ~[storm-core-0.10.0.jar:0.10.0]
        at backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:48) ~[storm-core-0.10.0.jar:0.10.0]
        ... 11 more
2015-12-08 14:26:41.028 b.s.util [ERROR] Halting process: ("Error when processing an event")
java.lang.RuntimeException: ("Error when processing an event")
        at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:336) [storm-core-0.10.0.jar:0.10.0]
        at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.6.0.jar:?]
        at backtype.storm.event$event_manager$fn__7258.invoke(event.clj:48) [storm-core-0.10.0.jar:0.10.0]
        at clojure.lang.AFn.run(AFn.java:22) [clojure-1.6.0.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.7.0_91]

Answer 1

我跟你遵循同样的指导，遇到了同样的问题。什么为我解决了问题：

以相同的方式编辑三台计算机(zookeeper、nimbus和slave1)的3/etc/hosts文件

• 首先删除以::1开头的IPv6行，apache Storm不支持这一点。
• 在包含本地别名的文件的第一行中，将本地计算机的公共主机名（集群的其他节点已知的主机名）放在127.0.0.1之后。我想这是storm要考虑的别名。
• 最后，如指南中所述，列出所有其他计算机和Storm已知的主机名

127.0.0.1 vm-matthias-02 localhost.localdomain localhost
192.168.200.48  vm-matthias-01
192.168.200.49  vm-matthias-02
192.168.200.50  vm-matthias-03

编辑配置文件时，请注意使用与计算机相同的名称。