Storm螺栓未打印/记录Kafka喷口

Question

编辑：我向Bolt添加了一个。ack（）（这要求我使用一个丰富的Bolt而不是基本的Bolt）并且遇到了同样的问题--没有任何信息告诉我Bolt正在处理元组。

如果有关系的话，我会在EC2实例上的CentOS映像上运行这个。如有任何帮助，不胜感激。

查看生成的Storm worker日志，我看到这一行:从以下内容读取分区信息：/helloworld spout/partition_0-->{“topic”：“helloworld”、“partition”：0、“topology”：{“id”：“ ”、“name”：“kafka-storm test”}、“broker”：{“port”：6667、“host”：“ip-10-0-0-35.ec2.internal”}、“offset”：20}

下面几行如下：

s.k.PartitionManager [INFO] Last commit offset from zookeeper: 0
s.k.PartitionManager [INFO] Commit offset 0 is more than 9223372036854775807 behind, resetting to startOffsetTime=-2
s.k.PartitionManager [INFO] Starting Kafka ip-10-0-0-35.ec2.internal:0 from offset 0
s.k.ZkCoordinator [INFO] Task [1/1] Finished refreshing
s.k.ZkCoordinator [INFO] Task [1/1] Refreshing partition manager connections
s.k.DynamicBrokersReader [INFO] Read partition info from zookeeper: GlobalPartitionInformation{partitionMap={0=ip-10-0-0-35.ec2.internal:6667}}

工作日志的其余部分没有显示螺栓处理的消息的日志/打印。我不明白为什么螺栓似乎没有从Kafka集群得到任何信息。任何帮助都会很好。谢了。

private static KafkaSpout setupSpout() {
  BrokerHosts hosts = new ZkHosts("localhost:2181");
  SpoutConfig spoutConfig = new SpoutConfig(hosts, "helloworld", "", "HelloWorld Spout");
  spoutConfig.scheme = new SchemeAsMultiScheme(new StringScheme());
  spoutConfig.forceFromStart = true;
  spoutConfig.startOffsetTime = kafka.api.OffsetRequest.EarliestTime();
  return new KafkaSpout(spoutConfig);
}

构建拓扑并提交

public static void main(String[] args) {
  TopologyBuilder builder = new TopologyBuilder();
  builder.setSpout("Kafka Spout", setupSpout());
  builder.setBolt("Echo Bolt", new SystemOutEchoBolt());

  try {
    System.setProperty("storm.jar", "/tmp/storm.jar");
    StormSubmitter.submitTopology("Kafka-Storm test", new Config(), builder.createTopology());
  } //catchExceptionsHere
}

螺栓

public class SystemOutEchoBolt extends BaseRichBolt {

  private static final long serialVersionUID = 1L;
  private static final Logger logger = LoggerFactory.getLogger(SystemOutEchoBolt.class);

  private OutputCollector m_collector;

  @SuppressWarnings("rawtypes")
  @Override
  public void prepare(Map _map, TopologyContext _conetxt, OutputCollector _collector) {
    m_collector = _collector;
  }

  @Override
  public void execute(Tuple _tuple) {
    System.out.println("Printing tuple with toString(): " + _tuple.toString());
    System.out.println("Printing tuple with getString(): " + _tuple.getString(0));
    logger.info("Logging tuple with logger: " + _tuple.getString(0));
    m_collector.ack(_tuple);
  }

  @Override
  public void declareOutputFields(OutputFieldsDeclarer _declarer) {}
}

Answer 1

答案很简单。我从来没有告诉螺栓订阅哪条流。添加.shufflegrouping（“Kafka spout”）；修复了这个问题。