多重消费者Spring Kafka

Question

问题是Spring Kafka侦听器只配置了主题名。

@Service
public class Consumer {

    @KafkaListener(topics = "${app.topic}")
    public void receive(@Payload String message,
                        @Headers MessageHeaders headers) {
        System.out.println("Received message="+message);
        headers.keySet().forEach(key -> System.out.println(key+"->"+headers.get(key)));
    }
}

我似乎可以让Kafka产生100个消费者来处理来自“队列”（日志）的消息。怎么能做到呢？

Answer 1

请查看以下答案，以了解Apache中的Kafka使用者Kafka为什么不能有比分区更多的使用者实例？

要在单个使用者组中正确地分发消息，您必须有多个分区。一旦为负载找到了正确的分区数量，我将使用spring cloud streaming来更好地管理并发性和使用者组分配。

    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-stream-kafka</artifactId>
    </dependency>

水槽样品

@SpringBootApplication
@EnableBinding(Sink.class)
public class LoggingConsumerApplication {

public static void main(String[] args) {
    SpringApplication.run(LoggingConsumerApplication.class, args);
}

@StreamListener(Sink.INPUT)
public void handle(Person person) {
    System.out.println("Received: " + person);
}

public static class Person {
    private String name;
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
    public String toString() {
        return this.name;
    }
}
}

cloud:
  stream:
    bindings:
      input:
        destination: <topic-name>
        group: <consumer-group>
        consumer:
          headerMode: raw
          partitioned: true
          concurrency: 20