为什么不能用Kafka正确使用消费者？

Question

192.168.0.1  kafka1
192.168.0.2  kafka2
192.168.0.3  kafka3

/usr/local/kafka2.12-2.6.0/config/server.properties

#
broker.id=1
listeners=PLAINTEXT://kafka1:9092
advertised.listeners=PLAINTEXT://kafka1:9092

#
broker.id=2
listeners=PLAINTEXT://kafka2:9092
advertised.listeners=PLAINTEXT://kafka2:9092

#
broker.id=3
listeners=PLAINTEXT://kafka3:9092
advertised.listeners=PLAINTEXT://kafka3:9092

在开始动物园管理员和Kafka之后，创建一个新的主题

[kafka@kafka1 ~]$ bin/kafka-topics.sh --create --zookeeper kafka1:2181,kafka2:2181,kafka3:2181 --replication-factor 1 --partitions 6 --topic topic1 --config cleanup.policy=delete --config delete.retention.ms=60000

检查所有三个节点上的集群状态

[kafka@kafka1 ~]$ bin/kafka-topics.sh --describe --zookeeper kafka1:2181 --topic topic1
Topic: topic1   PartitionCount: 6   ReplicationFactor: 1    Configs: cleanup.policy=delete,delete.retention.ms=60000
    Topic: topic1   Partition: 0    Leader: 2   Replicas: 2 Isr: 2
    Topic: topic1   Partition: 1    Leader: 3   Replicas: 3 Isr: 3
    Topic: topic1   Partition: 2    Leader: 1   Replicas: 1 Isr: 1
    Topic: topic1   Partition: 3    Leader: 2   Replicas: 2 Isr: 2
    Topic: topic1   Partition: 4    Leader: 3   Replicas: 3 Isr: 3
    Topic: topic1   Partition: 5    Leader: 1   Replicas: 1 Isr: 1

[kafka@kafka2 ~]$ bin/kafka-topics.sh --describe --zookeeper kafka1:2181 --topic topic1
Topic: topic1   PartitionCount: 6   ReplicationFactor: 1    Configs: cleanup.policy=delete,delete.retention.ms=60000
    Topic: topic1   Partition: 0    Leader: 2   Replicas: 2 Isr: 2
    Topic: topic1   Partition: 1    Leader: 3   Replicas: 3 Isr: 3
    Topic: topic1   Partition: 2    Leader: 1   Replicas: 1 Isr: 1
    Topic: topic1   Partition: 3    Leader: 2   Replicas: 2 Isr: 2
    Topic: topic1   Partition: 4    Leader: 3   Replicas: 3 Isr: 3
    Topic: topic1   Partition: 5    Leader: 1   Replicas: 1 Isr: 1

[kafka@kafka3 ~]$ bin/kafka-topics.sh --describe --zookeeper kafka1:2181 --topic topic1
Topic: topic1   PartitionCount: 6   ReplicationFactor: 1    Configs: cleanup.policy=delete,delete.retention.ms=60000
    Topic: topic1   Partition: 0    Leader: 2   Replicas: 2 Isr: 2
    Topic: topic1   Partition: 1    Leader: 3   Replicas: 3 Isr: 3
    Topic: topic1   Partition: 2    Leader: 1   Replicas: 1 Isr: 1
    Topic: topic1   Partition: 3    Leader: 2   Replicas: 2 Isr: 2
    Topic: topic1   Partition: 4    Leader: 3   Replicas: 3 Isr: 3
    Topic: topic1   Partition: 5    Leader: 1   Replicas: 1 Isr: 1

[kafka@kafka1 ~]$ bin/kafka-console-producer.sh --broker-list kafka1:9092 --topic topic1
>

bin/kafka-console-consumer.sh --bootstrap-server kafka1:9092 --topic topic1

# kafka1
[2020-08-21 02:09:57,299] WARN [Consumer clientId=consumer-console-consumer-39789-1, groupId=console-consumer-39789] Connection to node 2147483645 (kafka2/192.168.0.2:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
# kafka2
[2020-08-21 03:05:00,573] WARN [Consumer clientId=consumer-console-consumer-71891-1, groupId=console-consumer-71891] Connection to node -1 (kafka1/192.168.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
# kafka3
[2020-08-21 03:05:14,331] WARN [Consumer clientId=consumer-console-consumer-55574-1, groupId=console-consumer-55574] Connection to node -1 (kafka1/192.168.0.1:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

listeners=PLAINTEXT://localhost:9092
advertised.listeners=PLAINTEXT://localhost:9092

Answer 1

“侦听器”应该表示生产者和消费者可以访问的节点FQDN和端口。当生产者/消费者与Kafka联系起来时，就像你在帖子中所说的：

bin/kafka-console-consumer.sh --bootstrap-server kafka1:9092 --topic topic1

它打开到Kafka1:9092的连接，然后从Kafka中获取集群的映射，其中包含所有代理、它们的FQDN、端口和每个代理所领导的主题分区。

您设法创建了主题，这是使用zookeeper完成的，并成功了。

根据您的警告日志，“Broker1,2,3”被解析为192.168.0.1/2/3，因此IP解析是正常的。

尝试检查端口9092上的网络连接：从broker1运行：“Telnet broker2 9092”

或者运行producer命令访问远程代理，即：

[kafka@**kafka1** ~]$ bin/kafka-console-producer.sh --broker-list **kafka2**:9092 --topic topic1