远程Kafka连接问题-Broker可能不可用

Question

import org.apache.kafka.clients.KafkaClient;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringSerializer;

import java.util.Properties;

public class ProducerDemo {

    public static void main(String[] args) {
        // create producer properties
        Properties properties = new Properties();

        properties.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "<IP_TO_REMOTE_SERVER>:9092");

        properties.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        properties.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,StringSerializer.class.getName());

        //create producer
        KafkaProducer<String,String> producer = new KafkaProducer<String, String>(properties);

       //producer record
        ProducerRecord <String,String> record = new ProducerRecord<String, String>("first_topic", "jello there");
        System.out.println("SENDING RECORD");
        //send data - async
        producer.send(record);

        producer.flush();

        producer.close();
        System.out.println("complete");
    }
}

[kafka-producer-network-thread>producer-1]警告org.apache.kafka.clients.networkclient-[Producer clientid=producer-1]无法建立到节点0的连接(/xx.xx.xx.xx.xx:9092)。代理可能不可用。

[主]信息org.apache.Kafka.clients.Producer.kafkaProducer-[Producer clientid=producer-1]关闭Kafka生产者，时间为9223372036854775807毫秒。

在查看了Stackoverflow之后，我将server.Properties listeners部分更新为服务器的私有IP

# The address the socket server listens on. It will get the value returned from 
# java.net.InetAddress.getCanonicalHostName() if not configured.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
listeners=PLAINTEXT://10.0.1.51:9092

我应该如何将服务器上的Kafka配置为可以远程访问和监听？

Answer 1

我想您面临的主要问题是从配置的角度来看。在通过生产者沟通之前，请检查您是否进行了所有必要的更改。您需要进行以下更改：

Kafka更改：您需要在Zookeeper.properties中为相关代理添加配置。

AWS更改：当连接到AWS时，您需要设置传递。pem文件的方式。您可能需要在AWS实例中启用直接访问。默认情况下，它将阻止所有未知流量。