使用Flume将数据从kafka摄取到HDFS:：ConfigurationException：必须指定引导服务器

Question

我正在尝试使用flume将数据从Kafka源接收到hdfs。下面是我的flume配置文件。

flume1.sources = kafka-source-1
flume1.channels = hdfs-channel-1
flume1.sinks = hdfs-sink-1
flume1.sources.kafka-source-1.type = org.apache.flume.source.kafka.KafkaSource
flume1.sources.kafka-source-1.bootstrap.servers = localhost:9092
flume1.sources.kafka-source-1.zookeeperConnect = localhost:2181
flume1.sources.kafka-source-1.topic = MYNEWSFEEDS
flume1.sources.kafka-source-1.batchSize = 100
flume1.sources.kafka-source-1.channels = hdfs-channel-1
flume1.channels.hdfs-channel-1.type   = memory
flume1.sinks.hdfs-sink-1.channel = hdfs-channel-1
flume1.sinks.hdfs-sink-1.type = hdfs
flume1.sinks.hdfs-sink-1.hdfs.writeFormat = Text
flume1.sinks.hdfs-sink-1.hdfs.fileType = DataStream
flume1.sinks.hdfs-sink-1.hdfs.filePrefix = test-events
flume1.sinks.hdfs-sink-1.hdfs.useLocalTimeStamp = true
flume1.sinks.hdfs-sink-1.hdfs.path = hdfs://quickstart.cloudera:8020/tmp
flume1.sinks.hdfs-sink-1.hdfs.rollCount=100
flume1.sinks.hdfs-sink-1.hdfs.rollSize=0
flume1.channels.hdfs-channel-1.capacity = 10000
flume1.channels.hdfs-channel-1.transactionCapacity = 1000

我正在使用以下命令运行flume agent：

sudo flume-ng agent --name flume1 --conf-file '/etc/flume-ng/conf/flafka.conf' Dflume.root.logger=TRACE,console

但我得到以下错误：

18/03/12 16:49:18 ERROR节点. AbstractConfigurationProvider： Source kafka-source-1已被删除，由于配置过程中的错误unnable.runConfigurationException： Bootstrap Servers必须在rovider.java:140指定KafkaSjava.util.concurrent.配置（KafkaSdapter.call）在xecutors.java:471BasicSourceSjava.util.concurrent.（BasicSourceSask.run）在ask.java:304AbstractPollableSjava.util.concurrent.（AbstractPollableSource.java:63）在org.apache.flume.conf.Configurables.configure（Configurables.java:41）在org.apache.flume.node.AbstractConfigurationProvider.load源（AbstractConfigurationProvider.java:326）在org.apache.flume.node.AbstractConfigurationProvider.get配置（AbstractConfigurationProvider.java:97）在org.apache.flume.node.PollingProperty tiesFileConfigurationProvider$FileWatcherRorg.apache.flume.conf.（PollingProperty tiesFileConfigurationPorg.apache.flume.source.kafka.）在ource.do执行人$RunnableAource.java:330（Eorg.apache.flume.source.）在emantics.configureFutureTemantics.java:65AndReset（FutureTorg.apache.flume.source.）在ource.configureSchduledThreadPoolExecitor$Schdudu

虽然我已经在conf文件中指定了引导服务器，但它仍然给出了相同的错误。尝试了许多排列和组合，但没有成功。

Answer 1

根据官方的JavaDoc，你应该更换

flume1.sources.kafka-source-1.bootstrap.servers = localhost:9092

具有

flume1.sources.kafka-source-1.kafka.bootstrap.servers = localhost:9092