启动Zookeeper和Kafka服务器文件意外

Question

我是Kafka的初学者

1/我下载了Kafka的1.0.0版本

2/I更改了bith server.properties和zookeeper.properties中的数据目录位置属性

你能帮帮我吗？

Answer 1

我假设每个用户自己的文件夹中的zookeeper&kafka为“Kafkaflume”

在kafkaflume文件夹中有两个文件夹：一个是zookeeper，另一个是kafka。这里还提供了配置文件：flumekafka.conf

您需要根据您的要求编辑此文件。

打开一个终端，转到zookeeper文件夹并启动它：

bin/zkServer.sh start

打开另一个终端，转到kafka文件夹并将其启动为：

bin/kafka-server-start.sh config/server.properties

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic  <topicname>

# Name the components on this agent
agent.sources = r1
agent.sinks =  k2
agent.channels = c1


# Describe/configure the source
agent.sources.r1.type = org.apache.flume.source.kafka.KafkaSource
agent.sources.r1.zookeeperConnect = localhost:2181
agent.sources.r1.topic = <topicname>
agent.sources.r1.groupId = group1
agent.sources.r1.channels = c1
agent.sources.r1.interceptors = i1
agent.sources.r1.interceptors.i1.type = timestamp
agent.sources.r1.kafka.consumer.timeout.ms = 10000



# Describing/Configuring the sink
 agent.sinks.k2.type = hdfs
 agent.sinks.k2.hdfs.path =  hdfs://localhost:8020/user/<username>/<foldername>/%y-%m-%d
 agent.sinks.k2.hdfs.rollInterval = 5
 agent.sinks.k2.hdfs.rollSize = 0
 agent.sinks.k2.hdfs.rollCount = 0
 agent.sinks.k2.hdfs.fileType = DataStream
 agent.sinks.k2.channel = c1


# Describing/Configuring the channel agent.channels.MemChannel.type = memory
 agent.channels.c1.type=memory
 agent.channels.c1.capacity = 10000
 agent.channels.c1.transactionCapacity = 1000