水槽配置file_roll

Question

我对Flume和/或log4j有问题。我有带log4j和flume appender的JavaEE应用程序=一切都很好，工作正常。

当我用下面的配置运行水槽时，一切都很好。

这里是flume配置文件

#define components of agent
a1.sources = avroSource
a1.channels = ch1
a1.sinks = hdfsSink
#define source
a1.sources.avroSource.type = avro
a1.sources.avroSource.bind = localhost
a1.sources.avroSource.port = 41415
a1.sources.avroSource.channels = ch1
#define interceptors
a1.sources.avroSource.interceptors = i1 i2 i3 i4 i5
a1.sources.avroSource.interceptors.i1.type = timestamp
a1.sources.avroSource.interceptors.i2.type = host
a1.sources.avroSource.interceptors.i3.type = regex_filter
a1.sources.avroSource.interceptors.i4.type = regex_filter
a1.sources.avroSource.interceptors.i5.type = static
a1.sources.avroSource.interceptors.i2.useIP = false
a1.sources.avroSource.interceptors.i2.hostHeader = host
a1.sources.avroSource.interceptors.i3.regex = .*AppLogging.*
a1.sources.avroSource.interceptors.i3.excludeEvents = false
a1.sources.avroSource.interceptors.i4.regex = .*;ERORR;.*
a1.sources.avroSource.interceptors.i4.excludeEvents = true
a1.sources.avroSource.interceptors.i5.key = instance
a1.sources.avroSource.interceptors.i5.value = JBT_TEST
#define channel
a1.channels.ch1.type = memory
a1.channels.ch1.capacity = 1000
a1.channels.ch1.transactionCapacity = 100
#define sink
a1.sinks.hdfsSink.type = hdfs
a1.sinks.hdfsSink.channel = ch1
a1.sinks.hdfsSink.hdfs.writeFormat = Text
a1.sinks.hdfsSink.hdfs.filePrefix =  %{host}
a1.sinks.hdfsSink.hdfs.fileType = DataStream
a1.sinks.hdfsSink.hdfs.path = hdfs://192.168.79.128:9000/hdfs/mr/input/flume/%{instance}/
a1.sinks.hdfsSink.hdfs.useLocalTimeStamp = true
a1.sinks.hdfsSink.hdfs.rollInterval = 0
a1.sinks.hdfsSink.hdfs.rollCount = 0
a1.sinks.hdfsSink.hdfs.rollSize = 0
a1.sinks.hdfsSink.hdfs.batchSize = 1

如果我将接收器从 hdfs 更改为 file_roll，则水槽代理会创建日志文件，但没有任何内容。这是来自水槽的新 conf-file。

#define components of agent
a1.sources = avroSource
a1.channels = ch1
a1.sinks = file

#define source
a1.sources.avroSource.type = avro
a1.sources.avroSource.bind = localhost
a1.sources.avroSource.port = 41415
a1.sources.avroSource.channels = ch1

#define channel
a1.channels.ch1.type = memory
a1.channels.ch1.capacity = 1000
a1.channels.ch1.transactionCapacity = 100

#define sink
a1.sinks.file.type = file_roll
a1.sinks.file.channel = ch1
a1.sinks.file.sink.directory = c:/temp/
a1.sinks.file.sink.rollInterval = 600

我的错误是什么？

Answer 1

在配置的末尾添加以下两行，以将源和接收器与通道连接起来。让我知道这是否有效。

a1.sources.avroSource.channels = ch1
a1.sinks.file.channel = ch1