graylog的log4j2配置

Question

我们希望将所有java应用程序日志集中在Graylog服务器上。我们使用apache tomcat作为容器，使用log4j作为日志框架。log4j2.xml

        <Socket name="GELF" protocol="tcp" host="graylog.domain.com" port="12201">
            <GelfLayout host="%host">
                    <KeyValuePair key="projectName" value="magus"/>
                    <KeyValuePair key="level" value="%level"/>
                    <KeyValuePair key="timestamp" value="%d"/>
                    <KeyValuePair key="server" value="%host"/>
                    <KeyValuePair key="logStream" value="magus"/>
                    <KeyValuePair key="version" value="1.1"/>
                    <KeyValuePair key="projectName" value="magus"/>
                    <KeyValuePair key="logStream" value="magus"/>
                    <KeyValuePair key="className" value="%C"/>
                    <KeyValuePair key="simpleClassName" value="%C{1}"/>
            </GelfLayout>
    </Socket>

伐木工人

    <Loggers>
       <Root level="INFO">
          <AppenderRef ref="CONSOLE"/>
          <AppenderRef ref="LOG"/>
          <AppenderRef ref='GELF'/>
       </Root>
    </Loggers>

日志详细信息

2021-01-26 20:05:01,343 http-nio-31381-exec-1 DEBUG Reconnecting /graylog.domain.com:12201
2021-01-26 20:05:01,344 http-nio-31381-exec-1 DEBUG Creating socket /graylog.domain.com:12201
2021-01-26 20:05:01,344 http-nio-31381-exec-1 DEBUG Closing SocketOutputStream    java.net.SocketOutputStream@8cb01fa
2021-01-26 20:05:01,345 http-nio-31381-exec-1 DEBUG Connection to graylog.domain.com:12201 reestablished: Socket[addr=/graylog.domain.com,port=12201,localport=41482]

正如你们所见，我的应用程序与gray log server创建了一个套接字连接。但是我们在灰色日志服务器上没有看到任何日志

版本：tomcat-9.0.16.0 jdk-1.8.0_201-b09（64位）
log4j2-1.12/1.14（均选中）
os-Linux 3.10.0-1062.12.1.el7.x86_64
graylogsrv上的Graylog 3.0.2 1686930（Linux 3.10.0-1062.9.1上的Oracle Corporation 1.8.0_232.el7.x86_64）

文档https://logging.apache.org/log4j/2.x/manual/layouts.html#GELFLayout

我想使用log4j2以外的外部库，如logstash-gelle

更新这是灰色日志服务器日志

2021-01-27T12:18:45.079+04:00 ERROR [DecodingProcessor] Unable to decode raw message RawMessage{id=45f04b90-6078-11eb-80bf-00505696a882, journalOffset=2771397770, codec=gelf, payloadSize=11, timestamp=2021-01-27T08:18:45.065Z, remoteAddress=/graylog.domain:58258} on input <600ecd97f7c4b60478f4504e>.
2021-01-27T12:18:45.079+04:00 ERROR [DecodingProcessor] Error processing message RawMessage{id=45f04b90-6078-11eb-80bf-00505696a882, journalOffset=2771397770, codec=gelf, payloadSize=11, timestamp=2021-01-27T08:18:45.065Z, remoteAddress=/graylog.domain:58258}
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('�' (code 65533 / 0xfffd)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: �Y�n�8��h; line: 1, column: 2]
    at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1702) ~[graylog.jar:?]
    at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:558) ~[graylog.jar:?]
    at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:456) ~[graylog.jar:?]
    at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1906) ~[graylog.jar:?]
    at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:749) ~[graylog.jar:?]
    at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3850) ~[graylog.jar:?]
    at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3799) ~[graylog.jar:?]
    at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2397) ~[graylog.jar:?]
    at org.graylog2.inputs.codecs.GelfCodec.decode(GelfCodec.java:127) ~[graylog.jar:?]
    at org.graylog2.shared.buffers.processors.DecodingProcessor.processMessage(DecodingProcessor.java:150) ~[graylog.jar:?]
    at org.graylog2.shared.buffers.processors.DecodingProcessor.onEvent(DecodingProcessor.java:91) [graylog.jar:?]
    at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:74) [graylog.jar:?]
    at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:42) [graylog.jar:?]
    at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
    at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]

我们如何获得原始数据来发现错误？

Answer 1

终于解决了。根据文件

由于使用空字节（\0）作为帧分隔符，GELF TCP不支持压缩。

因此，在禁用log4j2配置上的compress之后，我们在灰色日志服务器上看到了我们的日志。下面的代码段是一个工作示例

<Socket name="GELF" protocol="tcp" host="graylog.domain.com" port="12201">
            <!-- gelf tcp does not support compression-->
            <GelfLayout includeStackTrace="true" host="${hostName}" includeThreadContext="true" includeNullDelimiter="true"
                  compressionType="OFF">
                
                <KeyValuePair key="host" value="${hostName}"/>
                <KeyValuePair key="version" value="1.1"/>
                <!--<KeyValuePair key="short_message" value="$${event:Message}"/>--><!--   not required             -->
                <KeyValuePair key="application_name" value="${web:contextPathName}"/>
                <KeyValuePair key="thread_id" value="$${event:ThreadId}"/>
                <KeyValuePair key="thread_name" value="$${event:ThreadName}"/>
                <KeyValuePair key="timestamp" value="$${event:Timestamp}"/>
                <!--<KeyValuePair key="level" value="1"/>--><!-- default level type is number so we don't use-->
                <KeyValuePair key="log_level" value="$${event:Level}"/><!-- for readabilty and filtering-->
            </GelfLayout>
        </Socket>