logstash收集tomcat日志

目录

简介

JULI 组件的Handler与Formatter

修改tomcat控制台日志

修改tomcat访问access日志修改为json格式

修改tomcat其他日志

---

简介

Tomcat 的内部日志使用 JULI 组件，这是一个 Apache Commons 日志的重命名的打包分支，默认被硬编码，使用 java.util.logging 架构。这能保证 Tomcat 内部日志与 Web 应用的日志保持独立，即使 Web 应用使用的是 Apache Commons Logging。

tomcat 自身的日志有：

catalina.out
catalina.2018-01-05.log
host-manager.2018-01-05.log
localhost.2018-01-05.log
manager.2018-01-05.log
localhost_access_log.2018-01-05.log

JULI 组件的Handler与Formatter

在与logstash集成时，为了方便管理，我们修改tomcat日志格式以 “.txt”结尾，access访问日志以json格式输出，在这里我们需要注意两个关键类：

java.util.logging.AsyncFileHandler extends Handler

Handler从日志记录器获取日志消息并导出它们,。通常使用 LogManager 属性来设置 Handler 的 Filter、Formatter 和 Level 的默认值。例如，可以将它们写入控制台或写入文件，或者将它们发送到网络日志服务，或者将它们转发到操作系统日志，等等。

java.util.logging.SimpleFormatter extends Formatter

Formatter 为格式化 LogRecords 提供支持。 
一般来说，每个日志记录 Handler 都有关联的 Formatter。Formatter 接受 LogRecord，并将它转换为一个字符串。 
有些 formatter（如 XMLFormatter）需要围绕一组格式化记录来包装头部和尾部字符串。SimpleFormatter接收我们的格式化模板，按格式要求输出LogRecord。

level 设置日志级别
Level等级 SEVERE（最高值） 、WARNING 、INFO 、CONFIG 、FINE 、FINER 、FINEST（最低值）、

rotatable 是否采用日期分片，tomcat默认设置为true。文件名称如：catalina.2018-01-05.log

formatter中pattern配置时可选值

%a - Remote IP address
%A - Local IP address
%b - Bytes sent, excluding HTTP headers, or '-' if zero
%B - Bytes sent, excluding HTTP headers
%h - Remote host name (or IP address if
    <code>enableLookups</code> for the connector is false)
%H - Request protocol
%l - Remote logical username from identd (always returns
    '-')
%m - Request method (GET, POST, etc.)
%p - Local port on which this request was received.
    See also <code>%{xxx}p</code> below.
%q - Query string (prepended with a '?' if it exists)
%r - First line of the request (method and request URI)
%s - HTTP status code of the response
%S - User session ID
%t - Date and time, in Common Log Format
%u - Remote user that was authenticated (if any), else '-'
%U - Requested URL path
%v - Local server name
%D - Time taken to process the request, in millis
%T - Time taken to process the request, in seconds
%F - Time taken to commit the response, in millis
%I - Current request thread name (can compare later with stacktraces)

修改tomcat控制台日志

修改catalina.sh文件中CATALINA_OUT值

修改前

if [ -z "$CATALINA_OUT" ] ; then
  CATALINA_OUT="$CATALINA_BASE"/logs/catalina.out
fi

修该后

if [ -z "$CATALINA_OUT" ] ; then
  CATALINA_OUT=/var/{APP_LOG_PATH}/console/catalina.txt
fi

修改tomcat访问access日志修改为json格式

修改Server.xml文件中 Server>Service>Engine>Host>Valve元素

修改前
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log" suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />
修改后
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="/var/{APP_LOG_PATH}/tomcat_access"
               prefix="localhost_access_log" suffix=".txt"
               pattern="{&quot;clientip&quot;:&quot;%h&quot;,&quot;ClientUser&quot;:&quot;%l&quot;,&quot;authenticated&quot;:&quot;%u&quot;,&quot;AccessTime&quot;:&quot;%t&quot;,&quot;method&quot;:&quot;%r&quot;,&quot;status&quot;:&quot;%s&quot;,&quot;SendBytes&quot;:&quot;%b&quot;,&quot;Query?string&quot;:&quot;%q&quot;,&quot;partner&quot;:&quot;%{Referer}i&quot;,&quot;AgentVersion&quot;:&quot;%{User-Agent}i&quot;}" />

修改tomcat其他日志

对于catalina.*.log、host-manager.*.log、localhost.*.log、manager.*.log进行修改在conf/logging.properties文件中配置

……
1catalina.org.apache.juli.AsyncFileHandler.level = FINE
1catalina.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
1catalina.org.apache.juli.AsyncFileHandler.prefix = catalina.

2localhost.org.apache.juli.AsyncFileHandler.level = FINE
2localhost.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
2localhost.org.apache.juli.AsyncFileHandler.prefix = localhost.

3manager.org.apache.juli.AsyncFileHandler.level = FINE
3manager.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
3manager.org.apache.juli.AsyncFileHandler.prefix = manager.

4host-manager.org.apache.juli.AsyncFileHandler.level = FINE
4host-manager.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
4host-manager.org.apache.juli.AsyncFileHandler.prefix = host-manager.
……

修改后conf/logging.properties文件

……
1catalina.org.apache.juli.AsyncFileHandler.level = FINE
1catalina.org.apache.juli.AsyncFileHandler.directory = /var/{APP_LOG_PATH}/console
1catalina.org.apache.juli.AsyncFileHandler.rotatable=false
1catalina.org.apache.juli.AsyncFileHandler.prefix = catalina
1catalina.org.apache.juli.AsyncFileHandler.suffix = .txt

2localhost.org.apache.juli.AsyncFileHandler.level = FINE
2localhost.org.apache.juli.AsyncFileHandler.directory = /var/{APP_LOG_PATH}/console
2localhost.org.apache.juli.AsyncFileHandler.rotatable=false
2localhost.org.apache.juli.AsyncFileHandler.prefix = localhost
2localhost.org.apache.juli.AsyncFileHandler.suffix = .txt

3manager.org.apache.juli.AsyncFileHandler.level = FINE
3manager.org.apache.juli.AsyncFileHandler.directory = /var/{APP_LOG_PATH}/console
3manager.org.apache.juli.AsyncFileHandler.rotatable=false
3manager.org.apache.juli.AsyncFileHandler.prefix = manager
3manager.org.apache.juli.AsyncFileHandler.suffix = .txt

4host-manager.org.apache.juli.AsyncFileHandler.level = FINE
4host-manager.org.apache.juli.AsyncFileHandler.directory = /var/{APP_LOG_PATH}/console
4host-manager.org.apache.juli.AsyncFileHandler.rotatable=false
4host-manager.org.apache.juli.AsyncFileHandler.prefix = host-manager
4host-manager.org.apache.juli.AsyncFileHandler.suffix = .txt

java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter
java.util.logging.SimpleFormatter.format=[%4$s] [%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS.%1$tL] [%2$s] - %5$s%6$s%n
……