分布追踪与弹性叠加可视化

Question

2019-06-03 10:45:00.051  INFO [currency-exchange,411a0496b048bcf4,8d40fcfea92613ad,true] 45648 --- [x-Controller-10] logger                                   : inside exchange

这是我的控制台中的日志格式。我正在使用spring cloud stream将我的日志从应用程序传输到logstash，这是logstash中的日志解析格式

grok {
              match => { "message" => "%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:severity}\s+\[%{DATA:service},%{DATA:trace},%{DATA:span},%{DATA:exportable}\]\s+%{DATA:pid}\s+---\s+\[%{DATA:thread}\]\s+%{DATA:class}\s+:\s+%{GREEDYDATA:rest}" }
       }

这是我的logstash.conf

输入{kafka{topics=>['zipkin']}}筛选{#pattern matching logback pattern grok{match=>{“message”=>“%{timestamp_iso8601:timestamp}\s+%{loglevel：severity}\s+\[%{data：service},%{data：trace},%{data：span},%{data：exportable}\]\s+%{data：pid}\s+---\s+\[%{data：thread}\]\s+%{data：class}\s+：\s+\[%{data：thread}\\s+%{greedydata：rest}”}}}}输出{elasticsearch{hosts=>}

这是我在log-stash控制台中的输出。这是解析异常

{“message”=>“[{\”traceid\“:\”411a0496b048bcf4\“,\”parentId\“:\”8d40fcfea92613ad\“,\”id\“:\”f14c1c332d2ef077\“,\”kind\“:\”client\“,\”name\“:\”get\“,\”timestamp\“:1559538900053889,\”duration\“:16783\”localendpoint\“：{\”serviceName\“:\”currency-exchange\“,\”ipv4\“:\”10.8.0.7\“},\”tags\“：{\”http.method\“：\”get\“,\”http.path\“：\”/convert/1/to/4\“}},{\”traceId\“：\”411a0496b048bcf4\“,\”parentId\“：\”411a0496b048bcf4\“,\”id\“：\”8d40fcfea92613ad\“,\”name\“：\”hystrix\“,\”timestamp\“：1559538900050039,\”duration\“：34500,\”localendpoint\“：{\”serviceName\“：\”currency-exchange\“,\”IPv4\“：”10.8.0.7\“}},{\”traceId\“:\”411a0496b048bcf4\“,\”id\“:\”411a0496b048bcf4\“,\”kind\“:\”server\“,\”name\“:\”get/convert\“,\”timestamp\“:1559538900041446,\”duration\“:44670,\”localendpoint\“：{\”serviceName\“:\”currency-exchange\“,\”ipv4\“:\”10.8.0.7\“},\”remoteEndpoint\“：{\”ipv6\“:\”::1\“,\”port\“：62200},\”tags\“：{\”http.method\“：\”get\“,\”http.path\“：\”/convert\“,\”mvc.controller.class\“：\”controll\“,\”mvc.controller.method\“:\”convert\“}}]”,“@timestamp”=>2019-06-03T05:15:00.296z,“@version”=>“1”,“tags”=>[[0]“_GrokParseFailure”]}

Answer 1

当我使用内置在Kibana中的Grok调试器（在Dev Tools下）时，我从示例日志和Grok模式中得到以下结果：

{
  "severity": "DEBUG",
  "rest": "GET \"/convert/4/to/5\", parameters={}",
  "pid": "35973",
  "thread": "nio-9090-exec-1",
  "trace": "62132b44a444425e",
  "exportable": "true",
  "service": "currency-conversion",
  "class": "o.s.web.servlet.DispatcherServlet",
  "timestamp": "2019-05-31 05:31:42.667",
  "span": "62132b44a444425e"
}

在我看来是正确的。那么缺少的部分是什么呢？

您显示的日志记录输出还包含“IPv4”：“192.168.xx.xxx”}、“RemoteEndpoint”：{“IPv6”：“：：1”、“Port”：55394}、“Tags”：...，但示例日志中没有这些内容。这些内容来自哪里？