使用logstash将json文件数据导入弹性搜索

Question

我想将json文件数据导入弹性搜索。这是我的logstash配置文件-

输入{file{type=

输出{stdout{codec=

这是我的json文件---

{"水果"："苹果"，"大小"："小"，"颜色"："红色" }, { "水果"："木瓜"，"大小"："大"，"颜色"："黄色"测试"："甜"}

我使用这个命令执行了上面的配置文件----

logstash -f logstashcon.conf

但我在弹性搜索索引中得到了如下数据--

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 10,
    "max_score": 1,
    "hits": [
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablA",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.302Z",
          "@version": "1",
          "host": "user-102",
          "message": "{\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablB",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.694Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"fruit\": \"Apple\",\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablE",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.696Z",
          "@version": "1",
          "host": "user-102",
          "message": "},\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablC",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.695Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"size\": \"Large\",\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablD",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.696Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"color\": \"Red\"\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablG",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.698Z",
          "@version": "1",
          "host": "user-102",
          "message": "\"fruit\": \"Papaya\",\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablJ",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.699Z",
          "@version": "1",
          "host": "user-102",
          "message": "}\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablH",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.699Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"size\": \"Large\",\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablF",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.698Z",
          "@version": "1",
          "host": "user-102",
          "message": "{\r",
          "type": "json"
        }
      },
      {
        "_index": "jsondata1",
        "_type": "json",
        "_id": "AWNniXbgMkzPgBTTablI",
        "_score": 1,
        "_source": {
          "path": "C:\\Users\\Desktop\\newJSON.json",
          "@timestamp": "2018-05-16T06:00:48.699Z",
          "@version": "1",
          "host": "user-102",
          "message": "    \"color\": \"Yellow\"\r",
          "type": "json"
        }
      }
    ]
  }
}

请帮我获得正确的输出谢谢！

Answer 1

你需要使用logstash的JSON过滤器插件来构建你的输出，

例如，如果在消息字段中有JSON数据，则过滤器将，

filter {
  json {
    source => "message"
  }
}

文件上是这么说的，

它接受一个包含JSON的现有字段，并将其扩展为Logstash事件中的实际数据结构。

请阅读更多关于用法和示例的信息。