如何在Elasticsearch提示器中使用Logstash导入CSV数据以完成字段类型

Question

ElasticSearch索引创建

curl -XPOST 'http://localhost:9200/music/' -d '{}'

场图

curl -XPUT 'http://localhost:9200/music/_mapping/song' -d '
{
  "properties": {
    "name" : {
      "type" : "string"
    },
    "suggest": {
      "type" : "completion"
    }
  }
}'

LogStash配置文件musicStash.config

input {
    file {
        path => "pathToCsv"
        start_position => beginning
    }
}

filter {  
    csv {
        columns => ["id", "name", "suggest"]
        separator => ","
    }
}

output {
    elasticsearch {
        hosts => "localhost"
        index => "music"
        document_id => "%{id}"
    }
}

现在，在执行logstash配置文件时，在elasticsearch控制台中收到以下异常

failed to put mappings on indices [[music]], type [logs]
java.lang.IllegalArgumentException: Mapper for [suggest] conflicts with existing mapping in other types:
[mapper [suggest] cannot be changed from type [completion] to [string]]
at org.elasticsearch.index.mapper.FieldTypeLookup.checkCompatibility(FieldTypeLookup.java:117)

在Logstash控制台中收到错误，

response=>{"index"=>{"_index"=>"music", "_type"=>"logs", "_id"=>"5", "status"=>400, 
"error"=>{"type"=>"illegal_argument_exception", 
"reason"=>"Mapper for [suggest] conflicts with existing mapping in other types:\n[mapper [suggest] cannot be changed from type [completion] to [string]]"}}}, :level=>:warn}

那么如何通过Logstash导入csv文件来实现elasticsearch自动完成功能。

Answer 1

您在elasticsearch输出中缺少以下设置：

document_type => "song"

发生的是，logstash创建了一个新类型logs（默认情况下），并且由于从ES 2.0版本开始，它被禁止在同一个索引中包含两个具有相同名称但不同类型（stringvs
completion）的字段，因此会出错。

只需像这样修改您的输出，它将起作用：

output {
    elasticsearch {
        hosts => "localhost"
        index => "music"
        document_type => "song"
        document_id => "%{id}"
    }
}