Elasticsearch停止令牌过滤器不工作

Question

我在Elasticsearch 7.10中创建了一个索引，如下所示：

{
  "mappings": {
    "properties": {
      "id": {
        "type": "keyword"
      },
      "title": {
        "type": "text"
      },
      "description": {
        "type": "text"
      }
    }
  },
  "settings": {
    "analysis": {
      "analyzer": {
        "my_analyzer": {
          "filter": [
            "lowercase",
            "stemmer",
            "stop"
          ],
          "type": "custom",
          "tokenizer": "standard"
        }
      }
    }
  }
}

如您所见，我配置了一个名为my_analyzer的自定义分析器，该分析器应用了令牌过滤器。基于文档，我希望这个过滤器在索引时从文档的所有类型属性中删除英语stopwords。

事实上，如果我向http://localhost:30200/my_index/_analyze使用此请求主体：

{
  "analyzer": "my_analyzer",
  "text": "If you are a horse, I do not want that cake"
}

我得到一个响应，表明令牌if、a、not和已从提供的文本中删除：

{
    "tokens": [
        {
            "token": "you",
            "start_offset": 3,
            "end_offset": 6,
            "type": "<ALPHANUM>",
            "position": 1
        },
        {
            "token": "ar",
            "start_offset": 7,
            "end_offset": 10,
            "type": "<ALPHANUM>",
            "position": 2
        },
        {
            "token": "hors",
            "start_offset": 13,
            "end_offset": 18,
            "type": "<ALPHANUM>",
            "position": 4
        },
        {
            "token": "i",
            "start_offset": 20,
            "end_offset": 21,
            "type": "<ALPHANUM>",
            "position": 5
        },
        {
            "token": "do",
            "start_offset": 22,
            "end_offset": 24,
            "type": "<ALPHANUM>",
            "position": 6
        },
        {
            "token": "want",
            "start_offset": 29,
            "end_offset": 33,
            "type": "<ALPHANUM>",
            "position": 8
        },
        {
            "token": "cake",
            "start_offset": 39,
            "end_offset": 43,
            "type": "<ALPHANUM>",
            "position": 10
        }
    ]
}

但是，如果我索引一个文档，其属性包含字符串“if you are horse，I don’t want that cake”，然后通过请求http://localhost:30200/my_index/_search使用此请求主体：

{
  "query": {
    "multi_match" : {
      "query": "that", 
      "fields": ["description"]
    }
  }
}

返回文档，即使本应被分析仪删除的单词“that”：

{
    "took": 3,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 0.2876821,
        "hits": [
            {
                "_index": "my_index",
                "_type": "_doc",
                "_id": "27ibobulhqhc7s96jbz6653ud",
                "_score": 0.2876821,
                "_source": {
                    "id": "27ibobulhqhc7s96jbz6653ud",
                    "title": "muscular yak",
                    "description": "If you are a horse, I do not want that cake"
                }
            }
        ]
    }
}

那么是什么？如果停止过滤器从索引的文本属性中剥离英语停止词，我希望查询其中一个停止词会返回零结果。在索引文档或处理查询时，我是否必须明确告诉Elasticsearch使用my_analyzer？

值得一提的是，我配置的其他过滤器（小写和词干分析器）似乎按预期工作。这只会给我带来麻烦。

Answer 1

您就快到了。您只需要将您的描述字段与您创建的客户分析器进行映射，如下所示。这将确保描述字段的内容在索引和搜索时使用my_analyzer。

{
  "mappings": {
    "properties": {
      "id": {
        "type": "keyword"
      },
      "title": {
        "type": "text"
      },
      "description": {
        "type": "text",
        "analyzer": "my_analyzer"          // note this
      }
    }
  },
  "settings": {
    "analysis": {
      "analyzer": {
        "my_analyzer": {
          "filter": [
            "lowercase",
            "stemmer",
            "stop"
          ],
          "type": "custom",
          "tokenizer": "standard"
        }
      }
    }
  }
}