搜索字符串时solr索引中出现意外结果

Question

我已经建立了一个SOLR环境，正在使用一个文本字段类型，我用其他几个字段填充它。

我正在经历一些奇怪的行为。每当我搜索“new”时，查询都会返回索引中包含new的结果，但也会返回一些没有“new”字符串的结果。我已经关闭了所有的过滤器工厂，但是没有用。我在查询中不断得到结果，其中不包含这个词。

下面你会发现我的solrconfig.xml和schema.xml.

字段类型text_nl：

<fieldType name="text_nl" class="solr.TextField" positionIncrementGap="100">
    <analyzer type="index">
        <tokenizer class="solr.StandardTokenizerFactory" />
        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
        <filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_nl.txt" format="snowball" />
        <filter class="solr.EdgeNGramFilterFactory" minGramSize="3" maxGramSize="15" />
        <filter class="solr.ReversedWildcardFilterFactory"/>
        <filter class="solr.RemoveDuplicatesTokenFilterFactory" />
        <filter class="solr.LowerCaseFilterFactory"/>
    </analyzer>
    <analyzer type="query">
        <tokenizer class="solr.StandardTokenizerFactory" />
        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
        <filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_nl.txt" format="snowball" />
        <!-- <filter class="solr.EdgeNGramFilterFactory" minGramSize="3" maxGramSize="15" /> -->
        <filter class="solr.LowerCaseFilterFactory"/>
    </analyzer>
</fieldType>

字段名称：

<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="Merk" type="text_nl" indexed="false" stored="true"/>
<field name="Model" type="text_nl" indexed="false" stored="true" multiValued="true" />
<field name="Kleur" type="text_nl" indexed="false" stored="true"/>
<field name="Collectie" type="text_nl" indexed="false" stored="true"/>
<field name="Categorie" type="text_nl" indexed="true" stored="true"/>
<field name="MateriaalSoort" type="text_nl" indexed="false" stored="true"/>
<field name="Zool" type="text_nl" indexed="false" stored="true"/>
<field name="Omschrijving" type="text_nl" indexed="false" stored="true"/>
    <field name="text" type="text_nl" indexed="true" stored="true" multiValued="true"/>

Solrconfig。xml

<requestHandler name="/query" class="solr.SearchHandler">
 <lst name="defaults">
   <str name="echoParams">explicit</str>
   <int name="rows">50000</int>
   <str name="wt">json</str>
   <str name="indent">true</str>
   <str name="df">text</str>
   <str name="fl">id,Merk,Model,Kleur,Collectie,Categorie,Zool,Omschrijving</str>
   <str name="qf">Merk^100 Model^0.8 Omschrijving^0.3 id^1.0</str>
   <str name="pf">Merk^100 Model^0.8 Omschrijving^0.3 id^1.0</str>
 </lst>

数据如下：/query？q=新

产量：

{
        "id":"3215.70.101204",
        "Merk":"New balance",
        "Model":["M576"],
        "Kleur":"Groen",
        "Collectie":"Herenschoenen",
        "Categorie":"Sneakers",
        "Zool":"Rubber",
        "Omschrijving":"Groene nubuck special runner van het merk New Balance. Het logo is van groen nubuck."},
      {
        "id":"3215.26.104592",
        "Merk":"Greve",
        "Model":["6260"],
        "Kleur":"Jeans",
        "Collectie":"Herenschoenen",
        "Categorie":"Sneakers",
        "Zool":"Rubber",
        "Omschrijving":"Deze jeans blauwe suède/lederen runner is van het merk Greve. De runner heeft een merklabel van Greve aan de achterzijde. De runner heeft een witte met houten middenzool en een rubberen zool, verder heeft de runner zilveren studs details."},

如你所见，第二个id的结果中没有“新”。

这是调试查询的结果：

debug":{
    "rawquerystring":"new",
    "querystring":"new",
    "parsedquery":"text:new",
    "parsedquery_toString":"text:new",
    "explain":{
      "3215.13.101204":"\n1.4514455 = (MATCH) weight(text:new in 2047) [DefaultSimilarity], result of:\n  1.4514455 = fieldWeight in 2047, product of:\n    1.7320508 = tf(freq=3.0), with freq of:\n      3.0 = termFreq=3.0\n    4.469293 = idf(docFreq=113, maxDocs=3661)\n    0.1875 = fieldNorm(doc=2047)\n",
      "3215.30.101204":"\n1.4514455 = (MATCH) weight(text:new in 2142) [DefaultSimilarity], result of:\n  1.4514455 = fieldWeight in 2142, product of:\n    1.7320508 = tf(freq=3.0), with freq of:\n      3.0 = termFreq=3.0\n    4.469293 = idf(docFreq=113, maxDocs=3661)\n    0.1875 = fieldNorm(doc=2142)\n",
      "3215.70.101204":"\n1.4514455 = (MATCH) weight(text:new in 2217) [DefaultSimilarity], result of:\n  1.4514455 = fieldWeight in 2217, product of:\n    1.7320508 = tf(freq=3.0), with freq of:\n      3.0 = termFreq=3.0\n    4.469293 = idf(docFreq=113, maxDocs=3661)\n    0.1875 = fieldNorm(doc=2217)\n",
      "3215.26.104592":"\n1.3966541 = (MATCH) weight(text:new in 2137) [DefaultSimilarity], result of:\n  1.3966541 = fieldWeight in 2137, product of:\n    2.0 = tf(freq=4.0), with freq of:\n      4.0 = termFreq=4.0\n    4.469293 = idf(docFreq=113, maxDocs=3661)\n    0.15625 = fieldNorm(doc=2137)\n",
      "3215.34.104592":"\n1.3966541 = (MATCH) weight(text:new in 2185) [DefaultSimilarity], result of:\n  1.3966541 = fieldWeight in 2185, product of:\n    2.0 = tf(freq=4.0), with freq of:\n      4.0 = termFreq=4.0\n    4.469293 = idf(docFreq=113, maxDocs=3661)\n    0.15625 = fieldNorm(doc=2185)\n",
      "3215.70.104592":"\n1.3966541 = (MATCH) weight(text:new in 2232) [DefaultSimilarity], result of:\n  1.3966541 = fieldWeight in 2232, product of:\n    2.0 = tf(freq=4.0), with freq of:\n      4.0 = termFreq=4.0\n    4.469293 = idf(docFreq=113, maxDocs=3661)\n    0.15625 = fieldNorm(doc=2232)\n",

Answer 1

这可能是由于EdgeNGramFilter和ReversedWildcardFilter的组合而发生的。EdgeNGramFilter首先将术语拆分为大小为3或更多的ngram。然后每一个都以正向和反向的形式索引，所以如果你索引“去”这个词，你最终会得到：

• ngrams：“文”、“耳鼻喉科”、“去了”

所以你得到了一个匹配的术语“去”和一个查询“新”。任何包含“new”或“wen”的单词都可以匹配。

真的，我认为这两种方法都太过分了。对我来说，倒车没有多大意义。这两种方法都是解决类似问题的方法，在我看来，它们一起使用是没有意义的。

此外，您还可以在“synonyms.txt”中为单词“new”定义同义词。