Solr DataImport中的汉字问题

Question

我在Solr3.4中的中文/日文文本索引有困难。我正在使用DIH导入数据，连接块是

<dataSource type="JdbcDataSource"
    driver="com.mysql.jdbc.Driver"
    url="jdbc:mysql://localhost/db_development?useUnicode=true&amp;characterEncoding=UTF-8&amp;characterSetResults=UTF-8"
    user="user"
    useUnicode="true"
    characterEncoding="UTF-8"
    encoding="UTF-8"
    password="password"
    zeroDateTimeBehavior="convertToNull"
    name="app" />

此字段的fieldtype defn为

  <fieldType name="text_commongrams" class="solr.TextField">
    <analyzer>
      <charFilter class="solr.HTMLStripCharFilterFactory" />
      <tokenizer class="solr.ICUTokenizerFactory" />
      <filter class="solr.ICUTransformFilterFactory" id="Traditional-Simplified"/>
      <filter class="solr.ICUFoldingFilterFactory"/>
      <filter class="solr.ASCIIFoldingFilterFactory"/>
      <filter class="solr.ICUNormalizer2FilterFactory" name="nfkc_cf" mode="compose"/>
      <filter class="solr.RemoveDuplicatesTokenFilterFactory" />
      <filter class="solr.TrimFilterFactory" />
      <filter class="solr.LowerCaseFilterFactory" />
      <filter class="solr.SnowballPorterFilterFactory" language="English" protected="protwords.txt"/>
    <filter class="solr.SynonymFilterFactory"
      synonyms="synonyms.txt"
      ignoreCase="true"
      expand="true" />
    <filter class="solr.CommonGramsFilterFactory"
      words="stopwords_en.txt"
      ignoreCase="true" />
    <filter class="solr.StopFilterFactory"
      words="stopwords_en.txt"
      ignoreCase="true" />
    <filter class="solr.WordDelimiterFilterFactory"
      generateWordParts="1"
      splitOnNumerics="0"
      generateNumberParts="1"
      catenateWords="1"
      catenateNumbers="1"
      catenateAll="0"
      preserveOriginal="1" />
  </analyzer>
</fieldType>

MySQL字符编码详细信息如

+--------------------------+-----------------------------------------+
| Variable_name            | Value                                   |
+--------------------------+-----------------------------------------+
| character_set_client     | latin1                                  |
| character_set_connection | latin1                                  |
| character_set_database   | latin1                                  |
| character_set_filesystem | binary                                  |
| character_set_results    | latin1                                  |
| character_set_server     | utf8                                    |
| character_set_system     | utf8                                    |
| character_sets_dir       | /opt/local/share/mysql5/mysql/charsets/ |
+--------------------------+-----------------------------------------+

我使用Java参数-dfile.encoding=utf-8启动Solr。

输入文本为JavaOne Tokyo 2012当我将其导入到Solr中，并使用ID查询该文档时，我看到的文本为JavaOne Tokyo 2012？？®？™：？？，©？？？？？？？？？

谁能告诉我我哪里错了？

Answer 1

所以我最后不得不修改我的MySQL表，将字符串存储在UTF8中。有关如何将现有表从latin1转换为utf8的详细信息可以在这里找到。