一、编译
git clone https://github.com/mit-nlp/MITIE.git
cd MITIE
#编译python接口
python setup.py install
#编译原生c++程序
cd MITIE/tools/wordrep
mkdir build
cd build
cmake ..
make
二、使用
1.python使用
from mitie import *
2.训练工具wordrep使用
#结构目录
-MITIE/tools/wordrep/build
- CMakeCache.txt
- CMakeFiles
- cmake_install.cmake
- dlib_build
- Makefile
- mitie_build
- wordrep
#训练预料
cd ~
mkdir temp
cd temp
mv MITIE/tools/wordrep/build/wordrep .
mkdir zh
cp ~/text.txt ./zh
wordrep -e ./zh
#预料格式
你 可 不要 小看 我
你 的 身高 是 多少
我 只有 二十厘米 高 , 现在 还 在 买 半价票 呢
你 有 多 高
我 个子 不高 呢 , 只有 二十厘米 , 出门 要 把 我 抱 在 怀里
#训练完毕之后的目录
/temp
- substring_set.dat
- top_word_counts.dat
- total_word_feature_extractor.dat
- wordrep
- zh/
- substrings.txt
- top_words.txt
- word_morph_feature_extractor.dat
- word_vects.dat
#测试代码
from mitie import *
print ("loading Total Word Feature Extractor...")
twfe = total_word_feature_extractor('zh/total_word_feature_extractor.dat')
# Get fingerprint of feature dictionary
print ("Fingerprint of feature dictionary", twfe.fingerprint)
print ()
# Get number of dimensions of feature vectors
print ("Number of dimensions of feature vectors", twfe.num_dimensions)
print ()
# Get number of words in the dictionary
print ("Number of words in the dictionary", twfe.num_words_in_dictionary)
print ()
# Get list of words in the dictionary
words=twfe.get_words_in_dictionary()
print ("First 10 words in dictionary", words[0:200])
print ()
# Get features for one word
feats = twfe.get_feature_vector("我")
print ("First 5 features of word 'home'", feats[0:])
# The total word feature extractor will generate feature vectors for words not
# in its dictionary as well. It does this by looking at word morphology.
feats = twfe.get_feature_vector("_word_not_in_dictionary_")
print ("First 5 features of word '_word_not_in_dictionary_'", feats[0:50])
#输出
Number of dimensions of feature vectors 271
Number of words in the dictionary 2582
.....
...