训练数据opennlp分类器中应该有多少行和文档

Question

我遵循Apache open-nlp的留档。我能够理解句子检测、标记器、名称查找器。但是我被分类器卡住了。原因是，我不明白，如何创建分类模型。

我确实明白我需要创建一个文件。格式非常清楚，它需要是一个类别空间和一个单行文档。以. train扩展名保存文件。

所以我创建了以下文件：

Refund What is the refund status for my order #342 ?
NewOffers Are there any new offers for your products ?

我给这个命令-

Opennlp DoccatTrainer模型en-doccat.binlang en数据en-doccat.train编码UTF-8

它开始做某事，然后返回一个错误。以下是命令提示符中的内容：

Indexing events using cutoff of 5

    Computing event counts...  done. 2 events
    Indexing...  Dropped event Refund:[bow=What, bow=is, bow=the, bow=refund, bow=status, bow=for, bow=my, bow=order, bow=#342, bow=?]
Dropped event NewOffers:[bow=Are, bow=there, bow=any, bow=new, bow=offers, bow=for, bow=your, bow=products, bow=?]
done.
Sorting and merging events... Done indexing.
Incorporating indexed data for training...  
Exception in thread "main" java.lang.NullPointerException
    at opennlp.maxent.GISTrainer.trainModel(GISTrainer.java:263)
    at opennlp.maxent.GIS.trainModel(GIS.java:256)
    at opennlp.model.TrainUtil.train(TrainUtil.java:184)
    at opennlp.tools.doccat.DocumentCategorizerME.train(DocumentCategorizerME.java:162)
    at opennlp.tools.cmdline.doccat.DoccatTrainerTool.run(DoccatTrainerTool.java:61)
    at opennlp.tools.cmdline.CLI.main(CLI.java:222)

我只是不明白为什么这里会出现空指针异常？我还试图增加两行，但没有结果。

Refund What is the refund status for my order #342 ?
NewOffers Are there any new offers for your products ?
Refund Can I place a refund request for electronics ?
NewOffers Is there any new offer on buying worth 5000 ?  

我找到了这个博客，但这里也做了几乎相同的事情。在尝试他的训练文件时，它很有魅力。我的文件中有什么问题？我如何解决错误。

当我尝试opennlp doccatrainer时，它会为我打开帮助，所以路径不是问题。感谢您的帮助。

编辑：我把文件改成了

Refund What is the refund status for my order #342 ? Can I place a refund request for clothes ?
NewOffers Are there any new offers for your products ? what are the offers on new products or new offers on old products?
Refund Can I place a refund request for electronics ?
NewOffers Is there any new offer on buying worth 5000 ? 

它是有效的，我认为它与文档有关（显然应该是两句话），并删除了最后两行。

使它

Refund What is the refund status for my order #342 ? Can I place a refund request for clothes ?
NewOffers Are there any new offers for your products ? what are the offers on new products or new offers on old products? 

但它又一次失败了，现在的问题是它需要什么样的数据/格式/文档？

谢谢

Answer 1

可以在DoccatTrainer命令中使用-cutoff标志来更改默认值。在您的情况下，您可以添加-cutoff 1，将每个类别的最小文档数设置为1。

Answer 2

您必须从每个类别中添加超过5个样本。因为默认截止标记大小为5，

请参考这篇博文http://madhawagunasekara.blogspot.com/2014/11/nlp-categorizer.html