Stanford NLP令牌Regex--不识别NER

Question

我从几个网络搜索中拼凑出以下内容。我可以让简单的Java regex/john/与之匹配，但在使用NER时，我没有尝试过任何匹配（所有这些都是从web搜索中复制来的示例，并稍微调整了一下）。

为了清晰起见，请编辑：（下面代码中的matcher2.matches()中的成功/失败为true/false。）

我不知道我是否需要明确地提到某个模型或注释或其他东西，或者我是否遗漏了其他东西，或者我是否只是以完全错误的方式处理它。

import edu.stanford.nlp.ling.CoreAnnotations;
import edu.stanford.nlp.ling.CoreLabel;
import edu.stanford.nlp.ling.tokensregex.TokenSequenceMatcher;
import edu.stanford.nlp.ling.tokensregex.TokenSequencePattern;
import edu.stanford.nlp.pipeline.Annotation;
import edu.stanford.nlp.pipeline.StanfordCoreNLP;
import edu.stanford.nlp.util.CoreMap;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.junit.Test;

public class StanfordSandboxTest {
    private static final Log log = LogFactory.getLog(StanfordSandboxTest.class);

    @Test
    public void testFirstAttempt() {

        Properties props2;
        StanfordCoreNLP pipeline2;
        TokenSequencePattern pattern2;
        Annotation document2;
        List<CoreMap> sentences2;
        TokenSequenceMatcher matcher2;
        String text2;

        props2 = new Properties();
        props2.put("annotators", "tokenize, ssplit, pos, lemma, ner, regexner, parse, dcoref");
        pipeline2 = new StanfordCoreNLP(props2);
        text2 = "March 1, 1999";
        pattern2 = TokenSequencePattern.compile("pattern: (([{ner:DATE}])");
        document2 = new Annotation(text2);
        pipeline2.annotate(document2);
        sentences2 = document2.get(CoreAnnotations.SentencesAnnotation.class);
        matcher2 = pattern2.getMatcher(sentences2);
        log.info("testFirstAttempt: Matches2: " + matcher2.matches());

        props2 = new Properties();
        props2.put("annotators", "tokenize, ssplit, pos, lemma, ner, regexner, parse, dcoref");
        pipeline2 = new StanfordCoreNLP(props2);
        text2 = "John";
        pattern2 = TokenSequencePattern.compile("/John/");
        document2 = new Annotation(text2);
        pipeline2.annotate(document2);
        sentences2 = document2.get(CoreAnnotations.SentencesAnnotation.class);
        matcher2 = pattern2.getMatcher(sentences2);
        log.info("testFirstAttempt: Matches2: " + matcher2.matches());
    }
}

Answer 1

示例代码：

package edu.stanford.nlp.examples;

import edu.stanford.nlp.util.*;
import edu.stanford.nlp.ling.*;
import edu.stanford.nlp.pipeline.*;

import java.util.*;


public class TokensRegexExampleTwo {

  public static void main(String[] args) {

    // set up properties
    Properties props = new Properties();
    props.setProperty("annotators", "tokenize,ssplit,pos,lemma,ner,tokensregex");
    props.setProperty("tokensregex.rules", "multi-step-per-org.rules");
    props.setProperty("tokensregex.caseInsensitive", "true");

    // set up pipeline
    StanfordCoreNLP pipeline = new StanfordCoreNLP(props);

    // set up text to annotate
    Annotation annotation = new Annotation("Joe Smith works for Apple Inc.");

    // annotate text
    pipeline.annotate(annotation);

    // print out found entities
    for (CoreMap sentence : annotation.get(CoreAnnotations.SentencesAnnotation.class)) {
      for (CoreLabel token : sentence.get(CoreAnnotations.TokensAnnotation.class)) {
        System.out.println(token.word() + "\t" + token.ner());
      }
    }
  }
}

示例规则文件：

ner = { type: "CLASS", value: "edu.stanford.nlp.ling.CoreAnnotations$NamedEntityTagAnnotation" }

$ORGANIZATION_TITLES = "/inc\.|corp\./"

$COMPANY_INDICATOR_WORDS = "/company|corporation/"

ENV.defaults["stage"] = 1

{ pattern: (/works/ /for/ ([{pos: NNP}]+ $ORGANIZATION_TITLES)), action: (Annotate($1, ner, "RULE_FOUND_ORG") ) }

ENV.defaults["stage"] = 2

{ pattern: (([{pos: NNP}]+) /works/ /for/ [{ner: "RULE_FOUND_ORG"}]), action: (Annotate($1, ner, "RULE_FOUND_PERS") ) }

这将对“Joe Smith”和“Apple Inc.”应用NER标记。您可以根据您的具体情况调整这一点。请让我知道，如果你想做一些更先进的东西，而不仅仅是应用NER标签。注意：确保将这些规则放在一个名为“multi-step-per-org.rules”的文件中。