-->

算法来从复合(混合)的句子中提取简单句?(algorithm to extract simple s

2019-09-16 08:40发布

是否有可用于提取段落的简单句的算法?

我的最终目标是要在以后运行的结果简单的句子另一种算法来确定作者的情绪。

我从来源,如蔡-DEUG公园研究这一点,但没有讨论准备简单的句子作为训练数据。

提前致谢

Answer 1:

我刚使用openNLP的一样。

public static List<String> breakIntoSentencesOpenNlp(String paragraph) throws FileNotFoundException, IOException,
        InvalidFormatException {

    InputStream is = new FileInputStream("resources/models/en-sent.bin");
    SentenceModel model = new SentenceModel(is);
    SentenceDetectorME sdetector = new SentenceDetectorME(model);

    String[] sentDetect = sdetector.sentDetect(paragraph);
    is.close();
    return Arrays.asList(sentDetect);
}

    //Failed at Hi.
    paragraph = "Hi. How are you? This is Mike.";
    SentenceDetector.breakIntoSentencesOpenNlp(paragraph).forEach(sentence -> System.out.println(sentence));

    //Failed at Door.Noone
    paragraph = "Close the Door.Noone is out there";
    SentenceDetector.breakIntoSentencesOpenNlp(paragraph).forEach(sentence -> System.out.println(sentence));//not able to break on noone

    paragraph = "Really!! I cant believe. Mr. Wilson can come any moment to receive mrs. watson.";
    SentenceDetector.breakIntoSentencesOpenNlp(paragraph).forEach(sentence -> System.out.println(sentence));

    //Failed at dr.
    paragraph = "Radhika, Mohan, and Shaik went to meet dr. Kashyap to raise fund for poor patients.";
    SentenceDetector.breakIntoSentencesOpenNlp(paragraph).forEach(sentence -> System.out.println(sentence));//breaking on dr.

    paragraph = "This is how I tried to split a paragraph into a sentence. But, there is a problem. My paragraph includes dates like Jan.13, 2014 , words like U.S. and numbers like 2.2. They all got splitted by the above code.";
    SentenceDetector.breakIntoSentencesOpenNlp(paragraph).forEach(sentence -> System.out.println(sentence));//breaking on dr.

    paragraph = "www.thinkzarahatke.com is the second site I developed. You can send mail to admin@thinkzarahatke.com";
    SentenceDetector.breakIntoSentencesOpenNlp(paragraph).forEach(sentence -> System.out.println(sentence));

它失败时,才会有人类的错误。 例如。 “博士” 缩写应该有资本d,并且至少有1空间2句之间的预期。

您也可以使用实现它RE在下面的方式;

public static List<String> breakIntoSentencesCustomRESplitter(String paragraph){
    List<String> sentences = new ArrayList<String>();
    Pattern re = Pattern.compile("[^.!?\\s][^.!?]*(?:[.!?](?!['\"]?\\s|$)[^.!?]*)*[.!?]?['\"]?(?=\\s|$)", Pattern.MULTILINE | Pattern.COMMENTS);
    Matcher reMatcher = re.matcher(paragraph);
    while (reMatcher.find()) {
        sentences.add(reMatcher.group());
    }
    return sentences;

}

    paragraph = "Hi. How are you? This is Mike.";
    SentenceDetector.breakIntoSentencesCustomRESplitter(paragraph).forEach(sentence -> System.out.println(sentence));

    //Failed at Door.Noone
    paragraph = "Close the Door.Noone is out there";
    SentenceDetector.breakIntoSentencesCustomRESplitter(paragraph).forEach(sentence -> System.out.println(sentence));

    //Failed at Mr., mrs.
    paragraph = "Really!! I cant believe. Mr. Wilson can come any moment to receive mrs. watson.";
    SentenceDetector.breakIntoSentencesCustomRESplitter(paragraph).forEach(sentence -> System.out.println(sentence));

    //Failed at dr.
    paragraph = "Radhika, Mohan, and Shaik went to meet dr. Kashyap to raise fund for poor patients.";
    SentenceDetector.breakIntoSentencesCustomRESplitter(paragraph).forEach(sentence -> System.out.println(sentence));

    //Failed at U.S.
    paragraph = "This is how I tried to split a paragraph into a sentence. But, there is a problem. My paragraph includes dates like Jan.13, 2014 , words like U.S. and numbers like 2.2. They all got splitted by the above code.";
    SentenceDetector.breakIntoSentencesCustomRESplitter(paragraph).forEach(sentence -> System.out.println(sentence));

    paragraph = "www.thinkzarahatke.com is the second site I developed. You can send mail to admin@thinkzarahatke.com";
    SentenceDetector.breakIntoSentencesCustomRESplitter(paragraph).forEach(sentence -> System.out.println(sentence));

但是,错误是竞争力高。 另一种方法是使用的BreakIterator;

public static List<String> breakIntoSentencesBreakIterator(String paragraph){
    List<String> sentences = new ArrayList<String>();
    BreakIterator sentenceIterator =
            BreakIterator.getSentenceInstance(Locale.ENGLISH);
    BreakIterator sentenceInstance = sentenceIterator.getSentenceInstance();
    sentenceInstance.setText(paragraph);

    int end = sentenceInstance.last();
     for (int start = sentenceInstance.previous();
          start != BreakIterator.DONE;
          end = start, start = sentenceInstance.previous()) {
         sentences.add(paragraph.substring(start,end));
     }

     return sentences;
}

例:

    paragraph = "Hi. How are you? This is Mike.";
    SentenceDetector.breakIntoSentencesBreakIterator(paragraph).forEach(sentence -> System.out.println(sentence));

    //Failed at Door.Noone
    paragraph = "Close the Door.Noone is out there";
    SentenceDetector.breakIntoSentencesBreakIterator(paragraph).forEach(sentence -> System.out.println(sentence));

    //Failed at Mr.
    paragraph = "Really!! I cant believe. Mr. Wilson can come any moment to receive mrs. watson.";
    SentenceDetector.breakIntoSentencesBreakIterator(paragraph).forEach(sentence -> System.out.println(sentence));

    //Failed at dr.
    paragraph = "Radhika, Mohan, and Shaik went to meet dr. Kashyap to raise fund for poor patients.";
    SentenceDetector.breakIntoSentencesBreakIterator(paragraph).forEach(sentence -> System.out.println(sentence));


    paragraph = "This is how I tried to split a paragraph into a sentence. But, there is a problem. My paragraph includes dates like Jan.13, 2014 , words like U.S. and numbers like 2.2. They all got splitted by the above code.";
    SentenceDetector.breakIntoSentencesBreakIterator(paragraph).forEach(sentence -> System.out.println(sentence));

    paragraph = "www.thinkzarahatke.com is the second site I developed. You can send mail to admin@thinkzarahatke.com";
    SentenceDetector.breakIntoSentencesBreakIterator(paragraph).forEach(sentence -> System.out.println(sentence));

标杆

  • 定制RE:7毫秒
  • 的BreakIterator:143毫秒
  • openNlp:255毫秒


Answer 2:

看看阿帕奇OpenNLP ,它有一个句子的检测器模块。 该文件对如何从命令行和API使用它的例子。



文章来源: algorithm to extract simple sentences from complex(mixed) sentences?