Stemming english words using Lucene 6

Question

I'm looking to stem English words using Lucene 6.5. I've seen quite a number of examples of using Lucene to achieve this. However, the examples I've seen so far seem to be using old versions of Lucene and replicating the same using Lucene 6 has not been possible.

A case in point is this one. The suggested and accepted solutions use org.apache.lucene.analysis.PorterStemmer which doesn't seem to be in the same package in Lucene 6.

UPDATE: I've found out that the current full path for the PorterStemmer stemmer is org.apache.lucene.analysis.en.PorterStemFilter. Additionally, one needs the dependency "org.apache.lucene" % "lucene-queryparser" % "6.5.0".

I'm now working on stemming a list of words. And I think I'll just pivot this question to word stemming since the examples I've seen using this stemmer do not seem to work nor compile with the current version of Lucene (ver 6.5.0).

Answer 1

I finally found a way to stem words using Lucene 6:

public List<String> stem(String term) throws Exception {
    Analyzer analyzer = new StandardAnalyzer();
    TokenStream result = analyzer.tokenStream(null, term);
    result = new PorterStemFilter(result);
    result = new StopFilter(result, StopAnalyzer.ENGLISH_STOP_WORDS_SET);
    CharTermAttribute resultAttr = result.addAttribute(CharTermAttribute.class);
    result.reset();

    List<String> tokens = new ArrayList<>();
    while (result.incrementToken()) {
        tokens.add(resultAttr.toString());
    }
    return tokens;
}

Calling this method with an input string, term, will return a list of string tokens generated from the input string. The method, additionally, removes stop words from the input. I'll leave this here in the hope that it'll be helpful to someone.