I am using lucene to remove English Stop words but my requirement is remove English stop words and Custom stop words. Below is my code to remove English stop words using lucene.
My Sample Code:
public class Stopwords_remove {
public String removeStopWords(String string) throws IOException
{
StandardAnalyzer ana = new StandardAnalyzer(Version.LUCENE_30);
TokenStream tokenStream = new StandardTokenizer(Version.LUCENE_36,newStringReader(string));
StringBuilder sb = new StringBuilder();
tokenStream = new StopFilter(Version.LUCENE_36, tokenStream, ana.STOP_WORDS_SET);
CharTermAttribute token = tokenStream.getAttribute(CharTermAttribute.class);
while (tokenStream.incrementToken())
{
if (sb.length() > 0)
{
sb.append(" ");
}
sb.append(token.toString());
}
return sb.toString();
}
public static void main(String args[]) throws IOException
{
String text = "this is a java project written by james.";
Stopwords_remove stopwords = new Stopwords_remove();
stopwords.removeStopWords(text);
}
}
output: java project written james.
required output: java project james.
How can I do this?