Converting a sentence string to a string array of

Question

I need my Java program to take a string like:

"This is a sample sentence."

and turn it into a string array like:

{"this","is","a","sample","sentence"}

No periods, or punctuation (preferably). By the way, the string input is always one sentence.

Is there an easy way to do this that I'm not seeing? Or do we really have to search for spaces a lot and create new strings from the areas between the spaces (which are words)?

Answer 1

You can use BreakIterator.getWordInstance to find all words in a string.

public static List<String> getWords(String text) {
    List<String> words = new ArrayList<String>();
    BreakIterator breakIterator = BreakIterator.getWordInstance();
    breakIterator.setText(text);
    int lastIndex = breakIterator.first();
    while (BreakIterator.DONE != lastIndex) {
        int firstIndex = lastIndex;
        lastIndex = breakIterator.next();
        if (lastIndex != BreakIterator.DONE && Character.isLetterOrDigit(text.charAt(firstIndex))) {
            words.add(text.substring(firstIndex, lastIndex));
        }
    }

    return words;
}

Test:

public static void main(String[] args) {
    System.out.println(getWords("A PT CR M0RT BOUSG SABN NTE TR/GB/(G) = RAND(MIN(XXX, YY + ABC))"));
}

Ouput:

[A, PT, CR, M0RT, BOUSG, SABN, NTE, TR, GB, G, RAND, MIN, XXX, YY, ABC]

Answer 2

Following is a code snippet which splits a sentense to word and give its count too.

 import java.util.HashMap;
 import java.util.Iterator;
 import java.util.Map;

 public class StringToword {
public static void main(String[] args) {
    String s="a a a A A";
    String[] splitedString=s.split(" ");
    Map m=new HashMap();
    int count=1;
    for(String s1 :splitedString){
         count=m.containsKey(s1)?count+1:1;
          m.put(s1, count);
        }
    Iterator<StringToword> itr=m.entrySet().iterator();
    while(itr.hasNext()){
        System.out.println(itr.next());         
    }
    }

}

Answer 3

Another way to do that is StringTokenizer. ex:-

 public static void main(String[] args) {

    String str = "This is a sample string";
    StringTokenizer st = new StringTokenizer(str," ");
    String starr[]=new String[st.countTokens()];
    while (st.hasMoreElements()) {
        starr[i++]=st.nextElement();
    }
}

Answer 4

String.split() will do most of what you want. You may then need to loop over the words to pull out any punctuation.

For example:

String s = "This is a sample sentence.";
String[] words = s.split("\\s+");
for (int i = 0; i < words.length; i++) {
    // You may want to check for a non-word character before blindly
    // performing a replacement
    // It may also be necessary to adjust the character class
    words[i] = words[i].replaceAll("[^\\w]", "");
}

Answer 5

Now, this can be accomplished just with split as it takes regex:

String s = "This is a sample sentence with []s.";
String[] words = s.split("\\W+");

this will give words as: {"this","is","a","sample","sentence", "s"}

The \\W+ will match all non-alphabetic characters occurring one or more times. So there is no need to replace. You can check other patterns also.

Answer 6

You can use simple following code

String str= "This is a sample sentence.";
String[] words = str.split("[[ ]*|[//.]]");
for(int i=0;i<words.length;i++)
System.out.print(words[i]+" ");