Difference between using StringTokenizer and Strin

Question

This question already has an answer here:

• Scanner vs. StringTokenizer vs. String.Split 10 answers

I've been using String[] split(String) of String class to split any string for some given delimiter, and it worked fine.

However, now it is expected to re-factor the same logic with StringTokenizer. But what are the differences and benifits of using one over the other.

Also, I feel that String[] returned by split() in a single call is much efficient option than using the objects of the class StringTokenizer.

Answer 1

I have the following program,

The string "x" is a tab separated 12s34;

    public class Testoken {
        public static void main(String[] args) {
            String x = "1   2   s       3           4   ";
            StringTokenizer st = new StringTokenizer(x,"\t");
            int i = 0;
            while(st.hasMoreTokens()){
                System.out.println("token-->"+st.nextToken());            
                i++;
            }
            System.out.println("i-->"+i);//elements from tokenizer
            String [] a = x.split("\t");
            System.out.println("length--->"+a.length);
            for(int y = 0;y<a.length;y++){
            System.out.println("value-->"+a[y]);//elements from split
            }
        }
    }   





Output: 

token-->1 
token-->2 
token-->s 
token-->3 
token-->4 
i-->5 
length--->8 
value-->1 
value-->2 
value-->s 
value--> 
value-->3 
value--> 
value--> 
value-->4

Answer 2

String#split accepts a regular expression whether StringTokenizer just accepts a String by which will split the string. You should always stick to the String#split, it's more robust then StringTokenizer.

Answer 3

Take a look at the JavaDocs

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.

The following example illustrates how the String.split method can be used to break up a string into its basic tokens:

 String[] result = "this is a test".split("\\s");
 for (int x=0; x<result.length; x++)
     System.out.println(result[x]);

Answer 4

-> String.split() and Pattern.split() give you an easy syntax for doing the latter, but that's essentially all that they do. If you want to parse the resulting strings, or change the delimiter halfway through depending on a particular token, they won't help you with that.

-> StringTokenizer is even more restrictive than String.split(), and also a bit fiddlier to use. It is essentially designed for pulling out tokens delimited by fixed substrings. Because of this restriction, it's about twice as fast as String.split(). (See my comparison of String.split() and StringTokenizer.) It also predates the regular expressions API, of which String.split() is a part.

You'll note from my timings that String.split() can still tokenize thousands of strings in a few milliseconds on a typical machine. In addition, it has the advantage over StringTokenizer that it gives you the output as a string array, which is usually what you want. Using an Enumeration, as provided by StringTokenizer, is too "syntactically fussy" most of the time. From this point of view, StringTokenizer is a bit of a waste of space nowadays, and you may as well just use String.split().

Answer from this link

Answer 5

http://docs.oracle.com/javase/6/docs/api/java/util/StringTokenizer.html say:

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.

So I would say, don't change it and show that line to the person who recommended refactoring it. Maybe they have old information or another good reason to tell you.

Answer 6

Read this

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.