Last Updated on January 17, 2021 by Scientech Easy

StringTokenizer in Java is a utility class that breaks a String into pieces or multiple strings called “tokens”.

For example, strings separated by space and tab characters are tokens. These tokens are separated with the help of a delimiter.

A Delimiter is a group of characters that separate tokens. Each character in the delimiter is considered a valid delimiter. For example, comma, semicolon, and colon are sets of delimiters.

By default, space, tab, newline, and carriage return are used to separate the strings. But we can also use any other characters to separate tokens.

Let’s understand Java Stringtokenizer with an example.

Suppose we have a string “Hello Scientech Easy”. If we define a space as a delimiter, this string has the following three tokens:

1. Hello
2. Scientech
3. Easy

If we define a comma as a delimiter after “Hello”, the same string has the following two tokens:

1. Hello
2. Scientech Easy

StringTokenizer was added in Java 1.0 version and now it is not part of the Java Collections Framework. It is present in java.util package.

If you want to use StringTokenizer, you must import java.util package or at least StringTokenizer class into your application.

StringTokenizer class declaration

StringTokenizer class can be declared in a general form as follows:

public class StringTokenizer
   extends Object
      implements Enumeration

StringTokenizer class implements Java Enumeration interface and extends Object class.

Constructors of StringTokenizer class

The StringTokenizer class defines the following constructors that are as follows:

1. StringTokenizer(String str): This constructor is used to construct a string tokenizer for the specified string str. The parameter str is a string that will be tokenized. The default delimiters are used.

2. StringTokenizer(String str, String delim): This constructor is used to construct a string tokenizer for the specified string and delimiter. The parameter delim is a string that represents delimiters to separate the tokens.

3. StringTokenizer(String str, String delim, boolean delimAsToken): This form of constructor constructs StringTokenizer with specified string, delimiter, and returnValue.

If the delimAsToken is true, delimiter characters are returned as tokens. If it is false, delimiter characters are not returned and serve to separate tokens. Delimiters are not returned as tokens by the first two forms of the constructor.

How to use StringTokenizer in Java?

StringTokenizer class in Java is useful to separate tokens. These tokens are then stored in the StringTokenizer object from where they can be retrieved.

To use StringTokenizer class in java, we need to create an object of StringTokenizer class is as follows:

StringTokenizer st = new StringTokenizer(str, "delimiter");

In the above statement, the actual string str is broken into multiple strings or tokens at the position marked by delimiters. A StringTokenizer object returns one token at a time. We can also change the delimiter anytime.

For example, to break the string “Hello Scientech easy” whenever a comma is found, we can write as follows:

StringTokenizer st = new StringTokenizer("Hello Scientech Easy", ",");

Similarly, to break a string whenever a comma, or semi-colon, or both are found, we can specify delimiters as follows:

String delimiters = ",;";
StringTokenizer st = new StringTokenizer("Hello Scientech Easy", delimiters);

Similarly, to break a string with default delimiters such as space, tab, or newline, we can create object as:

StringTokenizer st = new StringTokenizer("Hello Scientech Easy");

Methods of StringTokenizer in Java

StringTokenizer class has defined the following methods that are as follows:

1. int countTokens(): This method counts and returns the number of tokens available in the StringTokenizer object.

2. boolean hasMoreTokens(): This method checks if there are more tokens available in the StringTokenizer object or not. If the next token is available, it will return true.

3. String nextToken(): This method returns the next token from the string tokenizer object.

4. String nextToken(String delim): It returns the next token from the string tokenizer object based on the delimiter.

5. boolean hasMoreElements(): It returns the same value as the hasMoreTokens method.

6. Object nextElement(): It returns the same value as the nextToken method, but its return type is Object.

Java StringTokenizer Example Programs

1. Let’s write a program where we will break a string into tokens using StringTokenizer. We will use hasMoreTokens() method to check if we have more tokens and nextToken() method to retrieve the next token from the string.

Look at the program source code to understand better.

Program source code 1:

import java.util.StringTokenizer;
public class StringTokens {
public static void main(String[] args) 
{
// Take a string.	
  String str = "He is a gentle man";

// Take a string for delimiters.
   String delimiters = " ,"; // Here, delimiters are a space and a comma.
   
// Create an object of string tokenizer and break into tokens. Here, delimiters are a space and a comma.
   StringTokenizer st = new StringTokenizer(str, delimiters);
   
// Counts the number of tokens available in string tokenizer object.
   int counts = st.countTokens();
   System.out.println("Number of tokens: " +counts);
   
// Now retrieves tokens from st and display them.
   System.out.println("Tokens are: ");
   while(st.hasMoreTokens()){
	  String tokens = st.nextToken();
	  System.out.println(tokens);
   }
 }
}

Output:
      Number of tokens: 5
      Tokens are: 
      He
      is
      a
      gentle
      man

Explanation: In this example program, we take a string and break it into tokens whenever a space and comma are found in the string. The tokens are retrieved from string tokenizer object as string and then displayed on the console. Look at the below figure.

2. Let’s take another example program where we will split a string into tokens based on delimiters using split() method of String class.

The split() method of String class takes a regular expression as a delimiter. Look at the source code.

Program source code 2:

public class StringTokens {
public static void main(String[] args) 
{
// Take a string.	
  String str = "You are very sweet girl";

// Take a string for delimiters.
   String delimiters = "[ ,]+"; // Here, delimiters are a space and a comma.
   
// Split the string into tokens using String.split() method.
   System.out.println("Tokens are: ");
   String[] s = str.split(delimiters);
   
   for(int i = 0 ; i 

Output:
     Tokens are: 
     You
     are
     very
     sweet
     girl

Explanation: As you can see in the output, spilt() method of String class breaks the string tokens and returns each tokens as a string.

Hope that this tutorial has covered almost all the basics points related to StringTokenizer class in Java with example programs. I hope that you will have understood this topic.
Thanks for reading!!!

The post StringTokenizer in Java | Use, Example appeared first on Scientech Easy.