How to split a string into tokens in C?

Question

How to split a string into tokens by '&' in C?

Answer 1

I would do it something like this (using strchr()):

#include <string.h>

char *data = "this&&that&other";
char *next;
char *curr = data;
while ((next = strchr(curr, '&')) != NULL) {
    /* process curr to next-1 */
    curr = next + 1;
}
/* process the remaining string (the last token) */

strchr(const char *s, int c) returns a pointer to the next location of c in s, or NULL if c isn't found in s.

You might be able to use strtok(), however, I don't like strtok(), because:

• it modifies the string being tokenized, so it doesn't work for literal strings, or is not very useful when you want to keep the string for other purposes. In that case, you must copy the string to a temporary first.
• it merges adjacent delimiters, so if your string was "a&&b&c", the returned tokens are "a", "b", and "c". Note that there is no empty token after "a".
• it is not thread-safe.

Answer 2

You can use the strok() function as shown in the example below.

/// Function to parse a string in separate tokens 

int parse_string(char pInputString[MAX_STRING_LENGTH],char *Delimiter,
                   char *pToken[MAX_TOKENS])
{
  int i;
  i = 0;

  pToken[i] = strtok(pInputString, Delimiter);
  i++;

  while ((pToken[i] = strtok(NULL, Delimiter)) != NULL){
     i++;
  }
  return i;
}

/// The array pTokens[] now contains the pointers to the start of each token in the (unchanged) original string.

sprintf(String,"Token1&Token2");
NrOfParameters = parse_string(String,"&",pTokens);

sprintf("%s, %s",pToken[0],pToken[1]);

Answer 3

strtok / strtok_r

char *token;
char *state;

for (token = strtok_r(input, "&", &state);
     token != NULL;
     token = strtok_r(NULL, "&", &state))
{
    ...
}