C - Unexpected Segmentation Fault on strtok(…)

Question

I am using strtok(...) of the library and it appears to be working fine until the end condition, where it results in a segmentation fault and program crash. The API claims that strtok(...) will output a NULL when there are no more tokens to be found, which meant, I thought, that you had to catch this NULL in order to terminate any loops that you were running using strtok(...). What do I need to do to catch this NULL to prevent my program from crashing? I imagined the NULL was allowed for use as a terminating condition.

I have prepared a SSCCE for you to observe this behavior. I need strtok(...) to work for a much larger piece of software I am writing, and I am getting the exact same segmentation behavior. The output at the command line is shown below this code vignette (yes I know you use <...> to enclose libraries, but I was having difficulty getting this post to display the code libraries). I am using gcc version 4.5.3, on a Windows 8 OS, and below shows two different flavors of how I imagine one could try to catch the NULL in a loop.

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <string.h>

main(){
  char* from = "12.34.56.78";
  char * ch = ".";
  char * token = strtok(from, ch);
  printf("%s\n",token);
  while(token != NULL){
    token = strtok(NULL, ch);
    printf("%s\n", token);
  }
  printf("Broke out of loop!");
  while(strcmp(token, 0) != 0){
    printf("%s\n",token);
    token = strtok(NULL, ch);
  }
}

############ OUTPUT: ############

$ ./test
12
34
56
78
Segmentation fault (core dumped)

Answer 1

If purpose of code is only to print element separated by '.', Only change in char declaration and before printing token check for its value NULL or not !

 main(){
        char from[] = "12.34.56.78.100.101";
        char * ch = ".";
        char * token = strtok(from, ch);
        //printf("%s\n",token);
        while(token != NULL){
            printf("%s\n", token);
            token = strtok(NULL, ch);
        }
   }

OUTPUT

  ./test1
 12
 12
 34
 56
 78
 100
 101

Answer 2

The problem is that even though you terminate the loop when strtok() returns NULL, you try to print the NULL first:

  while(token != NULL){
    token = strtok(NULL, ch);
    printf("%s\n", token);    // not good when token is NULL
  }

It turns out there are several opportunities in addition to this one for segfaults in this example, as pointed out by other answers.

Here's one way to handle your example tokenization:

char from[] = "12.34.56.78";
char * ch = ".";
char * token = strtok(from, ch);
while (token != NULL){
    printf("%s\n", token);
    token = strtok(NULL, ch);
}

Answer 3

You have both memory access errors and logic errors. I will only address the memory access errors that are causing your program to crash.

strtok modifies it's first argument. Since you are passing in a string literal, it is unable to modify the string (string literals are not modifiable.)

Here's a possible fix to define from as a modifiable string array:

char from[] = "12.34.56.78";

Because strtok modifies the string passed into it, you cannot process that string again in your second while loop. You are essentially passing in a NULL into the strcmp function there. A possible fix would be to copy the from array into another buffer each time you wish to use strtok.

Answer 4

you are first checking if token is not equal to NULL(when it is, it breaks out of the while loop). Then you are comparing token, which is a NULL with a constant NUMBER? here: strcmp(token, 0) when strcmp expects 2 strings, you provide a number. strcmp will try to fetch a string at 0th address(or NULL) giving you a segmentation fault.

while(strcmp(token, 0) != 0){
    token = strtok(NULL, ch);
    printf("%s\n",token);
  }

Also this piece of code should be something like the following:

change

  char * token = strtok(from, ch);
  printf("%s\n",token);
  while(token != NULL){
    token = strtok(NULL, ch);
    printf("%s\n", token);
  }

to

  char * token = strtok(from, ch);
  printf("%s\n",token);
  while(token != NULL){
    printf("%s\n", token);
    token = strtok(NULL, ch);
  }

Answer 5

This is a problem:

  while(token != NULL){
    token = strtok(NULL, ch);
    printf("%s\n", token);
  }

You're checking for NULL, but then calling strtok again and not checking after that but before printing.

There are other problems with the code, but I suspect this is why it crashes where it does now.

Answer 6

strtok modifies its first argument. You are passing it a string from read-only memory, and the segfault occurs when strtok tries to change it. Try changing from:

char* from = "12.34.56.78";

to

char from[] = "12.34.56.78";