How can I wait for any/all pthreads to complete?

Question

I just want my main thread to wait for any and all my (p)threads to complete before exiting.

The threads come and go a lot for different reasons, and I really don't want to keep track of all of them - I just want to know when they're all gone.

wait() does this for child processes, returning ECHILD when there are no children left, however wait does not (appear to work with) (p)threads.

I really don't want to go through the trouble of keeping a list of every single outstanding thread (as they come and go), then having to call pthread_join on each.

As there a quick-and-dirty way to do this?

Answer 1

The proper way is to keep track of all of your pthread_id's, but you asked for a quick and dirty way so here it is. Basically:

• just keep a total count of running threads,
• increment it in the main loop before calling pthread_create,
• decrement the thread count as each thread finishes.
• Then sleep at the end of the main process until the count returns to 0.

.

volatile int running_threads = 0;
pthread_mutex_t running_mutex = PTHREAD_MUTEX_INITIALIZER;

void * threadStart()
{
   // do the thread work
   pthread_mutex_lock(&running_mutex);
   running_threads--;
   pthread_mutex_unlock(&running_mutex);
}

int main()
{
  for (i = 0; i < num_threads;i++)
  {
     pthread_mutex_lock(&running_mutex);
     running_threads++;
     pthread_mutex_unlock(&running_mutex);
     // launch thread

  }

  while (running_threads > 0)
  {
     sleep(1);
  }
}

Answer 2

If you don't want to keep track of your threads then you can detach the threads so you don't have to care about them, but in order to tell when they are finished you will have to go a bit further.

One trick would be to keep a list (linked list, array, whatever) of the threads' statuses. When a thread starts it sets its status in the array to something like THREAD_STATUS_RUNNING and just before it ends it updates its status to something like THREAD_STATUS_STOPPED. Then when you want to check if all threads have stopped you can just iterate over this array and check all the statuses.

Don't forget though that if you do something like this, you will need to control access to the array so that only one thread can access (read and write) it at a time, so you'll need to use a mutex on it.

Answer 3

Thanks all for the great answers! There has been a lot of talk about using memory barriers etc - so I figured I'd post an answer that properly showed them used for this.

#define NUM_THREADS 5

unsigned int thread_count;
void *threadfunc(void *arg) {
  printf("Thread %p running\n",arg);
  sleep(3);
  printf("Thread %p exiting\n",arg);
  __sync_fetch_and_sub(&thread_count,1);
  return 0L;
}

int main() {
  int i;
  pthread_t thread[NUM_THREADS];

  thread_count=NUM_THREADS;
  for (i=0;i<NUM_THREADS;i++) {
    pthread_create(&thread[i],0L,threadfunc,&thread[i]);
  }

  do {
    __sync_synchronize();
  } while (thread_count);
  printf("All threads done\n");
}

Note that the __sync macros are "non-standard" GCC internal macros. LLVM supports these too - but if your using another compiler, you may have to do something different.

Another big thing to note is: Why would you burn an entire core, or waste "half" of a CPU spinning in a tight poll-loop just waiting for others to finish - when you could easily put it to work? The following mod uses the initial thread to run one of the workers, then wait for the others to complete:

  thread_count=NUM_THREADS;
  for (i=1;i<NUM_THREADS;i++) {
    pthread_create(&thread[i],0L,threadfunc,&thread[i]);
  }

  threadfunc(&thread[0]);

  do {
    __sync_synchronize();
  } while (thread_count);
  printf("All threads done\n");
}

Note that we start creating the threads starting at "1" instead of "0", then directly run "thread 0" inline, waiting for all threads to complete after it's done. We pass &thread[0] to it for consistency (even though it's meaningless here), though in reality you'd probably pass your own variables/context.

Answer 4

Do you want your main thread to do anything in particular after all the threads have completed?

If not, you can have your main thread simply call pthread_exit() instead of returning (or calling exit()).

If main() returns it implicitly calls (or behaves as if it called) exit(), which will terminate the process. However, if main() calls pthread_exit() instead of returning, that implicit call to exit() doesn't occur and the process won't immediately end - it'll end when all threads have terminated.

• http://pubs.opengroup.org/onlinepubs/007908799/xsh/pthread_exit.html

Can't get too much quick-n-dirtier.

Here's a small example program that will let you see the difference. Pass -DUSE_PTHREAD_EXIT to the compiler to see the process wait for all threads to finish. Compile without that macro defined to see the process stop threads in their tracks.

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <time.h>

static
void sleep(int ms)
{
    struct timespec waittime;

    waittime.tv_sec = (ms / 1000);
    ms = ms % 1000;
    waittime.tv_nsec = ms * 1000 * 1000;

    nanosleep( &waittime, NULL);
}

void* threadfunc( void* c)
{
    int id = (int) c;
    int i = 0;

    for (i = 0 ; i < 12; ++i) {
        printf( "thread %d, iteration %d\n", id, i);
        sleep(10);
    }

    return 0;
}


int main()
{
    int i = 4;

    for (; i; --i) {
        pthread_t* tcb = malloc( sizeof(*tcb));

        pthread_create( tcb, NULL, threadfunc, (void*) i);
    }

    sleep(40);

#ifdef USE_PTHREAD_EXIT
    pthread_exit(0);
#endif

    return 0;
}

Answer 5

you could keep a list all your thread ids and then do pthread_join on each one, of course you will need a mutex to control access to the thread id list. you will also need some kind of list that can be modified while being iterated on, maybe a std::set<pthread_t>?

int main() {
   pthread_mutex_lock(&mutex);

   void *data;
   for(threadId in threadIdList) {
      pthread_mutex_unlock(&mutex);
      pthread_join(threadId, &data);
      pthread_mutex_lock(&mutex);
   }

   printf("All threads completed.\n");
}

// called by any thread to create another
void CreateThread()
{
   pthread_t id;

   pthread_mutex_lock(&mutex);
   pthread_create(&id, NULL, ThreadInit, &id); // pass the id so the thread can use it with to remove itself
   threadIdList.add(id);
   pthread_mutex_unlock(&mutex);  
}

// called by each thread before it dies
void RemoveThread(pthread_t& id)
{
   pthread_mutex_lock(&mutex);
   threadIdList.remove(id);
   pthread_mutex_unlock(&mutex);
}