safe method to wait for all thread timer callbacks

Question

In case of one-shot timer i can use semaphore to wait for timer callback completion. But if timer was fired several times it doesn't help. Consider the following code:

#include <stdlib.h>
#include <stdint.h>
#include <stdio.h>
#include <signal.h>
#include <time.h>
#include <unistd.h>
#include <pthread.h>

#define N 10

void timer_threaded_function(sigval_t si)
{
    uint8_t *shared_resource = si.sival_ptr;

    sleep(rand() % 7);

    /* ... manipulate with shared_resource */

    return;
}

int main()
{
    struct sigevent sig_ev = {0};
    uint8_t *shared_resource = malloc(123);
    timer_t timer_id;
    int i;

    sig_ev.sigev_notify = SIGEV_THREAD;
    sig_ev.sigev_value.sival_ptr = shared_resource;
    sig_ev.sigev_notify_function = timer_threaded_function;
    sig_ev.sigev_notify_attributes = NULL;

    timer_create(CLOCK_REALTIME, &sig_ev, &timer_id);

    for (i = 0; i < N; i++) {
        /* arm timer for 1 nanosecond */
        timer_settime(timer_id, 0,
                      &(struct itimerspec){{0,0},{0,1}}, NULL);

        /* sleep a little bit, so timer will be fired */
        usleep(1);
    }

    /* only disarms timer, but timer callbacks still can be running */
    timer_delete(timer_id);

    /* 
     * TODO: safe wait for all callbacks to end, so shared resource
     * can be freed without races.
     */
    ...

    free(shared_resource);

    return 0;
}

timer_delete() only disarms timer (if it is was armed) and frees assocoated with timer resources. But timer callbacks still can be running. So we cannot free shared_resource, otherwise race condition may occur. Is there any method to cope with this situation?

I thougth about reference counting, but it doesn't help, because we doesn't know how much threads actually will try to access shared resource (cause of timer overruns).

Answer 1

It is thoroughly unsatisfactory :-(. I have looked, and there does not seem to be any way to discover whether the sigevent (a) has not been fired, or (b) is pending, or (c) is running, or (d) has completed.

Best I can suggest is an extra level of indirection, and a static to point at the shared resource. So:

  static foo_t* p_shared ;
   ....
  p_shared = shared_resourse ;
   .....
  sig_ev.sigev_value.sival_ptr = &p_shared ;

where foo_t is the type of the shared resource.

Now we can use some atomics... in timer_threaded_function():

  foo_t** pp_shared ;
  foo_t*  p_locked ;
  foo_t*  p_shared ;

  pp_shared = so.sival_ptr ;
  p_locked  = (void*)UINPTR_MAX ;
  p_shared  = atomic_swap(pp_shared, p_locked) ;

  if (p_shared == p_locked)
    return ;                    // locked already.

  if (p_shared == NULL)
    return ;                    // destroyed already.

  .... proceed to do the usual work ...

  if (atomic_cmp_swap(pp_shared, &p_locked, p_shared))
    return ;                    // was locked and is now restored

  assert(p_locked == NULL) ;

  ... the shared resource needs to be freed ...

And in the controlling thread:

  timer_delete(timer_id) ;              // no more events, thank you

  p_s = atomic_swap(&p_shared, NULL) ;  // stop processing events

  if (p_s == (void*)UINTPTR_MAX)
    // an event is being processed.

  if (p_s != NULL)
    ... the shared resource needs to be freed ...

When the event thread finds that the shared resourse needs to be freed, it can do it itself, or signal to the controlling thread that the event has been processed, so that the controlling thread can go ahead and do the free. That's largely a matter of taste.

Basically, this is using atomics to provide a sort of a lock, whose value is tri-state: NULL <=> destroyed ; UINTPTR_MAX <=> locked ; anything else <=> unlocked.

The down-side is the static p_shared, which has to remain in existence until the timer_threaded_function() has finished and will never be called again... and since those are precisely the things that are unknowable, static p_shared is, effectively, a fixture :-(.