Using pthread condition variable with rwlock

Question

I'm looking for a way to use pthread rwlock structure with conditions routines in C++.

I have two questions:

First: How is it possible and if we can't, why ?

Second: Why current POSIX pthread have not implemented this behaviour ?

To understand my purpose, I explain what will be my use: I've a producer-consumer model dealing with one shared array. The consumer will cond_wait when the array is empty, but rdlock when reading some elems. The producer will wrlock when adding(+signal) or removing elems from the array.

The benefit of using rdlock instead of mutex_lock is to improve performance: when using mutex_lock, several readers would block, whereas using rdlock several readers would not block.

Answer 1

C++0x is getting multithreading support, and that support includes a new type called condition_variable_any:

class condition_variable_any
{
public:
    condition_variable_any();
    ~condition_variable_any();

    condition_variable_any(const condition_variable_any&) = delete;
    condition_variable_any& operator=(const condition_variable_any&) = delete;

    void notify_one();
    void notify_all();

    template <class Lock>
        void wait(Lock& lock);
    template <class Lock, class Predicate>
        void wait(Lock& lock, Predicate pred);

    template <class Lock, class Clock, class Duration>
        cv_status
        wait_until(Lock& lock,
                   const chrono::time_point<Clock, Duration>& abs_time);

    template <class Lock, class Clock, class Duration, class Predicate>
        bool
        wait_until(Lock& lock,
                   const chrono::time_point<Clock, Duration>& abs_time,
                   Predicate pred);

    template <class Lock, class Rep, class Period>
        cv_status
        wait_for(Lock& lock,
                 const chrono::duration<Rep, Period>& rel_time);

    template <class Lock, class Rep, class Period, class Predicate>
        bool
        wait_for(Lock& lock,
                 const chrono::duration<Rep, Period>& rel_time,
                 Predicate pred);
};

There is an explanation of how to implement condition_variable_any here:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2406.html#gen_cond_var

but at this link it is named gen_cond_var. The magic thing about condition_variable_any is that it will wait on anything that has lock() and unlock() members. Once you have condition_variable_any, then all you need is a rwlock. The link above also introduces shared_mutex and shared_lock and shows example code doing what you want:

std::tr2::shared_mutex mut;
std::gen_cond_var cv;

void wait_in_shared_ownership_mode()
{
    std::tr2::shared_lock<std::tr2::shared_mutex> shared_lk(mut);
    // mut is now shared-locked
    // ...
    while (not_ready_to_proceed())
        cv.wait(shared_lk);  // shared-lock released while waiting
    // mut is now shared-locked
    // ...
}   // mut is now unlocked

void wait_in_unique_ownership_mode()
{
    std::unique_lock<std::tr2::shared_mutex> lk(mut);
    // mut is now unique-locked
    // ...
    while (not_ready_to_proceed())
        cv.wait(lk);  // unique-lock released while waiting
    // mut is now unique-locked
    // ...
}   // mut is now unlocked

The above document is somewhat dated. There is a more up-to-date implementation and description of shared_mutex / shared_lock here:

http://howardhinnant.github.io/shared_mutex http://howardhinnant.github.io/shared_mutex.cpp

All of this is implemented on top of POSIX pthreads. I hope to get the shared-locking stuff into a C++ technical report (tr2), but of course there is no guarantee of that.

Answer 2

There are several RWLocks implemented on top of mutexes and condvars. Pick any one and add some condvars for your custom needs.

Answer 3

To solve issue signaled by Doomsday one have to check again condition after grab_mutex and before wait_on_condition:

consumer() {

  get_readlock();
  if(array_empty()) {
    release_readlock();
    grab_mutex();
    if(array_empty()) {
       wait_on_condition();
    }
    release_mutex();
    get_readlock();
  }
  process_elements();
  release_readlock();
}  

Answer 4

For what you want, you need to just have 1 set of rwlock and 1 set of mutex/cond variable, pseudo code (though you'll need the usual loops on the posix cond variables)

consumer() {

  get_readlock();
  if(array_empty()) {
    release_readlock();
    grab_mutex();
    wait_on_condition();
    release_mutex();
    get_readlock();
  }
  process_elements();
  release_readlock();
}

producer()
{
  get_writelock();
  get_mutex();
  insert_elements();
  signal_condition();
  release_mutex();
  release_writelock();
}

I'd guess condition variables only works with mutexes, because waiting or signalling a conditions needs mutual exclusiveness.

Answer 5

I have the same requirement as you needed.

Here is my solution:

Wrap pthread_rwlock_t

class rwlock
{
public:
    rwlock()
    {
        pthread_rwlock_init(&_lock, nullptr);
    }

    ~rwlock()
    {
        pthread_rwlock_destroy(&_lock);
    }

    void read_lock()
    {
        pthread_rwlock_rdlock(&_lock);
    }

    void write_lock()
    {
        pthread_rwlock_wrlock(&_lock);
    }

    bool try_read_lock()
    {
        return pthread_rwlock_tryrdlock(&_lock) == 0;
    }

    bool try_write_lock()
    {
        return pthread_rwlock_trywrlock(&_lock) == 0;
    }

    void lock()
    {
        read_lock();
    }

    void try_lock()
    {
        try_read_lock();
    }

    void unlock()
    {
        pthread_rwlock_unlock(&_lock);
    }

private:
    pthread_rwlock_t _lock;
};

Usage

rwlock lock;

std::condition_variable_any cond;

bool ready = false;

Producer

lock.write_lock();
...
if (!ready) {
    ready = true;
    cond.notify_all();
}
lock.unlock();

Consumer

std::unique_lock<rwlock> lock_(lock);
while (!ready) {
    cond.wait(lock_, []{ return ready; });
}
...
ready = false;

Answer 6

I assume that by "conditions", you mean "conditional variables". They're different things.

No, you cannot use a rwlock when waiting on a conditional variable. I cannot answer as to the "why", but that's the way POSIX has decided to do it. Perhaps just to keep things simple.

You can still get the behavior you want, however, by making your own rwlock class by using only a mutex and 2 conditional variables without using a POSIX rwlock:

getReadLock():
     lock(mutex)
     while(array.empty())
         wait(readersCondVar, mutex)
     readers++;
     unlock(mutex)       

releaseReadLock():
     lock(mutex)
     if (--readers == 0)
           broadcast(writerCondVar, mutex) // or signal, if only 1 producer
     unlock(mutex)

readerThread:
     forever() {
         getReadLock()
         read()
         releaseReadLock()
      }

getWriteLock():
     lock(mutex)
     while(readers) {
         wait(writerCondVar, mutex)
     }

releaseWriteLock():
     broadcast(readersCondVar, mutex)
     unlock(mutex)

writerThread():
      forever() {
         getWriteLock()
         write()
         releaseWriteLock()
      }

Simple and does what you want.