How to iterate over a container in a thread-safe w

Question

I have a container (C++) on which I need to operate in two ways, from different threads: 1) Add and remove elements, and 2) iterate through its members. Clearly, remove element while iteration is happening = disaster. The code looks something like this:

class A
{
public:
   ...
   void AddItem(const T& item, int index) { /*Put item into my_stuff at index*/ }
   void RemoveItem(const T& item) { /*Take item out of m_stuff*/ }
   const list<T>& MyStuff() { return my_stuff; } //*Hate* this, but see class C
private:
   Mutex mutex; //Goes in the *Item methods, but is largely worthless in MyStuff()
   list<T> my_stuff; //Just as well a vector or deque
};
extern A a; //defined in the .cpp file

class B
{
   ...
   void SomeFunction() { ... a.RemoveItem(item); }
};

class C
{
   ...
   void IterateOverStuff()
   {
      const list<T>& my_stuff(a.MyStuff());
      for (list<T>::const_iterator it=my_stuff.begin(); it!=my_stuff.end(); ++it)
      {
          ...
      }
   }
};

Again, B::SomeFunction() and C::IterateOverStuff() are getting called asynchronously. What's a data structure I can use to ensure that during the iteration, my_stuff is 'protected' from add or remove operations?

Answer 1

When you return the list, return it enclosed in a class that locks/unlocks the mutex in its constructor/destructor. Something along the lines of

class LockedIterable {
public:
  LockedIterable(const list<T> &l, Mutex &mutex) : list_(l), mutex_(mutex)
  {lock(mutex);}
  LockedIterable(const LockedIterable &other) : list_(other.list_), mutex_(other.mutex_) {
    // may be tricky - may be wrap mutex_/list_ in a separate structure and manage it via shared_ptr?
  }
  ~LockedIterable(){unlock(mutex);}
  list<T>::iterator begin(){return list_.begin();}
  list<T>::iterator end(){return list_.end();}
private:
  list<T> &list_;
  Mutex &mutex_;
};

class A {
  ...
  LockedIterable MyStuff() { return LockedIterable(my_stuff, mutex); }  
};

The tricky part is writing copy constructor so that your mutex does not have to be recursive. Or just use auto_ptr.

Oh, and reader/writer lock is indeed a better solution than mutex here.

Answer 2

IMHO it is a mistake to have a private mutex in a data structure class and then write the class methods so that the whole thing is thread safe no matter what the code that calls the methods does. The complexity that is required to do this completely and perfectly is way over the top.

The simpler way is to have a public ( or global ) mutex which the calling code is responsible for locking when it needs to access the data.

Here is my blog article on this subject.

Answer 3

sounds like a reader/writer lock is needed. Basically, the idea is that you may have 1 or more readers OR a single writer. Never can you have a read and write lock at the same time.

EDIT: An example of usage which I think fits your design involves making a small change. Add an "iterate" function to the class which owns the list and make it templated so you can pass a function/functor to define what to do for each node. Something like this (quick and dirty pseudo code, but you get the point...):

class A {
public:
    ...
    void AddItem(const T& item, int index) {
        rwlock.lock_write();
        // add the item
        rwlock.unlock_write();
    }

    void RemoveItem(const T& item) {
        rwlock.lock_write();
        // remove the item
        rwlock.unlock_write();
    }

    template <class P>
    void iterate_list(P pred) {
        rwlock.lock_read();
        std::for_each(my_stuff.begin(), my_stuff.end(), pred);
        rwlock.unlock_read();
    }

private:
    rwlock_t rwlock;
    list<T> my_stuff; //Just as well a vector or deque
};


extern A a; //defined in the .cpp file

class B {
    ...
    void SomeFunction() { ... a.RemoveItem(item); }
};

class C {
    ...

    void read_node(const T &element) { ... }

    void IterateOverStuff() {
        a.iterate_list(boost::bind(&C::read_node, this));
   }
};

Another Option would be to make the reader/writer lock publicly accessible and have the caller responsible for correctly using the lock. But that's more error prone.