Python Conditional “With” Lock Design

Question

I am trying to do some shared locking using with statements

def someMethod(self, hasLock = False):
     with self.my_lock:
         self.somethingElse(hasLock=True)


def somethingElse(self, hasLock = False):
    #I want this to be conditional...
    with self.my_lock:
          print 'i hate hello worlds"

That make sense? I basically only want to do the with if I don't already have the lock.

On top of being able to accomplish this, is it a bad design? Should I just acquire/release myself?

Answer 1

Just use a threading.RLock which is re-entrant meaning it can be acquired multiple times by the same thread.

http://docs.python.org/library/threading.html#rlock-objects

For clarity, the RLock is used in the with statements, just like in your sample code:

lock = threading.RLock()

def func1():
    with lock:
        func2()

def func2():
    with lock: # this does not block even though the lock is acquired already
        print 'hello world'

As far as whether or not this is bad design, we'd need more context. Why both of the functions need to acquire the lock? When is func2 called by something other than func1?

Answer 2

The Python or is short circuiting so you can make the locking conditional:

def somethingElse(self, hasLock = False):
    #I want this to be conditional...
    with hasLock or self.my_lock:
          print 'i hate hello worlds'

Unfortunately it's not quite that easy, because a boolean isn't a valid return from a with statement. You'll need to create a class with the __enter__ and __exit__ to wrap the boolean True value.

Here's one possible implementation that I haven't tested.

from contextlib import contextmanager

@contextmanager
def withTrue():
    yield True

def withbool(condition):
    if condition:
        return withTrue()
    return False

def somethingElse(self, hasLock = False):
    with withbool(hasLock) or self.my_lock():
          print 'i hate hello worlds'

This is a lot of boilerplate for something so simple, so the RLock solution looks like a winner. This solution might be useful in a different context though.

Answer 3

Why not:

def someMethod(self):
     with self.my_lock:
         self.somethingNoLock()

def somethingElse(self):
    with self.my_lock:
         self.somethingNoLock()

def somethingNoLock(self):
    print 'i hate hello worlds"

Note that while someMethod and somethingElse are identical in my solution, in general they would be different. You could put another wrapper around somethingNoLock so that the lock acquisition and release is not repeated multiple times.

This is far simpler and straightforward. Just because the re-entrant lock hammer is available, I wouldn't recommend using it when there is a more straightforward, less fragile way to nail it.

The more specific criticism of rlock is that the line that creates the re-entrant lock is far away from the code that is acquiring the lock in a re-entrant way. This is slightly fragile if someone say coalesces the re-entrant lock with another lock that is not re-entrant or otherwise changes the line that creates the lock.

Answer 4

Using with statement is better than just acquire() and release() functions. This way, if an error occurs, the locks will be released.

Answer 5

The with statement is a great way to implement locking, as locking is a perfect resource acquisition pattern. Though, your current example won't work, you'll need an if statement around the with statement within somethingElse().