With-Statement and Threading :Making function exec

Question

This question is a follow up from following question:With statement and python threading

I have been experimenting with python threading api. I have this code which works for what I want to achieve :---->function execution before invoking run on python thread.

However to do this, I invariably have to call time.sleep(1) in the run() method to make it proceed to execute().Otherwise the thread exits without function assignment and execution.Is there a better way to achieve this type of waiting?

from __future__ import print_function
import threading
import time
import functools
import contextlib
import thread
from threading import Lock
#import contextlib
#Thread module for dealing with lower level thread operations.Thread is limited use Threading instead.

def timeit(fn):
    '''Timeit function like this doesnot work with the thread calls'''
    def wrapper(*args,**kwargs):
        start = time.time()
        fn(*args,**kwargs)
        end = time.time()
        threadID = ""
        print ("Duration for func %s :%d\n"%(fn.__name__ +"_"+ threading.current_thread().name ,end-start))
    return wrapper

exitFlag = 0

@timeit
def print_time(counter,delay):
    while counter:
        if exitFlag:
            thread.exit()
        time.sleep(delay)
        print("%s : %s_%d"%(threading.current_thread().name,time.ctime(time.time()),counter))
        counter -= 1

class Mythread(threading.Thread):
    def __init__(self,threadID,name):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self._f = None

    def run(self):
        print("Starting%s\n" % self.name)
        time.sleep(1)
        if self._f:
            self._f()
            print("Exiting%s\n" % self.name)
        else:
            print("Exiting%s without function execution\n" % self.name ) 

#     def set_f(self,f):
#         self._f = f

    def execute(self,f,*args,**kwargs):
        self._f=functools.partial(f,*args,**kwargs)

    def __enter__(self):
        self.start()

    def __exit__(self,type,value,traceback):
        self.join()




class ThreadContainer(object):
    def __init__(self,id,name):
        self._t = Mythread(id,name)

    def execute(self,f,*args,**kwargs):
        self._f=functools.partial(f,*args,**kwargs)
        self._t.set_f(self._f)
#        self._t.start()
#         self._t.join()


    def __enter__(self):
        self._t.start()

    def __exit__(self,type,value,traceback):
        self._t.join()




if __name__ == '__main__':
    '''
    print_time(5, 1)
     threadLock = threading.Lock()
     threads = []
     thread1 = Mythread(1,"Thread1",5,1)
     thread2 = Mythread(2,"Thread2",5,2)
     thread1.start()
     thread2.start()
     threads.append(thread1)
     threads.append(thread2)
     for t in threads:
         t.join()
    '''
#     thread1 = Mythread(1,"Thread1")
#     thread2 = Mythread(2,"Thread2")
#     with contextlib.nested(ThreadContainer(1,"Thread1"),ThreadContainer(2,"Thread2")) as (t1,t2):
#         t1.execute(print_time,5,1)
#         t2.execute(print_time,5,2)
    t1 = Mythread(1,"Thread1")
    t2 = Mythread(2,"Thread2")
    with contextlib.nested(t1,t2):
        t1.execute(print_time,5,1)
        t2.execute(print_time,5,2)


    print("Exiting main thread ")

Answer 1

The problem here is that you want the run function to wait until the execute function is called.

Of course the obvious solution is to call execute before you call start:

t1.execute(print_time,5,1)
t2.execute(print_time,5,2)
with contextlib.nested(t1, t2):
    pass

… or just make execute call start, or pass the function in to the constructor or the start call, or…

Also, your intended design is a bit weird. The thread function is designed to handle the case where _f hasn't been set… but you want it to wait until _f has been set?

---

But it's conceivable that this kind of problem could come up in a more realistic design, so, let's look at how to solve it.

First, adding sleep to solve a threading problem is almost always a sign that you're doing something very wrong. It's also a great way to cause either horrible performance problems (as in: by the time you add enough sleeps in enough places to make everything mostly work, it takes 30 seconds for your app to start instead of 30 milliseconds)—and, worse, race-condition bugs (surely 1 second is always enough time, right? unless the computer is thrashing swap, or waking up from hibernate, or busy with other programs using all the CPU, or…).

If you're trying to synchronize actions across threads, you need to use a synchronization object. The trick is knowing the right one. Read the docs for Lock through Event (and 3.x adds Barrier), and find a tutorial on threading in general to get a broader idea of what all of these things are for.*

In this case, you've got code that's waiting for some change to saved state, and other code that's making the change, which is the prototypical use case for a 'Condition'. So:

class Mythread(threading.Thread):
    def __init__(self, threadID, name, condition):
        self.condition = condition
        # ... same as before

    def run(self):
        # ... setup before checking for _f

        with self.condition:
            while not self._f:
                self.condition.wait()
        self._f()

        # ... anything else you want

Now, you need to create the Condition, pass it to the threads, and notify it.

You could use a single Condition:

condition = threading.Condition()
t1 = Mythread(1, "Thread1", condition)
t2 = Mythread(2, "Thread2", condition)
with contextlib.nested(t1,t2):
    with condition:
        t1.execute(print_time, 5, 1)
        t2.execute(print_time, 5, 2)
        condition.notify_all()

---

Alternatively, you can give each thread its own Condition:

class Mythread(threading.Thread):
    def __init__(self, threadID, name):
        self.condition = Condition()
        # ... same as before

# ...

t1 = Mythread(1, "Thread1")
t2 = Mythread(2, "Thread2")
with contextlib.nested(t1,t2):
    with t1.condition:
        t1.execute(print_time, 5, 1)
        t1.condition.notify()
    with t2.condition:
        t2.execute(print_time, 5, 1)
        t2.condition.notify()

---

Note that this doesn't allow you to explicit "not set" _f, but it's pretty easy to do that. For example, you can add an _f_set attribute, and check that instead of _f, so someone can call execute(None) (and then notify) to wake you up and get you to the "no _f" case.

---

* Warning: Some of the naming is inconsistent. There's a different thing also called "barrier", and a different different thing also called "fence", and there are many variants of "event" that are pretty different from Pythons (some of which are more like a condition, but aren't actually usable as such), and sometimes a "condition variable" is the actual shared state protected by the sync object rather than the sync object, and so on…