Parallelism in python isn't working right

Question

I was developing an app on gae using python 2.7, an ajax call requests some data from an API, a single request could take ~200 ms, however when I open two browsers and make two requests at a very close time they take more than the double of that, I've tried putting everything in threads but it didn't work.. (this happens when the app is online, not just on the dev-server)

So I wrote this simple test to see if this is a problem in python in general (in case of a busy wait), here is the code and the result:

def work():
    t = datetime.now()
    print threading.currentThread(), t
    i = 0
    while i < 100000000:
        i+=1
    t2 = datetime.now()
    print threading.currentThread(), t2, t2-t

if __name__ == '__main__': 
    print "single threaded:"
    t1 = threading.Thread(target=work)
    t1.start()
    t1.join()

    print "multi threaded:"
    t1 = threading.Thread(target=work)
    t1.start()
    t2 = threading.Thread(target=work)
    t2.start()
    t1.join()
    t2.join()

The result on mac os x, core i7 (4 cores, 8 threads), python2.7:

single threaded:
<Thread(Thread-1, started 4315942912)> 2011-12-06 15:38:07.763146
<Thread(Thread-1, started 4315942912)> 2011-12-06 15:38:13.091614 0:00:05.328468

multi threaded:
<Thread(Thread-2, started 4315942912)> 2011-12-06 15:38:13.091952
<Thread(Thread-3, started 4323282944)> 2011-12-06 15:38:13.102250
<Thread(Thread-3, started 4323282944)> 2011-12-06 15:38:29.221050 0:00:16.118800
<Thread(Thread-2, started 4315942912)> 2011-12-06 15:38:29.237512 0:00:16.145560

This is pretty shocking!! if a single thread would take 5 seconds to do this.. I thought starting two threads at the same time will take the same time to finish both tasks, but it takes almost triple the time.. this makes the whole threading idea useless, as it would be faster to do them sequentially!

what am I missing here..

Answer 1

I would look at where the time is going. Suppose, for example, the server can only answer one query every 200ms. Then there's nothing you can do, you'll only get one reply every 200ms because that's all the server can provide you.

Answer 2

David Beazley gave a talk about this issue at PyCon 2010. As others have already stated, for some tasks, using threading especially with multiple cores can lead to slower performance than the same task performed by a single thread. The problem, Beazley found, had to do with multiple cores having a "GIL battle":

To avoid GIL contention, you may get better results having the tasks run in separate processes instead of separate threads. The multiprocessing module provides a convenient way to do that especially since multiprocessing API is very similar to the threading API.

import multiprocessing as mp
import datetime as dt
def work():
    t = dt.datetime.now()
    print mp.current_process().name, t
    i = 0
    while i < 100000000:
        i+=1
    t2 = dt.datetime.now()
    print mp.current_process().name, t2, t2-t

if __name__ == '__main__': 
    print "single process:"
    t1 = mp.Process(target=work)
    t1.start()
    t1.join()

    print "multi process:"
    t1 = mp.Process(target=work)
    t1.start()
    t2 = mp.Process(target=work)
    t2.start()
    t1.join()
    t2.join()

yields

single process:
Process-1 2011-12-06 12:34:20.611526
Process-1 2011-12-06 12:34:28.494831 0:00:07.883305
multi process:
Process-3 2011-12-06 12:34:28.497895
Process-2 2011-12-06 12:34:28.503433
Process-2 2011-12-06 12:34:36.458354 0:00:07.954921
Process-3 2011-12-06 12:34:36.546656 0:00:08.048761

PS. As zeekay pointed out in the comments, The GIL battle is only severe for CPU-bound tasks. It should not be a problem for IO-bound tasks.

Answer 3

the CPython interpreter will not allow more then one thread to run. read about GIL http://wiki.python.org/moin/GlobalInterpreterLock

So certain tasks cannot be done concurrently in an efficient way in the CPython with threads.

If you want to do things parallel in GAE, then start them parallel with separate requests.

Also, you may want to consult to the Python parallel wiki http://wiki.python.org/moin/ParallelProcessing