Multiprocessing in following python code does not

Question

I'm trying to increase the execution time of my python program by using some multiprocessing. Suppose I have this sample code:

def foo(q,x,y):
     ....
     q.put(result)

def parallel_funtion(x):

    q1 = Queue(); q2 = Queue()

    p1 = Process(target=foo,
                 args=[q1,x,0])

    p2 = Process(target=foo,
                 args=[q2,x,1])

    p1.start(); p2.start()
    p1.join(); p2.join()

    z = max(q1.get(), q2.get())

    return z

def function(list)
    .....

    for i in list:
        parallel_function(i)


main():
    function(aList)

After the first iteration in the cycle in "function" the program freezes specifically in this row:

z = max(q1.get(), q2.get())

Why?

Answer 1

The question is short on specifics, but this works for me... I modified your use of list, since that appears to destroy python's list method (although as you say, the code still executes):

from multiprocessing import Process, Queue
import time

def foo1(queue, procid, arg1, arg2):
    # Measure execution time and return the total time in the queue
    print "%s got arg1=%s, arg2=%s " % (procid, arg1, arg2)
    start = time.time()
    ii = arg1
    while (ii > 0):
        ii = ii - 1
        time.sleep(0.01)
    # return the output of the call through the Queue
    queue.put((time.time() - start)*arg2)

def parallel_function(x):
    q1 = Queue()
    q2 = Queue()

    p1 = Process(target=foo1, args=[q1, 'Proc1', x, 1])
    p2 = Process(target=foo1, args=[q2, 'Proc2', x, 2])

    p1.start(); p2.start()
    p1.join(); p2.join()
    # Get return values from each Queue
    z = max(q1.get(), q2.get())
    return z

def function(_list):
    for ii in _list:
        print "FUNCTION RESULT input=%s, result=%s" % (ii, 
            parallel_function(ii))



function([100,120,130,140,150])

Output:

Proc1 got arg1=100, arg2=1 
Proc2 got arg1=100, arg2=2 
FUNCTION RESULT input=100, result=2.01133012772
Proc1 got arg1=120, arg2=1 
Proc2 got arg1=120, arg2=2 
FUNCTION RESULT input=120, result=2.4130563736
Proc1 got arg1=130, arg2=1 
Proc2 got arg1=130, arg2=2 
FUNCTION RESULT input=130, result=2.61448001862
Proc1 got arg1=140, arg2=1 
Proc2 got arg1=140, arg2=2 
FUNCTION RESULT input=140, result=2.81632232666
Proc1 got arg1=150, arg2=1 
Proc2 got arg1=150, arg2=2 
FUNCTION RESULT input=150, result=3.01693964005