Parallelize for loops in python

2019-07-23 17:19发布

Is there any possibility to parallelize the following code in python? I was wondering how to convert this code with map and lambda functions..

values = (1,2,3,4,5 )

def op(x,y):
    return x+y

[(i, j, op(i, j))
        for i in values
        for j in values
        if i is not j]

2条回答
Lonely孤独者°
2楼-- · 2019-07-23 17:58

You can parallelize the function op with multiprocessing and map:

from multiprocessing.dummy import Pool as ThreadPool
from itertools import permutations

pool = ThreadPool(4)  # Number of threads

values = (1,2,3,4,5)
aux_val = [(i, j) for i,j in permutations(values,2)]

def op(tupx):
    result = (tupx[0], tupx[1], tupx[0] + tupx[1])
    return result

results = pool.map(op, aux_val)
查看更多
Lonely孤独者°
3楼-- · 2019-07-23 18:21

Check this out:

from itertools import permutations

values = (1,2,3,4,5 )
[(i, j, i+j) for i, j in permutations(values, 2)]

It's in python's stdlib.

If you want run in parallel, check out this using python3:

import multiprocessing
from itertools import permutations

values = [1, 2, 3, 4, 5]
l = permutations(values, 2)


def f(x):
    return x[0], x[1], x[0] + x[1]

with multiprocessing.Pool(5) as p:
    data = p.map(f, l)
查看更多
登录 后发表回答