studying parallel programming python

Question

import multiprocessing
from multiprocessing import Pool
from source.RUN import*

def func(r,grid,pos,h):
    return r,grid,pos,h
    p = multiprocessing.Pool()  # Creates a pool with as many workers as you have CPU cores
    results = []


if __name__ == '__main__':
    for i in  pos[-1]<2:
    results.append(Pool.apply_async(LISTE,(r,grid,pos[i,:],h)))
    p.close()
    p.join()

for result in results:
    print('liste', result.get())

I want to create Pool for (LISTE,(r,grid,pos[i,:],h)) process and i is in pos which is variable in different file which is a ndarray[] and I have to call this whole function in another file in between one While Loop. but this code gives error and if I am using if __name__ == '__main__': it will not pass through below the if __name__ == '__main__': module please give me idea how I can make it

Answer 1

I'm still having a somewhat difficult time understanding your question. But I think this is what you're looking for:

You want to be able to call a function that creates a pool given r, grid, pos, h Iterate over pos feed it to the Pool, then return the results. You also want to be able to access that function from different modules. If that's what you're asking, you can do it like this:

async_module.py:

from multiprocessing import Pool

# Not sure where the LISTE function gets defined, but it needs to be in here.

def do_LISTE(*args):
    # args is a tuple containing (r, grid, pos[i, :], h)
    # we use tuple expansion (*args( to send each parameter to LISTE
    return LISTE(*args)

def async_process(r,grid,pos,h):
    return r,grid,pos,h
    p = multiprocessing.Pool()  # Creates a pool with as many workers as you have CPU cores
    results = p.map(do_LISTE, [(r,grid,pos[i,:], h) for i in pos[-1]<2])
    p.close()
    p.join()
    return results

Then in some other module:

from async_module import async_process

def do_async_processing():
    r = "something"
    grid = get_grid()
    pos = get_pos()
    h = 345
    results = async_process(r, grid, pos, h)

if __name__ == "__main__":
    do_async_processing()  # Make sure the entry point is protected by `if __name__ == "__main__":`.