I want my Python script to be able to run one of its functions as subprocesses. How should I do that?
Here is a mock-up script of my intention:
#!/urs/bin/env python
def print_mynumber(foo):
"""This function is obviously more complicated in my script.
It should be run as a subprocess."""
print(foo)
for foo in [1,2,3]:
print_mynumber(foo) # Each call of this function should span a new process.
# subprocess(print_mynumber(foo))
Thank you for your suggestions. It is a little hard for me to formulate the problem correctly, and thus to make the appropriate web search.
Use the multiprocessing module:
import multiprocessing as mp
def print_mynumber(foo):
"""This function is obviously more complicated in my script.
It should be run as a subprocess."""
print(foo)
if __name__ == '__main__':
for foo in [1,2,3]:
proc = mp.Process(target = print_mynumber, args = (foo, ))
proc.start()
You might not want to be creating one process for each call to print_mynumber, especially if the list foo iterates over is long. A better way in that case would be to use a multiprocessing pool:
import multiprocessing as mp
def print_mynumber(foo):
"""This function is obviously more complicated in my script.
It should be run as a subprocess."""
print(foo)
if __name__ == '__main__':
pool = mp.Pool()
pool.map(print_mynumber, [1,2,3])
The pool, be default, will create N worker processes, where N is the number of cpus (or cores) the machine possesses. pool.map behaves much like the Python builtin map command, except that it farms out tasks to the pool of workers.
