Using the python multiprocessing module for IO wit

问题:

I use pygame for running experiments in cognitive science, and often I have heavy I/O demands so I like to fork off these tasks to separate processes (when using a multi-core machine) to improve performance of my code. However, I encountered a scenario where some code works on my colleague's linux machine (Ubuntu LTS), but not on my mac. Below is code representing a minimal reproducible example. My mac is a 2011 Macbook Air running 10.7.2 and using the default python 2.7.1. I tried both pygame as installed via pre-built binary, and I also then tried after installing both SDL and pygame from source.

import pygame
import multiprocessing
pygame.init()

def f():
    while True:
        pygame.event.pump() #if this is replaced by pass, this code works

p = multiprocessing.Process(target=f)
p.start()

while True:
    pass

As noted in the code, it seems that the culprit is putting pygame.event.pump() in a separate process. When I run this on my mac, I first get the following printed repeatedly in terminal:

The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.

Then I get a crash report as copied to this gist.

Any suggestions for how to fix this?

回答1:

Maybe you should initialize the pygame (which initialize SDL-> OpenGL) in each forked (child) process like in sample:

import multiprocessing

def f():
  import pygame
  pygame.init()

  while True:
    pygame.event.pump()

if __module__ == "__main__"
  p = multiprocessing.Process(target=f)
  p.start()

  import pygame
  pygame.init()

  while True:
    pygame.event.pump()

回答2:

Try this link:

http://www.slideshare.net/dabeaz/an-introduction-to-python-concurrency#btnPrevious

It may help. The problem is that you are creating a process that never stops. This should be declared as a daemon:

p = multiprocessing.Process(target=f)
p.daemon = True
p.start()

Not sure if this will solve the problem, I'm just learning about the multiprocessing module as I'm posting this.

回答3:

Have you tried using threads instead of processes? I've had issues before using the python multiprocessing module in OS X. http://docs.python.org/library/threading.html