Cannot change class variables with multiprocessing

2019-02-26 17:06发布

问题:

If I write a class with a class variable, generate two class objects, and change the value of the class variable by using a method of one of the two objects, the class variable value is of course also changed for the other object. Here's what I mean in code:

class DemoClass:
    ClassVariable = False

    def __init__(self):
        pass

    def do(self):
        print(DemoClass.ClassVariable)
        DemoClass.ClassVariable = True


class1 = DemoClass()
class1.do()  # False
class2 = DemoClass()
class2.do()  # True

However, if I do the same with multiprocessing.Process, it does not work. The class variable value will only change for the object that changed it:

import multiprocessing

class DemoProcess(multiprocessing.Process):
    ClassVariable = False

    def __init__(self):
        multiprocessing.Process.__init__(self)

    def run(self):
        print(DemoProcess.ClassVariable)
        DemoProcess.ClassVariable = True
        print(DemoProcess.ClassVariable)


if __name__ == '__main__':
    process_list = []
    p1 = DemoProcess()
    process_list.append(p1)
    p1.start()  # False True
    p2 = DemoProcess()
    process_list.append(p2)
    p2.start()  # False True; should be: True True

    for p in process_list:
        p.join()

The code behaves as if each process generates a new class variable. Am I doing something wrong?

回答1:

With the help of the commenters of my original question I came to the conclusion that I had not yet understood how processes work.

Every DemoProcess.start() creates a new Process which can not share its class variables with other processes.

I solved the issue by using a multprocessing.Value object like Mike McKerns proposed in the comments. The value of this object can be shared with multiple processes.

import multiprocessing

class DemoProcess(multiprocessing.Process):

    def __init__(self, class_variable):
        multiprocessing.Process.__init__(self)
        self.class_variable = class_variable

    def run(self):
        print(self.class_variable.value)
        with self.class_variable.get_lock():
            self.class_variable.value = True
        print(self.class_variable.value)


if __name__ == '__main__':
    ClassVariable = multiprocessing.Value('b', False)

    process_list = []
    p1 = DemoProcess(ClassVariable)
    process_list.append(p1)
    p1.start()  # Output: 0 1
    p2 = DemoProcess(ClassVariable)
    process_list.append(p2)
    p2.start()  # Output: 1 1

    for p in process_list:
        p.join()