Python - why does time.sleep cause memory leak?

问题:

When I ran the code below, the memory was increasing. However if I deleted time.sleep(3), it was 0.1 in top and never increased.

It seems process not be terminated correctly, but why?

Code(Python 2.7.11):

import time
import multiprocessing

def process():
    #: FIXME
    time.sleep(3)
    return

def main():
    pool = multiprocessing.Pool(processes=10)
    while 1:
        pool.apply_async(process)
    pool.close()
    pool.join()

if __name__ == '__main__':
    main()

回答1:

As far as I know, I think that as you spawn new processes in the same Pool, the garbage collection is never done, so you don't release memory from old processes, even if you are done using them. One fix would be to enforce the garbage collection in your while loop:

import time
import multiprocessing
import gc

def process():
    time.sleep(3)
    return

def main():
    pool = multiprocessing.Pool(processes=10)
    while 1:
        pool.apply_async(process)
        gc.collect()
    pool.close()
    pool.join()


if __name__ == '__main__':
    main()

This fixed the memory leaks for me, as you force the garbage collection before launching another set of processes. I hope someone could explain in a more detailed way the reason behing this memory leak.