I am using multiprocessing.Process to load some images and store them in a shared memory as explained here. The problem is, sometimes my code crashes due to a huge memory spike at completely random times. I just had an idea of what might be causing this: the process does not have had enough time to copy the contents of the image into the shared memory in RAM by the time join(). To test my hypothesis I added time.sleep(0.015) after doing join() on each of my processes and this has already reduced the number of memory spikes by about 90% or more. However, I'm still not 100% certain whether not getting memory spikes as often is because that little amount of time could help the data to get fully transferred to the shared memory or not.
So I wonder, is there a way to make sure a child process has finished copying data to memory before .join() is called? I do not want to use a fixed number when calling time.sleep(). It would be great to to know when the data has been fully transferred to the shared memory and then do join().
