Python copy.deepcopy() fails without raising warni

Question

This question is related to another question I posted yesterday, although it is much more general in nature.

Because of the thread I mentionned, I have been trying to determine what objects can be copied, pickled, marshaled and what objects cannot.

While doing that, I stumbled on this puzzle:

new_obj = copy.deepcopy(my_obj)
function_that_uses_my_new_obj(new_obj)

throws:

    function_that_uses_my_new_obj(new_obj)

RuntimeError: Internal C++ object (Pyside.QtGui.QWidget) already deleted

Now, since my_obj is a C++ object, that error I can understand. And the reason for that particular problem is the main topic of the other thread.

However, when I try:

function_that_uses_my_new_obj(copy.deepcopy(my_obj))

I don't get anything at all. The program runs normally to this line, stops there for a few seconds and the execution is stopped, the code after that line is not run, no exception/error/warning is thrown and the Python prompt is ready to accept any new command.

EDIT

For some reason, using the copy() method instead of deepcopy() like so:

function_that_uses_my_new_obj(copy.copy(my_obj))

results in the same exception being thrown. So there has to be some point at which deepcopy decides to stop or is being stopped and that triggers the end of the execution. What I don't get is why nothing is raised to inform the user...

Answer 1

Copy and pickle will try to do their job in all cases, but as for many things in python, the responsibility of the result is placed on the programmer only.

In order of rising difficulty:

1. The objects that really can be safely copied or pickled are basically the primitive types, strings, lists, dictionaries, numbers, etc.

2. Then there are many objects that explicitly support the magic methods (like __copy__ and __deepcopy__)

3. Then there are the objects on which copy will do its best and succeed. For simple objects, that contain only references to objects in the previous levels.

4. Finally the really unsafe to copy:

   • Large object graphs with many circular references
   • Object that are being used and modified by other threads.
   • Object with external resources: files, connections, etc.
   • Object that wrap or proxy C libraries

Answer 2

Your assertion that "my_obj is a C++ object" looks pretty obviously false: my_obj is actually a python wrapper around a C++ object. So code such as:

    widget = QtGui.QWidget()
    my_obj = copy.deepcopy(widget)

will only create a copy of the python wrapper, leaving the underlying C++ object untouched. This explains why attempting to call one of the copied wrapped methods will produce that RuntimeError. The copy of the wrapper never had a corresponding underlying C++ object, and so it behaves as if it has been deleted.

This kind of thing can happen quite easily in "normal" PySide/PyQt code. Sometimes, if you don't take care to keep a reference to an object on the python side, Qt can delete the C++ part, leaving you with an "empty" wrapper. And in such situations, you will see the exact same RuntimeError.