Can `weakref` callbacks replace `__del__`?

Question

Is there any obstacle that prevents weakref from doing everything that __del__ does but with much stronger guarantees (e.g., finalize guarantees that the call will be made before the interpreter exits, and the order of calls is well-defined, etc.)?

It seems that in the distant past it was thought that weakref would eventually lead to the removal of __del__ from the language.

What prevented this from happening?

There seems to be few use cases for __del__, and all the ones I'm aware of seem to work at least as well (and usually much better) with weakref callbacks or weakref.finalize.

Update:

With PEP 442 dramatically improving the behavior of __del__, and the concerns with weakref mentioned by @gz and @user2357112, I'm wondering if the language is generally moving towards making __del__ more reliable, or towards using weakref instead of __del__, or both.

Answer 1

There's a somewhat pragmatic reason __del__ is still around. Several signficant weakref improvements, including finalize, were new in Python 3.4. So, replacing __del__ with better weakrefs missed the window for language breaking changes with py3k.

I think most uses can be replaced by the base weakref functionality, but I'm struck by this observation from Richard Oudkerk in issue 15528 where proposed and implemented finalize:

[Weakref callbacks] are low level, and working out how to use them correctly requires a bit of head scratching. One must find somewhere to store the weakref till after the referent is dead, and without accidentally keeping the referent alive. Then one must ensure that the callback frees the weakref (without leaving any remnant ref-cycles).

When it is an option, using a __del__ method is far less hassle.

Anyway, perhaps the question should be brought up again when Python 4 is being considered? ;)

Answer 2

Answer to the question is really depends on the use case and would also suggest you consider that different Python interpreter implementations do not have the same GC behaviour as CPython.

PyPy in particular does not call __del__ as-soon-as an object is del-eleted or goes out of scope but 'some time later'. So code that relies on CPython's __del__ behaviour will break on PyPy, and other alternative interpreters.

What I would recommend is to make use of __enter__ and __exit__ along with the with keyword. For example

from __future__ import print_function

class MyClass(object):

    def __enter__(self):
        print("Entering")

    def __exit__(self, exc_type, exc_val, exc_tb):
        print("Exiting")

# 'with MyClass()' can be used but want to show that
# code in exit is executed regardless of object
# life time
obj = MyClass()
with obj:
    print("Stuff happening in between")

Result is:

Entering
Stuff happening in between
Exiting

The above order of statements is guaranteed and does not depend on GC behaviour.

Things that should be done as soon as the code in the with block completes can go in __exit__, such as things that would normally be put in a destructor - clearing file handles, releasing locks and so on.

A subsequent del on the object, or it going out of scope, will clear the object reference eventually, again depending on interpreter implementation, but things that should be done immediately best not rely on that behaviour.