Can someone explain why the following code behaves the way it does:
import types
class Dummy():
def __init__(self, name):
self.name = name
def __del__(self):
print "delete",self.name
d1 = Dummy("d1")
del d1
d1 = None
print "after d1"
d2 = Dummy("d2")
def func(self):
print "func called"
d2.func = types.MethodType(func, d2)
d2.func()
del d2
d2 = None
print "after d2"
d3 = Dummy("d3")
def func(self):
print "func called"
d3.func = types.MethodType(func, d3)
d3.func()
d3.func = None
del d3
d3 = None
print "after d3"
The output (note that the destructor for d2 is never called) is this (python 2.7)
delete d1
after d1
func called
after d2
func called
delete d3
after d3
Is there a way to "fix" the code so the destructor is called without deleting the method added? I mean, the best place to put the d2.func = None would be in the destructor!
Thanks
[edit] Based on the first few answers, I'd like to clarify that I'm not asking about the merits (or lack thereof) of using __del__
. I tried to create the shortest function that would demonstrate what I consider to be non-intuitive behavior. I'm assuming a circular reference has been created, but I'm not sure why. If possible, I'd like to know how to avoid the circular reference....
Instead of del, you can use the
with
operator.http://effbot.org/zone/python-with-statement.htm
just like with filetype objects, you could something like
A full example of a context manager.
I'm providing my own answer because, while I appreciate the advice to avoid
__del__
, my question was how to get it to work properly for the code sample provided.Short version: The following code uses
weakref
to avoid the circular reference. I thought I'd tried this before posting the question, but I guess I must have done something wrong.Longer version: When I posted the question, I did search for similar questions. I know you can use
with
instead, and that the prevailing sentiment is that__del__
is BAD.Using
with
makes sense, but only in certain situations. Opening a file, reading it, and closing it is a good example wherewith
is a perfectly good solution. You've gone a specific block of code where the object is needed, and you want to clean up the object and the end of the block.A database connection seems to be used often as an example that doesn't work well using
with
, since you usually need to leave the section of code that creates the connection and have the connection closed in a more event-driven (rather than sequential) timeframe.If
with
is not the right solution, I see two alternatives:__del__
works (see this blog for a better description of weakref usage)atexit
module to run a callback when your program closes. See this topic for example.While I tried to provide simplified code, my real problem is more event-driven, so
with
is not an appropriate solution (with
is fine for the simplified code). I also wanted to avoidatexit
, as my program can be long-running, and I want to be able to perform the cleanup as soon as possible.So, in this specific case, I find it to be the best solution to use
weakref
and prevent circular references that would prevent__del__
from working.This may be an exception to the rule, but there are use-cases where using
weakref
and__del__
is the right implementation, IMHO.It seems to me the real heart of the matter is here:
I sense that what you are really after is a flexible way to bind different functionality to an object representing program state, also known as polymorphism. Python does that quite well, not by attaching/detaching methods, but by instantiating different classes. I suggest you look again at your class organization. Perhaps you need to separate a core, persistent data object from transient state objects. Use the has-a paradigm rather than is-a: each time state changes, you either wrap the core data in a state object, or you assign the new state object to an attribute of the core.
If you're sure you can't use that kind of pythonic OOP, you could still work around your problem another way by defining all your functions in the class to begin with and subsequently binding them to additional instance attributes (unless you're compiling these functions on the fly from user input):
An alternative solution to using
weakref
is to dynamically bind the function to the instance only when it is called by overriding__getattr__
or__getattribute__
on the class to returnfunc.__get__(self, type(self))
instead of justfunc
for functions bound to the instance. This is how functions defined on the class behave. Unfortunately (for some use cases) python doesn't perform the same logic for functions attached to the instance itself, but you can modify it to do this. I've had similar problems with descriptors bound to instances. Performance here probably isn't as good as usingweakref
, but it is an option that will work transparently for any dynamically assigned function with the use of only python builtins.If you find yourself doing this often, you might want a custom metaclass that does dynamic binding of instance-level functions.
Another alternative is to add the function directly to the class, which will then properly perform the binding when it's called. For a lot of use cases, this would have some headaches involved: namely, properly namespacing the functions so they don't collide. The instance id could be used for this, though, since the id in cPython isn't guaranteed unique over the life of the program, you'd need to ponder this a bit to make sure it works for your use case... in particular, you probably need to make sure you delete the class function when an object goes out of scope, and thus its id/memory address is available again.
__del__
is perfect for this :). Alternatively, you could clear out all methods namespaced to the instance on object creation (in__init__
or__new__
).Another alternative (rather than messing with python magic methods) is to explicitly add a method for calling your dynamically bound functions. This has the downside that your users can't call your function using normal python syntax:
Just to make this post complete, I'll show your
weakref
option as well:You cannot assume that
__del__
will ever be called - it is not a place to hope that resources are automagically deallocated. If you want to make sure that a (non-memory) resource is released, you should make arelease()
or similar method and then call that explicitly (or use it in a context manager as pointed out by Thanatos in comments below).At the very least you should read the
__del__
documentation very closely, and then you should probably not try to use__del__
. (Also refer to thegc.garbage
documentation for other bad things about__del__
)