Understanding the difference between __getattr__ a

Question

I am trying to understand the difference between __getattr__ and __getattribute__, however, I am failing at it.

The answer to the Stack Overflow question Difference between __getattr__ vs __getattribute__ says:

__getattribute__ is invoked before looking at the actual attributes on the object, and so can be tricky to implement correctly. You can end up in infinite recursions very easily.

I have absolutely no idea what that means.

Then it goes on to say:

You almost certainly want __getattr__.

Why?

I read that if __getattribute__ fails, __getattr__ is called. So why are there two different methods doing the same thing? If my code implements the new style classes, what should I use?

I am looking for some code examples to clear this question. I have Googled to best of my ability, but the answers that I found don't discuss the problem thoroughly.

If there is any documentation, I am ready to read that.

Answer 1

I think the other answers have done a great job of explaining the difference between __getattr__ and __getattribute__, but one thing that might not be clear is why you would want to use __getattribute__. The cool thing about __getattribute__ is that it essentially allows you to overload the dot when accessing a class. This allows you to customize how attributes are accessed at a low level. For instance, suppose I want to define a class where all methods that only take a self argument are treated as properties:

# prop.py
import inspect

class PropClass(object):
    def __getattribute__(self, attr):
        val = super(PropClass, self).__getattribute__(attr)
        if callable(val):
            argcount = len(inspect.getargspec(val).args)
            # Account for self
            if argcount == 1:
                return val()
            else:
                return val
        else:
            return val

And from the interactive interpreter:

>>> import prop
>>> class A(prop.PropClass):
...     def f(self):
...             return 1
... 
>>> a = A()
>>> a.f
1

Of course this is a silly example and you probably wouldn't ever want to do this, but it shows you the power you can get from overriding __getattribute__.

Answer 2

Some basics first.

With objects, you need to deal with its attributes. Ordinarily we do instance.attribute. Sometimes we need more control (when we do not know the name of the attribute in advance).

For example, instance.attribute would become getattr(instance, attribute_name). Using this model, we can get the attribute by supplying the attribute_name as a string.

Use of __getattr__

You can also tell a class how to deal with attributes which it doesn't explicitly manage and do that via __getattr__ method.

Python will call this method whenever you request an attribute that hasn't already been defined, so you can define what to do with it.

A classic use case:

class A(dict):
    def __getattr__(self, name):
       return self[name]
a = A()
# Now a.somekey will give a['somekey']

Caveats and use of __getattribute__

If you need to catch every attribute regardless whether it exists or not, use __getattribute__ instead. The difference is that __getattr__ only gets called for attributes that don't actually exist. If you set an attribute directly, referencing that attribute will retrieve it without calling __getattr__.

__getattribute__ is called all the times.

Answer 3

__getattribute__ is called whenever an attribute access occurs.

class Foo(object):
    def __init__(self, a):
        self.a = 1

    def __getattribute__(self, attr):
        try:
            return self.__dict__[attr]
        except KeyError:
            return 'default'
f = Foo(1)
f.a

This will cause infinite recursion. The culprit here is the line return self.__dict__[attr]. Let's pretend (It's close enough to the truth) that all attributes are stored in self.__dict__ and available by their name. The line

f.a

attempts to access the a attribute of f. This calls f.__getattribute__('a'). __getattribute__ then tries to load self.__dict__. __dict__ is an attribute of self == f and so python calls f.__getattribute__('__dict__') which again tries to access the attribute '__dict__'. This is infinite recursion.

If __getattr__ had been used instead then

1. It never would have run because f has an a attribute.
2. If it had run, (let's say that you asked for f.b) then it would not have been called to find __dict__ because it's already there and __getattr__ is invoked only if all other methods of finding the attribute have failed.

The 'correct' way to write the above class using __getattribute__ is

class Foo(object):
    # Same __init__

    def __getattribute__(self, attr):
        return super(Foo, self).__getattribute__(attr)

super(Foo, self).__getattribute__(attr) binds the __getattribute__ method of the 'nearest' superclass (formally, the next class in the class's Method Resolution Order, or MRO) to the current object self and then calls it and lets that do the work.

All of this trouble is avoided by using __getattr__ which lets Python do it's normal thing until an attribute isn't found. At that point, Python hands control over to your __getattr__ method and lets it come up with something.

It's also worth noting that you can run into infinite recursion with __getattr__.

class Foo(object):
    def __getattr__(self, attr):
        return self.attr

I'll leave that one as an exercise.