Metaclasses and when/how functions are called

Question

I'm trying to learn how metaclasses work in python 3. Things I want to know are: which functions are called, in what order, and their signatures and returns.

As an example, I know __prepare__ gets called when a class with a metaclass is instantiated with arguments metaclass, name_of_subclass, bases and returns a dictionary representing the future namespace of the instantiated object.

I feel like I understand __prepare__'s step in the process well. What I don't, though, are __init__, __new__, and __call__. What are their arguments? What do they return? How do they all call each other, or in general how does the process go? Currently, I'm stuck on understanding when __init__ is called.

Here is some code I've been messing around with to answer my questions:

#!/usr/bin/env python3

class Logged(type):

    @classmethod
    def __prepare__(cls, name, bases):
        print('In meta __prepare__')
        return {}

    def __call__(subclass):
        print('In meta __call__')
        print('Creating {}.'.format(subclass))
        return subclass.__new__(subclass)

    def __new__(subclass, name, superclasses, attributes, **keyword_arguments):
        print('In meta __new__')
        return type.__new__(subclass, name, superclasses, attributes)

    def __init__(subclass, name, superclasses, attributes, **keyword_arguments):
        print('In meta __init__')

class Thing(metaclass = Logged):

    def __new__(this, *arguments, **keyword_arguments):
        print('In sub __new__')
        return super(Thing, this).__new__(this)

    def __init__(self, *arguments, **keyword_arguments):
        print('In sub __init__')

    def hello(self):
        print('hello')

def main():
    thing = Thing()
    thing.hello()

if __name__ == '__main__':
    main()

From this and some googling, I know that __new__ is really a static method that returns an instance of some object (usually the object where __new__ is defined, but not always), and that __init__ is called of an instance when it is made. By that logic, I'm confused as to why Thing.__init__() isn't being called. Could someone illuminate?

The output of this code prints 'hello', so an instance of Thing is being created, which further confuses me about init. Here's the output:

In meta __prepare__
In meta __new__
In meta __init__
In meta __call__
Creating <class '__main__.Thing'>
In sub __new__
hello

Any help understanding metaclasses would be appreciated. I've read quite a few tutorials, but I've missed some of these details.

Answer 1

In the metaclass's __call__ method, you're calling Thing's __new__ only, but not __init__. It seems that the default behaviour of __call__ is to invoke both of them, as seen when we call the metaclass's inherited __call__:

    def __call__(subclass):
        print('In meta __call__')
        print('Creating {}.'.format(subclass))
        return super().__call__(subclass)

This prints:

Creating <class '__main__.Thing'>.
In sub __new__
In sub __init__

Answer 2

First of all: __prepare__ is optional, you don't need to supply an implementation if all you are doing is return a default {} empty dictionary.

Metaclasses work exactly like classes, in that when you call them, then they produce an object. Both classes and metaclasses are factories. The difference is that a metaclass produces a class object when called, a class produces an instance when called.

Both classes and metaclasses define a default __call__ implementation, which basically does:

1. Call self.__new__ to produce a new object.
2. if that new object is an instance of self / a class with this metaclass, then also call __init__ on that object.

You produced your own __call__ implementation, which doesn't implement that second step, which is why Thing.__init__ is never called.

You may ask: but the __call__ method is defined on the metaclass. That's correct, so it is exactly that method that is called when you call the class with Thing(). All special methods (starting and ending with __) are called on the type (e.g. type(instance) is the class, and type(class) is the metaclass) precisely because Python has this multi-level hierarchy of instances from classes from metaclasses; a __call__ method on the class itself is used to make instances callable. For metaclass() calls, it is the type object itself that provides the __call__ implementation. That's right, metaclasses are both subclasses and instances of type, at the same time.

When writing a metaclass, you should only implement __call__ if you want to customise what happens when you call the class. Leave it at the default implementation otherwise.

If I remove the __call__ method from your metaclass (and ignore the __prepare__ method), then Thing.__init__ is once again called:

>>> class Logged(type):
...     def __new__(subclass, name, superclasses, attributes, **keyword_arguments):
...         print('In meta __new__')
...         return type.__new__(subclass, name, superclasses, attributes)
...     def __init__(subclass, name, superclasses, attributes, **keyword_arguments):
...         print('In meta __init__')
...
>>> class Thing(metaclass = Logged):
...     def __new__(this, *arguments, **keyword_arguments):
...         print('In sub __new__')
...         return super(Thing, this).__new__(this)
...     def __init__(self, *arguments, **keyword_arguments):
...         print('In sub __init__')
...     def hello(self):
...         print('hello')
...
In meta __new__
In meta __init__
>>> thing = Thing()
In sub __new__
In sub __init__