What does __flags__ in python type used for

Question

I have been read pickle source code recently.

The following code in copy_reg make me confused:

_HEAPTYPE = 1<<9

def _reduce_ex(self, proto):
    assert proto < 2
    for base in self.__class__.__mro__:
        if hasattr(base, '__flags__') and not base.__flags__ & _HEAPTYPE:
            break
    else:
        base = object # not really reachable
    if base is object:
        state = None

So what does __flags__ using for?

I found it is defined in type object:

type.__flags__ = 2148423147

I was tried to search it in the official doc, but nothing was found.

But interesting thing is that __class__.__flags__ & _HEAPTYPE is always 0 when the __class__ is python internal type. And the result will be 1 when __class__ is a subclass of python internal type.

Can anyone help me to solve this puzzle?

Answer 1

__flags__ is a wrapper, to access CPython type object structure member tp_flags, constants used to compose this flag defined in object.h, following is quoted from the source:

Type flags (tp_flags) These flags are used to extend the type structure in a backwards-compatible fashion. Extensions can use the flags to indicate (and test) when a given type structure contains a new feature. The Python core will use these when introducing new functionality between major revisions (to avoid mid-version changes in the PYTHON_API_VERSION).

See more detail on python document on tp_flags.

But interesting thing is that class.flags & _HEAPTYPE is always 0 when the class is a python internal type. And the result will be 1 when class is a subclass of python internal type.

Subclass of python built-in type, like other user defined type, allocated on heap with PyType_GenericAlloc().

---

to break down type.__flags__:

import re

def flags_to_name(type_obj):
    tp_flag_consts = {}        
    with open('/path/to/Include/object.h') as f:
        for l in f:
            m = re.search(r'^#define (Py_TPFLAGS_\w+)\s+\(.+?<< (\d+)\)', l.strip())
            if m:
                tp_flag_consts[int(m.group(2))] = m.group(1)
    bin_str = bin(type_obj.__flags__)[2:][::-1]
    return ', '.join(tp_flag_consts[n] for n, c in enumerate(bin_str) if c == '1')

print(flags_to_name(type))

yields:

Py_TPFLAGS_BASETYPE, Py_TPFLAGS_READY, Py_TPFLAGS_HAVE_GC, Py_TPFLAGS_HAVE_VERSION_TAG, Py_TPFLAGS_VALID_VERSION_TAG, Py_TPFLAGS_TYPE_SUBCLASS

Answer 2

__flags__ is to be viewed as binary, if you want to see the flags.

These type flags are defined here in Python's source code.