What exactly is contained within a obj.__closure__

Question

Beazley pg 100 mentions:

>>>python.__closure__
(<cell at 0x67f50: str object at 0x69230>,)
>>>python.__closure__[0].cell_contents

my understanding is that __closure__ is a list but what's all this cell stuff and str object?? That looks like a 1-ary tuple?

Answer 1

It is the new Python 3 name for the old func_closure.

http://docs.python.org/3.0/whatsnew/3.0.html

The function attributes named func_X have been renamed to use the __X__ form, freeing up these names in the function attribute namespace for user-defined attributes. To wit, func_closure, func_code, func_defaults, func_dict, func_doc, func_globals, func_name were renamed to __closure__, __code__, __defaults__, __dict__, __doc__, __globals__, __name__, respectively.

In a nutshell:

__closure__ is None or a tuple of cells that contain binding for the function's free variables.

Also, it is NOT writable.

Reference: http://docs.python.org/ref/types.html

Example Python < 3 (so I am using func_closure)

def foo():
    x = "I am used"
    y = "I am free"
    z = "I am free too"

    def bar(x):
        return x, y, z

    return bar

c = foo().func_closure

print [i.cell_contents for i in c]

Output:

>>> 
['I am free', 'I am free too']

As foo is returning the function bar which is using its own value x, but not y or z. So, they come under __closure__.

Answer 2

when a nested function(closure) is defined in python:
the outer function uses co_cellvars to note variables defined in the outer function that could be referenced by inner function.
the inner function uses co_freevars to note variables defined in the outer function, which could be referenced for later use.

Example:

# python3
Python 3.4.5 (default, May 29 2017, 15:17:55) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def foo(n):
...     a = 1
...     def g(n):
...             return a - n
...     return g
... 
>>> foo.__closure__
>>> foo.__code__.co_freevars
()
>>> foo.__code__.co_cellvars
('a',)
>>> foo(0).__closure__
(<cell at 0x7f2cd98db1c8: int object at 0x7f2cd9847960>,)
>>> foo(0).__closure__[0].cell_contents
1
>>> foo(0).__code__.co_freevars
('a',)
>>> foo(0).__code__.co_cellvars
()

Answer 3

>>> def f():
...     a = "HELO"
...     b = 1.0
...     def w(c):
...         return a,b,c
...     return w

>>> w = f()
>>> w.__closure__
(<cell at 0xa05c4ac: str object at 0x9e91b74>, <cell at 0xa05c3bc: float object at 0xb733dde8>)
>>> w.__closure__[0].cell_contents
'HELO'
>>> w.__closure__[1].cell_contents
1.0

I have never seen the cell type used anywhere else. It seems to be purpose-built to hold closure variables.

Answer 4

Closure cells refer to values needed by the function but are taken from the surrounding scope.

When Python compiles a nested function, it notes any variables that it references but are only defined in a parent function (not globals) in the code objects for both the nested function and the parent scope. These are the co_freevars and co_cellvars attributes on the __code__ objects of these functions, respectively.

Then, when you actually create the nested function (which happens when the parent function is executed), those references are then used to attach a closure to the nested function.

A function closure holds a tuple of cells, one each for each free variable (named in co_freevars); cells are special references to local variables of a parent scope, that follow the values those local variables point to. This is best illustrated with an example:

def foo():
    def bar():
        print(spam)

    spam = 'ham'
    bar()
    spam = 'eggs'
    bar()
    return bar

b = foo()
b()

In the above example, the function bar has one closure cell, which points to spam in the function foo. The cell follows the value of spam. More importantly, once foo() completes and bar is returned, the cell continues to reference the value (the string eggs) even though the variable spam inside foo no longer exists.

Thus, the above code outputs:

>>> b=foo()
ham
eggs
>>> b()
eggs

and b.__closure__[0].cell_contents is 'eggs'.

Note that the closure is dereferenced when bar() is called; the closure doesn't capture the value here. That makes a difference when you produce nested functions (with lambda expressions or def statements) that reference the loop variable:

def foo():
    bar = []
    for spam in ('ham', 'eggs', 'salad'):
        bar.append(lambda: spam)
    return bar

for bar in foo():
    print bar()

The above will print salad three times in a row, because all three lambda functions reference the spam variable, not the value it was bound to when the function object was created. By the time the for loop finishes, spam was bound to 'salad', so all three closures will resolve to that value.