Closure in python?

Question

When I run this code, I get this result:

15
15

I expect the output should be

15
17

but it is not. The question is: why?

def make_adder_and_setter(x):
    def setter(n):
        x = n

    return (lambda y: x + y, setter)

myadder, mysetter = make_adder_and_setter(5)
print myadder(10)
mysetter(7)
print myadder(10)

Answer 1

You are setting a local variable x in the setter() function. Assignment to a name in a function marks it as a local, unless you specifically tell the Python compiler otherwise.

In Python 3, you can explicitly mark x as non-local using the nonlocal keyword:

def make_adder_and_setter(x):
    def setter(n):
        nonlocal x
        x = n

    return (lambda y: x + y, setter)

Now x is marked as a free variable and looked up in the surrounding scope instead when assigned to.

In Python 2 you cannot mark a Python local as such. The only other option you have is marking x as a global. You'll have to resort to tricks where you alter values contained by a mutable object that lives in the surrounding scope.

An attribute on the setter function would work, for example; setter is local to the make_adder_and_setter() scope, attributes on that object would be visible to anything that has access to setter:

def make_adder_and_setter(x):
    def setter(n):
        setter.x = n
    setter.x = x

    return (lambda y: setter.x + y, setter)

Another trick is to use a mutable container, such as a list:

def make_adder_and_setter(x):
    x = [x]
    def setter(n):
        x[0] = n

    return (lambda y: x[0] + y, setter)

In both cases you are not assigning to a local name anymore; the first example uses attribute assignment on the setter object, the second alters the x list, not assign to x itself.

Answer 2

Python 2.x has a syntax limitation that doesn't allow to capture a variable in read/write.

The reason is that if a variable is assigned in a function there are only two possibilities:

1. the variable is a global and has been declared so with global x
2. the variable is a local of the function

more specifically it's ruled out that the variable is a local of an enclosing function scope

This has been superseded in Python 3.x with the addition of nonlocal declaration. Your code would work as expected in Python 3 by changing it to

def make_adder_and_setter(x):
    def setter(n):
        nonlocal x
        x = n

    return (lambda y: x + y, setter)

The python 2.x runtime is able to handle read-write closed over variable at a bytecode level, however the limitation is in the syntax that the compiler accepts.

You can see a lisp compiler that generates python bytecode directly that creates an adder closure with read-write captured state at the end of this video. The compiler can generate bytecode for Python 2.x, Python 3.x or PyPy.

If you need closed-over mutable state in Python 2.x a trick is to use a list:

def make_adder_and_setter(x):
    x = [x]
    def setter(n):
        x[0] = n

    return (lambda y: x[0] + y, setter)

Answer 3

Your inner def setter(n) function defines its own local variable x. That hides the other x variable that was a parameter of make_adder_and_setter (makes a hole in the scope). So the setter function has no side effect. It just sets the value of an inner local variable and exits.

Maybe it will be clear for you if you try the code below. It does exactly the same thing, just uses the name z instead of x.

def make_adder_and_setter(x):
    def setter(n):
        z = n

    return (lambda y: x + y, setter)

myadder, mysetter = make_adder_and_setter(5)
print myadder(10)
mysetter(7)
print myadder(10)