What limitations have closures in Python compared

Question

Where X is any programming language (C#, Javascript, Lisp, Perl, Ruby, Scheme, etc) which supports some flavour of closures.

Some limitations are mentioned in the Closures in Python (compared to Ruby's closures), but the article is old and many limitations do not exist in modern Python any more.

Seeing a code example for a concrete limitation would be great.

Related questions:

• Can you explain closures (as they relate to Python)?
• What is a ‘Closure’?
• How does a javascript closure work ?

Answer 1

The most important limitation, currently, is that you cannot assign to an outer-scope variable. In other words, closures are read-only:

>>> def outer(x): 
...     def inner_reads():
...         # Will return outer's 'x'.
...         return x
...     def inner_writes(y):
...         # Will assign to a local 'x', not the outer 'x'
...         x = y
...     def inner_error(y):
...         # Will produce an error: 'x' is local because of the assignment,
...         # but we use it before it is assigned to.
...         tmp = x
...         x = y
...         return tmp
...     return inner_reads, inner_writes, inner_error
... 
>>> inner_reads, inner_writes, inner_error = outer(5)
>>> inner_reads()
5
>>> inner_writes(10)
>>> inner_reads()
5
>>> inner_error(10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 11, in inner_error
UnboundLocalError: local variable 'x' referenced before assignment

A name that gets assigned to in a local scope (a function) is always local, unless declared otherwise. While there is the 'global' declaration to declare a variable global even when it is assigned to, there is no such declaration for enclosed variables -- yet. In Python 3.0, there is (will be) the 'nonlocal' declaration that does just that.

You can work around this limitation in the mean time by using a mutable container type:

>>> def outer(x):
...     x = [x]
...     def inner_reads():
...         # Will return outer's x's first (and only) element.
...         return x[0]
...     def inner_writes(y):
...         # Will look up outer's x, then mutate it.      
...         x[0] = y
...     def inner_error(y):
...         # Will now work, because 'x' is not assigned to, just referenced.
...         tmp = x[0]
...         x[0] = y
...         return tmp
...     return inner_reads, inner_writes, inner_error
... 
>>> inner_reads, inner_writes, inner_error = outer(5)
>>> inner_reads()
5
>>> inner_writes(10)
>>> inner_reads()
10
>>> inner_error(15)
10
>>> inner_reads()
15

Answer 2

A limitation (or "limitation") of Python closures, comparing to Javascript closures, is that it cannot be used for effective data hiding

Javascript

var mksecretmaker = function(){
    var secrets = [];
    var mksecret = function() {
        secrets.push(Math.random())
    }
    return mksecret
}
var secretmaker = mksecretmaker();
secretmaker(); secretmaker()
// privately generated secret number list
// is practically inaccessible

Python

import random
def mksecretmaker():
    secrets = []
    def mksecret():
        secrets.append(random.random())
    return mksecret

secretmaker = mksecretmaker()
secretmaker(); secretmaker()
# "secrets" are easily accessible,
# it's difficult to hide something in Python:
secretmaker.__closure__[0].cell_contents # -> e.g. [0.680752847190161, 0.9068475951742101]

Answer 3

_{comment for @Kevin Little's answer to include the code example}

nonlocal does not solve completely this problem on python3.0:

x = 0 # global x
def outer():
    x = 1 # local to `outer`
    def inner():
        global x
        x = 2 # change global
        print(x) 
        x = 3 # change global
        return x
    def inner2():
##        nonlocal x # can't use `nonlocal` here
        print(x)     # prints global
##        x = 4      # can't change `x` here
        return x
    x = 5
    return (inner, inner2)

for inner in outer():
    print(inner())
# -> 2 3 3 3

On the other hand:

x = 0
def outer():
    x = 1 # local to `outer`
    def inner():
##        global x
        x = 2
        print(x) # local to `inner` 
        x = 3 
        return x
    def inner2():
        nonlocal x
        print(x)
        x = 4  # local to `outer`
        return x
    x = 5
    return (inner, inner2)

for inner in outer():
    print(inner())
# -> 2 3 5 4

it works on python3.1-3.3

Answer 4

The better workaround until 3.0 is to include the variable as a defaulted parameter in the enclosed function definition:

def f()
    x = 5
    def g(y, z, x=x):
        x = x + 1

Answer 5

@John Millikin

def outer():
    x = 1 # local to `outer()`

    def inner():
        x = 2     # local to `inner()`
        print(x)
        x = 3
        return x

    def inner2():
        nonlocal x
        print(x)  # local to `outer()`
        x = 4     # change `x`, it is not local to `inner2()`
        return x

    x = 5         # local to `outer()`
    return (inner, inner2)

for inner in outer():
    print(inner()) 

# -> 2 3 5 4

Answer 6

The only difficulty I've seen people encounter with Python's in particular is when they try to mix non-functional features like variable reassignment with closures, and are surprised when this doesn't work:

def outer ():
    x = 1
    def inner ():
        print x
        x = 2
    return inner
outer () ()

Usually just pointing out that a function has its own local variables is enough to deter such silliness.