Need help understanding Python closures

Question

I have this code:

import re

def doReplace(toReplace):
    i = 1
    def chapterReplacer(_):
        result = 'Chapter %i' % i
        i += 1
        return result

    return re.sub('Chapter [a-zA-Z]+', chapterReplacer, test)

test = 'Chapter one Chapter Two Chapter three'
print doReplace(test)

when I run it, I get the following error:

Traceback (most recent call last):
  File "C:/Python26/replace.py", line 13, in <module>
    print doReplace(test)
  File "C:/Python26/replace.py", line 10, in doReplace
    return re.sub('Chapter [a-zA-Z]+', chapterReplacer, test)
  File "C:\Python26\lib\re.py", line 151, in sub
    return _compile(pattern, 0).sub(repl, string, count)
  File "C:/Python26/replace.py", line 6, in chapterReplacer
    result = 'Chapter %i' % i
UnboundLocalError: local variable 'i' referenced before assignment

I was under the impression that chapterReplacer would capture the local variable i, but that doesn't seem to be happening?

Answer 1

In Python, if you assign to variable inside a function (even if with a compound assignment operator such as +=), that variable is considered local unless specified otherwise by a global or nonlocal statement.

Answer 2

You can make i a function attribute

def doReplace(toReplace):
    chapterReplacer.i = 1
    def chapterReplacer(_):
        result = 'Chapter %i' % chapterReplacer.i
        chapterReplacer.i += 1
        return result

    return re.sub('Chapter [a-zA-Z]+', chapterReplacer, test)

EDIT: As of python 3, you can use nonlocal a la @MartijnPieters 's solution.

Answer 3

Nope, and in python 2 you can't at all without resorting to using tricks with mutables:

def doReplace(toReplace):
    i = [1]
    def chapterReplacer(_):
        result = 'Chapter %i' % i[0]
        i[0] += 1
        return result

    return re.sub('Chapter [a-zA-Z]+', chapterReplacer, test)

Normally, python will only look in the surrounding scope for a variable if it is not being assigned to locally; as soon as the bytecompiler sees a direct assignment (i = something) and no global i statement to persuade it otherwise, a variable is considered local.

But in the above code we never assign to i in the chapterReplacer function. Yes, we do change i[0] but the value stored in i itself, a list, does not change.

In python 3, just use the nonlocal statement to have python look in it's closure for the variable:

def doReplace(toReplace):
    i = 1
    def chapterReplacer(_):
        nonlocal i
        result = 'Chapter %i' % i
        i += 1
        return result

    return re.sub('Chapter [a-zA-Z]+', chapterReplacer, test)

Answer 4

When compiler sees that variable i gets another value inside function chapterReplacer it treats it as local, and no 'closure magic' is applied. If you remove line i += 1, your code will run.