Variable scope in nested functions

Question

Could someone explain why the following program fails:

def g(f):
  for _ in range(10):
    f()

def main():
  x = 10
  def f():
    print x
    x = x + 1
  g(f)

if __name__ == '__main__':
  main()

with the message:

Traceback (most recent call last):
  File "a.py", line 13, in <module>
    main()
  File "a.py", line 10, in main
    g(f)
  File "a.py", line 3, in g
    f()
  File "a.py", line 8, in f
    print x
UnboundLocalError: local variable 'x' referenced before assignment

But if I simply change the variable x to an array, it works:

def g(f):
  for _ in range(10):
    f()

def main():
  x = [10]
  def f():
    print x[0]
    x[0] = x[0] + 1
  g(f)

if __name__ == '__main__':
  main()

with the output

10
11
12
13
14
15
16
17
18
19

The reason I am confused is, if from f() it can't access x, why it becomes accessible if x is an array?

Thanks.

Answer 1

The reason is in first example you used an assignment operation, x = x + 1, so when the functions was defined python thought that x is local variable. But when you actually called the function python failed to find any value for the x on the RHS locally, So raised an Error.

In your second example instead of assignment you simply changed a mutable object, so python will never raise any objection and will fetch x[0]'s value from the enclosing scope(actually it looks for it firstly in the enclosing scope, then global scope and finally in the builtins, but stops as soon as it was found).

In python 3x you can handle this using the nonlocal keyword and in py2x you can either pass the value to the inner function or use a function attribute.

Using function attribute:

def main():
  main.x = 1
  def f():
      main.x = main.x + 1
      print main.x
  return f

main()()   #prints 2

Passing the value explicitly:

def main():
  x = 1
  def f(x):
      x = x + 1
      print x
      return x
  x = f(x)     #pass x and store the returned value back to x

main()   #prints 2

Using nonlocal in py3x:

def main():
  x = 1
  def f():
      nonlocal x
      x = x + 1
      print (x)
  return f

main()()  #prints 2

Answer 2

The problem is that the variable x is picked up by closure. When you try to assign to a variable that is picked up from the closure, python will complain unless you use the global or nonlocal¹ keywords. In the case where you are using a list, you're not assigning to the name -- You can modify an object which got picked up in the closure, but you can't assign to it.

---

Basically, the error occurs at the print x line because when python parses the function, It sees that x is assigned to so it assumes x must be a local variable. When you get to the line print x, python tries to look up a local x but it isn't there. This can be seen by using dis.dis to inspect the bytecode. Here, python uses the LOAD_FAST instruction which is used for local variables rather than the LOAD_GLOBAL instruction which is used for non-local variables.

Normally, this would cause a NameError, but python tries to be a little more helpful by looking for x in func_closure or func_globals ². If it finds x in one of those, it raises an UnboundLocalError instead to give you a better idea about what is happening -- You have a local variable which couldn't be found (isn't "bound").

^{1python3.x only}

^{2python2.x -- On python3.x, those attributes have changed to __closure__ and __globals__ respectively}

Answer 3

The problem is in the line

x = x + 1

This is the first time x being assigned in function f(), telling the compiler that x is a local name. It conflicts with the previous line print x, which can't find any previous assignment of the local x. That's where your error UnboundLocalError: local variable 'x' referenced before assignment comes from.

Note that the error happens when compiler tries to figure out which object the x in print x refers to. So the print x doesn't executes.

Change it to

x[0] = x[0] + 1

No new name is added. So the compiler knows you are referring to the array outside f().

Answer 4

But this answer says the problem is with assigning to x. If that's it, then printing it should work just fine, shouldn't it?

You have to understand the order in which things happen. Before your python code is even compiled and executed, something called a parser reads through the python code and checks the syntax. Another thing the parser does is mark variables as being local. When the parser sees an assignment in the code in a local scope, the variable on the lefthand side of the assignment is marked as local. At that point, nothing has even been compiled yet--let alone executed, and therefore no assignment takes place; the variable is merely marked as a local variable.

After the parser is finished, the code is compiled and executed. When execution reaches the print statement:

def main():
  x = 10     #<---x in enclosing scope

  def f():
    print x    #<-----

    x = x + 1  #<-- x marked as local variable inside the function f()

the print statement looks like it should go ahead and print the x in the enclosing scope (the 'E' in the LEGB lookup process). However, because the parser previously marked x as a local variable inside f(), python does not proceed past the local scope (the 'L' in the LEGB lookup process) to lookup x. Because x has not been assigned to in the local scope at the time 'print x' executes, python spits out an error.

Note that even if the code where an assignment occurs will NEVER execute, the parser still marks the variable on the left of an assignment as a local variable. The parser has no idea about how things will execute, so it blindly searches for syntax errors and local variables throughout your file--even in code that can never execute. Here are some examples of that:

def dostuff ():
    x = 10 

    def f():
        print x

        if False:  #The body of the if will never execute...
            a b c  #...yet the parser finds a syntax error here


    return f

f = dostuff()
f()



--output:--
File "1.py", line 8
     a b c
      ^
SyntaxError: invalid syntax

The parser does the same thing when marking local variables:

def dostuff ():
    x = 10 

    def f():
        print x

        if False:  #The body of the if will never execute...
            x = 0  #..yet the parser marks x as a local variable

    return f

f = dostuff()
f()

Now look what happens when you execute that last program:

Traceback (most recent call last):
  File "1.py", line 11, in <module>
    f()
  File "1.py", line 4, in f
    print x
UnboundLocalError: local variable 'x' referenced before assignment

When the statement 'print x' executes, because the parser marked x as a local variable the lookup for x stops at the local scope.

That 'feature' is not unique to python--it happens in other languages too.

As for the array example, when you write:

x[0] = x[0] + 1

that tells python to go lookup up an array named x and assign something to its first element. Because there is no assignment to anything named x in the local scope, the parser does not mark x as a local variable.