We Keep Coding

Why does global keyword required to access variable defined in outer scope?

Another question does not explain why I can't change variable time_verf without using 'global' in the example (2) but still can do it to the list in the example (4).
On the resource I found that I can not change global variable from within a function, which is clearly illustrated by these examples:
from datetime import datetime, timedelta
time_verf = datetime.now()
I think I understand why the following is working (1):
def check_global():
global time_verf
clean_list = time_verf + timedelta(hours=12) # время очистки листа
if clean_list < datetime.now():
list_verf.clear()
time_verf = datetime.now()
print('ok')
>> check_global()
<< ok
Next it throws exception when I comment out line with global keyword (2):
def check_global():
# global time_verf
clean_list = time_verf + timedelta(hours=12) # время очистки листа
if clean_list < datetime.now():
list_verf.clear()
time_verf = datetime.now()
print('ok')
>> check_global()
<< Traceback (most recent call last):
File "<input>", line 1, in <module>
File "<input>", line 3, in check_global
UnboundLocalError: local variable 'time_verf' referenced before assignment
And then again can be again referenced without 'global' when line with assignment commented out(3):
def check_global():
# global time_verf
clean_list = time_verf + timedelta(hours=12) # время очистки листа
if clean_list < datetime.now():
list_verf.clear()
# time_verf = datetime.now()
print('ok')
>> check_global()
<< ok
But why am I allowed to update list defined in the outer scope without using global (4)?
list = []
def check_global_list():
list.append('check')
>> check_global_list()
>> list
<< ['check']

When you comment out the global time_verf statement in line 2,
1: def check_global():
2: # global time_verf
3: clean_list = time_verf + timedelta(hours=12) # время очистки листа
4: if clean_list < datetime.now():
5: list_verf.clear()
6: time_verf = datetime.now()
line 6 assigns a value to a local variable time_verf. It is local because assigning it creates it as a local variable. You get an error on line 3 because it refers to the local variable that your code creates in line 6. If you did not have that assignment then time_verf would be global by default.
But whether a variable is global or not does not depend on the order of execution. You appear to expect that because line 3 on its own would make time_verf global by default, then it becomes global and remains global in line 6. But that isn't the way local variables work. The presence of line 6 changes the meaning (and correctness) of line 3. The interpreter examines the entire scope of your function and creates local variables for any names that the code assigns values to. So because of line 6, time_verf is local, even if line 3 on its own would make it global.
There are very good reasons for this behaviour. Suppose line 3 were enclosed in an if-test. Then, according to the behaviour you seem to expect, the variable would be global if the if-test were true, but local if the if-test were false.

The resource you found states "If we need to assign a new value to a global variable then we can do that by declaring the variable as global.", the keyword being "assign". You can access global variables, call methods on them, mutate them without declaring them as global.
It's when you need to assign to them that you need to declare them as global.

Edit: I agree with Roel Schroeven that the core issue here is assignment!
Nevertheless, BoarGules gave a good answer regarding variable lifetime imho. And in addition I think an important point to notice here is mutability vs. immutability. This especially refers to (4) of the question. While the following works fine
a = 2.72 # global scope, defined BEFORE the function
def modify():
b = a + 3.14 # no assignmnet made to a (but to b in this case)
return b # scope of b is local -> 'return' needed to make it available in the outer scope
In [106]: modify()
Out[106]: 5.86
this fails:
a = 2.72
def modify(): # assignment!
a += 3.14 # you would need 'global a' before this to make it work
return a
In [108]: modify()
UnboundLocalError: local variable 'a' referenced before assignment
While a can be called inside modify(), it cannot be modified since it is type float and thus an immutable object.
On the other hand if you do the same with a mutable object like a list, you get
a = [2.72]
def modify():
a[0] = 3.14 # change the reference to another value...
# no return needed
In [110]: modify()
In [110]: a
Out[110]: [3.14]
It doesn't fail AND a changed, even outside the scope of the function! It will fail again if you call a previously undefined variable in the function. Note that a[0] = 3.14 is not an assignment of a to some value but changes a reference stored in a to another value. This is very important if you have multiple functions in a script and pass things around.
For furter reading, there are good resources on the topic out there, e.g. this as a starter and maybe also on the python data model in the docs.

Remembered values, and scope gone from memory, in a Python Closure

Below is a simple piece of code I found in this tutorial.
Here's a nice definition of Closure I found here: "a function object that remembers values in enclosing scopes regardless of whether those scopes are still present in memory."
I gather that rotate() below is a closure. Please help me understand what values is it remembering even after their scope is gone from memory (and why does their scope leave memory)?
def make_rotater(seq):
def rotate():
val = seq.pop(0)
seq.append(val)
return val
return rotate
r = make_rotater([1,2,3])
r()
# 1
r()
# 2
(Update) Part 2: Why does the (closure-less) code below not work?
def make_rotater(seq):
val = seq.pop(0)
seq.append(val)
return val
r = make_rotater([1,2,3])
r()
# File "<stdin>", line 1, in <module>
# TypeError: 'int' object is not callable

It remembers local values from make_rotator so if u do:
def make_rotater():
seq=[1,2,3]
def rotate():
val = seq.pop(0)
seq.append(val)
return val
return rotate
The seq is referenced by rotate, so it will remain in memory for when you will call rotate, even though it was defined in make_rotater (that is already done, and cleaned from memory)
When you call make_rotater it creates a new seq and defines the rotate method that references seq, so one you leave make_rotater it's memory isn't needed except seq, (cause rotate still uses it). When you will no longer reference rotate, seq will also be cleaned
part 2:
your method now doesnt return another method it returns the number directly, so you dont need to do r()
you can use it like this:
seq = [1,2,3]
def make_rotater(seq):
val = seq.pop(0)
seq.append(val)
return val
r = make_rotater(seq)
print r # prints 1, note there is no r() just r
r = make_rotater(seq)
print r # prints 2
r = make_rotater(seq)
print r # prints 3

That definition is sort of right, sort of wrong. It depends on what you mean by a scope being "still present in memory". I'd say a better definition would be "a function object that remembers variables in enclosing scopes regardless of whether those scopes are still present on the call stack."
When you call make_rotater:
def make_rotater(seq):
def rotate():
val = seq.pop(0)
seq.append(val)
return val
return rotate
The rotate closure keeps the seq variable alive even after execution leaves the scope of make_rotater. Ordinarily, when execution leaves a function, its local variables would cease to exist.

local variable referenced before assignment in python when i set it global

from random import randint
shifts = [4, 4.2, 5, 6, 7]
days_names = ['Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday']
workers_names = ['Itai', 'Or', 'Reut', 'Kuka', 'Aviel']
counter = 1
def shift_arrange(worker):
for day in days.values():
counter+=1
global avilable_shifts
avilable_shifts = check_avilable_shifts(day)
if not random_shifte_selector(worker,day): soft_reset(worker)
I set counter as a global variable, and when i try to run this code i get the local variable error:
Traceback (most recent call last):
File "C:\Or\mypy\shift creator\shift cretor.py", line 144, in <module>
for w in workers.values(): shift_arrange(w)
File "C:\Or\mypy\shift creator\shift cretor.py", line 105, in shift_arrange
counter+=1
UnboundLocalError: local variable 'counter' referenced before assignmen
I saw some guy ask this question here, he deleted his pyc file or something(i don't know what is it) and its work fine. Why this is happen? Its not happen to other variables in the program.
Thanks, Or

You need to declare a global variable
def shift_arrange(worker):
global counter
for day in days.values():
counter+=1
...
Since you modify counter in that scope, python treats it as a local variable, unless you declare it as global. If you only need to read it, that isn't necessary.
Consider the following:
This works:
c = 0
def f():
print c
f()
While this does not:
c = 0
def f():
print c
c = 1
f()
While this does:
c = 0
def f():
global c
print c
c = 1
f()
print c # prints 1, f() modified the global value

There's a lot to look up here, but in general python has names, and things that are assigned to them. The name could be either in the local or global scope.
For reads, python looks through local scope, then through global scope, and uses the first one it finds (or error if it doesn't).
For writes... python needs to know where to put it. Normally, what it would do is look in the local scope, and if it's not there, create a variable there and assign the value. This would hide the global variable. You could have it also look in globals and use that one if it exists - but that could be undesirable & unexpected. So, you need a way to tell python to use a global variable instead if it exists (and then, it will create if it doesn't).
This leads to some odd behaviour sometimes as well. In addition to the earlier answer...
c = 0
# Passes. We are just looking up the global variable.
def f1(x):
return x + c
# Passes, but may not be as expected. Sets a local variable c to a value, does not
# modify the global one.
def f2(x):
c = x
# Correct way to do the above; now it sets the global variable.
def f3(x):
global c
c = x
# What if you mix them?
def f4(x):
c = c + x
# This fails. What happens is that first it sees that it's writing to c, and it's
# not marked global, so it assigns c to the local space. Then it tries to dereference
# it. Since we've marked it local, it masks the global one, and since it doesn't
# have a value, it throws an error. c += x works the same way and also fails.
# However, this works, though is just as equally wrong:
def f5(x):
d = c
c = d + x
# This one works, because we copy the value of the global c into the local d.
# Then, the second line assigns a local c to addition of local d and x.
# But does not update the global.
# Each line is separate though:
def f6(x):
d = c
c = c + 1
# You might think that d=c already made c local for the whole function. But it doesn't
# work like that. The first line just looks up the value for c, which it finds
# globally, but then forgets about it - it cares about the object c is the name of,
# not the name c itself. The second line still fails.

Why do I get Unbound Local Error?

So the functions in question are very long, so I will summarize it.
def func1( X = None, Y = None ) :
if X :
dostuff
if condition :
Z += 1
if Y :
print Y
func1.Z = 0
def func2( A )
for loop that does stuff and calls func1
When I run this, it tells me that the line Z += 1 has an error "UnboundLocalError: local variable 'Z' referenced before assignment"
I've read all the Unbound Local Error and Referanced before assignment questions I could find already, but none of the problems seem similar, and none of the solutions work.
Should I just make Z a global?

Because setting an attribute on a function object does not create a local name. Reference it on the function object within the function too:
func1.Z += 1
or make it a global. Better still, avoid such shenanigans and just make Z a local or a function argument, and return Z at the end of the function to the caller.

Variable scope in nested functions

Could someone explain why the following program fails:
def g(f):
for _ in range(10):
f()
def main():
x = 10
def f():
print x
x = x + 1
g(f)
if __name__ == '__main__':
main()
with the message:
Traceback (most recent call last):
File "a.py", line 13, in <module>
main()
File "a.py", line 10, in main
g(f)
File "a.py", line 3, in g
f()
File "a.py", line 8, in f
print x
UnboundLocalError: local variable 'x' referenced before assignment
But if I simply change the variable x to an array, it works:
def g(f):
for _ in range(10):
f()
def main():
x = [10]
def f():
print x[0]
x[0] = x[0] + 1
g(f)
if __name__ == '__main__':
main()
with the output
10
11
12
13
14
15
16
17
18
19
The reason I am confused is, if from f() it can't access x, why it becomes accessible if x is an array?
Thanks.

But this answer says the problem is with assigning to x. If that's it,
then printing it should work just fine, shouldn't it?
You have to understand the order in which things happen. Before your python code is even compiled and executed, something called a parser reads through the python code and checks the syntax. Another thing the parser does is mark variables as being local. When the parser sees an assignment in the code in a local scope, the variable on the lefthand side of the assignment is marked as local. At that point, nothing has even been compiled yet--let alone executed, and therefore no assignment takes place; the variable is merely marked as a local variable.
After the parser is finished, the code is compiled and executed. When execution reaches the print statement:
def main():
x = 10 #<---x in enclosing scope
def f():
print x #<-----
x = x + 1 #<-- x marked as local variable inside the function f()
the print statement looks like it should go ahead and print the x in the enclosing scope (the 'E' in the LEGB lookup process). However, because the parser previously marked x as a local variable inside f(), python does not proceed past the local scope (the 'L' in the LEGB lookup process) to lookup x. Because x has not been assigned to in the local scope at the time 'print x' executes, python spits out an error.
Note that even if the code where an assignment occurs will NEVER execute, the parser still marks the variable on the left of an assignment as a local variable. The parser has no idea about how things will execute, so it blindly searches for syntax errors and local variables throughout your file--even in code that can never execute. Here are some examples of that:
def dostuff ():
x = 10
def f():
print x
if False: #The body of the if will never execute...
a b c #...yet the parser finds a syntax error here
return f
f = dostuff()
f()
--output:--
File "1.py", line 8
a b c
^
SyntaxError: invalid syntax
The parser does the same thing when marking local variables:
def dostuff ():
x = 10
def f():
print x
if False: #The body of the if will never execute...
x = 0 #..yet the parser marks x as a local variable
return f
f = dostuff()
f()
Now look what happens when you execute that last program:
Traceback (most recent call last):
File "1.py", line 11, in <module>
f()
File "1.py", line 4, in f
print x
UnboundLocalError: local variable 'x' referenced before assignment
When the statement 'print x' executes, because the parser marked x as a local variable the lookup for x stops at the local scope.
That 'feature' is not unique to python--it happens in other languages too.
As for the array example, when you write:
x[0] = x[0] + 1
that tells python to go lookup up an array named x and assign something to its first element. Because there is no assignment to anything named x in the local scope, the parser does not mark x as a local variable.

The reason is in first example you used an assignment operation, x = x + 1, so when the functions was defined python thought that x is local variable. But when you actually called the function python failed to find any value for the x on the RHS locally, So raised an Error.
In your second example instead of assignment you simply changed a mutable object, so python will never raise any objection and will fetch x[0]'s value from the enclosing scope(actually it looks for it firstly in the enclosing scope, then global scope and finally in the builtins, but stops as soon as it was found).
In python 3x you can handle this using the nonlocal keyword and in py2x you can either pass the value to the inner function or use a function attribute.
Using function attribute:
def main():
main.x = 1
def f():
main.x = main.x + 1
print main.x
return f
main()() #prints 2
Passing the value explicitly:
def main():
x = 1
def f(x):
x = x + 1
print x
return x
x = f(x) #pass x and store the returned value back to x
main() #prints 2
Using nonlocal in py3x:
def main():
x = 1
def f():
nonlocal x
x = x + 1
print (x)
return f
main()() #prints 2

The problem is that the variable x is picked up by closure. When you try to assign to a variable that is picked up from the closure, python will complain unless you use the global or nonlocal1 keywords. In the case where you are using a list, you're not assigning to the name -- You can modify an object which got picked up in the closure, but you can't assign to it.
Basically, the error occurs at the print x line because when python parses the function, It sees that x is assigned to so it assumes x must be a local variable. When you get to the line print x, python tries to look up a local x but it isn't there. This can be seen by using dis.dis to inspect the bytecode. Here, python uses the LOAD_FAST instruction which is used for local variables rather than the LOAD_GLOBAL instruction which is used for non-local variables.
Normally, this would cause a NameError, but python tries to be a little more helpful by looking for x in func_closure or func_globals 2. If it finds x in one of those, it raises an UnboundLocalError instead to give you a better idea about what is happening -- You have a local variable which couldn't be found (isn't "bound").
1python3.x only
2python2.x -- On python3.x, those attributes have changed to __closure__ and __globals__ respectively

The problem is in the line
x = x + 1
This is the first time x being assigned in function f(), telling the compiler that x is a local name. It conflicts with the previous line print x, which can't find any previous assignment of the local x.
That's where your error UnboundLocalError: local variable 'x' referenced before assignment comes from.
Note that the error happens when compiler tries to figure out which object the x in print x refers to. So the print x doesn't executes.
Change it to
x[0] = x[0] + 1
No new name is added. So the compiler knows you are referring to the array outside f().