Where X is any programming language (C#, Javascript, Lisp, Perl, Ruby, Scheme, etc) which supports some flavour of closures.
Some limitations are mentioned in the Closures in Python (compared to Ruby's closures), but the article is old and many limitations do not exist in modern Python any more.
Seeing a code example for a concrete limitation would be great.
Related questions:
Can you explain closures (as they relate to Python)?
What is a ‘Closure’?
How does a javascript closure work ?
The most important limitation, currently, is that you cannot assign to an outer-scope variable. In other words, closures are read-only:
>>> def outer(x):
... def inner_reads():
... # Will return outer's 'x'.
... return x
... def inner_writes(y):
... # Will assign to a local 'x', not the outer 'x'
... x = y
... def inner_error(y):
... # Will produce an error: 'x' is local because of the assignment,
... # but we use it before it is assigned to.
... tmp = x
... x = y
... return tmp
... return inner_reads, inner_writes, inner_error
...
>>> inner_reads, inner_writes, inner_error = outer(5)
>>> inner_reads()
5
>>> inner_writes(10)
>>> inner_reads()
5
>>> inner_error(10)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 11, in inner_error
UnboundLocalError: local variable 'x' referenced before assignment
A name that gets assigned to in a local scope (a function) is always local, unless declared otherwise. While there is the 'global' declaration to declare a variable global even when it is assigned to, there is no such declaration for enclosed variables -- yet. In Python 3.0, there is (will be) the 'nonlocal' declaration that does just that.
You can work around this limitation in the mean time by using a mutable container type:
>>> def outer(x):
... x = [x]
... def inner_reads():
... # Will return outer's x's first (and only) element.
... return x[0]
... def inner_writes(y):
... # Will look up outer's x, then mutate it.
... x[0] = y
... def inner_error(y):
... # Will now work, because 'x' is not assigned to, just referenced.
... tmp = x[0]
... x[0] = y
... return tmp
... return inner_reads, inner_writes, inner_error
...
>>> inner_reads, inner_writes, inner_error = outer(5)
>>> inner_reads()
5
>>> inner_writes(10)
>>> inner_reads()
10
>>> inner_error(15)
10
>>> inner_reads()
15
The only difficulty I've seen people encounter with Python's in particular is when they try to mix non-functional features like variable reassignment with closures, and are surprised when this doesn't work:
def outer ():
x = 1
def inner ():
print x
x = 2
return inner
outer () ()
Usually just pointing out that a function has its own local variables is enough to deter such silliness.
A limitation (or "limitation") of Python closures, comparing to Javascript closures, is that it cannot be used for effective data hiding
Javascript
var mksecretmaker = function(){
var secrets = [];
var mksecret = function() {
secrets.push(Math.random())
}
return mksecret
}
var secretmaker = mksecretmaker();
secretmaker(); secretmaker()
// privately generated secret number list
// is practically inaccessible
Python
import random
def mksecretmaker():
secrets = []
def mksecret():
secrets.append(random.random())
return mksecret
secretmaker = mksecretmaker()
secretmaker(); secretmaker()
# "secrets" are easily accessible,
# it's difficult to hide something in Python:
secretmaker.__closure__[0].cell_contents # -> e.g. [0.680752847190161, 0.9068475951742101]
Fixed in Python 3 via the nonlocal statement:
The nonlocal statement causes the listed identifiers to refer to previously bound variables in the nearest enclosing scope excluding globals. This is important because the default behavior for binding is to search the local namespace first. The statement allows encapsulated code to rebind variables outside of the local scope besides the global (module) scope.
#John Millikin
def outer():
x = 1 # local to `outer()`
def inner():
x = 2 # local to `inner()`
print(x)
x = 3
return x
def inner2():
nonlocal x
print(x) # local to `outer()`
x = 4 # change `x`, it is not local to `inner2()`
return x
x = 5 # local to `outer()`
return (inner, inner2)
for inner in outer():
print(inner())
# -> 2 3 5 4
comment for #Kevin Little's answer to include the code example
nonlocal does not solve completely this problem on python3.0:
x = 0 # global x
def outer():
x = 1 # local to `outer`
def inner():
global x
x = 2 # change global
print(x)
x = 3 # change global
return x
def inner2():
## nonlocal x # can't use `nonlocal` here
print(x) # prints global
## x = 4 # can't change `x` here
return x
x = 5
return (inner, inner2)
for inner in outer():
print(inner())
# -> 2 3 3 3
On the other hand:
x = 0
def outer():
x = 1 # local to `outer`
def inner():
## global x
x = 2
print(x) # local to `inner`
x = 3
return x
def inner2():
nonlocal x
print(x)
x = 4 # local to `outer`
return x
x = 5
return (inner, inner2)
for inner in outer():
print(inner())
# -> 2 3 5 4
it works on python3.1-3.3
The better workaround until 3.0 is to include the variable as a defaulted parameter in the enclosed function definition:
def f()
x = 5
def g(y, z, x=x):
x = x + 1
Related
I'm trying to make a little choose your own adventure game for fun and can't find a way to make this work. I want to be able to have a consistent pick up method where a specific action adds one to a variable and unlocks lines of dialogue under other actions. What I've got right now is something like this.
item = 0
while True:
act = input()
if "action" in act:
if(item != 1):
print("You don't have that item.")
continue
else:
print("You use the item.")
continue
if "take item" in act:
print("You take the item.")
global item
item = item + 1
continue
The issue I'm finding is that when I try to set the global variable, the program claims a syntax warning:
main.py:107: SyntaxWarning: name 'item' is assigned to before global declaration global item
since the variable is stated before the global, but if I don't state the variable before hand, the system I have in place to prevent the wrong dialogue from printing won't work. How do I have a conditional global variable that updates an existing local variable?
TLDR
You don't need the global keyword. Remove that line and your code will work.
Short Explanation
In Python, everything is "global" (not accurate, I will explain in long explanation) so you rarely need to use the global keyword. You can try the following code in the interactive Python REPL:
>>> def test(y):
... return x, y
...
>>> test(3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in test
NameError: name 'x' is not defined
>>> x = 4
>>> test(3)
(4, 3)
Here, I'm using a function as an example, if and while/for loops are exactly the same.
I'm declaring a function called test that takes in a parameter y, but it will return the tuple (x, y). Here, x is undefined. You would expect that it returns an error, but it actually doesn't until you run it.
When we run test(3), Python will complain that x is not defined so it cannot run the function. However, if we define x = 4 even after our function declaration, the function will work. Hence test(3) will return (4, 3). Here, x is treated as a global variable.
Long Explanation
>>> x = 3
>>> y = 4
>>> def test2():
... x = 1
... y = 2
... return x, y
...
>>> test2()
(1, 2)
>>> x
3
>>> y
4
>>>
Notice that y is not touched, even though it's clearly being assigned a different value in the function test2.
How is this possible? Actually, Python functions technically do have a scope of their own, and changes to function variables are isolated and do not change the variables of the same name outside the function. Loops and if statements don't have their own scope, so the variables inside if statements are the exact variables outside them.
>>> x = 3
>>> y = 4
>>> if True:
... x = 1
... y = 2
...
>>> x
1
>>> y
2
When you reference a variable in functions, Python will check if a value is assigned to a variable of that name in that function (either as a parameter or inside the function block). If there is a variable with that name, Python will use the variable in the function (within the scope of the function). If not, Python will try to look for a function that's in the scope outside (either another function or the global scope). If Python searched till the global scope and there is no variable of that name, it will finally produce an error.
>>> x = 3
>>> y = 4
>>> def function1():
... x = 4 # this x is being used
... def function2():
... print(x)
... function2()
...
>>> function1()
4
>>> def test3():
... x = 1
... y = 2
... return (x, y)
...
>>> test3()
(1, 2)
>>> x
3
>>> y
4
This means, only when you are trying to change the value of a variable outside of the current scope, you will use the global keyword. However, this is BAD PRACTICE. Whenever you find yourself doing this, you need to think about whether you should refactor your code to using classes. It's a very bad practice to change the value of global variables inside functions no matter the language you are using.
>>> x = 3
>>> y = 4
>>> def test4():
... global x, y # refers to the global x and y
... x = 1
... y = 2
... return (x, y)
...
>>> test4()
(1, 2)
>>> x
1
>>> y
2
I have tried creating a function like this:
n = ["one", "two"]
result=""
def join_strings(words):
for word in words:
result+=word
return result
print (join_strings(n))
This does not work and outputs:
UnboundLocalError: local variable 'result' referenced before assignment
The result variable has to be inside the function for it to work.
However, when I have this simple function:
x=3
def price():
return (x+3)
print (price())
This works and prints the price, even though x is outside of the function. Why is this happening?
Actually there is no inconsistency as the examples you gave are different from each other. It would still fail in the second function if you have tried to assign x to itself, like:
>>> x = 3
>>> def price():
... x +=3
... return x
...
>>> price()
UnboundLocalError: local variable 'x' referenced before assignment
Instead of assigning back to x if you choose another name, it would run with no problem:
>>> def price():
... y = x + 3
... return y
...
>>> price()
6
But, why it happens?
It's because Python's scoping rules. You can read the value of a variable outside of function but you can't change it**. When you do x += 3, which is same as x = x + 3 for integers, that means "I have a variable x that I have write access to it in the current scope." You don't have such variable, thus as the error says: you are referencing a "local variable" before assignment.
Is there a way to modify them in the function?
Yes. You can use global keyword, changes will be applied to your global variable:
>>> x = 3
>>> def price():
... global x
... x += 3
... return x
...
>>> x
3
>>> price()
6
>>> x
6
** By changing, I mean assigning something else to it so it's id will change.
The difference is that in the second example you aren't trying to modify or reassign x. (You'll get the same error if you say something like x += 3 in the price function.) Once you use an assignment operator, you're binding a new value to that name, shadowing the outer scope.
If you want to be able to modify something from an outer scope, put it inside a mutable container (like a list); you can then modify the contents without reassigning that variable.
result = [""]
def join_strings(words):
for word in words:
result[0] += word
return result[0]
If you just want to be able to reference the result value, without modifying it, that's fine too, but you have to assign it to a new variable then:
result = ""
def join_strings(words):
ret = result
for word in words:
ret += word
return ret
I am really new to programming and Python, so please really forgive me for my ignorance.
I just learned that using global can make a variable inside function be a global one.
However, I discovered something not with my expectation:
I tried the following code in Python 3.8 (forgive me for my ignorance as I don't know what else information I should provide):
>>> x = 0
>>>
>>> def function():
... if False:
... global x
... x = 1
...
>>> function()
>>> print(x)
and the result is 1.
However, I expected the code to have the same effect as the following code:
>>> x = 0
>>>
>>> def function():
... x = 1
...
>>> function()
>>> print(x)
which the result should be 0.
In my mind, the statement inside if False should not be executed, so it sounds strange to me.
Also, personally, I think that in some situation I would expect the variable inside a function, whether local or global, to be dependent on other codes... what I mean is, I would like to change if False to something like if A == 'A', while (I hope) I can control whether the x is global/local according to my conditional statement.
I tried to change if to while, but it's the same... there isn't a infinite loop, but the code global x is still executed/compiled...
I admit that it may sounds naive, and perhaps it just won't work in Python, but I really wonder why... It seems that the code global x is unreachable, but how come it is not ignored?
Can anyone please tell me about the reason? I would like to know more about the mechanism behind compilation(?)
Any help would be appreciated, thank you!
In python the global statement (and the nonlocal statement) are very different from the normal python code. Essentially no matter where a global statement in a function is, it influences always the current codeblock, and is never "executed". You should think more of it as a compiler directive instead of a command.
Note that the statement itself must come before any usage of the variable it modifies, i.e.
print(x)
global x
is a syntax error. The global statement can only modify variable behavior in the whole codeblock, you can't first have a non-global variable that later gets global and you can also not have conditional global variable
(I couldn't really find good documentation for this behavior, here it says "The global statement is a declaration which holds for the entire current code block." and "global is a directive to the parser. It applies only to code parsed at the same time as the global statement." but that doesn't seem super clear to me.)
There are more compiler directives in python, although they don't always look like one. One is the from __future__ import statements which look like module imports but change python behavior.
Global is not in execution path but in a scope. The scope is whole function. Statements like if for don't make scopes. If you use any assignment you create local variable. The same with global or nonlocal you bind symbol to variable from outside.
As Stanislas Morbieu typed, see doc.
Programmer’s note: global is a directive to the parser. It applies only to code parsed at the same time as the global statement.
Not at execution time.
x = 1
def fun():
y = x + 1
print(f'local x = {x}, y = {y}')
fun()
print(f'global x = {x}')
# Output:
# local x = 1, y = 2
# global x = 1
In example above, y uses global x (and adds 1).
x = 1
def fun():
y = x
x = y + 1
print(f'local x = {x}')
fun()
print(f'global x = {x}')
# Output:
# UnboundLocalError: local variable 'x' referenced before assignment
Look at last example. It doesn't assign y from global x because assignment in second line creates local x and y can not read local x before x assignment. The same with:
x = 1
def fun():
if False:
x += 1
fun()
# Output
# UnboundLocalError: local variable 'x' referenced before assignment
x assignment creates local variable.
If you want to change global variable under condition you can use globals().
x = 1
def set_x(do_set, value):
if do_set:
globals()['x'] = value
print(f'global x = {x} (init)')
set_x(False, 2)
print(f'global x = {x} (false)')
set_x(True, 3)
print(f'global x = {x} (true)')
# Output
# global x = 1 (init)
# global x = 1 (false)
# global x = 3 (true)
Proxy
I you want to decide with variable you want to use later (in the same scope) you need some kind of proxy IMO.
x = 1
def fun(use_global):
x = 2 # local
scope = globals() if use_global else locals()
scope['x'] += 1
print(f'local ({use_global}): x = {scope["x"]}')
print(f'global: x = {x} (init)')
fun(False)
print(f'global: x = {x} (false)')
fun(True)
print(f'global: x = {x} (true)')
# Output:
# global: x = 1 (init)
# local (False): x = 3
# global: x = 1 (false)
# local (True): x = 2
# global: x = 2 (true)
Maybe you can think about refactoring of your code if you need it.
If you can change local variable name (if not use globals() as above), you can proxy:
use dict (like in example above)
use list (x=[1]) and usage x[0]
use object (with builtin dict), example:
class X:
def __init__(self, x):
self.x = x
x = X(1)
def fun(use_global):
global x
my_x = x if use_global else X(2)
my_x.x += 1
print(f'local ({use_global}): x = {my_x.x}')
print(f'global: x = {x.x} (init)')
fun(False)
print(f'global: x = {x.x} (false)')
fun(True)
print(f'global: x = {x.x} (true)')
# Output:
# global: x = 1 (init)
# local (False): x = 3
# global: x = 1 (false)
# local (True): x = 2
# global: x = 2 (true)
Note. Variables in Python are only references. It is way you can not change x = 1 without global (or globals()). You change reference to local value 1.
But you can change z[0] or z['x'] or z.x. Because z referents to list or dict or object and you modify it content.
See: https://realpython.com/python-variables/#object-references
You can check real object by id() function, ex. print(id(x), id(my_x)).
As per the Python documentation, global is a directive to the parser so it is taken into account before the execution, therefore it does not matter if the code is reachable or not. The variable is global for the entire scope, which is the function in your case.
>>> x = 1
>>> def f():
... print x
...
>>> f()
1
>>> x = 1
>>> def f():
... x = 3
... print x
...
>>> f()
3
>>> x
1
>>> x = 1
>>> def f():
... print x
... x = 5
...
>>> f()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in f
UnboundLocalError: local variable 'x' referenced before assignment
>>> x = 1
>>> def f():
... global x
... print x
... x = 5
... print x
...
>>> f()
1
5
>>> x
5
How to treat the variable "x" inside the function as local without altering the global one when I have print statement above the variable assignment?
I expect the result of "x" to be 5 inside the function and the global x should be unaltered and remains the same in value (i.e) 1
I guess, there is no keyword called local in python contrary to global
>>> x = 1
>>> def f():
... print x
... global x
... x = 5
...
<stdin>:3: SyntaxWarning: name 'x' is used prior to global declaration
In Python, variables that are only referenced inside a function are implicitly global. If a variable is assigned a value anywhere within the function’s body, it’s assumed to be a local unless explicitly declared as global.
Source.
It's true there's no local keyword in Python; instead, Python has this rule to decide which variables are local.
Any variable in your function is either local or global. It can't be local in one part of the function and global in another. If you have a local variable x, then the function can't access the global x. If you want a local variable while accessing the global x, you can call the local variable some other name.
The behaviour is already what you want. The presence of x = inside the function body makes x a local variable which entirely shadows the outer variable. You're merely trying to print it before you assign any value to it, which is causing an error. This would cause an error under any other circumstance too; you can't print what you didn't assign.
I'm reading the python reference name resolution, which reads
A class definition is an executable statement that may use and define names. These references follow the normal rules for name resolution with an exception that unbound local variables are looked up in the global namespace.
Based on that, I would expect the following code
x = 10
def f():
x = 5
class Test:
y = x
return Test
print(f().y)
to print 10, however it prints 5. is this a mistake in the reference, or am I misunderstanding something?
In this case, 'normal' rules apply:
x = 'global'
def f():
x = 'defined in f'
class Test:
print(x) # not local, normal rules apply
f()
# defined in f
In this second case, we would expect an UnboundLocalError: local variable 'x' referenced before assignment if we were inside a function:
x = 'global'
def f():
x = 'defined in f'
class Test:
print(x) # unbound local at this time
x = 'assigned in Test'
print(x)
But for the first print(x), x will be taken from the global namespace:
f()
# global
# assigned in Test
I think #khelwood gave the answer. The value in the variable:
Test.y
is an integer that you define, but you never give the x = 10 to the function.
Maybe what you want is actually:
x = 10
def f(x=5):
class Test:
y = x
return Test
print(f().y) #print default 5
print(f(x).y)
The last line print 10 since x is given to the function, and therefore the class set y as x
You can see the exception by wrapping f in another function that defines a local variable x:
x = 10
def g():
x = 7
def f():
class Test:
y = x
return Test
return f()
print(g().y)
Now the output is 7, as the lookup of x in the class statement bypasses the local scope of g to find the global value of x.
In your code, because class establishes a new namespace, but not a new scope, the value of x is taken from the current local scope, that of f.