When I run this code, I get this result:
15
15
I expect the output should be
15
17
but it is not. The question is: why?
def make_adder_and_setter(x):
def setter(n):
x = n
return (lambda y: x + y, setter)
myadder, mysetter = make_adder_and_setter(5)
print myadder(10)
mysetter(7)
print myadder(10)
You are setting a local variable x in the setter() function. Assignment to a name in a function marks it as a local, unless you specifically tell the Python compiler otherwise.
In Python 3, you can explicitly mark x as non-local using the nonlocal keyword:
def make_adder_and_setter(x):
def setter(n):
nonlocal x
x = n
return (lambda y: x + y, setter)
Now x is marked as a free variable and looked up in the surrounding scope instead when assigned to.
In Python 2 you cannot mark a Python local as such. The only other option you have is marking x as a global. You'll have to resort to tricks where you alter values contained by a mutable object that lives in the surrounding scope.
An attribute on the setter function would work, for example; setter is local to the make_adder_and_setter() scope, attributes on that object would be visible to anything that has access to setter:
def make_adder_and_setter(x):
def setter(n):
setter.x = n
setter.x = x
return (lambda y: setter.x + y, setter)
Another trick is to use a mutable container, such as a list:
def make_adder_and_setter(x):
x = [x]
def setter(n):
x[0] = n
return (lambda y: x[0] + y, setter)
In both cases you are not assigning to a local name anymore; the first example uses attribute assignment on the setter object, the second alters the x list, not assign to x itself.
Python 2.x has a syntax limitation that doesn't allow to capture a variable in read/write.
The reason is that if a variable is assigned in a function there are only two possibilities:
the variable is a global and has been declared so with global x
the variable is a local of the function
more specifically it's ruled out that the variable is a local of an enclosing function scope
This has been superseded in Python 3.x with the addition of nonlocal declaration. Your code would work as expected in Python 3 by changing it to
def make_adder_and_setter(x):
def setter(n):
nonlocal x
x = n
return (lambda y: x + y, setter)
The python 2.x runtime is able to handle read-write closed over variable at a bytecode level, however the limitation is in the syntax that the compiler accepts.
You can see a lisp compiler that generates python bytecode directly that creates an adder closure with read-write captured state at the end of this video. The compiler can generate bytecode for Python 2.x, Python 3.x or PyPy.
If you need closed-over mutable state in Python 2.x a trick is to use a list:
def make_adder_and_setter(x):
x = [x]
def setter(n):
x[0] = n
return (lambda y: x[0] + y, setter)
Your inner def setter(n) function defines its own local variable x. That hides the other x variable that was a parameter of make_adder_and_setter (makes a hole in the scope). So the setter function has no side effect. It just sets the value of an inner local variable and exits.
Maybe it will be clear for you if you try the code below. It does exactly the same thing, just uses the name z instead of x.
def make_adder_and_setter(x):
def setter(n):
z = n
return (lambda y: x + y, setter)
myadder, mysetter = make_adder_and_setter(5)
print myadder(10)
mysetter(7)
print myadder(10)
Related
I'm currently learning about closure and this code was presented:
def outer_func(x):
y = 4
def inner_func(z):
print(f"x = {x}, y = {y}, z = {z}")
return x + y + z
return inner_func
for i in range(3):
closure = outer_func(i)
print(f"closure({i+5}) = {closure(i+5)}")
I understand x is defined at the point outer_func is assigned to closure, and y is defined within the function each time. My question is how is z defined? Shouldn't the call to closure overwrite the value of x? How does python know to assign this new value to z?
My question is how is z defined?
z is a parameter of inner_func. It means it's always local to the inner_func function(local variable). When the outer_func function is called the body of it will be executed, which indeed first create inner_func and give you back a reference to it. Then whatever you pass to inner_func, it uses as the z. It's a normal argument passing.
Shouldn't the call to closure overwrite the value of x?
inner_func is your closure. It has access to it's enclosing scope which is outer_func. x is a parameter of outer_func, it gets it's value from executing the line:
closure = outer_func(i)
So whatever you pass to outer_func becomes available to the inner_func's body. (Closure mechanism)
How does python know to assign this new value to z?
I already answered this in the first part. In the line:
print(f"closure({i+5}) = {closure(i+5)}")
you are passing the value of z in closure(i+5) part.
Q: Now where does inner_func get y's value from? A: Again it's a closure, it has access to outer_func namespace. y is always defined when outer_func() is called. Nothing can change it's value in your code. That loop can only change x because it comes from outside. (argument)
Your question is:
I understand x is defined at the point outer_func is assigned to closure, and y is defined within the function each time. My question is how is z defined? Shouldn't the call to closure overwrite the value of x? How does python know to assign this new value to z?
Your code is:
def outer_func(x):
y = 4
def inner_func(z):
print(f"x = {x}, y = {y}, z = {z}")
return x + y + z
return inner_func
for i in range(3):
closure = outer_func(i)
print(f"closure({i+5}) = {closure(i+5)}")
The call to closure is effectively a call to inner_func (not outer_func) and the argument passed to closure will be assigned to z (not x).
Explanation:
Each execution of closure = outer_func(i) does the following:
call outer_func() thereby effectively assigning the passed argument i to the variable x in the scope of outer_func()
execute y = 4
leave outer_func() with a return value of type function whose value is inner_func
assign the return value of outer_func() (namely the function inner_func) to closure, with ongoing access to the variables in the containing scope of inner_func() (namely, the scope of outer_func()).
Each call to closure() within the for loop does the following:
call inner_func() thereby assigning the passed argument to z in the scope of inner_func()
execute the body of inner_func() where x and y have the values they had at the time of the call to outer_func() that created this copy of inner_func as a closure.
This question already has an answer here:
Where is nonlocals()?
(1 answer)
Closed 5 months ago.
I have a local variable x = "local" which unfortunately shares its name with both a global and a non-local variable. Without changing any of the names, can I access all three values? For x = "global" there is globals(), but what about the non-local variable?
Minimal example which illustrates the issue:
x = "global"
def f(x="nonlocal"):
def g():
x = "local"
print(x) # same as locals()["x"]
print(globals()["x"])
# here I want to print the non-local x
return g
f()()
I don't get your context that you have to use same name.
Anyway, you can capture outer function's locals as nonlocal variable.
x = "global"
def f(x="nonlocal"):
nonlocals = locals()
def g():
x = "local"
print(x)
print(nonlocals['x'])
print(globals()["x"])
return g
f()()
output:
local
nonlocal
global
Though you couldn't do this with the code written exactly as given, you can use inspect to get non-local variables. Note the changes to the call and return. If the caller is the outer scope instead of global scope, the previous frame will be f.
import inspect
x = "global"
def f(x="nonlocal"):
def g():
x = "local"
print(x)
print(globals()["x"])
print(inspect.currentframe().f_back.f_locals["x"])
return g()
f()
Output
local
global
nonlocal
This might not help in this specific situation, it really depends on how much control you have over the contents of f. If you don't have control over that, you can also monkey-patch f. A lot depends on context.
Edit: I didn't notice that the question specifically asked for this without using nonlocal. Leaving this here in case others find it useful.
I question the rationale behind this, but in Python 3, you can use the nonlocal keyword to access the previous scope, store that before re-declaration, then get it later.
x = "global"
def f(x="nonlocal"):
def g():
nonlocal x
y = x
x = "local"
print(x) # same as locals()["x"]
print(globals()["x"])
print(y)
return g
f()()
Output
local
global
nonlocal
here is my code:
def f(x):
def g(n):
if n < 10:
x = x + 1
g(n + 1)
g(0)
When I evaluate f(0), there would be an error "x referenced before assignment".
However, when I use "print x" instead of "x = x + 1" , it will work.
It seems that in the scope of g, I can only use x as an "use occurrence" but not a "binding occurrence". I guess the problem is that f passes to g only the VALUE of x.
Am I understanding it correctly or not? If not, can someone explain why the left side of "x = x + 1" is not defined before reference?
Thanks
You are understanding it correctly. You cannot use x to assign to in a nested scope in Python 2.
In Python 3, you can still use it as a binding occurrence by marking the variable as nonlocal; this is a keyword introduced for just this usecase:
def f(x):
def g(n):
nonlocal x
if n < 10:
x = x + 1
g(n + 1)
g(0)
In python 2, you have a few work-arounds; using a mutable to avoid needing to bind it, or (ab)using a function property:
def f(x):
x = [x] # lists are mutable
def g(n):
if n < 10:
x[0] = x[0] + 1 # not assigning, but mutating (x.__setitem__(0, newvalue))
g(n + 1)
g(0)
or
def f(x):
def g(n):
if n < 10:
g.x = g.x + 1
g(n + 1)
g.x = x # attribute on the function!
g(0)
Yes, assigning to names is different than reading their values. Any names that are assigned to in a function are considered local variables of that function unless you specify otherwise.
In Python 2, the only way to "specify otherwise" is to use a global statement to allow you assign to a global variable. In Python 3, you also have the nonlocal statement to assign a value to a variable in a higher (but not necessarily global) scope.
In your example, x is in a higher but non-global scope (the function f). So there is no way to assign to x from within g in Python 2. In Python 3 you could do it with a nonlocal x statement in g.
In python what is the equivalent of the quote operator? I am finding the need to delay evaluation. For example, suppose in the following lisp psuedocode I have:
a = '(func, 'g)
g = something
(eval a)
What I am doing is deferring evaluation of g till a later time. This is necessary because I want to define g later. What is the equivalent idea of this psuedocode in python?
a = lambda: func(g)
g = something
a()
This isn't quite the most literal translation - the most literal translation would use a string and eval - but it's probably the best fit. Quoting probably isn't what you wanted in Lisp anyway; you probably wanted to delay something or create a lambda. Note that func and g are closure variables in the lambda function, rather than symbols, so if you call a from an environment with different bindings for func or g, it'll still use the variables from a's environment of definition.
Using eval for delaying evaluation is bad, both in Lisp and Python.
in Python, and in Lisp, you can delay evaluation using a closure:
def print_it(x):
def f():
print g(x)
return f
f = print_it(42)
def g(x):
return x * x
f()
Please note that what is captured in a closure is not the value of a variable, but the variable itself and this is sometimes surprising:
fa = []
for x in range(10):
def g():
print x
fa.append(g)
for f in fa:
f() # all of them will print 9
x = 42
fa[0]() # the output will be 42
to solve this problem (that can also be present in Common Lisp) you may see things like:
for x in range(10):
def g(x = x):
print x
fa.append(g)
or (in CL) things like
(let ((a a))
(lambda () (print a)))
Python also has a lambda special form for anonymous functions, but they are limited to one single expression and cannot contain any statement. A locally def-ined function instead is a regular function without any limitations.
for x in range(10):
# print is a statement in Python 2.x and cannot be in a lambda
fa.append(lambda x=x: sys.stdout.write(str(x) + "\n"))
Finally Python 2.x has a syntax limitation and closed-over variables are read-only because if there is an assignment (or augmented-assignment) in a function there are only two possibilities:
The variable is a global (and has been previously declared so with global x)
The variable is a local
and in particular it's ruled out that a variable being assigned could be a local of an enclosing function scope.
Python 3.x removed this limitation by providing a new possible declaration nonlocal x and now the famous adder example can be implemented as
def adder(x):
def f(y):
nonlocal x
x += y
return x
return f
I'm wondering if it's possible for a closure in Python to manipulate variables in its namespace. You might call this side-effects because the state is being changed outside the closure itself. I'd like to do something like this
def closureMaker():
x = 0
def closure():
x+=1
print x
return closure
a = closureMaker()
a()
1
a()
2
Obviously what I hope to do is more complicated, but this example illustrates what I'm talking about.
You can't do exactly that in Python 2.x, but you can use a trick to get the same effect: use a mutable object such as a list.
def closureMaker():
x = [0]
def closure():
x[0] += 1
print x[0]
return closure
You can also make x an object with a named attribute, or a dictionary. This can be more readable than a list, especially if you have more than one such variable to modify.
In Python 3.x, you just need to add nonlocal x to your inner function. This causes assignments to x to go to the outer scope.
What limitations have closures in Python compared to language X closures?
nonlocal keyword in Python 2.x
Example:
def closureMaker():
x = 0
def closure():
nonlocal x
x += 1
print(x)
return closure