I'm wondering if it's possible for a closure in Python to manipulate variables in its namespace. You might call this side-effects because the state is being changed outside the closure itself. I'd like to do something like this
def closureMaker():
x = 0
def closure():
x+=1
print x
return closure
a = closureMaker()
a()
1
a()
2
Obviously what I hope to do is more complicated, but this example illustrates what I'm talking about.
You can't do exactly that in Python 2.x, but you can use a trick to get the same effect: use a mutable object such as a list.
def closureMaker():
x = [0]
def closure():
x[0] += 1
print x[0]
return closure
You can also make x an object with a named attribute, or a dictionary. This can be more readable than a list, especially if you have more than one such variable to modify.
In Python 3.x, you just need to add nonlocal x to your inner function. This causes assignments to x to go to the outer scope.
What limitations have closures in Python compared to language X closures?
nonlocal keyword in Python 2.x
Example:
def closureMaker():
x = 0
def closure():
nonlocal x
x += 1
print(x)
return closure
Related
Python
Can anyone help me to understand this code, I am new to Python, how does this function work?
def makeInc(x):
def inc(y):
return y + x
return inc
incOne = makeInc(1)
incFive = makeInc(5)
print(incOne(5)) # returns 6
print(incFive(5)) # returns 10
Higher-order functions
Functions like makeInc that in turn, return another function are called higher order functions. Usually, functions are known to accept data as input and return data as output. With higher order functions, functions instead of data, either return code as output or accept code as input. This code is wrapped into a function. In Python, functions are first class citizens which means functions, just like data, can be passed around. For instance:
myvariable = print
Notice, how I have assigned print to myvariable and how I have dropped the parentheses after print Functions without parentheses are called function objects. This means myvariable now is just another name for print:
print("Hello World!")
myvariable("Hello World!")
Both of the above statements do the exact same thing. What can be assigned to variables can also be returned from functions:
def myfunction():
return print
myfunction()("Hello World!");
Now let's look at your example:
def makeInc(x):
def inc(y):
return y + x
return inc
makeInc is a function that accepts a parameter called x. It then defines another nested inner function called inc which takes in a parameter called y. The thing about nested functions is that they have access to the variables of the enclosing function as well. Here, inc is the inner function but it has access to x which is a variable of the enclosing outer scope.
The last statement return inc returns the inner function to the caller of makeInc. What makeInc essentially is doing, is creating a custom function based on the parameter it receives.
For instance:
x = makeInc(10)
makeInc will first accept 10 and then return a function that takes in an argument y and it increments y by 10.
Here, x is a function that takes in any argument y and then increments it by 10:
x(42) # Returns 52
nonlocal
However, there is a caveat when using nested functions:
def outer():
x = 10
def inner():
x = 20
inner()
print(x) # prints 10
Here, you would assume that the last print statement will print 20. But no! When you assign x = 20 in the inner function, it creates a new local variable called x which is initialized to 20. The outer x remains untouched. To modify the outer x, use the nonlocal keyword:
def outer():
x = 10
def inner():
nonlocal x = 20
inner()
print(x) # prints 20
If you are directly reading x inside inner() instead of assigning to it, you do not need nonlocal.
What is happening here is that makeInc() returns a function handle pointing to specific implementation of inc(). So, calling makeInc(5) "replaces" the x in inc(y) to 5 and returns the callable handle of that function. This handle is saved in incFive. You can now call the function as defined (inc(y)). Since you set x=5 before, the result will be y+5.
This question already has an answer here:
Where is nonlocals()?
(1 answer)
Closed 5 months ago.
I have a local variable x = "local" which unfortunately shares its name with both a global and a non-local variable. Without changing any of the names, can I access all three values? For x = "global" there is globals(), but what about the non-local variable?
Minimal example which illustrates the issue:
x = "global"
def f(x="nonlocal"):
def g():
x = "local"
print(x) # same as locals()["x"]
print(globals()["x"])
# here I want to print the non-local x
return g
f()()
I don't get your context that you have to use same name.
Anyway, you can capture outer function's locals as nonlocal variable.
x = "global"
def f(x="nonlocal"):
nonlocals = locals()
def g():
x = "local"
print(x)
print(nonlocals['x'])
print(globals()["x"])
return g
f()()
output:
local
nonlocal
global
Though you couldn't do this with the code written exactly as given, you can use inspect to get non-local variables. Note the changes to the call and return. If the caller is the outer scope instead of global scope, the previous frame will be f.
import inspect
x = "global"
def f(x="nonlocal"):
def g():
x = "local"
print(x)
print(globals()["x"])
print(inspect.currentframe().f_back.f_locals["x"])
return g()
f()
Output
local
global
nonlocal
This might not help in this specific situation, it really depends on how much control you have over the contents of f. If you don't have control over that, you can also monkey-patch f. A lot depends on context.
Edit: I didn't notice that the question specifically asked for this without using nonlocal. Leaving this here in case others find it useful.
I question the rationale behind this, but in Python 3, you can use the nonlocal keyword to access the previous scope, store that before re-declaration, then get it later.
x = "global"
def f(x="nonlocal"):
def g():
nonlocal x
y = x
x = "local"
print(x) # same as locals()["x"]
print(globals()["x"])
print(y)
return g
f()()
Output
local
global
nonlocal
This question already has answers here:
UnboundLocalError trying to use a variable (supposed to be global) that is (re)assigned (even after first use)
(14 answers)
Closed 6 months ago.
def some_func(a):
def access_a():
print(a)
access_a()
outputs the value of a. However, if I want to change a in the nested function like this:
def some_func(a):
def change_a():
a += 1
print(a)
change_a()
it raises UnboundLocalError exception.
I know a is a nonlocal variable, but why can I access it without declaring nonlocal a?
Python scoping rules 101:
a name bound in a function body is considered local unless explicitely declared global (Python 2.x and 3.x) or nonlocal (Python 3.x only). This holds true whereever the assignment happens in the function's body. Trying to read a local variable before it's bound is of course an error.
if a name is read but not bound in a function's body, it will be looked up in enclosing scopes (outer function(s) if any then global scope). NB: functions arguments are de facto local names so they will never be looked up in enclosing scopes.
Note that a += 1 is mainly a shortcut for a = a + 1, so in your example a is local (bound in the function's body and not explicitely declared global or nonlocal), but you try to read it (the rhs of a = a+1) before it's bound.
In Python 3 you can solve this with a nonlocal statement:
>>> def outer(a):
... def change():
... nonlocal a
... a += 1
... print("before : {}".format(a))
... change()
... print ("after : {}".format(a))
...
>>> outer(42)
before : 42
after : 43
Python 2 doesn't have nonlocal so the canonical hack is to wrap the variable in a mutable container (typically a list but any mutable object would do):
>>> def outer(a):
... _a = [a]
... def change():
... _a[0] += 1
... print("before : {}".format(_a[0]))
... change()
... print ("after : {}".format(_a[0]))
...
>>> outer(42)
before : 42
after : 43
which is quite ugly to say the least.
Now while closures are quite handy, they are mostly the functional counterpart of objects : a way to share state between a set of functions while preserving encapsulation of this state, so if you find you have a need for a nonlocal variable perhaps a proper class might be a cleaner solution (though possibly not for your example that doesn't return the inner function but only uses it internally).
i have two solutions for you:
#first one:
# try with list, compound data types dict/list
def some_func(a):
def change_a():
a[0] += 1
print(a[0])
change_a()
some_func([1])
>>> 2
#second one
#reference pointer
from ctypes import *
def some_func_ctypes(a):
def change_a():
a[0] += 1
print a.contents, a[0]
change_a()
i = c_int(1)
pi = pointer(i)
some_func_ctypes(pi)
>>> c_int(2) 2
When you use the += operator, there is an assignment of a new value to a. This turns a to a local in the eyes of the interpreter.
In python what is the equivalent of the quote operator? I am finding the need to delay evaluation. For example, suppose in the following lisp psuedocode I have:
a = '(func, 'g)
g = something
(eval a)
What I am doing is deferring evaluation of g till a later time. This is necessary because I want to define g later. What is the equivalent idea of this psuedocode in python?
a = lambda: func(g)
g = something
a()
This isn't quite the most literal translation - the most literal translation would use a string and eval - but it's probably the best fit. Quoting probably isn't what you wanted in Lisp anyway; you probably wanted to delay something or create a lambda. Note that func and g are closure variables in the lambda function, rather than symbols, so if you call a from an environment with different bindings for func or g, it'll still use the variables from a's environment of definition.
Using eval for delaying evaluation is bad, both in Lisp and Python.
in Python, and in Lisp, you can delay evaluation using a closure:
def print_it(x):
def f():
print g(x)
return f
f = print_it(42)
def g(x):
return x * x
f()
Please note that what is captured in a closure is not the value of a variable, but the variable itself and this is sometimes surprising:
fa = []
for x in range(10):
def g():
print x
fa.append(g)
for f in fa:
f() # all of them will print 9
x = 42
fa[0]() # the output will be 42
to solve this problem (that can also be present in Common Lisp) you may see things like:
for x in range(10):
def g(x = x):
print x
fa.append(g)
or (in CL) things like
(let ((a a))
(lambda () (print a)))
Python also has a lambda special form for anonymous functions, but they are limited to one single expression and cannot contain any statement. A locally def-ined function instead is a regular function without any limitations.
for x in range(10):
# print is a statement in Python 2.x and cannot be in a lambda
fa.append(lambda x=x: sys.stdout.write(str(x) + "\n"))
Finally Python 2.x has a syntax limitation and closed-over variables are read-only because if there is an assignment (or augmented-assignment) in a function there are only two possibilities:
The variable is a global (and has been previously declared so with global x)
The variable is a local
and in particular it's ruled out that a variable being assigned could be a local of an enclosing function scope.
Python 3.x removed this limitation by providing a new possible declaration nonlocal x and now the famous adder example can be implemented as
def adder(x):
def f(y):
nonlocal x
x += y
return x
return f
When I run this code, I get this result:
15
15
I expect the output should be
15
17
but it is not. The question is: why?
def make_adder_and_setter(x):
def setter(n):
x = n
return (lambda y: x + y, setter)
myadder, mysetter = make_adder_and_setter(5)
print myadder(10)
mysetter(7)
print myadder(10)
You are setting a local variable x in the setter() function. Assignment to a name in a function marks it as a local, unless you specifically tell the Python compiler otherwise.
In Python 3, you can explicitly mark x as non-local using the nonlocal keyword:
def make_adder_and_setter(x):
def setter(n):
nonlocal x
x = n
return (lambda y: x + y, setter)
Now x is marked as a free variable and looked up in the surrounding scope instead when assigned to.
In Python 2 you cannot mark a Python local as such. The only other option you have is marking x as a global. You'll have to resort to tricks where you alter values contained by a mutable object that lives in the surrounding scope.
An attribute on the setter function would work, for example; setter is local to the make_adder_and_setter() scope, attributes on that object would be visible to anything that has access to setter:
def make_adder_and_setter(x):
def setter(n):
setter.x = n
setter.x = x
return (lambda y: setter.x + y, setter)
Another trick is to use a mutable container, such as a list:
def make_adder_and_setter(x):
x = [x]
def setter(n):
x[0] = n
return (lambda y: x[0] + y, setter)
In both cases you are not assigning to a local name anymore; the first example uses attribute assignment on the setter object, the second alters the x list, not assign to x itself.
Python 2.x has a syntax limitation that doesn't allow to capture a variable in read/write.
The reason is that if a variable is assigned in a function there are only two possibilities:
the variable is a global and has been declared so with global x
the variable is a local of the function
more specifically it's ruled out that the variable is a local of an enclosing function scope
This has been superseded in Python 3.x with the addition of nonlocal declaration. Your code would work as expected in Python 3 by changing it to
def make_adder_and_setter(x):
def setter(n):
nonlocal x
x = n
return (lambda y: x + y, setter)
The python 2.x runtime is able to handle read-write closed over variable at a bytecode level, however the limitation is in the syntax that the compiler accepts.
You can see a lisp compiler that generates python bytecode directly that creates an adder closure with read-write captured state at the end of this video. The compiler can generate bytecode for Python 2.x, Python 3.x or PyPy.
If you need closed-over mutable state in Python 2.x a trick is to use a list:
def make_adder_and_setter(x):
x = [x]
def setter(n):
x[0] = n
return (lambda y: x[0] + y, setter)
Your inner def setter(n) function defines its own local variable x. That hides the other x variable that was a parameter of make_adder_and_setter (makes a hole in the scope). So the setter function has no side effect. It just sets the value of an inner local variable and exits.
Maybe it will be clear for you if you try the code below. It does exactly the same thing, just uses the name z instead of x.
def make_adder_and_setter(x):
def setter(n):
z = n
return (lambda y: x + y, setter)
myadder, mysetter = make_adder_and_setter(5)
print myadder(10)
mysetter(7)
print myadder(10)