Consider the following function, which we'd like to not be constant on integers a, but which always returns (1,2):
def foo(a):
b = 1
c = 2
def bar(b,c):
b = b + a
c = c + a
bar(b,c)
return (b,c)
If I understand correctly, the idiomatic way to implement bar(b,c) would be to give it no parameters at all and declare b and c nonlocal inside its definition. I'm curious, however: how do we make an inner function have nonlocal effects on its parameters?
As stated in this answer for Python 2.x:
Python doesn't allow you to reassign the value of a variable from an
outer scope in an inner scope (unless you're using the keyword
"global", which doesn't apply in this case).
This will return (2,3):
def foo(a):
b = 1
c = 2
def bar(b,c):
b = b + a
c = c + a
return b, c
b, c = bar(b,c)
return (b,c)
print(foo(1))
Make b and c function attributes.
def foo(a):
foo.b = 1
foo.c = 2
def bar():
foo.b = foo.b + a
foo.c = foo.c + a
bar()
return (foo.b,foo.c)
Notice you no longer pass b or c into the function bar.
Function parameters are always locals. You could pass in mutable objects however, and apply your operations to the indirectly referenced value:
def foo(a):
b = [1]
c = [2]
def bar(b, c):
b[0] += a
c[0] += a
bar(b, c)
return (b[0], c[0])
The changes you make to the mutable object are shared with any other references to those objects, including locals in foo().
However, if you want something to be a closure, just make it a closure. There are no use cases for this that cannot be handled by nonlocal values and return.
Related
Consider this example:
def A():
b = 1
def B():
# I can access 'b' from here.
print(b)
# But can i modify 'b' here?
B()
A()
For the code in the B function, the variable b is in a non-global, enclosing (outer) scope. How can I modify b from within B? I get an UnboundLocalError if I try it directly, and using global does not fix the problem since b is not global.
Python implements lexical, not dynamic scope - like almost all modern languages. The techniques here will not allow access to the caller's variables - unless the caller also happens to be an enclosing function - because the caller is not in scope. For more on this problem, see How can I access variables from the caller, even if it isn't an enclosing scope (i.e., implement dynamic scoping)?.
On Python 3, use the nonlocal keyword:
The nonlocal statement causes the listed identifiers to refer to previously bound variables in the nearest enclosing scope excluding globals. This is important because the default behavior for binding is to search the local namespace first. The statement allows encapsulated code to rebind variables outside of the local scope besides the global (module) scope.
def foo():
a = 1
def bar():
nonlocal a
a = 2
bar()
print(a) # Output: 2
On Python 2, use a mutable object (like a list, or dict) and mutate the value instead of reassigning a variable:
def foo():
a = []
def bar():
a.append(1)
bar()
bar()
print a
foo()
Outputs:
[1, 1]
You can use an empty class to hold a temporary scope. It's like the mutable but a bit prettier.
def outer_fn():
class FnScope:
b = 5
c = 6
def inner_fn():
FnScope.b += 1
FnScope.c += FnScope.b
inner_fn()
inner_fn()
inner_fn()
This yields the following interactive output:
>>> outer_fn()
8 27
>>> fs = FnScope()
NameError: name 'FnScope' is not defined
I'm a little new to Python, but I've read a bit about this. I believe the best you're going to get is similar to the Java work-around, which is to wrap your outer variable in a list.
def A():
b = [1]
def B():
b[0] = 2
B()
print(b[0])
# The output is '2'
Edit: I guess this was probably true before Python 3. Looks like nonlocal is your answer.
No you cannot, at least in this way.
Because the "set operation" will create a new name in the current scope, which covers the outer one.
I don't know if there is an attribute of a function that gives the __dict__ of the outer space of the function when this outer space isn't the global space == the module, which is the case when the function is a nested function, in Python 3.
But in Python 2, as far as I know, there isn't such an attribute.
So the only possibilities to do what you want is:
1) using a mutable object, as said by others
2)
def A() :
b = 1
print 'b before B() ==', b
def B() :
b = 10
print 'b ==', b
return b
b = B()
print 'b after B() ==', b
A()
result
b before B() == 1
b == 10
b after B() == 10
.
Nota
The solution of Cédric Julien has a drawback:
def A() :
global b # N1
b = 1
print ' b in function B before executing C() :', b
def B() :
global b # N2
print ' b in function B before assigning b = 2 :', b
b = 2
print ' b in function B after assigning b = 2 :', b
B()
print ' b in function A , after execution of B()', b
b = 450
print 'global b , before execution of A() :', b
A()
print 'global b , after execution of A() :', b
result
global b , before execution of A() : 450
b in function B before executing B() : 1
b in function B before assigning b = 2 : 1
b in function B after assigning b = 2 : 2
b in function A , after execution of B() 2
global b , after execution of A() : 2
The global b after execution of A() has been modified and it may be not whished so
That's the case only if there is an object with identifier b in the global namespace
The short answer that will just work automagically
I created a python library for solving this specific problem. It is released under the unlisence so use it however you wish. You can install it with pip install seapie or check out the home page here https://github.com/hirsimaki-markus/SEAPIE
user#pc:home$ pip install seapie
from seapie import Seapie as seapie
def A():
b = 1
def B():
seapie(1, "b=2")
print(b)
B()
A()
outputs
2
the arguments have following meaning:
The first argument is execution scope. 0 would mean local B(), 1 means parent A() and 2 would mean grandparent <module> aka global
The second argument is a string or code object you want to execute in the given scope
You can also call it without arguments for interactive shell inside your program
The long answer
This is more complicated. Seapie works by editing the frames in call stack using CPython api. CPython is the de facto standard so most people don't have to worry about it.
The magic words you are probably most likely interesed in if you are reading this are the following:
frame = sys._getframe(1) # 1 stands for previous frame
parent_locals = frame.f_locals # true dictionary of parent locals
parent_globals = frame.f_globals # true dictionary of parent globals
exec(codeblock, parent_globals, parent_locals)
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),ctypes.c_int(1))
# the magic value 1 stands for ability to introduce new variables. 0 for update-only
The latter will force updates to pass into local scope. local scopes are however optimized differently than global scope so intoducing new objects has some problems when you try to call them directly if they are not initialized in any way. I will copy few ways to circumvent these problems from the github page
Assingn, import and define your objects beforehand
Assingn placeholder to your objects beforehand
Reassign object to itself in main program to update symbol table: x = locals()["x"]
Use exec() in main program instead of directly calling to avoid optimization. Instead of calling x do: exec("x")
If you are feeling that using exec() is not something you want to go with you can
emulate the behaviour by updating the the true local dictionary (not the one returned by locals()). I will copy an example from https://faster-cpython.readthedocs.io/mutable.html
import sys
import ctypes
def hack():
# Get the frame object of the caller
frame = sys._getframe(1)
frame.f_locals['x'] = "hack!"
# Force an update of locals array from locals dict
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),
ctypes.c_int(0))
def func():
x = 1
hack()
print(x)
func()
Output:
hack!
I don't think you should want to do this. Functions that can alter things in their enclosing context are dangerous, as that context may be written without the knowledge of the function.
You could make it explicit, either by making B a public method and C a private method in a class (the best way probably); or by using a mutable type such as a list and passing it explicitly to C:
def A():
x = [0]
def B(var):
var[0] = 1
B(x)
print x
A()
For anyone looking at this much later on a safer but heavier workaround is. Without a need to pass variables as parameters.
def outer():
a = [1]
def inner(a=a):
a[0] += 1
inner()
return a[0]
You can, but you'll have to use the global statment (not a really good solution as always when using global variables, but it works):
def A():
global b
b = 1
def B():
global b
print( b )
b = 2
B()
A()
Consider this example:
def A():
b = 1
def B():
# I can access 'b' from here.
print(b)
# But can i modify 'b' here?
B()
A()
For the code in the B function, the variable b is in a non-global, enclosing (outer) scope. How can I modify b from within B? I get an UnboundLocalError if I try it directly, and using global does not fix the problem since b is not global.
Python implements lexical, not dynamic scope - like almost all modern languages. The techniques here will not allow access to the caller's variables - unless the caller also happens to be an enclosing function - because the caller is not in scope. For more on this problem, see How can I access variables from the caller, even if it isn't an enclosing scope (i.e., implement dynamic scoping)?.
On Python 3, use the nonlocal keyword:
The nonlocal statement causes the listed identifiers to refer to previously bound variables in the nearest enclosing scope excluding globals. This is important because the default behavior for binding is to search the local namespace first. The statement allows encapsulated code to rebind variables outside of the local scope besides the global (module) scope.
def foo():
a = 1
def bar():
nonlocal a
a = 2
bar()
print(a) # Output: 2
On Python 2, use a mutable object (like a list, or dict) and mutate the value instead of reassigning a variable:
def foo():
a = []
def bar():
a.append(1)
bar()
bar()
print a
foo()
Outputs:
[1, 1]
You can use an empty class to hold a temporary scope. It's like the mutable but a bit prettier.
def outer_fn():
class FnScope:
b = 5
c = 6
def inner_fn():
FnScope.b += 1
FnScope.c += FnScope.b
inner_fn()
inner_fn()
inner_fn()
This yields the following interactive output:
>>> outer_fn()
8 27
>>> fs = FnScope()
NameError: name 'FnScope' is not defined
I'm a little new to Python, but I've read a bit about this. I believe the best you're going to get is similar to the Java work-around, which is to wrap your outer variable in a list.
def A():
b = [1]
def B():
b[0] = 2
B()
print(b[0])
# The output is '2'
Edit: I guess this was probably true before Python 3. Looks like nonlocal is your answer.
No you cannot, at least in this way.
Because the "set operation" will create a new name in the current scope, which covers the outer one.
I don't know if there is an attribute of a function that gives the __dict__ of the outer space of the function when this outer space isn't the global space == the module, which is the case when the function is a nested function, in Python 3.
But in Python 2, as far as I know, there isn't such an attribute.
So the only possibilities to do what you want is:
1) using a mutable object, as said by others
2)
def A() :
b = 1
print 'b before B() ==', b
def B() :
b = 10
print 'b ==', b
return b
b = B()
print 'b after B() ==', b
A()
result
b before B() == 1
b == 10
b after B() == 10
.
Nota
The solution of Cédric Julien has a drawback:
def A() :
global b # N1
b = 1
print ' b in function B before executing C() :', b
def B() :
global b # N2
print ' b in function B before assigning b = 2 :', b
b = 2
print ' b in function B after assigning b = 2 :', b
B()
print ' b in function A , after execution of B()', b
b = 450
print 'global b , before execution of A() :', b
A()
print 'global b , after execution of A() :', b
result
global b , before execution of A() : 450
b in function B before executing B() : 1
b in function B before assigning b = 2 : 1
b in function B after assigning b = 2 : 2
b in function A , after execution of B() 2
global b , after execution of A() : 2
The global b after execution of A() has been modified and it may be not whished so
That's the case only if there is an object with identifier b in the global namespace
The short answer that will just work automagically
I created a python library for solving this specific problem. It is released under the unlisence so use it however you wish. You can install it with pip install seapie or check out the home page here https://github.com/hirsimaki-markus/SEAPIE
user#pc:home$ pip install seapie
from seapie import Seapie as seapie
def A():
b = 1
def B():
seapie(1, "b=2")
print(b)
B()
A()
outputs
2
the arguments have following meaning:
The first argument is execution scope. 0 would mean local B(), 1 means parent A() and 2 would mean grandparent <module> aka global
The second argument is a string or code object you want to execute in the given scope
You can also call it without arguments for interactive shell inside your program
The long answer
This is more complicated. Seapie works by editing the frames in call stack using CPython api. CPython is the de facto standard so most people don't have to worry about it.
The magic words you are probably most likely interesed in if you are reading this are the following:
frame = sys._getframe(1) # 1 stands for previous frame
parent_locals = frame.f_locals # true dictionary of parent locals
parent_globals = frame.f_globals # true dictionary of parent globals
exec(codeblock, parent_globals, parent_locals)
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),ctypes.c_int(1))
# the magic value 1 stands for ability to introduce new variables. 0 for update-only
The latter will force updates to pass into local scope. local scopes are however optimized differently than global scope so intoducing new objects has some problems when you try to call them directly if they are not initialized in any way. I will copy few ways to circumvent these problems from the github page
Assingn, import and define your objects beforehand
Assingn placeholder to your objects beforehand
Reassign object to itself in main program to update symbol table: x = locals()["x"]
Use exec() in main program instead of directly calling to avoid optimization. Instead of calling x do: exec("x")
If you are feeling that using exec() is not something you want to go with you can
emulate the behaviour by updating the the true local dictionary (not the one returned by locals()). I will copy an example from https://faster-cpython.readthedocs.io/mutable.html
import sys
import ctypes
def hack():
# Get the frame object of the caller
frame = sys._getframe(1)
frame.f_locals['x'] = "hack!"
# Force an update of locals array from locals dict
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),
ctypes.c_int(0))
def func():
x = 1
hack()
print(x)
func()
Output:
hack!
I don't think you should want to do this. Functions that can alter things in their enclosing context are dangerous, as that context may be written without the knowledge of the function.
You could make it explicit, either by making B a public method and C a private method in a class (the best way probably); or by using a mutable type such as a list and passing it explicitly to C:
def A():
x = [0]
def B(var):
var[0] = 1
B(x)
print x
A()
For anyone looking at this much later on a safer but heavier workaround is. Without a need to pass variables as parameters.
def outer():
a = [1]
def inner(a=a):
a[0] += 1
inner()
return a[0]
You can, but you'll have to use the global statment (not a really good solution as always when using global variables, but it works):
def A():
global b
b = 1
def B():
global b
print( b )
b = 2
B()
A()
I have some function which uses outside variables. A (substantially) simplified example:
a = 2
b = 3
def f(x):
return x * a + b
While I need a and b in f, I don't need them anywhere else. In particular, one can write a = 5, and that will change the behavior of f. How should I make a and b invisible to the outside?
Other languages allow me to write roughly the following code:
let f =
a = 2
b = 3
lambda x: x * a + b
What I want:
f must work as intended and have the same signature
a and b must be computed only once
a and b must not exist in the scope outside of f
Assignments a = ... and b = ... don't affect f
The cleanest way to do this. E.g. the following solution formally works, but it introduces g and then deletes it, which I don't like (e.g. there is a risk of overriding an existing g and I believe that it's simply ugly):
def g():
a = 2
b = 3
return lambda x: x * a + b
f = g()
del g
One method is to simply use a class. This allows you to place a and b in the scope of the class while f can still access them.
custom class
class F:
def __init__(self):
self.a = 2
self.b = 3
def __call__(self, x):
return x * self.a + self.b
f = F()
f(1)
# returns:
5
If you don't like having to call the class constructor, you can override __new__ to essentially create a callable with internal stored variables. This is an antipattern though and not very pythonic.
custom callable
class f:
a = 2
b = 3
def __new__(cls, x):
return x * cls.a + cls.b
f(1)
# returns:
5
This approach is based on the answers provided in this thread, though scoped to the specific problem above. You can use a decorator to update the global variables available to the function while also storin a and b within a closure.
decorator with closure
from functools import wraps
def dec_ab(fn):
a = 2
b = 3
#wraps(fn)
def wrapper(*args, **kwargs):
# get global scope
global_scope = f.__globals__
# copy current values of variables
var_list = ['a', 'b']
current_vars = {}
for var in var_list:
if var in global_scope:
current_vars[var] = global_scope.get(var)
# update global scope
global_scope.update({'a': a, 'b': b})
try:
out = fn(*args, **kwargs)
finally:
# undo the changes to the global scope
for var in var_list:
global_scope.pop(var)
global_scope.update(current_vars)
return out
return wrapper
#dec_ab
def f(x):
"""hello world"""
return x * a + b
This preserves the functions signature and keeps a and b from being altered
f(1)
# returns:
5
a
# raises:
NameError: name 'a' is not defined
You can use default arguments to accomplish this. Default arguments are only computed once, when the closure is created (that is why if you have mutable objects as default arguments, the state is retained between invocations).
def f(x, a=2, b=3):
return x * a + b
How can I make a Python function default value be evaluated each time the function is called?
Take this dummy code:
b=0
def a():
global b
return b
def c(d=a()):
return d
What I would expect as output:
>>> c()
0
>>> b=1
>>> a()
1
>>> c()
1
What I actually get:
>>> c()
0
>>> b=1
>>> a()
1
>>> c()
0
One more solution, in closer resemblance to your original answer.
b = 0
def a():
return b
def c(d=a): # When it's a parameter, the call will be evaluated and its return
# value will be used. Instead, just use the function name, because
return d() # within the scope of the function, the call will be evaluated every time.
When a function name is paired with the parentheses and its parameters, like f(x), it is assumed your intention is to call it at that time
d=a() is evaluated at start of program when function c is defined (ie a() gets called while it returns 0 ...)
def c(d=None):
if d == None: d=a()
return d
will cause it to be evaluated at the time you want
The problem here is, as you probably already know, that the d=a() (default argument assignement) is evaluated when the function is defined.
To change that, it is pretty common to use eg. None as default argument and evaluate it in the body of the function:
b=0
def a():
global b
return b
def c(d=None):
if d is None:
d = a()
return d
I'll give a slight variation on the above:
b = 0
def a():
# unless you are writing changes to b, you do not have to make it a global
return b
def c(d=None, get_d=a):
if d is None:
d = get_d()
return d
Given a function:
def A(a, b, c):
a *= 2
b *= 4
c *= 8
return a+b+c
How can I set the 'c' var to be called-by-reference, so if i call d = A(a,b,c), c will point to the same int object, and be updated from within the function?
You're getting into murky territory: you're talking about declaring global (or nonlocal) variables in functions, which should simply not be done. Functions are meant to do one thing, and do them without the consent of other values or functions affecting their state or output.
It's difficult to suggest a working example: are you alright with having copies of the variables left behind for later reference? You could expand this code to pass back a tuple, and reference the members of the tuple if needed, or the sum:
>>> def A(a, b, c):
return (a*2, b*4, c*8)
>>> d = A(2, 4, 8)
>>> sum(d)
84
>>> d[-1] #or whatever index you'd need...this may serve best as a constant
64
You can do this if c is a list:
c = [2]
def A(a, b, c):
a *= 2
b *= 4
c[0] *= 8
return a+b+c[0]
print c # gives [16]
You can't. Python cannot do that.
What you can do is pass a wrapper that has __imul__() defined and an embedded int, and the augmented assignment will result in the embedded attribute being mutated instead.
All calls in Python are "by reference". Integers are immutable in Python. You can't change them.
class C:
def __init__(self, c):
self.c = c
def __call__(self, a, b):
a *= 2
b *= 4
self.c *= 8
return a + b + self.c
Example
A = C(1)
print A(1, 1), A.c
print A(1, 1), A.c
Output
14 8
70 64