I just ran across Eric Lippert's Closing over the loop variable considered harmful via SO, and, after experimenting, realized that the same problem exists (and is even harder to get around) in Python.
>>> l = []
>>> for r in range(10):
... def foo():
... return r
... l.append(foo)
...
>>> for f in l:
... f()
...
9
9
9
# etc
and, the standard C# workaround doesn't work (I assume because of the nature of references in Python)
>>> l = []
>>> for r in range(10):
... r2 = r
... def foo():
... return r2
... l.append(foo)
...
>>> for f in l:
... f()
...
9
9
9
# etc
I recognize that this isn't much of a problem in Python with its general emphasis on non-closure object structures, but I'm curious if there is an obvious Pythonic way to handle this, or do we have to go the JS route of nested function calls to create actually new vars?
>>> l = []
>>> for r in range(10):
... l.append((lambda x: lambda: x)(r))
...
>>> for f in l:
... f()
...
0
1
2
# etc
One way is to use a parameter with default value:
l = []
for r in range(10):
def foo(r = r):
return r
l.append(foo)
for f in l:
print(f())
yields
0
1
2
3
4
5
6
7
8
9
This works because it defines an r in foo's local scope, and binds the default value to it at the time foo is defined.
Another way is to use a function factory:
l = []
for r in range(10):
def make_foo(r):
def foo():
return r
return foo
l.append(make_foo(r))
for f in l:
print(f())
This works because it defines r in make_foo's local scope, and binds a value to it when make_foo(r) is called. Later, when f() is called, r is looked-up using the LEGB rule. Although r is not found in the local scope of foo, it is found in the enclosing scope of make_foo.
Related
Lets say I have a function with a variable duration inside it.
Is there any way to set the Duration` value outside of the function in some other nonparent function without adjusting any parameter?
With Python 3.x you can declare it with the nonlocal keyword
def make_slow(action):
slowDuration = None
def slowAction(self, colony):
nonlocal slowDuration
slowDuration = 10 # It is changing the variable from the scope above
If you want to change a value from somewhere else and you mustn't return the value, try going global... Note this may pollute your current namespace.
For a more pythonic approach, you should use something like self.slowDuration. That's what objects are for.
slowDuration is a local variable of the slowAction function. The point of local variables is that they are only accessible inside the function.
You should change the slowAction function so it uses a slowDuration variable that is defined somewhere else, for example as a member variable of the class that make_slow apparently belongs to.
You can also make slowAction be an instance of a class that overrides the __call__ method.
>>> class Counter:
... def __init__(self):
... self.count = 0
... def __call__(self, delta):
... self.count += delta
... print(self.count)
... def set_count(self, c):
... self.count = c
...
>>> c = Counter()
>>> c(1)
1
>>> c(3)
4
>>> c(3)
7
>>> c(3)
10
>>> c.set_count(42)
>>> c(-2)
40
You could also use some trickery to make the shared variable available on the function object itself:
def makeCounter():
counter = None
def counter_func():
counter.count += 1
print(counter.count)
counter = counter_func
counter.count = 0
return counter
and use it like this:
>>> c = makeCounter()
>>> c()
1
>>> c()
2
>>> c()
3
>>> c()
4
>>> c.count = 42
>>> c()
43
>>> c()
44
>>> c()
45
But in general, "clever" code like that should be avoided unless you have a very good reason to use it, because it makes the code base harder to understand.
Where X is any programming language (C#, Javascript, Lisp, Perl, Ruby, Scheme, etc) which supports some flavour of closures.
Some limitations are mentioned in the Closures in Python (compared to Ruby's closures), but the article is old and many limitations do not exist in modern Python any more.
Seeing a code example for a concrete limitation would be great.
Related questions:
Can you explain closures (as they relate to Python)?
What is a ‘Closure’?
How does a javascript closure work ?
The most important limitation, currently, is that you cannot assign to an outer-scope variable. In other words, closures are read-only:
>>> def outer(x):
... def inner_reads():
... # Will return outer's 'x'.
... return x
... def inner_writes(y):
... # Will assign to a local 'x', not the outer 'x'
... x = y
... def inner_error(y):
... # Will produce an error: 'x' is local because of the assignment,
... # but we use it before it is assigned to.
... tmp = x
... x = y
... return tmp
... return inner_reads, inner_writes, inner_error
...
>>> inner_reads, inner_writes, inner_error = outer(5)
>>> inner_reads()
5
>>> inner_writes(10)
>>> inner_reads()
5
>>> inner_error(10)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 11, in inner_error
UnboundLocalError: local variable 'x' referenced before assignment
A name that gets assigned to in a local scope (a function) is always local, unless declared otherwise. While there is the 'global' declaration to declare a variable global even when it is assigned to, there is no such declaration for enclosed variables -- yet. In Python 3.0, there is (will be) the 'nonlocal' declaration that does just that.
You can work around this limitation in the mean time by using a mutable container type:
>>> def outer(x):
... x = [x]
... def inner_reads():
... # Will return outer's x's first (and only) element.
... return x[0]
... def inner_writes(y):
... # Will look up outer's x, then mutate it.
... x[0] = y
... def inner_error(y):
... # Will now work, because 'x' is not assigned to, just referenced.
... tmp = x[0]
... x[0] = y
... return tmp
... return inner_reads, inner_writes, inner_error
...
>>> inner_reads, inner_writes, inner_error = outer(5)
>>> inner_reads()
5
>>> inner_writes(10)
>>> inner_reads()
10
>>> inner_error(15)
10
>>> inner_reads()
15
The only difficulty I've seen people encounter with Python's in particular is when they try to mix non-functional features like variable reassignment with closures, and are surprised when this doesn't work:
def outer ():
x = 1
def inner ():
print x
x = 2
return inner
outer () ()
Usually just pointing out that a function has its own local variables is enough to deter such silliness.
A limitation (or "limitation") of Python closures, comparing to Javascript closures, is that it cannot be used for effective data hiding
Javascript
var mksecretmaker = function(){
var secrets = [];
var mksecret = function() {
secrets.push(Math.random())
}
return mksecret
}
var secretmaker = mksecretmaker();
secretmaker(); secretmaker()
// privately generated secret number list
// is practically inaccessible
Python
import random
def mksecretmaker():
secrets = []
def mksecret():
secrets.append(random.random())
return mksecret
secretmaker = mksecretmaker()
secretmaker(); secretmaker()
# "secrets" are easily accessible,
# it's difficult to hide something in Python:
secretmaker.__closure__[0].cell_contents # -> e.g. [0.680752847190161, 0.9068475951742101]
Fixed in Python 3 via the nonlocal statement:
The nonlocal statement causes the listed identifiers to refer to previously bound variables in the nearest enclosing scope excluding globals. This is important because the default behavior for binding is to search the local namespace first. The statement allows encapsulated code to rebind variables outside of the local scope besides the global (module) scope.
#John Millikin
def outer():
x = 1 # local to `outer()`
def inner():
x = 2 # local to `inner()`
print(x)
x = 3
return x
def inner2():
nonlocal x
print(x) # local to `outer()`
x = 4 # change `x`, it is not local to `inner2()`
return x
x = 5 # local to `outer()`
return (inner, inner2)
for inner in outer():
print(inner())
# -> 2 3 5 4
comment for #Kevin Little's answer to include the code example
nonlocal does not solve completely this problem on python3.0:
x = 0 # global x
def outer():
x = 1 # local to `outer`
def inner():
global x
x = 2 # change global
print(x)
x = 3 # change global
return x
def inner2():
## nonlocal x # can't use `nonlocal` here
print(x) # prints global
## x = 4 # can't change `x` here
return x
x = 5
return (inner, inner2)
for inner in outer():
print(inner())
# -> 2 3 3 3
On the other hand:
x = 0
def outer():
x = 1 # local to `outer`
def inner():
## global x
x = 2
print(x) # local to `inner`
x = 3
return x
def inner2():
nonlocal x
print(x)
x = 4 # local to `outer`
return x
x = 5
return (inner, inner2)
for inner in outer():
print(inner())
# -> 2 3 5 4
it works on python3.1-3.3
The better workaround until 3.0 is to include the variable as a defaulted parameter in the enclosed function definition:
def f()
x = 5
def g(y, z, x=x):
x = x + 1
Consider the following code:
>>> b = 3
>>> def change (b):
... b = 2
...
>>> change(b)
>>> print(b)
3
I am not confused by the fact that the print statement returns 3. Here's the question: is this because of scope, or because the argument that the function takes is not related to the outside b?
The b inside change is of a different scope than b outside of change. Inside the scope of the function, it does not matter what you call the variable before you pass it in - for now, it's going to be called b. As long as you don't return b from change and assign it to your "original" b, the first one won't change.
def change(b):
b = 2
return b
change(b)
print(b) # 3
b = change(b)
print(b) # 2
You are reassigning b. It does not change the parameter. You can test this using the id() function before and after the assignement.
The scope answers are partially correct for the object type you are passing.
Check out this link for information about how python "passes by value reference"
http://stupidpythonideas.blogspot.com/2013/11/does-python-pass-by-value-or-by.html
Here is another link that explains pythons Local -> Enclosed -> Global -> Built-in scope resolution with some examples:
http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/not_so_obvious_python_stuff.ipynb#python_legb
FWIW, the behavior is slightly different if we consider mutable objects:
>>> b = [3]
>>> def change(b):
... b[0] = 2
...
>>> change(b)
>>> print(b)
[2]
>>>
You have asked quite a tricky question, in the sense that the terminology you have used in framing the question is making people give you differing and weird answers.
You've given two choices:
we have a scope issue
b inside the function isn't related to b outside the function
Well, they are both true, and they are really just two ways to say the same thing.
Forgetting about terminology for a moment, realize that there is absolutely no difference between
b = 3
def f(a):
print(a)
f(b)
and
b = 3
def f(b):
print(b)
f(b)
The parameter names in a function's signature (the def line of a function) are local to the function, no matter what names are outside the function.
Now, you probably noticed that I sidestepped the issue of assignment within a function. That's because it's harder to explain quickly, but still be understandable and memorable to someone still in the fairly early stages of learning Python (which, no offense, it seems like you are).
Here is some reading that should help you understand fundamental concepts in Python:
http://nedbatchelder.com/text/names.html
http://www.jeffknupp.com/blog/2013/02/14/drastically-improve-your-python-understanding-pythons-execution-model/
You need to look at global variables. Just add global b at the beginning of your code and function, and it should work:
>>> global b
>>> b = 3
>>> def change():
... global b
... b = 2
...
>>> b
3
>>> change()
>>> b
2
>>>
You also need to remove the b parameter from def change(b), because that makes b both local and global. You can also return b:
>>> b = 3
>>> def change(b):
... b = 2
... return b
...
>>> b
3
>>> b = change(b)
>>> b
2
It is a scoping problem. b in the function is a local variable.
globals() is dictionary which hold all global variables.
b=3
def change (a):
globals()[a] = 2
print b
# 3
change("b")
print b
# 2
Given a function in Python defined as follows:
a = 3
b = 4
c = 5
def add():
d = a+b+c
is it possible to get a code object or similar that gives me:
a = 3
b = 4
c = 5
d = a+b+c
The function object has a code object associated with it; you can exec that code object:
>>> a = 3
>>> b = 4
>>> c = 5
>>> def add():
... d = a+b+c
...
>>> exec add.func_code
This will not however set d in the local namespace, because it is compiled to create a local name only, using the STORE_FAST opcode. You'd have to transform this bytecode to use STORE_DEREF opcodes instead, requiring you to learn AST transformations.
If you did want to go down this route, then read Exploring Dynamic Scoping in Python, which teaches you the ins and outs of Python scoping and byte code.
You can use the global command, which
is the most simple way in my opinion
>>> a = 3
>>> b = 4
>>> c = 5
>>> def add():
global d
... d = a+b+c
In Perl 5.10, I can say:
sub foo () {
state $x = 1;
say $x++;
}
foo();
foo();
foo();
...and it will print out:
1
2
3
Does Python have something like this?
A class may be a better fit here (and is usually a better fit for anything involving "state"):
class Stateful(object):
def __init__(self):
self.state_var = 0
def __call__(self):
self.state_var = self.state_var + 1
print self.state_var
foo = Stateful()
foo()
foo()
The closest parallel is probably to attach values to the function itself.
def foo():
foo.bar = foo.bar + 1
foo.bar = 0
foo()
foo()
foo()
print foo.bar # prints 3
Python has generators which do something similar:
What does the "yield" keyword do in Python?
Not sure if this is what you're looking for, but python has generator functions that don't return a value per se, but a generator object that generates a new value everytime
def gen():
x = 10
while True:
yield x
x += 1
usage:
>>> a = gen()
>>> a.next()
10
>>> a.next()
11
>>> a.next()
12
>>> a.next()
13
>>>
look here for more explanation on yield:
What does the "yield" keyword do in Python?
Here's one way to implement a closure in python:
def outer():
a = [4]
def inner():
print a[0]
a[0] = a[0] + 1
return inner
fn = outer()
fn() # => 4
fn() # => 5
fn() # => 6
I borrowed this example verbatim from a python mailing list post.
Not that I'm recommending this, but just for fun:
def foo(var=[1]):
print var[0]
var[0] += 1
This works because of the way mutable default arguments work in Python.
Yes, though you have to declare your global variable first before it is encountered in foo:
x = 0
def foo():
global x
x += 1
print x
foo()
foo()
foo()
EDIT: In response to the comment, it's true that python has no static variables scoped within a function. Note that x in this example is only exposed as global to the rest of the module. For example, say the code above is in test.py. Now suppose you write the following module:
from test import foo
x = 100
foo()
foo()
The output will be only 1 and 2, not 101 and 102.
You could also use something like
def static_num2():
k = 0
while True:
k += 1
yield k
static = static_num2().next
for i in range(0,10) :
print static()
to avoid a global var. Lifted from this link about the same question.
The preferable way is to use class or generator (yield).
For the sake of completeness here's a variant w/ closure in Python 3.x:
>>> def make_foo():
... x = 1
... def foo():
... nonlocal x
... print(x)
... x += 1
... return foo
...
>>> foo = make_foo()
>>> foo()
1
>>> foo()
2
>>> foo()
3
>>> def foo():
x = 1
while True:
yield x
x += 1
>>> z = iter(foo())
>>> next(z)
1
>>> next(z)
2
>>> next(z)
3
Here's another dirty cheap way to do it, it's a variation on Tryiptich's answer, but using decorators
def static_var( name, value ):
def dec( function ):
setattr( function, name, value )
return function
return dec
#static_var( 'counter', 0 )
def counting_function():
counting_function.counter = counting_function.counter + 1
print counting_function.counter
"""
>>> counting_function()
1
>>> counting_function()
2
>>> counting_function()
3
>>> counting_function()
4
>>> counting_function()
5
>>>
"""