We Keep Coding

Properly handling dicts where the keys are lambdas [duplicate]

I'm trying to create functions inside of a loop:
functions = []
for i in range(3):
def f():
return i
# alternatively: f = lambda: i
functions.append(f)
The problem is that all functions end up being the same. Instead of returning 0, 1, and 2, all three functions return 2:
print([f() for f in functions])
# expected output: [0, 1, 2]
# actual output: [2, 2, 2]
Why is this happening, and what should I do to get 3 different functions that output 0, 1, and 2 respectively?

You're running into a problem with late binding -- each function looks up i as late as possible (thus, when called after the end of the loop, i will be set to 2).
Easily fixed by forcing early binding: change def f(): to def f(i=i): like this:
def f(i=i):
return i
Default values (the right-hand i in i=i is a default value for argument name i, which is the left-hand i in i=i) are looked up at def time, not at call time, so essentially they're a way to specifically looking for early binding.
If you're worried about f getting an extra argument (and thus potentially being called erroneously), there's a more sophisticated way which involved using a closure as a "function factory":
def make_f(i):
def f():
return i
return f
and in your loop use f = make_f(i) instead of the def statement.

The Explanation
The issue here is that the value of i is not saved when the function f is created. Rather, f looks up the value of i when it is called.
If you think about it, this behavior makes perfect sense. In fact, it's the only reasonable way functions can work. Imagine you have a function that accesses a global variable, like this:
global_var = 'foo'
def my_function():
print(global_var)
global_var = 'bar'
my_function()
When you read this code, you would - of course - expect it to print "bar", not "foo", because the value of global_var has changed after the function was declared. The same thing is happening in your own code: By the time you call f, the value of i has changed and been set to 2.
The Solution
There are actually many ways to solve this problem. Here are a few options:
Force early binding of i by using it as a default argument
Unlike closure variables (like i), default arguments are evaluated immediately when the function is defined:
for i in range(3):
def f(i=i): # <- right here is the important bit
return i
functions.append(f)
To give a little bit of insight into how/why this works: A function's default arguments are stored as an attribute of the function; thus the current value of i is snapshotted and saved.
>>> i = 0
>>> def f(i=i):
... pass
>>> f.__defaults__ # this is where the current value of i is stored
(0,)
>>> # assigning a new value to i has no effect on the function's default arguments
>>> i = 5
>>> f.__defaults__
(0,)
Use a function factory to capture the current value of i in a closure
The root of your problem is that i is a variable that can change. We can work around this problem by creating another variable that is guaranteed to never change - and the easiest way to do this is a closure:
def f_factory(i):
def f():
return i # i is now a *local* variable of f_factory and can't ever change
return f
for i in range(3):
f = f_factory(i)
functions.append(f)
Use functools.partial to bind the current value of i to f
functools.partial lets you attach arguments to an existing function. In a way, it too is a kind of function factory.
import functools
def f(i):
return i
for i in range(3):
f_with_i = functools.partial(f, i) # important: use a different variable than "f"
functions.append(f_with_i)
Caveat: These solutions only work if you assign a new value to the variable. If you modify the object stored in the variable, you'll experience the same problem again:
>>> i = [] # instead of an int, i is now a *mutable* object
>>> def f(i=i):
... print('i =', i)
...
>>> i.append(5) # instead of *assigning* a new value to i, we're *mutating* it
>>> f()
i = [5]
Notice how i still changed even though we turned it into a default argument! If your code mutates i, then you must bind a copy of i to your function, like so:
def f(i=i.copy()):
f = f_factory(i.copy())
f_with_i = functools.partial(f, i.copy())

To add onto #Aran-Fey's excellent answer, in the second solution you might also wish to modify the variable inside your function which can be accomplished with the keyword nonlocal:
def f_factory(i):
def f(offset):
nonlocal i
i += offset
return i # i is now a *local* variable of f_factory and can't ever change
return f
for i in range(3):
f = f_factory(i)
print(f(10))

You have to save the each of the i value in a separate space in memory e.g.:
class StaticValue:
val = None
def __init__(self, value: int):
StaticValue.val = value
#staticmethod
def get_lambda():
return lambda x: x*StaticValue.val
class NotStaticValue:
def __init__(self, value: int):
self.val = value
def get_lambda(self):
return lambda x: x*self.val
if __name__ == '__main__':
def foo():
return [lambda x: x*i for i in range(4)]
def bar():
return [StaticValue(i).get_lambda() for i in range(4)]
def foo_repaired():
return [NotStaticValue(i).get_lambda() for i in range(4)]
print([x(2) for x in foo()])
print([x(2) for x in bar()])
print([x(2) for x in foo_repaired()])
Result:
[6, 6, 6, 6]
[6, 6, 6, 6]
[0, 2, 4, 6]

You can try like this:
l=[]
for t in range(10):
def up(y):
print(y)
l.append(up)
l[5]('printing in 5th function')

just modify the last line for
functions.append(f())
Edit: This is because f is a function - python treats functions as first-class citizens and you can pass them around in variables to be called later on. So what your original code is doing is appending the function itself to the list, while what you want to do is append the results of the function to the list, which is what the line above achieves by calling the function.

How to use nonlocal statement with several inner functions?

Might someone explain me why this function is not working ? Shouldn't the "nonlocal" statement makes x understandable in g, and therefore in h ?
def f():
def g():
nonlocal x
x= 1
def h():
print(x)
>>> SyntaxError: no binding for nonlocal 'x' found
Edit : I used nonlocal in order not to define x anywhere else than in g() : I have to define several variables in my code, and want to do it in a function init_var(). For lisibility, I want to avoid declaring them in my main function. Is there a way to adapt the previous code for this aim ?

From docs:
Names listed in a nonlocal statement, unlike those listed in a global statement, must refer to pre-existing bindings in an enclosing scope (the scope in which a new binding should be created cannot be determined unambiguously).
Your x is not pre-existing at the point of nonlocal. Try x = None just before def g(): to create a binding for nonlocal to refer to.
The ambiguity problem stated by the docs are easy to see here:
def f():
def g():
def h():
def i():
def j():
nonlocal x
Which functions should have access to x, and which shouldn't? On the other hand, here it is clear:
def f():
def g():
def h():
x = None
def i():
def j():
nonlocal x
In this case, f and g don't know x, while h, i and j do.

Not sure whether what you are trying really helps readability. If you want to to structure your code, you might want to consider using a class and (instance) attributes. That being said, if you want to avoid multiple assignment lines, you can use attributes of the outer function:
>>> def f():
... def g():
... f.a = 5
... def h():
... print(f.a)
... g()
... h()
...
>>> f()
5

Understanding weird behaviour of python list comprehension [duplicate]

I'm trying to create functions inside of a loop:
functions = []
for i in range(3):
def f():
return i
# alternatively: f = lambda: i
functions.append(f)
The problem is that all functions end up being the same. Instead of returning 0, 1, and 2, all three functions return 2:
print([f() for f in functions])
# expected output: [0, 1, 2]
# actual output: [2, 2, 2]
Why is this happening, and what should I do to get 3 different functions that output 0, 1, and 2 respectively?

You're running into a problem with late binding -- each function looks up i as late as possible (thus, when called after the end of the loop, i will be set to 2).
Easily fixed by forcing early binding: change def f(): to def f(i=i): like this:
def f(i=i):
return i
Default values (the right-hand i in i=i is a default value for argument name i, which is the left-hand i in i=i) are looked up at def time, not at call time, so essentially they're a way to specifically looking for early binding.
If you're worried about f getting an extra argument (and thus potentially being called erroneously), there's a more sophisticated way which involved using a closure as a "function factory":
def make_f(i):
def f():
return i
return f
and in your loop use f = make_f(i) instead of the def statement.

The Explanation
The issue here is that the value of i is not saved when the function f is created. Rather, f looks up the value of i when it is called.
If you think about it, this behavior makes perfect sense. In fact, it's the only reasonable way functions can work. Imagine you have a function that accesses a global variable, like this:
global_var = 'foo'
def my_function():
print(global_var)
global_var = 'bar'
my_function()
When you read this code, you would - of course - expect it to print "bar", not "foo", because the value of global_var has changed after the function was declared. The same thing is happening in your own code: By the time you call f, the value of i has changed and been set to 2.
The Solution
There are actually many ways to solve this problem. Here are a few options:
Force early binding of i by using it as a default argument
Unlike closure variables (like i), default arguments are evaluated immediately when the function is defined:
for i in range(3):
def f(i=i): # <- right here is the important bit
return i
functions.append(f)
To give a little bit of insight into how/why this works: A function's default arguments are stored as an attribute of the function; thus the current value of i is snapshotted and saved.
>>> i = 0
>>> def f(i=i):
... pass
>>> f.__defaults__ # this is where the current value of i is stored
(0,)
>>> # assigning a new value to i has no effect on the function's default arguments
>>> i = 5
>>> f.__defaults__
(0,)
Use a function factory to capture the current value of i in a closure
The root of your problem is that i is a variable that can change. We can work around this problem by creating another variable that is guaranteed to never change - and the easiest way to do this is a closure:
def f_factory(i):
def f():
return i # i is now a *local* variable of f_factory and can't ever change
return f
for i in range(3):
f = f_factory(i)
functions.append(f)
Use functools.partial to bind the current value of i to f
functools.partial lets you attach arguments to an existing function. In a way, it too is a kind of function factory.
import functools
def f(i):
return i
for i in range(3):
f_with_i = functools.partial(f, i) # important: use a different variable than "f"
functions.append(f_with_i)
Caveat: These solutions only work if you assign a new value to the variable. If you modify the object stored in the variable, you'll experience the same problem again:
>>> i = [] # instead of an int, i is now a *mutable* object
>>> def f(i=i):
... print('i =', i)
...
>>> i.append(5) # instead of *assigning* a new value to i, we're *mutating* it
>>> f()
i = [5]
Notice how i still changed even though we turned it into a default argument! If your code mutates i, then you must bind a copy of i to your function, like so:
def f(i=i.copy()):
f = f_factory(i.copy())
f_with_i = functools.partial(f, i.copy())

To add onto #Aran-Fey's excellent answer, in the second solution you might also wish to modify the variable inside your function which can be accomplished with the keyword nonlocal:
def f_factory(i):
def f(offset):
nonlocal i
i += offset
return i # i is now a *local* variable of f_factory and can't ever change
return f
for i in range(3):
f = f_factory(i)
print(f(10))

You have to save the each of the i value in a separate space in memory e.g.:
class StaticValue:
val = None
def __init__(self, value: int):
StaticValue.val = value
#staticmethod
def get_lambda():
return lambda x: x*StaticValue.val
class NotStaticValue:
def __init__(self, value: int):
self.val = value
def get_lambda(self):
return lambda x: x*self.val
if __name__ == '__main__':
def foo():
return [lambda x: x*i for i in range(4)]
def bar():
return [StaticValue(i).get_lambda() for i in range(4)]
def foo_repaired():
return [NotStaticValue(i).get_lambda() for i in range(4)]
print([x(2) for x in foo()])
print([x(2) for x in bar()])
print([x(2) for x in foo_repaired()])
Result:
[6, 6, 6, 6]
[6, 6, 6, 6]
[0, 2, 4, 6]

You can try like this:
l=[]
for t in range(10):
def up(y):
print(y)
l.append(up)
l[5]('printing in 5th function')

just modify the last line for
functions.append(f())
Edit: This is because f is a function - python treats functions as first-class citizens and you can pass them around in variables to be called later on. So what your original code is doing is appending the function itself to the list, while what you want to do is append the results of the function to the list, which is what the line above achieves by calling the function.

Dynamically Adding Methods to Python Class [duplicate]

I'm trying to create functions inside of a loop:
functions = []
for i in range(3):
def f():
return i
# alternatively: f = lambda: i
functions.append(f)
The problem is that all functions end up being the same. Instead of returning 0, 1, and 2, all three functions return 2:
print([f() for f in functions])
# expected output: [0, 1, 2]
# actual output: [2, 2, 2]
Why is this happening, and what should I do to get 3 different functions that output 0, 1, and 2 respectively?

You're running into a problem with late binding -- each function looks up i as late as possible (thus, when called after the end of the loop, i will be set to 2).
Easily fixed by forcing early binding: change def f(): to def f(i=i): like this:
def f(i=i):
return i
Default values (the right-hand i in i=i is a default value for argument name i, which is the left-hand i in i=i) are looked up at def time, not at call time, so essentially they're a way to specifically looking for early binding.
If you're worried about f getting an extra argument (and thus potentially being called erroneously), there's a more sophisticated way which involved using a closure as a "function factory":
def make_f(i):
def f():
return i
return f
and in your loop use f = make_f(i) instead of the def statement.

The Explanation
The issue here is that the value of i is not saved when the function f is created. Rather, f looks up the value of i when it is called.
If you think about it, this behavior makes perfect sense. In fact, it's the only reasonable way functions can work. Imagine you have a function that accesses a global variable, like this:
global_var = 'foo'
def my_function():
print(global_var)
global_var = 'bar'
my_function()
When you read this code, you would - of course - expect it to print "bar", not "foo", because the value of global_var has changed after the function was declared. The same thing is happening in your own code: By the time you call f, the value of i has changed and been set to 2.
The Solution
There are actually many ways to solve this problem. Here are a few options:
Force early binding of i by using it as a default argument
Unlike closure variables (like i), default arguments are evaluated immediately when the function is defined:
for i in range(3):
def f(i=i): # <- right here is the important bit
return i
functions.append(f)
To give a little bit of insight into how/why this works: A function's default arguments are stored as an attribute of the function; thus the current value of i is snapshotted and saved.
>>> i = 0
>>> def f(i=i):
... pass
>>> f.__defaults__ # this is where the current value of i is stored
(0,)
>>> # assigning a new value to i has no effect on the function's default arguments
>>> i = 5
>>> f.__defaults__
(0,)
Use a function factory to capture the current value of i in a closure
The root of your problem is that i is a variable that can change. We can work around this problem by creating another variable that is guaranteed to never change - and the easiest way to do this is a closure:
def f_factory(i):
def f():
return i # i is now a *local* variable of f_factory and can't ever change
return f
for i in range(3):
f = f_factory(i)
functions.append(f)
Use functools.partial to bind the current value of i to f
functools.partial lets you attach arguments to an existing function. In a way, it too is a kind of function factory.
import functools
def f(i):
return i
for i in range(3):
f_with_i = functools.partial(f, i) # important: use a different variable than "f"
functions.append(f_with_i)
Caveat: These solutions only work if you assign a new value to the variable. If you modify the object stored in the variable, you'll experience the same problem again:
>>> i = [] # instead of an int, i is now a *mutable* object
>>> def f(i=i):
... print('i =', i)
...
>>> i.append(5) # instead of *assigning* a new value to i, we're *mutating* it
>>> f()
i = [5]
Notice how i still changed even though we turned it into a default argument! If your code mutates i, then you must bind a copy of i to your function, like so:
def f(i=i.copy()):
f = f_factory(i.copy())
f_with_i = functools.partial(f, i.copy())

To add onto #Aran-Fey's excellent answer, in the second solution you might also wish to modify the variable inside your function which can be accomplished with the keyword nonlocal:
def f_factory(i):
def f(offset):
nonlocal i
i += offset
return i # i is now a *local* variable of f_factory and can't ever change
return f
for i in range(3):
f = f_factory(i)
print(f(10))

You have to save the each of the i value in a separate space in memory e.g.:
class StaticValue:
val = None
def __init__(self, value: int):
StaticValue.val = value
#staticmethod
def get_lambda():
return lambda x: x*StaticValue.val
class NotStaticValue:
def __init__(self, value: int):
self.val = value
def get_lambda(self):
return lambda x: x*self.val
if __name__ == '__main__':
def foo():
return [lambda x: x*i for i in range(4)]
def bar():
return [StaticValue(i).get_lambda() for i in range(4)]
def foo_repaired():
return [NotStaticValue(i).get_lambda() for i in range(4)]
print([x(2) for x in foo()])
print([x(2) for x in bar()])
print([x(2) for x in foo_repaired()])
Result:
[6, 6, 6, 6]
[6, 6, 6, 6]
[0, 2, 4, 6]

You can try like this:
l=[]
for t in range(10):
def up(y):
print(y)
l.append(up)
l[5]('printing in 5th function')

just modify the last line for
functions.append(f())
Edit: This is because f is a function - python treats functions as first-class citizens and you can pass them around in variables to be called later on. So what your original code is doing is appending the function itself to the list, while what you want to do is append the results of the function to the list, which is what the line above achieves by calling the function.

Convention for declaring variables in Python, when required for scope?

Take this example:
def f():
myvar = None
def g():
print myvar
...
myvar = get_real_value()
g()
Is "myvar = None" a conventional (or at least, reasonable) way of declaring the variable, to make it visible to g()? Is there a better way? (Python 2.6.x, if relevant)

There is no need to declare the variable before the definition for g():
>>> def f():
... def g():
... print myvar
... myvar = 1
... g()
...
>>> f()
1
However, if you can avoid referencing non-local variables in g() that would be preferable, which you would probably do here by having myvar be a parameter to g().

You don't need to pre-initialize the variable; it only has to be initialized before you call g(). Just remove the myvar = None line.
For clarity, I prefer to move local functions as close to their point of invocation as practicable, so that the initialisation of local variables that a function uses comes before the function itself.