This question already has answers here:
Use value of variable in lambda expression [duplicate]
(3 answers)
Closed 7 years ago.
I've experienced some strange behaviour when storing lambda functions into a dictionary: If you try to pass some default value to a function in a loop, only the last default value is being used.
Here some minimal example:
#!/usr/bin/env python
# coding: utf-8
def myfct(one_value, another_value):
"do something with two int values"
return one_value + another_value
fct_dict = {'add_{}'.format(number): (lambda x: myfct(x, number))
for number in range(10)}
print('add_3(1): {}, id={}'.format(fct_dict['add_3'](1), id(fct_dict['add_3'])))
print('add_5(1): {}, id={}'.format(fct_dict['add_5'](1), id(fct_dict['add_5'])))
print('add_9(1): {}, id={}'.format(fct_dict['add_9'](1), id(fct_dict['add_9'])))
The output reads as follows
add_3(1): 10, id=140421083875280
add_5(1): 10, id=140421083875520
add_9(1): 10, id=140421083876000
You get dissimilar functions (id not identical) but every function uses the same second argument.
Can somebody explain what's going on?
The same holds with python2, python3, pypy...
The fix:
def make_closure(number):
return lambda x: myfct(x, number)
used as
{'add_{}'.format(number): make_closure(number) for number in range(10)}
The reason for this behaviour is, that the variable number (think: named memory location here) is the same during all iterations of the loop (though its actual value changes in each iteration). "Loop" here refers to the dictionary comprehension, which internally is based on a loop. All lambda instances created in the loop will close over the same "location", which retains the value last assigned to it (in the last iteration of the loop).
The following code is not what actually happens underneath. It is merely provided to shed light on the concepts:
# Think of a closure variable (like number) as being an instance
# of the following class
class Cell:
def __init__(self, init=None):
self.value = None
# Pretend, the compiler "desugars" the dictionary comprehension into
# something like this:
hidden_result_dict = {}
hidden_cell_number = Cell()
for number in range(10):
hidden_cell_number.value = number
hidden_result_dictionary['add_{}'.format(number)] = create_lambda_closure(hidden_cell_number)
All lambda closures created by the create_lambda_closure operation share the very same Cell instance and will grab the value attribute at run-time (i.e., when the closure is actually called). By that time, value will refer to last value ever assigned to it.
The value of hidden_result_dict is then answered as the result of the dict comprehension. (Again: this is only meant as be read on a "conceptual" level; it has no relation to the actual code executed by the Python VM).
number is a variable which has different value for each iteration of the dict comprehension. But when you do lambda x: myfct(x, number), it does not use value of number. It just creates a lambda method that will use value of number when it will be called/used. So when you use you add_{} methods, number has value 9 which is used in every call to myfct(x, number).
Related
I originally had a problem with this code as I was missing the 'n =' in the last line of code and, as a result, was stuck in an infinite loop.
At this point, while I understand what needed to be corrected I don't understand why. Why can't 'collatz(n)' be enough to call the function and use n as its variable? If anyone could explain this in simple terms (beginner here), I'd really appreciate it.
def collatz(number):
if number % 2 == 0:
print (number // 2)
return number // 2
elif number % 2 == 1:
print (3 * number + 1)
return 3 * number + 1
print ('Please enter a number.')
n = int(input())
while n != 1:
n = collatz(n)
In Python, functions accept one or more arguments and return a single value or object. Most of the time they don't modify their arguments (and indeed your collatz function doesn't attempt to modify its argument).
As an example, this function accepts a variable x, and returns x**2.
def f(x):
return x**2
This function doesn't modify x in place, and the return value won't automatically get assigned to x. Automatic assignment to x would often be unhelpful, and it would be unclear what to do if your function accepted multiple arguments -- which one should get the return result?
You can call this function in various ways, but if you want to do something with the result, you have to store it to a variable or use it immediately:
y = 2
z = f(y)
z = f(2)
y = 2
print(f(y))
Note that all of these make sense if you think of the function f as an object that converts its argument to something else and returns that, but none of them make sense if you expect f to modify its argument in place (then f(2) would somehow have to convert the number 2 to mean 4 during later references).
For what it's worth, even if you did replace one of the arguments with a new value inside the function, that would not change the value of the corresponding variable outside the function. This is because the variables within the function only point to the corresponding value or object. If you assign a new value to the variable, the local variable within the function will now point to the new value, but the original variable outside the function still points to the old value. On the other hand, you can sometimes modify the underlying value or object rather than creating a new object and pointing the local variable to it. For example, adding an item to a list or dictionary will modify the underlying object, and that change will be visible outside your function.
But your collatz function does neither of these - it just calculates a new value and returns it. If you want to do anything with that value, you have to store the result of the function call explicitly. It won't automatically be stored in the argument variable.
When you pass a variable as an argument to a function, a copy of the variable is sent to the function and not the variable itself.
So in your case n_copy (for example) is sent to your function and not n.
Now when you modify it within the function it remains in the scope of the function (accessible only by the function) and not the main program.
So when the function ends, nothing happens to n because a copy of n was modified.
Now we come to the return function. Because of the above problem, there is a return function. This will return a value from the function to the main program.
As you modified n within your function, you need to return the modified value to the main program.
Once you return it to the main program, it has to be stored in a variable, in your case it is n.
As you have started learning Python, you should read about namespace, scopes also.
Here is the first link from google search
https://matthew-brett.github.io/teaching/global_scope.html
This question already has answers here:
Creating lambda inside a loop [duplicate]
(3 answers)
Closed 6 years ago.
I'm encountering some strange behavior with lambda functions in a loop in python. When I try to assign lambda functions to dictionary entries in a list, and when other entries in the dictionary are used in the function, only the last time through the loop is the lambda operator evaluated. So all of the functions end up having the same value!
Below is stripped-down code that captures just the parts of what I'm trying that is behaving oddly. My actual code is more complex, not as trivial as this, so I'm looking for an explanation and, preferably, a workaround.
n=4
numbers=range(n)
entries = [dict() for x in numbers]
for number, entry in zip(numbers,entries):
n = number
entry["number"] = n
entry["number2"] = lambda x: n*1
for number in numbers:
print(entries[number]["number"], entries[number]["number2"](2))
The output is:
0 3
1 3
2 3
3 3
In other words, the dictionary entires that are just integers are fine, and were filled properly by the loop. But the lambda functions — which are trivial and should just return the same value as the "number" entries — are all set to the last pass through.
What's going on?
Try this
N=4
numbers=range(N)
entries = [dict() for x in numbers]
for number, entry in zip(numbers,entries):
entry["number"] = number
entry["number2"] = lambda x,n=number: n*1
for number in numbers:
print(entries[number]["number"], entries[number]["number2"](2))
It prints (python3)
0 0
1 1
2 2
3 3
To avoid confusion, n referred to different things in your code. I used it only at one place.
It is a closure problem.
By the end of your for loop, the n variable - which, unlike in static languages such as C#, is set to 3, which is then being accessed in the lambda expression. The variable value is not fixed; as another answer on the site points out, lambda expressions are fluid and will retain references to the variables involved instead of capturing the values at the time of creation. This question also discusses your issue.
To fix it, you need to give the lambdas new, local variable via default parameters:
entry["number2"] = lambda x, n=n: n*1
This creates a new variable in the lambda's scope, called n, which sets its default value to the "outside" value of n. Note that this is the solution endorsed by the official FAQ, as this answer by Adrien Plisson states.
Now, you can call your lambda like normal and ignore the optional parameter, with no ill effect.
EDIT: As originally stated by Sci Prog, this solution makes n = number redundant. Your final code will look similar to this:
lim = 4
numbers = range(lim)
entries = [dict() for x in numbers]
for number, entry in zip(numbers, entries):
entry["number"] = number
entry["number2"] = lambda x, n = number: n*1
for number in numbers:
print(entries[number]["number"], entries[number]["number2"](2))
You are probably reaching the problem that the method is created as referencing a variable n. The function is only evaluated after the loop so you are going to call the function which references n. If you're ok with having the function evaluated at the time of assignment you could put a function call around it:
(lambda x: n*1)(2)
or if you want to have the functions to use, have them reference the specific value you want. From your code you could use a default argument as a workaround:
entry["number"] = n
entry["number2"] = lambda x, n=n: n*1
The difference comes down to a question of memory addressing. I imagine it went something like this:
You: Python, please give me a variable called "n"
Python: Ok! Here it is, it is at memory slot 1
You: Cool! I will now create functions which say take that variable "n"
value (at memory slot 1) and multiply it by 1 and return that to me.
Python: Ok! Got it:
1. Take the value at memory slot 1.
2. Multiply by 1.
3. Return it to you.
You: Done with my looping, now evaluate those instructions!
Python: Ok! Now I will take the value of at memory slot 1 and multiply by 1
and give that to you.
You: Hey, I wanted each function to reference different values!
Python: I followed your instructions exactly!
This question already has answers here:
Use value of variable in lambda expression [duplicate]
(3 answers)
Closed 7 years ago.
I've experienced some strange behaviour when storing lambda functions into a dictionary: If you try to pass some default value to a function in a loop, only the last default value is being used.
Here some minimal example:
#!/usr/bin/env python
# coding: utf-8
def myfct(one_value, another_value):
"do something with two int values"
return one_value + another_value
fct_dict = {'add_{}'.format(number): (lambda x: myfct(x, number))
for number in range(10)}
print('add_3(1): {}, id={}'.format(fct_dict['add_3'](1), id(fct_dict['add_3'])))
print('add_5(1): {}, id={}'.format(fct_dict['add_5'](1), id(fct_dict['add_5'])))
print('add_9(1): {}, id={}'.format(fct_dict['add_9'](1), id(fct_dict['add_9'])))
The output reads as follows
add_3(1): 10, id=140421083875280
add_5(1): 10, id=140421083875520
add_9(1): 10, id=140421083876000
You get dissimilar functions (id not identical) but every function uses the same second argument.
Can somebody explain what's going on?
The same holds with python2, python3, pypy...
The fix:
def make_closure(number):
return lambda x: myfct(x, number)
used as
{'add_{}'.format(number): make_closure(number) for number in range(10)}
The reason for this behaviour is, that the variable number (think: named memory location here) is the same during all iterations of the loop (though its actual value changes in each iteration). "Loop" here refers to the dictionary comprehension, which internally is based on a loop. All lambda instances created in the loop will close over the same "location", which retains the value last assigned to it (in the last iteration of the loop).
The following code is not what actually happens underneath. It is merely provided to shed light on the concepts:
# Think of a closure variable (like number) as being an instance
# of the following class
class Cell:
def __init__(self, init=None):
self.value = None
# Pretend, the compiler "desugars" the dictionary comprehension into
# something like this:
hidden_result_dict = {}
hidden_cell_number = Cell()
for number in range(10):
hidden_cell_number.value = number
hidden_result_dictionary['add_{}'.format(number)] = create_lambda_closure(hidden_cell_number)
All lambda closures created by the create_lambda_closure operation share the very same Cell instance and will grab the value attribute at run-time (i.e., when the closure is actually called). By that time, value will refer to last value ever assigned to it.
The value of hidden_result_dict is then answered as the result of the dict comprehension. (Again: this is only meant as be read on a "conceptual" level; it has no relation to the actual code executed by the Python VM).
number is a variable which has different value for each iteration of the dict comprehension. But when you do lambda x: myfct(x, number), it does not use value of number. It just creates a lambda method that will use value of number when it will be called/used. So when you use you add_{} methods, number has value 9 which is used in every call to myfct(x, number).
I am quite confused on this, so I hope someone can help me.
So what I have is basically a list in Python, then I have set a variable which within I have used the list name and called the first element such as myVariable = myList[0], now basically what I want to do is to add an integer value to myVariable to make it so that I get the next value in the list myList[1] and subtract the list as a whole by one each time. I have 8 different elements/items in the list so far.
I keep getting an error usually something along the lines that I can't add an integer (the number value) to the myVariable as it sees it as a string, and I'm unsure on how to get the list to move one result forward and once that's done then subtract the list by one result as it is intended to be removed each time it loops.
Example:
def testFunction(testValue):
myVariable = myList[0]
testResult (myVariable + testValue) - 1
print testResult
The testValue being an integer, I need to get the value from myList depending on the integer value of testValue (which it's being added with), so if the testValue was like 3 it would add that to the default myVariable value which is currently 0, then after that result from the list and continue doing this until there is only one result left in this list, which currently consists of 8 elements.
UPDATE:
Sample list:
["KaPow", "Boom", "Pow", "Shapow", "Crash", "Whoosh", "Bang", "Pew"]
I enter a value into a function such as 3, function below:
bam(bamtypes, choice):
I enter the suggested value of 3 as "choice" and the bamtypes is just the given list above, then I should receive back the value Shapow. Then what I want to do is basically add myVariable to testValue (I referred to the integer of "choice" in the function when referring to testValue, so in this case it'll be 3). I also referred to bamtypes when saying myList.
As myVariable basically set to tell the function to begin from the first element in the list such as myVariable = bamtypes[0], but then I want the 3 value from "choices" to be added to the myVariable, so that it becomes bamtypes[3] then I can use .pop to extract that value, then does it again so it is bamtypes[6] and so on until the list only leaves one result. Also note as the list has 8 elements, what I want to do is once the 3 adds to the bamtypes[6] it should give me the value of [0] again as it resets, counting twice to the 8th element then returns once more to 0, using up the value 3 for that given retrieval.
When you are doing an arithmetic operation, you can either store the result in a variable or ignore the result. If we want to store in a variable, we need to use assignment operator, like this
testResult = (myVariable + testValue) - 1 # Note the `=` symbol
The error you get means that the data stored in myVariable is not a number but a character string. So, you should convert that to an integer like this
testResult = (int(myVariable) + testValue) - 1 # Note the `int` function
This question already has answers here:
"Least Astonishment" and the Mutable Default Argument
(33 answers)
Closed 6 months ago.
#!/usr/bin/env python3.2
def f1(a, l=[]):
l.append(a)
return(l)
print(f1(1))
print(f1(1))
print(f1(1))
def f2(a, b=1):
b = b + 1
return(a+b)
print(f2(1))
print(f2(1))
print(f2(1))
In f1 the argument l has a default value assignment, and it is only evaluated once, so the three print output 1, 2, and 3. Why f2 doesn't do the similar?
Conclusion:
To make what I learned easier to navigate for future readers of this thread, I summarize as the following:
I found this nice tutorial on the topic.
I made some simple example programs to compare the difference between mutation, rebinding, copying value, and assignment operator.
This is covered in detail in a relatively popular SO question, but I'll try to explain the issue in your particular context.
When your declare your function, the default parameters get evaluated at that moment. It does not refresh every time you call the function.
The reason why your functions behave differently is because you are treating them differently. In f1 you are mutating the object, while in f2 you are creating a new integer object and assigning it into b. You are not modifying b here, you are reassigning it. It is a different object now. In f1, you keep the same object around.
Consider an alternative function:
def f3(a, l= []):
l = l + [a]
return l
This behaves like f2 and doesn't keep appending to the default list. This is because it is creating a new l without ever modifying the object in the default parameter.
Common style in python is to assign the default parameter of None, then assign a new list. This gets around this whole ambiguity.
def f1(a, l = None):
if l is None:
l = []
l.append(a)
return l
Because in f2 the name b is rebound, whereas in f1 the object l is mutated.
This is a slightly tricky case. It makes sense when you have a good understanding of how Python treats names and objects. You should strive to develop this understanding as soon as possible if you're learning Python, because it is central to absolutely everything you do in Python.
Names in Python are things like a, f1, b. They exist only within certain scopes (i.e. you can't use b outside the function that uses it). At runtime a name refers to a value, but can at any time be rebound to a new value with assignment statements like:
a = 5
b = a
a = 7
Values are created at some point in your program, and can be referred to by names, but also by slots in lists or other data structures. In the above the name a is bound to the value 5, and later rebound to the value 7. This has no effect on the value 5, which is always the value 5 no matter how many names are currently bound to it.
The assignment to b on the other hand, makes binds the name b to the value referred to by a at that point in time. Rebinding the name a afterwards has no effect on the value 5, and so has no effect on the name b which is also bound to the value 5.
Assignment always works this way in Python. It never has any effect on values. (Except that some objects contain "names"; rebinding those names obviously effects the object containing the name, but it doesn't affect the values the name referred to before or after the change)
Whenever you see a name on the left side of an assignment statement, you're (re)binding the name. Whenever you see a name in any other context, you're retrieving the (current) value referred to by that name.
With that out of the way, we can see what's going on in your example.
When Python executes a function definition, it evaluates the expressions used for default arguments and remembers them somewhere sneaky off to the side. After this:
def f1(a, l=[]):
l.append(a)
return(l)
l is not anything, because l is only a name within the scope of the function f1, and we're not inside that function. However, the value [] is stored away somewhere.
When Python execution transfers into a call to f1, it binds all the argument names (a and l) to appropriate values - either the values passed in by the caller, or the default values created when the function was defined. So when Python beings executing the call f3(5), the name a will be bound to the value 5 and the name l will be bound to our default list.
When Python executes l.append(a), there's no assignment in sight, so we're referring to the current values of l and a. So if this is to have any effect on l at all, it can only do so by modifying the value that l refers to, and indeed it does. The append method of a list modifies the list by adding an item to the end. So after this our list value, which is still the same value stored to be the default argument of f1, has now had 5 (the current value of a) appended to it, and looks like [5].
Then we return l. But we've modified the default list, so it will affect any future calls. But also, we've returned the default list, so any other modifications to the value we returned will affect any future calls!
Now, consider f2:
def f2(a, b=1):
b = b + 1
return(a+b)
Here, as before, the value 1 is squirreled away somewhere to serve as the default value for b, and when we begin executing f2(5) call the name a will become bound to the argument 5, and the name b will become bound to the default value 1.
But then we execute the assignment statement. b appears on the left side of the assignment statement, so we're rebinding the name b. First Python works out b + 1, which is 6, then binds b to that value. Now b is bound to the value 6. But the default value for the function hasn't been affected: 1 is still 1!
Hopefully that's cleared things up. You really need to be able to think in terms of names which refer to values and can be rebound to point to different values, in order to understand Python.
It's probably also worth pointing out a tricky case. The rule I gave above (about assignment always binding names with no effect on the value, so if anything else affects a name it must do it by altering the value) are true of standard assignment, but not always of the "augmented" assignment operators like +=, -= and *=.
What these do unfortunately depends on what you use them on. In:
x += y
this normally behaves like:
x = x + y
i.e. it calculates a new value with and rebinds x to that value, with no effect on the old value. But if x is a list, then it actually modifies the value that x refers to! So be careful of that case.
In f1 you are storing the value in an array or better yet in Python a list where as in f2 your operating on the values passed. Thats my interpretation on it. I may be wrong
Other answers explain why this is happening, but I think there should be some discussion of what to do if you want to get new objects. Many classes have the method .copy() that allows you create copies. For instance, if we rewrite f1 as
def f1(a, l=[]):
new_l = l.copy()
new_l.append(a)
return(new_l)
then it will continue to return [1]no matter how many times we call it. There is also the library https://docs.python.org/3/library/copy.html for managing copies.
Also, if you're looping through the elements of a container and mutating them one by one, using comprehensions not only is more Pythonic, but can avoid the issue of mutating the original object. For instance, suppose we have the following code:
data = [1,2,3]
scaled_data = data
for i, value in enumerate(scaled_data):
scaled_data[i] = value/sum(data)
This will set scaled_data to [0.16666666666666666, 0.38709677419354843, 0.8441754916792739]; each time you set a value of scaled_data to the scaled version, you also change the value in data. If you instead have
data = [1,2,3]
scaled_data = [x/sum(data) for x in data]
this will set scaled_data to [0.16666666666666666, 0.3333333333333333, 0.5] because you're not mutating the original object but creating a new one.