I've read about python scopes and browsed questions here on stackoverflow but theres something i'd like to clarify.
I'm extracting a piece of code to a function, from what i used to it should take all the variables it's using as parameters. But, in python the variable address is determined at runtime so theres actually no need for the parameters. Since i'm new to python i wanted to know if there are other implications or conventions i should know about.
x = 5
x += 1
print x
is there any difference between the following refactoring of the above code :
def f(x):
x += 1
return x
x = 5
x = f(x)
print x
and:
def f():
x++
x = 5
f()
print x
If not then, is one of the ways more commonly used or preferred in python ?
It's preferred not to use global variables, if not absolutely necessary. That said, in the second example you'd need global x declared before you refer to x.
So, first way:
f takes an argument x
Increments x
returns x + 1
the global x is not affected
The second way:
def f():
global x
x += 1
x = 1
f()
f has no arguments
Increments the global x
P.S. Python has no ++ operator. x += 1 is used instead
I upvote the question for two reasons:
1)
importing the notation x++ from other languages (C++ and Java) is a venial sin; who has never been absent-minded ?
not testing codes deserves a downvote, that's right, because it denotes that no tests have been performed to try to obtain oneself a certain number of observations and that's baaad
however I find the desire to understand notions concerning scopes, namespaces, global/local more commendable than the faults to be reproached
2)
It is certainly an unsatisfactory situation to be downvoted because of an approximate question while receiving an upvoted answer that contains itself some inadequate terms according to me. Terminology is particularly important in subjects in which controversial debates and confusionning descriptions may happen.
For exemple, for
def f(x):
x++
return x
I wouldn't say that f receives an argument x, but that f receives x as an argument.
x is not incremented; rather: another object is created with a value resulting of the incrementation of the value of initial x
f doesn't returns x+1 , it returns the new object with the incremented value
EDIT 1
# f takes an argument x
In the call f(x) , I wouldn't say that f receives an argument x, because x isn't an argument in the absolute. x "becomes" an argument only relatively to a function, at the moment when it is passed to the function. So I rather say that in the call f(x) , the function f receives x AS an argument.
It may happen that one says "the argument Y" one time, as an understatement to express "Y , considered at a moment when it is passed as an argument to a function". This understatement is shorter; but what I think is that in a precise explanation such an easy way to express must be banned.
#Increments x
Refering to the code written in the question (case 1), this sentence is ambiguous because of the function written with a parameter x: does x refer to the x in the function or the x outside ? You'll say I nitpick, but for a newbie that doesn't know the data model and the working of functions well, the question is valid. What does f(x) mean in Python ? , is the x passed by value or passed by reference ? It could be that the operations inside f could change the x outside, why not, that's precisely what is in discussion, no ? That's why it is preferable to name the parameter of f with a different name than any of the names of the outside objects; I think the same as for the previous sentence: in a precise explanation, only cautious sentences and codes should be employed, to avoid confusionning ambiguities:
def f(N):
N += 1
return N
x = 5
x = f(x)
print x
And it should be explained that the instruction N += 1 triggers the creation of a new object with value incremented and that the local identifier N is rebound to this new object. And at this point, the global identifier x is still bound to the same object as before the call.
# returns x + 1
f doesn't returns x+1 , it returns the new object newly assigned to N with the incremented value
# the global x is not affected
What does it mean ? Before , global x has a value 5. After, global x has a value 6. During the execution of the operations inside f, the global x isn't affected, that's right: it is only when the new object (in fact its address..) is returned to the outside of f, and that the assignement x = f(x) is executed that global x is affected. But this sentence "the global x is not affected" is placed after "returns x + 1
". No really, I understand nothing in such explanations.
End of EDIT 1
.
.
Concerning the second case
def f():
global x
x += 1
I prefer to say that f has no argument than it has no parameter
It doesn't increments the global x, it provokes a new assignement of the global identifier x to a new object with incremented value.
EDIT 2
In this case, it isn't possible to use another name than x inside the function, otherwise the exemple would mean another thing.
def f():
global x
x += 1
x = 5
f()
print x
# f has no arguments
Why the hell did I write "I prefer to say that f has no argument than it has no parameter" to comment that ? I still wonder.
At first, I had written "I prefer to say that f has no parameter than it has no argument" , thinking to the definition of f . I don't know why, I "corrected" by reversing the sentence and the result expresses nothing of what I think. I am completely puzzled.
Precisely, I think that the correct manners to express are :
in the definition of the function, f -> HAS no PARAMETER,
in the call f() , f -> RECEIVES no ARGUMENTS.
# Increments the global x
It doesn't increments the global x, it provokes a new assignement of the global identifier x to a new object with incremented value.
End of EDIT 2
.
It may seem minor nuances , and I am sure that it is a way to express things in a condensed manner.
But for me it is very important because, when a newbie, I had a lot of difficulty because of this kind of imprecision to understand all these questions linked to the peculiar data model of Python that are badly explained in the official docs of Python and that are debated in awkard discussions, such as the one concerning "pass by value or pass by reference" for exemple (that has no sense in Python), in which the terms are floating from right to left and left to right like on a sea of vagueness.
Related
I originally had a problem with this code as I was missing the 'n =' in the last line of code and, as a result, was stuck in an infinite loop.
At this point, while I understand what needed to be corrected I don't understand why. Why can't 'collatz(n)' be enough to call the function and use n as its variable? If anyone could explain this in simple terms (beginner here), I'd really appreciate it.
def collatz(number):
if number % 2 == 0:
print (number // 2)
return number // 2
elif number % 2 == 1:
print (3 * number + 1)
return 3 * number + 1
print ('Please enter a number.')
n = int(input())
while n != 1:
n = collatz(n)
In Python, functions accept one or more arguments and return a single value or object. Most of the time they don't modify their arguments (and indeed your collatz function doesn't attempt to modify its argument).
As an example, this function accepts a variable x, and returns x**2.
def f(x):
return x**2
This function doesn't modify x in place, and the return value won't automatically get assigned to x. Automatic assignment to x would often be unhelpful, and it would be unclear what to do if your function accepted multiple arguments -- which one should get the return result?
You can call this function in various ways, but if you want to do something with the result, you have to store it to a variable or use it immediately:
y = 2
z = f(y)
z = f(2)
y = 2
print(f(y))
Note that all of these make sense if you think of the function f as an object that converts its argument to something else and returns that, but none of them make sense if you expect f to modify its argument in place (then f(2) would somehow have to convert the number 2 to mean 4 during later references).
For what it's worth, even if you did replace one of the arguments with a new value inside the function, that would not change the value of the corresponding variable outside the function. This is because the variables within the function only point to the corresponding value or object. If you assign a new value to the variable, the local variable within the function will now point to the new value, but the original variable outside the function still points to the old value. On the other hand, you can sometimes modify the underlying value or object rather than creating a new object and pointing the local variable to it. For example, adding an item to a list or dictionary will modify the underlying object, and that change will be visible outside your function.
But your collatz function does neither of these - it just calculates a new value and returns it. If you want to do anything with that value, you have to store the result of the function call explicitly. It won't automatically be stored in the argument variable.
When you pass a variable as an argument to a function, a copy of the variable is sent to the function and not the variable itself.
So in your case n_copy (for example) is sent to your function and not n.
Now when you modify it within the function it remains in the scope of the function (accessible only by the function) and not the main program.
So when the function ends, nothing happens to n because a copy of n was modified.
Now we come to the return function. Because of the above problem, there is a return function. This will return a value from the function to the main program.
As you modified n within your function, you need to return the modified value to the main program.
Once you return it to the main program, it has to be stored in a variable, in your case it is n.
As you have started learning Python, you should read about namespace, scopes also.
Here is the first link from google search
https://matthew-brett.github.io/teaching/global_scope.html
This question already has answers here:
Creating lambda inside a loop [duplicate]
(3 answers)
Closed 6 years ago.
I'm encountering some strange behavior with lambda functions in a loop in python. When I try to assign lambda functions to dictionary entries in a list, and when other entries in the dictionary are used in the function, only the last time through the loop is the lambda operator evaluated. So all of the functions end up having the same value!
Below is stripped-down code that captures just the parts of what I'm trying that is behaving oddly. My actual code is more complex, not as trivial as this, so I'm looking for an explanation and, preferably, a workaround.
n=4
numbers=range(n)
entries = [dict() for x in numbers]
for number, entry in zip(numbers,entries):
n = number
entry["number"] = n
entry["number2"] = lambda x: n*1
for number in numbers:
print(entries[number]["number"], entries[number]["number2"](2))
The output is:
0 3
1 3
2 3
3 3
In other words, the dictionary entires that are just integers are fine, and were filled properly by the loop. But the lambda functions — which are trivial and should just return the same value as the "number" entries — are all set to the last pass through.
What's going on?
Try this
N=4
numbers=range(N)
entries = [dict() for x in numbers]
for number, entry in zip(numbers,entries):
entry["number"] = number
entry["number2"] = lambda x,n=number: n*1
for number in numbers:
print(entries[number]["number"], entries[number]["number2"](2))
It prints (python3)
0 0
1 1
2 2
3 3
To avoid confusion, n referred to different things in your code. I used it only at one place.
It is a closure problem.
By the end of your for loop, the n variable - which, unlike in static languages such as C#, is set to 3, which is then being accessed in the lambda expression. The variable value is not fixed; as another answer on the site points out, lambda expressions are fluid and will retain references to the variables involved instead of capturing the values at the time of creation. This question also discusses your issue.
To fix it, you need to give the lambdas new, local variable via default parameters:
entry["number2"] = lambda x, n=n: n*1
This creates a new variable in the lambda's scope, called n, which sets its default value to the "outside" value of n. Note that this is the solution endorsed by the official FAQ, as this answer by Adrien Plisson states.
Now, you can call your lambda like normal and ignore the optional parameter, with no ill effect.
EDIT: As originally stated by Sci Prog, this solution makes n = number redundant. Your final code will look similar to this:
lim = 4
numbers = range(lim)
entries = [dict() for x in numbers]
for number, entry in zip(numbers, entries):
entry["number"] = number
entry["number2"] = lambda x, n = number: n*1
for number in numbers:
print(entries[number]["number"], entries[number]["number2"](2))
You are probably reaching the problem that the method is created as referencing a variable n. The function is only evaluated after the loop so you are going to call the function which references n. If you're ok with having the function evaluated at the time of assignment you could put a function call around it:
(lambda x: n*1)(2)
or if you want to have the functions to use, have them reference the specific value you want. From your code you could use a default argument as a workaround:
entry["number"] = n
entry["number2"] = lambda x, n=n: n*1
The difference comes down to a question of memory addressing. I imagine it went something like this:
You: Python, please give me a variable called "n"
Python: Ok! Here it is, it is at memory slot 1
You: Cool! I will now create functions which say take that variable "n"
value (at memory slot 1) and multiply it by 1 and return that to me.
Python: Ok! Got it:
1. Take the value at memory slot 1.
2. Multiply by 1.
3. Return it to you.
You: Done with my looping, now evaluate those instructions!
Python: Ok! Now I will take the value of at memory slot 1 and multiply by 1
and give that to you.
You: Hey, I wanted each function to reference different values!
Python: I followed your instructions exactly!
What do the following expression actually does?
list = [lambda n=n: lambda x: x+n for n in range(10)]
More specifically:
What does n=n mean?
What will be the content of 'list'?
What will be the output of
print(list[0](14)) and print(list[0]()(14))
and why?
What does n=n mean?
lambda lets you define functions that take parameters, just like def. And those parameters can have default argument values. So, lambda n=n: is the same as def foo(n=n):.
In fact, when faced with an expression that's too complicated for you to read, it's often worth unpacking into simple statements:
list = []
for n in range(10):
def spam(n=n):
def eggs(x):
return x+n
return eggs
list.append(spam)
Now, why would you want to create a parameter named n with default value n? Why not just lambda:? The official FAQ explains this, but let's try to summarize.
If you just write this:
funcs = [lambda: n for n in range(10)]
… what you get is 10 functions of no parameters, that are all closures over the same variable, n. Because n has the value 9 at the end of the loop, when called, they're all going to return 9.
But if you do this:
funcs = [lambda n=n: n for n in range(10)]
… what you get is 10 functions of one optional parameter n (which hides the closure n from view), whose default value is the value of n at the time each function was defined. So, when called with no arguments, the first one will return 0, the second 1, and so on.
In your case, of course, the functions aren't just returning n, they're returning a function that takes a parameter, adds n to it, and returns the result. But the idea is the same; you want them to return different functions, which add 0, 1, … 9 to their arguments, not all return equal functions that all add 9.
What will be the content of list?
list will be 10 functions of one optional parameter whose default values range from 0 to 9, each of which returns a function of one parameter. That returned function is a closure over the value of n from the outer function. So, when it's called, it returns its argument, x, plus the n variable that ranges from 0 through 9.
What will be the output of
print(list[0](14))
Here, you're calling the first outer function, list[0], with the argument 14. So, instead of its default value 0 for n, it's going to have 14. So, what you'll get is a function that takes one argument and adds 14 to it. But it will print out as something like:
<function <listcomp>.<lambda>.<locals>.<lambda> at 0x105f21f28>
That long mess is Python 3.4+ trying to be helpful by telling you where to find the function definition. Usually, when a function is nested this deeply, most of the steps along the way have names. In this case, you've got three layers of anonymous functions, so none of the names are very useful…
In order to see it do anything useful, you'll have to call it:
print(list[0](14)(20))
And this will give you 34.
You could also use the inspect module, or just dir, to poke around inside the function. For example, print(list[0](14).__code__.co_freevars[0], list[0](14).__closure__[0].cell_contents) will tell you that it's stashed the number 14 under the name n for use by its internal function.
…
print(list[0]()(14))
Here, you're again calling list[0], but this time with no argument, so its n gets the default value of 0. So, it returns a function that adds 0 to its argument. You then call that function with 14, so you get 14.
To answer the last part first:
In [1]: list = [lambda n=n: lambda x: x+n for n in range(10)]
In [2]: print(list[0](14))
<function <lambda> at 0x7f47b5ca7cf8>
In [3]: print(list[0]()(14))
14
Obtained by running the code. list bad name by the way as list is a python builtin gives you 10 lambda functions that don't do much - the first will return the original argument x, the second the argument + 1, ect. as n is stored as the index of the lambda by n=n local to that lambda.
This question already has answers here:
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 6 months ago.
I am trying to use closures to eliminate a variable from a function signature (the application is to make writing all the functions needed for connecting Qt signals for an interface to control a largish number of parameters to the dictionary that stores the values ).
I do not understand why the case of using the lambda not wrapped in another function returns the last name for all cases.
names = ['a', 'b', 'c']
def test_fun(name, x):
print(name, x)
def gen_clousure(name):
return lambda x: test_fun(name, x)
funcs1 = [gen_clousure(n) for n in names]
funcs2 = [lambda x: test_fun(n, x) for n in names]
# this is what I want
In [88]: for f in funcs1:
....: f(1)
a 1
b 1
c 1
# I do not understand why I get this
In [89]: for f in funcs2:
....: f(1)
c 1
c 1
c 1
The reason is that closures (lambdas or otherwise) close over names, not values. When you define lambda x: test_fun(n, x), the n is not evaluated, because it is inside the function. It is evaluated when the function is called, at which time the value that is there is the last value from the loop.
You say at the beginning that you want to "use closures to eliminate a variable from a function signature", but it doesn't really work that way. (See below, though, for a way that may satisfy you, depending on what you mean by "eliminate".) Variables inside the function body will not be evaluated when the function is defined. In order to get the function to take a "snapshot" of the variable as it exists at function-definition time, you must pass the variable as an argument. The usual way to do this is to give the function an argument whose default value is the variable from the outer scope. Look at the difference between these two examples:
>>> stuff = [lambda x: n+x for n in [1, 2, 3]]
>>> for f in stuff:
... print f(1)
4
4
4
>>> stuff = [lambda x, n=n: n+x for n in [1, 2, 3]]
>>> for f in stuff:
... print f(1)
2
3
4
In the second example, passing n as an argument to the function "locks in" the current value of n to that function. You have to do something like this if you want to lock in the value in this way. (If it didn't work this way, things like global variables wouldn't work at all; it's essential that free variables be looked up at the time of use.)
Note that nothing about this behavior is specific to lambdas. The same scoping rules are in effect if you use def to define a function that references variables from the enclosing scope.
If you really want to, you can avoid adding the extra argument to your returned function, but to do so you must wrap that function in yet another function, like so:
>>> def makeFunc(n):
... return lambda x: x+n
>>> stuff = [makeFunc(n) for n in [1, 2, 3]]
>>> for f in stuff:
... print f(1)
2
3
4
Here, the inner lambda still looks up the value of n when it is called. But the n it refers to is no longer a global variable but a local variable inside the enclosing function makeFunc. A new value of this local variable is created every time makeFunc is called, and the returned lambda creates a closure that "saves" the local variable value that was in effect for that invocation of makeFunc. Thus each function created in the loop has its own "private" variable called x. (For this simple case, this can also be done using a lambda for the outer function --- stuff = [(lambda n: lambda x: x+n)(n) for n in [1, 2, 3]] --- but this is less readable.)
Notice that you still have to pass your n as an argument, it's just that, by doing it this way, you don't pass it as an argument to the same function that winds up going into the stuff list; instead you pass it as an argument to a helper function that creates the function you want to put into stuff. The advantage of using this two-function approach is that the returned function is "clean" and doesn't have the extra argument; this could be useful if you were wrapping functions that accepted a lot of arguments, in which case it could become confusing to remember where the n argument was in the list. The disadvantage is that, doing it this way, the process of making the functions is more complicated, since you need another enclosing function.
The upshot is that there is a tradeoff: you can make the function-creation process simpler (i.e., no need for two nested functions), but then you must make the resulting function a bit more complicated (i.e., it has this extra n=n argument). Or you can make the function simpler (i.e., it has no n=n argument), but then you must make the function-creation process more complicated (i.e., you need two nested functions to implement the mechanism).
I'm learning python using the tutorial on the official python website and came across this example:
>>> def make_incrementor(n):
... return lambda x: x + n
...
>>> f = make_incrementor(42)
>>> f(0)
42
>>> f(1)
43
Where does x get it's value from? I'm not familiar with how lambda works, I understand anonymous functions just fine from javascript but this has me stumped. Anyone care to shed some light? I'd be grateful.
Consider this. f is the object created by the make_incrementor function.
It is a lambda, an "anonymous function".
>>> f= lambda x: x+42
>>> f(10)
52
The value for x showed up when we applied f to a value.
First off, here's a translation into JavaScript:
function makeIncrementor(n) {
return function (x) {return x + n}
}
A lambda expression consists of three parts.
The word lambda (In JavaScript, the word function)
A list of parameters (In JavaScript, the part in (...))
Some expression to evaluate (In JavaScript, the part in {...}, automatically returned)
The list of parameters is the part before the colon. These parameters are visible within the scope of the expression. Whatever the expression evaluates to is returned as the result of the function.
Unlike in JavaScript, a lambda can only contain a single expression; you can't make an anonymous multi-line function with lambda.
Disclaimer: I have pretty much no Python background. This is going off my Scheme/lambda calculus knowledge.
make_incrementor defines a function to make functions. make_incrementor(42) returns a function with x bounded to the lambda, and n with a value of 42. When you call the function f, the argument of f replaces the bound variable.
lambda brings some lambda calculus to Python. In essence, this is what's happening: normally in lambda calculus a statement would look something like this:
sum = add 3 4
This would pass 3 and 4 to the add function and store the result in sum. However, we could also write something along the lines of add 3. Now, since add expects two arguments, we now get an object (or function, if you will) expecting one argument. That function will then call add with 3 as its first argument and whatever we pass it as the second argument. We can now do this:
func = add 3
sum = func 4
This will be equivalent to the previous example. However, you can now use func whenever you want to add 3 to something. (Granted, this doesn't seem useful in this example, but in more complex situations it is).
All this is closely related to currying, something very central in most functional languages. If you're interested in lambda calculus and its similarity to regular mathematics, I highly recommend that you take a look at Haskell.
When make_incrementor() is called it creates and returns a lambda function at that time. In the process the value of the argument n gets stored or remembered in the function object created. If you called it again with a different n a different function would be returned.
x gets its value from the parameter used to call f.
n gets its value from the parameter used to call make_incrementor.
make_incrementor is a function that returns a function:
def make_incrementor(n):
return lambda x: x + n
When we call make_incrementor, we bind the parameter 42 to n, and return the concrete function:
f = make_incrementor(42) # fill in n at this point and return a function
This function we will return will therefore look like:
lambda(x): x + 42
f is bound to the returned function, so f will conceptually look like:
def f(x):
return x + 42
When we call f, we provide the value for x
f(1) # x == 1
f(2) # x == 2
Here's how I understand lambda:
When we write x2, we often confuse two distinct ideas. Consider:
x2 has an odd number of factors when x is an integer.
x2 is larger than x when x>1.
x2 has derivative 2x.
x2 has an inverse, namely sqrt(x), for x>0.
The first two statements are about the square of a particular but unspecified number. In those statements "x" represents an arbitrary single thing, and x2 represents a single related thing.
The third and fourth statement are about x2, the function. But this is not clearly communicated in 3. and 4. except by context and shared understanding of Calculus. We need a notational device to distinguish between x2 the single (but arbitrary) value, and x2 the function. The device also needs to communicate that it is a function of x. Therefore lambda is invented:
"lambda x.x2" is written to make this distinction and communicate "the function of x whose value at x is x2". The lambda operator takes a name (x) and an expression (x2) and returns a function. This has the same consequences as normal function definition, except that the function doesn't automatically receive a name.