We Keep Coding

Python lambda - too much optimized [duplicate]

This question already has answers here:
What do lambda function closures capture?
(7 answers)
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 6 years ago.
I have a back-ground of C++ and trying to learn some python.
Whilst i understand virtual functions for C++, I unfortunately do not understand what is meant by late binding of closures in python.
Link: https://gist.github.com/deemson/8efabf56d67623ead804 (no longer works)
Copy-pasted from a tutorial:
functions = []
for n in [1, 2, 3]:
def func(x):
return n*x
functions.append(func)
# You would expect this to print [2, 4, 6]
print(
'calling a list of bad closures and output is: {}'
.format(str([function(2) for function in functions]))
)
What exactly is happening here? When the function is appended in to the list, what values does it have? Can someone please simplify this code for me to understand?

Notice this, you can create functions on runtime, more or less like lambdas in c++. So basically you are iterating over a list, making n take values 1,2 and 3
for n in [1, 2, 3]:
def func(x):
return n*x
so, by each iteration you are building a function named func, with takes a value and multiplies it for n. By appending it to the functions list you will have this functions stored, so you can iterate over the list to call the functions.
[function(2) for function in functions]
By doing this you call each of the functions stored with the value 2, you would expect this to output [2, 4, 6] ([1*2, 2*2, 3*2]), but instead it returns [6, 6, 6], WHY?, thats because every function use n for its computation, so they are not really doing 1*x, 2*x and 3*x but actually n*x and since n is bonded in last time to 3 all functions are doing 3*2 which becomes 6.
Play around with the python console to check it properly.

In the language of C++, a pointer to the function is what's being appended to the list. After the for loop, functions contains pointers to three different functions (func(x) = n * x, func(x) = n * x and func(x) = n * x). Note the dependency on n. As n changes, so will the behavior of these functions, and they are all equivalent.
In the second part of the code, the pointers are extracted from the list and each of the three functions is evaluated with an argument of 2.
Here's a further example to clarify. Imagine we do this:
>>> functions
[<function func at 0x0239AA70>, <function func at 0x0239AAB0>, <function func at 0x0239AB30>]
>>> g = functions[2]
>>> g
<function func at 0x0239AB30>
>>> g(10)
20
>>> g(100)
200
What we're seeing in that first lines is that functions contains pointers to three different functions. The next line extracts the third pointer from the list (which refers to func(x) = n * x) and assigns it to g. Effectively, we have defined a function g(x) = n * x with this call. We can now evaluate g with arguments.
Note that since all functions depend on n, you could change n, and the behavior would change.
>>> n = 100
>>> g(10)
1000

Creating a copy of a function with some vars fixed

Assume I have a function
def multiply_by(x, multiplier):
return x * multiplier
How can I create a copy of that function and fix the multiplier in that function?
multiply_by_5 = multiply_by? <-- here I need python magic
such that multiply_by_5 would have only one argument x and the multiplier would be 5? So that
multiply_by_5(2)
10
Is there a way in Python 2.7 to do that?

You can use functools.partial with keyword argument:
>>> def multiply_by(x, multiplier):
... return x * multiplier
...
>>> from functools import partial
>>> multiply_by_5 = partial(multiply_by, multiplier=5)
>>> multiply_by_5(2)
10

functools.partial is made exactly for this.
you can use it like
import functools
multiply_by_5=functools.partial(multiply_by,multiplier=5)

As suggested by #niemmi's answer, functools.partial is probably the way to go.
However, similar work can be done using curried functions:
def multiply_by(multiplier):
def multiply(x):
return multiplier * x
return multiply
>>> multiply_by_5 = multiply_by(5) # no magic
>>> multiply_by_5(2)
10
Or using the lambda syntax:
def multiply_by(multiplier):
return lambda x: multiplier * x
Note that partial is more succinct, more efficient, and more directly express your intent in a standard way. The above technique is an example of the concept called closure, which is means that a function defined in inner scope may refer to variables defined in enclosing scopes, and "close" over them, remembering them, and even mutating them.
Since this technique is more general, it might take the reader of your code more time to understand what exactly do you mean in your code, since your code may be arbitrarily complicated.
Specifically for multiplication (and other operators) partial can be combined with operator.mul:
>>> import functools, operator
>>> multiply_by_5 = functools.partial(operator.mul, 5)
>>> multiply_by_5(2)
10

Here's an alternative that doesn't use functools.partial. Instead we define a function inside a function. The inner function "remembers" any of the local variables of the outer function that it needs (including the outer function's arguments). The magic that makes this happen is called closure.
def multiply_factory(multiplier):
def fixed_multiply(x):
return x * multiplier
return fixed_multiply
multiply_by_3 = multiply_factory(3)
multiply_by_5 = multiply_factory(5)
for i in range(5):
print(i, multiply_by_3(i), multiply_by_5(i))
output
0 0 0
1 3 5
2 6 10
3 9 15
4 12 20
If you want, you can use your existing multiply_by function in the closure, although that's slightly less efficient, due to the overhead of an extra function call. Eg:
def multiply_factory(multiplier):
def fixed_multiply(x):
return multiply_by(x, multiplier)
return fixed_multiply
That can be written more compactly using lambda syntax:
def multiply_factory(multiplier):
return lambda x: multiply_by(x, multiplier)

If you cannot change the multiply_by() function, the simplest and perhaps best way is probably
def multiply_by_5(x):
return multiply_by(x, 5)
You can also use lambda if you really want a one-liner.
However, you may want to change your first function to
def multiply_by(x, multiplier = 5):
return x * multiplier
Then you can do either of these:
print(multiply_by(4, 3))
12
print(multiply_by(2))
10

Are python parameters actually aliases to arguments? [duplicate]

This question already has answers here:
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 6 months ago.
Please consider the code below.
a = []
def func1(x):
return x
for i in range(3):
def func2():
return func1(i)
a.append(func2)
for k in range(3):
print(a[k]())
This prints out
2
2
2
From 'The use of aliases' in http://gestaltrevision.be/wiki/python/aliases (last section) and in 'Scope' section in http://gestaltrevision.be/wiki/python/functions_basics, I learnt that function parameters are actually aliases of arguments that are passed.
So according to that, in
def func1(x): return x
for i in range(3):
def func2(): return func1(i)
I reasoned since x would be stored as an alias to i, even though i is reassigned each time the loop is executed, it would not matter to its alias, x.
So I expected the first three lines to output 0, 1, 2 instead of 2, 2, 2.
Can you explain what I did wrong here? Thanks

You create here a closure func2 that uses variable i from the enclosing scope.
The func2's instances are created by DEF statement, at the time of the FOR loop execution.
Then you execute the func2's after the FOR loop is exited.
In python a loop variable doesn't destroyed after the loop exit.
So your closure uses the current value of the i in enclosing scope, at the moment of exiting from the loop.
So in this code func1 changes nothing, the result will be the same without it.

If you want your code to work the way you want, do as follows
def func2(i):
def func1():
return i
return func1
a = [func2(i) for i in range(3)]
for k in range(3):
print(a[k]()) # prints 0 1 2
Now, why didn't your code work? Well it has to do with when objects are bound in to names in a closure, which func1 is. In your code the parameter x to func1 is being bound at runtime. Hence as each function in a has func1(i) and the value of i at printing time is 2, you get all 2. So the solution is to bind it at compile time i.e when func2 returns func1, i is already bound in func1.

When you do this:
for i in range(3):
def func2():
return func1(i)
You're "redefining" your func2 for every i in [0, 1, 2]. The final definition that lives is:
def func2():
return func1(2)
It's as simple as that. Unfortunately it does not behave the way you expect.

Python functions within lists

So today in computer science I asked about using a function as a variable. For example, I can create a function, such as returnMe(i) and make an array that will be used to call it. Like h = [help,returnMe] and then I can say h1 and it would call returnMe("Bob"). Sorry I was a little excited about this. My question is is there a way of calling like h.append(def function) and define a function that only exists in the array?
EDIT:
Here Is some code that I wrote with this!
So I just finished an awesome FizzBuzz with this solution thank you so much again! Here's that code as an example:
funct = []
s = ""
def newFunct(str, num):
return (lambda x: str if(x%num==0) else "")
funct.append(newFunct("Fizz",3))
funct.append(newFunct("Buzz",5))
for x in range(1,101):
for oper in funct:
s += oper(x)
s += ":"+str(x)+"\n"
print s

You can create anonymous functions using the lambda keyword.
def func(x,keyword='bar'):
return (x,keyword)
is roughly equivalent to:
func = lambda x,keyword='bar':(x,keyword)
So, if you want to create a list with functions in it:
my_list = [lambda x:x**2,lambda x:x**3]
print my_list[0](2) #4
print my_list[1](2) #8

Not really in Python. As mgilson shows, you can do this with trivial functions, but they can only contain expressions, not statements, so are very limited (you can't assign to a variable, for example).
This is of course supported in other languages: in Javascript, for example, creating substantial anonymous functions and passing them around is a very idiomatic thing to do.

You can create the functions in the original scope, assign them to the array and then delete them from their original scope. Thus, you can indeed call them from the array but not as a local variable. I am not sure if this meets your requirements.
#! /usr/bin/python3.2
def a (x): print (x * 2)
def b (x): print (x ** 2)
l = [a, b]
del a
del b
l [0] (3) #works
l [1] (3) #works
a (3) #fails epicly

You can create a list of lambda functions to increment by every number from 0 to 9 like so:
increment = [(lambda arg: (lambda x: arg + x))(i) for i in range(10)]
increment[0](1) #returns 1
increment[9](10) #returns 19
Side Note:
I think it's also important to note that this (function pointers not lambdas) is somewhat like how python holds methods in most classes, except instead of a list, it's a dictionary with function names pointing to the functions. In many but not all cases instance.func(args) is equivalent to instance.__dict__['func'](args) or type(class).__dict__['func'](args)

How do lexical closures work? [duplicate]

This question already has answers here:
What do lambda function closures capture?
(7 answers)
Closed 6 months ago.
While I was investigating a problem I had with lexical closures in Javascript code, I came along this problem in Python:
flist = []
for i in xrange(3):
def func(x): return x * i
flist.append(func)
for f in flist:
print f(2)
Note that this example mindfully avoids lambda. It prints "4 4 4", which is surprising. I'd expect "0 2 4".
This equivalent Perl code does it right:
my #flist = ();
foreach my $i (0 .. 2)
{
push(#flist, sub {$i * $_[0]});
}
foreach my $f (#flist)
{
print $f->(2), "\n";
}
"0 2 4" is printed.
Can you please explain the difference ?
Update:
The problem is not with i being global. This displays the same behavior:
flist = []
def outer():
for i in xrange(3):
def inner(x): return x * i
flist.append(inner)
outer()
#~ print i # commented because it causes an error
for f in flist:
print f(2)
As the commented line shows, i is unknown at that point. Still, it prints "4 4 4".

Python is actually behaving as defined. Three separate functions are created, but they each have the closure of the environment they're defined in - in this case, the global environment (or the outer function's environment if the loop is placed inside another function). This is exactly the problem, though - in this environment, i is modified, and the closures all refer to the same i.
Here is the best solution I can come up with - create a function creater and invoke that instead. This will force different environments for each of the functions created, with a different i in each one.
flist = []
for i in xrange(3):
def funcC(j):
def func(x): return x * j
return func
flist.append(funcC(i))
for f in flist:
print f(2)
This is what happens when you mix side effects and functional programming.

The functions defined in the loop keep accessing the same variable i while its value changes. At the end of the loop, all the functions point to the same variable, which is holding the last value in the loop: the effect is what reported in the example.
In order to evaluate i and use its value, a common pattern is to set it as a parameter default: parameter defaults are evaluated when the def statement is executed, and thus the value of the loop variable is frozen.
The following works as expected:
flist = []
for i in xrange(3):
def func(x, i=i): # the *value* of i is copied in func() environment
return x * i
flist.append(func)
for f in flist:
print f(2)

Here's how you do it using the functools library (which I'm not sure was available at the time the question was posed).
from functools import partial
flist = []
def func(i, x): return x * i
for i in range(3):
flist.append(partial(func, i))
for f in flist:
print(f(2))
Outputs 0 2 4, as expected.

look at this:
for f in flist:
print f.func_closure
(<cell at 0x00C980B0: int object at 0x009864B4>,)
(<cell at 0x00C980B0: int object at 0x009864B4>,)
(<cell at 0x00C980B0: int object at 0x009864B4>,)
It means they all point to the same i variable instance, which will have a value of 2 once the loop is over.
A readable solution:
for i in xrange(3):
def ffunc(i):
def func(x): return x * i
return func
flist.append(ffunc(i))

What is happening is that the variable i is captured, and the functions are returning the value it is bound to at the time it is called. In functional languages this kind of situation never arises, as i wouldn't be rebound. However with python, and also as you've seen with lisp, this is no longer true.
The difference with your scheme example is to do with the semantics of the do loop. Scheme is effectively creating a new i variable each time through the loop, rather than reusing an existing i binding as with the other languages. If you use a different variable created external to the loop and mutate it, you'll see the same behaviour in scheme. Try replacing your loop with:
(let ((ii 1)) (
(do ((i 1 (+ 1 i)))
((>= i 4))
(set! flist
(cons (lambda (x) (* ii x)) flist))
(set! ii i))
))
Take a look here for some further discussion of this.
[Edit] Possibly a better way to describe it is to think of the do loop as a macro which performs the following steps:
Define a lambda taking a single parameter (i), with a body defined by the body of the loop,
An immediate call of that lambda with appropriate values of i as its parameter.
ie. the equivalent to the below python:
flist = []
def loop_body(i): # extract body of the for loop to function
def func(x): return x*i
flist.append(func)
map(loop_body, xrange(3)) # for i in xrange(3): body
The i is no longer the one from the parent scope but a brand new variable in its own scope (ie. the parameter to the lambda) and so you get the behaviour you observe. Python doesn't have this implicit new scope, so the body of the for loop just shares the i variable.

The problem is that all of the local functions bind to the same environment and thus to the same i variable. The solution (workaround) is to create separate environments (stack frames) for each function (or lambda):
t = [ (lambda x: lambda y : x*y)(x) for x in range(5)]
>>> t[1](2)
2
>>> t[2](2)
4

I'm still not entirely convinced why in some languages this works one way, and in some another way. In Common Lisp it's like Python:
(defvar *flist* '())
(dotimes (i 3 t)
(setf *flist*
(cons (lambda (x) (* x i)) *flist*)))
(dolist (f *flist*)
(format t "~a~%" (funcall f 2)))
Prints "6 6 6" (note that here the list is from 1 to 3, and built in reverse").
While in Scheme it works like in Perl:
(define flist '())
(do ((i 1 (+ 1 i)))
((>= i 4))
(set! flist
(cons (lambda (x) (* i x)) flist)))
(map
(lambda (f)
(printf "~a~%" (f 2)))
flist)
Prints "6 4 2"
And as I've mentioned already, Javascript is in the Python/CL camp. It appears there is an implementation decision here, which different languages approach in distinct ways. I would love to understand what is the decision, exactly.

The variable i is a global, whose value is 2 at each time the function f is called.
I would be inclined to implement the behavior you're after as follows:
>>> class f:
... def __init__(self, multiplier): self.multiplier = multiplier
... def __call__(self, multiplicand): return self.multiplier*multiplicand
...
>>> flist = [f(i) for i in range(3)]
>>> [g(2) for g in flist]
[0, 2, 4]
Response to your update: It's not the globalness of i per se which is causing this behavior, it's the fact that it's a variable from an enclosing scope which has a fixed value over the times when f is called. In your second example, the value of i is taken from the scope of the kkk function, and nothing is changing that when you call the functions on flist.

The reasoning behind the behavior has already been explained, and multiple solutions have been posted, but I think this is the most pythonic (remember, everything in Python is an object!):
flist = []
for i in xrange(3):
def func(x): return x * func.i
func.i=i
flist.append(func)
for f in flist:
print f(2)
Claudiu's answer is pretty good, using a function generator, but piro's answer is a hack, to be honest, as it's making i into a "hidden" argument with a default value (it'll work fine, but it's not "pythonic").

I didn't like how solutions above created wrappers in the loop. Note: python 3.xx
flist = []
def func(i):
return lambda x: x * i
for i in range(3):
flist.append(func(i))
for f in flist:
print f(2)