I'm having a bit of trouble understanding how Python treats and evaluates lambdas at runtime.
Iteratively building up an integer
Consider the following code (Python 3.5.2):
x = 0
for iteration in range(3):
x = x + 1
print(x)
As expected, this prints 3. Here is my understanding of the way x changes over 3 iterations:
Initial Value: x
Iteration 1: x+1
Iteration 2: (x+1) + 1
Iteration 3: ((x+1) + 1) + 1
Iteratively building up a lambda
Consider the following code:
add3 = lambda x: x
for iteration in range(3):
add3 = lambda x: add3(x) + 1
print(add3(0))
Here is my understanding of the way add3 should change over 3 iterations:
Initial Value: lambda x: x
Iteration 1: lambda x: (lambda x: x)(x) + 1
Iteration 2: lambda x: (lambda x: (lambda x: x)(x) + 1)(x) + 1
Iteration 3: lambda x: (lambda x: (lambda x: (lambda x: x)(x) + 1)(x) + 1)(x) + 1
Instead, calling add3 causes the maximum recursion depth to be exceeded.
My first thought was that Python is dynamically looking up the function body from its name at call time, rather than storing the function's code as part of the function. However, even the following code does not work:
functionList = [lambda x: x] #Store each iteration separately
for i in range(3):
oldFunction = functionList[-1]
newFunction = lambda x: oldFunction(x) + 1 #Should be a completely new lambda object
functionList.append(newFunction)
print(functionList[-1](0)) #Causes infinite recursion
Even with no named functions whatsoever, and following the suggestion here (although I may have misunderstood his answer), it still fails:
functionList = [lambda x: x]
for i in range(3):
functionList.append(lambda x, i=i: functionList[-1](x) + 1)
print(functionList[-1](0)) #Causes infinite recursion
The four lambdas contained in functionList are completely separate objects in memory:
>>> print(functionList)
[<function <lambda> at 0x00000266D41A12F0>, <function <lambda> at 0x00000266D41D7E18>, <function <lambda> at 0x00000266D41D7730>, <function <lambda> at 0x00000266D41D7840>]
Could someone please explain this behavior?
This behavior has nothing to do with 'iterational' lambda generation. When you say add3 = lambda x: add3(x) + 1, the add3 object is replaced with a lambda calling itself recursively with no termination condition.
So when you call add3(0), it becomes:
add3(0) = add3(0) + 1 = (add3(0) + 1) + 1 = ((add3(0) + 1) + 1) + 1
And this goes on forever (well, until the maximum recursion depth is exceeded).
As for other examples, the second function in your list already fails with RecursionError: maximum recursion depth exceeded.
I've got this code for you:
import copy
flist=[lambda x: x]
flist.append(lambda x: copy.deepcopy(flist[-1])(x) + 1)
>>> flist
[<function <lambda> at 0x101d45268>, <function <lambda> at 0x101bf1a60>]
So we made sure that we call a copy of a function. flist[-1](0) results in a RecursionError, and the error is raised in the deepcopy module. So, this means that copy.deepcopy(flist[-1])(x) means 'copy the last element currently in the list and run the copy'.
Here it is: the last element of the list calls itself over and over again.
You are correct that it is evaluated at run time. Because of that, add3, when referenced in itself, is calling itself, not the old add3. In the case of your loop, you are always using [-1]. Since it is evaluated at run time, the first one calls the one at the end of the list. The one at the end of the list then calls the one at the end of the list which calls the one at the end of the list... The list is not changing by the time the functions are being called. Therefore, the first function is called once, and then the last function is called infinitely. What you want is to use [i] instead of [-1]. It is very good that you used a default argument for i in your lambdas because default arguments are evaluated at definition.
Python allows you to access variables in the enclosing scope, but you are changing those variables during the loop.
add3 = lambda x: x
for iteration in range(3):
# This add3 will call itself!
# It always uses the *current* value of add3,
# NOT the value of add3 when it was created.
add3 = lambda x: add3(x) + 1
print(add3(0))
You will need to create a new binding:
add3 = lambda x: x
for _ in range(3):
add3 = (lambda f: lambda x: f(x) + 1)(add3)
print(add3(0))
This creates a new scope each time through the loop, which lets you bind with the current value of add3 rather than the future value of add3.
Languages which behave the way you describe are somewhat rare. Haskell is one example of such a language. In C++ you could achieve this by implicitly copying the lambdas using [=] for the capture list. Python just doesn't do this without some extra work.
Here's how add3 actually changes throughout the loop:
Initial Value: lambda x: x
Iteration 1: lambda x: add3(x) + 1
add3 is not immediately substituted inside the lambda body! It gets looked up when the function is executed.
Iteration 2: lambda x: add3(x) + 1 again
Iteration 3: still lambda x: add3(x) + 1
At the end, when you call add3, the recursive call to add3 inside the lambda body looks up the current value of add3, not the value when the lambda was defined. add3 calls itself, which calls itself, which calls itself, and so on until stack overflow.
Your attempted fix doesn't work because oldFunction still gets looked up at function execution time, so it still finds the final oldFunction instead of the oldFunction from function definition time. You didn't do anything to change that.
For comparison, when you do x = x + 1, x is immediately substituted, so in the first version, the values of x are successively 0, 1, 2, and 3. You don't have x = ((x+1) + 1) + 1 at the end; you just have x = 3.
Except for the name and name binding, the expression lambda <args>: <expression> creates a function that equals the result of def f(<args>): return <expression>. The statement f = lambda <args>: <expression> does not even have the name binding difference. However, some people falsely assume other differences. Many puzzles with lambda are illuminated by replacing it with equivalent defs. For the code above, we get the following.
def add3(x): return x
for iteration in range(3):
def add3(x): return add3(x) + 1
print(add3(0))
Does this make it more obvious that the second def negates the first, by rebinding add3 to a new function object? And that executing the second def statement more than once is redundant? Is it still a surprise that a recursive function with no terminal base case to stop it raises Recursion Error?
Related
I have no clue how the code executes. This is a question on a practice test.
(lambda x: x(x))(lambda y:4)
The output is 4 but I don't know how the code runs. I think the steps are as follows:
lambda (x) is defined
executes lambda (x)
returns x(lambda(y))
lambda (y) is defined
returns 4
x(4)?
I know step 6 is wrong.
This is a tricky question.
Lambdas are anonymous functions. Let's define them to help you understand what is happening.
# (lambda x: x(x))
def func_1(func):
return func(func)
# (lambda y: 4)
def func_2(y):
return 4
>>> func_1(func_2) # Equivalent to (lambda x: x(x))(lambda y:4)
4
The second function is just a callable that returns the constant value of 4 regardless of the value of y, so func_2(10) and func_2(0) both return 4.
The first function is a callable with itself as the single argument to the function (x(x)). We just saw that the second function is a callable that will return the constant value 4 regardless of the input argument, so func_2(func_2) simply returns 4.
lambda expression works just like an unnamed function (anonymous functions). And works like this:
lambda (1): (2)
(1) are parameters
(2) the return
See:
def square(x):
return x**2
Using lambda expression:
square = lambda x: x**2
>>> square(2)
4
But think about the use, maybe you just want to use an function a few times, why store it? Create an anonymous function! How? running a lambda function:
>>> (lambda x: x**2)(2)
4
See? Not that hard. Let's think together.
(lambda x: x(x))(lambda y:4)
lambda y: 4 is passed to our lambda x function, now x = (lambda y: 4) (crazy thing!). x(x) are equals to (lambda y: 4)((lambda y: 4)). Look! We got an parameter to our first 'y' it is (lambda y: 4)! then the first function are executed, and returns 4. What about the second? It doesn't run, and don't need to run! if you do x(x(x)) the second function will run, but not the third one.
Don't think about "lambda is defined". A lambda is a value, like 4. You don't "define" a 4 - it just is. When evaluating a lambda, just replace its (evaluated) arguments into its body (adding parentheses where neccessary). Work it like a rewriting problem.
(lambda x: x(x))(lambda y:4)
# rewrite all `x` in `x(x)` to `lambda y:4` (two appearances):
(lambda y:4)(lambda y:4)
# rewrite all `y` in `4` to `lambda y:4` (zero appearances):
4
This question already has answers here:
What do lambda function closures capture?
(7 answers)
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 6 months ago.
I am trying to make a program that creates a list of lambda functions of the format y=mx+b, where 'm' and 'b' are predetermined values
My overall goal is to implement a function that
Takes a picture
Finds the lines on it
Extends them across the whole picture in a solid colour
Basically, something like a Hough transforms if you know what that is.
Once I have the lines for a specific image, I can create a lambda function to represent the slope of the line and where it begins. I'm having an issue not being able to append a lambda function to the list.
I have tried this :
if __name__ == "__main__":
nums = []
for i in range(10):
j = lambda x: x + i
nums.append(j)
for i in nums:
print(i(1))
Here is the error I'm getting :
Traceback (most recent call last):
File "C:/Users/me/.PyCharmCE2018.3/config/scratches/scratch_3.py", line 7, in <module>
print(i(1))
File "C:/Users/me/.PyCharmCE2018.3/config/scratches/scratch_3.py", line 4, in <lambda>
j = (lambda x: x + i)
TypeError: unsupported operand type(s) for +: 'int' and 'function'
The problem is that the lambdas you create are referring to the current value of i in the active stack frame. When you later reuse i for the second for loop, it is bound to the lambdas in your list. When invoked as i(1), the lambdas are trying to evaluate 1 + i where i is the lambda, so of course you get an error.
Probably what you want is to freeze the value of i at the point at which the lambda is created. You can do this by replacing:
j = lambda x: x + i
with:
j = (lambda y: lambda x: x + y)(i)
This effectively captures the current value of i by binding it to a lambda variable, then immediately applying that lambda, after which the binding remains fixed.
This will give you a clue:
>>> i=1
>>> a=lambda x:x+i
>>> a(5)
6
>>> i=2
>>> a(5)
7
lambda uses i in the outer scope. In the OP case, all the functions are the same. Using i in the final loop makes i a function, not an integer. Change it to something else, and you'll find the functions are all the same, using the last value of i:
nums = []
for i in range(10):
j = lambda x: x + i
nums.append(j)
for f in nums:
print(f(1))
10
10
10
10
10
10
10
10
10
10
The fix is, make i a parameter to the function, to capture the value as a local variable:
nums = []
for i in range(10):
j = lambda x,i=i: x + i
nums.append(j)
for f in nums:
print(f(1))
1
2
3
4
5
6
7
8
9
10
Your value of i has changed and it's not what you think.
First you create lambda:
j = lambda x: x + i
in hope, that i will remain as CURRENT value (so 0, 1, 2 and so on).
Then you execute it:
print(i(1))
Do you see, how you named your second iterator variable i? Change it to j and your example will work. Why? Because python resolves value of i in your lambda, when you execute it, not when you define it. So when you execute your lambda (i(1)) it will go to your lambda body and try x + i. Then it will lookup i, which now contains your lambda (not the INTEGER!). Hence your problem.
You need to do double function to make it work properly. Try this:
if __name__ == "__main__":
nums = []
for i in range(10):
def generate_lambda(i):
return lambda x: x + i
j = generate_lambda(i)
nums.append(j)
for i in nums:
print(i(1))
Why does this work? When you call generate_lambda, there will be i variable with your INTEGER value. It will shadow variable i used later on to iterate over lambdas. And since you never modify i variable inside generate_lambda function, it will stay like this forever.
I think you need to learn something more about lambda functions...
Actually, it's syntax is like : [lambda arguments: expression]
So, the issue is you have two variables in the expression, so you need to pass two arguments.
I don't really get what you want to achieve by this function, but I guess you need to have two arguments for m and b.
In your code, you need to initialize x and pass it as an argument to lambda.
nums = []
x=0
for i in range(10):
j = lambda x,i : x + i
nums.append(j)
for i in nums:
print(i(1,1))
You can use operator.add and functools.partial and do not lambda at all:
import operator
import functools
if __name__ == "__main__":
nums = []
for i in range(10):
nums.append(functools.partial(operator.add, i))
for i in nums:
print(i(1))
In my code, I first define two lambda functions:
func1 = lambda x: x + 2
func2 = lambda x: x * 2
and later in my code, I want to create a lambda function that adds the results of the two lambda functions above, but this function must have the same name as one of the functions above. That is, I wanted to do something like this:
func2 = lambda x: func1(x) + func2(x)
Notice the function name func2 appears again.
I know this is silly and there are other ways of achieving the same desired outcome. However this example is just a very simplified version of what I tried to achieve. However, this won't work, as it gives me the error:
RecursionError: maximum recursion depth exceeded
I think that's because func2 keeps calling itself. Is there a way to prevent this from happening? I cannot change the names func1 or func2 (note that also means I cannot create two new functions with different names but the same content). I tried something like this
func2Copy = lambda x: func2(x)
func1Copy = lambda x: func1(x)
func2 = lambda x: func1Copy(x) + func2Copy(x)
but saw the same error (as expected). Is what I'm trying to do possible?
You could use some indirection
func2 = (lambda f1, f2: lambda x: f1(x) + f2(x))(func1, func2)
so then
>>> func2(1)
5
This is a bit silly though - the short story is you need another function.
You need to save a reference of the original functions. A quick and dirty way to achieve this is to make them default parameters to your lambda.
func2 = lambda x, func1=func1, func2=func2: func1(x) + func2(x)
Otherwise, a much more safe way to do this is to create a function that would return your altered function.
def new_func2(func1, func2):
return lambda x: func1(x) + func2(x)
func2 = new_func2(func1, func2)
Very rarely I'll come across some code in python that uses an anonymous function which returns an anonymous function...?
Unfortunately I can't find an example on hand, but it usually takes the form like this:
g = lambda x,c: x**c lambda c: c+1
Why would someone do this? Maybe you can give an example that makes sense (I'm not sure the one I made makes any sense).
Edit: Here's an example:
swap = lambda a,x,y:(lambda f=a.__setitem__:(f(x,(a[x],a[y])),
f(y,a[x][0]),f(x,a[x][1])))()
You could use such a construct to do currying:
curry = lambda f, a: lambda x: f(a, x)
You might use it like:
>>> add = lambda x, y: x + y
>>> add5 = curry(add, 5)
>>> add5(3)
8
swap = lambda a,x,y:(lambda f=a.__setitem__:(f(x,(a[x],a[y])),
f(y,a[x][0]),f(x,a[x][1])))()
See the () at the end? The inner lambda isn't returned, its called.
The function does the equivalent of
def swap(a, x, y):
a[x] = (a[x], a[y])
a[y] = a[x][0]
a[x] = a[x][1]
But let's suppose that we want to do this in a lambda. We cannot use assignments in a lambda. However, we can call __setitem__ for the same effect.
def swap(a, x, y):
a.__setitem__(x, (a[x], a[y]))
a.__setitem__(y, a[x][0])
a.__setitem__(x, a[x][1])
But for a lambda, we can only have one expression. But since these are function calls we can wrap them up in a tuple
def swap(a, x, y):
(a.__setitem__(x, (a[x], a[y])),
a.__setitem__(y, a[x][0]),
a.__setitem__(x, a[x][1]))
However, all those __setitem__'s are getting me down, so let's factor them out:
def swap(a, x, y):
f = a.__setitem__
(f(x, (a[x], a[y])),
f(y, a[x][0]),
f(x, a[x][1]))
Dagnamit, I can't get away with adding another assignment! I know let's abuse default parameters.
def swap(a, x, y):
def inner(f = a.__setitem__):
(f(x, (a[x], a[y])),
f(y, a[x][0]),
f(x, a[x][1]))
inner()
Ok let's switch over to lambdas:
swap = lambda a, x, y: lambda f = a.__setitem__: (f(x, (a[x], a[y])), f(y, a[x][0]), f(x, a[x][1]))()
Which brings us back to the original expression (plus/minus typos)
All of this leads back to the question: Why?
The function should have been implemented as
def swap(a, x, y):
a[x],a[y] = a[y],a[x]
The original author went way out of his way to use a lambda rather then a function. It could be that he doesn't like nested function for some reason. I don't know. All I'll say is its bad code. (unless there is a mysterious justification for it.)
It can be useful for temporary placeholders. Suppose you have a decorator factory:
#call_logger(log_arguments=True, log_return=False)
def f(a, b):
pass
You can temporarily replace it with
call_logger = lambda *a, **kw: lambda f: f
It can also be useful if it indirectly returns a lambda:
import collections
collections.defaultdict(lambda: collections.defaultdict(lambda: collections.defaultdict(int)))
It's also useful for creating callable factories in the Python console.
And just because something is possible doesn't mean that you have to use it.
I did something like this just the other day to disable a test method in a unittest suite.
disable = lambda fn : lambda *args, **kwargs: None
#disable
test_method(self):
... test code that I wanted to disable ...
Easy to re-enable it later.
This can be used to pull out some common repetitive code (there are of course other ways to achieve this in python).
Maybe you're writing a a logger, and you need to prepend the level to the log string. You might write something like:
import sys
prefixer = lambda prefix: lambda message: sys.stderr.write(prefix + ":" + message + "\n")
log_error = prefixer("ERROR")
log_warning = prefixer("WARNING")
log_info = prefixer("INFO")
log_debug = prefixer("DEBUG")
log_info("An informative message")
log_error("Oh no, a fatal problem")
This program prints out
INFO:An informative message
ERROR:Oh no, a fatal problem
It is most oftenly - at least in code I come accross and that I myself write - used to "freeze" a variable with the value it has at the point the lambda function is created. Otherwise, nonlocals variable reference a variable in the scope they exist, which can lead to undesied results sometimes.
For example, if I want to create a list of ten functions, each one being a multiplier for a scalar from 0 to 9. One might be tempted to write it like this:
>>> a = [(lambda j: i * j) for i in range(10)]
>>> a[9](10)
90
Whoever, if you want to use any of the other factoried functions you get the same result:
>>> a[1](10)
90
That is because the "i" variable inside the lambda is not resolved when the lambda is created. Rather, Python keeps a reference to the "i" in the "for" statement - on the scope it was created (this reference is kept in the lambda function closure). When the lambda is executed, the variable is evaluated, and its value is the final one it had in that scope.
When one uses two nested lambdas like this:
>>> a = [(lambda k: (lambda j: k * j))(i) for i in range(10)]
The "i" variable is evaluated durint the execution of the "for" loop. Itś value is passed to "k" - and "k" is used as the non-local variable in the multiplier function we are factoring out. For each value of i, there will be a different instance of the enclosing lambda function, and a different value for the "k" variable.
So, it is possible to achieve the original intent :
>>> a = [(lambda k: (lambda j: k * j))(i) for i in range(10)]
>>> a[1](10)
10
>>> a[9](10)
90
>>>
It can be used to achieve a more continuation/trampolining style of programming,
See Continuation-passing style
Basically, with this you can modify functions instead of values
One example I stumbled with recently: To compute approximate derivatives (as functions) and use it as an input function in another place.
dx = 1/10**6
ddx = lambda f: lambda x: (f(x + dx) - f(x))/dx
f = lambda x: foo(x)
newton_method(func=ddx(f), x0=1, n=10)
I'm trying to write a lambda-expression that calls itself, but i can't seem to find any syntax for that, or even if it's possible.
Essentially what I wanted to transfer the following function into the following lambda expression: (I realize it's a silly application, it just adds, but I'm exploring what I can do with lambda-expressions in python)
def add(a, b):
if a <= 0:
return b
else:
return 1 + add(a - 1, b)
add = lambda a, b: [1 + add(a-1, b), b][a <= 0]
but calling the lambda form of add results in a runtime error because the maximum recursion depth is reached. Is it even possible to do this in python? Or am I just making some stupid mistake? Oh, I'm using python3.0, but I don't think that should matter?
Maybe you need a Y combinator?
Edit - make that a Z combinator (I hadn't realized that Y combinators are more for call-by-name)
Using the definition of the Z combinator from Wikipedia
>>> Z = lambda f: (lambda x: f(lambda *args: x(x)(*args)))(lambda x: f(lambda *args: x(x)(*args)))
Using this, you can then define add as a completely anonymous function (ie. no reference to its name in its definition)
>>> add = Z(lambda f: lambda a, b: b if a <= 0 else 1 + f(a - 1, b))
>>> add(1, 1)
2
>>> add(1, 5)
6
Perhaps you should try the Z combinator, where this example is from:
>>> Z = lambda f: (lambda x: f(lambda *args: x(x)(*args)))(lambda x: f(lambda *args: x(x)(*args)))
>>> fact = lambda f: lambda x: 1 if x == 0 else x * f(x-1)
>>> Z(fact)(5)
120
First of all recursive lambda expressions are completely unnecessary. As you yourself point out, for the lambda expression to call itself, it needs to have a name. But lambda expressions is nothing else than anonymous functions. So if you give the lambda expression a name, it's no longer a lambda expression, but a function.
Hence, using a lambda expression is useless, and will only confuse people. So create it with a def instead.
But yes, as you yourself discovered, lambda expressions can be recursive. Your own example is. It's in fact so fantastically recursive that you exceed the maximum recursion depth. So it's recursive alright. Your problem is that you always call add in the expression, so the recursion never stops. Don't do that. Your expression can be expressed like this instead:
add = lambda a, b: a > 0 and (1 + add(a-1, b)) or b
Which takes care of that problem. However, your first def is the correct way of doing it.
add = lambda a, b: b if a <= 0 else 1 + add(a - 1, b)
You want the Y combinator, or some other fixed point combinator.
Here's an example implementation as a Python lambda expression:
Y = lambda g: (lambda f: g(lambda arg: f(f)(arg))) (lambda f: g(lambda arg: f(f)(arg)))
Use it like so:
factorial = Y(lambda f: (lambda num: num and num * f(num - 1) or 1))
That is, you pass into Y() a single-argument function (or lambda), which receives as its argument a recursive version of itself. So the function doesn't need to know its own name, since it gets a reference to itself instead.
Note that this does get tricky for your add() function because the Y combinator only supports passing a single argument. You can get more arguments by currying -- but I'll leave that as an exercise for the reader. :-)
a little late ... but I just found this gem # http://metapython.blogspot.com/2010/11/recursive-lambda-functions.html
def myself (*args, **kw):
caller_frame = currentframe(1)
code = caller_frame.f_code
return FunctionType(code, caller_frame.f_globals)(*args,**kw)
print "5! = "
print (lambda x:1 if n <= 1 else myself(n-1)*n)(5)