We Keep Coding

python lambda evaluate expression

I am trying out lambda in python and came across this question:
def foo(y):
return lambda x: x(x(y))
def bar(x):
return lambda y: x(y)
print((bar)(bar)(foo)(2)(lambda x:x+1))
can someone explain/breakdown how this code works? I am having problems trying to figure out what is x and y.

Lambda functions are just functions. They're almost syntatic sugar, as you can think of this structure:
anony_mouse = lambda x: x # don't actually assign lambdas
as equivalent to this structure:
def anony_mouse(x):
return x
(Almost, as there is no other way of getting a function without assigning it to some variable, and the syntax prevents you doing some things with them, such as using multiple lines.)
Thus let's write out the top example using standard function notation:
def foo(y):
# note that y exists here
def baz(x):
return x(x(y))
return baz
So we have a factory function, which generates a function which... expects to be called with a function (x), and returns x(x(arg_to_factory_function)). Consider:
>>> def add_six(x):
return x + 6
>>> bazzer = foo(3)
>>> bazzer(add_six) # add_six(add_six(3)) = 6+(6+3)
I could go on, but does that make it clearer?
Incidentally that code is horrible, and almost makes me agree with Guido that lambdas are bad.

The 1st ‘(bar)’ is equal to just ‘bar’ so it is an ordinary function call, the 2nd — argument to that call, i.e. bar(bar) — substitute ‘x’ to ‘bar’ there any you will get what is result of bar(bar); the’(foo)’ argument passing to the result of bar(bar) it will be a lambda-function with some arg. — substitute it to ‘foo’ and get result and so on until you reach the end of expression

I slightly modify your original function to make clearer what's going on (so it should be clearer which parameter is callable!)
# given a function it evaluates it at value p
def eval(func): # your foo
return lambda p: func(p)
# given a value p perform a double composition of the function at this value (2-step recursion)
def iter_2(p): # your bar
return lambda func: func(func(p))
increment = lambda x: x + 1 # variable binding only for readability
This example is quite hard to understand because one of the function, eval just do nothing special, and it composition is equivalent to the identity! ... so it could be quite confusing.
(foo)(2)(lambda x:x+1)):
x = 2
iter_2(x)(increment) # increment by 2 because iter_2 calls increment twice
# 4
idempotency: (or composition with itself return the identity function)
increment(3) == eval(increment)(3)
# True
# idempotency - second composition is equivalent to the identity
eval(increment)(3) == eval(eval)(increment)(3)
# True
eval(increment)(3) == eval(eval)(eval)(increment)(3)
# True
# ... and so on
final: consequence of idempotency -> bar do nothing, just confusion
eval(eval)(iter_2)(x)(increment) == iter_2(x)(increment)
# True
Remark:
in (bar)(bar)(foo)(2)(lambda x:x+1) you can omit the brackets around the 1st term, just bar(bar)(foo)(2)(lambda x:x+1)
Digression: [since you example is quite scaring]
Lambda functions are also known as anonymous function. Why this? Simply because that they don't need to be declared. They are designed to be single purpose, so you should "never" assign to a variable. The arise for example in the context of functional programming where the basic ingredients are... functions! They are used to modify the behavior of other functions (for example by decoration!). Your example it is just a standalone syntactical one... essentially a nonsense example which hides the truth "power" of the lambda functions. There is also a branch mathematics which based on them called lambda calculus.
Here a totally different example of application of the lambda functions, useful for decoration (but this is another story):
def action(func1):
return lambda func2: lambda p: func2(p, func1())
def save(path, content):
print(f'content saved to "{path}"')
def content():
return 'content' # i.e. from a file, url, ...
# call
action(content)(save)('./path')
# with each key-parameter would be
action(func1=content)(func2=save)(p='./path')
Output
content saved to "./path"

lambda function in different lines

I am a complete newbie in python.
I start doing lambda functions and they end up a bit longer than my initial goal:
Can I split it in different lines for better readability?, like this:
parts.map(lambda p: (p[0]\
,p[1]\
,int(p[1].split("-")[0])\
,int(p[1].split("-")[1])\
,p[2]\
,float(p[3])\
,p[4]))
or it defeats the purpose of using a lambda function?
I feel when I write it is ok to use lambda function in one line, is quick and good, but when I check again my code later I feel is not legible all of it in one line...

If it's not clearly readable as a simple one-liner, then it's not a good candidate for a lambda. Remember that the lambda statement is just syntactic sugar, technically it IS a function:
>>> def foo(): pass
...
>>> bar = lambda: None
>>>
>>> type(foo)
<class 'function'>
>>> type(bar)
<class 'function'>
>>>
So yes, in your example it does definitly "defeat the purpose of using a lambda function". As far as I'm concerned, if I had to maintain this code, I'd rather find something like:
def prepare(p):
p1a, p1b = (int(x) for x in p[1].split("-"))
p3f = float(p3)
return p[0], p[1], p1a, p1b, p[2], p3f, p[4]
whatever = [prepare(part) for part in parts]

If you are interested in style and readability I can't recommend the PEP8 style guide enough. Overall, it explains the best practices to write readable Python.
It will in particular give you advice on where to put commas when you start a new line, when to use parenthesis and how and when to write to a new line.
On lambda functions in particular it states:
Always use a def statement instead of an assignment statement that
binds a lambda expression directly to an identifier.
Yes:
def f(x): return 2*x
No:
f = lambda x: 2*x
In your case, I would use a function instead.

You could define a normal function and just use it in the map function if it gets too long.
def foo(p):
'''your code'''
result = list(map(foo, your_list)) # the list wrapper to convert map object to a list

Lambda functions python 3

I got the following code:
g = lambda x: x+7
foo = lambda f: (lambda x: f(x+1)*2)
print( g(3), (foo(g))(3), (foo(foo(g))((3) )
Could I get an explanation on how (foo(foo(g))((3) works?

The first thing to remember is that lambdas are regular functions that:
Don't automatically have names
Can be used as expressions
Must consist of a single expression
Implicitly return the results of that expression
So you can always rewrite them as normal def functions with names if you're confused. For example, foo can become:
def foo(f):
def foo_inner(x):
return f(x + 1) * 2
return foo_inner
So calling foo with any function (f) returns a new function which takes a numeric type, adds one to it, calls f with the value, and doubles the result.
All the rest of it is just tracing the multiple layers of wrapping here, which I'll leave to you; this isn't an interesting problem in general. In real code that uses factory functions like this, the intent and behavior is generally much more clear (because it's being done for a purpose, rather than as a brainteaser).

Function composition, tuples and unpacking

(disclaimed: not a Python kid, so please be gentle)
I am trying to compose functions using the following:
def compose(*functions):
return functools.reduce(lambda acc, f: lambda x: acc(f(x)), functions, lambda x: x)
which works as expected for scalar functions. I'd like to work with functions returning tuples and others taking multiple arguments, eg.
def dummy(name):
return (name, len(name), name.upper())
def transform(name, size, upper):
return (upper, -size, name)
# What I want to achieve using composition,
# ie. f = compose(transform, dummy)
transform(*dummy('Australia'))
=> ('AUSTRALIA', -9, 'Australia')
Since dummy returns a tuple and transform takes three arguments, I need to unpack the value.
How can I achieve this using my compose function above? If I try like this, I get:
f = compose(transform, dummy)
f('Australia')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in <lambda>
File "<stdin>", line 2, in <lambda>
TypeError: transform() takes exactly 3 arguments (1 given)
Is there a way to change compose such that it will unpack where needed?

This one works for your example but it wont handle just any arbitrary function - it will only works with positional arguments and (of course) the signature of any function must match the return value of the previous (wrt/ application order) one.
def compose(*functions):
return functools.reduce(
lambda f, g: lambda *args: f(*g(*args)),
functions,
lambda *args: args
)
Note that using reduce here, while certainly idiomatic in functional programming, is rather unpythonic. The "obvious" pythonic implementation would use iteration instead:
def itercompose(*functions):
def composed(*args):
for func in reversed(functions):
args = func(*args)
return args
return composed
Edit:
You ask "Is there a way to make have a compose function which will work in both cases" - "both cases" here meaning wether the functions returns an iterable or not (what you call "scalar functions", a concept that has no meaning in Python).
Using the iteration-based implementation, you could just test if the return value is iterable and wrap it in a tuple ie:
import collections
def itercompose(*functions):
def composed(*args):
for func in reversed(functions):
if not isinstance(args, collections.Iterable):
args = (args,)
args = func(*args)
return args
return composed
but this is not garanteed to work as expected - actually this is even garanteed to NOT work as expected for most use cases. There are a lot of builtin iterable types in Python (and even more user-defined ones) and just knowing an object is iterable doesn't say much about it's semantic.
For example a dict or str are iterable but in this case should obviously be considered a "scalar". A list is iterable too, and how it should be interpreted in this case is actually just undecidable without knowing exactly what it contains and what the "next" function in composition order expects - in some cases you will want to treat it as a single argument, in other cases ase a list of args.
IOW only the caller of the compose() function can really tell how each function result should be considered - actually you might even have cases where you want a tuple to be considered as a "scalar" value by the next function. So to make a long story short: no, there's no one-size-fits-all generic solution in Python. The best I could think of requires a combination of result inspection and manual wrapping of composed functions so the result is properly interpreted by the "composed" function but at this point manually composing the functions will be both way simpler and much more robust.
FWIW remember that Python is first and mostly a dynamically typed object oriented language so while it does have a decent support for functional programming idioms it's obviously not the best tool for real functional programming.

You might consider inserting a "function" (really, a class constructor) in your compose chain to signal the unpacking of the prior/inner function's results. You would then adjust your composer function to check for that class to determine if the prior result should be unpacked. (You actually end up doing the reverse: tuple-wrap all function results except those signaled to be unpacked -- and then have the composer unpack everything.) It adds overhead, it's not at all Pythonic, it's written in a terse lambda style, but it does accomplish the goal of being able to properly signal in a function chain when the composer should unpack a result. Consider the following generic code, which you can then adapt to your specific composition chain:
from functools import reduce
from operator import add
class upk: #class constructor signals composer to unpack prior result
def __init__(s,r): s.r = r #hold function's return for wrapper function
idt = lambda x: x #identity
wrp = lambda x: x.r if isinstance(x, upk) else (x,) #wrap all but unpackables
com = lambda *fs: ( #unpackable compose, unpacking whenever upk is encountered
reduce(lambda a,f: lambda *x: a(*wrp(f(*x))), fs, idt) )
foo = com(add, upk, divmod) #upk signals divmod's results should be unpacked
print(foo(6,4))
This circumvents the problem, as called out by prior answers/comments, of requiring your composer to guess which types of iterables should be unpacked. Of course, the cost is that you must explicitly insert upk into the callable chain whenever unpacking is required. In that sense, it is by no means "automatic", but it is still a fairly simple/terse way of achieving the intended result while avoiding unintended wraps/unwraps in many corner cases.

The compose function in the answer contributed by Bruno did do the job for functions with multiple arguments but didn't work any more for scalar ones unfortunately.
Using the fact that Python `unpacks' tuples into positional arguments, this is how I solved it:
import functools
def compose(*functions):
def pack(x): return x if type(x) is tuple else (x,)
return functools.reduce(
lambda acc, f: lambda *y: f(*pack(acc(*pack(y)))), reversed(functions), lambda *x: x)
which now works just as expected, eg.
#########################
# scalar-valued functions
#########################
def a(x): return x + 1
def b(x): return -x
# explicit
> a(b(b(a(15))))
# => 17
# compose
> compose(a, b, b, a)(15)
=> 17
########################
# tuple-valued functions
########################
def dummy(x):
return (x.upper(), len(x), x)
def trans(a, b, c):
return (b, c, a)
# explicit
> trans(*dummy('Australia'))
# => ('AUSTRALIA', 9, 'Australia')
# compose
> compose(trans, dummy)('Australia')
# => ('AUSTRALIA', 9, 'Australia')
And this also works with multiple arguments:
def add(x, y): return x + y
# explicit
> b(a(add(5, 3)))
=> -9
# compose
> compose(b, a, add)(5, 3)
=> -9

Scope of lambda functions and their parameters? [duplicate]

This question already has answers here:
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 6 months ago.
I need a callback function that is almost exactly the same for a series of gui events. The function will behave slightly differently depending on which event has called it. Seems like a simple case to me, but I cannot figure out this weird behavior of lambda functions.
So I have the following simplified code below:
def callback(msg):
print msg
#creating a list of function handles with an iterator
funcList=[]
for m in ('do', 're', 'mi'):
funcList.append(lambda: callback(m))
for f in funcList:
f()
#create one at a time
funcList=[]
funcList.append(lambda: callback('do'))
funcList.append(lambda: callback('re'))
funcList.append(lambda: callback('mi'))
for f in funcList:
f()
The output of this code is:
mi
mi
mi
do
re
mi
I expected:
do
re
mi
do
re
mi
Why has using an iterator messed things up?
I've tried using a deepcopy:
import copy
funcList=[]
for m in ('do', 're', 'mi'):
funcList.append(lambda: callback(copy.deepcopy(m)))
for f in funcList:
f()
But this has the same problem.

When a lambda is created, it doesn't make a copy of the variables in the enclosing scope that it uses. It maintains a reference to the environment so that it can look up the value of the variable later. There is just one m. It gets assigned to every time through the loop. After the loop, the variable m has value 'mi'. So when you actually run the function you created later, it will look up the value of m in the environment that created it, which will by then have value 'mi'.
One common and idiomatic solution to this problem is to capture the value of m at the time that the lambda is created by using it as the default argument of an optional parameter. You usually use a parameter of the same name so you don't have to change the body of the code:
for m in ('do', 're', 'mi'):
funcList.append(lambda m=m: callback(m))

The problem here is the m variable (a reference) being taken from the surrounding scope.
Only parameters are held in the lambda scope.
To solve this you have to create another scope for lambda:
def callback(msg):
print msg
def callback_factory(m):
return lambda: callback(m)
funcList=[]
for m in ('do', 're', 'mi'):
funcList.append(callback_factory(m))
for f in funcList:
f()
In the example above, lambda also uses the surounding scope to find m, but this
time it's callback_factory scope which is created once per every callback_factory
call.
Or with functools.partial:
from functools import partial
def callback(msg):
print msg
funcList=[partial(callback, m) for m in ('do', 're', 'mi')]
for f in funcList:
f()

Python does uses references of course, but it does not matter in this context.
When you define a lambda (or a function, since this is the exact same behavior), it does not evaluate the lambda expression before runtime:
# defining that function is perfectly fine
def broken():
print undefined_var
broken() # but calling it will raise a NameError
Even more surprising than your lambda example:
i = 'bar'
def foo():
print i
foo() # bar
i = 'banana'
foo() # you would expect 'bar' here? well it prints 'banana'
In short, think dynamic: nothing is evaluated before interpretation, that's why your code uses the latest value of m.
When it looks for m in the lambda execution, m is taken from the topmost scope, which means that, as others pointed out; you can circumvent that problem by adding another scope:
def factory(x):
return lambda: callback(x)
for m in ('do', 're', 'mi'):
funcList.append(factory(m))
Here, when the lambda is called, it looks in the lambda' definition scope for a x. This x is a local variable defined in factory's body. Because of this, the value used on lambda execution will be the value that was passed as a parameter during the call to factory. And doremi!
As a note, I could have defined factory as factory(m) [replace x by m], the behavior is the same. I used a different name for clarity :)
You might find that Andrej Bauer got similar lambda problems. What's interesting on that blog is the comments, where you'll learn more about python closure :)

Yes, that's a problem of scope, it binds to the outer m, whether you are using a lambda or a local function. Instead, use a functor:
class Func1(object):
def __init__(self, callback, message):
self.callback = callback
self.message = message
def __call__(self):
return self.callback(self.message)
funcList.append(Func1(callback, m))

the soluiton to lambda is more lambda
In [0]: funcs = [(lambda j: (lambda: j))(i) for i in ('do', 're', 'mi')]
In [1]: funcs
Out[1]:
[<function __main__.<lambda>>,
<function __main__.<lambda>>,
<function __main__.<lambda>>]
In [2]: [f() for f in funcs]
Out[2]: ['do', 're', 'mi']
the outer lambda is used to bind the current value of i to j
at the
each time the outer lambda is called it makes an instance of the inner lambda with j bound to the current value of i as i's value

First, what you are seeing is not a problem, and not related to call-by-reference or by-value.
The lambda syntax you defined has no parameters, and as such, the scope you are seeing with parameter m is external to the lambda function. This is why you are seeing these results.
Lambda syntax, in your example is not necessary, and you would rather be using a simple function call:
for m in ('do', 're', 'mi'):
callback(m)
Again, you should be very precise about what lambda parameters you are using and where exactly their scope begins and ends.
As a side note, regarding parameter passing. Parameters in python are always references to objects. To quote Alex Martelli:
The terminology problem may be due to
the fact that, in python, the value of
a name is a reference to an object.
So, you always pass the value (no
implicit copying), and that value is
always a reference. [...] Now if you
want to coin a name for that, such as
"by object reference", "by uncopied
value", or whatever, be my guest.
Trying to reuse terminology that is
more generally applied to languages
where "variables are boxes" to a
language where "variables are post-it
tags" is, IMHO, more likely to confuse
than to help.

The variable m is being captured, so your lambda expression always sees its "current" value.
If you need to effectively capture the value at a moment in time, write a function takes the value you want as a parameter, and returns a lambda expression. At that point, the lambda will capture the parameter's value, which won't change when you call the function multiple times:
def callback(msg):
print msg
def createCallback(msg):
return lambda: callback(msg)
#creating a list of function handles with an iterator
funcList=[]
for m in ('do', 're', 'mi'):
funcList.append(createCallback(m))
for f in funcList:
f()
Output:
do
re
mi

there are actually no variables in the classic sense in Python, just names that have been bound by references to the applicable object. Even functions are some sort of object in Python, and lambdas do not make an exception to the rule :)

As a side note, map, although despised by some well known Python figure, forces a construction which prevents this pitfall.
fs = map (lambda i: lambda: callback (i), ['do', 're', 'mi'])
NB : the first lambda i acts like the factory in other answers.