Related
I am trying out lambda in python and came across this question:
def foo(y):
return lambda x: x(x(y))
def bar(x):
return lambda y: x(y)
print((bar)(bar)(foo)(2)(lambda x:x+1))
can someone explain/breakdown how this code works? I am having problems trying to figure out what is x and y.
Lambda functions are just functions. They're almost syntatic sugar, as you can think of this structure:
anony_mouse = lambda x: x # don't actually assign lambdas
as equivalent to this structure:
def anony_mouse(x):
return x
(Almost, as there is no other way of getting a function without assigning it to some variable, and the syntax prevents you doing some things with them, such as using multiple lines.)
Thus let's write out the top example using standard function notation:
def foo(y):
# note that y exists here
def baz(x):
return x(x(y))
return baz
So we have a factory function, which generates a function which... expects to be called with a function (x), and returns x(x(arg_to_factory_function)). Consider:
>>> def add_six(x):
return x + 6
>>> bazzer = foo(3)
>>> bazzer(add_six) # add_six(add_six(3)) = 6+(6+3)
I could go on, but does that make it clearer?
Incidentally that code is horrible, and almost makes me agree with Guido that lambdas are bad.
The 1st ‘(bar)’ is equal to just ‘bar’ so it is an ordinary function call, the 2nd — argument to that call, i.e. bar(bar) — substitute ‘x’ to ‘bar’ there any you will get what is result of bar(bar); the’(foo)’ argument passing to the result of bar(bar) it will be a lambda-function with some arg. — substitute it to ‘foo’ and get result and so on until you reach the end of expression
I slightly modify your original function to make clearer what's going on (so it should be clearer which parameter is callable!)
# given a function it evaluates it at value p
def eval(func): # your foo
return lambda p: func(p)
# given a value p perform a double composition of the function at this value (2-step recursion)
def iter_2(p): # your bar
return lambda func: func(func(p))
increment = lambda x: x + 1 # variable binding only for readability
This example is quite hard to understand because one of the function, eval just do nothing special, and it composition is equivalent to the identity! ... so it could be quite confusing.
(foo)(2)(lambda x:x+1)):
x = 2
iter_2(x)(increment) # increment by 2 because iter_2 calls increment twice
# 4
idempotency: (or composition with itself return the identity function)
increment(3) == eval(increment)(3)
# True
# idempotency - second composition is equivalent to the identity
eval(increment)(3) == eval(eval)(increment)(3)
# True
eval(increment)(3) == eval(eval)(eval)(increment)(3)
# True
# ... and so on
final: consequence of idempotency -> bar do nothing, just confusion
eval(eval)(iter_2)(x)(increment) == iter_2(x)(increment)
# True
Remark:
in (bar)(bar)(foo)(2)(lambda x:x+1) you can omit the brackets around the 1st term, just bar(bar)(foo)(2)(lambda x:x+1)
Digression: [since you example is quite scaring]
Lambda functions are also known as anonymous function. Why this? Simply because that they don't need to be declared. They are designed to be single purpose, so you should "never" assign to a variable. The arise for example in the context of functional programming where the basic ingredients are... functions! They are used to modify the behavior of other functions (for example by decoration!). Your example it is just a standalone syntactical one... essentially a nonsense example which hides the truth "power" of the lambda functions. There is also a branch mathematics which based on them called lambda calculus.
Here a totally different example of application of the lambda functions, useful for decoration (but this is another story):
def action(func1):
return lambda func2: lambda p: func2(p, func1())
def save(path, content):
print(f'content saved to "{path}"')
def content():
return 'content' # i.e. from a file, url, ...
# call
action(content)(save)('./path')
# with each key-parameter would be
action(func1=content)(func2=save)(p='./path')
Output
content saved to "./path"
In Python 3, here is my compose function one-liner, which I am trying to modify:
def compose(*fncs):
return functools.reduce(lambda f,g: lambda x: f(g(x)), fncs, lambda x: x)
When I compose a function with c = compose(h, g, f), calling c(x) is equivalent to calling h(g(f(x))
By changing my existing one-liner as little as possible, I would like to create a compose_intermed(*fncs) function which returns a slightly different kind of composed function. This function, when called, returns not the final value of the composed functions, but a list whose first element is the final value, followed by all the intermediate values at each step in which composed functions are applied.
When I compose a function with ci = compose_intermed(h, g, f), calling ci(x) would return the list [h(g(f(x))), g(f(x)), f(x)].
I would like to modify the compose function as little as possible, continuing to use either reduce or perhaps a list comprehension, rather than loops. I know there may be easier ways to do this, but I'm trying to use this as an exercise to improve my general understanding of the nexus of functional programming and Python 3.
Bonus question: Does this function have another more standardized name in the functional programming world? I've searched several libraries, and I haven't yet found a library function for what I am trying to do.
Ry's comment is a good starting point. In this post, I'll try to demonstrate what he/she is talking about -
from functools import reduce
def identity(x):
return x
def pipeline(f = identity, *fs):
return reduce(lambda r,f: lambda x: f(r(x)), fs, f)
Make two simple functions and test it out. Notice how pipeline applies the functions in left-to-right order -
def add1(x):
return x + 1
def mult2(x):
return x * 2
f = pipeline(mult2, add1, add1, add1)
print(f(10))
# 23
Next, implement pipeline_intermediate. Just as Ry comments, the output is reversed at the end using [::-1] -
def pipeline_intermediate(f = identity, *fs):
return lambda x: reduce(lambda r,f: [f(r[0])]+r, fs, [f(x)]) [::-1]
g = pipeline_intermediate(mult2, add1, add1, add1)
print(g(10))
# [20, 21, 22, 23]
Now can you see how to implement right-to-left compose_intermediate? Can you see why it's more challenging?
(disclaimed: not a Python kid, so please be gentle)
I am trying to compose functions using the following:
def compose(*functions):
return functools.reduce(lambda acc, f: lambda x: acc(f(x)), functions, lambda x: x)
which works as expected for scalar functions. I'd like to work with functions returning tuples and others taking multiple arguments, eg.
def dummy(name):
return (name, len(name), name.upper())
def transform(name, size, upper):
return (upper, -size, name)
# What I want to achieve using composition,
# ie. f = compose(transform, dummy)
transform(*dummy('Australia'))
=> ('AUSTRALIA', -9, 'Australia')
Since dummy returns a tuple and transform takes three arguments, I need to unpack the value.
How can I achieve this using my compose function above? If I try like this, I get:
f = compose(transform, dummy)
f('Australia')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in <lambda>
File "<stdin>", line 2, in <lambda>
TypeError: transform() takes exactly 3 arguments (1 given)
Is there a way to change compose such that it will unpack where needed?
This one works for your example but it wont handle just any arbitrary function - it will only works with positional arguments and (of course) the signature of any function must match the return value of the previous (wrt/ application order) one.
def compose(*functions):
return functools.reduce(
lambda f, g: lambda *args: f(*g(*args)),
functions,
lambda *args: args
)
Note that using reduce here, while certainly idiomatic in functional programming, is rather unpythonic. The "obvious" pythonic implementation would use iteration instead:
def itercompose(*functions):
def composed(*args):
for func in reversed(functions):
args = func(*args)
return args
return composed
Edit:
You ask "Is there a way to make have a compose function which will work in both cases" - "both cases" here meaning wether the functions returns an iterable or not (what you call "scalar functions", a concept that has no meaning in Python).
Using the iteration-based implementation, you could just test if the return value is iterable and wrap it in a tuple ie:
import collections
def itercompose(*functions):
def composed(*args):
for func in reversed(functions):
if not isinstance(args, collections.Iterable):
args = (args,)
args = func(*args)
return args
return composed
but this is not garanteed to work as expected - actually this is even garanteed to NOT work as expected for most use cases. There are a lot of builtin iterable types in Python (and even more user-defined ones) and just knowing an object is iterable doesn't say much about it's semantic.
For example a dict or str are iterable but in this case should obviously be considered a "scalar". A list is iterable too, and how it should be interpreted in this case is actually just undecidable without knowing exactly what it contains and what the "next" function in composition order expects - in some cases you will want to treat it as a single argument, in other cases ase a list of args.
IOW only the caller of the compose() function can really tell how each function result should be considered - actually you might even have cases where you want a tuple to be considered as a "scalar" value by the next function. So to make a long story short: no, there's no one-size-fits-all generic solution in Python. The best I could think of requires a combination of result inspection and manual wrapping of composed functions so the result is properly interpreted by the "composed" function but at this point manually composing the functions will be both way simpler and much more robust.
FWIW remember that Python is first and mostly a dynamically typed object oriented language so while it does have a decent support for functional programming idioms it's obviously not the best tool for real functional programming.
You might consider inserting a "function" (really, a class constructor) in your compose chain to signal the unpacking of the prior/inner function's results. You would then adjust your composer function to check for that class to determine if the prior result should be unpacked. (You actually end up doing the reverse: tuple-wrap all function results except those signaled to be unpacked -- and then have the composer unpack everything.) It adds overhead, it's not at all Pythonic, it's written in a terse lambda style, but it does accomplish the goal of being able to properly signal in a function chain when the composer should unpack a result. Consider the following generic code, which you can then adapt to your specific composition chain:
from functools import reduce
from operator import add
class upk: #class constructor signals composer to unpack prior result
def __init__(s,r): s.r = r #hold function's return for wrapper function
idt = lambda x: x #identity
wrp = lambda x: x.r if isinstance(x, upk) else (x,) #wrap all but unpackables
com = lambda *fs: ( #unpackable compose, unpacking whenever upk is encountered
reduce(lambda a,f: lambda *x: a(*wrp(f(*x))), fs, idt) )
foo = com(add, upk, divmod) #upk signals divmod's results should be unpacked
print(foo(6,4))
This circumvents the problem, as called out by prior answers/comments, of requiring your composer to guess which types of iterables should be unpacked. Of course, the cost is that you must explicitly insert upk into the callable chain whenever unpacking is required. In that sense, it is by no means "automatic", but it is still a fairly simple/terse way of achieving the intended result while avoiding unintended wraps/unwraps in many corner cases.
The compose function in the answer contributed by Bruno did do the job for functions with multiple arguments but didn't work any more for scalar ones unfortunately.
Using the fact that Python `unpacks' tuples into positional arguments, this is how I solved it:
import functools
def compose(*functions):
def pack(x): return x if type(x) is tuple else (x,)
return functools.reduce(
lambda acc, f: lambda *y: f(*pack(acc(*pack(y)))), reversed(functions), lambda *x: x)
which now works just as expected, eg.
#########################
# scalar-valued functions
#########################
def a(x): return x + 1
def b(x): return -x
# explicit
> a(b(b(a(15))))
# => 17
# compose
> compose(a, b, b, a)(15)
=> 17
########################
# tuple-valued functions
########################
def dummy(x):
return (x.upper(), len(x), x)
def trans(a, b, c):
return (b, c, a)
# explicit
> trans(*dummy('Australia'))
# => ('AUSTRALIA', 9, 'Australia')
# compose
> compose(trans, dummy)('Australia')
# => ('AUSTRALIA', 9, 'Australia')
And this also works with multiple arguments:
def add(x, y): return x + y
# explicit
> b(a(add(5, 3)))
=> -9
# compose
> compose(b, a, add)(5, 3)
=> -9
I'm curious about the difference between lambda function and a regular function (defined with def) - in the python level. (I know what is the difference for programmers and when to use each one.)
>>> def a():
return 1
>>> b = lambda: 1
>>> a
<function a at 0x0000000004036F98>
>>> b
<function <lambda> at 0x0000000004031588>
As we can see - python knows that b is a lambda function and a is a regular function. why is that? what is the difference between them to python?
They are the same type so they are treated the same way:
>>> type(a)
<type 'function'>
>>> type(b)
<type 'function'>
Python also knows that b was defined as a lambda function and it sets that as function name:
>>> a.func_name
'a'
>>> b.func_name
'<lambda>'
In other words, it influences the name that the function will get but as far as Python is concerned, both are functions which means they can be mostly used in the same way. See mgilson's comment below for an important difference between functions and lambda functions regarding pickling.
The only difference is that (a) the body of a lambda can consist of only a single expression, the result of which is returned from the function created and (b) a lambda expression is an expression which evaluates to a function object, while a def statement has no value, and creates a function object and binds it to a name.
In all other material respects they result in identical objects - the same scope and capture rules apply. (Immaterial differences are that lambda-created functions have a default func_name of "<lambda>". This may affect operation in esoteric cases - e.g. attempts to pickle functions.).
Both lambda and def create the same kind of function – they have the same kind of metadata and capabilities. Their technical difference is syntactical:
A lambda is an expression producing a function.
A def is a statement producing a function.
This is everything that dictates how they can be used. Other apparent differences simply come from the information lambda/def can capture.
>>> def def_func(): pass
>>> lambda_func = lambda: None
>>> type(def_func) == type(lambda_func)
True
Usage: Expression vs. Statement
A lambda is more flexible as expressions can be part of more language constructs.
# v--------------v arguments must be expressions
sort(values, key=lambda x: abs(x))
In contrast, a def is more powerful as it can consist of more language constructs.
def encode(num, base):
while num: # statements must be inside statements
num, bit = divmod(num, base)
yield bit
These differences derive directly from one being an expression and the other being a statement. Python has no special rules to decide where a lambda/def may be used.
Where the wild <lambda>s grow
The primary reason to assume lambda and def correspond to different kinds of function is metadata: lambda is often referred to as an "anonymous function" and miraculously it always produces a function <lambda>. Other quirks include "lambda functions can't be pickled", and recently typing also does "not work" for lambda.
That is because compared to def syntax, the lambda syntax has no way of specifying name, type annotations and similar. As such, Python simply fills in sane defaults for either: the name becomes <lambda> and annotations are left empty.
>>> identity = lambda a: a
>>> identity.__qualname__
'<lambda>'
>>> identity.__annotations__
{}
Since <lambda> is not a valid identifier, everything using this metadata to find the function – most prominently pickle – fails.
However, that does not make the function an "anonymous function" type. The metadata can be patched up to insert what def would provide:
>>> identity.__qualname__ = identity.__name__ = 'identity'
>>> identity
<function __main__.identity(a)>
Of course at that one point one can just use def…
First consider the diff b/w the two.
Lambda functions: are operator can have any number of arguments, but it can have only one expression. It cannot contain any statements and it returns a function object which can be assigned to any variable. They can be used in the block they were created.
def functions: Functions help break our program into smaller and modular chunks. As our program grows larger and larger, functions make it more organised and manageable. They can be called and used anywhere we want.
Here you can get more clear difference by following example.
Defining a function
def add(a,b):
return a+b
print(add(4,5))
Defining a lambda
add = lambda x, y : x + y
print(add(4,5))
Lambda is an inline function where we can do any functionality without a function name.
It is helpful when we use it as an argument to a higher-order function.
Eg: A function that takes in other functions as arguments.
Example of Function definition:
>>> def func(a, b):
return a * b
>>> func(2,3)
6
>>> type(func)
<class 'function'>
>>> func
<function func at 0x034B6E88>
Example of Lambda expression:
>>> multiply = lambda a, b: a * b
>>> multiply(2, 3)
6
>>> type(multiply)
<class 'function'>
>>> multiply
<function <lambda> at 0x034B6ED0>
Both returns same output value. Only object returned are different. "func" name for Function and for Lambda.
lambda creates an anonymous function. This idea has been taken from functional programming languages. In this way you can create and pass the function to other functions like map and filter. (look here)
You can pass normal functions to these functions too, but since mostly they are simple and they are not used anywhere else, it's inconvenient to go through the whole process of definfing a new function.
As an example take a look at this:
>>> a = [1, 2, 3, 4]
>>> print map( lambda x : x*2 + 1, a )
[3, 5, 7, 9, 11]
What does the following code do?
a = lambda _:True
From what I read and tested in the interactive prompt, it seems to be a function that returns always True.
Am I understanding this correctly? I hope to understand why an underscore (_) was used as well.
The _ is variable name. Try it.
(This variable name is usually a name for an ignored variable. A placeholder so to speak.)
Python:
>>> l = lambda _: True
>>> l()
<lambda>() missing 1 required positional argument: '_'
>>> l("foo")
True
So this lambda does require one argument. If you want a lambda with no argument that always returns True, do this:
>>> m = lambda: True
>>> m()
True
Underscore is a Python convention to name an unused variable (e.g. static analysis tools does not report it as unused variable). In your case lambda argument is unused, but created object is single-argument function which always returns True. So your lambda is somewhat analogous to Constant Function in math.
it seems to be a function that returns True regardless.
Yes, it is a function (or lambda) that returns True. The underscore, which is usually a placeholder for an ignored variable, is unnecessary in this case.
An example use case for such a function (that does almost nothing):
dd = collections.defaultdict(lambda: True)
When used as the argument to a defaultdict, you can have True as a general default value.
Below is the line of code in question:
a = lambda _:True
It creates a function having one input parameter: _. Underscore is a rather strange choice of variable name, but it is just a variable name. You can use _ anywhere, even when not using lambda functions. For example, instead of....
my_var = 5
print(my_var)
You could write:
_ = 5
print(_)
However, there was a reason that _ was used as the name of parameter name instead of something like x or input. We'll get to that in a moment.
First, we need to know that the lambda-keyword constructs a function, similar to def, but with different syntax. The definition of the lambda function, a = lambda _:True, is similar to writing:
def a(_):
return True
It creates a function named a with an input parameter _, and it returns True. One could have just as easily written a = lambda x:True, with an x instead of an underscore. However, the convention is to use _ as a variable name when we do not intend to use that variable. Consider the following:
for _ in range(1, 11):
print('pear')
Notice that the loop index is never used inside of the loop-body. We simply want the loop to execute a specified number of times. As winklerrr has written, "the variable name _ is [...] like a "throw-away-variable", just a placeholder which is of no use. "
Likewise, with ``a = lambda x:True the input parameter is not used inside the body of the function. It does not really matter what the input argument is, as long as there is one. The author of that lambda-function wrote _ instead of something like x, to indicate that the variable would not be used.
Note that the lambda does have an argument; So, writing
a(), will raise an error.
If you want a lambda with no argument write something like this:
bar = lambda: True
Now calling bar(), with no args, will work just fine.
A lambda which takes no arguments need not always return the same value:
import random
process_fruit = lambda : random.random()
The lambda function above is more complex that just a something which always returns the same constant.
One reason that programmers sometimes us the lambda keyword instead of def is for functions which are especially short and simple. Note that a lambda definition can usually fit all on one line, whereas, it is difficult to do the same with a def statement. Another reason to use lambda instead of def sf when the function will not be used again. If we don't want to call the function again later, then there is no need to give the function a name. For example consider the following code:
def apply_to_each(transform, in_container):
out_container = list()
for idx, item in enumerate(container, 0):
out_container[idx] = transform(item)
return out_container
Now we make the following call:
squares = apply_to_each(lambda x: x**2 range(0, 101))
Notice that lambda x: x**2 is not given a label. This is because we probably won't call it again later, it was just something short and simple we needed temporarily.
The fact that lambda functions need not be given a name is the source of another name to describe them: "anonymous functions."
Also note that lambda-statements are like a function-call in that they return a reference to the function they create. The following is illegal:
apply_to_each(def foo(x): x**2 , range(0, 101))
Whereas, apply_to_each(lambda x: x**2 range(0, 101)) is just fine.
So, we use lambda instead of def and _ instead of a long variable name when we want something short, sweet and probably won't want use again later.
Lambda means a function.
The above statement is same as writing
def f(_):
return True
For lambda a variable needs to be present. So you pass it a variable called _(Similarly you could pass x, y..)
Underscore _ is a valid identifier and is used here as a variable name. It will always return True for the argument passed to the function.
>>>a('123')
True