I have a bunch of functions in a list:
funcs = [f1, f2, f3, f4, f5]
and all of the functions take in return a single argument, eg.
f1 = lambda x: x*2
I'd like to map all these functions together
result = lambda x: f5(f4(f3(f2(f1(x)))))
or, iterating over funcs
def dispatch(x):
for f in funcs:
x = f(x)
return x
dispatch works fine, but I couldn't figure out a clean way to do this using iterools. Is it possible? Does this sequential function mapping idiom have a name?
There is no point in using itertools here; you are producing one output, and you could not apply this to an infinite iterable. You have to have a finite number of functions in the input iterable for this to work at all.
Use the reduce() function:
from functools import reduce
x = reduce(lambda res, func: func(res), funcs, x)
The functools.reduce() import helps the above work in both Python 2 and 3.
reduce(), together with map(), filter() and, yes, itertools, is an often used tool in functional programming.
Another (less efficient, alas) way of looking at Martijn's answer is to realize that you want to compose the list of functions.
# function composition: compose(f,g)(x) = f(g(x))
def compose(f, g):
return lambda x: f(g(x))
# Identity for function composition
# compose(f, identity)(x) = f(x)
identity = lambda x: x
# result(x) = f1(f2(...fn(x)...))
result = reduce(compose, funcs, identity)
Related
What's the pythonic way to combine a list of lambdas in a single function? For example:
lambdas = [lambda x, k=k: x+k for k in range(3)]
I would like to get this all in a single lambda similar to this but without having to type it out:
f = lambda x: lambdas[2](lambdas[1](lambdas[0](x)))
You can do this with functools.reduce like below:
from functools import reduce
lambdas = [lambda x, k=k: x+k for k in range(3)]
# x = 0
reduce(lambda x, l: l(x), lambdas, x)
# -> l[2](l[1](l[0](x)))
# step_1 : x = x , l = lambda[0] -> lambda[0](x)
# step_2 : x = lambda[0](x), l = lambda[1] -> lambda[1](lambda[0](x))
# step_3 : x = lambda[1](lambda[0](x)), l = lambda[2] -> lambda[2](lambda[1](lambda[0](x)))
The reduce function is defined to be exactly what you want. An alternative is to use a simple for loop.
def f(x):
for func in lambdas:
x = func(x)
return x
to do this with a lambda seems kind of weird.
Is there any specific reason why we cannot:
def function_chainer(lambdas):
def chained(x):
for function in lambdas:
x = function(x)
return chained
This solution is not a one-liner, but it is pythonic I believe.
If you really need a one-liner, you can use functools.reduce:
lambda x: functools.reduce(lambda a, f: f(a), lambdas, x)
The first argument to reduce governs the way of applying each subsequent element, the second is the iterable (here - our iterable of lambdas) and the last one is the initializer - the first value we want to pass to those lambda functions.
Question
Is there a readily available Pythonic way to compose a multiple nested function g = f3(f2(f1())) from a list of functions [f1, f2, f3] where there are more functions in a list.
If there are a few, I may do:
g = lambda x: f3(f2(f1(x)))
However when I have dozens of functions e.g layers in a deep neural network, it is un-manageable. Prefer not creating another function to compose g but finding an available way.
Update
Based on the answer from #Chris. For sequential neural network layers [ batchnorm, matmul, activation, softmaxloss ], each of which has a forward(X) method to calculate its output to the next layer, the loss function L and loss would be:
L = reduce(lambda f, g: lambda X: g(f(X)), [ layer.forward for layer in layers ] ) # Loss function
network_loss = L(X)
One way using functools.reduce:
from functools import reduce
f1 = lambda x: x+1
f2 = lambda x: x*2
f3 = lambda x: x+3
funcs = [f1, f2, f3]
g = reduce(lambda f, g: lambda x: g(f(x)), funcs)
Output:
g(1)==7 # ((1+1) * 2) + 3
g(2)==9 # ((2+1) * 2) + 3
Insight:
functools.reduce will chain its second argument (funcs here) according to its first argument (lambda here).
That being said, it will start chaining f1 and f2 as f_21(x) = f2(f1(x)), then f3 and f_21 as f3(f_21(x)) which becomes g(x).
One problem with the reduce-baed approach is that you introduce O(n) additional function calls. An alternative is to define a single function that remembers the functions to compose; when called, it simply calls each function in sequence on the given argument.
def compose(*args):
"""compose(f1, f2, ..., fn) == lambda x: fn(...(f2(f1(x))...)"""
def _(x):
result = x
for f in args:
result = f(result)
return result
return _
You can implement it yourself, but you could also try a module named compose which implements this, written by #mtraceur. It takes care to handle various details, such as correct function signature forwarding.
pip install compose
from compose import compose
def doubled(x):
return 2*x
octupled = compose(doubled, doubled, doubled)
print(octupled(1))
# 8
Lets say we have functions in python:
def _abs(iterable): #cause its normally for one element only
return [abs(i) for i in iterable]
def A(list, foo):
return foo(list)
list = [2,3,-5]
print( A(list,foo=sum) )
>> 0
while I may pass foo=sum to A, I am looking for an elegant way to pass something like foo=sum(_abs) to perform sum(_abs(list)).
The only way I see it now is to send a list of functions [sum, _abs] and apply them in order. Is there a better way?
Or, to compose more generally (i.e. with an arbitrary number of argument functions):
from functools import partial, reduce
def compose(*funcs):
return partial(reduce, lambda x, f: f(x), reversed(funcs))
(see the docs on partial and reduce - note you don't need to import reduce in Python 2.x)
Then:
>>> compose(sum, _abs)([2, 3, -5])
10
You could make an explicit compose function
>>> def compose(f, g):
... return lambda x: f(g(x))
...
Then use it
>>> A(list, compose(sum, _abs))
10
def compose(funcs, funcargs):
for func in reversed(funcs):
funcargs = func(funcargs)
return funcargs
Is there a function in Python that would do this:
val = f3(f2(f1(arg)))
by typing this (for example):
val = chainCalling(arg,f3,f2,f1)
I just figured, since python is (arguably) a functional language the function I'm looking for will make syntax brighter
Use the reduce() function to chain calls:
from functools import reduce
val = reduce(lambda r, f: f(r), (f1, f2, f3), arg)
I used the forward-compatible functools.reduce() function; in Python 3 reduce() is no longer in the built-in namespace.
This can also be made a separate function, of course:
from functools import reduce
def chain(*funcs):
def chained_call(arg):
return reduce(lambda r, f: f(r), funcs, arg)
return chained_call
You can use the reduce() functool — as Martijn briantly suggested, or you can write it yourself quite simply:
def chainCalling(arg, *funcs):
if len(funcs) > 0:
return chainCalling(funcs[0](arg), funcs[1:])
return arg
or, as an alternative not using recursion — so not bound to the call stack limitation, as suggested by Martijn:
def chainCalling(arg, *funcs):
result = arg
for f in funcs:
result = f(result)
return result
Obviously, you'll want to call it that way, to avoid an useless reversal of the arguments:
chainCalling(arg, f1, f2, f3)
In case you want to apply the chain of functions to multiple arguments, you can create an aggregated function.
g = lambda x: f3(f2(f1(x)))
or more flexible (when there is an arbitrary list of functions):
from functools import reduce, partial
f3 = lambda x: -x
f2 = lambda x: x ** 2
f1 = lambda x: x + 1
function_list = (f1, f2, f3)
g = partial(reduce, lambda r, f: f(r), function_list)
print(g(3)) # results in -16
It might be a tad late, but try my solution below. chain_func essentially creates a new wrapper function that calls all underlying functions in chain when invoked in runtime.
def chain_func(*funcs):
def _chain(*args, **kwargs):
cur_args, cur_kwargs = args, kwargs
ret = None
for f in reversed(funcs):
cur_args, cur_kwargs = (f(*cur_args, **cur_kwargs), ), {}
ret = cur_args[0]
return ret
return _chain
I have a rather lengthy equation that I need to integrate over using scipy.integrate.quad and was wondering if there is a way to add lambda functions to each other. What I have in mind is something like this
y = lambda u: u**(-2) + 8
x = lambda u: numpy.exp(-u)
f = y + x
int = scipy.integrate.quad(f, 0, numpy.inf)
The equations that I am really using are far more complicated than I am hinting at here, so for readability it would be useful to break up the equation into smaller, more manageable parts.
Is there a way to do with with lambda functions? Or perhaps another way which does not even require lambda functions but will give the same output?
In Python, you'll normally only use a lambda for very short, simple functions that easily fit inside the line that's creating them. (Some languages have other opinions.)
As #DSM hinted in their comment, lambdas are essentially a shortcut to creating functions when it's not worth giving them a name.
If you're doing more complex things, or if you need to give the code a name for later reference, a lambda expression won't be much of a shortcut for you -- instead, you might as well define a plain old function.
So instead of assigning the lambda expression to a variable:
y = lambda u: u**(-2) + 8
You can define that variable to be a function:
def y(u):
return u**(-2) + 8
Which gives you room to explain a bit, or be more complex, or whatever you need to do:
def y(u):
"""
Bloopinate the input
u should be a positive integer for fastest results.
"""
offset = 8
bloop = u ** (-2)
return bloop + offset
Functions and lambdas are both "callable", which means they're essentially interchangable as far as scipy.integrate.quad() is concerned.
To combine callables, you can use several different techniques.
def triple(x):
return x * 3
def square(x):
return x * x
def triple_square(x):
return triple(square(x))
def triple_plus_square(x):
return triple(x) + square(x)
def triple_plus_square_with_explaining_variables(x):
tripled = triple(x)
squared = square(x)
return tripled + squared
There are more advanced options that I would only consider if it makes your code clearer (which it probably won't). For example, you can put the callables in a list:
all_the_things_i_want_to_do = [triple, square]
Once they're in a list, you can use list-based operations to work on them (including applying them in turn to reduce the list down to a single value).
But if your code is like most code, regular functions that just call each other by name will be the simplest to write and easiest to read.
There's no built-in functionality for that, but you can implement it quite easily (with some performance hit, of course):
import numpy
class Lambda:
def __init__(self, func):
self._func = func
def __add__(self, other):
return Lambda(
lambda *args, **kwds: self._func(*args, **kwds) + other._func(*args, **kwds))
def __call__(self, *args, **kwds):
return self._func(*args, **kwds)
y = Lambda(lambda u: u**(-2) + 8)
x = Lambda(lambda u: numpy.exp(-u))
print((x + y)(1))
Other operators can be added in a similar way.
With sympy you can do function operation like this:
>>> import numpy
>>> from sympy.utilities.lambdify import lambdify, implemented_function
>>> from sympy.abc import u
>>> y = implemented_function('y', lambda u: u**(-2) + 8)
>>> x = implemented_function('x', lambda u: numpy.exp(-u))
>>> f = lambdify(u, y(u) + x(u))
>>> f(numpy.array([1,2,3]))
array([ 9.36787944, 8.13533528, 8.04978707])
Use code below to rich same result with writing as less code as possible:
y = lambda u: u**(-2) + 8
x = lambda u: numpy.exp(-u)
f = lambda u, x=x, y=y: x(u) + y(u)
int = scipy.integrate.quad(f, 0, numpy.inf)
As a functional programmer, I suggest generalizing the solutions to an applicative combinator:
In [1]: def lift2(h, f, g): return lambda x: h(f(x), g(x))
In [2]: from operator import add
In [3]: from math import exp
In [4]: y = lambda u: u**(-2) + 8
In [5]: x = lambda u: exp(-u)
In [6]: f = lift2(add, y, x)
In [7]: [f(u) for u in range(1,5)]
Out[7]: [9.367879441171443, 8.385335283236612, 8.160898179478975, 8.080815638888733]
Using lift2, you can combine the output of two functions using arbitrary binary functions in a pointfree way. And most of the stuff in operator should probably be enough for typical mathematical combinations, avoiding having to write any lambdas.
In a similar fasion, you might want to define lift1 and maybe lift3, too.