I have a set of functions that take integers and functions as arguments. I'd like to memoize them.
I know that using this solution, I could use pickle to encode both sets of arguments and memoize the encoded values. In this particular use case, however, the function arguments are large and constant, and I'd rather not take up the lru_cache space with a function argument that, in the course of a program run, never changes.
Is there a way for me to memoize a partial function, where I've fixed the function arguments and have received a partial object that takes only hashable arguments? I can't figure out how to use the functools.lru_cache decorator as a function.
Here's what I've tried on a toy example. It doesn't work; the binomial tree still revisits nodes.
import functools
import logging
logging.basicConfig(level=logging.DEBUG)
def binomial_tree(x, y, fn):
logging.debug(f"binomial_tree({x}, {y})")
"""Note: this does not recombine, and we can't memoize function."""
if x == 10:
return fn(x, y)
else:
return 0.5 * binomial_tree(x + 1, y, fn) + 0.5 * binomial_tree(x + 1, y + 1, fn)
def memoize_fn(fn):
#functools.lru_cache(maxsize=None)
def inner(*args, **kwargs):
return fn(*args, **kwargs)
return inner
memoized_binomial_tree = memoize_fn(functools.partial(binomial_tree, fn=lambda x, y: 10 * x * y))
print(memoized_binomial_tree(0, 0))
Here is a way to memoize your toy example with binomial_tree without encoding and memoizing function arguments:
import functools
import logging
logging.basicConfig(level=logging.DEBUG)
def create_binomial_tree(fn):
#functools.lru_cache(maxsize=None)
def binomial_tree(x, y):
logging.debug(f"binomial_tree({x}, {y})")
if x == 10:
return fn(x, y)
else:
return 0.5 * binomial_tree(x + 1, y) + 0.5 * binomial_tree(x + 1, y + 1)
return binomial_tree
memoized_binomial_tree = create_binomial_tree(fn=lambda x, y: 10 * x * y)
print(memoized_binomial_tree(0, 0))
Maybe it can be applicable in your real use case?
Related
I'm looking for a nice functional way to do the following:
def add(x, y):
return x + y
def neg(x):
return -x
def c(x, y):
# Apply neg to inputs for add
_x = neg(x)
_y = neg(y)
return add(_x, _y)
neg_sum = c(2, 2) # -4
It seems related to currying, but all of the examples I can find use functions that only have one input variable. I would like something that looks like this:
def add(x, y):
return x + y
def neg(x):
return -x
c = apply(neg, add)
neg_sum = c(2, 2) # -4
This is a fairly direct way to do it:
def add(x, y):
return x + y
def neg(x):
return -x
def apply(g, f):
# h is a function that returns
# f(g(arg1), g(arg2), ...)
def h(*args):
return f(*map(g, args))
return h
# or this:
# def apply(g, f):
# return lambda *args: f(*map(g, args))
c = apply(neg, add)
neg_sum = c(2, 2) # -4
Note that when you use *myvar as an argument in a function definition, myvar becomes a list of all non-keyword arguments that are received. And if you call a function with *expression as an argument, then all the items in expression are unpacked and sent as separate arguments to the function. I use these two behaviors to make h accept an unknown list of arguments, then apply function g to each one (with map), then pass all of them as arguments to f.
A different approach, depending on how extensible you need this to be, is to create an object which implements your operator methods, which each return the same object, allowing you to chain operators together in arbitrary orders.
If you can cope with it always returning a list, you might be able to make it work.
class mathifier:
def __init__(self,values):
self.values = values
def neg(self):
self.values = [-value for value in self.values]
return self
def add(self):
self.values = [sum(self.values)]
return self
print (mathifier([2,3]).neg().add().values)
And you can still get your named function for any set of chained functions:
neg_add = lambda x : mathifier(x).neg().add()
print(neg_add([2,3]).values)
From Matthias Fripp's answer, I asked myself : I'd like to compose add and neg both ways : add_neg(*args) and neg_add(*args). This requires hacking Matthias suggestion a bit. The idea is to get some hint on the arity (number of args) of the functions to compose. This information is obtained with a bit of introspection, thanks to inspect module. With this in mind, we adapt the way args are passed through the chain of funcs. The main assumption here is that we deal with real functions, in the mathematical sense, i.e. functions returning ONE float, and taking at least one argument.
from functools import reduce
from inspect import getfullargspec
def arity_one(func):
spec = getfullargspec(func)
return len(spec[0])==1 and spec[1] is None
def add(*args):
return reduce(lambda x,y:x+y, args, 0)
def neg(x):
return -x
def compose(fun1,fun2):
def comp(*args):
if arity_one(fun2): return fun1(*(map( fun2, args)))
else: return fun1(fun2(*args))
return comp
neg_add = compose(neg, add)
add_neg = compose(add, neg)
print(f"-2+(-3) = {add_neg(2, 3)}")
print(f"-(2+3) = {neg_add(2, 3)}")
The solution is still very adhoc...
class well_2D:
def __init__(self,d):
self.d=d
def φ_x(self,l,x):
if l%2==0 :
return np.sqrt(2/self.d)*np.sin(np.pi*l*x/self.d)
else :
return np.sqrt(2/self.d)*np.cos(np.pi*l*x/self.d)
def φ_y(self,m,y):
if l%2==0 :
return np.sqrt(2/self.d)*np.sin(np.pi*m*y/self.d)
else :
return np.sqrt(2/self.d)*np.cos(np.pi*m*y/self.d)
def Ψ(self,l,m,x,y):
return φ_x*φ_y
Usage:
Q=well_2D(2)
Q.Ψ(2,1,2,2)
>>>TypeError: unsupported operand type(s) for *: 'function' and 'function'
Based on the function signatures, you meant to write this:
return self.φ_x(l, x) * self.φ_y(m, y)
BTW l is a bad variable name since it looks like 1 and I. You could use L instead.
For various reasons, you cannot multiply functions themselves, only the results of calling them. In this case, I would refactor your code significantly:
class Well2D:
def __init__(self, d):
self.d = d
def φ(self, l, x):
f = np.sin if l % 2 == 0 else np.cos
return np.sqrt(2/self.d) * f(np.pi * l * x / self.d)
def Ψ(self, l, m, x, y):
return self.φ(l, x) * self.φ(m, y)
Inside φ, the only difference between the two cases is the trigonometric function you call based on the parity of l. There is no difference between φ_x and φ_y, other than the arguments you pass to each call, so just define a single function φ.
As explained here, to use scipy.optimize.minimize_scalar we need to define the objective function such as:
def f(x):
return (x - 2) * x * (x + 2)**2
Then, we will optimize it by:
from scipy.optimize import minimize_scalar
res = minimize_scalar(f)
Now, I want to define my function with a variable to optimize and several parameters. For example, some thing like:
def f(x, a, b):
return (x - a) * x * (x + a)**a + b
res = minimize_scalar(f(x, 2, 3))
How can I define the function and use it like that?
Please note that because a and b can be different each time, I cannot define them within the function definition.
Use the args argument:
args : tuple, optional
Extra arguments passed to the objective function.
The correct syntax looks like this:
res = minimize_scalar(f, args=(2, 3))
I am studying the properties of functions in Python and I came across an exercise that asks to:
Write a function which returns de power of a number. Conditions: The function may only take 1 argument and must use another function to return the value of the power of a given number.
The code that solves this exercise is:
def power(x):
return lambda y: y**x
For example, if we would like to know the value of the power: 2^3, we would call the function like this: power(3)(2)
Here is what I would like to know:
Is there any way to write a function that, when called, has a similar structure: function()()().
In other words, is it possible to write a function, that requires three or more parentheses ()()() when called?
If it is possible, could you please give me an example code of that function and briefly explain it?
Also:
def power(x):
def power_extra(y):
return y
def power_another(z):
return z
return power_extra and power_another
Possible?
Sure you can:
def power_times(k):
"""use as power_times(k)(x)(y) => k * y^x"""
return lambda x: lambda y: k * y**x
print power_times(2)(3)(4) # returns 2 * 4^3 = 128
When you call this function with argument 2 (power_times(2)), it returns a lambda function that works like lambda x: lambda y: 2 * y ** x (that is, like your original function, only with an extra "times 2").
You can stack as many lambdas on top of each other as you like:
def many_lambdas(x):
"""many_lambdas(x)(y)(z)(q) => x + y * z^q"""
return lambda y: lambda z: lambda q: x + y * z ** q
print many_lambdas(1)(2)(3)(4) # prints 163
Indeed, it might be even clearer if you skipped using def at all, and just wrote:
many_lambdas = lambda x: lambda y: lambda z: lambda q: x + y * z ** q
Or, alternatively, you could skip using lambda ever and just use them as nested functions:
def many_funcs(x):
def many_funcs_y(y):
def many_funcs_z(z):
def many_funcs_q(q):
return x + y * z ** q
return many_funcs_q
return many_funcs_z
return many_funcs_y
print many_funcs(1)(2)(3)(4) # prints 163
#David's answer would aptly answer you question for fixed nested function calls. For undefined nesting, you may want to define a class and overload the __call__ method along with __repr__ and __int__ to serve your Purpose.
>>> class Power(object):
def __init__(self, value):
self.value = value
def __call__(self, value):
self.value **= value
return self
def __int__(self):
return self.value
def __repr__(self):
return str(self.value)
>>> print Power(2)(2)(2)(2)(2)
65536
>>> int(Power(2)(2)(2)(2)(2)) / 2
32768
This question already has answers here:
Python function as a function argument?
(10 answers)
Python Argument Binders
(7 answers)
Closed 5 months ago.
Is it possible to pass functions with arguments to another function in Python?
Say for something like:
def perform(function):
return function()
But the functions to be passed will have arguments like:
action1()
action2(p)
action3(p,r)
Do you mean this?
def perform(fun, *args):
fun(*args)
def action1(args):
# something
def action2(args):
# something
perform(action1)
perform(action2, p)
perform(action3, p, r)
This is what lambda is for:
def perform(f):
f()
perform(lambda: action1())
perform(lambda: action2(p))
perform(lambda: action3(p, r))
You can use the partial function from functools like so.
from functools import partial
def perform(f):
f()
perform(Action1)
perform(partial(Action2, p))
perform(partial(Action3, p, r))
Also works with keywords
perform(partial(Action4, param1=p))
Use functools.partial, not lambdas! And ofc Perform is a useless function, you can pass around functions directly.
for func in [Action1, partial(Action2, p), partial(Action3, p, r)]:
func()
This is called partial functions and there are at least 3 ways to do this. My favorite way is using lambda because it avoids dependency on extra package and is the least verbose. Assume you have a function add(x, y) and you want to pass add(3, y) to some other function as parameter such that the other function decides the value for y.
Use lambda
# generic function takes op and its argument
def runOp(op, val):
return op(val)
# declare full function
def add(x, y):
return x+y
# run example
def main():
f = lambda y: add(3, y)
result = runOp(f, 1) # is 4
Create Your Own Wrapper
Here you need to create a function that returns the partial function. This is obviously lot more verbose.
# generic function takes op and its argument
def runOp(op, val):
return op(val)
# declare full function
def add(x, y):
return x+y
# declare partial function
def addPartial(x):
def _wrapper(y):
return add(x, y)
return _wrapper
# run example
def main():
f = addPartial(3)
result = runOp(f, 1) # is 4
Use partial from functools
This is almost identical to lambda shown above. Then why do we need this? There are few reasons. In short, partial might be bit faster in some cases (see its implementation) and that you can use it for early binding vs lambda's late binding.
from functools import partial
# generic function takes op and its argument
def runOp(op, val):
return op(val)
# declare full function
def add(x, y):
return x+y
# run example
def main():
f = partial(add, 3)
result = runOp(f, 1) # is 4
(months later) a tiny real example where lambda is useful, partial not:
say you want various 1-dimensional cross-sections through a 2-dimensional function,
like slices through a row of hills.
quadf( x, f ) takes a 1-d f and calls it for various x.
To call it for vertical cuts at y = -1 0 1 and horizontal cuts at x = -1 0 1,
fx1 = quadf( x, lambda x: f( x, 1 ))
fx0 = quadf( x, lambda x: f( x, 0 ))
fx_1 = quadf( x, lambda x: f( x, -1 ))
fxy = parabola( y, fx_1, fx0, fx1 )
f_1y = quadf( y, lambda y: f( -1, y ))
f0y = quadf( y, lambda y: f( 0, y ))
f1y = quadf( y, lambda y: f( 1, y ))
fyx = parabola( x, f_1y, f0y, f1y )
As far as I know, partial can't do this --
quadf( y, partial( f, x=1 ))
TypeError: f() got multiple values for keyword argument 'x'
(How to add tags numpy, partial, lambda to this ?)
Although all the responses are very accurate and well explained.
I want to make a clarification that you also can pass anonymous functions.
def perform(fun, *arg):
return fun(*arg)
# Pass anonymous function
print(perform(lambda x: x + 1, 3)) # output: 4
print(perform(lambda x, y: x + y + 1, 3, 2)) # output: 6
# Pass defined function
perform(lambda: action1())
perform(lambda: action2(p))
perform(lambda: action3(p, r))
Here is a way to do it with a closure:
def generate_add_mult_func(func):
def function_generator(x):
return reduce(func,range(1,x))
return function_generator
def add(x,y):
return x+y
def mult(x,y):
return x*y
adding=generate_add_mult_func(add)
multiplying=generate_add_mult_func(mult)
print adding(10)
print multiplying(10)
I think this is what you're looking for...
def action1(action):
print(f'doing {action} here!')
def perform(function):
return function()
perform(lambda : action1('business action'))
lambda packages up func and args in closure and passes to perform()
Thanks to David Beasley.