How can I bind arguments to a Python function so that I can call it later without arguments (or with fewer additional arguments)?
For example:
def add(x, y):
return x + y
add_5 = magic_function(add, 5)
assert add_5(3) == 8
What is the magic_function I need here?
It often happens with frameworks and libraries that people accidentally call a function immediately when trying to give arguments to a callback: for example on_event(action(foo)). The solution is to bind foo as an argument to action, using one of the techniques described here. See for example How to pass arguments to a Button command in Tkinter? and Using a dictionary as a switch statement in Python.
Some APIs, however, allow you to pass the to-be-bound arguments separately, and will do the binding for you. Notably, the threading API in the standard library works this way. See thread starts running before calling Thread.start. If you are trying to set up your own API like this, see How can I write a simple callback function?.
Explicitly binding arguments is also a way to avoid problems caused by late binding when using closures. This is the problem where, for example, a lambda inside a for loop or list comprehension produces separate functions that compute the same result. See What do lambda function closures capture? and Creating functions (or lambdas) in a loop (or comprehension).
functools.partial returns a callable wrapping a function with some or all of the arguments frozen.
import sys
import functools
print_hello = functools.partial(sys.stdout.write, "Hello world\n")
print_hello()
Hello world
The above usage is equivalent to the following lambda.
print_hello = lambda *a, **kw: sys.stdout.write("Hello world\n", *a, **kw)
Using functools.partial:
>>> from functools import partial
>>> def f(a, b):
... return a+b
...
>>> p = partial(f, 1, 2)
>>> p()
3
>>> p2 = partial(f, 1)
>>> p2(7)
8
If functools.partial is not available then it can be easily emulated:
>>> make_printer = lambda s: lambda: sys.stdout.write("%s\n" % s)
>>> import sys
>>> print_hello = make_printer("hello")
>>> print_hello()
hello
Or
def partial(func, *args, **kwargs):
def f(*args_rest, **kwargs_rest):
kw = kwargs.copy()
kw.update(kwargs_rest)
return func(*(args + args_rest), **kw)
return f
def f(a, b):
return a + b
p = partial(f, 1, 2)
print p() # -> 3
p2 = partial(f, 1)
print p2(7) # -> 8
d = dict(a=2, b=3)
p3 = partial(f, **d)
print p3(), p3(a=3), p3() # -> 5 6 5
lambdas allow you to create a new unnamed function with fewer arguments and call the function:
>>> def foobar(x, y, z):
... print(f'{x}, {y}, {z}')
...
>>> foobar(1, 2, 3) # call normal function
1, 2, 3
>>> bind = lambda x: foobar(x, 10, 20) # bind 10 and 20 to foobar
>>> bind(1)
1, 10, 20
>>> bind = lambda: foobar(1, 2, 3) # bind all elements
>>> bind()
1, 2, 3
You can also use functools.partial. If you are planning to use named argument binding in the function call this is also applicable:
>>> from functools import partial
>>> barfoo = partial(foobar, x=10)
>>> barfoo(y=5, z=6)
10, 5, 6
Note that if you bind arguments from the left you need to call the arguments by name. If you bind from the right it works as expected.
>>> barfoo(5, 6)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foobar() got multiple values for argument 'x'
>>> f = partial(foobar, z=20)
>>> f(1, 1)
1, 1, 20
This would work, too:
def curry(func, *args):
def curried(*innerargs):
return func(*(args+innerargs))
curried.__name__ = "%s(%s, ...)" % (func.__name__, ", ".join(map(str, args)))
return curried
>>> w=curry(sys.stdout.write, "Hey there")
>>> w()
Hey there
Functors can be defined this way in Python. They're callable objects. The "binding" merely sets argument values.
class SomeFunctor( object ):
def __init__( self, arg1, arg2=None ):
self.arg1= arg1
self.arg2= arg2
def __call___( self, arg1=None, arg2=None ):
a1= arg1 or self.arg1
a2= arg2 or self.arg2
# do something
return
You can do things like
x= SomeFunctor( 3.456 )
x( arg2=123 )
y= SomeFunctor( 3.456, 123 )
y()
The question asks generally about binding arguments, but all answers are about functions. In case you are wondering, partial also works with class constructors (i.e. using a class instead of a function as a first argument), which can be useful for factory classes. You can do it as follows:
from functools import partial
class Animal(object):
def __init__(self, weight, num_legs):
self.weight = weight
self.num_legs = num_legs
animal_class = partial(Animal, weight=12)
snake = animal_class(num_legs = 0)
print(snake.weight) # prints 12
Related
How can I bind arguments to a Python function so that I can call it later without arguments (or with fewer additional arguments)?
For example:
def add(x, y):
return x + y
add_5 = magic_function(add, 5)
assert add_5(3) == 8
What is the magic_function I need here?
It often happens with frameworks and libraries that people accidentally call a function immediately when trying to give arguments to a callback: for example on_event(action(foo)). The solution is to bind foo as an argument to action, using one of the techniques described here. See for example How to pass arguments to a Button command in Tkinter? and Using a dictionary as a switch statement in Python.
Some APIs, however, allow you to pass the to-be-bound arguments separately, and will do the binding for you. Notably, the threading API in the standard library works this way. See thread starts running before calling Thread.start. If you are trying to set up your own API like this, see How can I write a simple callback function?.
Explicitly binding arguments is also a way to avoid problems caused by late binding when using closures. This is the problem where, for example, a lambda inside a for loop or list comprehension produces separate functions that compute the same result. See What do lambda function closures capture? and Creating functions (or lambdas) in a loop (or comprehension).
functools.partial returns a callable wrapping a function with some or all of the arguments frozen.
import sys
import functools
print_hello = functools.partial(sys.stdout.write, "Hello world\n")
print_hello()
Hello world
The above usage is equivalent to the following lambda.
print_hello = lambda *a, **kw: sys.stdout.write("Hello world\n", *a, **kw)
Using functools.partial:
>>> from functools import partial
>>> def f(a, b):
... return a+b
...
>>> p = partial(f, 1, 2)
>>> p()
3
>>> p2 = partial(f, 1)
>>> p2(7)
8
If functools.partial is not available then it can be easily emulated:
>>> make_printer = lambda s: lambda: sys.stdout.write("%s\n" % s)
>>> import sys
>>> print_hello = make_printer("hello")
>>> print_hello()
hello
Or
def partial(func, *args, **kwargs):
def f(*args_rest, **kwargs_rest):
kw = kwargs.copy()
kw.update(kwargs_rest)
return func(*(args + args_rest), **kw)
return f
def f(a, b):
return a + b
p = partial(f, 1, 2)
print p() # -> 3
p2 = partial(f, 1)
print p2(7) # -> 8
d = dict(a=2, b=3)
p3 = partial(f, **d)
print p3(), p3(a=3), p3() # -> 5 6 5
lambdas allow you to create a new unnamed function with fewer arguments and call the function:
>>> def foobar(x, y, z):
... print(f'{x}, {y}, {z}')
...
>>> foobar(1, 2, 3) # call normal function
1, 2, 3
>>> bind = lambda x: foobar(x, 10, 20) # bind 10 and 20 to foobar
>>> bind(1)
1, 10, 20
>>> bind = lambda: foobar(1, 2, 3) # bind all elements
>>> bind()
1, 2, 3
You can also use functools.partial. If you are planning to use named argument binding in the function call this is also applicable:
>>> from functools import partial
>>> barfoo = partial(foobar, x=10)
>>> barfoo(y=5, z=6)
10, 5, 6
Note that if you bind arguments from the left you need to call the arguments by name. If you bind from the right it works as expected.
>>> barfoo(5, 6)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foobar() got multiple values for argument 'x'
>>> f = partial(foobar, z=20)
>>> f(1, 1)
1, 1, 20
This would work, too:
def curry(func, *args):
def curried(*innerargs):
return func(*(args+innerargs))
curried.__name__ = "%s(%s, ...)" % (func.__name__, ", ".join(map(str, args)))
return curried
>>> w=curry(sys.stdout.write, "Hey there")
>>> w()
Hey there
Functors can be defined this way in Python. They're callable objects. The "binding" merely sets argument values.
class SomeFunctor( object ):
def __init__( self, arg1, arg2=None ):
self.arg1= arg1
self.arg2= arg2
def __call___( self, arg1=None, arg2=None ):
a1= arg1 or self.arg1
a2= arg2 or self.arg2
# do something
return
You can do things like
x= SomeFunctor( 3.456 )
x( arg2=123 )
y= SomeFunctor( 3.456, 123 )
y()
The question asks generally about binding arguments, but all answers are about functions. In case you are wondering, partial also works with class constructors (i.e. using a class instead of a function as a first argument), which can be useful for factory classes. You can do it as follows:
from functools import partial
class Animal(object):
def __init__(self, weight, num_legs):
self.weight = weight
self.num_legs = num_legs
animal_class = partial(Animal, weight=12)
snake = animal_class(num_legs = 0)
print(snake.weight) # prints 12
It is easy to save function into a variable like
pr = print
pr(5) # 5
But if this possible to save function with arguments without a call, like
some_var = defer print(5) # No call!
some_var() # 5
I tried to use lambda, but it's lead to syntaxys error `l = lambda 5:
Why I need it? For example to no repeat multiple "if" branches:
example:
def foo()
l1 = lambda: 1
l2 = lambda: 2
if 1:
func = l1
elif 2:
func = l2
else:
func = some_outer_func, some_inner_func
return func # To use "func" need additional "if" branches for type and length of a returned value
functools.partial takes a function of many arguments and returns a function with fewer arguments with some of the arguments "saved"
from functools import partial
print_five = partial(print, 5)
print_five() # 5
It also works with keyword arguments
def foo(a, b=False):
print(a, b)
bar = partial(foo, b=True)
foo(1) # 1 False
bar(1) # 1 True
The way with lambda:
pr = lambda: print(5)
pr()
I am trying to use currying to make a simple functional add in Python. I found this curry decorator here.
def curry(func):
def curried(*args, **kwargs):
if len(args) + len(kwargs) >= func.__code__.co_argcount:
return func(*args, **kwargs)
return (lambda *args2, **kwargs2:
curried(*(args + args2), **dict(kwargs, **kwargs2)))
return curried
#curry
def foo(a, b, c):
return a + b + c
Now this is great because I can do some simple currying:
>>> foo(1)(2, 3)
6
>>> foo(1)(2)(3)
6
But this only works for exactly three variables. How do I write the function foo so that it can accept any number of variables and still be able to curry the result? I've tried the simple solution of using *args but it didn't work.
Edit: I've looked at the answers but still can't figure out how to write a function that can perform as shown below:
>>> foo(1)(2, 3)
6
>>> foo(1)(2)(3)
6
>>> foo(1)(2)
3
>>> foo(1)(2)(3)(4)
10
Arguably, explicit is better than implicit:
from functools import partial
def example(*args):
print("This is an example function that was passed:", args)
one_bound = partial(example, 1)
two_bound = partial(one_bound, 2)
two_bound(3)
#JohnKugelman explained the design problem with what you're trying to do - a call to the curried function would be ambiguous between "add more curried arguments" and "invoke the logic". The reason this isn't a problem in Haskell (where the concept comes from) is that the language evaluates everything lazily, so there isn't a distinction you can meaningfully make between "a function named x that accepts no arguments and simply returns 3" and "a call to the aforementioned function", or even between those and "the integer 3". Python isn't like that. (You could, for example, use a zero-argument call to signify "invoke the logic now"; but that would break special cases aren't special enough, and require an extra pair of parentheses for simple cases where you don't actually want to do any currying.)
functools.partial is an out-of-box solution for partial application of functions in Python. Unfortunately, repeatedly calling partial to add more "curried" arguments isn't quite as efficient (there will be nested partial objects under the hood). However, it's much more flexible; in particular, you can use it with existing functions that don't have any special decoration.
You can implement the same thing as the functools.partial example for yourself like this:
def curry (prior, *additional):
def curried(*args):
return prior(*(args + additional))
return curried
def add(*args):
return sum(args)
x = curry(add, 3,4,5)
y = curry(b, 100)
print y(200)
# 312
It may be easier to think of curry as a function factory rather than a decorator; technically that's all a decorator does but the decorator usage pattern is static where a factory is something you expect to be invoking as part of a chain of operations.
You can see here that I'm starting with add as an argument to curry and not add(1) or something: the factory signature is <callable>, *<args> . That gets around the problem in the comments to the original post.
FACT 1: It is simply impossible to implement an auto currying function for a variadic function.
FACT 2: You might not be searching for curry, if you want the function that will be passed to it * to know* that its gonna be curried, so as to make it behave differently.
In case what you need is a way to curry a variadic function, you should go with something along these lines below (using your own snipped):
def curryN(arity, func):
"""curries a function with a pre-determined number of arguments"""
def curried(*args, **kwargs):
if len(args) + len(kwargs) >= arity:
return func(*args, **kwargs)
return (lambda *args2, **kwargs2:
curried(*(args + args2), **dict(kwargs, **kwargs2)))
return curried
def curry(func):
"""automatically curries a function"""
return curryN(func.__code__.co_argcount, func);
this way you can do:
def summation(*numbers):
return sum(numbers);
sum_two_numbers = curryN(2, summation)
sum_three_numbers = curryN(3, summation)
increment = curryN(2, summation)(1)
decrement = curryN(2, summation)(-1)
I think this is a decent solution:
from copy import copy
import functools
def curry(function):
def inner(*args, **kwargs):
partial = functools.partial(function, *args, **kwargs)
signature = inspect.signature(partial.func)
try:
signature.bind(*partial.args, **partial.keywords)
except TypeError as e:
return curry(copy(partial))
else:
return partial()
return inner
This just allow you to call functools.partial recursively in an automated way:
def f(x, y, z, info=None):
if info:
print(info, end=": ")
return x + y + z
g = curry_function(f)
print(g)
print(g())
print(g(2))
print(g(2,3))
print(g(2)(3))
print(g(2, 3)(4))
print(g(2)(3)(4))
print(g(2)(3, 4))
print(g(2, info="test A")(3, 4))
print(g(2, info="test A")(3, 4, info="test B"))
Outputs:
<function curry.<locals>.inner at 0x7f6019aa6f28>
<function curry.<locals>.inner at 0x7f6019a9a158>
<function curry.<locals>.inner at 0x7f6019a9a158>
<function curry.<locals>.inner at 0x7f6019a9a158>
<function curry.<locals>.inner at 0x7f6019a9a0d0>
9
9
9
test A: 9
test B: 9
I try to understand python decorator
def dec(func):
def wrap(*args,**kw):
print args, kw
return func(*args,**kw)
return wrap
#dec
def myfunc(a=1,b=2,c=3):
return a+b+c
>>> myfunc()
() {}
6
>>> myfunc(1,2,3)
(1, 2, 3) {}
6
>>> myfunc(1,2,c=5)
(1, 2) {'c': 5}
8
>>>
When I run myfunc() args and kw are nothing, but when I run myfunc(1,2,3) and myfunc(1,2,c=5), args and kw were passed to dec function.
As I know,
#dec
def myfunc(...)
equals to myfunc = dec(myfunc) <-- no arguments were mentioned here.
How arguments were passed to wrap function in dec? How to understand these?
Not sure if i understand correctly your problem, but the default values for myfunc arguments are known only to myfunc - your decorator has no knowledge of them, so it cannot print them.
That's why:
myfunc()
results in printing:
() {}
Both *args and **kw are empty for the decorator, but the decorated function will use the default values in this case.
In the second and third case you get the parameters printed, as they are explicitly passed to the decorated function:
def wrap(*args,**kw): <- this is what is actually called when you invoke decorated function, no default values here
print args, kw
return func(*args,**kw) <- this is the function you decorate
#if func has default parameter values, then those will be used
#when you don't pass them to the wrapper but only **inside** func
return wrap
Edit:
It looks like you're mistaking calling the decorated function with decorating the function:
myfunc = dec(myfunc)
decorates myfunc using dec and is equivalent to
#dec
def myfunc(...)
On the other hand, after using either of them:
myfunc(whatever)
calls the wrap function defined in your decorator, which will in turn call the original myfunc
Another way to think of it is by saying:
def wrapper(some_function):
def _inner(*args, **kwargs):
#do some work
return func(*args, **kwargs)
return _inner
#wrapper
def foo(a, b, c):
return "Foo called, and I saw %d, %d, %d" % (a, b, c)
...you're getting a result which is roughly similar to the following:
def foo(a, b, c):
#do some work
return "Foo called, and I saw %d, %d, %d" % (a, b, c)
This isn't exactly right because the #do some work is occurring before the actual foo() call, but as an approximation this is what you're getting. For that reason, the wrapper can't really 'see' the default arguments for foo() if any exist. So a better way to think of it might be:
#always execute this code before calling...
def foo(a, b, c):
return "Foo called and I saw %d, %d, %d" % (a, b, c)
So something really basic might look like this.
def wrap(f):
... def _inner(*a, **k):
... new_a = (ar*2 for ar in a)
... new_k = {}
... for k, v in k.iteritems():
... new_k[k] = v*2
... return f(*new_a, **new_k)
... return _inner
...
>>> def foo(a=2, b=4, c=6):
... return "%d, %d, %d" % (a, b, c)
...
>>> foo()
'2, 4, 6'
>>> foo(1, 5, 7)
'1, 5, 7'
>>> foo = wrap(foo) #actually wrapping it here
>>> foo()
'2, 4, 6'
>>> foo(3, 5, 6)
'6, 10, 12'
>>> foo(3, 5, c=7)
'6, 10, 14'
>>>
Decorators are function wrappers. They give back a function that wraps the original one into some pre- and post-processing code, but still need to call the original function (normally with the same argument as you would call it in absence of a decorator).
Python has two types of arguments, positional and keyword arguments (this has nothing to do with decorators, that's generic python basics). * is for positional (internally is a list), ** for keyword (dictionary) arguments. By specifying both you allow your decorator to accept all at all possible types of arguments and pass them through to the underlying function. The contract of the call is, however, still defined by your function. E.g. if it only takes keyword arguments it will fail when the decorator function passes through a positional argument.
In your particular example, you have some pre-processing code (i.e. code that will run before the original function is called). For example in this code you can print out arguments *args that your original function might fail to accept all together because it does not take any position arguments.
You do not necessarily have to pass through *args and **kwargs. In fact you can define a decorator which makes some decisions based on the arguments you pass in about what to provide to the original function as arguments, e.g.
def dec(fun):
def wrap(*args, **kwargs):
if 'a' in kwargs:
return fun(kwargs[a])
else:
return fun(*args)
return wrap
So this works:
>>> x = 1
>>> y = 2
>>> "a={a}, b={b}, a+b={c}".format( a=x, b=y, c=x+y )
'a=1, b=2, a+b=3'
But this doesn't:
>>> "a={a}, b={b}, a+b={c}".format( a=x, b=y, c=a+b )
NameError: name 'a' is not defined
Is there any way to make the second one work? (Say for example that x and y are function calls and I don't want to recompute them during string formatting)
The most pythonic (readable, in this case) solution for this is not to use a lambda function, but to cache a and b before the format() call:
a = function_x()
b = function_y()
"a={a}, b={b}, a+b={c}".format(a=a, b=b, c=a+b)
You'll be thankful when looking at the code 6 months from now.
You can do it with lambda:
def x():
return 1
def y():
return 2
>>> "a={a},b={b}, a+b={c}".format(**(lambda a=x(),b=y():{'a':a,'b':b,'c':a+b})())
'a=1,b=2, a+b=3'
this lambda expression is equal to calling predefined function:
def twosumm(a, b):
return {'a':a, 'b':b, 'c': a+b}
>>> "a={a},b={b}, a+b={c}".format(**twosumm(x(), y()))
'a=1,b=2, a+b=3'
Im also think that it is better to use simple and readable solution and just call x() and y() to get results before formatiing:
>>> a, b = x(), y()
>>> "a={a},b={b}, a+b={c}".format(a=a, b=b, c=a+b)
'a=1,b=2, a+b=3'
x = 1
y = 2
def f(x,y):
return (x,y,x+y)
print "a={}, b={}, a+b={}".format( *f(x,y) )
# or
print "a={0[0]}, b={0[1]}, a+b={0[2]}".format( f(x,y) )
.
EDIT
I think your question is wrongly written and that induces blurry understanding of it, and then wrong answers.
x and y are not function calls. As they appear, they are just identifiers
If you evoke function calls, I think it is because, in fact, you wish to obtain the result of something like that:
"a={a}, b={b}, a+b={c}".format( a=f(), b=g(), c=f()+g() )
but without having to write c=f()+g() because it implies that f() and g() are each executed two times.
Firstly, it will forever be impossible in Python to write something like .format( a=x, b=y, c=a+b ) or .format( a=f(), b=g(), c=a+b ) where a and b in c=a+b will refer to the same objects as a and b in a=x and b=y.
Because any identifier at the left side of = is in the local namespace of format() while any identifier at the right side of = is in the namespace outside of the function format().
By the way, that's why the identifiers at the left are called parameters and the identifiers at the right are the identifiers of objects passed as arguments.
Secondly, if you want to avoid writing f() two times (one time as an alone argument and one time in the expression f()+g()), and the same for g(), that means you want to write each only one time, as alone argument.
So , if I understand you well, you essentially wish to write something like that:
"a={a}, b={b}, a+b={}".format( a=f(), b=g() )
With current method str.format , this expression with three replacement fields { } is evidently not correct.
No matter, let's redefine the method format ! And then it's possible to pass only two arguments to format().
def fx(): return 101
def fy(): return 45
class Pat(str):
def __init__(self,s):
self = s
def format(self,x,y):
return str.format(self,x,y,x+y)
p = Pat("a={}, b={}, a+b={}")
print 'p==',p
print p.format(fx(),fy())
result
p : a={}, b={}, a+b={}
a=101, b=45, a+b=146
We can even do more complex things:
from sys import exit
import re
def fx(): return 333
def fy(): return 6
class Pat(str):
def __init__(self,s):
for x in re.findall('(?<=\{)[^}]+(?=\})',s):
if x not in ('A','M'):
mess = " The replacement field {%s] isn't recognised" % x
exit(mess)
self.orig = s
self.mod = re.sub('\{[^}]*\}','{}',s)
def modif(self,R):
it = iter(R)
return tuple(sum(R) if x=='{A}'
else reduce(lambda a,b: a*b, R) if x=='{M}'
else next(it)
for x in re.findall('(\{[^}]*\})',self))
def format(self,*args):
return ''.join(self.mod.format(*self.modif(args)))
print Pat("a={}, b={}, a+b={A}").format(fx(),fy())
print '******************************************'
print Pat("a={}, b={}, c={}, a+b+c={A}").format(fx(),fy(),5000)
print '******************************************'
print Pat("a={}, b={}, a*b={M}").format(fx(),fy())
print '******************************************'
print Pat("a={}, b={}, axb={X}").format(fx(),fy())
result
a=333, b=6, a+b=339
******************************************
a=333, b=6, c=5000, a+b+c=5339
******************************************
a=333, b=6, a*b=1998
******************************************
Traceback (most recent call last):
File "I:\potoh\ProvPy\Copie de nb.py", line 70, in <module>
print Pat("a={}, b={}, axb={X}").format(fx(),fy())
File "I:\potoh\ProvPy\Copie de nb.py", line 51, in __init__
exit(mess)
SystemExit: The replacement field {X] isn't recognised