This question already has answers here:
Calling a Python function with *args,**kwargs and optional / default arguments
(4 answers)
Closed 4 years ago.
I'm trying to wrap my head around using args and kwargs in Python 3 (Python 3.7.0) but I'm running into some issues with my understanding.
Here is a simple function I have:
def add_args(y=10, *args, **kwargs):
return y, args, kwargs
And test it to see what is returned:
print(add_args(1, 5, 10, 20, 50))
print(add_args())
>>
(1, (5, 10, 20, 50), {}) # What happened to my default y=10?
(10, (), {})
What I don't understand is, what happened to y=10 in the first print statement? I can see it is being overridden by the 1 in args but I'm unsure why.
How can I rewrite this function so the default value is not overridden, or am I missing something with how the parameters are passed from the function signature to the return statement?
I tried looking here and here but did not find the answers I was looking for. As I thought putting the default values before the args and kwargs would prevent the overwriting.
*args only captures any positional arguments not otherwise defined; y=10 does not mean y can't be used as a positional argument. So y is assigned the first positional argument.
You can prevent y being used as a positional argument by making it a keyword-only argument. You do this by placing the argument after the *args var-positional catch-all parameter, or if you don't have a *name parameter, after a * single asterisk:
def add_args(*args, y=10, **kwargs):
return y, args, kwargs
or
def keyword_only_args(*, y=10, **kwargs):
return y, kwargs
Now y won't capture positional arguments any more:
>>> def add_args(*args, y=10, **kwargs):
... return y, args, kwargs
...
>>> add_args(1, 5, 10, 20, 50)
(10, (1, 5, 10, 20, 50), {}) # y is still 10
>>> add_args(1, 5, 10, 20, 50, y=42) # setting y explicitly
(42, (1, 5, 10, 20, 50), {})
You don't have to have a **kwargs keyword catch-all either:
def add_args(*args, y=10):
return y, args
but if it is present, it needs to be listed last.
Keyword-only arguments do not have to have a default value, the =10 can be omitted, but then the parameter becomes mandatory, and can only be specified in a call by using y=value:
>>> def add_args(*args, y): # mandatory keyword-only argument
... return y, args
...
>>> add_args(1, 5, 10, 20, 50)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: add_args() missing 1 required keyword-only argument: 'y'
>>> add_args(1, 5, 10, 20, 50, y=42)
(42, (1, 5, 10, 20, 50))
Hmm, y is the first positional argument in your function definition, so naturally it binds to the first actual positional argument at callsite, which is 1.
Related
I tried to use * and ** to pass any number of arguments to a function. In "Learning Python" authored by Mark Lutz, it says to follow the order of positional (value) first, then a combination of keyword arguments (name=value) and *sequence, followed by **dict. However, I found that the positional arguments need to come first if present, but the rest of three, to certain extent, can be mixed in order.
Code keywords3.py:
def func(a, b=1, *pArgs, **kwArgs):
print("a = {0}, b = {1}".format(a,b))
print("Positional args = {}".format(pArgs))
print("Keyword args = {}".format(kwArgs))
By trial-and-error,
[1] Between keywords and **dict, they can be in any order...
>>> import keywords3 as j
>>> j.func(b = 3, **{'a':2,'c':4,'d':5})
a = 2, b = 3
Positional args = ()
Keyword args = {'d': 5, 'c': 4}
>>> j.func( **{'a':2}, b = 3, **{'c':4})
a = 2, b = 3
Positional args = ()
Keyword args = {'c': 4}
[2] Between positional args and *sequence, they can be in any order...
>>> j.func(*(2, 3), 4, *(5, 6))
a = 2, b = 3
Positional args = (4, 5, 6)
Keyword args = {}
>>> j.func(2, *(3, 4), 5, *(6,7), **{'c':8})
a = 2, b = 3
Positional args = (4, 5, 6, 7)
Keyword args = {'c': 8}
[3] In general, positional or *sequence arguments need to appear before keyword or **dict arguments.
>>> j.func(*(3, 4), 5, *(6,7), d=15, **{'c':8}, e=16)
a = 3, b = 4
Positional args = (5, 6, 7)
Keyword args = {'e': 16, 'd': 15, 'c': 8}
>>> j.func(d=15, 5, *(6,7), **{'c':8}, e=16)
File "<stdin>", line 1
SyntaxError: positional argument follows keyword argument
>>> j.func(**{'a':2}, 5, *(6,7), **{'c':8}, e=16)
File "<stdin>", line 1
SyntaxError: positional argument follows keyword argument unpacking
>>> j.func(**{'a':2}, *(6,7), **{'c':8}, e=16)
File "<stdin>", line 1
SyntaxError: iterable argument unpacking follows keyword argument unpacking
[4] One exception is that iterable argument unpacking *(6,7) following keyword argument is ok...
>>> j.func(f=5, *(6,7), **{'c':8}, e=16)
a = 6, b = 7
Positional args = ()
Keyword args = {'e': 16, 'f': 5, 'c': 8}
Are these observations correct? Please comment.
There is one single rule consistent with all your examples: positional arguments go before named arguments.
In all your examples, * and ** are unpacking operators. So, for example, when you write
f(1, *(2,3), 4)
the language gets
f(1,2,3,4)
They are all positional arguments, the language doesn't know the difference. Similarly for the ** operator.
However, when you violate that only rule, eg
j.func(**{'a':2}, 5, *(6,7), **{'c':8}, e=16)
you get an error, becuase, in this example, **{'a':2} is equivalent to a=2, which precedes the positional argument 5.
I have a function that accepts *args, but I would like to set a default tuple, in case none are provided. (This is not possible through def f(*args=(1, 3, 5)), which raises a SyntaxError.) What would be the best way to accomplish this? The intended functionality is shown below.
f()
# I received 1, 2, 3!
f(1)
# I received 1!
f(9, 3, 72)
# I received 9, 3, 72!
The following function g will provide the correct functionality, but I would prefer *args.
def g(args=(1, 2, 3)):
return "I received {}!".format(', '.join(str(arg) for arg in args))
g()
# I received 1, 2, 3!
g((1,))
# I received 1!
g((9, 3, 72))
# I received 9, 3, 72!
You could check whether args are truthy in your function:
def g(*args):
if not args:
args = (1, 2, 3)
return "I received {}!".format(', '.join(str(arg) for arg in args))
If no args are passed to the function, it will result in a empty tuple, which evaluates to False.
If no arguments are received, args will be an empty tuple. You can't add a default value in the method signature itself, but you can check if args is empty and replace it with a fallback value inside the function.
def g(*args):
if not args:
args = (1, 2, 3)
return 'I received {}!'.format(', '.join(str(arg) for arg in args))
def g(*args):
if not args:
args = [1, 3, 5]
return "I received {}!".format(', '.join(str(arg) for arg in args))
This question already has answers here:
Python, default keyword arguments after variable length positional arguments
(2 answers)
Closed 9 years ago.
I am trying to figure out if I can leave an optional argument out (use it's default value) when using *args in Python. The following code works through "print(a)", so explicitly including the optional argument h in the deriv function call works. Can I leave it (h) out somehow? My attempts ("b = ...", "c = ...", "d = ...") fail. Is there another way?
def deriv(f, x, h=1.e-9, *params):
return (f(x+h, *params)-f(x-h, *params))/(2.*h)
def f1(x, a, p):
return a*x**p
a = deriv(f1, 3, 1.e-9, 4, 5)
print(a)
b = deriv(f1, 3, , 4, 5)
c = deriv(f1, 3, 4, 5)
d = deriv(f1, 3, h, 4, 5)
No, python applies positional arguments to all named arguments first; h is the third positional argument in the function signature and thus only argument positions 4 and over are captured by *params.
Instead, use a **kwargs argument catching arbitrary keyword arguments and look h up in that:
def deriv(f, x, *params, **kwargs):
h = kwargs.pop('h', 1.e-9)
You'll now have to name h explicitly when calling deriv:
b = deriv(f1, 3, 4, 5, h=2.0)
It looks like you're using Python 3, so you can use keyword-only arguments:
>>> def deriv(f, x, *params, h=1.0E-9):
print(f)
print(x)
print(params)
print(h)
>>> deriv(pow, 'x', 10, 20, 30)
<built-in function pow>
x
(10, 20, 30)
1e-09
>>> deriv(pow, 'x', 10, 20, 30, h=.2)
<built-in function pow>
x
(10, 20, 30)
0.2
Take for example the python built in pow() function.
xs = [1,2,3,4,5,6,7,8]
from functools import partial
list(map(partial(pow,2),xs))
>>> [2, 4, 8, 16, 32, 128, 256]
but how would I raise the xs to the power of 2?
to get [1, 4, 9, 16, 25, 49, 64]
list(map(partial(pow,y=2),xs))
TypeError: pow() takes no keyword arguments
I know list comprehensions would be easier.
No
According to the documentation, partial cannot do this (emphasis my own):
partial.args
The leftmost positional arguments that will be prepended to the positional arguments
You could always just "fix" pow to have keyword args:
_pow = pow
pow = lambda x, y: _pow(x, y)
I think I'd just use this simple one-liner:
import itertools
print list(itertools.imap(pow, [1, 2, 3], itertools.repeat(2)))
Update:
I also came up with a funnier than useful solution. It's a beautiful syntactic sugar, profiting from the fact that the ... literal means Ellipsis in Python3. It's a modified version of partial, allowing to omit some positional arguments between the leftmost and rightmost ones. The only drawback is that you can't pass anymore Ellipsis as argument.
import itertools
def partial(func, *args, **keywords):
def newfunc(*fargs, **fkeywords):
newkeywords = keywords.copy()
newkeywords.update(fkeywords)
return func(*(newfunc.leftmost_args + fargs + newfunc.rightmost_args), **newkeywords)
newfunc.func = func
args = iter(args)
newfunc.leftmost_args = tuple(itertools.takewhile(lambda v: v != Ellipsis, args))
newfunc.rightmost_args = tuple(args)
newfunc.keywords = keywords
return newfunc
>>> print partial(pow, ..., 2, 3)(5) # (5^2)%3
1
>>> print partial(pow, 2, ..., 3)(5) # (2^5)%3
2
>>> print partial(pow, 2, 3, ...)(5) # (2^3)%5
3
>>> print partial(pow, 2, 3)(5) # (2^3)%5
3
So the the solution for the original question would be with this version of partial list(map(partial(pow, ..., 2),xs))
Why not just create a quick lambda function which reorders the args and partial that
partial(lambda p, x: pow(x, p), 2)
You could create a helper function for this:
from functools import wraps
def foo(a, b, c, d, e):
print('foo(a={}, b={}, c={}, d={}, e={})'.format(a, b, c, d, e))
def partial_at(func, index, value):
#wraps(func)
def result(*rest, **kwargs):
args = []
args.extend(rest[:index])
args.append(value)
args.extend(rest[index:])
return func(*args, **kwargs)
return result
if __name__ == '__main__':
bar = partial_at(foo, 2, 'C')
bar('A', 'B', 'D', 'E')
# Prints: foo(a=A, b=B, c=C, d=D, e=E)
Disclaimer: I haven't tested this with keyword arguments so it might blow up because of them somehow. Also I'm not sure if this is what #wraps should be used for but it seemed right -ish.
you could use a closure
xs = [1,2,3,4,5,6,7,8]
def closure(method, param):
def t(x):
return method(x, param)
return t
f = closure(pow, 2)
f(10)
f = closure(pow, 3)
f(10)
You can do this with lambda, which is more flexible than functools.partial():
pow_two = lambda base: pow(base, 2)
print(pow_two(3)) # 9
More generally:
def bind_skip_first(func, *args, **kwargs):
return lambda first: func(first, *args, **kwargs)
pow_two = bind_skip_first(pow, 2)
print(pow_two(3)) # 9
One down-side of lambda is that some libraries are not able to serialize it.
One way of doing it would be:
def testfunc1(xs):
from functools import partial
def mypow(x,y): return x ** y
return list(map(partial(mypow,y=2),xs))
but this involves re-defining the pow function.
if the use of partial was not 'needed' then a simple lambda would do the trick
def testfunc2(xs):
return list(map(lambda x: pow(x,2), xs))
And a specific way to map the pow of 2 would be
def testfunc5(xs):
from operator import mul
return list(map(mul,xs,xs))
but none of these fully address the problem directly of partial applicaton in relation to keyword arguments
Even though this question was already answered, you can get the results you're looking for with a recipe taken from itertools.repeat:
from itertools import repeat
xs = list(range(1, 9)) # [1, 2, 3, 4, 5, 6, 7, 8]
xs_pow_2 = list(map(pow, xs, repeat(2))) # [1, 4, 9, 16, 25, 36, 49, 64]
Hopefully this helps someone.
Yes, you can do it, provided the function takes keyword arguments. You just need to know the name.
In the case of pow() (provided you are using Python 3.8 or newer) you need exp instead of y.
Try to do:
xs = [1,2,3,4,5,6,7,8]
print(list(map(partial(pow,exp=2),xs)))
As already said that's a limitation of functools.partial if the function you want to partial doesn't accept keyword arguments.
If you don't mind using an external library 1 you could use iteration_utilities.partial which has a partial that supports placeholders:
>>> from iteration_utilities import partial
>>> square = partial(pow, partial._, 2) # the partial._ attribute represents a placeholder
>>> list(map(square, xs))
[1, 4, 9, 16, 25, 36, 49, 64]
1 Disclaimer: I'm the author of the iteration_utilities library (installation instructions can be found in the documentation in case you're interested).
The very versatile funcy includes an rpartial function that exactly addresses this problem.
xs = [1,2,3,4,5,6,7,8]
from funcy import rpartial
list(map(rpartial(pow, 2), xs))
# [1, 4, 9, 16, 25, 36, 49, 64]
It's just a lambda under the hood:
def rpartial(func, *args):
"""Partially applies last arguments."""
return lambda *a: func(*(a + args))
If you can't use lambda functions, you can also write a simple wrapper function that reorders the arguments.
def _pow(y, x):
return pow(x, y)
and then call
list(map(partial(_pow,2),xs))
>>> [1, 4, 9, 16, 25, 36, 49, 64]
Yes
if you created your partial class
class MyPartial:
def __init__(self, func, *args):
self._func = func
self._args = args
def __call__(self, *args):
return self._func(*args, *self._args) # swap ordering
xs = [1,2,3,4,5,6,7,8]
list(map(MyPartial(pow,2),xs))
>>> [1, 4, 9, 16, 25, 36, 49, 64]
For python, I could use unpacking arguments as follows.
def hello(x, *y, **z):
print 'x', x
print 'y', y
print 'z', z
hello(1, *[1,2,3], a=1,b=2,c=3)
hello(1, *(1,2,3), **{'a':1,'b':2,'c':3})
x = 1
y = (1, 2, 3)
z = {'a': 1, 'c': 3, 'b': 2}
But, I got an error if I use keyword argument as follows.
hello(x=1, *(1,2,3), **{'a':1,'b':2,'c':3})
TypeError: hello() got multiple values for keyword argument 'x'
Why is this?
Regardless of the order in which they are specified, positional arguments get assigned prior to keyword arguments. In your case, the positional arguments are (1, 2, 3) and the keyword arguments are x=1, a=1, b=2, c=3. Because positional arguments get assigned first, the parameter x receives 1 and is not eligible for keyword arguments any more. This sounds a bit weird because syntactically your positional arguments are specified after the keyword argument, but nonetheless the order “positional arguments → keyword arguments” is always adhered to.
Here is a simpler example:
>>> def f(x): pass
...
>>> f(1, x=2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() got multiple values for keyword argument 'x'
>>> f(x=2, *(1,))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() got multiple values for keyword argument 'x'