Related
This question already has answers here:
What does ** (double star/asterisk) and * (star/asterisk) do for parameters?
(25 answers)
Closed 9 years ago.
So I have difficulty with the concept of *args and **kwargs.
So far I have learned that:
*args = list of arguments - as positional arguments
**kwargs = dictionary - whose keys become separate keyword arguments and the values become values of these arguments.
I don't understand what programming task this would be helpful for.
Maybe:
I think to enter lists and dictionaries as arguments of a function AND at the same time as a wildcard, so I can pass ANY argument?
Is there a simple example to explain how *args and **kwargs are used?
Also the tutorial I found used just the "*" and a variable name.
Are *args and **kwargs just placeholders or do you use exactly *args and **kwargs in the code?
The syntax is the * and **. The names *args and **kwargs are only by convention but there's no hard requirement to use them.
You would use *args when you're not sure how many arguments might be passed to your function, i.e. it allows you pass an arbitrary number of arguments to your function. For example:
>>> def print_everything(*args):
for count, thing in enumerate(args):
... print( '{0}. {1}'.format(count, thing))
...
>>> print_everything('apple', 'banana', 'cabbage')
0. apple
1. banana
2. cabbage
Similarly, **kwargs allows you to handle named arguments that you have not defined in advance:
>>> def table_things(**kwargs):
... for name, value in kwargs.items():
... print( '{0} = {1}'.format(name, value))
...
>>> table_things(apple = 'fruit', cabbage = 'vegetable')
cabbage = vegetable
apple = fruit
You can use these along with named arguments too. The explicit arguments get values first and then everything else is passed to *args and **kwargs. The named arguments come first in the list. For example:
def table_things(titlestring, **kwargs)
You can also use both in the same function definition but *args must occur before **kwargs.
You can also use the * and ** syntax when calling a function. For example:
>>> def print_three_things(a, b, c):
... print( 'a = {0}, b = {1}, c = {2}'.format(a,b,c))
...
>>> mylist = ['aardvark', 'baboon', 'cat']
>>> print_three_things(*mylist)
a = aardvark, b = baboon, c = cat
As you can see in this case it takes the list (or tuple) of items and unpacks it. By this it matches them to the arguments in the function. Of course, you could have a * both in the function definition and in the function call.
One place where the use of *args and **kwargs is quite useful is for subclassing.
class Foo(object):
def __init__(self, value1, value2):
# do something with the values
print value1, value2
class MyFoo(Foo):
def __init__(self, *args, **kwargs):
# do something else, don't care about the args
print 'myfoo'
super(MyFoo, self).__init__(*args, **kwargs)
This way you can extend the behaviour of the Foo class, without having to know too much about Foo. This can be quite convenient if you are programming to an API which might change. MyFoo just passes all arguments to the Foo class.
Here's an example that uses 3 different types of parameters.
def func(required_arg, *args, **kwargs):
# required_arg is a positional-only parameter.
print required_arg
# args is a tuple of positional arguments,
# because the parameter name has * prepended.
if args: # If args is not empty.
print args
# kwargs is a dictionary of keyword arguments,
# because the parameter name has ** prepended.
if kwargs: # If kwargs is not empty.
print kwargs
>>> func()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: func() takes at least 1 argument (0 given)
>>> func("required argument")
required argument
>>> func("required argument", 1, 2, '3')
required argument
(1, 2, '3')
>>> func("required argument", 1, 2, '3', keyword1=4, keyword2="foo")
required argument
(1, 2, '3')
{'keyword2': 'foo', 'keyword1': 4}
Here's one of my favorite places to use the ** syntax as in Dave Webb's final example:
mynum = 1000
mystr = 'Hello World!'
print("{mystr} New-style formatting is {mynum}x more fun!".format(**locals()))
I'm not sure if it's terribly fast when compared to just using the names themselves, but it's a lot easier to type!
One case where *args and **kwargs are useful is when writing wrapper functions (such as decorators) that need to be able accept arbitrary arguments to pass through to the function being wrapped. For example, a simple decorator that prints the arguments and return value of the function being wrapped:
def mydecorator( f ):
#functools.wraps( f )
def wrapper( *args, **kwargs ):
print "Calling f", args, kwargs
v = f( *args, **kwargs )
print "f returned", v
return v
return wrapper
*args and **kwargs are special-magic features of Python.
Think of a function that could have an unknown number of arguments. For example, for whatever reasons, you want to have function that sums an unknown number of numbers (and you don't want to use the built-in sum function). So you write this function:
def sumFunction(*args):
result = 0
for x in args:
result += x
return result
and use it like: sumFunction(3,4,6,3,6,8,9).
**kwargs has a diffrent function. With **kwargs you can give arbitrary keyword arguments to a function and you can access them as a dictonary.
def someFunction(**kwargs):
if 'text' in kwargs:
print kwargs['text']
Calling someFunction(text="foo") will print foo.
Just imagine you have a function but you don't want to restrict the number of parameter it takes.
Example:
>>> import operator
>>> def multiply(*args):
... return reduce(operator.mul, args)
Then you use this function like:
>>> multiply(1,2,3)
6
or
>>> numbers = [1,2,3]
>>> multiply(*numbers)
6
The names *args and **kwargs or **kw are purely by convention. It makes it easier for us to read each other's code
One place it is handy is when using the struct module
struct.unpack() returns a tuple whereas struct.pack() uses a variable number of arguments. When manipulating data it is convenient to be able to pass a tuple to struck.pack() eg.
tuple_of_data = struct.unpack(format_str, data)
# ... manipulate the data
new_data = struct.pack(format_str, *tuple_of_data)
without this ability you would be forced to write
new_data = struct.pack(format_str, tuple_of_data[0], tuple_of_data[1], tuple_of_data[2],...)
which also means the if the format_str changes and the size of the tuple changes, I'll have to go back and edit that really long line
Note that *args/**kwargs is part of function-calling syntax, and not really an operator. This has a particular side effect that I ran into, which is that you can't use *args expansion with the print statement, since print is not a function.
This seems reasonable:
def myprint(*args):
print *args
Unfortunately it doesn't compile (syntax error).
This compiles:
def myprint(*args):
print args
But prints the arguments as a tuple, which isn't what we want.
This is the solution I settled on:
def myprint(*args):
for arg in args:
print arg,
print
These parameters are typically used for proxy functions, so the proxy can pass any input parameter to the target function.
def foo(bar=2, baz=5):
print bar, baz
def proxy(x, *args, **kwargs): # reqire parameter x and accept any number of additional arguments
print x
foo(*args, **kwargs) # applies the "non-x" parameter to foo
proxy(23, 5, baz='foo') # calls foo with bar=5 and baz=foo
proxy(6)# calls foo with its default arguments
proxy(7, bar='asdas') # calls foo with bar='asdas' and leave baz default argument
But since these parameters hide the actual parameter names, it is better to avoid them.
You can have a look at python docs (docs.python.org in the FAQ), but more specifically for a good explanation the mysterious miss args and mister kwargs (courtesy of archive.org) (the original, dead link is here).
In a nutshell, both are used when optional parameters to a function or method are used.
As Dave says, *args is used when you don't know how many arguments may be passed, and **kwargs when you want to handle parameters specified by name and value as in:
myfunction(myarg=1)
This question already has answers here:
What does ** (double star/asterisk) and * (star/asterisk) do for parameters?
(25 answers)
Closed 9 years ago.
So I have difficulty with the concept of *args and **kwargs.
So far I have learned that:
*args = list of arguments - as positional arguments
**kwargs = dictionary - whose keys become separate keyword arguments and the values become values of these arguments.
I don't understand what programming task this would be helpful for.
Maybe:
I think to enter lists and dictionaries as arguments of a function AND at the same time as a wildcard, so I can pass ANY argument?
Is there a simple example to explain how *args and **kwargs are used?
Also the tutorial I found used just the "*" and a variable name.
Are *args and **kwargs just placeholders or do you use exactly *args and **kwargs in the code?
The syntax is the * and **. The names *args and **kwargs are only by convention but there's no hard requirement to use them.
You would use *args when you're not sure how many arguments might be passed to your function, i.e. it allows you pass an arbitrary number of arguments to your function. For example:
>>> def print_everything(*args):
for count, thing in enumerate(args):
... print( '{0}. {1}'.format(count, thing))
...
>>> print_everything('apple', 'banana', 'cabbage')
0. apple
1. banana
2. cabbage
Similarly, **kwargs allows you to handle named arguments that you have not defined in advance:
>>> def table_things(**kwargs):
... for name, value in kwargs.items():
... print( '{0} = {1}'.format(name, value))
...
>>> table_things(apple = 'fruit', cabbage = 'vegetable')
cabbage = vegetable
apple = fruit
You can use these along with named arguments too. The explicit arguments get values first and then everything else is passed to *args and **kwargs. The named arguments come first in the list. For example:
def table_things(titlestring, **kwargs)
You can also use both in the same function definition but *args must occur before **kwargs.
You can also use the * and ** syntax when calling a function. For example:
>>> def print_three_things(a, b, c):
... print( 'a = {0}, b = {1}, c = {2}'.format(a,b,c))
...
>>> mylist = ['aardvark', 'baboon', 'cat']
>>> print_three_things(*mylist)
a = aardvark, b = baboon, c = cat
As you can see in this case it takes the list (or tuple) of items and unpacks it. By this it matches them to the arguments in the function. Of course, you could have a * both in the function definition and in the function call.
One place where the use of *args and **kwargs is quite useful is for subclassing.
class Foo(object):
def __init__(self, value1, value2):
# do something with the values
print value1, value2
class MyFoo(Foo):
def __init__(self, *args, **kwargs):
# do something else, don't care about the args
print 'myfoo'
super(MyFoo, self).__init__(*args, **kwargs)
This way you can extend the behaviour of the Foo class, without having to know too much about Foo. This can be quite convenient if you are programming to an API which might change. MyFoo just passes all arguments to the Foo class.
Here's an example that uses 3 different types of parameters.
def func(required_arg, *args, **kwargs):
# required_arg is a positional-only parameter.
print required_arg
# args is a tuple of positional arguments,
# because the parameter name has * prepended.
if args: # If args is not empty.
print args
# kwargs is a dictionary of keyword arguments,
# because the parameter name has ** prepended.
if kwargs: # If kwargs is not empty.
print kwargs
>>> func()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: func() takes at least 1 argument (0 given)
>>> func("required argument")
required argument
>>> func("required argument", 1, 2, '3')
required argument
(1, 2, '3')
>>> func("required argument", 1, 2, '3', keyword1=4, keyword2="foo")
required argument
(1, 2, '3')
{'keyword2': 'foo', 'keyword1': 4}
Here's one of my favorite places to use the ** syntax as in Dave Webb's final example:
mynum = 1000
mystr = 'Hello World!'
print("{mystr} New-style formatting is {mynum}x more fun!".format(**locals()))
I'm not sure if it's terribly fast when compared to just using the names themselves, but it's a lot easier to type!
One case where *args and **kwargs are useful is when writing wrapper functions (such as decorators) that need to be able accept arbitrary arguments to pass through to the function being wrapped. For example, a simple decorator that prints the arguments and return value of the function being wrapped:
def mydecorator( f ):
#functools.wraps( f )
def wrapper( *args, **kwargs ):
print "Calling f", args, kwargs
v = f( *args, **kwargs )
print "f returned", v
return v
return wrapper
*args and **kwargs are special-magic features of Python.
Think of a function that could have an unknown number of arguments. For example, for whatever reasons, you want to have function that sums an unknown number of numbers (and you don't want to use the built-in sum function). So you write this function:
def sumFunction(*args):
result = 0
for x in args:
result += x
return result
and use it like: sumFunction(3,4,6,3,6,8,9).
**kwargs has a diffrent function. With **kwargs you can give arbitrary keyword arguments to a function and you can access them as a dictonary.
def someFunction(**kwargs):
if 'text' in kwargs:
print kwargs['text']
Calling someFunction(text="foo") will print foo.
Just imagine you have a function but you don't want to restrict the number of parameter it takes.
Example:
>>> import operator
>>> def multiply(*args):
... return reduce(operator.mul, args)
Then you use this function like:
>>> multiply(1,2,3)
6
or
>>> numbers = [1,2,3]
>>> multiply(*numbers)
6
The names *args and **kwargs or **kw are purely by convention. It makes it easier for us to read each other's code
One place it is handy is when using the struct module
struct.unpack() returns a tuple whereas struct.pack() uses a variable number of arguments. When manipulating data it is convenient to be able to pass a tuple to struck.pack() eg.
tuple_of_data = struct.unpack(format_str, data)
# ... manipulate the data
new_data = struct.pack(format_str, *tuple_of_data)
without this ability you would be forced to write
new_data = struct.pack(format_str, tuple_of_data[0], tuple_of_data[1], tuple_of_data[2],...)
which also means the if the format_str changes and the size of the tuple changes, I'll have to go back and edit that really long line
Note that *args/**kwargs is part of function-calling syntax, and not really an operator. This has a particular side effect that I ran into, which is that you can't use *args expansion with the print statement, since print is not a function.
This seems reasonable:
def myprint(*args):
print *args
Unfortunately it doesn't compile (syntax error).
This compiles:
def myprint(*args):
print args
But prints the arguments as a tuple, which isn't what we want.
This is the solution I settled on:
def myprint(*args):
for arg in args:
print arg,
print
These parameters are typically used for proxy functions, so the proxy can pass any input parameter to the target function.
def foo(bar=2, baz=5):
print bar, baz
def proxy(x, *args, **kwargs): # reqire parameter x and accept any number of additional arguments
print x
foo(*args, **kwargs) # applies the "non-x" parameter to foo
proxy(23, 5, baz='foo') # calls foo with bar=5 and baz=foo
proxy(6)# calls foo with its default arguments
proxy(7, bar='asdas') # calls foo with bar='asdas' and leave baz default argument
But since these parameters hide the actual parameter names, it is better to avoid them.
You can have a look at python docs (docs.python.org in the FAQ), but more specifically for a good explanation the mysterious miss args and mister kwargs (courtesy of archive.org) (the original, dead link is here).
In a nutshell, both are used when optional parameters to a function or method are used.
As Dave says, *args is used when you don't know how many arguments may be passed, and **kwargs when you want to handle parameters specified by name and value as in:
myfunction(myarg=1)
Why have a function as my_fun(an_arg, *arg) or even this a_func(dict=None, **args) why do people prefer to do this instead of saying just my_func(*args)? Are we not just repeating ourselves by using the former?
There's difference between my_fun(an_arg, *arg) and my_func(*args).
my_fun(an_arg, *arg)
Pass at least 1 argument or more arguments.
my_func(*args)
Pass any number of arguments, even 0.
Demo:
>>> def my_fun(an_arg, *arg):
pass
...
>>> def my_fun1(*arg):
pass
...
>>> my_fun()
...
TypeError: my_fun() missing 1 required positional argument: 'an_arg'
>>> my_fun1(1) #works fine
It's to help give your function a bit more meaning. Let's say that I'm trying to take in a function that increments a list of numbers by some parameter. Here's a silly example to illustrate:
def increase(increment, *nums):
return [num + increment for num in nums]
In this case, it's very clear what the first argument does, and what it's used for. In contrast, if we did this:
def increase(*args):
return [num + args[0] for num in args[1:]]
...then it's less clear what we're doing, and what all the arguments do.
In addition, it's also useful if we want to take in data, transform it, and pass in the rest of my arguments to another function.
Here's another contrived example:
def log(message, func, *args):
print message
func(*args)
Once again, if we just used only *args, our meaning is less clear:
def log(*args):
print args[0]
args[1](args[2:])
This would be much more error-prone, and hard to modify. It would also cause the function to fail if there weren't enough arguments -- by doing it the first way, you essentially make the first two elements mandatory and the rest optional.
They aren't equivalent forms. The other "repeat" forms bind arguments to discreet parameters and, more importantly, indicate that some parameters are required. Try to call def my_fun(an_arg, *arg): pass with 0 arguments.
While discreet (known) parameters work in many cases, *args allows for "sequence-variadic" functions and **kwargs allows "parameter-variadic" functions.
"{}, {}, {}".format(*(1,2,3,4,5))
Prints:
'1, 2, 3'
This works, as long as the number of {} in format does not exceed the length of a tuple. I want to make it work for a tuple of arbitrary length, padding it with -s if it is of insufficient length. And to avoid making assumptions about the number of {}'s, I wanted to use a generator. Here's what I had in mind:
def tup(*args):
for s in itertools.chain(args, itertools.repeat('-')):
yield s
print "{}, {}, {}".format(*tup(1,2))
Expected:
'1, 2, -'
But it never returns. Can you make it work with generators? Is there a better approach?
If you think about it, besides the fact that variable argument unpacking unpacks all at once, there's also the fact that format doesn't necessarily take its arguments in order, as in '{2} {1} {0}'.
You could work around this if format just took a sequence instead of requiring separate arguments, by building a sequence that does the right thing. Here's a trivial example:
class DefaultList(list):
def __getitem__(self, idx):
try:
return super(DefaultList, self).__getitem__(idx)
except IndexError:
return '-'
Of course your real-life version would wrap an arbitrary iterable, not subclass list, and would probably have to use tee or an internal cache and pull in new values as requested, only defaulting when you've passed the end. (You may want to search for "lazy list" or "lazy sequence" recipes at ActiveState, because there are a few of them that do this.) But this is enough to show the example.
Now, how does this help us? It doesn't; *lst on a DefaultList will just try to make a tuple out of the thing, giving us exactly the same number of arguments we already had. But what if you had a version of format that could just take a sequence of args instead? Then you could just pass your DefaultList and it would work.
And you do have that: Formatter.vformat.
>>> string.Formatter().vformat('{0} {1} {2}', DefaultList([0, 1]), {})
'0 1 -'
However, there's an even easier way, once you're using Formatter explicitly instead of implicitly via the str method. You can just override its get_value method and/or its check_unused_args:
class DefaultFormatter(string.Formatter):
def __init__(self, default):
self.default = default
# Allow excess arguments
def check_unused_args(self, used_args, args, kwargs):
pass
# Fill in missing arguments
def get_value(self, key, args, kwargs):
try:
return super(DefaultFormatter, self).get_value(key, args, kwargs)
except IndexError:
return '-'
f = DefaultFormatter('-')
print(f.vformat('{0} {2}', [0], {}))
print(f.vformat('{0} {2}', [0, 1, 2, 3], {}))
Of course you're still going to need to wrap your iterator in something that provides the Sequence protocol.
While we're at it, your problem could be solved more directly if the language had an "iterable unpacking" protocol. See here for a python-ideas thread proposing such a thing, and all of the problems the idea has. (Also note that the format function would make this trickier, because it would have to use the unpacking protocol directly instead of relying on the interpreter to do it magically. But, assuming it did so, then you'd just need to write a very simple and general-purpose wrapper around any iterable that handles __unpack__ for it.)
You cannot use endless generators to fill any *args arbitrary arguments call.
Python iterates over the generator to load all arguments to pass on to the callable, and if the generator is endless, that will never complete.
You can use non-endless generators without problems. You could use itertools.islice() to cap a generator:
from itertools import islice
print "{}, {}, {}".format(*islice(tup(1,2), 3))
After all, you already know how many slots your template has.
Martijn Pieters has the immediate answer, but if you wanted to create some sort of generic wrapper/helper for format autofilling, you could look at string.Formatter.parse. Using that, you can get a representation of how format sees the format string, and strip out the argument count/named argument names to dynamically figure out how long your iterator needs to be.
The naive approach would be to provide L/2 arguments to the format function where L is the length of the format string. Since a replacement token is at least 2 chars long, you are certain to always have enough values to unpack:
def tup(l, *args):
for s in args + (('-',) * l):
yield s
s = "{}, {}, {}"
print s.format(*list(tup(len(s)//2, 1, 2)))
As suggested by Silas Ray a more refined upper bound can be found using string.Formatter.parse
import string
def tup(l, *args):
for s in args + (('-',) * l):
yield s
s = "{}, {}, {}"
l = len(list(string.Formatter().parse(s)))
print s.format(*list(tup(l, 1, 2)))
Hi everyone this is probably something extremely simple that i'm overlooking but can someone point me in the right direction for how to handle this problem.
def nodeFunction(self,*args):
return self[1] + self[2]
Basically what I am trying to do is grab the data passed in through the arguments. I am just stuck on the syntax for referencing the arguments inside the function when using *args.
args is simply a tuple:
def nodeMethod(self, *args):
return args[0], args[1]
Is that what you mean?
Note that there's nothing special about "args". You could use any variable name. It's the * operator that counts.
>>> class Node(object):
... def nodeMethod(self, *cornucopia):
... return cornucopia[0], cornucopia[1]
...
>>> n = Node()
>>> n.nodeMethod(1, 2, 3)
(1, 2)
Still, "args" is the most idiomatic variable name; I wouldn't use anything else without a good reason that would be obvious to others.
def nodeFunction(self, arg1, arg2, *args)
*arg in argument list means: pass the remaning arguments as a list in variable arg. So check how to handle lists. Note: list index starts from 0.