We Keep Coding

Tuple metavar value for positional argument with nargs > 1

It seems that setting a tuple as a metavar for positional argument and requesting help does not work:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('size', type=int, nargs=2, help='size', metavar=('w', 'h'))
args = parser.parse_args()
print(args)
This produces an error when called as prog.py --help. The error differs between Python3 versions (I tried 3.5, 3.6, 3.8) and includes ValueError: too many values to unpack (expected 1) or TypeError: sequence item 0: expected str instance, tuple found. See live example on Wandbox.
For optional arguments, all is good:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--size', type=int, nargs=2, help='size', metavar=('w', 'h'))
args = parser.parse_args()
print(args)
Live example on Wandbox.
Is my code invalid, or did I find a bug in Python implementation?
Please note that simply parsing the arguments works as expected.

Last part of the traceback is
/usr/lib/python3.6/argparse.py in _format_action_invocation(self, action)
550 if not action.option_strings:
551 default = self._get_default_metavar_for_positional(action)
--> 552 metavar, = self._metavar_formatter(action, default)(1)
553 return metavar
554
So yes, it's specifically occurring with positionals (empty option_strings). The metavar, = ... assignment only works with the RHS returns one item. With your metavar it's returning a 2.
Usage displays ok
In [36]: parser.print_usage()
usage: ipython3 [-h] w h
It does look like a bug.
The (1) argument tells the function is should return a 1 element tuple:
metavar, = self._metavar_formatter(action, default)(1)
I suspect the issue has been raised already in Python bug/issues. I'll look for it later.
Instead of metavar, you could just as well use two positional arguments:
parser = argparse.ArgumentParser()
parser.add_argument('w', type=int)
parser.add_argument('h', type=int)
This has been a known bug for a long time - but so far no action:
https://bugs.python.org/issue14074
argparse allows nargs>1 for positional arguments but doesn't allow metavar to be a tuple

Following hpaulj's answer, here's another workaround using action='append':
for name in 'width', 'height':
parser.add_argument('size', type=int, help=name, metavar=name[0], action='append')
args = parser.parse_args(['4', '3'])
print(args)
parser.print_help()
Output:
Namespace(size=[4, 3])
usage: test.py [-h] w h
positional arguments:
w width
h height

Python argparse : How can I get Namespace objects for argument groups separately?

I have some command line arguments categorized in groups as follows:
cmdParser = argparse.ArgumentParser()
cmdParser.add_argument('mainArg')
groupOne = cmdParser.add_argument_group('group one')
groupOne.add_argument('-optA')
groupOne.add_argument('-optB')
groupTwo = cmdParser.add_argument_group('group two')
groupTwo.add_argument('-optC')
groupTwo.add_argument('-optD')
How can I parse the above, such that I end up with three different Namespace objects?
global_args - containing all the arguments not part of any group
groupOne_args - containing all the arguments in groupOne
groupTwo_args - containing all the arguments in groupTwo
Thank you!

you can do it in this way:
import argparse
parser = argparse.ArgumentParser()
group1 = parser.add_argument_group('group1')
group1.add_argument('--test1', help="test1")
group2 = parser.add_argument_group('group2')
group2.add_argument('--test2', help="test2")
args = parser.parse_args('--test1 one --test2 two'.split())
arg_groups={}
for group in parser._action_groups:
group_dict={a.dest:getattr(args,a.dest,None) for a in group._group_actions}
arg_groups[group.title]=argparse.Namespace(**group_dict)
This will give you the normal args, plus dictionary arg_groups containing namespaces for each of the added groups.
(Adapted from this answer)

Nothing in argparse is designed to do that.
For what it's worth, the parser starts off with two argument groups, one that displays as positionals and the other as optionals (I forget the exact titles). So in your example there will actually be 4 groups.
The parser only uses argument groups when formatting the help. For parsing all arguments are put in a master parser._actions list. And during parsing the parser only passes around one namespace object.
You could define separate parsers, with different sets of arguments, and call each with parse_known_args. That works better with optionals (flagged) arguments than with positionals. And it fragments your help.
I have explored in other SO questions a novel Namespace class that could nest values based on some sort of dotted dest (names like group1.optA, group2.optC, etc). I don't recall whether I had to customize the Action classes or not.
The basic point is that when saving a value to the namespace, a parser, or actually a Action (argument) object does:
setattr(namespace, dest, value)
That (and getattr/hasattr) is all that the parser expects of the namespace. The default Namespace class is simple, little more than a plain object subclass. But it could be more elaborate.

I was looking for a solution for this for a very long time,
And I think I finally got it.
So I will just put it here...
from argparse import ArgumentParser
def _parse_args():
parser = ArgumentParser()
parser.add_argument('-1', '--flag-1', action='store_true', default=False)
parser.add_argument('-2', '--flag-2', action='store_true', default=False)
parser.add_argument('-3', '--flag-3', action='store_true', default=False)
args, unknown = parser.parse_known_args()
print(f"args : {args}")
print(f"unknown : {unknown}")
hidden = ArgumentParser(add_help=False)
hidden.add_argument('-d', '--debug', action='store_true', default=False)
hidden_args = hidden.parse_args(unknown)
print(f"hidden_args : {hidden_args}")
if __name__ == "__main__":
_parse_args()
as a result:
show help:
ubuntu → playAround $ ./test.py -h
usage: test.py [-h] [-1] [-2] [-3]
optional arguments:
-h, --help show this help message and exit
-1, --flag-1
-2, --flag-2
-3, --flag-3
With debug flag:
ubuntu → playAround $ ./test.py -d
args : Namespace(flag_1=False, flag_2=False, flag_3=False)
unknown : ['-d']
hidden_args : Namespace(debug=True)
with flags 1 and 2:
ubuntu → playAround $ ./test.py -12
args : Namespace(flag_1=True, flag_2=True, flag_3=False)
unknown : []
hidden_args : Namespace(debug=False)
with flags 1 and 2 and debug:
ubuntu → playAround $ ./test.py -12 -d
args : Namespace(flag_1=True, flag_2=True, flag_3=False)
unknown : ['-d']
hidden_args : Namespace(debug=True)
The only thing you can't do with this approch is to pass the debug short flag along side to the other short flags:
ubuntu → playAround $ ./test.py -12d
usage: test.py [-h] [-1] [-2] [-3]
test.py: error: argument -2/--flag-2: ignored explicit argument 'd'

Here's a simple method that calls the parse_known_args() method after each group is defined to get the names for those arguments separately.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('mainArg')
global_names = set(vars(parser.parse_known_args()[0]).keys())
group1 = parser.add_argument_group('group 1')
group1.add_argument('-optA')
group1.add_argument('-optB')
group1_names = set(vars(parser.parse_known_args()[0]).keys()) - global_names
group2 = parser.add_argument_group('group 2')
group2.add_argument('-optC')
group2.add_argument('-optD')
group2_names = set(vars(parser.parse_known_args()[0]).keys()) - global_names - group1_names
args = parser.parse_args()
global_args = argparse.Namespace(**dict((k, v) for k, v in vars(args).items() if k in global_names))
group1_args = argparse.Namespace(**dict((k, v) for k, v in vars(args).items() if k in group1_names))
group2_args = argparse.Namespace(**dict((k, v) for k, v in vars(args).items() if k in group2_names))
print(global_args)
print(group1_args)
print(group2_args)
E.g. python args.py hi -optA fooA -optB fooB -optC fooC -optD fooD will output:
Namespace(mainArg='hi')
Namespace(optA='fooA', optB='fooB')
Namespace(optC='fooC', optD='fooD')

Argparse: how to handle variable number of arguments (nargs='*')

I thought that nargs='*' was enough to handle a variable number of arguments. Apparently it's not, and I don't understand the cause of this error.
The code:
p = argparse.ArgumentParser()
p.add_argument('pos')
p.add_argument('foo')
p.add_argument('--spam', default=24, type=int, dest='spam')
p.add_argument('vars', nargs='*')
p.parse_args('1 2 --spam 8 8 9'.split())
I think the resulting namespace should be Namespace(pos='1', foo='2', spam='8', vars=['8', '9']). Instead, argparse gives this error:
usage: prog.py [-h] [--spam SPAM] pos foo [vars [vars ...]]
error: unrecognized arguments: 9 8
Basically, argparse doesn't know where to put those additional arguments... Why is that?

For anyone who doesn't know what is nargs:
nargs stands for Number Of Arguments
3: 3 values, can be any number you want
?: a single value, which can be optional
*: a flexible number of values, which will be gathered into a list
+: like *, but requiring at least one value
argparse.REMAINDER: all the values that are remaining in the command
line
Example:
Python
import argparse
my_parser = argparse.ArgumentParser()
my_parser.add_argument('--input', action='store', type=int, nargs=3)
args = my_parser.parse_args()
print(args.input)
Console
$ python nargs_example.py --input 42
usage: nargs_example.py [-h] [--input INPUT INPUT INPUT]
nargs_example.py: error: argument --input: expected 3 arguments
$ python nargs_example.py --input 42 42 42
[42, 42, 42]
See more

The relevant Python bug is Issue 15112.
argparse: nargs='*' positional argument doesn't accept any items if preceded by an option and another positional
When argparse parses ['1', '2', '--spam', '8', '8', '9'] it first tries to match ['1','2'] with as many of the positional arguments as possible. With your arguments the pattern matching string is AAA*: 1 argument each for pos and foo, and zero arguments for vars (remember * means ZERO_OR_MORE).
['--spam','8'] are handled by your --spam argument. Since vars has already been set to [], there is nothing left to handle ['8','9'].
The programming change to argparse checks for the case where 0 argument strings is satisfying the pattern, but there are still optionals to be parsed. It then defers the handling of that * argument.
You might be able to get around this by first parsing the input with parse_known_args, and then handling the remainder with another call to parse_args.
To have complete freedom in interspersing optionals among positionals, in issue 14191, I propose using parse_known_args with just the optionals, followed by a parse_args that only knows about the positionals. The parse_intermixed_args function that I posted there could be implemented in an ArgumentParser subclass, without modifying the argparse.py code itself.
Here's a way of handling subparsers. I've taken the parse_known_intermixed_args function, simplified it for presentation sake, and then made it the parse_known_args function of a Parser subclass. I had to take an extra step to avoid recursion.
Finally I changed the _parser_class of the subparsers Action, so each subparser uses this alternative parse_known_args. An alternative would be to subclass _SubParsersAction, possibly modifying its __call__.
from argparse import ArgumentParser
def parse_known_intermixed_args(self, args=None, namespace=None):
# self - argparse parser
# simplified from http://bugs.python.org/file30204/test_intermixed.py
parsefn = super(SubParser, self).parse_known_args # avoid recursion
positionals = self._get_positional_actions()
for action in positionals:
# deactivate positionals
action.save_nargs = action.nargs
action.nargs = 0
namespace, remaining_args = parsefn(args, namespace)
for action in positionals:
# remove the empty positional values from namespace
if hasattr(namespace, action.dest):
delattr(namespace, action.dest)
for action in positionals:
action.nargs = action.save_nargs
# parse positionals
namespace, extras = parsefn(remaining_args, namespace)
return namespace, extras
class SubParser(ArgumentParser):
parse_known_args = parse_known_intermixed_args
parser = ArgumentParser()
parser.add_argument('foo')
sp = parser.add_subparsers(dest='cmd')
sp._parser_class = SubParser # use different parser class for subparsers
spp1 = sp.add_parser('cmd1')
spp1.add_argument('-x')
spp1.add_argument('bar')
spp1.add_argument('vars',nargs='*')
print parser.parse_args('foo cmd1 bar -x one 8 9'.split())
# Namespace(bar='bar', cmd='cmd1', foo='foo', vars=['8', '9'], x='one')

Simple solution: Specify the --spam flag before specifying pos and foo:
p = argparse.ArgumentParser()
p.add_argument('pos')
p.add_argument('foo')
p.add_argument('--spam', default=24, type=int, dest='spam')
p.add_argument('vars', nargs='*')
p.parse_args('--spam 8 1 2 8 9'.split())
The same works if you place the --spam flag after specifying your variable arguments.
p = argparse.ArgumentParser()
p.add_argument('pos')
p.add_argument('foo')
p.add_argument('--spam', default=24, type=int, dest='spam')
p.add_argument('vars', nargs='*')
p.parse_args('1 2 8 9 --spam 8'.split())
EDIT: For what it's worth, it seems that changing the * to a + will also fix the error.
p = argparse.ArgumentParser()
p.add_argument('pos')
p.add_argument('foo')
p.add_argument('--spam', default=24, type=int, dest='spam')
p.add_argument('vars', nargs='+')
p.parse_args('1 2 --spam 8 8 9'.split())

If you expect to have at least one optional argument then p.add_argument('vars', nargs='+') will work in your specific case

How to make an argument default if no argument passed?

I have a simple argparse script that takes two arguments; --encode and --decode. I want to make the --decode default if no argument is given. How can I do so?
I want this:
myscript.py --decode "some encoded string here"
to happen when I do:
myscript.py "some encoded string here"
by default.

Have a look at the 'store_true' action on the python documentation, or even the default keyword on the add argument method
You'll need to implement some logic, but here the idea:
parser.add_argument('--decode', rest_of_options..., default=True)
parser.add_argument('--encode', rest_of_options..., default=False)
values = parser.parse_args()
if values.decode:
do_some_stuff
elif values.encode:
do_some_other_stuff

In argparse.ArgumentParser() object when you use method: add_argument is a flag default="some value" for example:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--test', nargs='?', const=1, type=int)
args = parser.parse_args()
print(args)
output:
>python test.py
Namespace(test=None)
add the default flag:
parser.add_argument('--test', nargs='?', const=1, type=int, default=2)
after add_argument change:
>python test.py
Namespace(test=2)

You can use the default parameter to specify a default, along with dest to control the variable name of the option:
import argparse
p = argparse.ArgumentParser()
# --decode and --encode are usually considered mutually exclusive; this enforces that constraint
g = p.add_mutually_exclusive_group()
# Order matters: the first default for a given dest is used
g.add_argument('--decode', dest='action', action='store_const', const='decode', default='decode')
g.add_argument('--encode', dest='action', action='store_const', const='encode')

This approximates what you want, except for the missing '--'
p.add_argument('action',choices=['decode','encode'],default='decode',nargs='?')
p.add_argument('astring')
In [8]: p.parse_args(["a string"])
Out[8]: Namespace(action='decode', astring='a string')
In [9]: p.parse_args(['decode',"a string"])
Out[9]: Namespace(action='decode', astring='a string')
In [10]: p.parse_args(['encode',"a string"])
Out[10]: Namespace(action='encode', astring='a string')
If you must have the '--', nneonneo's solution is fine, producing the same namespaces. Both arguments write to the same destination attribute, and that attribute by default is 'decode'.
p.add_argument('--decode', dest='action', action='store_const', const='decode', default='decode')
p.add_argument('--encode', dest='action', action='store_const', const='encode')
If you don't use a mutually exclusive group, the last argument will have final say ('--decode --encode "a string to be encoded"')

Python,argparse: how to have nargs=2 with type=str and type=int

I spent some times on the argparse documentation, but I'm still struggling with this module for one option in my program:
parser.add_argument("-r", "--rmsd", dest="rmsd", nargs=2,
help="extract the poses that are close from a ref according RMSD",
metavar=("ref","rmsd"))
I'd like to the first argument to be a string (type str) and mandatory, while the second argument should have type int, and if no value is given have a default one (let's say default=50). I know how to do that when there is only one argument expected, but I have no idea how to proceed when nargs=2... Is that even possible?

You can do the following. The required keyword sets the field mandatory and the default=50 sets the default value of the option to 50 if not specified:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-s", "--string", type=str, required=True)
parser.add_argument("-i", "--integer", type=int, default=50)
args = parser.parse_args()
print args.string
print args.integer
Output:
$ python arg_parser.py -s test_string
test_string
50
$ python arg_parser.py -s test_string -i 100
test_string
100
$ python arg_parser.py -i 100
usage: arg_parser.py [-h] -s STRING [-i INTEGER]
arg_parser.py: error: argument -s/--string is required

I tend to agree with Mike's solution, but here's another way. It's not ideal, since the usage/help string tells the user to use 1 or more arguments.
import argparse
def string_integer(int_default):
"""Action for argparse that allows a mandatory and optional
argument, a string and integer, with a default for the integer.
This factory function returns an Action subclass that is
configured with the integer default.
"""
class StringInteger(argparse.Action):
"""Action to assign a string and optional integer"""
def __call__(self, parser, namespace, values, option_string=None):
message = ''
if len(values) not in [1, 2]:
message = 'argument "{}" requires 1 or 2 arguments'.format(
self.dest)
if len(values) == 2:
try:
values[1] = int(values[1])
except ValueError:
message = ('second argument to "{}" requires '
'an integer'.format(self.dest))
else:
values.append(int_default)
if message:
raise argparse.ArgumentError(self, message)
setattr(namespace, self.dest, values)
return StringInteger
And with that, you get:
>>> import argparse
>>> parser = argparse.ArgumentParser(description="")
parser.add_argument('-r', '--rmsd', dest='rmsd', nargs='+',
... action=string_integer(50),
... help="extract the poses that are close from a ref "
... "according RMSD")
>>> parser.parse_args('-r reference'.split())
Namespace(rmsd=['reference', 50])
>>> parser.parse_args('-r reference 30'.split())
Namespace(rmsd=['reference', 30])
>>> parser.parse_args('-r reference 30 3'.split())
usage: [-h] [-r RMSD [RMSD ...]]
: error: argument -r/--rmsd: argument "rmsd" requires 1 or 2 arguments
>>> parser.parse_args('-r reference 30.3'.split())
usage: [-h] [-r RMSD [RMSD ...]]
: error: argument -r/--rmsd: second argument to "rmsd" requires an integer

Sorry for jumping in way late. I'd use a function for type to call.
def two_args_str_int(x):
try:
return int(x)
except:
return x
parser.add_argument("-r", "--rmsd", dest="rmsd", nargs=2, type=two_args_str_int
help="extract the poses that are close from a ref according RMSD",
metavar=("ref","rmsd"))

I would recommend using two arguments:
import argparse
parser = argparse.ArgumentParser(description='Example with to arguments.')
parser.add_argument('-r', '--ref', dest='reference', required=True,
help='be helpful')
parser.add_argument('-m', '--rmsd', type=int, dest='reference_msd',
default=50, help='be helpful')
args = parser.parse_args()
print args.reference
print args.reference_msd

I had a similar problem, but "use two arguments" approach didn't work for me because I need a list of pairs: parser.add_argument('--replace', nargs=2, action='append') and if I use separate arguments then I would have to validate lengths of lists etc.
Here is what I did:
Use tuple for metavar to properly show help: tuple=('OLD', 'NEW') results in the help string being displayed as --replace OLD NEW. It is documented but I could not find it until tried different options.
Use custom validation: after parse_args, validate the resulting list's items and call parser.error() if something is wrong. That's because they have different data types.