When using an argument (optional and positional both have this problem) with the keyword choices, the generated help output shows those choices.
If that same argument also includes a metavar keyword, the list of choices is omitted from the generated output.
What I had in mind, was to show the metavar in the usage line, but actually show the available choices when the 'autohelp' lists positional/optional argument details.
Any simple fixes/workarounds?
I have already started an argparse wrapper for custom help functionality. Perhaps this should be another feature on my TODO list.
You can add the choices to the help text.
parser=argparse.ArgumentParser()
parser.add_argument('-f',metavar="TEST",choices=('a','b','c'),
help='choices, {%(choices)s}')
print parser.format_help()
produces:
usage: stack20328931.py [-h] [-f TEST]
optional arguments:
-h, --help show this help message and exit
-f TEST choices, {a, b, c}
Related
I'm using argparse and I have a custom argument group required arguments. Is there any way to change the order of the argument groups in the help message? I think it is more logical to have the required arguments before optional arguments, but haven't found any documentation or questions to help.
For example, changing this:
usage: foo.py [-h] -i INPUT [-o OUTPUT]
Foo
optional arguments:
-h, --help show this help message and exit
-o OUTPUT, --output OUTPUT
Output file name
required arguments:
-i INPUT, --input INPUT
Input file name
to this:
usage: foo.py [-h] -i INPUT [-o OUTPUT]
Foo
required arguments:
-i INPUT, --input INPUT
Input file name
optional arguments:
-h, --help show this help message and exit
-o OUTPUT, --output OUTPUT
Output file name
(example taken from this question)
You might consider adding an explicit optional arguments group:
import argparse
parser = argparse.ArgumentParser(description='Foo', add_help=False)
required = parser.add_argument_group('required arguments')
required.add_argument('-i', '--input', help='Input file name', required=True)
optional = parser.add_argument_group('optional arguments')
optional.add_argument("-h", "--help", action="help", help="show this help message and exit")
optional.add_argument('-o', '--output', help='Output file name', default='stdout')
parser.parse_args(['-h'])
You can move the help action to your optional group as
described here:
Move "help" to a different Argument Group in python argparse
As you can see, the code produces the required output:
usage: code.py -i INPUT [-h] [-o OUTPUT]
Foo
required arguments:
-i INPUT, --input INPUT
Input file name
optional arguments:
-h, --help show this help message and exit
-o OUTPUT, --output OUTPUT
Output file name
This is admittedly a hack, and is reliant on the changeable internal implementation, but after adding the arguments, you can simply do:
parser._action_groups.reverse()
This will effectively make the required arguments group display above the optional arguments group. Note that this answer is only meant to be descriptive, not prescriptive.
Credit: answer by hpaulj
The parser starts out with 2 argument groups, the usual positional and optionals. The -h help is added to optionals. When you do add_argument_group, a group is created (and returned to you). It is also appended to the parser._action_groups list.
When you ask for help (-h) parser.format_help() is called (you can do that as well in testing). Look for that method in argparse.py. That sets up the help message, and one step is:
# positionals, optionals and user-defined groups
for action_group in self._action_groups:
formatter.start_section(action_group.title)
formatter.add_text(action_group.description)
formatter.add_arguments(action_group._group_actions)
formatter.end_section()
So if we reorder the items in the parser._action_groups list, we will reorder the groups in the display. Since this is the only use of _action_groups it should be safe and easy. But some people aren't allowed to peak under the covers (look or change ._ attributes).
The proposed solution(s) is to make your own groups in the order you want to see them, and make sure that the default groups are empty (the add_help=False parameter). That's the only way to do this if you stick with the public API.
Demo:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('foo')
g1 = parser.add_argument_group('REQUIRED')
g1.add_argument('--bar', required=True)
g1.add_argument('baz', nargs=2)
print(parser._action_groups)
print([group.title for group in parser._action_groups])
print(parser.format_help())
parser._action_groups.reverse() # easy inplace change
parser.print_help()
Run result:
1504:~/mypy$ python stack39047075.py
_actions_group list and titles:
[<argparse._ArgumentGroup object at 0xb7247fac>,
<argparse._ArgumentGroup object at 0xb7247f6c>,
<argparse._ArgumentGroup object at 0xb721de0c>]
['positional arguments', 'optional arguments', 'REQUIRED']
default help:
usage: stack39047075.py [-h] --bar BAR foo baz baz
positional arguments:
foo
optional arguments:
-h, --help show this help message and exit
REQUIRED:
--bar BAR
baz
after reverse:
usage: stack39047075.py [-h] --bar BAR foo baz baz
REQUIRED:
--bar BAR
baz
optional arguments:
-h, --help show this help message and exit
positional arguments:
foo
1504:~/mypy$
Another way to implement this is to define a ArgumentParser subclass with a new format_help method. In that method reorder the list used in that for action_group... loop.
This program:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('files', metavar='INPUT', nargs='*',
help='File(s) containing words to include. If none given, stdin will be used.')
parser.add_argument('-x', '--exclude', nargs='*',
help='File(s) containing words to exclude.')
args = parser.parse_args()
print args.files
print args.exclude
produces this output when run in Python 2.7.9:
$ python prog.py --help
usage: prog.py [-h] [-x [EXCLUDE [EXCLUDE ...]]] [INPUT [INPUT ...]]
positional arguments:
INPUT File(s) containing words to include. If
none given, stdin will be used.
optional arguments:
-h, --help show this help message and exit
-x [EXCLUDE [EXCLUDE ...]], --exclude [EXCLUDE [EXCLUDE ...]]
File(s) containing words to exclude.
However, that "help" output instructs the user to use a nonsensical ordering for the arguments. It is nonsensical because if the -x option is used, then no INPUT arguments will be detected.
Argparse ought instead to advise the user to use this ordering:
usage: prog.py [-h] [INPUT [INPUT ...]] [-x [EXCLUDE [EXCLUDE ...]]]
Two questions:
Is this a bug in argparse? (I think it is.)
Regardless of whether it is a bug, how can I fix it so that $ python prog.py --help will output the help text I desire (see above), preferably in as DRY a way as possible?
When generating the usage line, flagged arguments are placed first, positional after. That fits with common commandline usage. It makes no effort to evaluate whether that is the best choice or not.
The simplest way around this is to provide a custom usage parameter.
(there have be SO questions about changing the order of arguments in the usage line. The solution requires customization of the HelpFormatter class. Change argparse usage message argument order
)
As you note, when the * positional is placed after the * optional, all arguments are assigned to the optional. You could use a '--' to separate the two lists of arguments.
There is a bug/issue with patch that should improve handling when the positional takes a known number of arguments (e.g. the default one). It does so by noting that the positional requires an argument, so it reserves one for it. But in the * * case, the positional is satisfied with none, so it has no way of knowing how to split the arguments between the 2.
I like the idea of turning that postional into a flagged argument. That should reduce the ambiguity inherent in lots of * arguments.
Add '-f', '--files' to the input option:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-f', '--files', metavar='INPUT', nargs='*', required=True,
help='File(s) containing words to include. If none given, stdin will be used.')
parser.add_argument('-x', '--exclude', nargs='*',
help='File(s) containing words to exclude.')
args = parser.parse_args()
print args.files
shows:
usage: argparse_test.py [-h] [-f [INPUT [INPUT ...]]]
[-x [EXCLUDE [EXCLUDE ...]]]
optional arguments:
-h, --help show this help message and exit
-f [INPUT [INPUT ...]], --files [INPUT [INPUT ...]]
File(s) containing words to include. If none given,
stdin will be used.
-x [EXCLUDE [EXCLUDE ...]], --exclude [EXCLUDE [EXCLUDE ...]]
File(s) containing words to exclude.
print args.exclude
You can make 'files' required. From the docs:
In general, the argparse module assumes that flags like -f and --bar indicate optional arguments, which can always be omitted at the command line. To make an option required, True can be specified for the required= keyword argument to add_argument():
The simplest way is to add usage="..." to argparse.ArgumentParser().
By viewing the source of argparse, I found a way to resort arguments which might be a little bit dirty:
class MyHelpFormatter(argparse.HelpFormatter):
def _format_actions_usage(self, actions, groups):
actions.sort(key=lambda a: bool(a.option_strings and a.nargs != 0))
return super(MyHelpFormatter, self)._format_actions_usage(actions, groups)
parser = argparse.ArgumentParser(formatter_class = MyHelpFormatter)
My script is accepting --full, --last and --check using ArgParse. If no option is provided, it just show the help message. But in that message, the parameters appear as optional.
usage: script.py [-h] [--full] [--last] [--check log_file]
If I use the keyword required, then the script will always expect the parameter, which is not correct.
usage: script.py [-h] --full --last --check log_file
So, how can I show something like:
usage: script.py [-h] (--full |--last |--check log_file)
Indicating that the help is optional but that at least one of those parameters is required.
On the question of customizing the usage:
The parser constructor takes a usage parameter. The immediate effect is to set an attribute:
parser = argparse.ArgumentParser( ... usage=custom_usage...)
print(parser.usage)
# should show None or the custom_usage string
Being a normal Python object attribute, you can change it after the parser was created.
usage_str = parser.format_usage()
The format_usage method ask the parser for create the usage that will be shown in the help (and error messages). If the parser.usage value is None, it formats it from the arguments. If a string, it is used as is (I think it fills in values like %(prog)s).
So you could write a usage string from scratch. Or you could set up the parser, get the current usage string, and edit that to suit your needs. Editing is most likely something you'd do during development, while testing the parser in an IDE. But it could be done on the fly.
A crude example:
In [441]: parser=argparse.ArgumentParser()
In [442]: g=parser.add_mutually_exclusive_group()
In [443]: g.add_argument('--foo')
In [444]: g.add_argument('--bar')
In [445]: ustr = parser.format_usage()
# 'usage: ipython3 [-h] [--foo FOO | --bar BAR]\n'
In [450]: parser.usage = ustr.replace('[','(').replace(']',')')
In [451]: parser.format_usage()
# 'usage: usage: ipython3 (-h) (--foo FOO | --bar BAR)\n'
I've replaced the [] with () (even on the -h :( ).
For now testing logical combinations of the args attributes is the best choice. Inside the parse_args functions the parser maintains a list (set actually) of arguments that it has seen. That is used to test for required arguments, and for mutually_exclusive_arguments, but it is not available outside that code.
For store_true (or false) arguments, just check their truth value. For others I like to test for the default None. If you use other default values, test accordingly. A nice thing about None is that the user cannot give you that value.
Perhaps the most general way to test for arguments is to count the number of attributes which are not None:
In [461]: args=argparse.Namespace(one=None, tow=2, three=None)
In [462]: ll = ['one','tow','three']
In [463]: sum([getattr(args,l,None) is not None for l in ll])
Out[463]: 1
0 means none are found; >0 at least one present; ==len(ll) all found; >1 violates mutually exclusivity; '==1' for required mutually exclusive.
As #doublep explained in his answer, if you want to use more than one option at a time:
Change the usage message manually to the one you want.
Add the following code from Python argparse: Make at least one argument required:
if not (args.full or args.last or args.check):
parse.error('[-] Error: DISPLAY_ERROR_MESSAGE')
You can use add_mutually_exclusive_group():
parser = argparse.ArgumentParser ()
group = parser.add_mutually_exclusive_group (required = True)
group.add_argument ('--foo')
group.add_argument ('--bar')
However, the main effect is that you won't be able to use more than one option at a time.
I have read a similar question asked on SO, which doesn't address my own problem. For illustration purpose, say I have a Python program using argpase that provides two subcommands: copy and resume:
prog copy src dest # src, dest positional, required
prog resume id # id positional, required
However, the most natural way to invoke the "copy" command is NOT explicitly give the copy subcommand, that is, I was hoping for:
prog src dest
will do the default copy action, while still keep the benefits of have two subparsers, each handles a different set of arguments. Is it possible with argparse package?
Formally there isn't. The subcommand argument is a required positional argument, where the 'choices' are the subparser names (and their aliases).
That's evident in the help message, were {cmd1,cmd} are shown as choices.
usage: ipython3 [-h] {cmd1,cmd2} ...
positional arguments:
{cmd1,cmd2}
optional arguments:
-h, --help show this help message and exit
The latest Python has a 'bug' that actually lets it be optional. In other words, it does not raise an error if you don't give any positionals. It's a bug because it changes previous behavior, and most people what it to be required. But even when it is optional, and cmd1 defined as the default, it won't run the cmd1 parser on the remaining arguments. And any 'positional' argument will be treated as a wrong command string.
I think the best you can do is define one or more positionals. One might have choices and default, and be optional (nargs='?'). The rest (or other) might has nargs='+'. By playing around with those options you could approximate the desired behavior, but it won't make use of the subparsers mechanism.
Another way to think about the issue, is to consider an input like
prog one two
Should it interpret that as prog src dest or prog cmd id. The only basis for saying it should be the former is the fact that one is not one of copy or resume. But it isn't that smart when it comes to handling 'choices'. It assigns a value based on position, and then tests whether it meets criteria like 'choices' or 'type' (ie. integer, float v string).
I am wrapping a class that exposes its OptionParser through a property named options_parser. I am wrapping this class in a 'runner' that I've written to use argparse. I use the ArgumentParser's parse_known_args() method to parse the wrapper's argument, and any of the remaining arguments I pass on to the instance of the wrapped class.
Running ./wrapper.py --help does not list the options from the wrapped class. Is there a convenient way to add the optparse options to the wrapper's argparse argument?
If it's solely for displaying the options, one way I can think of is use the format_help of optparse and put it in the epilog of argparse, for example:
In [303]: foo = OptionParser()
In [304]: foo.add_option("-f", "--file", dest="filename",help="read data from FILENAME")
In [305]: foo.add_option("-v", "--verbose",action="store_true", dest="verbose")
In [311]: bar = ArgumentParser(epilog = foo.format_help(), formatter_class = RawTextHelpFormatter)
In [312]: bar.add_argument('integers', metavar='N', type=int, nargs='+',help='an integer for the accumulator')
In [313]: bar.add_argument('--sum', dest='accumulate', action='store_const',const=sum, default=max,help='sum the integers (default: find the max)')
In [314]: bar.print_help()
usage: ipython [-h] [--sum] N [N ...]
positional arguments:
N an integer for the accumulator
optional arguments:
-h, --help show this help message and exit
--sum sum the integers (default: find the max)
Usage: ipython [options]
Options:
-h, --help show this help message and exit
-f FILENAME, --file=FILENAME
read data from FILENAME
-v, --verbose
You can of course format the epilog as you want, including some explanation about the two lists of options. You might also experiment with a different Formatter, though the default one doesn't work well in this case because it strips newlines from the epilog. If desperate about the layout you might also try to create your own formatter by subclassing argparse.HelpFormatter, though I'd not recommend this based on class docs:
"""Formatter for generating usage messages and argument help strings.
Only the name of this class is considered a public API. All the methods
provided by the class are considered an implementation detail.
"""