I am writing the script where I pass values with CLI through argparse module. I am wondering if this is possible to restrict variable to hold pre-defined values, to avoid user mistake. It is not the type restriction, values are consists of letters as well as digits, surely I can write an if block, but I have about 30 pre-defined values, so writing something like
if var is value1 or var is value2 ... or var is value30:
pass
else:
print("oops, your value does not fit")
would be painful. What is the proper way of doing this?
Instead of checking for equality for each item individually, check if it's in a set of valid items.
if var in {'foo', 'bar', 'etc.'}:
Also, don't use is check for string equality. Use ==. It is possible that a string in Python can contain the same contents as another, but not be the same object. The compiler should intern (i.e. reuse) strings from literals, but this is just an optimization. Strings generated at runtime should make new objects. There are ways to intern them manually though.
For a very long list of options like that, I'd probably split them from a string to make the set, like
options = set("""
spam
eggs
sausage
bacon
ham
""".split())
Then you can use var in options.
You can even union other hashable types into the set.
options |= {7, 42, False}
Although user input would start out as strings anyway.
Another option to consider is the re module. A regular expression can match a large set of related strings, sometimes very compactly. It depends on the kinds of options you want to allow.
With choices:
In [214]: parser = argparse.ArgumentParser()
In [215]: parser.add_argument('--foo', choices=['one','two','three','four']);
Accepted:
In [216]: parser.parse_args('--foo one'.split())
Out[216]: Namespace(foo='one')
rejected:
In [217]: parser.parse_args('--foo five'.split())
usage: ipython3 [-h] [--foo {one,two,three,four}]
ipython3: error: argument --foo: invalid choice: 'five' (choose from 'one', 'two', 'three', 'four')
help:
In [218]: parser.parse_args('-h'.split())
usage: ipython3 [-h] [--foo {one,two,three,four}]
optional arguments:
-h, --help show this help message and exit
--foo {one,two,three,four}
If I'd defined a metavar, the help will be
usage: ipython3 [-h] [--foo CHOICES]
optional arguments:
-h, --help show this help message and exit
--foo CHOICES
Or if the choices is too long, define a type function:
In [222]: def mychoices(astr):
...: if astr in ['one','two','three','four']:
...: return astr
...: else:
...: raise argparse.ArgumentTypeError('Wrong choice')
In [223]: parser = argparse.ArgumentParser()
In [224]: parser.add_argument('--foo', type=mychoices);
In [225]: parser.parse_args('--foo one'.split())
Out[225]: Namespace(foo='one')
In [226]: parser.parse_args('--foo five'.split())
usage: ipython3 [-h] [--foo FOO]
ipython3: error: argument --foo: Wrong choice
In [227]: parser.parse_args('-h'.split())
usage: ipython3 [-h] [--foo FOO]
optional arguments:
-h, --help show this help message and exit
--foo FOO
Related
I'm trying to learn how to use argparse. I'm writing a program with two positional arguments let's say ARG1 and ARG2 and I want to add some options based on the combination of those two positional arguments.
The interface would look like this:
D
C
B B
A A
^ ^
| |
app.py <ARG1> <ARG2> --opt1 --opt2 --opt3 --opt4
The first positional argument ARG1 would have 4 different options and the second positional argument ARG2 would have two different options.
Based on the combination of those two positional arguments I want to specify different options.
For example, when the user enters app.py A B I want only --opt1 --opt2 as required. Some combinations might not use some options at all.
I think subparsers is the way to go but I didn't manage to get it to work.
How can I do that?
Since argparse is designed to handle optionals in any order, it is awkward to force interactions. Subparsers give some control, but can only have one (unless you define subparsers of subparsers).
I think the simplest is to define 2 positionals with choices, and all the optionals, and do any "required" checking after parsing:
In [20]: parser = argparse.ArgumentParser()
In [21]: parser.add_argument('arg1', choices=['A','B','C','D']);
In [22]: parser.add_argument('arg2', choices=['A','B']);
In [23]: parser.add_argument('--opt1');
In [24]: parser.add_argument('--opt2');
In [26]: args = parser.parse_args('--opt2 foo C A'.split())
In [27]: print(args)
Namespace(arg1='C', arg2='A', opt1=None, opt2='foo')
A possible test:
In [29]: if args.arg1=='C' and args.arg2=='A' and args.opt1 is None:
...: parser.error('--opt1 is required with C and A')
...:
usage: ipython3 [-h] [--opt1 OPT1] [--opt2 OPT2] {A,B,C,D} {A,B}
ipython3: error: --opt1 is required with C and A
Keep in mind that it's hard to describe conditional interactions in the usage and help. An alternative to requiring an optional is to give it a reasonable default. That way it won't matter whether the user gives a value or not. But in the above test, I'm depending on the default default None.
The usage shows the optionals first, and in the commandline optionals can occur in an order, with respect to themselves and the positionals.
Not sure I fully understood your requirement but see if this example helps you:
from argparse import ArgumentParser
parser = ArgumentParser()
subparsers = parser.add_subparsers(help="sub-commands")
a_parser = subparsers.add_parser("ARG1")
a_parser.add_argument("-1", "--opt1", action="store_true", required=False)
a_parser.add_argument("-2", "--opt2", action="store_true", required=False)
a_parser.add_argument("-3", "--opt3", action="store_true", required=False)
a_parser.add_argument("-4", "--opt4", action="store_true", required=False)
subparsers2 = a_parser.add_subparsers(help="ARG1-ARG2 sub-commands")
ab_parser = subparsers2.add_parser("ARG2")
ab_parser.add_argument("-1", "--opt1", action="store_true", required=False)
ab_parser.add_argument("-2", "--opt2", action="store_true", required=False)
b_parser = subparsers.add_parser("ARG2")
b_parser.add_argument("-3", "--opt3", action="store_true", required=False)
b_parser.add_argument("-4", "--opt4", action="store_true", required=False)
argument = parser.parse_args()
Output:
± % python recovery_tools/test.py ARG1 --help
usage: test.py ARG1 [-h] [-1] [-2] [-3] [-4] {ARG2} ...
positional arguments:
{ARG2} ARG1-ARG2 sub-commands
optional arguments:
-h, --help show this help message and exit
-1, --opt1
-2, --opt2
-3, --opt3
-4, --opt4
± % python recovery_tools/test.py ARG1 ARG2 --help
usage: test.py ARG1 ARG2 [-h] [-1] [-2]
optional arguments:
-h, --help show this help message and exit
-1, --opt1
-2, --opt2
± % python recovery_tools/test.py ARG2 --help
usage: test.py ARG2 [-h] [-3] [-4]
optional arguments:
-h, --help show this help message and exit
-3, --opt3
-4, --opt4
(new-recovery-tools-env)
I want the user to be able to pass a mandatory argument to 'argparse', with either positional, or optional argument.
I.e.,
both following forms are valid:
my_prog arg
my_prog -m arg
I've seen Argparse optional positional arguments?
But the suggestions there make both form optional. I want that one of them will be mandatory.
Of course, I can add a manual checking after parsing that at least one of them has been set. But I got a hunch that there must be a better solution.
(And even with my manual approach, the 'help' section shows both of them as optional)
The mutually exclusive group mechanism can take a required parameter. It also works with one ? positional along with the optionals (flagged arguments). (more than one '?' positional doesn't make sense).
As for the help display there are 2 default groups, positonal and optional. So even if an optional (flagged) is set to required it is, by default, displayed in the optional group. The usage line is a better guide as to whether an argument is required or not. If you don't like the group labels in the help section, define your own argument groups.
In [146]: import argparse
In [147]: parser = argparse.ArgumentParser()
In [148]: gp = parser.add_mutually_exclusive_group(required=True)
In [149]: gp.add_argument('pos', nargs='?', default='foo');
In [150]: gp.add_argument('-f','--foo', default='bar');
In [151]: parser.parse_args('arg'.split())
Out[151]: Namespace(foo='bar', pos='arg')
In [152]: parser.parse_args('-f arg'.split())
Out[152]: Namespace(foo='arg', pos='foo')
In [153]: parser.parse_args('arg -f arg'.split())
usage: ipython3 [-h] [-f FOO] [pos]
ipython3: error: argument -f/--foo: not allowed with argument pos
In [154]: parser.parse_args(''.split())
usage: ipython3 [-h] [-f FOO] [pos]
ipython3: error: one of the arguments pos -f/--foo is required
In [155]: parser.parse_args('-h'.split())
usage: ipython3 [-h] [-f FOO] [pos]
positional arguments:
pos
optional arguments:
-h, --help show this help message and exit
-f FOO, --foo FOO
Oops, usage isn't showing the -f and pos in a mutually exlusive group. Sometimes that usage formatting is brittle.
Switching the order in which the arguments are defined gives a better usage
In [156]: parser = argparse.ArgumentParser()
In [157]: gp = parser.add_mutually_exclusive_group(required=True)
In [158]: gp.add_argument('-f','--foo', default='bar');
In [159]: gp.add_argument('pos', nargs='?', default='foo');
In [160]:
In [160]: parser.parse_args('-h'.split())
usage: ipython3 [-h] (-f FOO | pos)
positional arguments:
pos
optional arguments:
-h, --help show this help message and exit
-f FOO, --foo FOO
With a user defined argument group:
In [165]: parser = argparse.ArgumentParser()
In [166]: gp = parser.add_argument_group('Mutually exclusive')
In [167]: gpm = gp.add_mutually_exclusive_group(required=True)
In [168]: gpm.add_argument('-f','--foo', default='bar');
In [169]: gpm.add_argument('pos', nargs='?', default='foo');
In [170]:
In [170]: parser.parse_args('-h'.split())
usage: ipython3 [-h] (-f FOO | pos)
optional arguments:
-h, --help show this help message and exit
Mutually exclusive:
-f FOO, --foo FOO
pos
This is the one exception to the general rule argument_groups and mutually_exclusive_groups aren't designed for nesting.
The m-x-group was not required, usage would use []
usage: ipython3 [-h] [-f FOO | pos]
Is it possible to combine the functionality of argparse.ArgumentDefaultsHelpFormatter with argparse.MetavarTypeHelpFormatter?
There are examples on how to use any one of these in the docs, but it's not clear how one would create help text that combines the best of both?
For example, I would like something like the following:
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--foo', type=int, default=42, help='FOO!')
parser.add_argument('bar', nargs='*', default=[1, 2, 3], help='BAR!')
to produce something like the following:
usage: PROG [-h] [--foo int] [float [float ...]]
positional arguments:
float BAR! (default: [1, 2, 3])
optional arguments:
-h, --help show this help message and exit
--foo int FOO! (default: 42)
In theory you can define a new formatter class that is a subclass of both of the other classes. (I could find a bug/issue where that is suggested by the developer.) Whether that will work with this pair is needs to be tested or checked in the code. Basically we need to see whether the methods that these 2 classes change are compatible.
import argparse
class MyFormatter(argparse.ArgumentDefaultsHelpFormatter, argparse.MetavarTypeHelpFormatter):
pass
parser = argparse.ArgumentParser(prog='PROG', formatter_class=MyFormatter)
parser.add_argument('--foo', type=float, default=42, help='FOO!')
parser.add_argument('bar', nargs='*', default=[1, 2, 3], help='BAR!', type=int)
parser.print_help()
produces
2035:~/mypy$ python3.5 stack41195224.py
usage: PROG [-h] [--foo float] [int [int ...]]
positional arguments:
int BAR! (default: [1, 2, 3])
optional arguments:
-h, --help show this help message and exit
--foo float FOO! (default: 42)
Note that I had to specify a type for the positional as well. Without that I got an error:
AttributeError: 'NoneType' object has no attribute '__name__'
With this requirement, I suspect the MetavarTypeHelpFormatter does not get much use. The default 'string' type (None) will produce an error. This is a convenience formatter class, and likely to create more problems than it solves.
The defaults help formatter just adds a ' (default: %(default)s)' string to the help string. You can do that yourself. So it is just a convenience class, not something that anyone requires.
The alternative to subclassing these 2 classes, is to subclass their parent, and include all the methods that the 2 subclasses change.
adding:
parser.add_argument('foobar', type=str)
displays
1039:~/mypy$ python3 stack41195224_1.py
usage: PROG [-h] [--foo float] [int [int ...]] str
positional arguments:
int BAR! (default: [1, 2, 3])
str
optional arguments:
-h, --help show this help message and exit
--foo float FOO! (default: 42)
Using just
parser.add_argument('foobar')
produces
AttributeError: 'NoneType' object has no attribute '__name__'
I'm using argparse and I have a custom argument group required arguments. Is there any way to change the order of the argument groups in the help message? I think it is more logical to have the required arguments before optional arguments, but haven't found any documentation or questions to help.
For example, changing this:
usage: foo.py [-h] -i INPUT [-o OUTPUT]
Foo
optional arguments:
-h, --help show this help message and exit
-o OUTPUT, --output OUTPUT
Output file name
required arguments:
-i INPUT, --input INPUT
Input file name
to this:
usage: foo.py [-h] -i INPUT [-o OUTPUT]
Foo
required arguments:
-i INPUT, --input INPUT
Input file name
optional arguments:
-h, --help show this help message and exit
-o OUTPUT, --output OUTPUT
Output file name
(example taken from this question)
You might consider adding an explicit optional arguments group:
import argparse
parser = argparse.ArgumentParser(description='Foo', add_help=False)
required = parser.add_argument_group('required arguments')
required.add_argument('-i', '--input', help='Input file name', required=True)
optional = parser.add_argument_group('optional arguments')
optional.add_argument("-h", "--help", action="help", help="show this help message and exit")
optional.add_argument('-o', '--output', help='Output file name', default='stdout')
parser.parse_args(['-h'])
You can move the help action to your optional group as
described here:
Move "help" to a different Argument Group in python argparse
As you can see, the code produces the required output:
usage: code.py -i INPUT [-h] [-o OUTPUT]
Foo
required arguments:
-i INPUT, --input INPUT
Input file name
optional arguments:
-h, --help show this help message and exit
-o OUTPUT, --output OUTPUT
Output file name
This is admittedly a hack, and is reliant on the changeable internal implementation, but after adding the arguments, you can simply do:
parser._action_groups.reverse()
This will effectively make the required arguments group display above the optional arguments group. Note that this answer is only meant to be descriptive, not prescriptive.
Credit: answer by hpaulj
The parser starts out with 2 argument groups, the usual positional and optionals. The -h help is added to optionals. When you do add_argument_group, a group is created (and returned to you). It is also appended to the parser._action_groups list.
When you ask for help (-h) parser.format_help() is called (you can do that as well in testing). Look for that method in argparse.py. That sets up the help message, and one step is:
# positionals, optionals and user-defined groups
for action_group in self._action_groups:
formatter.start_section(action_group.title)
formatter.add_text(action_group.description)
formatter.add_arguments(action_group._group_actions)
formatter.end_section()
So if we reorder the items in the parser._action_groups list, we will reorder the groups in the display. Since this is the only use of _action_groups it should be safe and easy. But some people aren't allowed to peak under the covers (look or change ._ attributes).
The proposed solution(s) is to make your own groups in the order you want to see them, and make sure that the default groups are empty (the add_help=False parameter). That's the only way to do this if you stick with the public API.
Demo:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('foo')
g1 = parser.add_argument_group('REQUIRED')
g1.add_argument('--bar', required=True)
g1.add_argument('baz', nargs=2)
print(parser._action_groups)
print([group.title for group in parser._action_groups])
print(parser.format_help())
parser._action_groups.reverse() # easy inplace change
parser.print_help()
Run result:
1504:~/mypy$ python stack39047075.py
_actions_group list and titles:
[<argparse._ArgumentGroup object at 0xb7247fac>,
<argparse._ArgumentGroup object at 0xb7247f6c>,
<argparse._ArgumentGroup object at 0xb721de0c>]
['positional arguments', 'optional arguments', 'REQUIRED']
default help:
usage: stack39047075.py [-h] --bar BAR foo baz baz
positional arguments:
foo
optional arguments:
-h, --help show this help message and exit
REQUIRED:
--bar BAR
baz
after reverse:
usage: stack39047075.py [-h] --bar BAR foo baz baz
REQUIRED:
--bar BAR
baz
optional arguments:
-h, --help show this help message and exit
positional arguments:
foo
1504:~/mypy$
Another way to implement this is to define a ArgumentParser subclass with a new format_help method. In that method reorder the list used in that for action_group... loop.
i'm stuck on a task, which require either 2 positional args or 1 (a file) if flag enabled:
parser.add_argument('pos_arg1', help='desc')
parser.add_argument('pos_arg2', help='desc')
parser.add_argument('--add_arg1', help='desc', type=argparse.FileType('r'), nargs='?')
so using the script with either script.py arg1 arg2 or script.py --add_arg1 file are both valid. How would I do this?
If you define:
In [422]: parser=argparse.ArgumentParser()
In [423]: g=parser.add_mutually_exclusive_group()
In [424]: g.add_argument('--foo')
In [426]: g.add_argument('bar',nargs='*',default='test')
In [427]: parser.print_help()
usage: ipython2.7 [-h] [--foo FOO | bar [bar ...]]
positional arguments:
bar
optional arguments:
-h, --help show this help message and exit
--foo FOO
In [429]: parser.parse_args([])
Out[429]: Namespace(bar='test', foo=None)
In [430]: parser.parse_args(['one','two'])
Out[430]: Namespace(bar=['one', 'two'], foo=None)
In [431]: parser.parse_args(['--foo','two'])
Out[431]: Namespace(bar='test', foo='two')
With this you can specify two (really any number) of unlabeled values, or one value flagged with --foo. It will object if I try both. I could have marked the group as required.
A couple of notes:
Marking --foo as nargs='?' is relatively meaningless unless I specify both a default and const.
I can only specify one one positional in the exclusive group, and that argument has to have '?' or '*', and a 'default'. In other words, it has to genuinely optional.
Without the mutually_exclusive_group I could make both positionals ?, but I can't say 'zero or two arguments'.