There is probably an obvious answer to this question, but I've looked at it for a bit without figuring it out. This is some old Python code using argparse. I haven't used argparse recently, so I may have forgotten some nuance.
#test.py
def load_crossval_dataset(args):
schema, samplenum, permuted, search = args.schema, args.samplenum, args.permuted, args.search
print "schema", schema
print "samplenum", samplenum
print "permuted", permuted
print "search", search
import argparse
parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
subparsers = parser.add_subparsers()
# create the parser for the "crossval" command
parser_crossval = subparsers.add_parser('crossval', formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser_crossval.add_argument('schema', help='name of schema')
parser_crossval.add_argument("-n", "--samplenum", action="store", type=int, dest="samplenum", help="number of samples to do crossvalidation on")
parser_crossval.add_argument("-p", "--permuted", action="store_true", dest="permuted", help="permuted dataset", default=False)
parser_crossval.add_argument("-s", "--search", action="store_true", dest="search", help="model search", default=False)
parser_crossval.set_defaults(func=load_crossval_dataset)
args = parser.parse_args()
args.func(args)
Let us invoke this as:
python test.py
usage: test.py [-h] {crossval} ...
test.py: error: too few arguments
Now as
python test.py crossval -h
usage: test.py crossval [-h] [-n SAMPLENUM] [-p] [-s] schema
positional arguments:
schema name of schema
optional arguments:
-h, --help show this help message and exit
-n SAMPLENUM, --samplenum SAMPLENUM
number of samples to do crossvalidation on (default: None)
-p, --permuted permuted dataset (default: False)
-s, --search model search (default: False)
Now as
python test.py crossval -n 1 -s True
schema True
samplenum 1
permuted False
search True
Question: why does argparse not complain about the missing schema argument, and why does it set it to True?
At a glance, the -s option is boolean - so its presence implies True and it requires no argument. So when you say python test.py crossval -n 1 -s True, the True gets parsed as being the schema argument since the -s switch doesn't require a value.
This much can in fact be gleaned from the usage string in the help text:
usage: test.py crossval [-h] [-n SAMPLENUM] [-p] [-s] schema
The [-s] indicates that it's a nullary option, unlike -n which is listed as [-n SAMPLENUM] since it requires an argument (SAMPLENUM).
Edit:
This behavior is stated in the Python 2.7 Documentation for argparse, which I infer is the version you are using in your example since you are using the statement- rather than function-form of print. To quote section 15.4.3.2:
'store_true' and 'store_false' - These are special cases of 'store_const' using for storing the values True and False respectively. In addition, they create default values of False and True respectively.
The option -s doesn't take an argument (store_const, store_true and store_false actions don't take an argument — this could stand to be clarified in the documentation). So in python test.py crossval -n 1 -s True, the argument True is a positional argument of crossval, not an argument of -s; thus it's the value of schema.
python test.py crossval -n 1 -s correctly complains about the missing argument to test.py crossval.
Related
I'm trying to use argparse in a Python 3 application where there's an explicit list of choices, but a default if none are specified.
The code I have is:
parser.add_argument('--list', default='all', choices=['servers', 'storage', 'all'], help='list servers, storage, or both (default: %(default)s)')
args = parser.parse_args()
print(vars(args))
However, when I run this I get the following with an option:
$ python3 ./myapp.py --list all
{'list': 'all'}
Or without an option:
$ python3 ./myapp.py --list
usage: myapp.py [-h] [--list {servers,storage,all}]
myapp.py: error: argument --list: expected one argument
Am I missing something here? Or can I not have a default with choices specified?
Pass the nargs and const arguments to add_argument:
parser.add_argument('--list',
default='all',
const='all',
nargs='?',
choices=['servers', 'storage', 'all'],
help='list servers, storage, or both (default: %(default)s)')
If you want to know if --list was passed without an argument, remove the const argument, and check if args.list is None.
Documention:
nargs with '?'
One argument will be consumed from the command line if possible, and produced as a single item. If no command-line argument is present, the value from default will be produced. Note that for optional arguments, there is an additional case - the option string is present but not followed by a command-line argument. In this case the value from const will be produced.
const
When add_argument() is called with option strings (like -f or --foo) and nargs='?'. This creates an optional argument that can be followed by zero or one command-line arguments. When parsing the command line, if the option string is encountered with no command-line argument following it, the value of const will be assumed instead. See the nargs description for examples.
Thanks #ShadowRanger. Subcommands is exactly what I need, combined with nargs and const. The following works:
parser = argparse.ArgumentParser()
subparser = parser.add_subparsers()
parser_list = subparser.add_parser('list')
parser_list.add_argument('list_type', default='all', const='all', nargs='?', choices=['all', 'servers', 'storage'])
parser_create = subparser.add_parser('create')
parser_create.add_argument('create_type', default='server', const='server', nargs='?', choices=['server', 'storage'])
args = parser.parse_args()
pprint(vars(args))
$ python3 ./myapp.py -h
usage: dotool.py [-h] {list,create} ...
Digital Ocean tool
positional arguments:
{list,create}
optional arguments:
-h, --help show this help message and exit
list option alone:
$ python3 ./myapp.py list
{'list_type': 'all'}
List option with a parameter:
$ python3 ./myapp.py list servers
{'list_type': 'servers'}
My Program should include the following options, properly parsed by argparse:
purely optional: [-h, --help] and [-v, --version]
mutually exclusive: [-f FILE, --file FILE] and [-u URL, --url URL]
optional if --url was chosen: [-V, --verbose]
required if either --file or --url was chosen: [-F, --format FORMAT]
The desired usage pattern would be:
prog.py [-h] [-v] [-f FILE (-F FORMAT) | -u URL [-V] (-F FORMAT) ]
with the -F requirement applying to both members of the mutually exclusive group.
Not sure if it rather be a positional.
So it should be possible to run:
prog.py -u "http://foo.bar" -V -F csv
and the parser screaming in case i forgot the -F (as he's supposed to).
What i've done so far:
parser = ArgumentParser(decription='foo')
group = parser.add_mutually_exclusive_group()
group.add_argument('-f','--file', nargs=1, type=str, help='')
group.add_argument('-u','--url', nargs=1, type=str, help='')
parser.add_argument('-V','--verbose', action='store_true', default=False, help='')
parser.add_argument('-F','--format', nargs=1, type=str, help='')
Since it has a 'vanilla mode' to run without command line arguments, all arguments must be optional.
How can i implement points 3. and 4. into my code?
EDIT:
I tried -f and -u as subparsers, as described here, but subcommands seem to be treated like positionals and the parser gives me an error: too few arguments if i run it without arguments.
Use of nargs=2 and tuple metavar approximates your goal
parser = argparse.ArgumentParser(prog='PROG')
group = parser.add_mutually_exclusive_group()
group.add_argument('-f','--file', nargs=2, metavar=('FILE','FORMAT'))
group.add_argument('-u','--url', nargs=2, metavar=('URL','FORMAT'))
parser.add_argument('-V','--verbose', action='store_true',help='optional with url')
which produces:
usage: PROG [-h] [-f FILE FORMAT | -u URL FORMAT] [-V]
optional arguments:
-h, --help show this help message and exit
-f FILE FORMAT, --file FILE FORMAT
-u URL FORMAT, --url URL FORMAT
-V, --verbose optional with url
This requires the format along with filename or url, it just doesn't require the -F. As others noted -V can be ignored in the -f case.
I tried -f and -u as subparsers, as described here, but subcommands seem to be treated like positionals and the parser gives me an error: too few arguments if i run it without arguments.
In the latest version(s) subcommands are no longer treated as required positionals. This was, as best I can tell, a side effect of changing the error message to be more informative. Instead of _parse_known_args doing a:
if positionals:
self.error(_('too few arguments'))
it scans _actions to see which are required, and then lists them by name in the error message. This is discussed in http://bugs.python.org/issue9253 . I know this change is in development (3.4), and may also be in 3.3.
These points can enforced in optparse using a callback method when a certain option is present.
However, in argparse these are not available.
You can add a subparser for the url and the file sub-option, and parse these seperatly.
from the help:
Note that the object returned by parse_args() will only contain attributes for
the main parser and the subparser that was selected by the command line
(and not any other subparsers). So in the example above, when the a command
is specified, only the foo and bar attributes are present, and when the b command
is specified, only the foo and baz attributes are present.
But I would just properly document the usage, and just ignore the arguments that are not
applicable.
e.g. let these two command lines behave exactly the same:
prog.py -f FILE -V
prog.py -f FILE
I would like to use argparse to make some code to be used in the following two ways:
./tester.py all
./tester.py name someprocess
i.e. either all is specified OR name with some additional string.
I have tried to implement as follows:
import argparse
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument('all', action='store_true', \
help = "Stops all processes")
group.add_argument('name', \
help = "Stops the named process")
print parser.parse_args()
which gives me an error
ValueError: mutually exclusive arguments must be optional
Any idea how to do it right? I also would like to avoid sub parsers in this case.
The question is a year old, but since all the answers suggest a different syntax, I'll give something closer to the OP.
First, the problems with the OP code:
A positional store_true does not make sense (even if it is allowed). It requires no arguments, so it is always True. Giving an 'all' will produce error: unrecognized arguments: all.
The other argument takes one value and assigns it to the name attribute. It does not accept an additional process value.
Regarding the mutually_exclusive_group. That error message is raised even before parse_args. For such a group to make sense, all the alternatives have to be optional. That means either having a -- flag, or be a postional with nargs equal to ? or *. And doesn't make sense to have more than one such positional in the group.
The simplest alternative to using --all and --name, would be something like this:
p=argparse.ArgumentParser()
p.add_argument('mode', choices=['all','name'])
p.add_argument('process',nargs='?')
def foo(args):
if args.mode == 'all' and args.process:
pass # can ignore the process value or raise a error
if args.mode == 'name' and args.process is None:
p.error('name mode requires a process')
args = p.parse_args()
foo(args) # now test the namespace for correct `process` argument.
Accepted namespaces would look like:
Namespace(mode='name', process='process1')
Namespace(mode='all', process=None)
choices imitates the behavior of a subparsers argument. Doing your own tests after parse_args is often simpler than making argparse do something special.
import argparse
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-a','--all', action='store_true', \
help = "Stops all processes")
group.add_argument('-n','--name', \
help = "Stops the named process")
print parser.parse_args()
./tester.py -h
usage: zx.py [-h] (-a | -n NAME)
optional arguments:
-h, --help show this help message and exit
-a, --all Stops all processes
-n NAME, --name NAME Stops the named process
"OR name with some additional string."
positional argument cannot take additional string
I think the best solution for you is (named test.py):
import argparse
p = argparse.ArgumentParser()
meg = p.add_mutually_exclusive_group()
meg.add_argument('-a', '--all', action='store_true', default=None)
meg.add_argument('-n', '--name', nargs='+')
print p.parse_args([])
print p.parse_args(['-a'])
print p.parse_args('--name process'.split())
print p.parse_args('--name process1 process2'.split())
print p.parse_args('--all --name process1'.split())
$ python test.py
Namespace(all=None, name=None)
Namespace(all=True, name=None)
Namespace(all=None, name=['process'])
Namespace(all=None, name=['process1', 'process2'])
usage: t2.py [-h] [-a | -n NAME [NAME ...]]
t2.py: error: argument -n/--name: not allowed with argument -a/--all
I would agree that this looks exactly like a sub-parser problem, and that if you don't want to make it an optional argument by using --all and --name, one suggestion from me would be just to ignore the all and name altogether, and use the following semantics:
If tester.py is called without any arguments, stop all process.
If tester.py is called with some arguments, stop only those processes.
Which can be done using:
import argparse, sys
parser = argparse.ArgumentParser()
parser.add_argument('processes', nargs='*')
parsed = parser.parse(sys.argv[1:])
print parsed
which will behave as follows:
$ python tester.py
Namespace(processes=[])
$ python tester.py proc1
Namespace(processes=['proc1'])
Or, if you insist on your own syntax, you can create a custom class. And actually you're not having a "mutually exclusive group" case, since I assume if all is specified, you will ignore the rest of the arguments (even when name is one of the other arguments), and when name is specified, anything else after that will be regarded as processes' name.
import argparse
import sys
class AllOrName(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
if len(values)==0:
raise argparse.ArgumentError(self, 'too few arguments')
if values[0]=='all':
setattr(namespace, 'all', True)
elif values[0]=='name':
if len(values)==1:
raise argparse.ArgumentError(self, 'please specify at least one process name')
setattr(namespace, 'name', values[1:])
else:
raise argparse.ArgumentError(self, 'only "all" or "name" should be specified')
parser = argparse.ArgumentParser()
parser.add_argument('processes', nargs='*', action=AllOrName)
parsed = parser.parse_args(sys.argv[1:])
print parsed
with the following behaviour:
$ python argparse_test.py name
usage: argparse_test.py [-h] [processes [processes ...]]
argparse_test.py: error: argument processes: please specify at least one process name
$ python argparse_test.py name proc1
Namespace(name=['proc1'], processes=None)
$ python argparse_test.py all
Namespace(all=True, processes=None)
$ python argparse_test.py host
usage: argparse_test.py [-h] [processes [processes ...]]
argparse_test.py: error: argument processes: only "all" or "name" should be specified
$ python argparse_test.py
usage: argparse_test.py [-h] [processes [processes ...]]
argparse_test.py: error: argument processes: too few arguments
This is probably what you're looking for:
group.add_argument('--all', dest=is_all, action='store_true')
group.add_argument('--name', dest=names, nargs='+')
Passing --name will then require at list one value and store them as a list.
I need some help regarding using argparse. What I want to achieve is that I need to pass in only one argument, it could be one of the followings: --k, --r, --b, --p,(ignore the rest). If the argument count is not 1, print "usage" information and quit. Also the program needs to know which flag is passed in in order to create corresponding object. I tried several times but I doesn't work, can anyone give me a hint on this? Thanks.
What you need to use to accomplish that is a mutually exclusive group:
import argparse
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument('-k', action='store_true')
group.add_argument('-r', action='store_true')
group.add_argument('-b', action='store_true')
group.add_argument('-p', action='store_true')
parser.parse_args()
As it can be seen in the example below, only one option in a mutually exclusive group is allowed at the same time:
$ python test.py -k -r -b -p
usage: test.py [-h] [-k | -r | -b | -p]
test.py: error: argument -r: not allowed with argument -k
To check which flag was passed, you just need to look at the argparse.Namespace object returned by parse_args method (the flag passed will be set to True).
How about not using argparse at all? It doesn't seem really necessary.
if len(sys.argv) != 2:
print_usage()
arg = sys.argv[1]
if arg not in ["--k", "--r", "--b", "--p"]:
print_usage()
# Do whatever you want with arg
I've been using argparse for a Python program that can -process, -upload or both:
parser = argparse.ArgumentParser(description='Log archiver arguments.')
parser.add_argument('-process', action='store_true')
parser.add_argument('-upload', action='store_true')
args = parser.parse_args()
The program is meaningless without at least one parameter. How can I configure argparse to force at least one parameter to be chosen?
UPDATE:
Following the comments: What's the Pythonic way to parametrize a program with at least one option?
if not (args.process or args.upload):
parser.error('No action requested, add -process or -upload')
args = vars(parser.parse_args())
if not any(args.values()):
parser.error('No arguments provided.')
I know this is old as dirt, but the way to require one option but forbid more than one (XOR) is like this:
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-process', action='store_true')
group.add_argument('-upload', action='store_true')
args = parser.parse_args()
print args
Output:
>opt.py
usage: multiplot.py [-h] (-process | -upload)
multiplot.py: error: one of the arguments -process -upload is required
>opt.py -upload
Namespace(process=False, upload=True)
>opt.py -process
Namespace(process=True, upload=False)
>opt.py -upload -process
usage: multiplot.py [-h] (-process | -upload)
multiplot.py: error: argument -process: not allowed with argument -upload
If not the 'or both' part (I have initially missed this) you could use something like this:
parser = argparse.ArgumentParser(description='Log archiver arguments.')
parser.add_argument('--process', action='store_const', const='process', dest='mode')
parser.add_argument('--upload', action='store_const', const='upload', dest='mode')
args = parser.parse_args()
if not args.mode:
parser.error("One of --process or --upload must be given")
Though, probably it would be a better idea to use subcommands instead.
Requirements Review
use argparse (I will ignore this one)
allow one or two actions to be called (at least one required).
try to by Pythonic (I would rather call it "POSIX"-like)
There are also some implicit requirements when living on command line:
explain the usage to the user in a way which is easy to understand
options shall be optional
allow specifying flags and options
allow combining with other parameters (like file name or names).
Sample solution using docopt (file managelog.py):
"""Manage logfiles
Usage:
managelog.py [options] process -- <logfile>...
managelog.py [options] upload -- <logfile>...
managelog.py [options] process upload -- <logfile>...
managelog.py -h
Options:
-V, --verbose Be verbose
-U, --user <user> Username
-P, --pswd <pswd> Password
Manage log file by processing and/or uploading it.
If upload requires authentication, you shall specify <user> and <password>
"""
if __name__ == "__main__":
from docopt import docopt
args = docopt(__doc__)
print args
Try to run it:
$ python managelog.py
Usage:
managelog.py [options] process -- <logfile>...
managelog.py [options] upload -- <logfile>...
managelog.py [options] process upload -- <logfile>...
managelog.py -h
Show the help:
$ python managelog.py -h
Manage logfiles
Usage:
managelog.py [options] process -- <logfile>...
managelog.py [options] upload -- <logfile>...
managelog.py [options] process upload -- <logfile>...
managelog.py -h
Options:
-V, --verbose Be verbose
-U, --user <user> Username
-P, --pswd <pswd> P managelog.py [options] upload -- <logfile>...
Manage log file by processing and/or uploading it.
If upload requires authentication, you shall specify <user> and <password>
And use it:
$ python managelog.py -V -U user -P secret upload -- alfa.log beta.log
{'--': True,
'--pswd': 'secret',
'--user': 'user',
'--verbose': True,
'-h': False,
'<logfile>': ['alfa.log', 'beta.log'],
'process': False,
'upload': True}
Short alternative short.py
There can be even shorter variant:
"""Manage logfiles
Usage:
short.py [options] (process|upload)... -- <logfile>...
short.py -h
Options:
-V, --verbose Be verbose
-U, --user <user> Username
-P, --pswd <pswd> Password
Manage log file by processing and/or uploading it.
If upload requires authentication, you shall specify <user> and <password>
"""
if __name__ == "__main__":
from docopt import docopt
args = docopt(__doc__)
print args
Usage looks like this:
$ python short.py -V process upload -- alfa.log beta.log
{'--': True,
'--pswd': None,
'--user': None,
'--verbose': True,
'-h': False,
'<logfile>': ['alfa.log', 'beta.log'],
'process': 1,
'upload': 1}
Note, that instead of boolean values for "process" and "upload" keys there are counters.
It turns out, we cannot prevent duplication of these words:
$ python short.py -V process process upload -- alfa.log beta.log
{'--': True,
'--pswd': None,
'--user': None,
'--verbose': True,
'-h': False,
'<logfile>': ['alfa.log', 'beta.log'],
'process': 2,
'upload': 1}
Conclusions
Designing good command line interface can be challenging sometime.
There are multiple aspects of command line based program:
good design of command line
selecting/using proper parser
argparse offers a lot, but restricts possible scenarios and can become very complex.
With docopt things go much shorter while preserving readability and offering high degree of flexibility. If you manage getting parsed arguments from dictionary and do some of conversions (to integer, opening files..) manually (or by other library called schema), you may find docopt good fit for command line parsing.
For http://bugs.python.org/issue11588 I am exploring ways of generalizing the mutually_exclusive_group concept to handle cases like this.
With this development argparse.py, https://github.com/hpaulj/argparse_issues/blob/nested/argparse.py
I am able to write:
parser = argparse.ArgumentParser(prog='PROG',
description='Log archiver arguments.')
group = parser.add_usage_group(kind='any', required=True,
title='possible actions (at least one is required)')
group.add_argument('-p', '--process', action='store_true')
group.add_argument('-u', '--upload', action='store_true')
args = parser.parse_args()
print(args)
which produces the following help:
usage: PROG [-h] (-p | -u)
Log archiver arguments.
optional arguments:
-h, --help show this help message and exit
possible actions (at least one is required):
-p, --process
-u, --upload
This accepts inputs like '-u', '-up', '--proc --up' etc.
It ends up running a test similar to https://stackoverflow.com/a/6723066/901925, though the error message needs to be clearer:
usage: PROG [-h] (-p | -u)
PROG: error: some of the arguments process upload is required
I wonder:
are the parameters kind='any', required=True clear enough (accept any of the group; at least one is required)?
is usage (-p | -u) clear? A required mutually_exclusive_group produces the same thing. Is there some alternative notation?
is using a group like this more intuitive than phihag's simple test?
The best way to do this is by using python inbuilt module add_mutually_exclusive_group.
parser = argparse.ArgumentParser(description='Log archiver arguments.')
group = parser.add_mutually_exclusive_group()
group.add_argument('-process', action='store_true')
group.add_argument('-upload', action='store_true')
args = parser.parse_args()
If you want only one argument to be selected by command line just use required=True as an argument for group
group = parser.add_mutually_exclusive_group(required=True)
If you require a python program to run with at least one parameter, add an argument that doesn't have the option prefix (- or -- by default) and set nargs=+ (Minimum of one argument required). The problem with this method I found is that if you do not specify the argument, argparse will generate a "too few arguments" error and not print out the help menu. If you don't need that functionality, here's how to do it in code:
import argparse
parser = argparse.ArgumentParser(description='Your program description')
parser.add_argument('command', nargs="+", help='describe what a command is')
args = parser.parse_args()
I think that when you add an argument with the option prefixes, nargs governs the entire argument parser and not just the option. (What I mean is, if you have an --option flag with nargs="+", then --option flag expects at least one argument. If you have option with nargs="+", it expects at least one argument overall.)
This achieves the purpose and this will also be relfected in the argparse autogenerated --help output, which is imho what most sane programmers want (also works with optional arguments):
parser.add_argument(
'commands',
nargs='+', # require at least 1
choices=['process', 'upload'], # restrict the choice
help='commands to execute'
)
Official docs on this:
https://docs.python.org/3/library/argparse.html#choices
Maybe use sub-parsers?
import argparse
parser = argparse.ArgumentParser(description='Log archiver arguments.')
subparsers = parser.add_subparsers(dest='subparser_name', help='sub-command help')
parser_process = subparsers.add_parser('process', help='Process logs')
parser_upload = subparsers.add_parser('upload', help='Upload logs')
args = parser.parse_args()
print("Subparser: ", args.subparser_name)
Now --help shows:
$ python /tmp/aaa.py --help
usage: aaa.py [-h] {process,upload} ...
Log archiver arguments.
positional arguments:
{process,upload} sub-command help
process Process logs
upload Upload logs
optional arguments:
-h, --help show this help message and exit
$ python /tmp/aaa.py
usage: aaa.py [-h] {process,upload} ...
aaa.py: error: too few arguments
$ python3 /tmp/aaa.py upload
Subparser: upload
You can add additional options to these sub-parsers as well. Also instead of using that dest='subparser_name' you can also bind functions to be directly called on given sub-command (see docs).
For cases like
parser.add_argument("--a")
parser.add_argument("--b")
We can use the following
if not args.a and not args.b:
parser.error("One of --a or --b must be present")
Use append_const to a list of actions and then check that the list is populated:
parser.add_argument('-process', dest=actions, const="process", action='append_const')
parser.add_argument('-upload', dest=actions, const="upload", action='append_const')
args = parser.parse_args()
if(args.actions == None):
parser.error('Error: No actions requested')
You can even specify the methods directly within the constants.
def upload:
...
parser.add_argument('-upload', dest=actions, const=upload, action='append_const')
args = parser.parse_args()
if(args.actions == None):
parser.error('Error: No actions requested')
else:
for action in args.actions:
action()
Using
parser = argparse.ArgumentParser(description='Log archiver arguments.')
parser.add_argument('-process', action='store_true')
parser.add_argument('-upload', action='store_true')
args = parser.parse_args()
Maybe try:
if len([False for arg in vars(args) if vars(args)[arg]]) == 0:
parsers.print_help()
exit(-1)
At least this is what I just used; hopefully this helps someone in the future!