Does anyone know what is the main difference between specifying an argparse agument with a single dash -r and a double dash --r?
I came across this Julia language argparse which classifies them as short and long but doesn't say exactly why you would use one over the other.
The long options like --foo come from the GNU tradition:
GNU adds long options to these conventions. Long options consist of ‘--’ followed by a name made of alphanumeric characters and dashes. Option names are typically one to three words long, with hyphens to separate words. Users can abbreviate the option names as long as the abbreviations are unique.
It is very common to support both a short option -f and a long option --foo. The double dash -- is used to distinguish a long option from a collection of short options. Usually, you can write both
go -a -b -c
or the condensed form
go -abc
That's different from
go --abc
which is a different option.
If following the usual conventions, the single dash arguments only consist of a single char, thus, -abc is identical to -a -b -c. Parameter names with double-dash are treated as whole words, so --abc is only a single parameter named abc.
Although argparse supports the convention where multiple short options (using single-dash) can be specified using a single letter, e.g., -abc can be a shorter way to specify -a -b -c, argparse also allows unconventional use of words with single-dash options, which can render some short forms unusable due to ambiguity:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-ab', action='store_true')
parser.add_argument('-ac', action='store_true')
parser.add_argument('-b', action='store_true')
parser.add_argument('-c', action='store_true')
parser.add_argument('-bczzz', action='store_true')
parser.add_argument('-z', action='store_true')
args = parser.parse_args()
print(args)
$ python3 parse.py -a
usage: parse.py [-h] [-ab] [-ac] [-b] [-c] [-bczzz] [-z]
parse.py: error: ambiguous option: -a could match -ab, -ac
$ python3 parse.py -ab
Namespace(ab=True, ac=False, b=False, bczzz=False, c=False, z=False)
$ python3 parse.py -abz
usage: parse.py [-h] [-ab] [-ac] [-b] [-c] [-bczzz] [-z]
parse.py: error: unrecognized arguments: -abz
$ python3 parse.py -bc
usage: parse.py [-h] [-ab] [-ac] [-b] [-c] [-bczzz] [-z]
parse.py: error: ambiguous option: -bc could match -b, -bczzz
$ python3 parse.py -cb
Namespace(ab=False, ac=False, b=True, bczzz=False, c=True, z=False)
$ python3 parse.py -bczz
usage: parse.py [-h] [-ab] [-ac] [-b] [-c] [-bczzz] [-z]
parse.py: error: ambiguous option: -bczz could match -b, -bczzz
Related
I want to recreate [-A [-b value]] where in command would look like this:
test.py -A -b 123
Seems really simple but I can't get it right. My latest attempt has been:
byte = subparser.add_parser("-A")
byte.add_argument("-b", type=int)
While the add_parser command accepts '-A', the parser cannot use it. Look at the help:
usage: ipython3 [-h] {-A} ...
positional arguments:
{-A}
optional arguments:
-h, --help show this help message and exit
A subparser is really a special kind of positional argument. To the main parser, you have effectively defined
add_argument('cmd', choices=['-A'])
But to the parsing code, '-A' looks like an optional's flag, as though you had defined
add_argument('-A')
The error:
error: argument cmd: invalid choice: '123' (choose from '-A')
means that it has skipped over the -A and -b (which aren't defined for the main parser), and tried to parse '123' as the first positional. But it isn't in the list of valid choices.
So to use subparsers, you need specify 'A' as the subparser, not '-A'.
I'd like to support a command line interface where users can declare an arbitrary number of samples, with one or more input files corresponding to each sample. Something like this:
$ myprogram.py \
--foo bar \
--sample1 input1.tsv \
--sample2 input2a.tsv input2b.tsv input2c.tsv \
--sample3 input3-filtered.tsv \
--out output.tsv
The idea is that the option keys will match the pattern --sample(\d+), and each key will consume all subsequent arguments as option values until the next - or -- prefixed flag is encountered. For explicitly declared arguments, this is a common use case that the argparse module supports with the nargs='+' option. But since I need to support an arbitrary number of arguments I can't declare them explicitly.
The parse_known_args command will give me access to all user-supplied arguments, but those not explicitly declared will not be grouped into an indexed data structure. For these I would need to carefully examine the argument list, look ahead to see how many of the subsequent values correspond to the current flag, etc.
Is there any way I can parse these options without having to essentially re-implement large parts of an argument parser (almost) from scratch?
If you can live with a slightly different syntax, namely:
$ myprogram.py \
--foo bar \
--sample input1.tsv \
--sample input2a.tsv input2b.tsv input2c.tsv \
--sample input3-filtered.tsv \
--out output.tsv
where the parameter name doesn't contain a number, but still it performs grouping, try this:
parser.add_argument('--sample', action='append', nargs='+')
It produces a list of lists, ie. --sample x y --sample 1 2 will produce Namespace(sample=[['x', 'y'], ['1', '2']])
As I mentioned in my comment:
import argparse
argv = "myprogram.py \
--foo bar \
--sample1 input1.tsv \
--sample2 input2a.tsv input2b.tsv input2c.tsv \
--sample3 input3-filtered.tsv \
--out output.tsv"
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
parser.add_argument('--out')
for x in range(1, argv.count('--sample') + 1):
parser.add_argument('--sample' + str(x), nargs='+')
args = parser.parse_args(argv.split()[1:])
Gives:
print args
Namespace(foo='bar', out='output.tsv', sample1=['input1.tsv'], sample2=['input2a.tsv', 'input2b.tsv', 'input2c.tsv'], sample3=['input3-filtered.tsv'])
With the real sys.argv you'll probably have to replace the argv.count with the slightly longer ' '.join(sys.argv).count('--sample')
The major downside to this approach is the auto help generation will not cover these fields.
It would be simpler to make that number or key at separate argument value, and collect the related arguments in an nested list.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
parser.add_argument('--out')
parser.add_argument('--sample', nargs='+', action='append', metavar=('KEY','TSV'))
parser.print_help()
argv = "myprogram.py \
--foo bar \
--sample 1 input1.tsv \
--sample 2 input2a.tsv input2b.tsv input2c.tsv \
--sample 3 input3-filtered.tsv \
--out output.tsv"
argv = argv.split()
args = parser.parse_args(argv[1:])
print(args)
produces:
1031:~/mypy$ python3 stack44267794.py -h
usage: stack44267794.py [-h] [--foo FOO] [--out OUT] [--sample KEY [TSV ...]]
optional arguments:
-h, --help show this help message and exit
--foo FOO
--out OUT
--sample KEY [TSV ...]
Namespace(foo='bar', out='output.tsv',
sample=[['1', 'input1.tsv'],
['2', 'input2a.tsv', 'input2b.tsv', 'input2c.tsv'],
['3', 'input3-filtered.tsv']])
There have been questions about collecting general key:value pairs. There's nothing in argparse to directly support that. Various things have been suggested, but all boil down to parsing the pairs yourself.
Is it possible to use argparse to capture an arbitrary set of optional arguments?
You have added the complication that the number of arguments per key is variable. That rules out handling '--sample1=input1' as simple strings.
argparse has extended a well known POSIX commandline standard. But if you want to move beyond that, then be prepared to process the arguments either before (sys.argv) or after argparse (the parse_known_args extras).
It may well be possible to do the sort of thing that you are looking for with click rather than argparse.
To quote:
$ click_
Click is a Python package for creating beautiful command line
interfaces in a composable way with as little code as necessary.
It's the "Command Line Interface Creation Kit". It's highly
configurable but comes with sensible defaults out of the box.
It aims to make the process of writing command line tools quick and
fun while also preventing any frustration caused by the inability to
implement an intended CLI API.
Click in three points:
arbitrary nesting of commands
automatic help page generation
supports lazy loading of subcommands at runtime
Read the docs at http://click.pocoo.org/
One of the important features of click is the ability to construct sub-commands, (a bit like using git or image magic covert), which should allow you to structure your command line as:
myprogram.py \
--foo bar \
--sampleset input1.tsv \
--sampleset input2a.tsv input2b.tsv input2c.tsv \
--sampleset input3-filtered.tsv \
combinesets --out output.tsv
Or even:
myprogram.py \
--foo bar \
process input1.tsv \
process input2a.tsv input2b.tsv input2c.tsv \
process input3-filtered.tsv \
combine --out output.tsv
Which might be cleaner, in this case your code would have parameters called --foo and --out and functions called process and combine process would be called with the input file(s) specified and combine with no parameters.
There is probably an obvious answer to this question, but I've looked at it for a bit without figuring it out. This is some old Python code using argparse. I haven't used argparse recently, so I may have forgotten some nuance.
#test.py
def load_crossval_dataset(args):
schema, samplenum, permuted, search = args.schema, args.samplenum, args.permuted, args.search
print "schema", schema
print "samplenum", samplenum
print "permuted", permuted
print "search", search
import argparse
parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
subparsers = parser.add_subparsers()
# create the parser for the "crossval" command
parser_crossval = subparsers.add_parser('crossval', formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser_crossval.add_argument('schema', help='name of schema')
parser_crossval.add_argument("-n", "--samplenum", action="store", type=int, dest="samplenum", help="number of samples to do crossvalidation on")
parser_crossval.add_argument("-p", "--permuted", action="store_true", dest="permuted", help="permuted dataset", default=False)
parser_crossval.add_argument("-s", "--search", action="store_true", dest="search", help="model search", default=False)
parser_crossval.set_defaults(func=load_crossval_dataset)
args = parser.parse_args()
args.func(args)
Let us invoke this as:
python test.py
usage: test.py [-h] {crossval} ...
test.py: error: too few arguments
Now as
python test.py crossval -h
usage: test.py crossval [-h] [-n SAMPLENUM] [-p] [-s] schema
positional arguments:
schema name of schema
optional arguments:
-h, --help show this help message and exit
-n SAMPLENUM, --samplenum SAMPLENUM
number of samples to do crossvalidation on (default: None)
-p, --permuted permuted dataset (default: False)
-s, --search model search (default: False)
Now as
python test.py crossval -n 1 -s True
schema True
samplenum 1
permuted False
search True
Question: why does argparse not complain about the missing schema argument, and why does it set it to True?
At a glance, the -s option is boolean - so its presence implies True and it requires no argument. So when you say python test.py crossval -n 1 -s True, the True gets parsed as being the schema argument since the -s switch doesn't require a value.
This much can in fact be gleaned from the usage string in the help text:
usage: test.py crossval [-h] [-n SAMPLENUM] [-p] [-s] schema
The [-s] indicates that it's a nullary option, unlike -n which is listed as [-n SAMPLENUM] since it requires an argument (SAMPLENUM).
Edit:
This behavior is stated in the Python 2.7 Documentation for argparse, which I infer is the version you are using in your example since you are using the statement- rather than function-form of print. To quote section 15.4.3.2:
'store_true' and 'store_false' - These are special cases of 'store_const' using for storing the values True and False respectively. In addition, they create default values of False and True respectively.
The option -s doesn't take an argument (store_const, store_true and store_false actions don't take an argument — this could stand to be clarified in the documentation). So in python test.py crossval -n 1 -s True, the argument True is a positional argument of crossval, not an argument of -s; thus it's the value of schema.
python test.py crossval -n 1 -s correctly complains about the missing argument to test.py crossval.
I have been looking at argparse documentation but I am still confused how to use it.
I made a python script to get issues from either pmd, checkstyle, or findbugs after a code analysis. Theses issues are also categorized into severities such as major, blocker, and critical.
So I want to be able to pass in two arguments in the form python script.py arg1 arg2 where arg1 would be a combination of p,c,f which stands for pmd, checkstyle, or findbug and arg2 would be a combination of m,c,b which stands for major, critical, and blocker.
So for instance, if I write python script.py pf cb in the terminal, I would get pmd and findbugs issues of critical and blocker severity.
It would be awesome if someone can give me a general structure of how this should go.
Thanks.
Perhaps you'd rather let the user specify the flags more verbose, like this?
python script.py --checker p --checker f --level c --level b
If so, you can use append
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--checker', action='append')
>>> parser.add_argument('--level', action='append')
Then you get a parameter checker and level, both as lists to iterate over.
If you really want to use the combined flags:
for c in arg1:
run_checker(c, arg2)
Assuming that you just pass the severity levels to the checker in some way.
You could try setting boolean flags if each option is present in an argument:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("pcf", help="pmd, checkstyle, or findbug")
parser.add_argument("mcb", help="major, critical, and blocker")
args = parser.parse_args()
# Use boolean flags
pmd = 'p' in args.pcf
checkstyle = 'c' in args.pcf
findbug = 'f' in args.pcf
major = 'm' in args.mcb
critical = 'c' in args.mcb
blocker = 'b' in args.mcb
This would work using python script.py pf cb.
Also, just a helpful hint. If you place the following at the top of your python file and then make it executable with chmod +x script.py you can then call the file directly (using a *NIX operating system):
#!/usr/bin/env python
import argparse
...
Now run with ./script.py pf cb or even put it in your path and call script.py pf cb
https://stackoverflow.com/users/1401034/ewan is a good solution given your specification. But a slight variation gives users more options, and might be clearer.
parser = argparse.ArgumentParser()
parser.add_argument('-p', '--pmd', action='store_true')
parser.add_argument('-c', '--checkstyle', action='store_true')
parser.add_argument('-f', '--findbug', action='store_true')
parser.add_argument('-m', '--major', action='store_true')
parser.add_argument('-d', '--critical', action='store_true')
parser.add_argument('-b', '--blocker', action='store_true')
args = parser.parse_args()
print args
Sample runs:
1840:~/mypy$ python2.7 stack24265375.py -cfmd
Namespace(blocker=False, checkstyle=True, critical=True, findbug=True, major=True, pmd=False)
1841:~/mypy$ python2.7 stack24265375.py -h
usage: stack24265375.py [-h] [-p] [-c] [-f] [-m] [-d] [-b]
optional arguments:
-h, --help show this help message and exit
-p, --pmd
-c, --checkstyle
-f, --findbug
-m, --major
-d, --critical
-b, --blocker
1842:~/mypy$ python2.7 stack24265375.py --critical --major
Namespace(blocker=False, checkstyle=False, critical=True, findbug=False, major=True, pmd=False)
In this case, there are 6 optional, boolean flags. The short, single letter versions can be strung together in any combination. Except the necessary '-', this form can look just like the one using 2 positional arguments. But they can be mixed and matched in any combination. And the long form is self documenting.
The help display might be clearer if these 6 arguments were divided into 2 argument groups.
parser = argparse.ArgumentParser()
group1 = parser.add_argument_group('issue')
group1.add_argument('-p', '--pmd', action='store_true')
group1.add_argument('-c', '--checkstyle', action='store_true')
group1.add_argument('-f', '--findbug', action='store_true')
group2 = parser.add_argument_group('severity')
group2.add_argument('-m', '--major', action='store_true')
group2.add_argument('-d', '--critical', action='store_true')
group2.add_argument('-b', '--blocker', action='store_true')
args = parser.parse_args()
print args
with the help:
usage: stack24265375.py [-h] [-p] [-c] [-f] [-m] [-d] [-b]
optional arguments:
-h, --help show this help message and exit
issue:
-p, --pmd
-c, --checkstyle
-f, --findbug
severity:
-m, --major
-d, --critical
-b, --blocker
My Program should include the following options, properly parsed by argparse:
purely optional: [-h, --help] and [-v, --version]
mutually exclusive: [-f FILE, --file FILE] and [-u URL, --url URL]
optional if --url was chosen: [-V, --verbose]
required if either --file or --url was chosen: [-F, --format FORMAT]
The desired usage pattern would be:
prog.py [-h] [-v] [-f FILE (-F FORMAT) | -u URL [-V] (-F FORMAT) ]
with the -F requirement applying to both members of the mutually exclusive group.
Not sure if it rather be a positional.
So it should be possible to run:
prog.py -u "http://foo.bar" -V -F csv
and the parser screaming in case i forgot the -F (as he's supposed to).
What i've done so far:
parser = ArgumentParser(decription='foo')
group = parser.add_mutually_exclusive_group()
group.add_argument('-f','--file', nargs=1, type=str, help='')
group.add_argument('-u','--url', nargs=1, type=str, help='')
parser.add_argument('-V','--verbose', action='store_true', default=False, help='')
parser.add_argument('-F','--format', nargs=1, type=str, help='')
Since it has a 'vanilla mode' to run without command line arguments, all arguments must be optional.
How can i implement points 3. and 4. into my code?
EDIT:
I tried -f and -u as subparsers, as described here, but subcommands seem to be treated like positionals and the parser gives me an error: too few arguments if i run it without arguments.
Use of nargs=2 and tuple metavar approximates your goal
parser = argparse.ArgumentParser(prog='PROG')
group = parser.add_mutually_exclusive_group()
group.add_argument('-f','--file', nargs=2, metavar=('FILE','FORMAT'))
group.add_argument('-u','--url', nargs=2, metavar=('URL','FORMAT'))
parser.add_argument('-V','--verbose', action='store_true',help='optional with url')
which produces:
usage: PROG [-h] [-f FILE FORMAT | -u URL FORMAT] [-V]
optional arguments:
-h, --help show this help message and exit
-f FILE FORMAT, --file FILE FORMAT
-u URL FORMAT, --url URL FORMAT
-V, --verbose optional with url
This requires the format along with filename or url, it just doesn't require the -F. As others noted -V can be ignored in the -f case.
I tried -f and -u as subparsers, as described here, but subcommands seem to be treated like positionals and the parser gives me an error: too few arguments if i run it without arguments.
In the latest version(s) subcommands are no longer treated as required positionals. This was, as best I can tell, a side effect of changing the error message to be more informative. Instead of _parse_known_args doing a:
if positionals:
self.error(_('too few arguments'))
it scans _actions to see which are required, and then lists them by name in the error message. This is discussed in http://bugs.python.org/issue9253 . I know this change is in development (3.4), and may also be in 3.3.
These points can enforced in optparse using a callback method when a certain option is present.
However, in argparse these are not available.
You can add a subparser for the url and the file sub-option, and parse these seperatly.
from the help:
Note that the object returned by parse_args() will only contain attributes for
the main parser and the subparser that was selected by the command line
(and not any other subparsers). So in the example above, when the a command
is specified, only the foo and bar attributes are present, and when the b command
is specified, only the foo and baz attributes are present.
But I would just properly document the usage, and just ignore the arguments that are not
applicable.
e.g. let these two command lines behave exactly the same:
prog.py -f FILE -V
prog.py -f FILE