I have read a similar question asked on SO, which doesn't address my own problem. For illustration purpose, say I have a Python program using argpase that provides two subcommands: copy and resume:
prog copy src dest # src, dest positional, required
prog resume id # id positional, required
However, the most natural way to invoke the "copy" command is NOT explicitly give the copy subcommand, that is, I was hoping for:
prog src dest
will do the default copy action, while still keep the benefits of have two subparsers, each handles a different set of arguments. Is it possible with argparse package?
Formally there isn't. The subcommand argument is a required positional argument, where the 'choices' are the subparser names (and their aliases).
That's evident in the help message, were {cmd1,cmd} are shown as choices.
usage: ipython3 [-h] {cmd1,cmd2} ...
positional arguments:
{cmd1,cmd2}
optional arguments:
-h, --help show this help message and exit
The latest Python has a 'bug' that actually lets it be optional. In other words, it does not raise an error if you don't give any positionals. It's a bug because it changes previous behavior, and most people what it to be required. But even when it is optional, and cmd1 defined as the default, it won't run the cmd1 parser on the remaining arguments. And any 'positional' argument will be treated as a wrong command string.
I think the best you can do is define one or more positionals. One might have choices and default, and be optional (nargs='?'). The rest (or other) might has nargs='+'. By playing around with those options you could approximate the desired behavior, but it won't make use of the subparsers mechanism.
Another way to think about the issue, is to consider an input like
prog one two
Should it interpret that as prog src dest or prog cmd id. The only basis for saying it should be the former is the fact that one is not one of copy or resume. But it isn't that smart when it comes to handling 'choices'. It assigns a value based on position, and then tests whether it meets criteria like 'choices' or 'type' (ie. integer, float v string).
Related
I need to edit a Python 3 program, let's call it ./foo, that uses argeparse to parse its argument list.
Currently, this program allows one to specify an optional input file on the command line. If it is not specified, then foo reads the file setting.txt
$ ./foo
$ ./foo --input my_settings.txt
The code that does this pretty simple:
parser.add_argument(
'--input',
default='settings.txt'
)
The intent was to give personalized settings, but in practice, 99% of people who use the --input argument specify one particular file other_settings.txt. I'd like to add a mutually exclusive convenience argument, --common that is equivalent to the --input other_settings.txt
# These are the same.
$ ./foo --input other_settings.txt
$ $ ./foo --common
What is the recommended way to accomplish this? For example, should I have both arguments write to the same dest value? Do I have them write to separate destinations and then add some logic inside the body of the code, similar as was done here?
You can easily accomplish this by parsing the arguments, and then:
if common is not None:
input_file = "other_settings.txt"
The solution was to create a mutually exclusive argument group grp. I added two arguments to the group in this order:
The original argument
A new argument with
grp.add_argument(
'--input',
default='settings.txt'
)
grp.add_argument(
"--common",
action="store_const",
dest="input",
const="other_settings.txt",
)
This order seems to ensure that the value that is provided when neither of them are specified is settings.txt
This is a program that already exists and I'm trying to extend it - so my hands are tied somewhat :-(.
I have a program where I want to add an option that takes an unknown number of values so I'm trying to use nargs='+'. I can spot when a value is not actually for my option and is a positional argument and I can then use setattr to set the positional argument - but argparse doesn't get a chance to find the positional argument itself so complains.
The syntax for the command, as show in arparse generated help text, is
command [--option value [value...]] positional
In theory this is possible if I did this instead
command positional [--option value [value...]]
This is precisely how examples, even in the argparse documentation, work but that is NOT how the command is currently used, NOT how users typically provide programs with options and NOT how argparse generated help text shows the expected syntax.
So it there a way to somehow both handle the positional but also tell argparse 'oh, I found this positional so no need to complain that it is missing'?
In my script I have 3 positional arguments and 1 optional argument. One of the three positional arguments is required and the rest is optional (as specified using narg='?').
My optional argument doesn't pass any other arguments (action ='store_true') and is just there to enable sorting which will be implemented at a later time.
However, my problem is that my optional argument only works when it is the first or last argument in the script call.
Below is my script so far:
parser = argparse.ArgumentParser()
parser.add_argument("-s", "--sort", help="option to sort the ip addresses", action='store_true')
parser.add_argument("file_name", help="Name of the file containing the tcpdump")
parser.add_argument("source_IP", nargs='?', type=str, help="Source ip to search")
parser.add_argument("dest_IP", nargs='?', type=str, help="Destination ip to search")
args = parser.parse_args()
If I enter my -s between any of the other positional arguments I get an error.
Ex: ./my_script file.txt -s 192.168.152.32 192.168.0.25
usage: 1 [-h] [-s] file_name [source_IP] [dest_IP]
1: error: unrecognized arguments: 192.168.152.32 192.168.0.25
My goal is to be able to enter my optional argument (-s) anywhere in the script call and have it working.
You have three positional arguments, but two of them are also optional thanks to nargs='?'. argparse is getting screwed up because it sees the positional filename, and then has to choose arbitrarily between interpreting the -s as the optional positional source, or as the switch. Either interpretation is valid (it's not doing complicated backtracking parsing to try to find some legal interpretation of the arguments that would allow it to complete; doing so with some argument types could lead to very bad behavior, like opening a file, then backtracking, closing it, and opening something else).
Short answer: In general, optional arguments should be either all positional, or all switches. Mixing and matching introduces complications that would make parsing a complicated recursive process that could only heuristically guess at the correct parsing (particularly with nargs='*' and nargs='+', but even '?' causes problems as you see). Removing the nargs qualifier from source and dest, or leaving them optional and converting to switches will allow -s to be passed in whatever order you like.
This problem is a subtile one, and requires a good understanding of how argparse parses postionals and optionals.
If all the positionals took one argument (the default nargs), then the -s could occur anywhere - start, end, or between any positional.
What happens with:
./my_script file.txt -s 192.168.152.32 192.168.0.25
is that source_IP,dest_IP are both consumed (and set to their defaults) when file_name is parsed. It then handles -s. Now there are 2 strings left, but no positionals to consume them, hence the unrecognized arguments error. Note that ./my_script file.txt runs fine, as does ./my_script file.txt -s. In all 3 cases, the 2 IP arguments are consumed at the same time as file_name.
parse_args alternates between consuming positionals and optionals. It will consume as many positionals at time as there strings (think of a greedy regex expression). Since source_IP and dest_IP are ok with 0 args, all three are consumed the first time it handles positionals.
There isn't a neat fix for the user, except to be wary of using nargs='?' positionals.
There is a bug/issue that tries to fix this. But the fix isn't trivial. The parser has to 'look ahead', noting that it could delay parsing these '?' arguments.
http://bugs.python.org/issue15112
argparse: nargs='*' positional argument doesn't accept any items
if preceded by an option and another positional
Your parser runs fine with the argparse patched as proposed in that issue. But there's quite a backlog of potential patches for argparse.
My script is accepting --full, --last and --check using ArgParse. If no option is provided, it just show the help message. But in that message, the parameters appear as optional.
usage: script.py [-h] [--full] [--last] [--check log_file]
If I use the keyword required, then the script will always expect the parameter, which is not correct.
usage: script.py [-h] --full --last --check log_file
So, how can I show something like:
usage: script.py [-h] (--full |--last |--check log_file)
Indicating that the help is optional but that at least one of those parameters is required.
On the question of customizing the usage:
The parser constructor takes a usage parameter. The immediate effect is to set an attribute:
parser = argparse.ArgumentParser( ... usage=custom_usage...)
print(parser.usage)
# should show None or the custom_usage string
Being a normal Python object attribute, you can change it after the parser was created.
usage_str = parser.format_usage()
The format_usage method ask the parser for create the usage that will be shown in the help (and error messages). If the parser.usage value is None, it formats it from the arguments. If a string, it is used as is (I think it fills in values like %(prog)s).
So you could write a usage string from scratch. Or you could set up the parser, get the current usage string, and edit that to suit your needs. Editing is most likely something you'd do during development, while testing the parser in an IDE. But it could be done on the fly.
A crude example:
In [441]: parser=argparse.ArgumentParser()
In [442]: g=parser.add_mutually_exclusive_group()
In [443]: g.add_argument('--foo')
In [444]: g.add_argument('--bar')
In [445]: ustr = parser.format_usage()
# 'usage: ipython3 [-h] [--foo FOO | --bar BAR]\n'
In [450]: parser.usage = ustr.replace('[','(').replace(']',')')
In [451]: parser.format_usage()
# 'usage: usage: ipython3 (-h) (--foo FOO | --bar BAR)\n'
I've replaced the [] with () (even on the -h :( ).
For now testing logical combinations of the args attributes is the best choice. Inside the parse_args functions the parser maintains a list (set actually) of arguments that it has seen. That is used to test for required arguments, and for mutually_exclusive_arguments, but it is not available outside that code.
For store_true (or false) arguments, just check their truth value. For others I like to test for the default None. If you use other default values, test accordingly. A nice thing about None is that the user cannot give you that value.
Perhaps the most general way to test for arguments is to count the number of attributes which are not None:
In [461]: args=argparse.Namespace(one=None, tow=2, three=None)
In [462]: ll = ['one','tow','three']
In [463]: sum([getattr(args,l,None) is not None for l in ll])
Out[463]: 1
0 means none are found; >0 at least one present; ==len(ll) all found; >1 violates mutually exclusivity; '==1' for required mutually exclusive.
As #doublep explained in his answer, if you want to use more than one option at a time:
Change the usage message manually to the one you want.
Add the following code from Python argparse: Make at least one argument required:
if not (args.full or args.last or args.check):
parse.error('[-] Error: DISPLAY_ERROR_MESSAGE')
You can use add_mutually_exclusive_group():
parser = argparse.ArgumentParser ()
group = parser.add_mutually_exclusive_group (required = True)
group.add_argument ('--foo')
group.add_argument ('--bar')
However, the main effect is that you won't be able to use more than one option at a time.
I was wondering if it is possible to have a positional argument follow an argument with an optional parameter. Ideally the last argument entered into the command line would always apply toward 'testname'.
import argparse
parser = argparse.ArgumentParser(description='TAF')
parser.add_argument('-r','--release',nargs='?',dest='release',default='trunk')
parser.add_argument('testname',nargs='+')
args = parser.parse_args()
I would like both of these calls to have smoketest apply to testname, but the second one results in an error.
>> python TAF.py -r 1.0 smoketest
>> python TAF.py -r smoketest
TAF.py: error: too few arguments
I realize that moving the positional argument to the front would result in the correct behavior of the optional parameter, however this is not quite the format I am looking for. The choices flag looks like an attractive alternative, however it throws an error instead of ignoring the unmatched item.
EDIT:
I've found a hacky way around this. If anyone has a nicer solution I would appreciate it.
import argparse
parser = argparse.ArgumentParser(description='TAF')
parser.add_argument('-r','--release',nargs='?',dest='release',default='trunk')
parser.add_argument('testname',nargs=argparse.REMAINDER)
args = parser.parse_args()
if not args.testname:
args.testname = args.release
args.release = ''
As stated in the documentation:
'?'. One argument will be consumed from the command line if possible,
and produced as a single item. If no command-line argument is present,
the value from default will be produced. Note that for optional
arguments, there is an additional case - the option string is present
but not followed by a command-line argument. In this case the value
from const will be produced.
So, the behaviour you want is not obtainable using '?'. Probably you could write some hack using argparse.Action and meddling with the previous results.(1)
I think the better solution is to split the functionality of that option. Make it an option that requires an argument(but the option itself is optional) and add an option without argument that sets the release to 'trunk'. In this way you can obtain the same results without any hack. Also I think the interface is simpler.
In your example:
python TAF.py -r smoketest
It's quite clear that smoketest will be interpreted as an argument to -r. At least following unix conventions. If you want to keep nargs='?' then the user must use --:
$ python TAF.py -r -- sometest
Namespace(release=None, testname=['sometest']) #parsed result
(1) An idea on how to do this: check if the option has an argument. If it has one check if it is a valid test name. If so put into by hand into testname and set release to the default value. You'll also have to set a "flag" that tells you that this thing happened.
Now, before parsing sys.argv you must redirect sys.stderr. When doing the parsing you must catch SystemExit, check the stderr and see if the error was "too few arguments", check if the flag was set, if so ignore the error and continue running, otherwise you should reprint to the original stderr the error message and exit.
This approach does not look robust, and it's probably buggy.