I am trying to use argparse with subparser to switch between 3 fonctionnalities whereas one positional argument should be common to all subparser. Moreover, and it is the key point, i want to put the positional argument as the last argument provided as this one is an output file path. It makes no sense to me to put it at the beginning (as first argument)
import sys,argparse,os
files = argparse.ArgumentParser(add_help=False)
files.add_argument('outfile', help='output mesh file name')
parser = argparse.ArgumentParser(description="A data interpolation program.",prog='data_interpolate.py', parents=[files])
subparsers = parser.add_subparsers(help='Mode command.')
command_parser = subparsers.add_parser('cmd',help='Pass all argument in command line.',parents=[files])
command_parser.add_argument('-min', dest='MINFILE',help='Input file with min values', required=True)
command_parser.add_argument('-max', dest='MAXFILE',help='Input file with min values', required=True)
command_parser.add_argument('u', help='Interpolation parameter. Float between 0 and 1. Out of bound values are limited to 0 or 1.')
subparsers.add_parser('py',help='Pass all argument in python file.',parents=[files])
subparsers.add_parser('json',help='Pass all argument in json file.',parents=[files])
Which gives:
data_interpolation.py -h
usage: data_interpolation.py [-h] outfile {cmd,py,json}
But, to my opinion, the outfile should be given at the end following:
data_interpolation.py [-h] {cmd,py,json} outfile
This has even more sense when using the cmd command as I need to pass other parameter values. For intance:
data_interpolation.py cmd -min minfile.txt -max maxfile.txt 0.6 outfile.txt
How can I set up argparse to have such behaviour?
(note - this is an old question).
The order of positionals is determined by the order in which they are defined. That includes the subparsers argument (which is a positional with choices and a special action).
Defining outfile as an argument to both the main parser and the subparsers is redundant.
Positionals defined via parents will be placed first. So if 'outfile' must be last, it has to be defined separately for each subparser.
It could also be specified last as postional for the main parser (after the subparser definitions).
In [2]: p=argparse.ArgumentParser()
In [5]: sp=p.add_subparsers(dest='cmd')
In [6]: spp=sp.add_parser('cmd1')
In [7]: spp.add_argument('test')
In [8]: p.add_argument('out')
In [9]: p.print_help()
usage: ipython [-h] {cmd1} ... out
...
In [11]: spp.print_help()
usage: ipython cmd1 [-h] test
...
In [15]: p.parse_args('cmd1 test out'.split())
Out[15]: Namespace(cmd='cmd1', out='out', test='test')
cmd1 is interpreted as a subparser choice. test is interpreted by the subparser as a positional. out is left over, and returned to the main parser to use as it sees fit. This parsing could be messed up if the subparser does not return any extras. So I'd be wary of specifying a final positional like this.
You don't need to specify files as a parent of parser, just for each of the subparsers.
Related
I want to allow either 2 arguments, or 0 arguments and fall back to defaults. I thought that this should do it
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('myargs', nargs=2, default=['foo', 'bar'])
However, this throws on anything but 2 arguments (the defaults are thus never invoked):
print(parser.parse_args(['a', 'b'])) # 2 arguments accepted
print(parser.parse_args([])) # throws
My question is, (how) can this be done without extra code. I.e. I want to find something more elegant and more default argparse than for example this workaround:
import argparse
def parse(args):
parser = argparse.ArgumentParser()
parser.add_argument('myargs', nargs='*', default=['foo', 'bar'])
a = parser.parse_args(args)
if len(a.myargs) != 2:
raise IOError('Incorrect number of arguments')
return a
print(parse([])) # defaults
print(parse(['a', 'b'])) # 2 arguments accepted
print(parse(['a', 'b', 'c'])) # throws (as excepted)
In argparse an argument without prefix (the default prefix is - for abbreviation or -- for full argument name) is considered as mandatory. So, if you want to have an optional argument you can do something like this:
parser = argparse.ArgumentParser()
parser.add_argument('--myargs', nargs=2, default=['foo', 'bar'])
In this case, if you do not pass any argument, it works like expected:
print(parser.parse_args([]))
Namespace(myargs=['foo', 'bar'])
On the other hand, if you provide same values:
print(parser.parse_args(['--myargs', 'a', 'b']))
Namespace(myargs=['a', 'b'])
An error will be raised if you pass the wrong number of arguments after myargs:
print(parser.parse_args(['--myargs', 'a']))
usage: scratch_2.py [-h] [--myargs MYARGS MYARGS]
<your script name>: error: argument --myargs: expected 2 arguments
Another (longer) way is to define a custom action to parse the arguments:
class CustomParsePositional(argparse.Action):
"""Action to parse arguments with a custom processing"""
def __init__(self, option_strings, dest, nargs=None, **kwargs):
super().__init__(option_strings, dest, nargs='*', **kwargs)
self._custom_nargs = len(kwargs.get('default', []))
def __call__(self, parser, namespace, values, option_string=None):
if len(values) != self._custom_nargs:
parser.error('Incorrect number of arguments')
namespace.__setattr__(self.dest, values)
parser = argparse.ArgumentParser()
parser.add_argument('myargs', default=['foo', 'bar'], type=str, action=CustomParsePositional)
In this case the number of expected values is inferred by the number of items in the default argument of the add_argument. Here some examples:
print(parser.parse_args(['a', 'b']))
print(parser.parse_args([]))
print(parser.parse_args(['a']))
Namespace(myargs=['a', 'b'])
Namespace(myargs=['foo', 'bar'])
usage: scratch_2.py [-h] [myargs [myargs ...]]
<your script name>: error: Incorrect number of arguments
And if you pass 3 values you get the an error as well:
print(parser.parse_args(['a', 'b', 'c']))
usage: scratch_2.py [-h] [myargs [myargs ...]]
<your script name>: error: Incorrect number of arguments
A brief description of how argparse parsing works might help.
Values are collected in a namespace object. At the start of parsing, the default values are all placed in namespace. Then as arguments are encountered in the user input they are parsed as specified in the add_argument, and the values placed in namespace, over writing the defaults.
Flagged arguments are parsed when the flag is seen, e.g. '--foo bar', but positionals like yours are required, and take the exact number of strings specified by nargs. In this case it will use the 2 strings. So for this definition, the default parameter is useless.
With nargs='*', any number of strings satisfies it. The case of 0 strings gets special handling, and puts the default in the namespace.
There's nothing in-elegant about adding checks after parsing. The error message can be streamlined with:
if len(a.myargs) != 2:
parser.error('Incorrect number of arguments')
Older parsers like optparse handled all flagged arguments, and returned the rest in an extras for you to handle. argparse handles those extras. Handling positionals with a fixed nargs is easiest. One positional with a variable nargs generally works fine, but it is hard to use more than one. If you specified two actions with *, how is the parser supposed to allocate strings?
Yes, you can define custom Action classes to handle special cases, but often the result is less elegant (IMO) than simpler post-parsing checks. I don't give extra points for cleverness :)
I am attempting to get my script working, but argparse keeps overwriting my positional arguments from the parent parser. How can I get argparse to honor the parent's value for these? It does keep values from optional args.
Here is a very simplified version of what I need. If you run this, you will see that the args are overwritten.
testargs.py
#! /usr/bin/env python3
import argparse
import sys
def main():
preparser = argparse.ArgumentParser(add_help=False)
preparser.add_argument('first',
nargs='?')
preparser.add_argument('outfile',
nargs='?',
type=argparse.FileType('w', encoding='utf-8'),
default=sys.stdout,
help='Output file')
preparser.add_argument(
'--do-something','-d',
action='store_true')
# Parse args with preparser, and find config file
args, remaining_argv = preparser.parse_known_args()
print(args)
parser = argparse.ArgumentParser(
parents=[preparser],
description=__doc__)
parser.add_argument(
'--clear-screen', '-c',
action='store_true')
args = parser.parse_args(args=remaining_argv,namespace=args )
print(args)
if __name__ == '__main__':
main()
And call it with testargs.py something /tmp/test.txt -d -c
You will see it keeps the -d but drops both the positional args and reverts them to defaults.
EDIT: see additional comments in the accepted answer for some caveats.
When you specify parents=[preparser] it means that parser is an extension of preparser, and will parse all arguments relevent to preparser which it is never given.
Lets say the preparser only has one positional argument first and the parser only has one positional argument second, when you make parser a child of preparser it expects both arguments:
import argparse
parser1 = argparse.ArgumentParser(add_help=False)
parser1.add_argument("first")
parser2 = argparse.ArgumentParser(parents=[parser1])
parser2.add_argument("second")
args2 = parser2.parse_args(["arg1","arg2"])
assert args2.first == "arg1" and args2.second == "arg2"
However passing only the remaining arguments that are left over from parser1 would just be ['second'] which is not the correct arguments to parser2:
parser1 = argparse.ArgumentParser(add_help=False)
parser1.add_argument("first")
args1, remaining_args = parser1.parse_known_args(["arg1","arg2"])
parser2 = argparse.ArgumentParser(parents=[parser1])
parser2.add_argument("second")
>>> args1
Namespace(first='arg1')
>>> remaining_args
['arg2']
>>> parser2.parse_args(remaining_args)
usage: test.py [-h] first second
test.py: error: the following arguments are required: second
To only process the arguments that were not handled by the first pass, do not specify it as the parent to the second parser:
parser1 = argparse.ArgumentParser(add_help=False)
parser1.add_argument("first")
args1, remaining_args = parser1.parse_known_args(["arg1","arg2"])
parser2 = argparse.ArgumentParser() #parents=[parser1]) #NO PARENT!
parser2.add_argument("second")
args2 = parser2.parse_args(remaining_args,args1)
assert args2.first == "arg1" and args2.second == "arg2"
The 2 positionals are nargs='?'. A positional like that is always 'seen', since an empty list matches that nargs.
First time through 'text.txt' matches with first and is put in the Namespace. Second time through there isn't any string to match, so the default is used - same as if you had not given that string the first time.
If I change first to have the default nargs, I get
error: the following arguments are required: first
from the 2nd parser. Even though there's a value in the Namespace it still tries to get a value from the argv. (it's like a default, but not quite).
Defaults for positionals with nargs='?' (or *) are tricky. They are optional, but not in quite the same way as optionals. The positional Actions are still called, but with a empty list of values.
I don't think the parents feature does anything for you. preparser already handles that set of arguments; there's no need to handle them again in parser, especially since all the relevant argument strings have been stripped out.
Another option is to leave the parents in, but use the default sys.argv[1:] in the 2nd parser. (but beware of side effects like opening files)
args = parser.parse_args(namespace=args )
A third option is to parse the arguments independently and merge them with a dictionary update.
adict = vars(preparse_args)
adict.update(vars(parser_args))
# taking some care in who overrides who
For more details look in argparse.py file at ArgumentParser._get_values, specifically the not arg_strings cases.
A note about the FileType. That type works nicely for small scripts where you will use the files right away and exit. It isn't so good on large programs where you might want to close the file after use (close stdout???), or use files in a with context.
edit - note on parents
add_argument creates an Action object, and adds it to the parser's list of actions. parse_args basically matches input strings with these actions.
parents just copies those Action objects (by reference) from parent to child. To the child parser it is just as though the actions were created with add_argument directly.
parents is most useful when you are importing a parser and don't have direct access to its definition. If you are defining both parent and child, then parents just saves you some typing/cut-n-paste.
This and other SO questions (mostly triggered the by-reference copy) show that the developers did not intend you to use both the parent and child to do parsing. It can be done, but there are glitches that the they did not consider.
===================
I can imagine defining a custom Action class that would 'behave' in a situation like this. It might, for example, check the namespace for some not default value before adding its own (possibly default) value.
Consider, for example if I changed the action of first to 'append':
preparser.add_argument('first', action='append', nargs='?')
The result is:
1840:~/mypy$ python3 stack37147683.py /tmp/test.txt -d -c
Namespace(do_something=True, first=['/tmp/test.txt'], outfile=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>)
Namespace(clear_screen=True, do_something=True, first=['/tmp/test.txt', None], outfile=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>)
From the first parser, first=['/tmp/test.txt']; from the second, first=['/tmp/test.txt', None].
Because of the append, the item from the first is preserved, and a new default has been added by the second parser.
My script is accepting --full, --last and --check using ArgParse. If no option is provided, it just show the help message. But in that message, the parameters appear as optional.
usage: script.py [-h] [--full] [--last] [--check log_file]
If I use the keyword required, then the script will always expect the parameter, which is not correct.
usage: script.py [-h] --full --last --check log_file
So, how can I show something like:
usage: script.py [-h] (--full |--last |--check log_file)
Indicating that the help is optional but that at least one of those parameters is required.
On the question of customizing the usage:
The parser constructor takes a usage parameter. The immediate effect is to set an attribute:
parser = argparse.ArgumentParser( ... usage=custom_usage...)
print(parser.usage)
# should show None or the custom_usage string
Being a normal Python object attribute, you can change it after the parser was created.
usage_str = parser.format_usage()
The format_usage method ask the parser for create the usage that will be shown in the help (and error messages). If the parser.usage value is None, it formats it from the arguments. If a string, it is used as is (I think it fills in values like %(prog)s).
So you could write a usage string from scratch. Or you could set up the parser, get the current usage string, and edit that to suit your needs. Editing is most likely something you'd do during development, while testing the parser in an IDE. But it could be done on the fly.
A crude example:
In [441]: parser=argparse.ArgumentParser()
In [442]: g=parser.add_mutually_exclusive_group()
In [443]: g.add_argument('--foo')
In [444]: g.add_argument('--bar')
In [445]: ustr = parser.format_usage()
# 'usage: ipython3 [-h] [--foo FOO | --bar BAR]\n'
In [450]: parser.usage = ustr.replace('[','(').replace(']',')')
In [451]: parser.format_usage()
# 'usage: usage: ipython3 (-h) (--foo FOO | --bar BAR)\n'
I've replaced the [] with () (even on the -h :( ).
For now testing logical combinations of the args attributes is the best choice. Inside the parse_args functions the parser maintains a list (set actually) of arguments that it has seen. That is used to test for required arguments, and for mutually_exclusive_arguments, but it is not available outside that code.
For store_true (or false) arguments, just check their truth value. For others I like to test for the default None. If you use other default values, test accordingly. A nice thing about None is that the user cannot give you that value.
Perhaps the most general way to test for arguments is to count the number of attributes which are not None:
In [461]: args=argparse.Namespace(one=None, tow=2, three=None)
In [462]: ll = ['one','tow','three']
In [463]: sum([getattr(args,l,None) is not None for l in ll])
Out[463]: 1
0 means none are found; >0 at least one present; ==len(ll) all found; >1 violates mutually exclusivity; '==1' for required mutually exclusive.
As #doublep explained in his answer, if you want to use more than one option at a time:
Change the usage message manually to the one you want.
Add the following code from Python argparse: Make at least one argument required:
if not (args.full or args.last or args.check):
parse.error('[-] Error: DISPLAY_ERROR_MESSAGE')
You can use add_mutually_exclusive_group():
parser = argparse.ArgumentParser ()
group = parser.add_mutually_exclusive_group (required = True)
group.add_argument ('--foo')
group.add_argument ('--bar')
However, the main effect is that you won't be able to use more than one option at a time.
I create a argparser like this:
parser = argparse.ArgumentParser(description='someDesc')
parser.add_argument(-a,required=true,choices=[x,y,z])
parser.add_argument( ... )
However, only for choice "x" and not for choices "y,z", I want to have an additional REQUIRED argument. For eg.
python test -a x // not fine...needs additional MANDATORY argument b
python test -a y // fine...will run
python test -a z // fine...will run
python test -a x -b "ccc" // fine...will run
How can I accomplish that with ArgumentParser? I know its possible with bash optparser
To elaborate on the subparser approach:
sp = parser.add_subparsers(dest='a')
x = sp.add_parser('x')
y=sp.add_parser('y')
z=sp.add_parser('z')
x.add_argument('-b', required=True)
this differs from your specification in that no -a is required.
dest='a' argument ensures that there is an 'a' attribute in the namespace.
normally an optional like '-b' is not required. Subparser 'x' could also take a required positional.
If you must use a -a optional, a two step parsing might work
p1 = argparse.ArgumentParser()
p1.add_argument('-a',choices=['x','y','z'])
p2 = argparse.ArgumentParser()
p2.add_argument('-b',required=True)
ns, rest = p1.parse_known_args()
if ns.a == 'x':
p2.parse_args(rest, ns)
A third approach is to do your own test after the fact. You could still use the argparse error mechanism
parser.error('-b required with -a x')
ArgumentParser supports the creation of such sub-commands with the
add_subparsers() method. The add_subparsers() method is normally
called with no arguments and returns a special action object. This
object has a single method, add_parser(), which takes a command name
and any ArgumentParser constructor arguments, and returns an
ArgumentParser object that can be modified as usual.
ArgumentParser.add_subparsers()
I am using argparser to parse the command line arguments.
Now, I have something like
./script.py 1112323 0 --salary 100000 -- age 34
Here first two are positional arguments and rest are optional.
Now, I want to have a feature such that when the user gives a filename as input in command line, then it should override these above arguments and take the arguments from header of the file. I meam when user gives sth like
id|sequence|age|name|........... (header of the file with first two cols as positional arguments and rest positional)
On giving this in command line:
./script.py -f filename
it should not complain of above positional arguments.
Is this feasible over my current implementation?
You will most likely need to implement this check yourself. Make both arguments (positional and -f) optional (required=False and nargs="*") and then implement your custom check and use the error method of ArgumentParser. To make it easier for user mention the correct usage in help string.
Something like this:
parser = ArgumentParser()
parser.add_argument("positional", nargs="*", help="If you don't provide positional arguments you need use -f")
parser.add_argument("-f", "--file", required=False, help="...")
args = parser.parse_args()
if not args.file and not args.positional:
parser.error('You must use either -f or positional argument')