As documentation suggests:
argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--foo')
>>> parser.add_argument('command')
>>> parser.add_argument('args', nargs=argparse.REMAINDER)
>>> print parser.parse_args('--foo B cmd --arg1 XX ZZ'.split())
Namespace(args=['--arg1', 'XX', 'ZZ'], command='cmd', foo='B')
I tried to use this to exactly the same purpose, but in some circumstances it seems buggy for me (or perhaps I get the concept wrong):
import argparse
a = argparse.ArgumentParser()
a.add_argument('-qa', nargs='?')
a.add_argument('-qb', nargs='?')
a.add_argument('rest', nargs=argparse.REMAINDER)
a.parse_args('-qa test ./otherutil bar -q atr'.split())
Result:
test.py: error: ambiguous option: -q could match -qa, -qb
So apparently, if the otherutil has such arguments which somehow "collide" with the arguments given to argparse, it doesn't seem to work correctly.
I would expect when argparse reaches the REMAINDER kind of argument, it just uses up all the strings in the end of the list without any further parsing. Can I reach this effect somehow?
I ran into this while trying to dispatch options to an underlying utility. The solution I wound up using was nargs='*' instead of nargs=argparse.REMAINDER, and then just use the "pseudo-argument" -- to separate the options for my command and the underlying tool:
>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--myflag', action='store_true')
>>> parser.add_argument('toolopts', nargs='*')
>>> parser.parse_args('--myflag -- -a --help'.split())
Namespace(myflag=True, toolopts=['-a', '--help'])
This is reasonably easy to document in the help output.
This has more to do with handling of abbreviations than with the REMAINDER nargs.
In [111]: import argparse
In [112]: a = argparse.ArgumentParser()
...:
...: a.add_argument('-qa', nargs='?')
...: a.add_argument('-qb', nargs='?')
In [113]: a.parse_args('-qa test ./otherutil bar -q atr'.split())
usage: ipython3 [-h] [-qa [QA]] [-qb [QB]]
ipython3: error: ambiguous option: -q could match -qa, -qb
argparse does a 2 pass parsing. First it tries to categorize the strings as options (flags) or arguments. Second it alternates between parsing positionals and optionals, allocating arguments according to the nargs.
Here the ambiguity occurs in the first pass. It's trying to match '-q' with the two available optionals. REMAINDER's special action (absorbing '-q' as though it were an plain string) doesn't occur until the second pass.
Newer argparse versions allow us to turn off the abbreviation handling:
In [114]: a.allow_abbrev
Out[114]: True
In [115]: a.allow_abbrev=False
In [116]: a.parse_args('-qa test ./otherutil bar -q atr'.split())
usage: ipython3 [-h] [-qa [QA]] [-qb [QB]]
ipython3: error: unrecognized arguments: ./otherutil bar -q atr
And if I add the REMAINDER action:
In [117]: a.add_argument('rest', nargs=argparse.REMAINDER)
In [118]: a.parse_args('-qa test ./otherutil bar -q atr'.split())
Out[118]: Namespace(qa='test', qb=None, rest=['./otherutil', 'bar', '-q', 'atr'])
The use of '--' as #Colin suggests works because that string is recognized in the first pass:
In [119]: a.allow_abbrev=True
In [120]: Out[117].nargs='*'
In [121]: a.parse_args('-qa test -- ./otherutil bar -q atr'.split())
Out[121]: Namespace(qa='test', qb=None, rest=['./otherutil', 'bar', '-q', 'atr'])
You need to use two --.
a.add_argument('--qa', nargs='?')
a.add_argument('--qb', nargs='?')
So the options that you define collide with a -q, that accepts at least an argument, defined somewhere else
From argparse doc
ArgumentParser.add_argument(name or flags...)
name or flags - Either a name or a list of option strings, e.g. foo or -f, --foo.
EDIT to reply to #PDani first comment:
This post is interesting.
From what I have understood, argparse follows the POSIX and GNU style.
An important thing is that short (1 letter) option can be grouped together and if one option required one argument this can be be attached to the option letter. For example if you have something like this
a.add_argument('-a', action='store_true')
a.add_argument('-b', action='store_true')
a.add_argument('-c', action='store_true')
a.add_argument('-d', nargs=1)
a.add_argument('-e', nargs=1)
you can call them as -abcd3 -e5 or -a -b -c -d3 -e5 or -cba -e5 -d3, ...
Now, if you have
a.add_argument('-abc', action='store_true')
and you have
would be very hard for argparse to decide if -abc is 3 short arguments attached or one long. So you are forced to define the argument as --abc.
So I guess that you can't use long arguments name with one -.
I know of an alternative way to do the command line parsing called docopt: you can give a look but I doubt that it can solve your problem.
Perhaps some combination of ArgumentParser.parse_known_args() and some other bits of special handling?
This is not be perfect, but might lead in the right direction:
import argparse
import sys
a = argparse.ArgumentParser()
# treat the common-prefixed arguments as options to the prefix
a.add_argument("-q")
# allow a delimiter to set off your arguments from those which should go to the
# other utility, and use parse_known_args() if the delimiter is not present
argv = sys.argv[1:]
if "--" in argv:
i = argv.index("--")
args, extra = a.parse_args(argv[:i]), argv[i + 1:]
else:
a.add_argument("extra", nargs=argparse.REMAINDER)
args, _ = a.parse_known_args(argv)
extra = args.extra
# complain if the `-q` option was not specified correctly
if args.q not in ("something", "otherthing"):
a.error("Must specify '-qsomething' or '-qotherthing'")
print "q:", "-q%s" % (args.q,)
print "extra:", '"%s"' % (" ".join(extra),)
Result:
$ ./testcmd -qsomething test ./otherutil bar -q atr
q: -qsomething
extra: "test ./otherutil bar -q atr"
Caveats:
This will allow a space between -q and the rest of the -q-prefixed option.
This will consume one -q option, but I don't recall whether it will raise an exception (or do any other helpful thing) if more are specified.
Related
I'm trying to write a python script that can echo whatever a user types when running the script
Right now, the code I have is (version_msg and usage_msg don't matter right now)
from optparse import OptionParser
version_msg = ""
usage_msg = ""
parser = OptionParser(version=version_msg, usage=usage_msg)
parser.add_option("-e", "--echo", action="append", dest="input_lines", default=[])
But if I try to run the script (python options.py -e hello world), it echoes just ['hello']. How would I go about fixing this so it outputs ['hello', 'world']?
In argparse this is quite easy, with its nargs parameter:
In [245]: parser = argparse.ArgumentParser()
In [246]: parser.add_argument('-e','--echo', nargs='+');
In [247]: parser.parse_args(['-e','hello','world'])
Out[247]: Namespace(echo=['hello', 'world'])
nargs is used to specify how many strings the Action takes. '+' means one or more. The results are collected in a list. It models the nargs values on the regex usage (e.g. '?' and '*' also work).
In [248]: parser.print_help()
usage: ipython3 [-h] [-e ECHO [ECHO ...]]
optional arguments:
-h, --help show this help message and exit
-e ECHO [ECHO ...], --echo ECHO [ECHO ...]
Looking at the optparse docs, I see a nargs parameter, but it must be a number. For a variable number, we have to use a callback as described in:
https://docs.python.org/2/library/optparse.html#callback-example-6-variable-arguments
Using the function defined in this section:
In [266]: parser = optparse.OptionParser()
In [267]: parser.add_option('-e','--echo', dest='echo', action='callback', callback=vararg_callback);
In [268]: parser.parse_args(['-e','hello','world'])
Out[268]: (<Values at 0x7f0ff208a5c0: {'echo': ['hello', 'world']}>, [])
In argparse, nargs='+' collects values up to the next -- or -, but that allocation is done topdown, by the main parsing routine, not a callback defined for the option itself.
A slightly hacky way of doing it:
from optparse import OptionParser
version_msg = ""
usage_msg = ""
parser = OptionParser(version=version_msg, usage=usage_msg)
parser.add_option("-e", "--echo", action="append", dest="input_lines", default=[])
options, arguments = parser.parse_args()
print(options.input_lines + arguments)
I then run
python myscript.py -e hello world how are you
Output:
['hello', 'world', 'how', 'are', 'you']
I think this is best accomplished by quoting the argument ie hello world becomes 'hello world' this ensures that the -e option consumes the entire string.
If you really need the string to be broken up into pieces ie ['hello', 'world'] instead of ['hello world'] you could easily call split() on options.e[0]
strings = options.e[0].split()
For a more complex method, you can use a callback, below links to a relevant example for you.
https://docs.python.org/3/library/optparse.html#callback-example-6-variable-arguments
I have been looking at argparse documentation but I am still confused how to use it.
I made a python script to get issues from either pmd, checkstyle, or findbugs after a code analysis. Theses issues are also categorized into severities such as major, blocker, and critical.
So I want to be able to pass in two arguments in the form python script.py arg1 arg2 where arg1 would be a combination of p,c,f which stands for pmd, checkstyle, or findbug and arg2 would be a combination of m,c,b which stands for major, critical, and blocker.
So for instance, if I write python script.py pf cb in the terminal, I would get pmd and findbugs issues of critical and blocker severity.
It would be awesome if someone can give me a general structure of how this should go.
Thanks.
Perhaps you'd rather let the user specify the flags more verbose, like this?
python script.py --checker p --checker f --level c --level b
If so, you can use append
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--checker', action='append')
>>> parser.add_argument('--level', action='append')
Then you get a parameter checker and level, both as lists to iterate over.
If you really want to use the combined flags:
for c in arg1:
run_checker(c, arg2)
Assuming that you just pass the severity levels to the checker in some way.
You could try setting boolean flags if each option is present in an argument:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("pcf", help="pmd, checkstyle, or findbug")
parser.add_argument("mcb", help="major, critical, and blocker")
args = parser.parse_args()
# Use boolean flags
pmd = 'p' in args.pcf
checkstyle = 'c' in args.pcf
findbug = 'f' in args.pcf
major = 'm' in args.mcb
critical = 'c' in args.mcb
blocker = 'b' in args.mcb
This would work using python script.py pf cb.
Also, just a helpful hint. If you place the following at the top of your python file and then make it executable with chmod +x script.py you can then call the file directly (using a *NIX operating system):
#!/usr/bin/env python
import argparse
...
Now run with ./script.py pf cb or even put it in your path and call script.py pf cb
https://stackoverflow.com/users/1401034/ewan is a good solution given your specification. But a slight variation gives users more options, and might be clearer.
parser = argparse.ArgumentParser()
parser.add_argument('-p', '--pmd', action='store_true')
parser.add_argument('-c', '--checkstyle', action='store_true')
parser.add_argument('-f', '--findbug', action='store_true')
parser.add_argument('-m', '--major', action='store_true')
parser.add_argument('-d', '--critical', action='store_true')
parser.add_argument('-b', '--blocker', action='store_true')
args = parser.parse_args()
print args
Sample runs:
1840:~/mypy$ python2.7 stack24265375.py -cfmd
Namespace(blocker=False, checkstyle=True, critical=True, findbug=True, major=True, pmd=False)
1841:~/mypy$ python2.7 stack24265375.py -h
usage: stack24265375.py [-h] [-p] [-c] [-f] [-m] [-d] [-b]
optional arguments:
-h, --help show this help message and exit
-p, --pmd
-c, --checkstyle
-f, --findbug
-m, --major
-d, --critical
-b, --blocker
1842:~/mypy$ python2.7 stack24265375.py --critical --major
Namespace(blocker=False, checkstyle=False, critical=True, findbug=False, major=True, pmd=False)
In this case, there are 6 optional, boolean flags. The short, single letter versions can be strung together in any combination. Except the necessary '-', this form can look just like the one using 2 positional arguments. But they can be mixed and matched in any combination. And the long form is self documenting.
The help display might be clearer if these 6 arguments were divided into 2 argument groups.
parser = argparse.ArgumentParser()
group1 = parser.add_argument_group('issue')
group1.add_argument('-p', '--pmd', action='store_true')
group1.add_argument('-c', '--checkstyle', action='store_true')
group1.add_argument('-f', '--findbug', action='store_true')
group2 = parser.add_argument_group('severity')
group2.add_argument('-m', '--major', action='store_true')
group2.add_argument('-d', '--critical', action='store_true')
group2.add_argument('-b', '--blocker', action='store_true')
args = parser.parse_args()
print args
with the help:
usage: stack24265375.py [-h] [-p] [-c] [-f] [-m] [-d] [-b]
optional arguments:
-h, --help show this help message and exit
issue:
-p, --pmd
-c, --checkstyle
-f, --findbug
severity:
-m, --major
-d, --critical
-b, --blocker
I would like to use argparse to make some code to be used in the following two ways:
./tester.py all
./tester.py name someprocess
i.e. either all is specified OR name with some additional string.
I have tried to implement as follows:
import argparse
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument('all', action='store_true', \
help = "Stops all processes")
group.add_argument('name', \
help = "Stops the named process")
print parser.parse_args()
which gives me an error
ValueError: mutually exclusive arguments must be optional
Any idea how to do it right? I also would like to avoid sub parsers in this case.
The question is a year old, but since all the answers suggest a different syntax, I'll give something closer to the OP.
First, the problems with the OP code:
A positional store_true does not make sense (even if it is allowed). It requires no arguments, so it is always True. Giving an 'all' will produce error: unrecognized arguments: all.
The other argument takes one value and assigns it to the name attribute. It does not accept an additional process value.
Regarding the mutually_exclusive_group. That error message is raised even before parse_args. For such a group to make sense, all the alternatives have to be optional. That means either having a -- flag, or be a postional with nargs equal to ? or *. And doesn't make sense to have more than one such positional in the group.
The simplest alternative to using --all and --name, would be something like this:
p=argparse.ArgumentParser()
p.add_argument('mode', choices=['all','name'])
p.add_argument('process',nargs='?')
def foo(args):
if args.mode == 'all' and args.process:
pass # can ignore the process value or raise a error
if args.mode == 'name' and args.process is None:
p.error('name mode requires a process')
args = p.parse_args()
foo(args) # now test the namespace for correct `process` argument.
Accepted namespaces would look like:
Namespace(mode='name', process='process1')
Namespace(mode='all', process=None)
choices imitates the behavior of a subparsers argument. Doing your own tests after parse_args is often simpler than making argparse do something special.
import argparse
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-a','--all', action='store_true', \
help = "Stops all processes")
group.add_argument('-n','--name', \
help = "Stops the named process")
print parser.parse_args()
./tester.py -h
usage: zx.py [-h] (-a | -n NAME)
optional arguments:
-h, --help show this help message and exit
-a, --all Stops all processes
-n NAME, --name NAME Stops the named process
"OR name with some additional string."
positional argument cannot take additional string
I think the best solution for you is (named test.py):
import argparse
p = argparse.ArgumentParser()
meg = p.add_mutually_exclusive_group()
meg.add_argument('-a', '--all', action='store_true', default=None)
meg.add_argument('-n', '--name', nargs='+')
print p.parse_args([])
print p.parse_args(['-a'])
print p.parse_args('--name process'.split())
print p.parse_args('--name process1 process2'.split())
print p.parse_args('--all --name process1'.split())
$ python test.py
Namespace(all=None, name=None)
Namespace(all=True, name=None)
Namespace(all=None, name=['process'])
Namespace(all=None, name=['process1', 'process2'])
usage: t2.py [-h] [-a | -n NAME [NAME ...]]
t2.py: error: argument -n/--name: not allowed with argument -a/--all
I would agree that this looks exactly like a sub-parser problem, and that if you don't want to make it an optional argument by using --all and --name, one suggestion from me would be just to ignore the all and name altogether, and use the following semantics:
If tester.py is called without any arguments, stop all process.
If tester.py is called with some arguments, stop only those processes.
Which can be done using:
import argparse, sys
parser = argparse.ArgumentParser()
parser.add_argument('processes', nargs='*')
parsed = parser.parse(sys.argv[1:])
print parsed
which will behave as follows:
$ python tester.py
Namespace(processes=[])
$ python tester.py proc1
Namespace(processes=['proc1'])
Or, if you insist on your own syntax, you can create a custom class. And actually you're not having a "mutually exclusive group" case, since I assume if all is specified, you will ignore the rest of the arguments (even when name is one of the other arguments), and when name is specified, anything else after that will be regarded as processes' name.
import argparse
import sys
class AllOrName(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
if len(values)==0:
raise argparse.ArgumentError(self, 'too few arguments')
if values[0]=='all':
setattr(namespace, 'all', True)
elif values[0]=='name':
if len(values)==1:
raise argparse.ArgumentError(self, 'please specify at least one process name')
setattr(namespace, 'name', values[1:])
else:
raise argparse.ArgumentError(self, 'only "all" or "name" should be specified')
parser = argparse.ArgumentParser()
parser.add_argument('processes', nargs='*', action=AllOrName)
parsed = parser.parse_args(sys.argv[1:])
print parsed
with the following behaviour:
$ python argparse_test.py name
usage: argparse_test.py [-h] [processes [processes ...]]
argparse_test.py: error: argument processes: please specify at least one process name
$ python argparse_test.py name proc1
Namespace(name=['proc1'], processes=None)
$ python argparse_test.py all
Namespace(all=True, processes=None)
$ python argparse_test.py host
usage: argparse_test.py [-h] [processes [processes ...]]
argparse_test.py: error: argument processes: only "all" or "name" should be specified
$ python argparse_test.py
usage: argparse_test.py [-h] [processes [processes ...]]
argparse_test.py: error: argument processes: too few arguments
This is probably what you're looking for:
group.add_argument('--all', dest=is_all, action='store_true')
group.add_argument('--name', dest=names, nargs='+')
Passing --name will then require at list one value and store them as a list.
I need some help regarding using argparse. What I want to achieve is that I need to pass in only one argument, it could be one of the followings: --k, --r, --b, --p,(ignore the rest). If the argument count is not 1, print "usage" information and quit. Also the program needs to know which flag is passed in in order to create corresponding object. I tried several times but I doesn't work, can anyone give me a hint on this? Thanks.
What you need to use to accomplish that is a mutually exclusive group:
import argparse
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument('-k', action='store_true')
group.add_argument('-r', action='store_true')
group.add_argument('-b', action='store_true')
group.add_argument('-p', action='store_true')
parser.parse_args()
As it can be seen in the example below, only one option in a mutually exclusive group is allowed at the same time:
$ python test.py -k -r -b -p
usage: test.py [-h] [-k | -r | -b | -p]
test.py: error: argument -r: not allowed with argument -k
To check which flag was passed, you just need to look at the argparse.Namespace object returned by parse_args method (the flag passed will be set to True).
How about not using argparse at all? It doesn't seem really necessary.
if len(sys.argv) != 2:
print_usage()
arg = sys.argv[1]
if arg not in ["--k", "--r", "--b", "--p"]:
print_usage()
# Do whatever you want with arg
How do I add an optional flag to my command line args?
eg. so I can write
python myprog.py
or
python myprog.py -w
I tried
parser.add_argument('-w')
But I just get an error message saying
Usage [-w W]
error: argument -w: expected one argument
which I take it means that it wants an argument value for the -w option. What's the way of just accepting a flag?
I'm finding http://docs.python.org/library/argparse.html rather opaque on this question.
As you have it, the argument w is expecting a value after -w on the command line. If you are just looking to flip a switch by setting a variable True or False, have a look here (specifically store_true and store_false)
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-w', action='store_true')
where action='store_true' implies default=False.
Conversely, you could haveaction='store_false', which implies default=True.
Adding a quick snippet to have it ready to execute:
Source: myparser.py
import argparse
parser = argparse.ArgumentParser(description="Flip a switch by setting a flag")
parser.add_argument('-w', action='store_true')
args = parser.parse_args()
print args.w
Usage:
python myparser.py -w
>> True
Your script is right. But by default is of None type. So it considers true of any other value other than None is assigned to args.argument_name variable.
I would suggest you to add a action="store_true". This would make the True/False type of flag. If used its True else False.
import argparse
parser = argparse.ArgumentParser('parser-name')
parser.add_argument("-f","--flag",action="store_true",help="just a flag argument")
usage
$ python3 script.py -f
After parsing when checked with args.f it returns true,
args = parser.parse_args()
print(args.f)
>>>true
If you are looking for a binary flag, then the argparse actions store_true or store_false provide exactly this. This approach is well explained in the accepted answer by #Jdog.
The official docs are also fairly clear. I would only complete the example with one line, so to make it very clear how the store_true/store_false act:
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('--foo', action='store_true')
>>> parser.add_argument('--fov', action='store_true') # this is not in the docs!
>>> parser.add_argument('--bar', action='store_false')
>>> parser.add_argument('--baz', action='store_false')
>>> parser.parse_args('--foo --bar'.split()) # --baz and --fov are missing
Out[4]: Namespace(bar=False, baz=True, foo=True, fov=False) # mind the fov=False
A slightly more powerful approach is to use the count action. You typically have used this type of flag already when setting the verbosity level when running a command.
For example ssh's verbose mode flag -v is a counter:
-v Verbose mode. Causes ssh to print debugging messages about its progress. This is helpful in debugging connection, authentication, and configuration problems. Multiple -v
options increase the verbosity. The maximum is 3.
So if you run ssh it's non verbose, ssh -v is slightly verbose and ssh -vvv is maximally verbose.
With argparse in python such a counter flag can be defined as follows:
parser.add_argument('--verbose', '-v', action='count', default=0)
If you want to use it as a boolena (True/False) flag, then you need to cast args.verbose into a boolean. You can either do this explicitly yourself, or rely a conditional statement like if args.verbose: ....
Here is a full working example to illustrate how you can use the counter flag:
With the script test.py:
#!/usr/bin/env python3
# test.py
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--verbose', '-v', action='count', default=0)
args = parser.parse_args()
if args.verbose:
print('verbose')
print(f'verbosity level: {args.verbose}')
else:
print('non-verbose')
You get the following outputs:
python test.py
>> non-verbose
python test.py -v
>> verbose
>> verbosity level: 1
python test.py -vvv
>> verbose
>> verbosity level: 3
Here's a quick way to do it, won't require anything besides sys.. though functionality is limited:
flag = "--flag" in sys.argv[1:]
[1:] is in case if the full file name is --flag