We Keep Coding

Convert many argparse conditional arguements to a more pythonic solution

I am working on a cli application using python. I have a set of arguments that are mutually exclusive and a set of arguments that must be there if one of those mutually exclusive arguments is passed.
I have got it working using brute force and lengthy if conditions, but I feel like there is a neat way to do this. Researching about that told me subparsers might be useful, however I am not able to correctly use subparsers at all.
The conditions I want are the following-
Main activities
+-----+--------+--------+--------+---------+
| get | create | delete | update | remove |
+-----+--------+--------+--------+---------+
These main activities are mutually exclusive.
If get is specified, then there should not be any more arguements.
If delete is specified then there should not be any more arguements.
If remove is specified then -r is mandatory.
If update is specified then -f mandatory, and-cd is optional.
If create is specified, then -d is mandatory, -m and -f are optional where -m and -f are mutually exclusive.
The brute force code is as follows -
import argparse
parser = argparse.ArgumentParser(description='Check args')
#get
parser.add_argument('-g', '--get', help='')
#create
parser.add_argument('-c', '--create', help='')
parser.add_argument('-d', '--display', help='')
parser.add_argument('-m', '--message', help='')
parser.add_argument('-f', '--file', help='')
#update
parser.add_argument('-u', '--update', help='')
parser.add_argument('-cd', '--changed', help='')
#delete
parser.add_argument('-del', '--delete', help='')
#remove
parser.add_argument('-rm', help='')
parser.add_argument('-r', '--remove', help='')
args = vars(parser.parse_args())
if args['get'] is not None:
if args['create'] is None and args['display'] is None and args['message'] is None and args['file'] is None and args['update'] is None and args['changed'] is None and args['delete'] is None and args['rm'] is None and args['remove'] is None:
print(args['get'])
else:
print('Dont mix get with others')
exit()
if args['create']:
if args['get'] is None and args['message'] is None and args['file'] is None and args['update'] is None and args['changed'] is None and args['delete'] is None and args['rm'] is None and args['remove'] is None:
print(args['create'])
else:
print('Dont mix create with others')
exit()
if args['display'] is None:
print('Missing display')
if args['update']:
if args['get'] is None and args['create'] is None and args['display'] is None and args['message'] is None and args['delete'] is None and args['rm'] is None and args['remove'] is None:
print(args['update'])
else:
print('Dont mix update with others')
exit()
if args['file'] is None:
print('Missing file')
if args['delete']:
if args['get'] is None and args['create'] is None and args['display'] is None and args['message'] is None and args['file'] is None and args['update'] is None and args['changed'] is None and args['rm'] is None and args['remove'] is None:
print(args['delete'])
else:
print('Dont mix delete with others')
exit()
if args['rm']:
if args['get'] is None and args['create'] is None and args['display'] is None and args['message'] is None and args['file'] is None and args['update'] is None and args['changed'] is None and args['delete'] is None:
print(args['rm'])
else:
print('Dont mix resource management with others')
exit()
if args['remove'] is None:
print('Missing remove')
Is there any way to make it more pythonic ?
EDIT 1:
Why have I not included code using subparser ?
Because so far I have understood, subparsers don't themselves take any values. So in my case I want to be able to execute the script like this -
prog -g xxyy
prog -c xxyy -d 'Hello World'
prog -u xxyy -f 'file.md' -cd 'Foo bar baz'
Where as using subparsers, they would become something like (which I don't want, correct me if I am wrong)-
prog get -g xxyy
prog create -c xxyy -d 'Hello World'
EDIT 2
I figured out how to have mutually exclusive arguements using add_mutually_exclusive_group
parser = argparse.ArgumentParser(description='Mutex')
group = parser.add_mutually_exclusive_group()
group.add_argument('-g')
group.add_argument('-c')
group.add_argument('-u')
group.add_argument('-del')
group.add_argument('-rm')
EDIT 3
I am not able to get the subparse work. The following code (working with a subset) throws the error error: invalid choice: 'world' (choose from '-m', '-f') if the command is $ python3 parse2.py -c hello -m world
parser = argparse.ArgumentParser(description='Mutex')
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-g')
group.add_argument('-c')
subparser = parser.add_subparsers()
parser_a = subparser.add_parser('-m')
parser_b = subparser.add_parser('-f')
args = parser.parse_args()
print(args)
EDIT 4
I have almost solved my issue with the following -
import argparse
parser = argparse.ArgumentParser()
sp = parser.add_subparsers(dest='main') # , required=True) in Py3.7
sp.required = True # in py 3.6
p1 = sp.add_parser('get')
p1.add_argument('id')
p2 = sp.add_parser('update')
p2.add_argument('id')
p2.add_argument('-f', required=True)
p2.add_argument('-cd')
p3 = sp.add_parser('create')
p3.add_argument('name')
p3.add_argument('-d', required=True)
p3_mutex = p3.add_mutually_exclusive_group()
p3_mutex.add_argument('-f')
p3_mutex.add_argument('-m')
p4 = sp.add_parser('delete')
p4.add_argument('id')
p5 = sp.add_parser('remove')
p5.add_argument('id')
p5.add_argument('-r', required=True)
args = parser.parse_args()
print(args)
However I want to know if it is possible to add the add_mutually_exclusive_group for get, delete, create, update and remove or do I have to do that using if conditions?

A start of a subparser version:
import argparse
parser = argparse.ArgumentParser()
sp = parser.add_subparsers(dest='main') # , required=True) in Py3.7
sp.required = True # in py 3.6
p1 = sp.add_parser('get')
p1.add_argument('xxyy')
p2 = sp.add_parser('update')
p2.add_argument('xxyy')
p2.add_argument('-r', required=True)
p2.add_argument('--cd')
# and so forth
args = parser.parse_args()
print(args)
sample runs
0914:~/mypy$ python3 stack53307678.py -h
usage: stack53307678.py [-h] {get,update} ...
positional arguments:
{get,update}
optional arguments:
-h, --help show this help message and exit
0914:~/mypy$ python3 stack53307678.py get abc
Namespace(main='get', xxyy='abc')
0914:~/mypy$ python3 stack53307678.py update abc -r foo --cd x
Namespace(cd='x', main='update', r='foo', xxyy='abc')

python argparse doesn't take string option start with "--"

I simplifying my problem here. I need to do this:
python test.py --arg1 '--no-route53'
I added the parser like this:
parser.add_argument("--arg1", nargs="?", type=str, help="option")
args = parser.parse_args()
I wanted to get the '--no-route53' as a whole string and use it later in my script. But I keep getting this error:
test.py: error: unrecognized arguments: --no-route53
How can I work around it?
UPDATE1: if i give extra space after '--no-route53', like this and it worked:
python test.py --arg1 '--no-route53 '

I had the exact same issue a while ago. I ended up pre-parsing the sys.argvlist:
def bug_workaround(argv):
# arg needs to be prepended by space in order to not be interpreted as
# an option
add_space = False
args = []
for arg in argv[1:]: # Skip argv[0] as that should not be passed to parse_args
if add_space:
arg = " " + arg
add_space = True if arg == "--args" else False
args.append(arg)
return args
parser = argparse.ArgumentParser(description='My progrm.')
.
.
.
parser.add_argument('--args', type=str,
help='Extra args (in "quotes")')
args = parser.parse_args(bug_workaround(sys.argv))
# Prune added whitespace (for bug workaround)
if args.args:
args.args = args.args[1:]

argparse -- requiring either 2 values or none for an optional argument

I'm trying to make an optional argument for a script that can either take no values or 2 values, nothing else. Can you accomplish this using argparse?
# desired output:
# ./script.py -a --> works
# ./script.py -a val1 --> error
# ./script.py -a val1 val2 --> works
version 1 -- accepts 0 or 1 values:
parser = argparse.ArgumentParser()
parser.add_argument("-a", "--action", nargs="?", const=True, action="store", help="do some action")
args = parser.parse_args()
# output:
# ./script.py -a --> works
# ./script.py -a val1 --> works
# ./script.py -a val1 val2 --> error
version 2 - accepts exactly 2 values:
parser = argparse.ArgumentParser()
parser.add_argument("-a", "--action", nargs=2, action="store", help="do some action")
args = parser.parse_args()
# output:
# ./script.py -a --> error
# ./script.py -a val1 --> error
# ./script.py -a val1 val2 --> works
How do you combine these 2 different versions so that the script accepts 0 or 2 values for the argument, but rejects it when it only has 1 value?

You'll have to do your own error checking here. Accept 0 or more value, and reject anything other than 0 or 2:
parser = argparse.ArgumentParser()
parser.add_argument("-a", "--action", nargs='*', action="store", help="do some action")
args = parser.parse_args()
if args.action is not None and len(args.action) not in (0, 2):
parser.error('Either give no values for action, or two, not {}.'.format(len(args.action)))
Note that args.action is set to None when no -a switch was used:
>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument("-a", "--action", nargs='*', action="store", help="do some action")
_StoreAction(option_strings=['-a', '--action'], dest='action', nargs='*', const=None, default=None, type=None, choices=None, help='do some action', metavar=None)
>>> args = parser.parse_args([])
>>> args.action is None
True
>>> args = parser.parse_args(['-a'])
>>> args.action
[]

Just handle that case yourself:
parser.add_argument("-a", "--action", nargs='*', action="store", help="do some action")
args = parser.parse_args()
if args.action is not None:
if len(args.action) not in (0, 2):
parser.error('Specify no or two actions')
# action was specified but either there were two actions or no action
else:
# action was not specified
Of course you should update the help text in that case so that the user has a chance to know this before running into the error.

Have your option take a single, optional comma-separated string. You'll use a custom type to convert that string to a list and verify that it has exactly two items.
def pair(value):
rv = value.split(',')
if len(rv) != 2:
raise argparse.ArgumentParser()
return rv
parser.add_argument("-a", "--action", nargs='?',
type=pair, metavar='val1,val2',
help="do some action")
print parser.parse_args()
Then you'd use it like
$ ./script.py -a
Namespace(action=None)
$ ./script.py -a val1,val2
Namespace(action=['val1','val2'])
$ ./script.py -a val1
usage: tmp.py [-h] [-a [ACTION]]
script.py: error: argument -a/--action: invalid pair value: 'val1'
$ ./script.py -a val1,val2,val3
usage: tmp.py [-h] [-a [ACTION]]
script.py: error: argument -a/--action: invalid pair value: 'val1,val2,val3'
You can adjust the definition of pair to use a different separator and to return something other than a list (a tuple, for example).
The metavar provides a better indication that the argument to action is a pair of values, rather than just one.
$ ./script.py -h
usage: script.py [-h] [-a [val1,val2]]
optional arguments:
-h, --help show this help message and exit
-a [val1,val2], --action [val1,val2]
do some action

How about the required argument:parser.add_argument("-a", "--action", nargs=2, action="store", help="do some action", required=False)

Python argparse value range help message appearance

I have an argument for a program that is an integer from 1-100 and I just don't like the way that it shows up in the -h help message when using argparse (it literally lists 0, 1, 2, 3, 4, 5,... etc)
Is there any way to change this or have it represented in another way?
Thanks
EDIT:
Here is the code for those who asked:
norse = parser.add_argument_group('Norse')
norse.add_argument('-n', '--norse', required=False, help='Run the Norse IPViking scan.', action='store_true')
norse.add_argument('--threshold', required=False, type=int, choices=range(0,101), help='Threshold (0-100) denoting at what threat level to provide additional data on an IP \
address. Default is 49.', default=49)

Use the metavar parameter of add_argument().
For example:
norse = parser.add_argument_group('Norse')
norse.add_argument('-n', '--norse', required=False, help='Run the Norse IPViking scan.', action='store_true')
norse.add_argument('--threshold', required=False, type=int, choices=range(0,101),
metavar="[0-100]",
help='Threshold (0-100) denoting at what threat level to provide additional data on an IP \
address. Default is 49.', default=49)
Test:
from argparse import ArgumentParser
norse = ArgumentParser()
norse.add_argument('-n', '--norse', required=False, help='Run the Norse IPViking scan.', action='store_true')
norse.add_argument('--threshold', required=False, type=int, choices=range(0,101), metavar="[0-100]", help='Threshold (0-100) denoting at what threat level to provide additional data on an IP address. Default is 49.', default=49)
norse.print_help()
Results
usage: -c [-h] [-n] [--threshold [0-100]]
optional arguments:
-h, --help show this help message and exit
-n, --norse Run the Norse IPViking scan.
--threshold [0-100] Threshold (0-100) denoting at what threat level to
provide additional data on an IP address. Default is
49.

You can customize action, e.g:
#!/usr/bin/env python
import argparse
class Range(argparse.Action):
def __init__(self, minimum=None, maximum=None, *args, **kwargs):
self.min = minimum
self.max = maximum
kwargs["metavar"] = "[%d-%d]" % (self.min, self.max)
super(Range, self).__init__(*args, **kwargs)
def __call__(self, parser, namespace, value, option_string=None):
if not (self.min <= value <= self.max):
msg = 'invalid choice: %r (choose from [%d-%d])' % \
(value, self.min, self.max)
raise argparse.ArgumentError(self, msg)
setattr(namespace, self.dest, value)
norse = argparse.ArgumentParser('Norse')
norse.add_argument('--threshold', required=False, type=int, min=0, max=100,
action=Range,
help='Threshold [%(min)d-%(max)d] denoting at what threat \
level to provide additional data on an IP address. \
Default is %(default)s.', default=49)
args = norse.parse_args()
print args
Test it:
~: user$ ./test.py --threshold 10
Namespace(threshold=10)
~: user$ ./test.py --threshold -1
usage: Norse [-h] [--threshold [0-100]]
Norse: error: argument --threshold: invalid choice: -1 (choose from [0-100])
~: user$ ./test.py -h
usage: Norse [-h] [--threshold [0-100]]
optional arguments:
-h, --help show this help message and exit
--threshold [0-100] Threshold [0-100] denoting at what threat level to
provide additional data on an IP address. Default is
49.

With a custom type, it is easier to control the error message (via the ArgumentTypeError). I still need the metavar to control the usage display.
import argparse
def range_type(astr, min=0, max=101):
value = int(astr)
if min<= value <= max:
return value
else:
raise argparse.ArgumentTypeError('value not in range %s-%s'%(min,max))
parser = argparse.ArgumentParser()
norse = parser.add_argument_group('Norse')
...
norse.add_argument('--range', type=range_type,
help='Value in range: Default is %(default)s.',
default=49, metavar='[0-101]')
parser.print_help()
print parser.parse_args()
producing:
2244:~/mypy$ python2.7 stack25295487.py --ran 102
usage: stack25295487.py [-h] [-n] [--threshold [0:101]] [--range [0-101]]
optional arguments:
-h, --help show this help message and exit
Norse:
...
--range [0-101] Value in range: Default is 49.
usage: stack25295487.py [-h] [-n] [--threshold [0:101]] [--range [0-101]]
stack25295487.py: error: argument --range: value not in range 0-101
I could use functools.partial to customize the range values:
type=partial(range_type, min=10, max=90)

Here are a couple ways you can do it instead
def parseCommandArgs():
parser = argparse.ArgumentParser()
parser.add_argument('-i', dest='myDest', choices=range(1,101), type=int, required=True, metavar='INT[1,100]', help='my help message')
return parser.parse_args()
You can also use action instead, which I highly recommend since it allows more customization
def verify():
class Validity(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
if values < 1 or values > 100:
# do something
pass
return Validity
def parseCommandArgs():
parser = argparse.ArgumentParser()
parser.add_argument('-i', dest='myDest', required=True, metavar='INT[1,100]', help='my help message', action=verify())
return parser.parse_args()

I have written a library for this: argparse-range
>>> from argparse import ArgumentParser
>>> from argparse_range import range_action
>>> parser = ArgumentParser()
>>> parser.add_argument("rangedarg", action=range_action(0, 10), help="An argument")
>>> args = parser.parse_args(["0"])
>>> args.rangedarg
0
>>> parser.parse_args(["20"])
Traceback (most recent call last):
....
argparse.ArgumentTypeError: Invalid choice: 20 (must be in range 0..=10)
Helptext is added transparently:
foo.py --help
usage: foo.py [-h] rangedarg
positional arguments:
rangedarg An argument (must be in range 0..=10)
optional arguments:
-h, --help show this help message and exit
Other features include:
Handles default, metavar, and nargs consistently with default argparse handling
Type inference, with explicit overrides using the type argument

Control formatting of the argparse help argument list?

import argparse
parser = argparse.ArgumentParser(prog='tool')
args = [('-u', '--upf', 'ref. upf', dict(required='True')),
('-s', '--skew', 'ref. skew', {}),
('-m', '--model', 'ref. model', {})]
for args1, args2, desc, options in args:
parser.add_argument(args1, args2, help=desc, **options)
parser.print_help()
Output:
usage: capcheck [-h] -u UPF [-s SKEW] [-m MODEL]
optional arguments:
-h, --help show this help message and exit
-u UPF, --upf UPF ref. upf
-s SKEW, --skew SKEW ref. skew
-m MODEL, --model MODEL
ref. model
How do I print ref. model in the same line as -m MODEL, --model MODEL instead of that appearing on a separate line when I run the script with -h option?

You could supply formatter_class argument:
parser = argparse.ArgumentParser(prog='tool',
formatter_class=lambda prog: argparse.HelpFormatter(prog,max_help_position=27))
args = [('-u', '--upf', 'ref. upf', dict(required='True')),
('-s', '--skew', 'ref. skew', {}),
('-m', '--model', 'ref. model', {})]
for args1, args2, desc, options in args:
parser.add_argument(args1, args2, help=desc, **options)
parser.print_help()
Note: Implementation of argparse.HelpFormatter is private only the name is public. Therefore the code might stop working in future versions of argparse. File a feature request to provide a public interface for the customization of max_help_position on http://bugs.python.org/
Output
usage: tool [-h] -u UPF [-s SKEW] [-m MODEL]
optional arguments:
-h, --help show this help message and exit
-u UPF, --upf UPF ref. upf
-s SKEW, --skew SKEW ref. skew
-m MODEL, --model MODEL ref. model

Inspired by #jfs's answer, I have come up with this solution:
def make_wide(formatter, w=120, h=36):
"""Return a wider HelpFormatter, if possible."""
try:
# https://stackoverflow.com/a/5464440
# beware: "Only the name of this class is considered a public API."
kwargs = {'width': w, 'max_help_position': h}
formatter(None, **kwargs)
return lambda prog: formatter(prog, **kwargs)
except TypeError:
warnings.warn("argparse help formatter failed, falling back.")
return formatter
Having that, you can call it with any HelpFormatter that you like:
parser = argparse.ArgumentParser(
formatter_class=make_wide(argparse.ArgumentDefaultsHelpFormatter)
)
or
parser = argparse.ArgumentParser(
formatter_class=make_wide(argparse.HelpFormatter, w=140, h=20)
)
What this does is make sure that the wider formatter can actually be created using the width and max_help_position arguments. If the private API changes, that is noted by make_wide by a TypeError and the formatter is returned unchanged. That should make the code more reliable for deployed applications.
I'd welcome any suggestions to make this more pythonic.

If you are providing a custom formatter_class to your ArgumentParser
parser = argparse.ArgumentParser(formatter_class=help_formatter)
and then use subparsers, the formatter will only apply to the top-level help message. In order to use the same (or some other) formatter for all subparsers, you need to provide formatter_class argument for each add_parser call:
subparsers = parser.add_subparsers(metavar="ACTION", dest="action")
child_parser = subparsers.add_parser(
action_name, formatter_class=help_formatter
)

As the argparse library tries to use the COLUMNS environment variable to get the terminal width, we also can set this variable, and let argparse do its job.
import os
import argparse
rows, columns = os.popen('stty size', 'r').read().split()
os.environ["COLUMNS"] = str(columns)
parser = argparse.ArgumentParser(etc...
Tested and approved on RHEL/Python 2.7.5
Credits to https://stackoverflow.com/a/943921 for getting the real terminal width

Another approach: hijack sys.argv, check it for --help and -h, if found extract help text using argparse.format_help, massage it, print it, and exit.
import sys, re, argparse
RGX_MID_WS = re.compile(r'(\S)\s{2,}')
def main(argv):
# note add_help = False
parser = argparse.ArgumentParser(description = '%(prog)s: testing help mods', formatter_class= argparse.RawTextHelpFormatter, add_help = False)
parser.add_argument('bar', nargs='+', help='two bars that need to be frobbled')
parser.add_argument('--foo', action='store_true', help='foo the bars before frobbling\nfoo the bars before frobbling')
parser.add_argument('--xxxxx', nargs=2, help='many xes')
parser.add_argument('--bacon', help ='a striped food')
parser.add_argument('--badger', help='in a striped pyjamas')
parser.add_argument('--animal', dest='animal', choices=('zabra', 'donkey', 'bat') ,help ='could be one of these')
# may exit
lArgs = help_manage(parser)
args = parser.parse_args() # args = lArgs
print('bars are: ', args.bar)
def help_manage(parser):
"""
check for -h, --help, -h in a single-letter cluster;
if none found, return, otherwise clean up help text and exit
"""
lArgs = sys.argv[1:]
lArgsNoHelp = [sOpt for sOpt in lArgs if (not sOpt in ('--help', '-h')) and not (sOpt[0] == '-' and sOpt[1] != '-' and 'h' in sOpt)]
# no change? then no --help params
if len(lArgsNoHelp) == len(lArgs): return
sHelp = parser.format_help()
# to see help as formated by argparse, uncomment:
# print(sHelp)
# exit()
for sLine in sHelp.split('\n'): print(clean_line(sLine))
exit()
def clean_line(sLine):
"""
this is just an example, and goes nowhere near covering all possible
argument properties
"""
# avoid messing with usage: lines
if 'usage' in sLine: return sLine
if sLine.startswith(' ') and '[' in sLine: return sLine
if sLine.endswith(' arguments:'): return sLine + '\n'
sLine = sLine.lstrip()
sLine = RGX_MID_WS.sub(r'\1\n', sLine)
if sLine.startswith('-'): sLine = '\n' + sLine
return sLine.replace('{', '\n(can be: ').replace('}', ')').replace('\n\n', '\n')
if __name__ == '__main__':
bRes = main(sys.argv[1:])
sys.exit(bRes)
Help without formatting:
usage: argparse_fix_min2.py [--foo] [--xxxxx XXXXX XXXXX] [--bacon BACON]
[--badger BADGER] [--animal {zabra,donkey,bat}]
bar [bar ...]
argparse_fix_min2.py: testing help mods
positional arguments:
bar two bars that need to be frobbled
optional arguments:
--foo foo the bars before frobbling
foo the bars before frobbling
--xxxxx XXXXX XXXXX many xes
--bacon BACON a striped food
--badger BADGER in a striped pyjamas
--animal {zabra,donkey,bat}
could be one of these
with formatting:
usage: argparse_fix_min2.py [--foo] [--xxxxx XXXXX XXXXX] [--bacon BACON]
[--badger BADGER] [--animal {zabra,donkey,bat}]
bar [bar ...]
argparse_fix_min2.py: testing help mods
positional arguments:
bar
two bars that need to be frobbled
optional arguments:
--foo
foo the bars before frobbling
foo the bars before frobbling
--xxxxx XXXXX XXXXX
many xes
--bacon BACON
a striped food
--badger BADGER
in a striped pyjamas
--animal
(can be: zabra,donkey,bat)
could be one of these
"""