We Keep Coding

argparse: How to map choices to different values?

When using argparse, I would like a user to select from choices, but I would like their choice to determine a more complex value (similar to how store_const works).
For example, when choosing from a smoker status of ['current', 'former', 'never'], I would like current to map to 'Current every day smoker.'
Is there an elegant way to do this?
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--smoker', choices=['current','former','never'])
print(vars(parser.parse_args()))
Normal output:
$ ./script.py --smoker current
{'smoker': 'current'}
Desired:
$ ./script.py --smoker current
{'smoker': 'Current every day smoker.'}
I thought I could do this with a lambda type argument, but argparse enforces the list of choices:
choice_dict = {'current': 'Current everyday...'}
parser.add_argument('--smoker', type=lambda x: choice_dict[x], choices=choice_dict.keys())
print(vars(parser.parse_args()))
./script.py --smoker current
usage: script.py [-h] [--smoker {current}]
script.py: error: argument --smoker: invalid choice: 'Current everyday...' (choose from 'current')

Though I agree with 0x5452 comment that it is better to decouple this formatting from the parser, you can use an Action to do what you want:
import argparse
CONVERSION_TABLE = {'current': 'Currently smokes',
'former': 'Used to smoke',
'never': 'Never smoke'}
class RenameOption(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
setattr(namespace, self.dest, CONVERSION_TABLE[values])
parser = argparse.ArgumentParser()
parser.add_argument('--smoker', action=RenameOption, choices=CONVERSION_TABLE)
print(vars(parser.parse_args()))
The result is:
$ python test.py --smoker current
{'smoker': 'Currently smokes'}
$ python test.py --smoker no
usage: test.py [-h] [--smoker {current,former,never}]
test.py: error: argument --smoker: invalid choice: 'no' (choose from 'current', 'former', 'never')

Selected options activate right subparsers

I have to invoke my script in this way.
script.py multiple --ways aa bb -as abab -bs bebe
In this case -as abab reffers to "aa" option from --ways parameter and -bs bebe to "bb" option.
And all chosed options should affect what "fulfill" method will be used.
If we chose 'aa' and 'bb' there should only be options '-as' and '-bs' not '-cs'.
import sys
from argparse import ArgumentParser
def fulfill_aa_parser(aa_parser):
aa_parser.add_argument('--ass', '-as', type=str, required=True, choices=['abababa'])
def fulfill_bb_parser(aa_parser):
aa_parser.add_argument('--bass', '-bs', type=str, required=True, choices=['bebebe'])
def fulfill_cc_parser(aa_parser):
aa_parser.add_argument('--cass', '-cs', type=str, required=True, choices=['cycycyc'])
def fulfill_multiple_parser(multiple_parser):
multiple_parser.add_argument('--ways', '-w', type=str, choices=['aa','bb', 'cc'], nargs='+', required=True)
def main(argv):
parser = ArgumentParser(description='TEST CASE')
subparsers = parser.add_subparsers(dest='type')
multiple_parser = subparsers.add_parser(
'multiple'
)
aabbparsers = multiple_parser.add_subparsers()
aa_parser = aabbparsers.add_parser('aa')
bb_parser = aabbparsers.add_parser('bb')
cc_parser = aabbparsers.add_parser('cc')
fulfill_multiple_parser(multiple_parser)
fulfill_aa_parser(aa_parser)
fulfill_bb_parser(bb_parser)
fulfill_cc_parser(cc_parser)
args = parser.parse_args(argv)
if args.type is None:
parser.print_help()
return
if __name__ == '__main__':
main(sys.argv[1:])
Parsing this in this way:
fulfill_aa_parser(multiple_parser)
fulfill_bb_parser(multiple_parser)
fulfill_cc_parser(multiple_parser)
will lead to parser always asking for '-as', '-bs' ,'-cs' and options in '--ways' will not affect this
EDIT : \
This is it looks when there is some thought put to it.
Just simply pass parser to this function
def fulfill_apple_argparser(parser):
parser.add_argument("--apple_argument")
def fulfill_banana_argparser(parser):
parser.add_argument("--banana_argument")
def fulfill_peach_argparser(parser):
parser.add_argument("--peach_argument")
def many_fruits_parse(parser, progs=None, list_of_fruits=('apple', 'banana', 'peach')):
progs = progs or []
if len(list_of_fruits) == 0 or parser in progs:
return
fulfill = {'apple': fulfill_apple_argparser, 'banana': fulfill_banana_argparser,
'peach': fulfill_peach_argparser}
subparsers = parser.add_subparsers(title='subparser', dest=parser.prog)
progs.append(parser)
for fruit in list_of_fruits:
secondary = [x for x in list_of_fruits if x != fruit]
fruit_parser = subparsers.add_parser(fruit, help=fruit)
fulfill[fruit](fruit_parser)
many_fruits_parse(fruit_parser, progs, secondary)

add_subparsers creates a special kind of positional argument, one that uses the add_parser command to create choices. Once a valid choice is provided, parsing is passed to that parser.
With
script.py multiple --ways aa bb -as abab -bs bebe
parser passes the task to multiple_parser. The --ways optional then gets 2 values
Namespace(ways=['aa','bb'])
Neither of those strings is used as a value for aabbparsers, and multiple_parser doesn't know what to do with '-as` or '-bs', and (I expect) will raise an error.
With:
script.py multiple aa -as abab
parsing is passed from parser to multiple_parser to aa_parser, which in turn handles '-as abab', producing (I think)
Namespace(as='abab')
Nesting as you do with multiple and aa is the only way to use multiple subparsers. You can't have two subparses 'in-parallel' (e.g. 'aa' and 'bb').
Especially when testing it's a good idea to provide a dest to the add_subparsers command. It gives information on which subparsers is being invoked.

Argparse arguments from file and string CLI arguments both

I want to launch my script like this:
python3 main.py #params.conf 1 2
where params.conf is a file and 1, 2 are string arguments.
I know how to parse file alone:
argparser = ArgumentParser()
argparser.add_argument('arg1', help='heeelp')
...
args = argparser.parse_args()
But how to parse following arguments?

An argument prefixed with # is treated as if its contents were in the command line directly, one argument per line. So if the contents of params.confis
2
3
And you define a parser like
import argparse
p = argparse.ArgumentParser(fromfile_prefix_chars='#')
p.add_argument("a")
p.add_argument("b")
p.add_argument("c")
p.add_argument("d")
args = p.parse_args()
and you call your script as
script.py 1 #params.conf 4
then your arguments a through d will be set to 1 through 4, respectively.

You just add more argparser.add_argument calls.
Like this:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('arg1', type=str)
parser.add_argument('arg2', type=str)
parser.add_argument('arg3', type=str)
args = parser.parse_args()
print(args) # arguments are parsed

Defining different forms of argparse argument inputs

I have just started out in using argparse, my code works but I am trying to parse in different values as to how the user types in the argument.
Currently this is my code:
def setup_args():
"""
Set up args for the tool
"""
parser = argparse.ArgumentParser(
description=("Get all file versions of a status in a project"),
formatter_class=argparse.RawDescriptionHelpFormatter)
# Positional Arguments
parser.add_argument('project',
type=str,
help='Name of the to look into')
parser.add_argument('status',
type=str,
help='Define which status to look into')
# Optional Arguments
parser.add_argument('-o',
'--output',
action='store_true',
help='Write to output to text file if used')
if __name__ == "__main__":
args = setup_args()
# Check the status set
status_list = ['Pending', 'Work in Progress', 'Approved', 'Rejected']
if not args.status in site_list:
raise ValueError("Please input one of the status : 'Pending', 'Work in Progress', 'Approved', 'Rejected'")
output_query(args.project, status, args.client, args.output)
As you can see in my main.. It only register those case-sensitive status names that I have defined.
Are there any ways in which I can also make my code to register if they are typed in small caps - 'pending', 'work in progress', 'approved', 'rejected' or in short forms - 'p', 'wip', 'a', 'r'?
One of the way I can implement is using if..elif..
if args.client == ('pending' or 'p'):
args.client = 'Pending'
elif args.client == ('work in progress' or 'wip'):
args.client = 'Work in Progress'
elif args.client == ('approved' or 'a'):
args.client = 'Approved'
elif args.client == ('rejected' or 'r'):
args.client = 'Rejected'
Though it works, it looks a bit 'long-winded' to me. If I have multiple arguments, this would means I will need to put in a lot of if...elif... which may not be practical unless this is the only way.
Is there a better solution to get around this?
EDIT:
This is how I have been running my command : python prog.py my_project Pending
but I am thinking of scenarios where one could type it this way : python prog.py my_project pending or python prog.py my_project p, notice that the caps P has become small letter..

You could generalize the status check with .lower() and restricting the number of characters that you check.
For example, if I define an abbreviated list of 'choices', I can test anything that looks like the big names with:
In [239]: choices = ['pend', 'work', 'appr','reje']
In [240]: status_list = ['Pending', 'Work in Progress', 'Approved', 'Rejected']
In [241]: for wd in status_list:
...: if wd.lower()[:4] in choices:
...: print(wd)
...:
Pending
Work in Progress
Approved
Rejected
You probably shouldn't expect your user to enter the full 'Work in Progress' string. To do so would require quoting. Otherwise the shell will break that into 3 strings.
A variation on this test uses startswith:
for wd in status_list:
if any([wd.lower().startswith(n) for n in choices]):
print(wd)
You could also let the parser do value checking
parser.add_argument('status',
# type=str, # default, don't need to add it
choices = ['pending', 'work', 'approved', 'rejected'],
help='Define which status to look into')
That generates a nice error message if the string doesn't match. And it incorporates the choices into the help. Try it and see what happens.
The disadvantage is that it doesn't allow abbreviations or upper/lower case. (A custom type function can get around those restrictions, but that's a more advanced technique).
==================
A way to use type is to define a little function:
def abrev(astr):
return astr.lower()[:4]
which works in the above test:
for wd in status_list:
if abrev(wd) in choices:
print(wd)
In a parser it can be used as:
In [253]: p = argparse.ArgumentParser()
In [254]: p.add_argument('status', type=abrev, choices=choices);
In [255]: p.print_help()
usage: ipython3 [-h] {pend,work,appr,reje}
positional arguments:
{pend,work,appr,reje}
optional arguments:
-h, --help show this help message and exit
Sample calls:
In [256]: p.parse_args(['Work'])
Out[256]: Namespace(status='work')
In [257]: p.parse_args(['status'])
usage: ipython3 [-h] {pend,work,appr,reje}
ipython3: error: argument status: invalid choice: 'stat' (choose from 'pend', 'work', 'appr', 'reje')
...
In [258]: p.parse_args(['reject'])
Out[258]: Namespace(status='reje')
In [259]: p.parse_args(['Pending'])
Out[259]: Namespace(status='pend')

Though I haven't tested it, looking at the add_argument function documentation, you should be able to add them as the first arguments to it.
name or flags - Either a name or a list of option strings, e.g. foo or -f, --foo.
https://docs.python.org/3/library/argparse.html#the-add-argument-method

Control formatting of the argparse help argument list?

import argparse
parser = argparse.ArgumentParser(prog='tool')
args = [('-u', '--upf', 'ref. upf', dict(required='True')),
('-s', '--skew', 'ref. skew', {}),
('-m', '--model', 'ref. model', {})]
for args1, args2, desc, options in args:
parser.add_argument(args1, args2, help=desc, **options)
parser.print_help()
Output:
usage: capcheck [-h] -u UPF [-s SKEW] [-m MODEL]
optional arguments:
-h, --help show this help message and exit
-u UPF, --upf UPF ref. upf
-s SKEW, --skew SKEW ref. skew
-m MODEL, --model MODEL
ref. model
How do I print ref. model in the same line as -m MODEL, --model MODEL instead of that appearing on a separate line when I run the script with -h option?

You could supply formatter_class argument:
parser = argparse.ArgumentParser(prog='tool',
formatter_class=lambda prog: argparse.HelpFormatter(prog,max_help_position=27))
args = [('-u', '--upf', 'ref. upf', dict(required='True')),
('-s', '--skew', 'ref. skew', {}),
('-m', '--model', 'ref. model', {})]
for args1, args2, desc, options in args:
parser.add_argument(args1, args2, help=desc, **options)
parser.print_help()
Note: Implementation of argparse.HelpFormatter is private only the name is public. Therefore the code might stop working in future versions of argparse. File a feature request to provide a public interface for the customization of max_help_position on http://bugs.python.org/
Output
usage: tool [-h] -u UPF [-s SKEW] [-m MODEL]
optional arguments:
-h, --help show this help message and exit
-u UPF, --upf UPF ref. upf
-s SKEW, --skew SKEW ref. skew
-m MODEL, --model MODEL ref. model

Inspired by #jfs's answer, I have come up with this solution:
def make_wide(formatter, w=120, h=36):
"""Return a wider HelpFormatter, if possible."""
try:
# https://stackoverflow.com/a/5464440
# beware: "Only the name of this class is considered a public API."
kwargs = {'width': w, 'max_help_position': h}
formatter(None, **kwargs)
return lambda prog: formatter(prog, **kwargs)
except TypeError:
warnings.warn("argparse help formatter failed, falling back.")
return formatter
Having that, you can call it with any HelpFormatter that you like:
parser = argparse.ArgumentParser(
formatter_class=make_wide(argparse.ArgumentDefaultsHelpFormatter)
)
or
parser = argparse.ArgumentParser(
formatter_class=make_wide(argparse.HelpFormatter, w=140, h=20)
)
What this does is make sure that the wider formatter can actually be created using the width and max_help_position arguments. If the private API changes, that is noted by make_wide by a TypeError and the formatter is returned unchanged. That should make the code more reliable for deployed applications.
I'd welcome any suggestions to make this more pythonic.

If you are providing a custom formatter_class to your ArgumentParser
parser = argparse.ArgumentParser(formatter_class=help_formatter)
and then use subparsers, the formatter will only apply to the top-level help message. In order to use the same (or some other) formatter for all subparsers, you need to provide formatter_class argument for each add_parser call:
subparsers = parser.add_subparsers(metavar="ACTION", dest="action")
child_parser = subparsers.add_parser(
action_name, formatter_class=help_formatter
)

As the argparse library tries to use the COLUMNS environment variable to get the terminal width, we also can set this variable, and let argparse do its job.
import os
import argparse
rows, columns = os.popen('stty size', 'r').read().split()
os.environ["COLUMNS"] = str(columns)
parser = argparse.ArgumentParser(etc...
Tested and approved on RHEL/Python 2.7.5
Credits to https://stackoverflow.com/a/943921 for getting the real terminal width

Another approach: hijack sys.argv, check it for --help and -h, if found extract help text using argparse.format_help, massage it, print it, and exit.
import sys, re, argparse
RGX_MID_WS = re.compile(r'(\S)\s{2,}')
def main(argv):
# note add_help = False
parser = argparse.ArgumentParser(description = '%(prog)s: testing help mods', formatter_class= argparse.RawTextHelpFormatter, add_help = False)
parser.add_argument('bar', nargs='+', help='two bars that need to be frobbled')
parser.add_argument('--foo', action='store_true', help='foo the bars before frobbling\nfoo the bars before frobbling')
parser.add_argument('--xxxxx', nargs=2, help='many xes')
parser.add_argument('--bacon', help ='a striped food')
parser.add_argument('--badger', help='in a striped pyjamas')
parser.add_argument('--animal', dest='animal', choices=('zabra', 'donkey', 'bat') ,help ='could be one of these')
# may exit
lArgs = help_manage(parser)
args = parser.parse_args() # args = lArgs
print('bars are: ', args.bar)
def help_manage(parser):
"""
check for -h, --help, -h in a single-letter cluster;
if none found, return, otherwise clean up help text and exit
"""
lArgs = sys.argv[1:]
lArgsNoHelp = [sOpt for sOpt in lArgs if (not sOpt in ('--help', '-h')) and not (sOpt[0] == '-' and sOpt[1] != '-' and 'h' in sOpt)]
# no change? then no --help params
if len(lArgsNoHelp) == len(lArgs): return
sHelp = parser.format_help()
# to see help as formated by argparse, uncomment:
# print(sHelp)
# exit()
for sLine in sHelp.split('\n'): print(clean_line(sLine))
exit()
def clean_line(sLine):
"""
this is just an example, and goes nowhere near covering all possible
argument properties
"""
# avoid messing with usage: lines
if 'usage' in sLine: return sLine
if sLine.startswith(' ') and '[' in sLine: return sLine
if sLine.endswith(' arguments:'): return sLine + '\n'
sLine = sLine.lstrip()
sLine = RGX_MID_WS.sub(r'\1\n', sLine)
if sLine.startswith('-'): sLine = '\n' + sLine
return sLine.replace('{', '\n(can be: ').replace('}', ')').replace('\n\n', '\n')
if __name__ == '__main__':
bRes = main(sys.argv[1:])
sys.exit(bRes)
Help without formatting:
usage: argparse_fix_min2.py [--foo] [--xxxxx XXXXX XXXXX] [--bacon BACON]
[--badger BADGER] [--animal {zabra,donkey,bat}]
bar [bar ...]
argparse_fix_min2.py: testing help mods
positional arguments:
bar two bars that need to be frobbled
optional arguments:
--foo foo the bars before frobbling
foo the bars before frobbling
--xxxxx XXXXX XXXXX many xes
--bacon BACON a striped food
--badger BADGER in a striped pyjamas
--animal {zabra,donkey,bat}
could be one of these
with formatting:
usage: argparse_fix_min2.py [--foo] [--xxxxx XXXXX XXXXX] [--bacon BACON]
[--badger BADGER] [--animal {zabra,donkey,bat}]
bar [bar ...]
argparse_fix_min2.py: testing help mods
positional arguments:
bar
two bars that need to be frobbled
optional arguments:
--foo
foo the bars before frobbling
foo the bars before frobbling
--xxxxx XXXXX XXXXX
many xes
--bacon BACON
a striped food
--badger BADGER
in a striped pyjamas
--animal
(can be: zabra,donkey,bat)
could be one of these
"""