Argparse outputting help text twice - python

After an hour googling, I can't find anybody who has had anything resembling this issue besides myself. I created a command line interface with argparse. Originally I had tried to leverage argparse's built in help text behavior. But my boss isn't satisfied with the default help text, so he is having me write up the full usage/help text in a text file and just display the entire file.
For some reason, in a certain case, its outputting the text twice.
Here is the basics of how my program is broken down:
I have a top level parser. I read in my help text file, set it to a string help_text, and then set "usage=help_text" on the parser. Then I create subparsers (4 of them and then a base case) to create subcommands. Only one of those subparsers has any additional arguments (one positional, one optional). Before I reworked the help text, I had help text for each individual subcommand by using "help=" but now those are all blank. Lastly, I have set up a base case to display the help text whenever no subcommands are given.
Here is the behavior I'm getting:
When I call the main function with no subcommands and no arguments, my help_text from the text file outputs, and then like 2-3 additional lines of boiler plate I can't seem to get rid of. Also because the word usage appears in my text file, it says "usage: usage"
When I call the main command and then type --help, the exact same thing happens as above.
When I call the one subcommand that has a required positional argument and I don't include that argument... it spits out the entire help text twice. Right above the second time it prints, it prints the default usage line for that subcommand.
Lastly, when I use a different subcommand that has no arguments and give it an argument (one too many) it spits out everything completely correctly without even the extra couple lines at the end.
I don't know how to make heads or tales about this. Here is the main function of the script (I can verify that this problem occurs only in the main function where argparse is used, not the other functions that the main function calls):
def main():
# Import help text from file
p = Path(__file__).with_name("help_text.txt")
with p.open() as file:
help_text = file.read()
# Configure the top level Parser
parser = argparse.ArgumentParser(prog='hubmap-clt', description='Name of cli', usage=help_text)
subparsers = parser.add_subparsers()
# Create Subparsers to give subcommands
parser_transfer = subparsers.add_parser('subcommandone')
parser_transfer.add_argument('argument1', type=str)
parser_transfer.add_argument('--optionalargument', default='mydefault')
parser_login = subparsers.add_parser('subcommandtwo')
parser_whoami = subparsers.add_parser('subcommandthree')
parser_logout = subparsers.add_parser('subcommandfour')
# Assign subparsers to their respective functions
parser_subcommandone.set_defaults(func=subcommandone)
parser_subcommandtwo.set_defaults(func=subcommandtwo)
parser_subcommandthree.set_defaults(func=subcommandthree)
parser_subcommandfour.set_defaults(func=subcommandfour)
parser.set_defaults(func=base_case)
# Parse the arguments and call appropriate functions
args = parser.parse_args()
if len(sys.argv) == 1:
args.func(args, parser)
else:
args.func(args)
So to clarify:
Why does the extra couple lines of boiler-plat help text appear sometimes which looks like this:
name of cli
positional arguments:
{subcommandone,subcommandtwo,subcommandthree,subcommandfour}
optional arguments:
-h, --help show this help message and exit
Why does using subcommandone with too few arguments print out the help text twice (but NOT the extra lines of boiler-plate help text.
why does using subcommandtwo with one too MANY arguments print everything perfectly without any extra lines?

With a modification of your main:
def foo():
# Import help text from file
# p = Path(__file__).with_name("help_text.txt")
# with p.open() as file:
# help_text = file.read()
help_text = "cli usage: foobar\n morebar"
# Configure the top level Parser
parser = argparse.ArgumentParser(
prog="hubmap-clt", description="Name of cli", usage=help_text
)
subparsers = parser.add_subparsers()
# Create Subparsers to give subcommands
parser_transfer = subparsers.add_parser("subcommandone")
parser_transfer.add_argument("argument1", type=str)
parser_transfer.add_argument("--optionalargument", default="mydefault")
parser_login = subparsers.add_parser("subcommandtwo")
# parser_whoami = subparsers.add_parser("subcommandthree")
# parser_logout = subparsers.add_parser("subcommandfour")
# Assign subparsers to their respective functions
parser_transfer.set_defaults(func="subcommandone")
parser_login.set_defaults(func="subcommandtwo")
# parser_subcommandthree.set_defaults(func="subcommandthree")
# parser_subcommandfour.set_defaults(func="subcommandfour")
parser.set_defaults(func="base_case")
return parser
in an iteractive ipython session:
In [8]: p = foo()
In [9]: p.print_usage()
usage: cli usage: foobar
morebar
Usage is exactly as I specified. And the help for the main parser:
In [10]: p.print_help()
usage: cli usage: foobar
morebar
Name of cli
positional arguments:
{subcommandone,subcommandtwo}
optional arguments:
-h, --help show this help message and exit
That's what I expect given the arguments.
Help for a subparser:
In [11]: p.parse_args(["subcommandone", "-h"])
usage: cli usage: foobar
morebar subcommandone [-h] [--optionalargument OPTIONALARGUMENT] argument1
positional arguments:
argument1
optional arguments:
-h, --help show this help message and exit
--optionalargument OPTIONALARGUMENT
Usage is like the main's but with some added info on how to call this subparser and its arguments.
Error when calling the subparsers without enough values:
In [15]: p.parse_args(["subcommandone"])
usage: cli usage: foobar
morebar subcommandone [-h] [--optionalargument OPTIONALARGUMENT] argument1
cli usage: foobar
morebar subcommandone: error: the following arguments are required: argument1
Is this repeat of cli usage that bothering you? This error is raised by the subparser, and I suspect the extra comes from the prog of that subparser. I think I saw something like this on the Python bug/issues for argparse.
error with too much:
In [17]: p.parse_args(["subcommandone", "test", "extra"])
usage: cli usage: foobar
morebar
hubmap-clt: error: unrecognized arguments: extra
In this case error is produced by the main parser, hence the "hubmat-clt" prog.
change prog:
...: parser_transfer = subparsers.add_parser(
...: "subcommandone", prog="hubmap-clt sobcommandone"
...: )
In [21]: p.parse_args(["subcommandone", "test", "extra"])
usage: cli usage: foobar
morebar
hubmap-clt: error: unrecognized arguments: extra
In [22]: p.parse_args(["subcommandone"])
usage: hubmap-clt sobcommandone [-h] [--optionalargument OPTIONALARGUMENT] argument1
hubmap-clt sobcommandone: error: the following arguments are required: argument1
[21] is as before [17]. But [22] is now showing the prog that I set. I could also have specified a custom usage for the subparser.
If I modify the function to use default usage and prog, but also display the subparser's prog. And I gave the main an "main_foo" positional argument:
In [30]: p = foo()
hubmap-clt main_foo subcommandone
In [31]: p.parse_args(["subcommandone"])
Out[31]: Namespace(main_foo='subcommandone')
In [32]: p.parse_args(["foo", "subcommandone"])
usage: hubmap-clt main_foo subcommandone [-h] [--optionalargument OPTIONALARGUMENT] argument1
hubmap-clt main_foo subcommandone: error: the following arguments are required: argument1
Notice how the main's usage has been incorporated into the 'prog' for the subparser.
In the bug/issue I found the main parser's usage gets incorporated into the prog of the subparser. That's why you see the duplicate.
https://bugs.python.org/issue42297
[argparse] Bad error message formatting when using custom usage text
The relatively recent date of this bug issue indicates that custom usage is not that common, and even less so when used with subparsers. As my post on this issue indicates, the relation between the main parser, the "subparsers" command, and individual subparsers gets complicated.

Related

How to perform an argparse subparse for [-A[-b value]] in Python

I want to recreate [-A [-b value]] where in command would look like this:
test.py -A -b 123
Seems really simple but I can't get it right. My latest attempt has been:
byte = subparser.add_parser("-A")
byte.add_argument("-b", type=int)
While the add_parser command accepts '-A', the parser cannot use it. Look at the help:
usage: ipython3 [-h] {-A} ...
positional arguments:
{-A}
optional arguments:
-h, --help show this help message and exit
A subparser is really a special kind of positional argument. To the main parser, you have effectively defined
add_argument('cmd', choices=['-A'])
But to the parsing code, '-A' looks like an optional's flag, as though you had defined
add_argument('-A')
The error:
error: argument cmd: invalid choice: '123' (choose from '-A')
means that it has skipped over the -A and -b (which aren't defined for the main parser), and tried to parse '123' as the first positional. But it isn't in the list of valid choices.
So to use subparsers, you need specify 'A' as the subparser, not '-A'.

python script with 1 or 3 positional arguments

I am writing a python script, it takes either 3 positional arguments (name, date, location, let's say) or 1 argument, which is a setup file which contains that information.
I know that I can use argparse and I can make the positional arguments optional with:
parser.add_argument('name_OR_setupFile')
parser.add_argument('date', nargs='?')
parser.add_argument('location', nargs='?')
and then I can error-check, to make sure that the user didn't do anything stupid
The problem is that now the help message will be very confusing, because it's unclear what the 1st argument really is. I'd LIKE a way to do this as two different add_argument lines, somehow, but I'm not sure how.
I also know that I could use a --setupFile argument, and make the three optional... but I'd rather not do that as well, if I don't have to.
A third option is to use:
parser.add_argument('ARGS', nargs='+', help='ARGS is either of the form setupFile, or name date location')
and then error check later...
ETA for clarification:
I want to be able to call the script with either:
python foo.py setupFile
or
python foo.py name date location
I want the help text to be something like:
usage:
foo.py setupFile
foo.py name date location
I think the clearest design using argparse is:
parser = argparse.ArgumentParser()
g = parser.add_mutually_exclusive_group()
g.add_argument('--setup','-s',metavar='FILE',help='your help')
g.add_argument('--name',nargs=3,metavar=('NAME','DATE','LOCATION'),hel
...: p='your help')
parser.print_help() produces:
usage: ipython3 [-h] [--setup FILE | --name NAME DATE LOCATION]
optional arguments:
-h, --help show this help message and exit
--setup FILE, -s FILE
your help
--name NAME DATE LOCATION
your help
I've handled the 1 or 3 arguments requirement with mutually exclusive optionals. And used metavar to add clarity to the arguments. (As noted in another recent question, metavar does not work well with positionals.)
Another option is to use subparsers. That still requires a key word like setup and name, only they are entered without the --. And the help structure for subparsers is quite different.
Not totally sure this is what you meant, but if I understand you correctly:
if __name__ =='__main__':
def dem_args(*args):
if len(args) == 1:
if os.path.isfile(args[0]):
#go file
else:
#error regarding this being a bad filename or nonexistent file
elif len(args) == 3:
#try to process / raise errors regarding name, date, location
else:
#error reg. wrong number of arguments, possible arguments are either this or that
Ok, this is what I'm currently doing. I'm putting this here for people to comment on, and in case it ends up being useful, for posterity.
I'm actually solving an additional problem here. The problem is actually a little bit more complicated than even I specified. Because there's actually 3 ways to run the program, and I want to be able to have a --help option for only give me the details for one type. So I want -h, -h 1 and -h 2 to all do different things.
My current code is:
import argparse
baseParser = argparse.ArgumentParser(add_help=False)
baseParser.add_argument('-f', '--foo', help ='foo argument')
baseParser.add_argument('-h', '--help', nargs='?' , const = 'all')
parser1 = argparse.ArgumentParser(parents = [baseParser], add_help=False)
parser1.add_argument('name', help='name argument (type 1)')
parser1.add_argument('date', help='date argument')
parser1.add_argument('location', help='location argument')
setupParser=argparse.ArgumentParser(parents = [baseParser],add_help=False)
setupParser.add_argument('setup', help='setup file')
parser2 = argparse.ArgumentParser(parents = [baseParser],add_help=False)
parser2.add_argument('name', help='name argument (type 2)')
parser2.add_argument('baa', help='sheep?')
realParser = argparse.ArgumentParser(parents=[baseParser], add_help=False)
realParser.add_argument('ARGS', nargs = '*', help = 'positional arguments')
args = realParser.parse_args()
if args.help:
if args.help == 'all':
print 'This product can be used in multiple ways:'
print 'setup'
setupParser.print_usage()
print 'type1'
parser1.print_usage()
print'type2'
parser2.print_usage()
print 'use help [type] for more details'
elif args.help=='setup':
setupParser.print_help()
elif args.help=='1':
parser1.print_help()
else:
parser2.print_help()
exit(0)
#actually parse the args in args.ARGS, and work with that

Python: Allow the positional argument to be specified last or write it first in the help output when using argparse

This code
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('target', help='Specifiy who to attack!')
magic = ['fireball', 'heal', 'microwave']
parser.add_argument('-m', '--magic', nargs='*',
choices=magic,
help=('Magic'))
parsed_arguments = parser.parse_args()
Produces this help output
usage: Example.py [-h]
[-m [{fireball,heal,microwave} [{fireball,heal,microwave} ...]]]
target
positional arguments:
target Specifiy who to attack!
optional arguments:
-h, --help show this help message and exit
-m [{fireball,heal,microwave} [{fireball,heal,microwave} ...]], --magic [{fireball,heal,microwave} [{fireball,heal,microwave} ...]]
Magic
I think the help output is confusing and makes it look like the target should be specified last, which however does not work: python example.py -m fireball troll gives argument -m/--magic: invalid choice: 'troll'.
I realize the grammar of the language becomes ambiguous, but it would still be possible to tell that as there should exist one word (target) last in the sentence troll is not an argument to the -m option.
Questions:
Is there a way to let the positional argument be specified last without beating argparse to much?
Is there a way to let the argparse help output indicate that target should indeed be specified first?
As previously mentioned in a comment, the standard methods on POSIX systems to denote the end of options is to separate the arguments and options by --.
As for the second question: You might have to create your own HelpFormatter to achieve this, see formatter-class. You might be able to inherit from the default formatter and override only the necessary functions to generate the usage line.

In python's argparse module, how can I disable printing subcommand choices between curly brackets?

How can I disable printing subcommand choices, the ones between curly brackets? Using an example at http://docs.python.org/dev/library/argparse.html#sub-commands, the normal output is:
usage: [-h] {foo,bar} ...
optional arguments:
-h, --help show this help message and exit
subcommands:
{foo,bar} additional help
What I want is to print this:
usage: [-h] {foo,bar} ...
optional arguments:
-h, --help show this help message and exit
subcommands:
Removing just the last line.
To avoid spamming my users with the huge ugly curly-braced list of dozens of sub-commands, I simply set the metavar attribute of the subcommand object. My code looks like:
import argparse
parser = argparse.ArgumentParser(description='Stack Overflow example')
subs = parser.add_subparsers()
subs.metavar = 'subcommand'
sub = subs.add_parser('one', help='does something once')
sub = subs.add_parser('two', help='does something twice')
parser.parse_args()
And the output of running this script with a single -h argument is:
usage: tmp.py [-h] subcommand ...
Stack Overflow example
positional arguments:
subcommand
one does something once
two does something twice
optional arguments:
-h, --help show this help message and exit
The result is not exactly what you illustrate as your best desired case, but I think that it may be the closest you can get without subclassing argparse.ArgumentParser and overriding the things you need adjusted, which would be messy work.
Override ArgumentParser.print_usage() with your own method to print whatever, however you want. If all you want to do is eliminate the last line, call the original version, capture the results (by sending it to a file) and print just the part(s) you want.

In python, how to get subparsers to read in parent parser's argument?

Here is an example code:
import argparse
parser=argparse.ArgumentParser()
parser.add_argument('-main_arg')
subparser=parser.add_subparser()
a=subparser.add_parser('run')
a.add_argument('required_sub_arg')
a.add_argument('arg_a')
b=subparser.add_parser('b')
parser.parse_args()
I want it to read in -main_arg if I enter program run required_sub_arg -main_arg -arg_a
Right now, it doesn't recognize -main_arg as a valid argument.
PSA to recent readers
As this question still has visits in 2018, before doing anything this complex with argparse, please consider using docopt or click instead. It will improve both your sanity and that of anyone who might read or modify your code. Thank you.
Original answer
As is, you have a few issues.
First, parser.parse_args is a method that returns a namespace of parser's arguments, so you should do something like
args = parser.parse_args()
Then args.main_args to get-main_arg from a call like
program -main_arg run required_sub_arg -arg_a
Your issue with main_arg is that you have created a argument to parser named main_arg, and you make a call like
program run required_sub_arg -main_arg -arg_a
that refers to an argument to a named main_arg. Since a doesn't have such an argument, it is invalid.
In order to refer to a parser's argument from one of its subparser, you have to make said subparser inherit the arguments of its parent. This is done with
a=parser.add_subparser('run', parents=[parser])
You have mistaken subparser for child parser. See http://docs.python.org/dev/py3k/library/argparse.html and https://code.google.com/p/argparse/issues/detail?id=54 for more informations.
For anyone else using argparse that arrives here looking for a way to display "common" sub-parser arguments in the "main" help screen, here's one approach:
import argparse
common = argparse.ArgumentParser(add_help=False)
common.add_argument('--shared', action='store_true', help='some shared arg')
parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter)
parser.add_argument('--parent', action='store_true', help='parent only arg')
subparsers = parser.add_subparsers()
run = subparsers.add_parser('run', parents=[common])
run.add_argument('--fast', action='store_true', help='run only arg')
parser.epilog = "--- Arguments common to all sub-parsers ---" \
+ common.format_help().replace(common.format_usage(), '')
args = parser.parse_args()
Main help:
$ program.py -h
usage: program.py [-h] {run} ...
positional arguments:
{run}
optional arguments:
-h, --help show this help message and exit
--parent parent only arg
--- Arguments common to all sub-parsers ---
optional arguments:
--shared some shared arg
run sub-parser help:
$ program.py run -h
usage: program.py run [-h] [--shared]
optional arguments:
-h, --help show this help message and exit
--shared some shared arg
--fast run only arg
To address the actual question, since the accepted answer doesn't run for me, here's some additional information on why it doesn't seem possible to truly share argparse arguments with the same name across both parent and child/sub-parser parsers.
First, the problem with the following code:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-main_arg')
subparsers = parser.add_subparsers()
run = subparsers.add_parser('run', parents=[parser])
args = parser.parse_args()
Is that it leads to the following error, because both parent parser and sub-parser run define the -h/--help argument (by default).
Argparse.ArgumentError: argument -h/--help: conflicting option strings: -h, --help
While this error can be avoided by suppressing the -h/--help option (with add_help=False) on either the parent or the child, it's nice to have the help option at both levels.
Another potential way to avoid conflicting help options is to move common arguments to a shared parser, common:
import argparse
common = argparse.ArgumentParser(add_help=False)
common.add_argument('-main_arg', action='store_true')
parser = argparse.ArgumentParser(parents=[common])
subparsers = parser.add_subparsers()
run = subparsers.add_parser('run', parents=[common])
args = parser.parse_args()
print(args)
While this appears to work on the surface, in practice, it doesn't work as intended:
$ program.py run # OK
Namespace(main_arg=False)
$ program.py run -main_arg # OK
Namespace(main_arg=True)
$ program.py -main_arg run # BAD: expected main_arg to be True
Namespace(main_arg=False)
The behavior observed when parsing program.py -main_arg run illustrates a key relationship: a parent argparser and its sub-parsers are independent parsers, where the parent parses all arguments up to the sub-parser "command" positional argument, and then the selected sub-parser parses the remaining arguments in the same Namespace as the parent with no regard for attributes that may have been set by the parent.

Categories