We Keep Coding

How can I remove three-dots at the end of usage line in argparse

Python argparse keep putting space and three-dots ( ...) at the end of usage: line, example: usage: program.sh [-h] command [<options>...] .... Would it be possible to remove them?
Example code:
def helper():
parser = argparse.ArgumentParser(
"program.py",
)
subparsers = parser.add_subparsers(dest="command", metavar="command [<options>...]")
driver = subparsers.add_parser(
"driver", help="Example script")
driver.add_argument("--bn", type=int, default=0, help="Block number to start fetch blocks from")
return parser
Output:
$ ./program.sh --help
usage: program.sh [-h] command [<options>...] ...

Direct answer: you could write your own usage summary:
parser = argparse.ArgumentParser(
"program.py",
usage="usage: %(prog)s [-h] command [<options>...]",
)
However, there is something that does not make sense to me.
The ... is caused by using subparsers. There is usually one subparser per command, for example: 3 commands, i.e. 3 subparsers:
Usage:
myprog cmd1 [options and args specific for cmd1]
myprog cmd2 [different options and args for cmd2]
myprog cmd3 [yet another set of options and args]
And that can be hardly summarized in one line except with:
Usage: myprog command ...
unless you are not using any options and args for the commands, which means there is nothing to parse.
So, if you want to get rid of the trailing ... and still have a valid usage synopsis, you probably do not need subparsers at all.

That usage is produced by the implied nargs of the subparsers argument, 'A...'.
I get the same thing if I create a positional argument with the same nargs:
In [393]: import argparse
In [394]: p = argparse.ArgumentParser()
In [395]: p.add_argument('foo',nargs=argparse.PARSER)
Out[395]: _StoreAction(option_strings=[], dest='foo', nargs='A...', const=None, default=None, type=None, choices=None, help=None, metavar=None)
In [396]: p.print_help()
usage: ipython3 [-h] foo ...
positional arguments:
foo
optional arguments:
-h, --help show this help message and exit
Note the nargs string in Out[395]. add_subparsers creates a positional argument with 'A...' nargs value.
In [403]: p = argparse.ArgumentParser()
In [404]: p.add_subparsers(dest='foo', metavar='FOO')
Out[404]: _SubParsersAction(option_strings=[], dest='foo', nargs='A...', const=None, default=None, type=None, choices={}, help=None, metavar='FOO')
In [405]: p.print_help()
usage: ipython3 [-h] FOO ...
positional arguments:
FOO
optional arguments:
-h, --help show this help message and exit
That nargs takes all the remaining strings while requiring at least one. The add_parser lines add choices to that subparser Action.
In formatting its usage the main parser "knows nothing" about what the sub-parsers do. To it, the subparsers argument is just another positional argument with choices. The same applies when parsing. It just does the suparsers.__call__ with the remaining argv strings. That in turn passes those to the chosen parser.

Extending Help in Argparse to Subcommands [duplicate]

My argparse has only 3 flags (store_true) on the top level, everything else is handled through subparsers. When I run myprog.py --help, the output shows a list of all subcommands like normal, {sub1, sub2, sub3, sub4, ...}. So, the default is working great...
I usually can't remember the exact subcommand name I need, and all of its options. So I end up doing 2 help lookups:
myprog.py --help
myprog.py sub1 --help
I do this so often, I decided to cram this into one step. I would rather have my toplevel help output a huge summary, and then I scroll through the list manually. I find it is much faster (for me at least).
I was using a RawDescriptionHelpFormatter, and typing the long help output by hand. But now I have lots of subcommands, and its becoming a pain to manage.
Is there a way to get a verbose help output with just one program call?
If not, how can I iterate the subparsers of my argparse instance, and then retrieve the help output individually from each one (which I will then later glue together)?
Here is a quick outline of my argparse setup. I cleaned/stripped the code a fair bit, so this may not run without a bit of help.
parser = argparse.ArgumentParser(
prog='myprog.py',
formatter_class=argparse.RawDescriptionHelpFormatter,
description=textwrap.dedent(""" You can manually type Help here """) )
parser.add_argument('--debuglog', action='store_true', help='Verbose logging for debug purposes.')
parser.add_argument('--ipyonexit', action='store_true', help='Drop into an embeded Ipython session instead of exiting command.')
subparser = parser.add_subparsers()
### --- Subparser B
parser_b = subparser.add_parser('pdfreport', description="Used to output reports in PDF format.")
parser_b.add_argument('type', type=str, choices=['flatlist', 'nested', 'custom'],
help="The type of PDF report to generate.")
parser_b.add_argument('--of', type=str, default='',
help="Override the path/name of the output file.")
parser_b.add_argument('--pagesize', type=str, choices=['letter', '3x5', '5x7'], default='letter',
help="Override page size in output PDF.")
parser_b.set_defaults(func=cmd_pdf_report)
### ---- Subparser C
parser_c = subparser.add_parser('dbtables', description="Used to perform direct DB import/export using XLS files.")
parser_c.add_argument('action', type=str, choices=['push', 'pull', 'append', 'update'],
help="The action to perform on the Database Tables.")
parser_c.add_argument('tablename', nargs="+",
help="The name(s) of the DB-Table to operate on.")
parser_c.set_defaults(func=cmd_db_tables)
args = parser.parse_args()
args.func(args)

This is a bit tricky, as argparse does not expose a list of defined sub-parsers directly. But it can be done:
import argparse
# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--foo', action='store_true', help='foo help')
subparsers = parser.add_subparsers(help='sub-command help')
# create the parser for the "a" command
parser_a = subparsers.add_parser('a', help='a help')
parser_a.add_argument('bar', type=int, help='bar help')
# create the parser for the "b" command
parser_b = subparsers.add_parser('b', help='b help')
parser_b.add_argument('--baz', choices='XYZ', help='baz help')
# print main help
print(parser.format_help())
# retrieve subparsers from parser
subparsers_actions = [
action for action in parser._actions
if isinstance(action, argparse._SubParsersAction)]
# there will probably only be one subparser_action,
# but better safe than sorry
for subparsers_action in subparsers_actions:
# get all subparsers and print help
for choice, subparser in subparsers_action.choices.items():
print("Subparser '{}'".format(choice))
print(subparser.format_help())
This example should work for python 2.7 and python 3. The example parser is from Python 2.7 documentation on argparse sub-commands.
The only thing left to do is adding a new argument for the complete help, or replacing the built in -h/--help.

Here is complete soulution with custom help handler (almost all code from #Adaephon answer):
import argparse
class _HelpAction(argparse._HelpAction):
def __call__(self, parser, namespace, values, option_string=None):
parser.print_help()
# retrieve subparsers from parser
subparsers_actions = [
action for action in parser._actions
if isinstance(action, argparse._SubParsersAction)]
# there will probably only be one subparser_action,
# but better save than sorry
for subparsers_action in subparsers_actions:
# get all subparsers and print help
for choice, subparser in subparsers_action.choices.items():
print("Subparser '{}'".format(choice))
print(subparser.format_help())
parser.exit()
# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG', add_help=False) # here we turn off default help action
parser.add_argument('--help', action=_HelpAction, help='help for help if you need some help') # add custom help
parser.add_argument('--foo', action='store_true', help='foo help')
subparsers = parser.add_subparsers(help='sub-command help')
# create the parser for the "a" command
parser_a = subparsers.add_parser('a', help='a help')
parser_a.add_argument('bar', type=int, help='bar help')
# create the parser for the "b" command
parser_b = subparsers.add_parser('b', help='b help')
parser_b.add_argument('--baz', choices='XYZ', help='baz help')
parsed_args = parser.parse_args()

Perhaps an easier approach is to use parser.epilog:
def define_parser():
import argparse
parser = argparse.ArgumentParser(
prog='main',
formatter_class=argparse.RawDescriptionHelpFormatter,
)
commands = parser.add_subparsers(
title="required commands",
help='Select one of:',
)
command_list = commands.add_parser(
'list',
help='List included services',
)
command_ensure = commands.add_parser(
'ensure',
help='Provision included service',
)
command_ensure.add_argument(
"service",
help='Service name',
)
import textwrap
parser.epilog = textwrap.dedent(
f"""\
commands usage:\n
{command_list.format_usage()}
{command_ensure.format_usage()}
"""
)
return parser
parser = define_parser()
parser.print_help()
which results in the following output:
usage: main [-h] {list,ensure} ...
optional arguments:
-h, --help show this help message and exit
required commands:
{list,ensure} Select one of:
list List included services
ensure Provision included service
commands usage:
usage: main list [-h]
usage: main ensure [-h] service

A simpler way to iterate over the subparsers in Adaephon's example is
for subparser in [parser_a, parser_b]:
subparser.format_help()
Python does allow you to access hidden attributes like parser._actions, but that's not encouraged. It is just as easy to build your own list while defining the parser. Same goes for doing special things with the arguments. add_argument and add_subparser return their respective Action and Parser objects for a reason.
If I were making a subclass of ArgumentParser I would feel free to use _actions. But for a one off application, building my own list would be clearer.
An example:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('mainpos')
parser.add_argument('--mainopt')
sp = parser.add_subparsers()
splist = [] # list to collect subparsers
sp1 = sp.add_parser('cmd1')
splist.append(sp1)
sp1.add_argument('--sp1opt')
sp2 = sp.add_parser('cmd2')
splist.append(sp2)
sp2.add_argument('--sp2opt')
# collect and display for helps
helps = []
helps.append(parser.format_help())
for p in splist:
helps.append(p.format_help())
print('\n'.join(helps))
# or to show just the usage
helps = []
helps.append(parser.format_usage())
for p in splist:
helps.append(p.format_usage())
print(''.join(helps))
The combined 'usage' display is:
usage: stack32607706.py [-h] [--mainopt MAINOPT] mainpos {cmd1,cmd2} ...
usage: stack32607706.py mainpos cmd1 [-h] [--sp1opt SP1OPT]
usage: stack32607706.py mainpos cmd2 [-h] [--sp2opt SP2OPT]
The display of the combined helps is long and redundant. It could be edited in various ways, either after formatting, or with special help formatters. But who is going make such choices?

I was also able to print a short help for commands using _choices_actions.
def print_help(parser):
print(parser.description)
print('\ncommands:\n')
# retrieve subparsers from parser
subparsers_actions = [
action for action in parser._actions
if isinstance(action, argparse._SubParsersAction)]
# there will probably only be one subparser_action,
# but better save than sorry
for subparsers_action in subparsers_actions:
# get all subparsers and print help
for choice in subparsers_action._choices_actions:
print(' {:<19} {}'.format(choice.dest, choice.help))

add_subparsers().add_parser() accepts not only a description, which shows up in the help of the subcommand, but also a help= which is used as one-line description in the top-level parsers' help.
The docs have this hidden in the formulation
(A help message for each subparser command, however, can be given by supplying the help= argument to add_parser() as above.)
and even in the sample code around that sentence:
>>> # create the parser for the "b" command
>>> parser_b = subparsers.add_parser('b', help='b help')
>>> parser_b.add_argument('--baz', choices='XYZ', help='baz help')
[...]
usage: PROG [-h] [--foo] {a,b} ...
positional arguments:
{a,b} sub-command help
a a help
b b help
Yes, this is not the full help for everthing, but IMHO covers the basic use case very well and is not easily discoverable.

if __name__ == '__main__':
parser = argparse.ArgumentParser("TOML FILE OVERWRITE SCRIPT")
subparsers = parser.add_subparsers()
parser_set = subparsers.add_parser('set', help='Set Toml')
parser_set.add_argument('set', help='TOMl file edit set action', action='store_true')
parser_set.add_argument('-n', '--name', type=str, help='Service Name', required=True)
parser_set.add_argument('-s', '--section', type=str, help='Toml Section Name', required=True)
parser_set.add_argument('-k', '--key', type=str, help='Toml Key of Section', required=True)
parser_set.add_argument('-v', '--value', help='New Value', required=True)
args = parser.parse_args()
if args.set:
setter = ConfigurationSetter(args.name, args.section, args.key, args.value)
setter.execute()
else:
print("Ops! Something is wrong, type --help or -h")
You can check my code maybe inspires you!

python: argparse with optional command-line arguments

I'd like to implement arguments parsing.
./app.py -E [optional arg] -T [optional arg]
The script requires at least one of parameters: -E or -T
What should I pass in parser.add_argument to have such functionality?
UPDATE
For some reason(s) the suggested solution with add_mutually_exclusive_group didn't work, when I added nargs='?' and const= attributes:
parser = argparse.ArgumentParser(prog='PROG')
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-F', '--foo', nargs='?', const='/tmp')
group.add_argument('-B', '--bar', nargs='?', const='/tmp')
parser.parse_args([])
Running as script.py -F still throws error:
PROG: error: one of the arguments -F/--foo -B/--bar is required
However, the following workaround helped me:
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('-F', '--foo', nargs='?', const='/tmp')
parser.add_argument('-B', '--bar', nargs='?', const='/tmp')
args = parser.parse_args()
if (not args.foo and not args.bar) or (args.foo and args.bar):
print('Must specify one of -F/-B options.')
sys.exit(1)
if args.foo:
foo_func(arg.foo)
elif args.bar:
bar_func(args.bar)
...

You can make them both optional and check in your code if they are set.
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
parser.add_argument('--bar')
args = parser.parse_args()
if args.foo is None and args.bar is None:
parser.error("please specify at least one of --foo or --bar")
If you want only one of the two arguments to be present, see [add_mutually_exclusive_group] (https://docs.python.org/2/library/argparse.html#mutual-exclusion) with required=True
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> group = parser.add_mutually_exclusive_group(required=True)
>>> group.add_argument('--foo', action='store_true')
>>> group.add_argument('--bar', action='store_false')
>>> parser.parse_args([])
usage: PROG [-h] (--foo | --bar)
PROG: error: one of the arguments --foo --bar is required

python's argument parser printing the argument name in upper case

I am trying to write usage/help for my python script using the argparse library.
This is my sample code:
import argparse
parser = argparse.ArgumentParser(
description='My description')
parser.add_argument(
"-r", "--remote",
help="help message")
parser.print_help()
Output:
usage: [-h] [-r REMOTE]
My description
optional arguments:
-h, --help show this help message and exit
-r REMOTE, --remote REMOTE
help message
I have no idea why it is printing REMOTE after the -r and --remote switches in the above output.
Can anyone tell me what am I doing wrong here or what should I do to get rid of it?

You are looking at the metavar; it is autogenerated from the option string to form a placeholder. It tells the user that it is there that they need to fill in a value.
You can set an explicit metavar value with the metavar keyword argument:
When ArgumentParser generates help messages, it needs some way to refer to each expected argument. By default, ArgumentParser objects use the dest value as the “name” of each object. By default, for positional argument actions, the dest value is used directly, and for optional argument actions, the dest value is uppercased.
You see it because your argument takes a value; if you expected it to be a toggle, use action='store_true'; in that case the option defaults to False unless the user specifies the switch.
Demo of the latter:
>>> import argparse
>>> parser = argparse.ArgumentParser(
... description='My description')
>>> parser.add_argument("-r", "--remote", action='store_true', help="help message")
_StoreTrueAction(option_strings=['-r', '--remote'], dest='remote', nargs=0, const=True, default=False, type=None, choices=None, help='help message', metavar=None)
>>> parser.print_help()
usage: [-h] [-r]
My description
optional arguments:
-h, --help show this help message and exit
-r, --remote help message
>>> opts = parser.parse_args([])
>>> opts.remote
False
>>> opts = parser.parse_args(['-r'])
>>> opts.remote
True

You are missing action.
import argparse
parser = argparse.ArgumentParser(
description='My description')
parser.add_argument(
"-r", "--remote", action="store_true", # add action
help="help message")
parser.print_help()
usage: -c [-h] [-r]
My description
optional arguments:
-h, --help show this help message and exit
-r, --remote help message

Argparse: optional subparsers with store_true flags

Forgive me if my terminology is off, but I'm looking for a way to add a subparser to an optional argparse argument, with store_true flags on each of the args.
Ideally, I'd like to use the following syntax to reference the boolean value of --html in the shodan_parser subparser:
if args.shodan.html:
print("Doing a thing")
parser = argparse.ArgumentParser()
parser.add_argument("-r", "--reverse-dns", help="rDNS on host", action="store_true")
parser.add_argument("-s", "--shodan", help="perform SHODAN query on discovered IPs", action="store_true")
parser.add_argument("targets", help="IPv4 addresses to search for", nargs="+")
subparsers = parser.add_subparsers()
shodan_parser = subparsers.add_parser("shodan", help="SHODAN options")
shodan_parser.add_argument("--html", action="store_true")
shodan_parser.set_defaults(which='shodan')
Output:
(venv)[nott#admin gumdrop]$ python gumdrop.py google.ca --shodan --html
usage: gumdrop.py [-h] [-r] [-e] [-s] targets [targets ...] {shodan} ...
gumdrop.py: error: too few arguments
(venv)[nott#admin gumdrop]$ python gumdrop.py --shodan --html google.ca askjeeves.ca
usage: gumdrop.py [-h] [-r] [-e] [-s] targets [targets ...] {shodan} ...
gumdrop.py: error: invalid choice: 'askjeeves.ca' (choose from 'shodan')
Any suggestions?

parser = argparse.ArgumentParser()
parser.add_argument("-r", "--reverse-dns", help="rDNS on host", action="store_true")
parser.add_argument("-s", "--shodan", help="perform SHODAN query on discovered IPs", action="store_true")
parser.add_argument("targets", help="IPv4 addresses to search for", nargs="+")
Does this --shodan optional (flag) have anything to do with the subparser name? What is its purpose? Are you confusing a flag with a subparser?
The subparser is also a positional. Using both a positional with nargs='+' and a subparser might work, but is likely to cause confusion. When does the list of targets end, and the subparser (and its arguments) begin?
subparsers = parser.add_subparsers()
shodan_parser = subparsers.add_parser("shodan", help="SHODAN options")
...
your output
(venv)[nott#admin gumdrop]$ python gumdrop.py google.ca --shodan --html
....
I'm guessing that this sets targets=['google.ca'], shodan=True. But you haven't given it a subparser command. The --html is an unknown.
(venv)[nott#admin gumdrop]$ python gumdrop.py --shodan --html google.ca askjeeves.ca
...
Now you set shodan=True (the main parser flag). Again --html is unknown. It sets target=['google.ca']. But now it tries to interpret askjeeves.ca as a subparser command . But it does not match the available choices.
I think these lines would work:
python gumdrop.py google.ca askjeeves.ca shodan --html
python gumdrop.py --shodan google.ca shodan --html
I'd suggest dropping the whole subparser bit. It's just confusing things, for you and your users. Go ahead and include --html in the main parser. It's optional, so it can be used, or ignored, at will. It you must use subparsers, review the documentation, and try some simpler examples.
As for getting args.shodan.html, that's a much more advanced issue, involving the nesting of namespaces. For now be happy if you get args.html.

You'll need to change the switch from "-h" to something else (or disable the help), because the "-h" switch is already used by the help menu
parser = argparse.ArgumentParser()
parser.add_argument("-r", "--reverse-dns", help="rDNS on host", action="store_true")
parser.add_argument("-s", "--shodan", help="perform SHODAN query on discovered IPs", action="store_true")parser.add_argument("targets", help="IPv4 addresses to search for", nargs="+")
subparsers = parser.add_subparsers()
shodan_parser = subparsers.add_parser("shodan", help="SHODAN options")
shodan_parser.add_argument("--html", action="store_true")
shodan_parser.set_defaults(which='shodan')
args = parser.parse_args()
if args.html:
print("Doing a thing")