We Keep Coding

argparse: unable to get subparser_name after declaring a global argument

I've got a parser with some sub-parsers. I setup a global argument to be used on all subparser. Here's the relevant snippet
parser = argparse.ArgumentParser(prog="my_prog", add_help=False)
parser.add_argument('-d', '--debug', action='store_true', help='debug flag')
subparsers = parser.add_subparsers(dest="subparser_name", help='some help notes')
parser_cmd1 = subparsers.add_parser('cmd1', parents=[parser])
parser_cmd1.add_argument('-f', '-foo', type=str, action=foo, required=False, help='foo command')
parser_cmd2 = subparsers.add_parser('cmd2', parents=[parser])
parser_cmd2.add_argument('-b', '-bar', type=str, action=bar, required=False, help='bar command')
args = parser.parse_args()
parser = args.subparser_name
print(args)
if args.debug:
logging.basicConfig(level=logging.INFO)
if parser == 'cmd1':
if args.foo:
//do foo stuff
if parser == 'cmd2':
if args.bar:
//do bar stuff
So you can a command like such my_prog.py cmd1 -d -f inp_str. Here's the problem: subparser_name is None. The output of print(args) looks kind of like this
Namespace(debug=True, foo="inp_str", subparser_name=None)
Before I added the global debug argument, subparser_name would be the name of the command I ran, i.e. 'cmd1' or 'cmd2'. Now, it's 'None'. Even with the parents=[parser] addition in the subparser creation. How can I fix this? How do I know which command was called?

Split out the common args to a separate ArgumentParser, which is then used as parent for the sub parsers. Also your foo and bar options were specified using -foo and -bar whi should be --foo and --bar. Also you didn't have default values for these so e.g. when -f/--foo wasn't specified args.foo correctly didn't exist.
This works better:
import argparse
common_args = argparse.ArgumentParser(prog="my_prog", add_help=False)
common_args.add_argument('-d', '--debug', action='store_true', help='debug flag')
parser = argparse.ArgumentParser(prog="my_prog", add_help=True)
subparsers = parser.add_subparsers(dest="subparser_name", help='some help notes')
parser_cmd1 = subparsers.add_parser('cmd1', parents=[common_args])
parser_cmd1.add_argument('-f', '--foo', type=str, default='', required=False, help='foo command')
parser_cmd2 = subparsers.add_parser('cmd2', parents=[common_args])
parser_cmd2.add_argument('-b', '--bar', type=str, default='', required=False, help='bar command')
args = parser.parse_args()
parser = args.subparser_name
print(args)
if args.debug:
logging.basicConfig(level=logging.INFO)
if parser == 'cmd1':
if args.foo:
#//do foo stuff
print( f"foo {args.foo}" )
if parser == 'cmd2':
if args.bar:
#//do bar stuff
print( f"bar {args.bar}" )
run with:
args.py cmd1 -f asd
output:
Namespace(subparser_name='cmd1', debug=False, foo='asd')
foo asd
Update:
If you want to be able to use e.g. args.py -d cmd1 then on the creation of parser, specify parents=[common_args]
parser = argparse.ArgumentParser(prog="my_prog", add_help=True, parents=[common_args])
Next time you ask a question ensure you only post code as a minimal reproducible example - i.e. that can be run without adding anything

The subparser's defaults have priority of any values set by the main parser - default or user input. The main does set the subparser_name to 'cmd1', but the subparser changes it back to the default None.
While not evident in your test case, defining debug at both levels has the same problem. The subparser's default overwrites anything set in the main.
In general, it is not a good idea to use the same dest in the main and subparsers. Flags can be the same, but the dest should be different - at least if you want to see anything set by the main.
And using the main parser as a parent to the sub, is just asking for confusion.

Extending Help in Argparse to Subcommands [duplicate]

My argparse has only 3 flags (store_true) on the top level, everything else is handled through subparsers. When I run myprog.py --help, the output shows a list of all subcommands like normal, {sub1, sub2, sub3, sub4, ...}. So, the default is working great...
I usually can't remember the exact subcommand name I need, and all of its options. So I end up doing 2 help lookups:
myprog.py --help
myprog.py sub1 --help
I do this so often, I decided to cram this into one step. I would rather have my toplevel help output a huge summary, and then I scroll through the list manually. I find it is much faster (for me at least).
I was using a RawDescriptionHelpFormatter, and typing the long help output by hand. But now I have lots of subcommands, and its becoming a pain to manage.
Is there a way to get a verbose help output with just one program call?
If not, how can I iterate the subparsers of my argparse instance, and then retrieve the help output individually from each one (which I will then later glue together)?
Here is a quick outline of my argparse setup. I cleaned/stripped the code a fair bit, so this may not run without a bit of help.
parser = argparse.ArgumentParser(
prog='myprog.py',
formatter_class=argparse.RawDescriptionHelpFormatter,
description=textwrap.dedent(""" You can manually type Help here """) )
parser.add_argument('--debuglog', action='store_true', help='Verbose logging for debug purposes.')
parser.add_argument('--ipyonexit', action='store_true', help='Drop into an embeded Ipython session instead of exiting command.')
subparser = parser.add_subparsers()
### --- Subparser B
parser_b = subparser.add_parser('pdfreport', description="Used to output reports in PDF format.")
parser_b.add_argument('type', type=str, choices=['flatlist', 'nested', 'custom'],
help="The type of PDF report to generate.")
parser_b.add_argument('--of', type=str, default='',
help="Override the path/name of the output file.")
parser_b.add_argument('--pagesize', type=str, choices=['letter', '3x5', '5x7'], default='letter',
help="Override page size in output PDF.")
parser_b.set_defaults(func=cmd_pdf_report)
### ---- Subparser C
parser_c = subparser.add_parser('dbtables', description="Used to perform direct DB import/export using XLS files.")
parser_c.add_argument('action', type=str, choices=['push', 'pull', 'append', 'update'],
help="The action to perform on the Database Tables.")
parser_c.add_argument('tablename', nargs="+",
help="The name(s) of the DB-Table to operate on.")
parser_c.set_defaults(func=cmd_db_tables)
args = parser.parse_args()
args.func(args)

This is a bit tricky, as argparse does not expose a list of defined sub-parsers directly. But it can be done:
import argparse
# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--foo', action='store_true', help='foo help')
subparsers = parser.add_subparsers(help='sub-command help')
# create the parser for the "a" command
parser_a = subparsers.add_parser('a', help='a help')
parser_a.add_argument('bar', type=int, help='bar help')
# create the parser for the "b" command
parser_b = subparsers.add_parser('b', help='b help')
parser_b.add_argument('--baz', choices='XYZ', help='baz help')
# print main help
print(parser.format_help())
# retrieve subparsers from parser
subparsers_actions = [
action for action in parser._actions
if isinstance(action, argparse._SubParsersAction)]
# there will probably only be one subparser_action,
# but better safe than sorry
for subparsers_action in subparsers_actions:
# get all subparsers and print help
for choice, subparser in subparsers_action.choices.items():
print("Subparser '{}'".format(choice))
print(subparser.format_help())
This example should work for python 2.7 and python 3. The example parser is from Python 2.7 documentation on argparse sub-commands.
The only thing left to do is adding a new argument for the complete help, or replacing the built in -h/--help.

Here is complete soulution with custom help handler (almost all code from #Adaephon answer):
import argparse
class _HelpAction(argparse._HelpAction):
def __call__(self, parser, namespace, values, option_string=None):
parser.print_help()
# retrieve subparsers from parser
subparsers_actions = [
action for action in parser._actions
if isinstance(action, argparse._SubParsersAction)]
# there will probably only be one subparser_action,
# but better save than sorry
for subparsers_action in subparsers_actions:
# get all subparsers and print help
for choice, subparser in subparsers_action.choices.items():
print("Subparser '{}'".format(choice))
print(subparser.format_help())
parser.exit()
# create the top-level parser
parser = argparse.ArgumentParser(prog='PROG', add_help=False) # here we turn off default help action
parser.add_argument('--help', action=_HelpAction, help='help for help if you need some help') # add custom help
parser.add_argument('--foo', action='store_true', help='foo help')
subparsers = parser.add_subparsers(help='sub-command help')
# create the parser for the "a" command
parser_a = subparsers.add_parser('a', help='a help')
parser_a.add_argument('bar', type=int, help='bar help')
# create the parser for the "b" command
parser_b = subparsers.add_parser('b', help='b help')
parser_b.add_argument('--baz', choices='XYZ', help='baz help')
parsed_args = parser.parse_args()

Perhaps an easier approach is to use parser.epilog:
def define_parser():
import argparse
parser = argparse.ArgumentParser(
prog='main',
formatter_class=argparse.RawDescriptionHelpFormatter,
)
commands = parser.add_subparsers(
title="required commands",
help='Select one of:',
)
command_list = commands.add_parser(
'list',
help='List included services',
)
command_ensure = commands.add_parser(
'ensure',
help='Provision included service',
)
command_ensure.add_argument(
"service",
help='Service name',
)
import textwrap
parser.epilog = textwrap.dedent(
f"""\
commands usage:\n
{command_list.format_usage()}
{command_ensure.format_usage()}
"""
)
return parser
parser = define_parser()
parser.print_help()
which results in the following output:
usage: main [-h] {list,ensure} ...
optional arguments:
-h, --help show this help message and exit
required commands:
{list,ensure} Select one of:
list List included services
ensure Provision included service
commands usage:
usage: main list [-h]
usage: main ensure [-h] service

A simpler way to iterate over the subparsers in Adaephon's example is
for subparser in [parser_a, parser_b]:
subparser.format_help()
Python does allow you to access hidden attributes like parser._actions, but that's not encouraged. It is just as easy to build your own list while defining the parser. Same goes for doing special things with the arguments. add_argument and add_subparser return their respective Action and Parser objects for a reason.
If I were making a subclass of ArgumentParser I would feel free to use _actions. But for a one off application, building my own list would be clearer.
An example:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('mainpos')
parser.add_argument('--mainopt')
sp = parser.add_subparsers()
splist = [] # list to collect subparsers
sp1 = sp.add_parser('cmd1')
splist.append(sp1)
sp1.add_argument('--sp1opt')
sp2 = sp.add_parser('cmd2')
splist.append(sp2)
sp2.add_argument('--sp2opt')
# collect and display for helps
helps = []
helps.append(parser.format_help())
for p in splist:
helps.append(p.format_help())
print('\n'.join(helps))
# or to show just the usage
helps = []
helps.append(parser.format_usage())
for p in splist:
helps.append(p.format_usage())
print(''.join(helps))
The combined 'usage' display is:
usage: stack32607706.py [-h] [--mainopt MAINOPT] mainpos {cmd1,cmd2} ...
usage: stack32607706.py mainpos cmd1 [-h] [--sp1opt SP1OPT]
usage: stack32607706.py mainpos cmd2 [-h] [--sp2opt SP2OPT]
The display of the combined helps is long and redundant. It could be edited in various ways, either after formatting, or with special help formatters. But who is going make such choices?

I was also able to print a short help for commands using _choices_actions.
def print_help(parser):
print(parser.description)
print('\ncommands:\n')
# retrieve subparsers from parser
subparsers_actions = [
action for action in parser._actions
if isinstance(action, argparse._SubParsersAction)]
# there will probably only be one subparser_action,
# but better save than sorry
for subparsers_action in subparsers_actions:
# get all subparsers and print help
for choice in subparsers_action._choices_actions:
print(' {:<19} {}'.format(choice.dest, choice.help))

add_subparsers().add_parser() accepts not only a description, which shows up in the help of the subcommand, but also a help= which is used as one-line description in the top-level parsers' help.
The docs have this hidden in the formulation
(A help message for each subparser command, however, can be given by supplying the help= argument to add_parser() as above.)
and even in the sample code around that sentence:
>>> # create the parser for the "b" command
>>> parser_b = subparsers.add_parser('b', help='b help')
>>> parser_b.add_argument('--baz', choices='XYZ', help='baz help')
[...]
usage: PROG [-h] [--foo] {a,b} ...
positional arguments:
{a,b} sub-command help
a a help
b b help
Yes, this is not the full help for everthing, but IMHO covers the basic use case very well and is not easily discoverable.

if __name__ == '__main__':
parser = argparse.ArgumentParser("TOML FILE OVERWRITE SCRIPT")
subparsers = parser.add_subparsers()
parser_set = subparsers.add_parser('set', help='Set Toml')
parser_set.add_argument('set', help='TOMl file edit set action', action='store_true')
parser_set.add_argument('-n', '--name', type=str, help='Service Name', required=True)
parser_set.add_argument('-s', '--section', type=str, help='Toml Section Name', required=True)
parser_set.add_argument('-k', '--key', type=str, help='Toml Key of Section', required=True)
parser_set.add_argument('-v', '--value', help='New Value', required=True)
args = parser.parse_args()
if args.set:
setter = ConfigurationSetter(args.name, args.section, args.key, args.value)
setter.execute()
else:
print("Ops! Something is wrong, type --help or -h")
You can check my code maybe inspires you!

How to set optional arguments to positional arguments in Python's Argparse?

I have the following code:
# Get parsed arguments
args = argparse.ArgumentParser(description=Messages().Get(112))
# Get the arguments for sinit
args.add_argument('init', help=Messages().Get(100), action="store_true")
args.add_argument('--url', default=None, help=Messages().Get(101))
# Get the arguments for schema import
args.add_argument('schema-import', help=Messages().Get(104), action="store_true")
args.add_argument('--file', default=None, help=Messages().Get(104))
The --url argument should only be used with init. For example: script.py schema-import --url should not be accepted but script.py schema-import --file should.
How to set arguments as child arguments?

As mentioned there might be a way to do this with argparse, I'm not sure, but in any event I find it more transparent to explicitly handle argument dependencies in application logic. This should achieve what I think you want:
import argparse
import sys
args = argparse.ArgumentParser(description="please only use the '--url' argument if you also use the 'init' argument")
# Going to use aliases here it's more conventional. So user can use, eg,
# -i or --init for the first argument.
args.add_argument('-i', '--init', help='init help', action="store_true")
args.add_argument('-u', '--url', default=None, help='init help')
args.add_argument('-s', '--schema-import', help='schema-import help', action="store_true")
args.add_argument('-f', '--file', help='file help')
def main():
arguments = args.parse_args()
if arguments.url and not arguments.init:
# You can log an output or raise an exception if you want
# But most likely a print statment is most appropriate
# Since you are interacting with the CLI.
print("You can only use the URL option with init. Exiting")
sys.exit(0)
print("gaurd clauses passed. Here is my code...")
...
if __name__ == "__main__":
main()
Test results (my file called temp.py):
$python temp.py -u https://www.google.com
You can only use the URL option with init. Exiting
$
$python temp.py -i -u https://www.google.com
Gaurd clauses passed. Here is my code...

Why bother with doing all the logic when you can let argparse do all the work for you?
Simply use Sub-commands to define different "branches" of execution:
args = argparse.ArgumentParser(description=Messages().Get(112))
subparsers = args.add_subparsers()
parser_init = subparsers.add_parser('init', help=Messages().Get(100))
parser_init.add_argument('--url', default=None, help=Messages().Get(101))
parser_schema = subparsers.add_parser('schema-import', help=Messages().Get(104))
parser_schema.add_argument('--file', default=None, help=Messages().Get(104))
And this will give you what you want without any logic added:
>>> print(args.parse_args(['schema-import', '--url', "some.url"]))
usage: args.py [-h] {init,schema-import} ...
args.py: error: unrecognized arguments: --url some.url
>>> print(args.parse_args(['schema-import', '--file', "some.file"]))
Namespace(file='some.file')

python-daemon + argparse?

I made a script that takes a few arguments to run. Originally it could be run automatically with the argument 'auto', but I'm trying to daemonize it so it will run the run the script with the specified arguments as a daemon. The problem is that python-daemon and argparse don't seem to get along when it comes to deciding who parses what.
parser = argparse.ArgumentParser(usage='pyfilter.py <file> <options> <actions>')
parser.add_argument('file', help='blacklist file containing IPs', type=str)
subparsers = parser.add_subparsers(help='help', dest='action')
parser_update = subparsers.add_parser('update', help='update help')
parser_update.add_argument('-c', '--check', help="check IPs for abuse reports", dest="check", type=str, nargs=1)
parser_blacklist = subparsers.add_parser('blacklist', help='create iptables rules for malicious IPs specified'
'in the provided file')
parser_clear = subparsers.add_parser('clear', help='clear iptables')
parser_auto = subparsers.add_parser('auto', help='automatically run update and blacklist on a loop')
parser_auto.add_argument('-i', '--interval', help='specify the loop interval', dest='interval', type=int, nargs=1)
parser_auto.add_argument('-c', '--check', help="check IPs for abuse reports", dest="check", type=str, nargs=1)
parser.add_argument('-p', '--port', help='specify the port to block', type=int)
parser.add_argument('-v', '--verbose', help='write output to screen', nargs=1)
args = parser.parse_args()
. . .
class pyfilterDaemon():
def __init__(self):
self.stdin_path = '/dev/null'
self.stdout_path = '/dev/tty'
self.stderr_path = '/dev/tty'
self.pidfile_path = '/tmp/pyfilter.pid'
self.pidfile_timeout = 5
def run(self):
while True:
update()
blacklist()
time.sleep(interval)
. . .
def main():
#handle()
d = pyfilterDaemon()
daemon_runner = runner.DaemonRunner(d)
daemon_runner.start()
Here's the commands I've tried to make this work:
root#tfof:~# ./pyfilter.py start
ERROR: File 'start' not found # argparse parsed 'start' accidentally
root#tfof:~# ./pyfilter.py /etc/blacklist.lst -v yes auto
usage: checkfilter.py stop|restart|start # now the argparse arguments are satisfied, but python-daemon is looking for its argument
root#tfof:~# ./pyfilter.py /etc/blacklist.lst -v yes auto start
usage: pyfilter.py <file> <options> <actions>
pyfilter.py: error: unrecognized arguments: start # argparse is trying to parse 'start'
Would it be possible to pass off the 'start' argument to python-daemon or something? Or if I could just get rid of the argparsing it'd be fine, but the 'file' argument is a must.

Argparse takes arguments from sys.argv per default (see here).
It is not surprising that the behaviour you see here is happening,
as you just call the parse_args function with the default arguments.
You can just pass whatever you want to parse to it, instead of sys.argv.
See this question for an example.
So consume whatever you need for python-deamon and then parse the remaining args with argparse.

Remove long sub-command help output?

I'm using argparse to produce my CLI and using the add_subparsers function. This works exception the --help output is really ugly. It lists all the commands in the overall syntax. For example:
usage: redid [-h] [--config CONFIG] [--verbose] [--show-curl] [--save [SAVE]]
{setup,show-config,check-auth,version,get-profile,put-profile,delete-profile,get-profile-signature,list-profiles,list-resources,ls-resources,get-resource-record,delete-resource,get-resource,upload-resource,get-resource-url}
...
I'd much prefer to have a more traditional and clean output similar to:
usage: redid [OPTIONS...] Command ...
How can I do this?

Try adding the metavar argument to your subparser definition and give it no value:
parser.add_subparsers(title="the_commands", metavar="")
From the documentation:
Description of parameters:
....
metavar - string presenting available sub-commands in help; by default it is None and presents sub-commands in form {cmd1, cmd2, ..}
Here's an example, I'm not sure how you've set up your sub-parsers but:
parser = argparse.ArgumentParser(prog='PROG')
parser.add_argument('--foo', action='store_true', help='foo help')
subparsers = parser.add_subparsers(help="sub-command help", metavar="sub command")
parser_a = subparsers.add_parser('a', help='a help')
parser_a.add_argument('bar', type=int, help='bar help')
parser_b = subparsers.add_parser('b', help='b help')
parser_b.add_argument('--baz', choices='XYZ', help='baz help')
>>> parser.print_help()
usage: PROG [-h] [--foo] sub command ...
positional arguments:
sub command sub-command help
a a help
b b help
optional arguments:
-h, --help show this help message and exit
--foo foo help