So I'm writing this very small program to do http get and post requests. The requests are as follows:
requestApp.py help
requestApp.py help get
requestApp.py help post
requestApp.py get [-v] [-h key:value] URL
requestApp.py post [-v] [-h key:value] [-d inline-data] [-f file] URL
As you can see, the -v, -h, -d, -f, URL arguments are optional. The get and post arguments are non-optional. I'll show you the snippet of my program that is relevant to this situation:
parser = argparse.ArgumentParser(description='httpc is a curl-like application but supports HTTP protocol only.')
parser.add_argument('command', type=str, help=help_output())
parser.add_argument('url', action='store_true', help='The URL that will be provided to perform the requested command.')
parser.add_argument('-v', '--verbose', action='store_true')
The command argument will be help, get, or post, and the url argument is self explanatory. My question is related to the second and third commands above, namely:
requestApp.py help get
requestApp.py help post
How can I make sure that when typing help get, the get will not be registered in the URL (same for help post). In addition, when I do include a URL, I want it to be stored inside of the URL argument. Would I have to manually evaluate the arguments passed through if statements? Or there is a better way to do it?
Perhaps the closest an argparse solution can come, at least without going the subparser route, is:
import argparse
import sys
print(sys.argv)
parser = argparse.ArgumentParser()
parser.add_argument('-k', '--keyvalue')
parser.add_argument('-v', '--verbose', action='store_true')
parser.add_argument('-d', '--data')
parser.add_argument('-f', '--file')
parser.add_argument('pos1', choices = ['help', 'get', 'post'])
parser.add_argument('pos2')
args = parser.parse_args()
print(args)
The resulting help is:
1744:~/mypy$ python stack54383659.py get aurl -h
['stack54383659.py', 'get', 'aurl', '-h']
usage: stack54383659.py [-h] [-k KEYVALUE] [-v] [-d DATA] [-f FILE]
{help,get,post} pos2
positional arguments:
{help,get,post}
pos2
optional arguments:
-h, --help show this help message and exit
-k KEYVALUE, --keyvalue KEYVALUE
-v, --verbose
-d DATA, --data DATA
-f FILE, --file FILE
The fit isn't perfect. For example you can give just help, but you can provide just -h. The 2nd positional value can be any string, 'get', a valid url or something else. Your own code will have to valid that. The key:value bit requires your own parsing.
In the argparse way of parsing the optionals can occur in any order. The two positionals have to occur in the given order (relative to each other).
In newer Pythons I can change the last positional to be 'optional', and use the new intermixed parser. That would allow me to give just 'help' (or just 'get'):
parser.add_argument('pos2', nargs='?')
args = parser.parse_intermixed_args()
intermixed is needed if the two positional values are separated by flags. For some complex reasons, the regular parsing may consume the '?' argument prematurely leaving you with an extra unrecognized string.
Another approach is to define all the flagged arguments, and use parse_known_args. The non-flag values will be in the extras list, which you can parse as you like. Older parsers like optparse did essentially that. argparse added a limited ability to handle positional arguments as well, but strictly by position, not by value.
It is quite complicated to do this using argparse here is how to do it using docopt, docopt parses the usage pattern and returns a dictionary :
"""
Usage:
requestApp help [get|post]
requestApp get [-v] [-k=key:value] <URL>
requestApp post [-v] [-k=key:value] [-d=data] [-f file] <URL>
Options:
-v --verbose This is verbose mode
-d=data This option does this
-k=key:value This one does that
-f file This one is magic
"""
from docopt import docopt
ARGS = docopt(__doc__)
For example with requestApp.py post -k hello:world -f myfile.txt google.com docopt will return:
{
"--verbose": false,
"-d": None,
"-f": "myfile.txt",
"-k": "hello:world",
"<URL>": "google.com",
"get": false,
"help": false,
"post": true
}
Then you can do:
if ARGS['help']:
if ARGS['get']: pass # requestApp help get
else if ARGS['post']: pass # requestApp help post
else: pass # requestApp help
exit()
if ARGS['get']: pass # requestApp get
else if ARGS['post']: pass # requestApp post
if ARGS['--verbose']: print("this is just the beginning")
-h is a reserved option by default (for help) that makes docopt returns the usage pattern and exit.
docopt will return the usage pattern to stdout and exit if you try illegal commands such as requestApp help unicorn
Related
I'm building a command line argparser for my program and I try to give much detail in the -h option
I have the following code:
import argparse
legal_actions = ['act1', 'act2', 'act3']
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(help='available commands')
parser_cmd = subparsers.add_parser("cmd")
parser_cmd.add_argument("-a", "--action", type=str, metavar="", choices=legal_actions, required=True,
help='list of actions: {%(choices)s}')
parser_cmd.add_argument("nargs", type=str, nargs='*',
help="the rest of the arguments required to perform an action")
parser_cmd.set_defaults(func=cmd_handler)
python prog.py cmd -h will result the following prints in the command line
usage: cmd [-h] -a [nargs [nargs ...]]
positional arguments:
nargs the rest of the arguments required to perform an action
optional arguments:
-h, --help show this help message and exit
-a , --action list of actions: {act1, act2, act3}
Every action requires a different number of arguments, so I want to add something that will describe the actions (from the list of actions) like:
actions availble:
act1: requires 2 arguments (arg1, arg2)
act2: requires 0 arguments ()
act3: requires 1 arguments (arg1)
And I want it to have any link with the above "optional arguments", so it'll be easy to see the "acts" are under the -a option
If you want to add more information, you can use the epilog-parameter:
from argparse import RawDescriptionHelpFormatter # This is used to enable newlines in epilogs and descriptions(\n)
from argparse import ArgumentParser
description = 'Some description of program'
epilog = 'actions availble:\n\t'
epilog += 'act1: requires 2 arguments (arg1, arg2)\n\t'
epilog += 'act2: requires 0 arguments ()\n\t'
epilog += 'act3: requires 1 arguments (arg1)'
parser = argparse.ArgumentParser(description=description, epilog=epilog,
formatter_class=RawTextHelpFormatter)
This will print out
actions availble:
act1: requires 2 arguments (arg1, arg2)
act2: requires 0 arguments ()
act3: requires 1 arguments (arg1)
At the end of help output. The epilog-parameter can also be included in the add_parser() when using add_subparsers():
This object has a single method, add_parser(), which takes a command name and any ArgumentParser constructor arguments, and returns an ArgumentParser object that can be modified as usual.
NOTE: the default formatter will ignore newlines, so take a look at Python argparse: How to insert newline in the help text? where this is addressed, which describe how to replace the formatter รก la:
ArgumentParser(..., formatter_class=RawDescriptionHelpFormatter)
Read more about the epilog-parameter in docs.
Basically what I am trying to achieve is this :
python http_client.py (get|post) [-v] (-h "k:v")* [-d inline-data] [-f file] URL
Now, what I did was something like this :
parser.add_argument('get', help='Get executes a HTTP GET request for a given URL.', default='http://httpbin.org/get')
But it's not working. I did some SO and these are the other links which I tried but desired output was not achieved
Python argparse optional sub-arguments
How to have sub-parser arguments in separate namespace with argparse?
This parser produces a similar usage line:
import argparse
parser = argparse.ArgumentParser(prog='http_client.py')
parser.add_argument("gp", choices=['get','post'])
parser.add_argument('-v', action='version', version='0.0.1')
parser.add_argument('--how', action='append',help='k:v, may repeat')
parser.add_argument('-d', metavar='inline-data')
parser.add_argument('-f','--file')
parser.add_argument('url')
print(parser.parse_args())
sample help with usage. Note that the positionals are displayed at the end, but may be interspersed with the optionals:
1356:~/mypy$ python stack46357414.py -h
usage: http_client.py [-h] [-v] [--how HOW] [-d inline-data] [-f FILE]
{get,post} url
positional arguments:
{get,post}
url
optional arguments:
-h, --help show this help message and exit
-v show program's version number and exit
--how HOW k:v, may repeat
-d inline-data
-f FILE, --file FILE
I assume your -v is supposed to be version, though -v is also used for a verbosity flag.
1357:~/mypy$ python stack46357414.py -v
0.0.1
example with get/post, several how, and required url
1357:~/mypy$ python stack46357414.py get --how 3:3 --how a:b aurl
Namespace(d=None, file=None, gp='get', how=['3:3', 'a:b'], url='aurl')
argparse does not parse k:v strings for you. You get to do that after parsing. I assume the (-h "k:v")* means you want to enter several of the k:v pairs. nargs='*' is alternative to the action='append'.
I defined the simple gp positional with a choices. That restricts the input to these 2 strings. So far in your description I don't see a need for subparsers.
In [210]: args=argparse.Namespace(d=None, file=None, gp='get', how=['3:3', 'a:b'
...: ], url='aurl')
In [211]: args
Out[211]: Namespace(d=None, file=None, gp='get', how=['3:3', 'a:b'], url='aurl')
In [212]: vars(args)
Out[212]: {'d': None, 'file': None, 'gp': 'get', 'how': ['3:3', 'a:b'], 'url': 'aurl'}
In [213]: for k in args.__dict__:
...: print(k, args.__dict__[k])
...:
file None
d None
url aurl
gp get
how ['3:3', 'a:b']
I am writing a program that takes a list of urls as a parameter. Optionally, I would like to also be able to include a file containing a list of additional urls. Thus the program help would look like a combination of the below:
usage: [-h] url [url ...]
or
usage: [-h] --input-file file [ url [url ...]]
I am currently programming it as:
usage: [-h] [--input-file file] [url [url ...]]
where both usage of --input-file and specifying at least one url is optional. I am then checking myself if either the input-file or list of urls is specified.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-i','--input-file', help="File of urls to be read")
parser.add_argument('urls',nargs='*')
args = parser.parse_args()
if not args.input_list and not args.urls:
parser.error('must have either an input-file or a url')
Is there a way in argparse where I can force the user to specify a url if no input-file is given, but make it optional if it is?
If you specify a fromfile-prefix-chars (such as #), the user could specify a file to read these urls from. The lines of this file are read and added to the sys.argv just as though they had been typed in the command line.
http://docs.python.org/dev/library/argparse.html#fromfile-prefix-chars
parser = argparse.ArgumentParser(fromfile_prefix_chars='#')
parser.add_argument('urls',nargs='+') # require 1 or more url
args = parser.parse_args()
these all work (where urls.txt has one url string per line)
parser.parse_args('four.txt'.split())
parser.parse_args('#urls.txt four.txt'.split())
parser.parse_args('#urls.txt'.split())
while no file or url gives an error
parser.parse_args(''.split())
The default help does not say anything about allowing that #url.txt file name, but you could add that in a custom description string.
PS - Your post-argparse test is simpler than anything you could specify in argparse. A required mutually-exclusive-group involving the -i argument and the positional might work, but it doesn't do much more than your test. I wrote 'might' because I'd have to double whether a '*' positional works in such a group. But this is one of the first cases I've seen where that fromfile_prefix_chars option would be genuinely useful.
ArgumentParser.add_mutually_exclusive_group() should help you accomplish your goal. Have a look at the example at https://docs.python.org/2/library/argparse.html#mutual-exclusion or the code below:
~#: cat urls.py
import argparse
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-infile', dest='file', help='File of URLs to be read')
group.add_argument('-urls', dest='urls', nargs='+', help='One or more URLs')
args = parser.parse_args()
~#: python urls.py
usage: urls.py [-h] (-infile FILE | -urls URLS [URLS ...])
urls.py: error: one of the arguments -infile -urls is required
I'm using the Python argparse module for command line subcommands in my program. My code basically looks like this:
import argparse
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(title="subcommands", metavar="<command>")
subparser = subparsers.add_parser("this", help="do this")
subparser = subparsers.add_parser("that", help="do that")
parser.parse_args()
When running "python test.py --help" I would like to list the available subcommands. Currently I get this output:
usage: test.py [-h] <command> ...
optional arguments:
-h, --help show this help message and exit
subcommands:
<command>
this do this
that do that
Can I somehow remove the <command> line in the subcommands listing and still keep it in the usage line? I have tried to give help=argparse.SUPPRESS as argument to add_subparsers, but that just hides all the subcommands in the help output.
I solved it by adding a new HelpFormatter that just removes the line if formatting a PARSER action:
class SubcommandHelpFormatter(argparse.RawDescriptionHelpFormatter):
def _format_action(self, action):
parts = super(argparse.RawDescriptionHelpFormatter, self)._format_action(action)
if action.nargs == argparse.PARSER:
parts = "\n".join(parts.split("\n")[1:])
return parts
I would like to be able to use the global options from an argparse.ArgumentParser object to override/augment the defaults values for a sub-command.
An example would be that displayed help reflects the global updates, i.e., for the following toy example:
import argparse
import os
import sys
# Global parser and options.
parser = argparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("--user",
dest="user",
default=os.environ.get("USER"),
help="Override the setting of the $USER variable.")
# Sub-command parser and options.
subparsers = parser.add_subparsers()
command = subparsers.add_parser(
"command",
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
command.add_argument("--config",
dest="config",
default="~%s/config" % os.environ.get("USER"),
help="The config file.")
options = parser.parse_args()
Ideally when when I run this in help mode I would get,
> python example.py --user thing command --help
usage: example.py command [-h] [--config CONFIG]
optional arguments:
-h, --help show this help message and exit
--config CONFIG The config file. (default: ~thing/config)
i.e., the config file path is user specific (thing). I realize that I could change the default config to be "~%(user)s/config" and then resolve this at run-time with the options namespace, however I would like the help to be more explicit.
I gather an alternative solution would be to try to parse the arguments once to obtain the global options, i.e.,
if "--help" in sys.argv:
# Parse the command minus the help to obtain the global options.
args = sys.argv[1:]
args.remove("--help")
# Update the defaults with the global options.
options = parser.parse_args(args)
command.set_defaults(config="~%s/config" % options.user)
# Re-parse the options.
parser.parse_args()
Though this seems somewhat hacky. Is there a better approach/solution?
After defining the global options, but before defining the subcommands, call parse_known_args to find out what the value of --user is. Then, finish defining the subparser commands, using the value of --user to define the default of --config, before
calling parse_args to parse all options on the command line.
This is a little different from your alternative, but it keeps all the command-line processing inside the argparse object.
(I trimmed down your code a little just to shorten it for this answer.)
import os
import argparse
# Global preparser and options.
preparser = argparse.ArgumentParser(add_help=False)
preparser.add_argument("--user", dest="user", default=os.environ.get("USER"))
# ****** NEW *******
options, _ = preparser.parse_known_args() # Ignore what we haven't defined yet
user = options.user # Use this to define the default of --config
parser = argparse.ArgumentParser(parents=[ preparser ], add_help=True,
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
# Sub-command parser and options.
subparsers = parser.add_subparsers()
command = subparsers.add_parser("command")
# ****** MODIFIED *******
command.add_argument("--config", dest="config", default="~%s/config" % (user,))
options = parser.parse_args()
Here's a solution that modifies the help line directly, at runtime. It could also modify os.environ or some other global as well, but I'll keep it simple. The key is assigning the action created by add_argument('--config'...) to a variable. help is just an attribute of that variable. You are free to modify that.
class FooAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
config.help = 'new user (%s)'% values
setattr(namespace, self.dest, values)
parser = argparse.ArgumentParser()
parser.add_argument('--user',action=FooAction)
sub = parser.add_subparsers()
cmd = sub.add_parser('cmd')
config = cmd.add_argument('--config',help='initial help')
config.help = 'default help' # alt way of setting 'help'
# print config # to see other attributes of the config action
args = parser.parse_args()
print args
Invoked with just cmd -h we get the default help
$ python stack12167228.py cmd -h
usage: stack12167228.py cmd [-h] [--config CONFIG]
optional arguments:
-h, --help show this help message and exit
--config CONFIG default help
Invoked with --user xxx cmd -h, the help is customized
$ python stack12167228.py --user xxx cmd -h
usage: stack12167228.py cmd [-h] [--config CONFIG]
optional arguments:
-h, --help show this help message and exit
--config CONFIG new user (xxx)