I have to write a command-line interface and I've seen I can use docopt and argparse.
I would like to know what are the main differences between the two so that I can make an enlightened choice.
Please stick to the facts. I don't want Wow. docopt. So beautiful. Very useful.
Docopt parses a doc string, whereas argparse constructs its parsing by creating an object instance and adding behaviour to it by function calls.
Example for argparse:
parser = argparse.ArgumentParser()
parser.add_argument("operation", help="mathematical operation that will be performed",
choices=['add', 'subtract', 'multiply', 'divide'])
parser.add_argument("num1", help="the first number", type=int)
parser.add_argument("num2", help="the second number", type=int)
args = parser.parse_args()
Example for docopt:
"""Calculator using docopt
Usage:
calc_docopt.py <operation> <num1> <num2>
calc_docopt.py (-h | --help)
Arguments:
<operation> Math Operation
<num1> First Number
<num2> Second Number
Options:
-h, --help Show this screen.
"""
from docopt import docopt
if __name__ == '__main__':
arguments = docopt(__doc__, version='Calculator with docopt')
print(arguments)
Note, that docopt uses Usage: and Options: sections for parsing. Here Arguments: is provided only for end-user convenience.
The Why page for Click:
https://click.palletsprojects.com/en/7.x/why/
has a nice comparison between argparse, docopt and click itself.
Click is another command-line parsing utility for Python.
argparse is in the python default library, so this doesn't add any extra dependencies to your program. The differences are mainly the way of writing your code. Using argparse it is possible to add hooks for plugins so they can add their own argumnets to your program. For example flake8 uses this.
docopt is a third party module provides a simple way of parsing arguments. I personally like docopt because of its simplicity, but I'm not saying it is the best to use in all cases. In their documentation they mention that using docopt it is possible to use more combinations of argument passing than when using argparse.
Related
I'm trying to create a terminal application a bit similar to cutechess-cli, which has options like the following:
-epdout FILE Save the end position of the games to FILE in FEN format.
-recover Restart crashed engines instead of stopping the match
-repeat [N] Play each opening twice (or N times). Unless the -noswap...
which can all be done with argparse in Python.
However it also has "named arguments" like the following:
-resign movecount=COUNT score=SCORE [twosided=VALUE]
Adjudicate the game as a loss if an engine's score is
at least SCORE centipawns below zero for at least COUNT
consecutive moves.
-sprt elo0=ELO0 elo1=ELO1 alpha=ALPHA beta=BETA
Use a Sequential Probability Ratio Test as a termination
criterion for the match. This option should only be used...
I can implement that with argparse using nargs='*' and then writing my own parser (maybe just regex). However that doesn't give nice documentation, and if argparse can already do something like this, I would rarther use the builtin approach.
Summary: Does argparse have a concept of named arguments similar to resign and sprt above? And if not, would the best approach be to do this manyally using nargs='*'?
You can use a custom type to split the values and use the metavar argument to give a better description for the value:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--arg', nargs='*', type=lambda text: text.split('=', maxsplit=1), metavar='PARAM-NAME=PARAM-VALUE', help='Some other parameters')
args = parser.parse_args()
args.arg = {k: v for k,v in args.arg}
Which produces:
usage: [-h] [--arg [PARAM-NAME=PARAM-VALUE [PARAM-NAME=PARAM-VALUE ...]]]
optional arguments:
-h, --help show this help message and exit
--arg [PARAM-NAME=PARAM-VALUE [PARAM-NAME=PARAM-VALUE ...]]
Some other parameters
If you wanted to you could avoid the "postprocessing" step to build the dictionary by using a custom Action type. But it seems an overkill to me in this case.
I have a simple python script that uses docopt to parse command line arguments. It looks like this:
#!/usr/bin/env python
__doc__ = """
Usage: mycopy <src>... <dest>
"""
from docopt import docopt
options = docopt(__doc__)
When I run it:
./mycopy source1/ source2/ destination/
it just prints the usage info, meaning that the command line arguments I passed it were wrong. Is something wrong with the usage spec? Is it even possible to do something like this using docopt?
If you put <dest> before <src>..., it works. Accordingly, run it with ./mycopy destination/ source1/ source2/.
I think the docopt hasn't implemented the support for: ARGS... ARG. This case adds some complexity to the implementation. But I agree 'copy src1 src2 ... dest' is more straightforward usage. So maybe you could raise a request to this project: https://github.com/docopt/docopt
I have a program which can be used in the following way:
program install -a arg -b arg
program list
program update
There can only ever be one of the positional arguments specified (install, list or update). And there can only be other arguments in the install scenario.
The argparse documentation is a little dense and I'm having a hard time figuring out how to do this correctly. What should my add_arguments look like?
This seems like you want to use subparsers.
from argparse import ArgumentParser
parser = ArgumentParser()
subparsers = parser.add_subparsers()
install = subparsers.add_parser('install')
install.add_argument('-b')
install.add_argument('-a')
install.set_defaults(subparser='install')
lst = subparsers.add_parser('list')
lst.set_defaults(subparser='list')
update = subparsers.add_parser('update')
update.set_defaults(subparser='update')
print parser.parse_args()
As stated in the docs, I have combined with set_defaults so that you can know which subparser was invoked.
argparse fails at dealing with sub-commands receiving global options:
import argparse
p = argparse.ArgumentParser()
p.add_argument('--arg', action='store_true')
s = p.add_subparsers()
s.add_parser('test')
will have p.parse_args('--arg test'.split()) work,
but fails on p.parse_args('test --arg'.split()).
Anyone aware of a python argument parser that handles global options to sub-commands properly?
You can easily add this argument to both parsers (main parser and subcommand parser):
import argparse
main = argparse.ArgumentParser()
subparser = main.add_subparsers().add_parser('test')
for p in [main,subparser]:
p.add_argument('--arg', action='store_true')
print main.parse_args('--arg test'.split()).arg
print main.parse_args('test --arg'.split()).arg
Edit: As #hpaulj pointed in comment, there is also parents argument which you can pass to ArgumentParser constructor or to add_parser method. You can list in this value parsers which are bases for new one.
import argparse
base = argparse.ArgumentParser(add_help=False)
base.add_argument('--arg', action='store_true')
main = argparse.ArgumentParser(parents=[base])
subparser = main.add_subparsers().add_parser('test', parents=[base])
print main.parse_args('--arg test'.split()).arg
print main.parse_args('test --arg'.split()).arg
More examples/docs:
looking for best way of giving command line arguments in python, where some params are req for some option and some params are req for other options
Python argparse - Add argument to multiple subparsers (I'm not sure if this question is not overlaping with this one too much)
http://docs.python.org/dev/library/argparse.html#parents
Give docopt a try:
>>> from docopt import docopt
>>> usage = """
... usage: prog.py command [--test]
... prog.py another [--test]
...
... --test Perform the test."""
>>> docopt(usage, argv='command --test')
{'--test': True,
'another': False,
'command': True}
>>> docopt(usage, argv='--test command')
{'--test': True,
'another': False,
'command': True}
There's a ton of argument-parsing libs in the Python world. Here are a few that I've seen, all of which should be able to handle address the problem you're trying to solve (based on my fuzzy recollection of them when I played with them last):
opster—I think this is what mercurial uses, IIRC
docopt—This one is new, but uses an interesting approach
cliff—This is a relatively new project by Doug Hellmann (PSF member, virtualenvwrapper author, general hacker extraordinaire) is a bit more than just an argument parser, but is designed from the ground up to handle multi-level commands
clint—Another project that aims to be "argument parsing and more", this one by Kenneth Reitz (of Requests fame).
Here's a dirty workaround --
import argparse
p = argparse.ArgumentParser()
p.add_argument('--arg', action='store_true')
s = p.add_subparsers()
s.add_parser('test')
def my_parse_args(ss):
#parse the info the subparser knows about; don't issue an error on unknown stuff
namespace,leftover=p.parse_known_args(ss)
#reparse the unknown as global options and add it to the namespace.
if(leftover):
s.add_parser('null',add_help=False)
p.parse_args(leftover+['null'],namespace=namespace)
return namespace
#print my_parse_args('-h'.split()) #This works too, but causes the script to stop.
print my_parse_args('--arg test'.split())
print my_parse_args('test --arg'.split())
This works -- And you could modify it pretty easily to work with sys.argv (just remove the split string "ss"). You could even subclass argparse.ArgumentParser and replace the parse_args method with my_parse_args and then you'd never know the difference -- Although subclassing to replace a single method seems overkill to me.
I think however, that this is a lit bit of a non-standard way to use subparsers. In general, global options are expected to come before subparser options, not after.
The parser has a specific syntax: command <global options> subcommand <subcommand ptions>, you are trying to feed the subcommand with an option and but you didn't define one.
According to PEP 257 the docstring of command line script should be its usage message.
The docstring of a script (a
stand-alone program) should be usable
as its "usage" message, printed when
the script is invoked with incorrect
or missing arguments (or perhaps with
a "-h" option, for "help"). Such a
docstring should document the script's
function and command line syntax,
environment variables, and files.
Usage messages can be fairly elaborate
(several screens full) and should be
sufficient for a new user to use the
command properly, as well as a
complete quick reference to all
options and arguments for the
sophisticated user.
So my docstring would look something like this:
<tool name> <copyright info>
Usage: <prog name> [options] [args]
some text explaining the usage...
Options:
-h, --help show this help message and exit
...
Now I want to use the optparse module. optparse generates the "Options" sections and a "usage" explaining the command line syntax:
from optparse import OptionParser
if __name__ == "__main__":
parser = OptionParser()
(options, args) = parser.parse_args()
So calling the script with the "-h" flag prints:
Usage: script.py [options]
Options:
-h, --help show this help message and exit
This can be modified as follows:
parser = OptionParser(usage="Usage: %prog [options] [args]",
description="some text explaining the usage...")
which results in
Usage: script.py [options] [args]
some text explaining the usage...
Options:
-h, --help show this help message and exit
But how can I use the docstring here? Passing the docstring as the usage message has two problems.
optparse appends "Usage: " to the docstring if it does not start with "Usage: "
The placeholder '%prog' must be used in the docstring
Result
According to the answers it seems that there is no way to reuse the docstring intended by the optparse module. So the remaining option is to parse the docstring manually and construct the OptionParser. (So I'll accept S.Loot's answer)
The "Usage: " part is introduced by the IndentedHelpFormatter which can be replaced with the formatter parameter in OptionParser.__init__().
I wrote a module docopt to do exactly what you want – write usage-message in docstring and stay DRY.
It also allows to avoid writing tedious OptionParser code at all, since docopt is generating parser
based on the usage-message.
Check it out: http://github.com/docopt/docopt
"""Naval Fate.
Usage:
naval_fate.py ship new <name>...
naval_fate.py ship [<name>] move <x> <y> [--speed=<kn>]
naval_fate.py ship shoot <x> <y>
naval_fate.py mine (set|remove) <x> <y> [--moored|--drifting]
naval_fate.py -h | --help
naval_fate.py --version
Options:
-h --help Show this screen.
--version Show version.
--speed=<kn> Speed in knots [default: 10].
--moored Moored (anchored) mine.
--drifting Drifting mine.
"""
from docopt import docopt
if __name__ == '__main__':
arguments = docopt(__doc__, version='Naval Fate 2.0')
print(arguments)
Choice 1: Copy and paste. Not DRY, but workable.
Choice 2: Parse your own docstring to strip out the description paragraph. It's always paragraph two, so you can split on '\n\n'.
usage, description= __doc__.split('\n\n')[:2]
Since optparse generates usage, you may not want to supply the usage sentence to it. Your version of the usage my be wrong. If you insist on providing a usage string to optparse, I'll leave it as an exercise for the reader to work out how to remove "Usage: " from the front of the usage string produced above.
I think we have to be reasonable about this PEP's advice -- I would think it's fine to leave the module with __doc__ being the short description that summarizes long usage. But if you're perfectionist:
'''<tool name>
The full description and usage can be generated by optparse module.
Description: ...
'''
...
# Generate usage and options using optparse.
usage, options = ...
# Modify the docstring on the fly.
docstring = __doc__.split('\n\n')
docstring[1:2] = [__license__, usage, options]
__doc__ = '\n\n'.join(docstring)