I am trying to make my help strings helpful. To do this I have a function Function() with a doc string something like
def Function(x):
""" First line describing what Function does
Keyword Arguments
x = float -- A description of what x does that may be long
"""
Having done this I think have something like this at the end
def parse_command_line(argvs):
parser = optparse.OptionParser()
parser.add_option("-f","--Function", help=Function.__doc__,metavar="Bs" )
(options,arguments) = parser.parse_args(argvs)
return options, arguments
options, arguments = parse_command_line(sys.argv)
The trouble occurs when calling the program with -h or --help The output is line wrapped by OptParse, this means the KeyWord arguments are not started on a new line, is it possible to stop OptParse from wrapping the output or is there a better way to do this?
In case of interest, I have written two such formatter classes for argparse. The first supports much of the mediaWiki, MarkDown, and POD syntaxes (even intermixed):
import MarkupHelpFormatter
MarkupHelpFormatter.InputOptions["mediawiki"] = True
parser = argparse.ArgumentParser(
description="""...your text, with mediawiki markup...""",
epilog='...',
formatter_class=MarkupHelpFormatter.MarkupHelpFormatter
)
The other is called "ParagraphHelpFormatter". It merely wraps text like the default argparse formatter, except that it respects blank lines.
Both are at http://derose.net/steve/utilities/PY/MarkupHelpFormatter.py, and
licensed as CCLI Attribution-Share-alike. They format for ANSI terminal
interfaces. Not highly polished (for example, auto-numbering is unfinished), but you may find them helpful.
argparse provides you with a formatter for raw formatting, ie your lines wont get wrapped
parser = argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter)
optparse also lets you set the formatter.. I suppose you can write your own formatter, but theres nothing provided that i know of
Related
I have a use case where I'd like the user to be able to provide, as an argument to argparse, EITHER a single string OR a filename where each line has a string.
Assume the user launches ./myscript.py -i foobar
The logical flow I'm looking for is something like this:
The script determines whether the string foobar is a readable file.
IF it is indeed a readable file, we call some function from the script, passing each line in foobar as an argument to that function. If foobar is not a readable file, we call the same function but just use the string foobar as the argument and return.
I have no ability to guarantee that a filename argument will have a specific extension (or even an extension at all).
Is there a more pythonic way to do this OTHER than just coding up the logic exactly as I've described above? I looked through the argparse tutorial and didn't see anything, but it also seems reasonable to think that there would be some specific hooks for filenames as arguments, so I figured I'd ask.
A way would be:
Let's say that you have created a parser like this:
parser.add_argument('-i',
help='...',
type=function)
Where type points to the function which will be an outer function that evaluates the input of the user and decides if it is a string or a filename
More information about type you can find in the documentation.
Here is a minimal example that demonstrates this use of type:
parser.add_argument('-d','--directory',
type=Val_dir,
help='...')
# ....
def Val_dir(dir):
if not os.path.isdir(dir):
raise argparse.ArgumentTypeError('The directory you specified does not seem to exist!')
else:
return dir
The above example shows that with type we can control the input at parsing time. Of course in your case the function would implement another logic - evaluate if the input is a string or a filename.
This doesn't look like an argparse problem, since all you want from it is a string. That string can be a filename or a function argument. To a parser these will look the same. Also argparse isn't normally used to run functions. It is used to parse the commandline. Your code determines what to do with that information.
So here's a script (untested) that I think does your task:
import argparse
def somefunction(*args):
print(args)
if __name__=='__main__':
parser=argparse.ArgumentParser()
parser.add_argument('-i','--input')
args = parser.parse_args()
try:
with open(args.input) as f:
lines = f.read()
somefunction(*lines)
# or
# for line in lines:
# somefuncion(line.strip())
except:
somefunction(arg.input)
argparse just provides the args.input string. It's the try/except block that determines how it is used.
================
Here's a prefix char approach:
parser=argparse.ArgumentParser(fromfile_prefix_chars='#',
description="use <prog -i #filename> to load values from file")
parser.add_argument('-i','--inputs')
args=parser.parse_args()
for arg in args.inputs:
somefunction(arg)
this is supposed to work with a file like:
one
two
three
https://docs.python.org/3/library/argparse.html#fromfile-prefix-chars
I am using Argparse to parse shell input to my Python function.
The tricky part is that this script first reads in a file that partially determines what kind of arguments are available to Argparse (it's a JSON file containing criteria by which the user can specify what data to output).
But before these arguments are added to my parser, I would like to read in some arguments relating to the file reading itself. (e.g. whether to fix the formatting of the input file). Kinda like this:
test.py (fix_formatting=True, **more arguments added later)
When I try to run args = parser.parse_args() twice, after the initial input and after adding more keys, things fall apart: Argparse quite predictably complains that some of the user input are unrecognized arguments:. I thought I might use subparsers to that end.
So I tried variations of (following the example in the docs as best as I could):
def main():
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(help='sub-command help')
settingsparser = subparsers.add_parser('settings') #i want a subparser called 'settings'
settingsparser.add_argument('--fix_formatting', action='store_true') #this subparser shall have a --fix_formatting
Then I try to parse only the "settings" part like so:
settings=parser.parse_args(['settings'])
This seems to work. But then I add my other keys and things break:
keys=['alpha','beta','gamma','delta']
for key in keys:
parser.add_argument("--"+key, type=str, help="X")
args = parser.parse_args()
If I parse any input for any of the arguments from keys, Argparse complains that I make an invalid choice: [...] (choose from 'settings'). Now I don't understand why I have to choose from "settings"; the docs say that the parse
will only contain attributes for the main parser and the subparser that was selected by the command line (and not any other subparsers)
what is my error of understanding here?
and if this is the wrong approach, how would one go about parsing one bit of input before another bit?
Any help is much appreciated!
parse_args calls parse_known_args. This returns the args namesparse along with a list of strings (from sys.argv) that it could not process (extras). parse_args raises this error if this list is not empty.
https://docs.python.org/3/library/argparse.html#partial-parsing
Thus parse_known_args is useful if you want to parse some of the input.
sys.argv remains unchanged. Subsequent calls to a parser (whether it was the original one or not) use that again, unless you pass the extras.
I don't think subparsers help you here. They aren't meant for delayed or two stage parsing. I'd suggest playing with the documentation examples for subparsers first.
To the main parser, the subparsers look like
subparsers = parser.add_argument('cmd', choices=['select',...])
In other words, it adds a positional argument where the choices are the subparser names that you define. That may help you see why it expects you to name select. Positionals are normally required.
(there's an exception to this in recent versions, https://stackoverflow.com/a/22994500/901925)
It appears to me that there's no easy way to use the RawDescriptionHelpFormatter in the argparse module without either violating PEP8 or cluttering your namespace.
Here is the most obvious way to format it:
parser = argparse.ArgumentParser(prog='PROG',
....
formatter_class=argparse.RawDescriptionHelpFormatter)
This violates the stipulation that lines should not exceed 80 characters
Here's how the example in the argparse documentation looks (spoiler: this is actually correct; see comments below):
parser = argparse.ArgumentParser(
prog='PROG',
formatter_class=argparse.RawDescriptionHelpFormatter,
....
This violates PEP8 E128 regarding the indentation of continuation lines.
Here'another possibility:
parser = argparse.ArgumentParser(
prog='PROG',
formatter_class=
argparse.RawDescriptionHelpFormatter,
....
This violates PEP8 E251 regarding spaces around = for keyward arguments.
(Of course, this doesn't even address the fact that my character-count for the line assumes that the parser token starts on the first column, which is the best case scenario; what if we want to create a parser inside a class and/or a function?)
So the only remaining alternative, as far as I can tell, is to either clutter the namespace:
from argparse import RawDescriptionHelpFormatter, ArgumentParser
...or use a silly temporary variable (which also clutters the namespace):
rawformatter = argparse.RawDescriptionHelpFormatter
parser = argparse.ArgumentParser(prog='PROG',
....
formatter_class=rawformatter)
Am I missing something? I guess having RawDescriptionHelpFormatter and ArgumentParser directly in the current namespace isn't a big deal, but this seems like an unnecessary frustration.
Your second example looks fine to me, and seems to match the "# Hanging indents should add a level." example here: http://legacy.python.org/dev/peps/pep-0008/#indentation
Also seems to tally with this similar question/answer: What is PEP8's E128: continuation line under-indented for visual indent?
A couple of other variations:
from argparse import RawDescriptionHelpFormatter as formatter
parser = argparse.ArgumentFormatter(prog='PROG')
# you can reassign a parser attribute after initialization
parser.formatter_class = formatter
But there are other inputs to ArgumentParser that may be long enough to require wrapping or assignment to separate variables.
usage = 'PROG [-h] --foo FOO BAR etc'
description = """\
This is a long multiline description
that may require dedenting.
"""
description = textwrap.dedent(description)
parser=Argparse(usage=usage, description=description, formatter_class=formatter)
Take a look at test_argparse.py to see the many ways that a long and multifaceted parser can be defined.
http://bugs.python.org/issue13023 raises the issue of what if you wanted several formatter modifications, e.g.:
This means we can either pass argparse.RawDescriptionHelpFormatter or argparse.ArgumentDefaultsHelpFormatter, but not both.
The recommended solution is to subclass the formatter:
class MyFormatter(argparse.RawDescriptionHelpFormatter,
argparse.ArgumentDefaultsHelpformatter):
pass
Another tactic to keep the namespace clean is to wrap the parser definition in a function or module.
http://ipython.org/ipython-doc/2/api/generated/IPython.core.magic_arguments.html
is an example of how IPython wraps argparse to make new API for its users.
Another parser built on argparse, plac first builds a cfg dictionary:
https://code.google.com/p/plac/source/browse/plac_core.py
def pconf(obj):
...
cfg = dict(description=obj.__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
...
return cfg
def parser_from(obj, **confparams):
...
conf = pconf(obj).copy()
conf.update(confparams)
parser = ArgumentParser(**conf)
I set up my argument parser as follows:
parser=argparse.ArgumentParser()
parser.add_argument('--point',help='enter a point (e.g. 2,3,4)')
parser.parse_args('--point=-2,5,6'.split()) #works
parser.parse_args('--point -2,5,6'.split()) #doesn't work :(
Is there any way to tell argparse that strings which match the regular expression r"-\d+.*" are not options but an argument of an option?
Also note that I could do something like this:
parser.add_argument('--point',nargs='*')
parser.parse_args('--point -2 5 6'.split())
but that's not really how I want it to work.
I think preprocessing sys.argv is the most straightforward way here. Consider for example:
import argparse, re
parser=argparse.ArgumentParser()
parser.add_argument('--point',help='enter a point (e.g. 2,3,4)')
args = '--point -2,5,6'.split() # or sys.argv
is_list = re.compile(r'^-?[\d,.]+$')
args = ['"%s"' % x if is_list.match(x) else x for x in args]
print parser.parse_args(args)
This returns Namespace(point='"-2,5,6"') which should be easy to parse.
You could change the prefix char so - is no longer recognized as indicating the start of an argument. It does look a bit weird but it is useful when negative numbers may appear in the arguments.
import argparse
parser=argparse.ArgumentParser(prefix_chars = '#')
parser.add_argument('##point',help='enter a point (e.g. 2,3,4)')
args = parser.parse_args('##point=-2,5,6'.split()) #works
print(args)
# Namespace(point='-2,5,6')
args = parser.parse_args('##point -2,5,6'.split()) #work also
print(args)
# Namespace(point='-2,5,6')
If you don't mind messing around with argparse internals, argparse already does something very similar to what I want to do. One of the classes that ArgumentParser inherits from has this line in __init__
import re as _re
...
self._negative_number_matcher = _re.compile(r'^-\d+$|^-\d*\.\d+$')
So to get my example to work, we just need to swap out an appropriate regex...
parser._negative_number_matcher = re.compile(r'^-\d+|^-\d*\.\d+')
Usually I'm not very much in favor of messing around with the internals of a class when they're prefixed by an underscore (as it is implementation dependent and liable to change) -- However, in this case, I think it's probably ok because:
If argparse changes that variable name, it doesn't hurt, we're just back to the case where "--print -2,3,4" doesn't work again.
I can't think of any better way to determine if something is a number than regex (I suppose they could try to cast to a float and catch the exception, but if they did that, they wouldn't have a variable named _negative_number_matcher anymore and again, argparse would still work fine except in this corner case where it doesn't do what I want)
argparse uses per default abbreviation in unambiguous cases.
I don't want abbreviation and I'd like to disable it.
But didn't find it in the documentation.
Is it possible?
Example:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--send', action='store_true')
parser.parse_args(['--se']) # returns Namespace(send=True)
But I want it only to be true when the full parameter is supplied. To prevent user errors.
UPDATE:
I created a ticket at python bugtracker after Vikas answer. And it already has been processed.
As of Python 3.5.0 you can disable abbreviations by initiating the ArgumentParser with the following:
parser = argparse.ArgumentParser(allow_abbrev=False)
Also see the documentation.
No, apparently this is not possible. At least in Python 2.7.2.
First, I took a look into the documentation - to no avail.
Then I opened the Lib\argparse.py and looked through the source code. Omitting a lot of details, it seems that each argument is parsed by a regular expression like this (argparse:2152):
# allow one or more arguments
elif nargs == ONE_OR_MORE:
nargs_pattern = '(-*A[A-]*)'
This regex will successfully parse both '-' and '--', so we have no control over the short and long arguments. Other regexes use the -* construct too, so it does not depend on the type of the parameter (no sub-arguments, 1 sub-argument etc).
Later in the code double dashes are converted to one dash (only for non-optional args), again, without any flags to control by user:
# if this is an optional action, -- is not allowed
if action.option_strings:
nargs_pattern = nargs_pattern.replace('-*', '')
nargs_pattern = nargs_pattern.replace('-', '')
No, well not without ugly hacks.
The code snippet #Vladimir posted, i suppose that is not what you are looking for. The actual code that is doing this is:
def _get_option_tuples(self, option_string):
...
if option_string.startswith(option_prefix):
...
See the check is startswith not ==.
And you can always extend argparse.ArgumentParser to provide your own _get_option_tuples(self, option_string) to change this behavior. I just did by replacing two occurrence of option_string.startswith(option_prefix) to option_string == option_prefix and:
>>> parser = my_argparse.MyArgparse
>>> parser = my_argparse.MyArgparse()
>>> parser.add_argument('--send', action='store_true')
_StoreTrueAction(option_strings=['--send'], dest='send', nargs=0, const=True, default=False, type=None, choices=None, help=None, metavar=None)
>>> parser.parse_args(['--se'])
usage: [-h] [--send]
: error: unrecognized arguments: --se
A word of caution
The method _get_option_tuples is prefixed with _, which typically means a private method in python. And it is not a good idea to override a private.
Another way for Python 2.7. Let's get clunky! Say you want to recognize --dog without abbreviation.
p = argparse.ArgumentParser()
p.add_argument('--dog')
p.add_argument('--dox', help=argparse.SUPPRESS, metavar='IGNORE')
By adding a second argument --dox that differs from the argument you want only in the third letter, --d and --do become ambiguous. Therefore, the parser will refuse to recognize them. You would need to add code to catch the resulting exception and process it according to the context in which you are calling parse_args. You might also need to suppress/tweak the help text.
The help=... keeps the argument out of the option list on the default help message (per this), and metavar='IGNORE' is just to make it clear you really aren't doing anything with this option :) .