I set up my argument parser as follows:
parser=argparse.ArgumentParser()
parser.add_argument('--point',help='enter a point (e.g. 2,3,4)')
parser.parse_args('--point=-2,5,6'.split()) #works
parser.parse_args('--point -2,5,6'.split()) #doesn't work :(
Is there any way to tell argparse that strings which match the regular expression r"-\d+.*" are not options but an argument of an option?
Also note that I could do something like this:
parser.add_argument('--point',nargs='*')
parser.parse_args('--point -2 5 6'.split())
but that's not really how I want it to work.
I think preprocessing sys.argv is the most straightforward way here. Consider for example:
import argparse, re
parser=argparse.ArgumentParser()
parser.add_argument('--point',help='enter a point (e.g. 2,3,4)')
args = '--point -2,5,6'.split() # or sys.argv
is_list = re.compile(r'^-?[\d,.]+$')
args = ['"%s"' % x if is_list.match(x) else x for x in args]
print parser.parse_args(args)
This returns Namespace(point='"-2,5,6"') which should be easy to parse.
You could change the prefix char so - is no longer recognized as indicating the start of an argument. It does look a bit weird but it is useful when negative numbers may appear in the arguments.
import argparse
parser=argparse.ArgumentParser(prefix_chars = '#')
parser.add_argument('##point',help='enter a point (e.g. 2,3,4)')
args = parser.parse_args('##point=-2,5,6'.split()) #works
print(args)
# Namespace(point='-2,5,6')
args = parser.parse_args('##point -2,5,6'.split()) #work also
print(args)
# Namespace(point='-2,5,6')
If you don't mind messing around with argparse internals, argparse already does something very similar to what I want to do. One of the classes that ArgumentParser inherits from has this line in __init__
import re as _re
...
self._negative_number_matcher = _re.compile(r'^-\d+$|^-\d*\.\d+$')
So to get my example to work, we just need to swap out an appropriate regex...
parser._negative_number_matcher = re.compile(r'^-\d+|^-\d*\.\d+')
Usually I'm not very much in favor of messing around with the internals of a class when they're prefixed by an underscore (as it is implementation dependent and liable to change) -- However, in this case, I think it's probably ok because:
If argparse changes that variable name, it doesn't hurt, we're just back to the case where "--print -2,3,4" doesn't work again.
I can't think of any better way to determine if something is a number than regex (I suppose they could try to cast to a float and catch the exception, but if they did that, they wouldn't have a variable named _negative_number_matcher anymore and again, argparse would still work fine except in this corner case where it doesn't do what I want)
Related
I have a use case where I'd like the user to be able to provide, as an argument to argparse, EITHER a single string OR a filename where each line has a string.
Assume the user launches ./myscript.py -i foobar
The logical flow I'm looking for is something like this:
The script determines whether the string foobar is a readable file.
IF it is indeed a readable file, we call some function from the script, passing each line in foobar as an argument to that function. If foobar is not a readable file, we call the same function but just use the string foobar as the argument and return.
I have no ability to guarantee that a filename argument will have a specific extension (or even an extension at all).
Is there a more pythonic way to do this OTHER than just coding up the logic exactly as I've described above? I looked through the argparse tutorial and didn't see anything, but it also seems reasonable to think that there would be some specific hooks for filenames as arguments, so I figured I'd ask.
A way would be:
Let's say that you have created a parser like this:
parser.add_argument('-i',
help='...',
type=function)
Where type points to the function which will be an outer function that evaluates the input of the user and decides if it is a string or a filename
More information about type you can find in the documentation.
Here is a minimal example that demonstrates this use of type:
parser.add_argument('-d','--directory',
type=Val_dir,
help='...')
# ....
def Val_dir(dir):
if not os.path.isdir(dir):
raise argparse.ArgumentTypeError('The directory you specified does not seem to exist!')
else:
return dir
The above example shows that with type we can control the input at parsing time. Of course in your case the function would implement another logic - evaluate if the input is a string or a filename.
This doesn't look like an argparse problem, since all you want from it is a string. That string can be a filename or a function argument. To a parser these will look the same. Also argparse isn't normally used to run functions. It is used to parse the commandline. Your code determines what to do with that information.
So here's a script (untested) that I think does your task:
import argparse
def somefunction(*args):
print(args)
if __name__=='__main__':
parser=argparse.ArgumentParser()
parser.add_argument('-i','--input')
args = parser.parse_args()
try:
with open(args.input) as f:
lines = f.read()
somefunction(*lines)
# or
# for line in lines:
# somefuncion(line.strip())
except:
somefunction(arg.input)
argparse just provides the args.input string. It's the try/except block that determines how it is used.
================
Here's a prefix char approach:
parser=argparse.ArgumentParser(fromfile_prefix_chars='#',
description="use <prog -i #filename> to load values from file")
parser.add_argument('-i','--inputs')
args=parser.parse_args()
for arg in args.inputs:
somefunction(arg)
this is supposed to work with a file like:
one
two
three
https://docs.python.org/3/library/argparse.html#fromfile-prefix-chars
It appears to me that there's no easy way to use the RawDescriptionHelpFormatter in the argparse module without either violating PEP8 or cluttering your namespace.
Here is the most obvious way to format it:
parser = argparse.ArgumentParser(prog='PROG',
....
formatter_class=argparse.RawDescriptionHelpFormatter)
This violates the stipulation that lines should not exceed 80 characters
Here's how the example in the argparse documentation looks (spoiler: this is actually correct; see comments below):
parser = argparse.ArgumentParser(
prog='PROG',
formatter_class=argparse.RawDescriptionHelpFormatter,
....
This violates PEP8 E128 regarding the indentation of continuation lines.
Here'another possibility:
parser = argparse.ArgumentParser(
prog='PROG',
formatter_class=
argparse.RawDescriptionHelpFormatter,
....
This violates PEP8 E251 regarding spaces around = for keyward arguments.
(Of course, this doesn't even address the fact that my character-count for the line assumes that the parser token starts on the first column, which is the best case scenario; what if we want to create a parser inside a class and/or a function?)
So the only remaining alternative, as far as I can tell, is to either clutter the namespace:
from argparse import RawDescriptionHelpFormatter, ArgumentParser
...or use a silly temporary variable (which also clutters the namespace):
rawformatter = argparse.RawDescriptionHelpFormatter
parser = argparse.ArgumentParser(prog='PROG',
....
formatter_class=rawformatter)
Am I missing something? I guess having RawDescriptionHelpFormatter and ArgumentParser directly in the current namespace isn't a big deal, but this seems like an unnecessary frustration.
Your second example looks fine to me, and seems to match the "# Hanging indents should add a level." example here: http://legacy.python.org/dev/peps/pep-0008/#indentation
Also seems to tally with this similar question/answer: What is PEP8's E128: continuation line under-indented for visual indent?
A couple of other variations:
from argparse import RawDescriptionHelpFormatter as formatter
parser = argparse.ArgumentFormatter(prog='PROG')
# you can reassign a parser attribute after initialization
parser.formatter_class = formatter
But there are other inputs to ArgumentParser that may be long enough to require wrapping or assignment to separate variables.
usage = 'PROG [-h] --foo FOO BAR etc'
description = """\
This is a long multiline description
that may require dedenting.
"""
description = textwrap.dedent(description)
parser=Argparse(usage=usage, description=description, formatter_class=formatter)
Take a look at test_argparse.py to see the many ways that a long and multifaceted parser can be defined.
http://bugs.python.org/issue13023 raises the issue of what if you wanted several formatter modifications, e.g.:
This means we can either pass argparse.RawDescriptionHelpFormatter or argparse.ArgumentDefaultsHelpFormatter, but not both.
The recommended solution is to subclass the formatter:
class MyFormatter(argparse.RawDescriptionHelpFormatter,
argparse.ArgumentDefaultsHelpformatter):
pass
Another tactic to keep the namespace clean is to wrap the parser definition in a function or module.
http://ipython.org/ipython-doc/2/api/generated/IPython.core.magic_arguments.html
is an example of how IPython wraps argparse to make new API for its users.
Another parser built on argparse, plac first builds a cfg dictionary:
https://code.google.com/p/plac/source/browse/plac_core.py
def pconf(obj):
...
cfg = dict(description=obj.__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
...
return cfg
def parser_from(obj, **confparams):
...
conf = pconf(obj).copy()
conf.update(confparams)
parser = ArgumentParser(**conf)
I am trying to make my help strings helpful. To do this I have a function Function() with a doc string something like
def Function(x):
""" First line describing what Function does
Keyword Arguments
x = float -- A description of what x does that may be long
"""
Having done this I think have something like this at the end
def parse_command_line(argvs):
parser = optparse.OptionParser()
parser.add_option("-f","--Function", help=Function.__doc__,metavar="Bs" )
(options,arguments) = parser.parse_args(argvs)
return options, arguments
options, arguments = parse_command_line(sys.argv)
The trouble occurs when calling the program with -h or --help The output is line wrapped by OptParse, this means the KeyWord arguments are not started on a new line, is it possible to stop OptParse from wrapping the output or is there a better way to do this?
In case of interest, I have written two such formatter classes for argparse. The first supports much of the mediaWiki, MarkDown, and POD syntaxes (even intermixed):
import MarkupHelpFormatter
MarkupHelpFormatter.InputOptions["mediawiki"] = True
parser = argparse.ArgumentParser(
description="""...your text, with mediawiki markup...""",
epilog='...',
formatter_class=MarkupHelpFormatter.MarkupHelpFormatter
)
The other is called "ParagraphHelpFormatter". It merely wraps text like the default argparse formatter, except that it respects blank lines.
Both are at http://derose.net/steve/utilities/PY/MarkupHelpFormatter.py, and
licensed as CCLI Attribution-Share-alike. They format for ANSI terminal
interfaces. Not highly polished (for example, auto-numbering is unfinished), but you may find them helpful.
argparse provides you with a formatter for raw formatting, ie your lines wont get wrapped
parser = argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter)
optparse also lets you set the formatter.. I suppose you can write your own formatter, but theres nothing provided that i know of
argparse uses per default abbreviation in unambiguous cases.
I don't want abbreviation and I'd like to disable it.
But didn't find it in the documentation.
Is it possible?
Example:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--send', action='store_true')
parser.parse_args(['--se']) # returns Namespace(send=True)
But I want it only to be true when the full parameter is supplied. To prevent user errors.
UPDATE:
I created a ticket at python bugtracker after Vikas answer. And it already has been processed.
As of Python 3.5.0 you can disable abbreviations by initiating the ArgumentParser with the following:
parser = argparse.ArgumentParser(allow_abbrev=False)
Also see the documentation.
No, apparently this is not possible. At least in Python 2.7.2.
First, I took a look into the documentation - to no avail.
Then I opened the Lib\argparse.py and looked through the source code. Omitting a lot of details, it seems that each argument is parsed by a regular expression like this (argparse:2152):
# allow one or more arguments
elif nargs == ONE_OR_MORE:
nargs_pattern = '(-*A[A-]*)'
This regex will successfully parse both '-' and '--', so we have no control over the short and long arguments. Other regexes use the -* construct too, so it does not depend on the type of the parameter (no sub-arguments, 1 sub-argument etc).
Later in the code double dashes are converted to one dash (only for non-optional args), again, without any flags to control by user:
# if this is an optional action, -- is not allowed
if action.option_strings:
nargs_pattern = nargs_pattern.replace('-*', '')
nargs_pattern = nargs_pattern.replace('-', '')
No, well not without ugly hacks.
The code snippet #Vladimir posted, i suppose that is not what you are looking for. The actual code that is doing this is:
def _get_option_tuples(self, option_string):
...
if option_string.startswith(option_prefix):
...
See the check is startswith not ==.
And you can always extend argparse.ArgumentParser to provide your own _get_option_tuples(self, option_string) to change this behavior. I just did by replacing two occurrence of option_string.startswith(option_prefix) to option_string == option_prefix and:
>>> parser = my_argparse.MyArgparse
>>> parser = my_argparse.MyArgparse()
>>> parser.add_argument('--send', action='store_true')
_StoreTrueAction(option_strings=['--send'], dest='send', nargs=0, const=True, default=False, type=None, choices=None, help=None, metavar=None)
>>> parser.parse_args(['--se'])
usage: [-h] [--send]
: error: unrecognized arguments: --se
A word of caution
The method _get_option_tuples is prefixed with _, which typically means a private method in python. And it is not a good idea to override a private.
Another way for Python 2.7. Let's get clunky! Say you want to recognize --dog without abbreviation.
p = argparse.ArgumentParser()
p.add_argument('--dog')
p.add_argument('--dox', help=argparse.SUPPRESS, metavar='IGNORE')
By adding a second argument --dox that differs from the argument you want only in the third letter, --d and --do become ambiguous. Therefore, the parser will refuse to recognize them. You would need to add code to catch the resulting exception and process it according to the context in which you are calling parse_args. You might also need to suppress/tweak the help text.
The help=... keeps the argument out of the option list on the default help message (per this), and metavar='IGNORE' is just to make it clear you really aren't doing anything with this option :) .
Using python optparse.py, is there a way to work out whether a specific option value was set from the command line or from the default value.
Ideally I would like to have a dict just like defaults, but containing the options actually supplied from command line
I know that you could compare the value for each option with defaults, but this wouldn't distinguish a value was passed through command line which matched the default.
Thanks!
EDIT
Sorry my original phrasing wasn't very clear.
I have a large number of scripts which are called from batch files. For audit purposes, I would like to report on the options being passed, and whether they are passed from command line, default, or some other means, to a log file.
Using defaults you can tell whether an option matches a default value, but that still doesn't tell you whether it was actually supplied from command line. This can be relevant: if an option is passed from command line and agrees with the default, if you then change the default in the code the script will still get the same value.
To me it would feel quite natural to have an equivalent to defaults, containing the values actually supplied.
To make the question concrete, in this example:
>>> sys.argv = ['myscript.py','-a','xxx']
>>> import optparse
>>> parser = optparse.OptionParser()
>>> parser.add_option('-a', default = 'xxx')
>>> parser.add_option('-b', default = 'yyy')
How do I know that option a was passed from command line. Is the only way to parse the command line manually?
(I know this is a fairly minor point, but I thought it would be worth asking in case I'm missing smthing on optparse)
Thanks again
Instead of the below boilerplate code:
opts, args = parser.parse_args()
if you use the below code, you will have an additional values object opts_no_defaults with options that were explicitly specified by the user:
opts_no_defaults = optparse.Values()
__, args = parser.parse_args(values=opts_no_defaults)
opts = Values(parser.get_default_values().__dict__)
opts._update_careful(opts_no_defaults.__dict__)
At the end, opts should be identical as in the initial boilerplate code.
print opts_no_defaults.__dict__
print opts.__dict__
for opt in parser._get_all_options():
if opt.dest:
print "Value of %s: %s" % (opt._long_opts[0], getattr(opts, opt.dest))
print "Is %s specified by user? %s" % (opt._long_opts[0], hasattr(opt_no_defaults, opt.dest))
Not knowing you code is impossible to give the better answer, but...
simply don't pass defaults to the parser and check for None values. A None value is a default for optparse lib so you can retrieve your own default and act as usually;
extend optparse to specialize it.
I don't know your program but usually it is not a good design changing behavior when the configuration is the same.
def is_opt_provided (parser, dest):
if any (opt.dest == dest and (opt._long_opts[0] in sys.argv[1:] or opt._short_opts[0] in sys.argv[1:]) for opt in parser._get_all_options()):
return True
return False
Usage:
parser = OptionsParser()
parser.add_option('-o', '--opt', dest='opt_var', ...)
if is_opt_provided(parser, 'opt_var'):
print "Option -o or --opt has been provided"
It would be great if Python maintainers included the suggested function to OptionParser class.