So, I have a python script for parsing and plotting data from text files. Argument handling is done with argparse module. The problem is, that some arguments are optional, e.g. one of the is used to add text annotations on the plot. This argument is sent to plotting function via **kwargs. My question is - what is the most pythonic way to handle those optional arguments? Some pseudo code here:
parser = argparse.ArgumentParser()
...
parser.add_argument("-o", "--options", nargs="+", help="additional options")
args = parser.parse_args()
...
def some_function(arguments, **kwargs):
doing something with kwargs['options']
return something
...
arguments = ...
some_function(arguments, options=args.options)
If options are not specified by default None value is assigned. And it causes some problems. What is more pythonic - somehow check 'options' within some_function? Or maybe parse arguments before some_function is called?
You can just provide an explicit empty list as the default.
parser.add_argument("-o", "--options", nargs="+", default=[])
use get and set a default value if the key is not found in the dict
def some_function(arguments, **kwargs):
something = kwargs.get('options', 'Not found')
return something
or an if statement
if 'option' in kwargs:
pass # do something
Related
Is it possible to tell argparse to give the same errors on default argument values as it would on user-specified argument values?
For example, the following will not result in any error:
parser = argparse.ArgumentParser()
parser.add_argument('--choice', choices=['a', 'b', 'c'], default='invalid')
args = vars(parser.parse_args()) # args = {'choice': 'invalid'}
whereas omitting the default, and having the user specify --choice=invalid on the command-line will result in an error (as expected).
Reason for asking is that I would like to have the user to be able to specify default command-line options in a JSON file which are then set using ArgumentParser.set_defaults(), but unfortunately the behaviour demonstrated above prevents these user-specified defaults from being validated.
Update: argparse is inconsistent and I now consider the behavior above to be a bug. The following does trigger an error:
parser = argparse.ArgumentParser()
parser.add_argument('--num', type=int, default='foo')
args = parser.parse_args() # triggers exception in case --num is not
# specified on the command-line
I have opened a bug report for this: https://github.com/python/cpython/issues/100949
I took the time to dig into the source code, and what is happening is that a check is only happening for arguments you gave on the command line. The only way to enforce a check, in my opinion, is to subclass ArgumentParser and have it do the check when you add the argument:
class ValidatingArgumentParser(argparse.ArgumentParser):
def add_argument(self, *args, **kwargs):
super().add_argument(*args, **kwargs)
self._check_value(self._actions[-1],kwargs['default'])
No. Explicit arguments need to be validated because they originate from outside the source code. Default values originate in the source code, so it's the job of the programmer, not the argument parser, to ensure they are valid.
(This is the difference between validation and debugging.)
(Using set_defaults on unvalidated user input still falls under the purview of debugging, as it's not the argument parser itself adding the default values, but the programmer.)
I have to read a CLI argument for my python script and then pass that input argument by the user to a method. I want to provide user with a list of valid arguments that she can enter, sort of when you run the script with an invalid argument or with no argument passed- it shows you that you can only enter these arguments.
I know we can achieve this with argparse module, but argparse in my case will be an overkill- since I just want to pass whatever (valid)argument the user gave to a method. How should I achieve this?
parser.add_argument('--system-status', '-st', metavar='<host>', type=str)
if args.system-status:
method(args.system-status)
And the same stuff repeated for 5 other variables?
The argparse lib is the way to go, if you dont want to check for each parameter you could use something like this:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--foo')
parser.add_argument('--bar')
parser.add_argument('--baz')
args = parser.parse_args()
def f(foo=None, bar=None, baz=None, **kwargs):
print(foo)
print(bar)
print(baz)
f(**args.__dict__)
**args.__dict__ will turn args in a dictionary turn it into kwargs.
If you really absolutly don't want to use argparse then sys.argv will give you the list of parameters passed to the function.
This first link has the same question in the first section, but it is unanswered
(python argparse: parameter=value). And this second question is similar, but I can't seem to get it working for my particular case
( Using argparse to parse arguments of form "arg= val").
So my situation is this -- I am re-writing a Python wrapper which is used by many other scripts (I would prefer not to modify these other scripts). Currently, the Python wrapper is called with command line arguments of the form --key=value for a number of different arguments, but was parsed manually. I would like to parse them with argparse.
N.B. The argument names are unwieldy, so I am renaming using the dest option in add_argument.
parser = argparse.ArgumentParser(description='Wrappin Ronnie Reagan')
parser.add_argument("--veryLongArgName1", nargs=1, dest="arg1", required=True)
parser.add_argument("--veryLongArgName2", nargs=1, dest="arg2")
parser.add_argument("--veryLongArgName3", nargs=1, dest="arg3")
userOpts = vars(parser.parse_args())
Which, while apparently parsing the passed command lines correctly, displays this as the help:
usage: testing_argsparse.py [-h] --veryLongArgName1 ARG1
[--veryLongArgName2 ARG2]
[--veryLongArgName3 ARG3]
testing_argsparse.py: error: argument --veryLongArgName1 is required
But what I want is that all parameters are specified with the --key=value format, not --key value. i.e.
usage: testing_argsparse.py [-h] --veryLongArgName1=ARG1
[--veryLongArgName2=ARG2]
[--veryLongArgName3=ARG3]
testing_argsparse.py: error: argument --veryLongArgName1 is required
testing_argsparse.py --veryLongArgName1=foo
works. argparse module accepts both --veryLongArgName1=foo and --veryLongArgName1 foo formats.
What exact command line arguments are you trying to pass to argparse that's causing it to not work?
A little late but for anyone with a similar request as the OP you could use a custom HelpFormatter.
class ArgFormatter(argparse.HelpFormatter):
def _format_args(self, *args):
result = super(ArgFormatter, self)._format_args(*args)
return result and '%%%' + result
def _format_actions_usage(self, *args):
result = super(ArgFormatter, self)._format_actions_usage(*args)
return result and result.replace(' %%%', '=')
This can then be passed to ArgumentParser to give the wanted behavior.
parser = argparse.ArgumentParser(
description='Wrappin Ronnie Reagan',
formatter_class=ArgFormatter)
This intercepts the args (ARG1, ARG2, ...) and adds a custom prefix which is later replaced (along with the unwanted space) for an = symbol. The and in the return statements makes sure to only modify the result if it's non-empty.
I'm trying to make argparse ignore the fact that two normally required positional arguments shouldn't be evaluated when an optional argument (-l) is specified.
Basically I'm trying to replicate the behavior of --help: when you specify the -h, all missing required arguments are ignored.
Example code:
parser = argparse.ArgumentParser(description="Foo bar baz")
parser.add_argument('arg1', help='arg1 is a positional argument that does this')
parser.add_argument('arg2', help='arg2 is a positional argument that does this')
parser.add_argument('-l', '--list', dest='list', help='this is an optional argument that prints stuff')
options, args = parser.parse_args()
if options.list:
print "I list stuff"
And of course, if I run it now, I get :
error: too few arguments
I tried different things like nargs='?', but couldn't get anything working.
This question is quite similar but wasn't answered.
Unfortunately, argparse isn't quite flexible enough for this. The best you can do is to make arg1 and arg2 optional using nargs="?" and check yourself whether they are given if needed.
The internal help action is implemented by printing the help message and exiting the program as soon as -h or --help are encountered on the command line. You could write a similar action yourself, something like
class MyAction(argparse.Action):
def __call__(self, parser, values, namespace, option_string):
print "Whatever"
parser.exit()
(Warning: untested code!)
There are definite downsides to the latter approac, though. The help message will unconditionally show arg1 and arg2 as compulsory arguments. And parsing the command line simply stops when encountering -l or --list, ignoring any further arguments. This behaviour is quite acceptable for --help, but is less than desirable for other options.
I ran into this issue and decided to use subcommands. Subcommands might be overkill, but if you find your program not using some of the positional arguments in many instances (as I did), then subcommands might be a good solution.
For your given example, I'd use something like the following:
parser = argparse.ArgumentParser(description="Foo bar baz")
subparsers = parser.add_subparsers(description='available subcommands')
parser_main = subparsers.add_parser('<main_command_name>')
parser_main.add_argument('arg1', help='arg1 is a positional argument that does this')
parser_main.add_argument('arg2', help='arg2 is a positional argument that does this')
parser_list = subparsers.add_parser('list', help='this is a subcommand that prints stuff')
options, args = parser.parse_args()
I left out some details that you might want to include (like set_defaults(func=list)), which are mentioned in the argparse documentation.
I think the nargs='*' is helpful.
Positional arguments is ignorable, then you can use if to check the positional arguments is true or false.
http://docs.python.org/library/argparse.html#nargs
The cleanest approach I've been able to find so far is to split the parsing into two stages. First check for -l/--list:
parser = argparse.ArgumentParser(description="Foo bar baz")
parser.add_argument('-l', '--list', dest='list', action='store_true',
help='this is an optional argument that prints stuff')
options, remainder = parser.parse_known_args()
Now, since you used parse_known_args, you won't get an error up to here, and you can decide what to do with the remainder of the arguments:
if options.list:
print "I list stuff"
else:
parser = argparse.ArgumentParser(add_help=False)
parser.add_argument('arg1', help='arg1 is a positional argument that does this')
parser.add_argument('arg2', help='arg2 is a positional argument that does this')
options = parser.parse_args(remainder)
You may want to set the usage option in the first parser to make the help string a bit nicer.
I may have found a solution here. True, it is a dirty hack, but it works.
Note: all the following applies to Python 3.3.2.
As per the answer here, parse_args checks which actions are required and throws an error if any of them are missing. I propose to override this behavior.
By subclassing ArgumentParser we can define a new ArgumentParser.error method (original here) that will check whether the error was thrown because some arguments are missing and take necessary action. Code follows:
import argparse
import sys
from gettext import gettext as _
class ArgumentParser(argparse.ArgumentParser):
skip_list = []
def error(self, message):
# Let's see if we are missing arguments
if message.startswith('the following arguments are required:'):
missingArgs = message.split('required: ')[1].split(', ')
newArgs = [] # Creating a list of whatever we should not skip but is missing
for arg in missingArgs:
if arg not in self.skip_list:
newArgs.append(arg)
else:
self.skip_list.remove(arg) # No need for it anymore
if len(newArgs) == 0:
return # WARNING! UNTESTED! MAY LEAD TO SPACETIME MELTDOWN!
else: # Some required stuff is still missing, so we show a corrected error message
message = _('the following arguments are required: %s') % ', '.join(newArgs)
self.print_usage(sys.stderr) # Original method behavior
args = {'prog': self.prog, 'message': message}
self.exit(2, _('%(prog)s: error: %(message)s\n') % args)
The new method first checks whether the error is because arguments are missing from the command line (see here for the code that generates the error). If so, the method gets the names of the arguments from the error message and puts them into a list (missingArgs).
Then, we iterate over this list and check which arguments should be skipped, and which are still required. To determine which arguments to skip, we compare them against skip_list. It is a field in our ArgumentParser subclass that should contain the names of the arguments to skip even when they are required by the parser. Please note that arguments that happen to be in skip_list are removed from it when they are found.
If there are still required arguments that are missing from the command line, the method throws a corrected error message. If all the missing arguments should be skipped, however, the method returns.
WARNING! The original definition of ArgumentParser.error states that if it is overridden in a subclass it should not return, but rather exit or raise an exception. Therefore, what is shown here is potentially unsafe and may cause your program to crash, your computer to catch fire or worse - IT MAY EVAPORATE ALL YOUR TEA. However, it seems like in this particular case (missing required arguments) it is safe to return from the method. But it might not be. You have been warned.
In order to fill skip_list we could use code like this:
class SpecialHelp(argparse._HelpAction):
def __call__(self, parser, namespace, values, option_string=None):
parser.print_help()
print()
for action in parser._actions:
if action != self and action.required:
parser.skip_list.append(argparse._get_action_name(action))
This particular class imitates the built-in help action, but instead of exiting it inserts all the remaining required arguments into skip_list.
Hope my answer helps and best of luck.
It may be ugly, but that is what I normally do.
def print_list():
the_list = ["name1", "name2"]
return "{0}".format(the_list)
...
parser.add_argument("-l", "--list", action='version',
version=print_list(), help="print the list")
I want to use argparse to parse command lines of form "arg=val"
For example, the usage would be:
script.py conf_dir=/tmp/good_conf
To achieve it, I am doing this:
desc = "details"
parser = argparse.ArgumentParser(description=desc, add_help=False)
args = parser.add_argument("conf_dir")
args = parser.parse_args("conf_dir=FOO".split())
args = parser.parse_args()
print args.conf_dir
But, the problem is that, on invocation of the script with:
python script.py conf_dir=/tmp/good_conf
I get:
conf_dir=/tmp/good_conf
Where as I expect
/tmp/good_conf
So, the question is: Can I use argparse to parse cmd line, which contains name value pairs?
Any hints?
Edit: The reason I want to do this and not some thing like --conf_dir=/tmp/good_dir is because there are other tools (written in other language), which uses conf_dir=/tmp/good_dir style of arguments. To maintain consistency, I was to parse args in this way.
You need a custom action
class StoreNameValuePair(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
n, v = values.split('=', 1)
setattr(namespace, n, v)
args = parser.add_argument("conf_dir", action=StoreNameValuePair)
As per the documentation, argparse doesn't natively let you have unprefixed options like that. If you omit the leading -, it assumes you are describing a positional argument and expects it to be provided as:
python script.py /tmp/good_conf
If you want it to be optional, it needs to be correctly marked as a flag by calling it --conf_dir, and invoking the script like:
python script.py --conf_dir=/tmp/good_conf
However, to accept name-value pairs, you can implement a custom action. In combination with nargs, such an action could accept an arbitrary number of name-value pairs and store them on the argument parsing result object.
#chepner This is great. I improved this to support multiple args as well and store the result as dict:
class StoreDict(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
kv={}
if not isinstance(values, (list,)):
values=(values,)
for value in values:
n, v = value.split('=')
kv[n]=v
setattr(namespace, self.dest, kv)
The usual way to put name value pairs on the command line is with options. I.e. you would use
python script.py --confdir=/tmp/good_conf
argparse can certainly handle that case. See the docs at:
http://docs.python.org/library/argparse.html#option-value-syntax