I want to use argparse to parse command lines of form "arg=val"
For example, the usage would be:
script.py conf_dir=/tmp/good_conf
To achieve it, I am doing this:
desc = "details"
parser = argparse.ArgumentParser(description=desc, add_help=False)
args = parser.add_argument("conf_dir")
args = parser.parse_args("conf_dir=FOO".split())
args = parser.parse_args()
print args.conf_dir
But, the problem is that, on invocation of the script with:
python script.py conf_dir=/tmp/good_conf
I get:
conf_dir=/tmp/good_conf
Where as I expect
/tmp/good_conf
So, the question is: Can I use argparse to parse cmd line, which contains name value pairs?
Any hints?
Edit: The reason I want to do this and not some thing like --conf_dir=/tmp/good_dir is because there are other tools (written in other language), which uses conf_dir=/tmp/good_dir style of arguments. To maintain consistency, I was to parse args in this way.
You need a custom action
class StoreNameValuePair(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
n, v = values.split('=', 1)
setattr(namespace, n, v)
args = parser.add_argument("conf_dir", action=StoreNameValuePair)
As per the documentation, argparse doesn't natively let you have unprefixed options like that. If you omit the leading -, it assumes you are describing a positional argument and expects it to be provided as:
python script.py /tmp/good_conf
If you want it to be optional, it needs to be correctly marked as a flag by calling it --conf_dir, and invoking the script like:
python script.py --conf_dir=/tmp/good_conf
However, to accept name-value pairs, you can implement a custom action. In combination with nargs, such an action could accept an arbitrary number of name-value pairs and store them on the argument parsing result object.
#chepner This is great. I improved this to support multiple args as well and store the result as dict:
class StoreDict(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
kv={}
if not isinstance(values, (list,)):
values=(values,)
for value in values:
n, v = value.split('=')
kv[n]=v
setattr(namespace, self.dest, kv)
The usual way to put name value pairs on the command line is with options. I.e. you would use
python script.py --confdir=/tmp/good_conf
argparse can certainly handle that case. See the docs at:
http://docs.python.org/library/argparse.html#option-value-syntax
Related
Working with Python and argparse, trying to accomplish the following:
$> my_app
Namespace(positional=None)
$> my_app file.txt somedir
Namespace(positional=['file.txt', 'somedir'])
i.e., a positional argument of type list, whose default is None. I would expect the following code to accomplish this:
p = argparse.ArgumentParser()
p.add_argument("positional", nargs='*', default=None)
print(p.parse_args())
But I get:
$> my_app
Namespace(positional=[])
$> my_app file.txt somedir
Namespace(positional=['file.txt', 'somedir'])
The rest of my code uses None as defaults for lists. If None is provided, the code select a sensible default. Thus passing [] is not an option.
The behavior of argparse does rather feel like a bug, but maybe I'm missing something. Any thoughts?
Answer (thx hpaulj) seems to be "as intended" to which I would add "completely unintuitively".
You can achieve the desired behavior like this
import argparse
p = argparse.ArgumentParser()
p.add_argument("positional", nargs='*', default=argparse.SUPPRESS)
print(p.parse_args(namespace=argparse.Namespace(positional=None)))
This prevents the arguments from appearing at all in the namespace which on turn causes the default namespace to take over.
Your case is handled in
def _get_values(self, action, arg_strings):
...
# when nargs='*' on a positional, if there were no command-line
# args, use the default if it is anything other than None
elif (not arg_strings and action.nargs == ZERO_OR_MORE and
not action.option_strings):
if action.default is not None:
value = action.default
self._check_value(action, value)
else:
# since arg_strings is always [] at this point
# there is no need to use self._check_value(action, value)
value = arg_strings
Normally the default is placed in the Namespace at the start of parsing. During parsing that default is overwritten with value(s) provided by the user. optionals are 'seen' when the right flag is used. positionals are 'seen' when the right number of strings are present. Since '*' and '?' accept 0 strings, they will always be 'seen'. Thus their defaults require special handling, if at all.
So, I have a python script for parsing and plotting data from text files. Argument handling is done with argparse module. The problem is, that some arguments are optional, e.g. one of the is used to add text annotations on the plot. This argument is sent to plotting function via **kwargs. My question is - what is the most pythonic way to handle those optional arguments? Some pseudo code here:
parser = argparse.ArgumentParser()
...
parser.add_argument("-o", "--options", nargs="+", help="additional options")
args = parser.parse_args()
...
def some_function(arguments, **kwargs):
doing something with kwargs['options']
return something
...
arguments = ...
some_function(arguments, options=args.options)
If options are not specified by default None value is assigned. And it causes some problems. What is more pythonic - somehow check 'options' within some_function? Or maybe parse arguments before some_function is called?
You can just provide an explicit empty list as the default.
parser.add_argument("-o", "--options", nargs="+", default=[])
use get and set a default value if the key is not found in the dict
def some_function(arguments, **kwargs):
something = kwargs.get('options', 'Not found')
return something
or an if statement
if 'option' in kwargs:
pass # do something
Is it possible to use argparse to capture an arbitrary set of optional arguments?
For example both the following should be accepted as inputs:
python script.py required_arg1 --var1 value1 --var2 value2 --var3 value3
python script.py required_arg1 --varA valueA --var2 value2 --varB valueB
a priori I don't know what optional arguments would be specified receive but would handle them accordingly.
This is kind of a hackish way, but it works well:
Check, which arguments are not added and add them
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("foo")
parser.add_argument("-bar", type=int)
# parser can have any arguments, whatever you want!
parsed, unknown = parser.parse_known_args() # this is an 'internal' method
# which returns 'parsed', the same as what parse_args() would return
# and 'unknown', the remainder of that
# the difference to parse_args() is that it does not exit when it finds redundant arguments
for arg in unknown:
if arg.startswith(("-", "--")):
# you can pass any arguments to add_argument
parser.add_argument(arg.split('=')[0], type=<your type>, ...)
args = parser.parse_args()
For example:
python3 arbitrary_parser.py ha -bar 12 -extra1 value1 -extra2 value2
Then the result would be
args = Namespace(bar=12, foo='ha', extra1='value1' extra2='value2')
Possible? possibly, but I wouldn't recommend it. argparse is the not best tool for parsing this kind of input, or conversely, this a poor argument specification from an argparse perspective.
Have you thought about what the usage line should look like? How would explain this to your users?
How would you parse this working from sys.argv directly? It looks like you could collect 3 pieces:
prog = sys.argv[0]
arg1 = sys.argv[1]
keys = sys.argv[2::2]
# maybe strip -- off each
values = sys.argv[3::2]
kvdict = {k:v for k, v in zip(keys, values)}
There are other SO questions asking about generic key:value pairs. Things like:
--args key1:value1 key2:value2
This can be handled with nargs='*' and an action that splits each input string on : (or =) and stores things by key.
Your requirement is least amenable to argparse use because it requires bypassing the whole idea of matching argument flags with strings in argv. It requires, some how, turning off all the normal argparse parsing.
Looks like I suggested the same thing a couple of years ago
Parse non-pre-defined argument
or earlier
Using argparse to parse arguments of form "arg= val"
Here is my argparse sample say sample.py
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-p", nargs="+", help="Stuff")
args = parser.parse_args()
print args
Python - 2.7.3
I expect that the user supplies a list of arguments separated by spaces after the -p option. For example, if you run
$ sample.py -p x y
Namespace(p=['x', 'y'])
But my problem is that when you run
$ sample.py -p x -p y
Namespace(p=['y'])
Which is neither here nor there. I would like one of the following
Throw an exception to the user asking him to not use -p twice instead just supply them as one argument
Just assume it is the same option and produce a list of ['x','y'].
I can see that python 2.7 is doing neither of them which confuses me. Can I get python to do one of the two behaviours documented above?
Note: python 3.8 adds an action="extend" which will create the desired list of ['x','y']
To produce a list of ['x','y'] use action='append'. Actually it gives
Namespace(p=[['x'], ['y']])
For each -p it gives a list ['x'] as dictated by nargs='+', but append means, add that value to what the Namespace already has. The default action just sets the value, e.g. NS['p']=['x']. I'd suggest reviewing the action paragraph in the docs.
optionals allow repeated use by design. It enables actions like append and count. Usually users don't expect to use them repeatedly, or are happy with the last value. positionals (without the -flag) cannot be repeated (except as allowed by nargs).
How to add optional or once arguments? has some suggestions on how to create a 'no repeats' argument. One is to create a custom action class.
I ran into the same issue. I decided to go with the custom action route as suggested by mgilson.
import argparse
class ExtendAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
if getattr(namespace, self.dest, None) is None:
setattr(namespace, self.dest, [])
getattr(namespace, self.dest).extend(values)
parser = argparse.ArgumentParser()
parser.add_argument("-p", nargs="+", help="Stuff", action=ExtendAction)
args = parser.parse_args()
print args
This results in
$ ./sample.py -p x -p y -p z w
Namespace(p=['x', 'y', 'z', 'w'])
Still, it would have been much neater if there was an action='extend' option in the library by default.
This first link has the same question in the first section, but it is unanswered
(python argparse: parameter=value). And this second question is similar, but I can't seem to get it working for my particular case
( Using argparse to parse arguments of form "arg= val").
So my situation is this -- I am re-writing a Python wrapper which is used by many other scripts (I would prefer not to modify these other scripts). Currently, the Python wrapper is called with command line arguments of the form --key=value for a number of different arguments, but was parsed manually. I would like to parse them with argparse.
N.B. The argument names are unwieldy, so I am renaming using the dest option in add_argument.
parser = argparse.ArgumentParser(description='Wrappin Ronnie Reagan')
parser.add_argument("--veryLongArgName1", nargs=1, dest="arg1", required=True)
parser.add_argument("--veryLongArgName2", nargs=1, dest="arg2")
parser.add_argument("--veryLongArgName3", nargs=1, dest="arg3")
userOpts = vars(parser.parse_args())
Which, while apparently parsing the passed command lines correctly, displays this as the help:
usage: testing_argsparse.py [-h] --veryLongArgName1 ARG1
[--veryLongArgName2 ARG2]
[--veryLongArgName3 ARG3]
testing_argsparse.py: error: argument --veryLongArgName1 is required
But what I want is that all parameters are specified with the --key=value format, not --key value. i.e.
usage: testing_argsparse.py [-h] --veryLongArgName1=ARG1
[--veryLongArgName2=ARG2]
[--veryLongArgName3=ARG3]
testing_argsparse.py: error: argument --veryLongArgName1 is required
testing_argsparse.py --veryLongArgName1=foo
works. argparse module accepts both --veryLongArgName1=foo and --veryLongArgName1 foo formats.
What exact command line arguments are you trying to pass to argparse that's causing it to not work?
A little late but for anyone with a similar request as the OP you could use a custom HelpFormatter.
class ArgFormatter(argparse.HelpFormatter):
def _format_args(self, *args):
result = super(ArgFormatter, self)._format_args(*args)
return result and '%%%' + result
def _format_actions_usage(self, *args):
result = super(ArgFormatter, self)._format_actions_usage(*args)
return result and result.replace(' %%%', '=')
This can then be passed to ArgumentParser to give the wanted behavior.
parser = argparse.ArgumentParser(
description='Wrappin Ronnie Reagan',
formatter_class=ArgFormatter)
This intercepts the args (ARG1, ARG2, ...) and adds a custom prefix which is later replaced (along with the unwanted space) for an = symbol. The and in the return statements makes sure to only modify the result if it's non-empty.