I'm using the python optparse module in my program, and I'm having trouble finding an easy way to parse an option that contains a list of values.
For example:
--groups one,two,three.
I'd like to be able to access these values in a list format as options.groups[]. Is there an optparse option to convert comma separated values into a list? Or do I have to do this manually?
S.Lott's answer has already been accepted, but here's a code sample for the archives:
def foo_callback(option, opt, value, parser):
setattr(parser.values, option.dest, value.split(','))
parser = OptionParser()
parser.add_option('-f', '--foo',
type='string',
action='callback',
callback=foo_callback)
Look at option callbacks. Your callback function can parse the value into a list using a basic optarg.split(',')
Again, just for the sake of archive completeness, expanding the example above:
You can still use "dest" to specify the option name for later access
Default values cannot be used in such cases (see explanation in Triggering callback on default value in optparse)
If you'd like to validate the input, OptionValueError should be thrown from foo_callback
The code (with tiny changes) would then be:
def get_comma_separated_args(option, opt, value, parser):
setattr(parser.values, option.dest, value.split(','))
parser = OptionParser()
parser.add_option('-f', '--foo',
type='string',
action='callback',
callback=get_comma_separated_args,
dest = foo_args_list)
With optparse, to get a list value you can use action 'append':
from optparse import OptionParser
parser = OptionParser()
parser.add_option("--group",
action="append",
dest="my_groups")
(options, args) = parser.parse_args()
print options.my_groups
Then call your program like this:
$ python demo.py --group one --group two --group three
['one', 'two', 'three']
Related
I have a legacy Python application which uses some options in its CLI, using argparse, like:
parser = argparse.ArgumentParser()
parser.add_argument('-f', default='foo')
Now I need to remove this option, since now its value cannot be overwritten by users but it has to assume its default value (say 'foo' in the example). Is there a way to keep the option but prevent it to show up and be overwritten by users (so that I can keep the rest of the code as it is)?
It's not entirely clear what you can do, not with the parser. Can you edit the setup? Or just modify results of parsing?
If you can edit the setup, you could replace the add_argument line with a
parser.setdefaults(f='foo')
https://docs.python.org/3/library/argparse.html#parser-defaults
The -f won't appear in the usage or help, but it will appear in the args
Or you could leave it in, but suppress the help display
parser.add_argument('-f', default='foo', help=argparse.SUPPRESS)
https://docs.python.org/3/library/argparse.html#help
Setting the value after parsing is also fine.
Yes you can do that. After the parser is parsed (args = parser.parse_args()) it is a NameSpace so you can do this:
parser = argparse.ArgumentParser()
args = parser.parse_args()
args.foo = 'foo value'
print(args)
>>> Namespace(OTHER_OPTIONS, foo='foo value')
I assumed that you wanted to add test to your parser, so your original code will still work, but you do not want it as an option for the user.
I think it doesn't make sense for the argparse module to provide this as a standard option, but there are several easy ways to achieve what you want.
The most obvious way is to just overwrite the value after having called parse_args() (as already mentioned in comments and in another answer):
args.f = 'foo'
However, the user may not be aware that the option is not supported anymore and that the application is now assuming the value "foo". Depending on the use case, it might be better to warn the user about this. The argparse module has several options to do this.
Another possibility is to use an Action class to do a little magic. For example, you could print a warning if the user provided an option that is not supported anymore, or even use the built-in error handling.
import argparse
class FooAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
if values != 'foo':
print('Warning: option `-f` has been removed, assuming `-f foo` now')
# Or use the built-in error handling like this:
# parser.error('Option "-f" is not supported anymore.')
# You could override the argument value like this:
# setattr(namespace, self.dest, 'foo')
parser = argparse.ArgumentParser()
parser.add_argument('-f', default='foo', action=FooAction)
args = parser.parse_args()
print('option=%s' % args.f)
You could also just limit the choices to only "foo" and let argparse create an error for other values:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-f', default='foo', choices=['foo'])
args = parser.parse_args()
print('option=%s' % args.f)
Calling python test.py -f bar would then result in:
usage: test.py [-h] [-f {foo}]
test.py: error: argument -f: invalid choice: 'bar' (choose from 'foo')
I have a Python script which takes a lot of arguments.
I currently use a configuration.ini file (read using configparser), but would like to allow the user to override specific arguments using command line.
If I'd only have had two arguments I'd have used something like:
if not arg1:
arg1 = config[section]['arg1']
But I don't want to do that for 30 arguments.
Any easy way to take optional arguments from cmd line, and default to the config file?
Try the following, using dict.update():
import argparse
import configparser
config = configparser.ConfigParser()
config.read('config.ini')
defaults = config['default']
parser = argparse.ArgumentParser()
parser.add_argument('-a', dest='arg1')
parser.add_argument('-b', dest='arg2')
parser.add_argument('-c', dest='arg3')
args = vars(parser.parse_args())
result = dict(defaults)
result.update({k: v for k, v in args.items() if v is not None}) # Update if v is not None
With this example of ini file:
[default]
arg1=val1
arg2=val2
arg3=val3
and
python myargparser.py -a "test"
result would contain:
{'arg1': 'test', 'arg2': 'val2', 'arg3': 'val3'}
You can use a ChainMap from the collections module.
From the doc:
A ChainMap groups multiple dicts or
other mappings together to create a single, updateable view. [...]
Lookups search the underlying mappings successively until a key is
found. [...]
So, you could create
a config dict containing the key-value pairs from your config file,
a cmd_line_args dict containing the ones given on the command line
Then, create a ChainMap:
from collections import ChainMap
combined = ChainMap(cmd_line_args, config)
When you access combined['arg1'], arg1 will first be looked up in the cmd_line_args dict, and if it isn't found there, config[arg1] will be returned.
You can chain as many dicts as you wish, which lets you combine as many levels of defaults as you wish.
You can use parser.set_defaults() to do a bulk override of defaults (so that non-entered arguments get populated from the config). Conveniently, this allows the argparse argument default field to specify a last-resort default for the case where the argument was not provided on the commandline nor in the config. However, the arguments still need to be added to the parser somehow so that the parser is willing to recognize them. Mostly, set_defaults() is useful if you already have an argparse parser set up, but you just want to override the defaults to come from the config if they aren't specified on the commandline:
import argparse
config = dict(
a=11,
b=13,
c=19
)
parser = argparse.ArgumentParser()
# add arguments to parser ...
parser.add_argument('-a', type=int)
parser.add_argument('-b', type=int)
parser.add_argument('-c', type=int)
parser.set_defaults(**config)
args = parser.parse_args()
print(args)
If you weren't planning on already having a parser set up with all available parameters, then you could alternatively use your config to set one up (given the defaults directly for each argument, so no need to do the additional set_defaults() step:
import argparse
parser = argparse.ArgumentParser()
config = dict(
a=11,
b=13,
c=19
)
for key, value in config.items():
parser.add_argument(f'-{key}', default=value, type=type(value))
args = parser.parse_args()
print(args)
I'm using argparse with optional parameter, but I want to avoid having something like this : script.py -a 1 -b -a 2
Here we have twice the optional parameter 'a', and only the second parameter is returned. I want either to get both values or get an error message.
How should I define the argument ?
[Edit]
This is the code:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-a', dest='alpha', action='store', nargs='?')
parser.add_argument('-b', dest='beta', action='store', nargs='?')
params, undefParams = self.parser.parse_known_args()
append action will collect the values from repeated use in a list
parser.add_argument('-a', '--alpha', action='append')
producing an args namespace like:
namespace(alpha=['1','3'], b='4')
After parsing you can check args.alpha, and accept or complain about the number of values. parser.error('repeated -a') can be used to issue an argparse style error message.
You could implement similar functionality in a custom Action class, but that requires understanding the basic structure and operation of such a class. I can't think anything that can be done in an Action that can't just as well be done in the appended list after.
https://stackoverflow.com/a/23032953/901925 is an answer with a no-repeats custom Action.
Why are you using nargs='?' with flagged arguments like this? Without a const parameter this is nearly useless (see the nargs=? section in the docs).
Another similar SO: Python argparse with nargs behaviour incorrect
I have a problem with giving arguments to a function when using callback. I am new to the command line argument scripting in python so go easy on me. Here's my code:
from optparse import OptionParser
import urllib
def google(option, opt_str, value, parser):
print options.f_name
parser = OptionParser()
parser.add_option("-f", "--first", type="string", dest="f_name", help="Supply a first name to search", action="store")
parser.add_option("-l", "--last", type="string", dest="l_name", help="Supply a last name to search", action="store")
parser.add_option("-g", "--google", action="callback", callback=google)
(options, args) = parser.parse_args()
And can't seem to figure out why it wouldn't print out the user supplied input. I've looked at the doc for python on optparse and it just gets fuzzy. Anyways any possibly way I can use options.f_name in that function. I have been using something as such to put into the functions arguments to use.
first_name = options.f_name
Then would supply the function with one of the arguments that didn't work either.
Old post, but for posterity:
You need to specify the type of the argument in order to have optparse inject it into the value parameter:
def google(option, opt_str, value, parser):
print value
parser.add_option("-g", "--google", action="callback", callback=google, type="string")
Full example showing how to return the value to the parser for inclusion into options:
from optparse import OptionParser
def doSomethingWithValue(value):
return "baked beans and {}".format(value)
def google(option, opt_str, value, parser):
setattr(parser.values, option.dest, doSomethingWithValue(value))
parser = OptionParser()
parser.add_option("-g", "--google", action="callback", callback=google, type="string", dest="googleOption")
(options, args) = parser.parse_args()
print(options)
# ./script.py ==> {'googleOption': None}
# ./script.py -g spam ==> {'googleOption': 'baked beans and spam'}
optparse is hard to use. You should try docopt. http://docopt.org/
As a side note, urllib is hard to use too. Check out the requests module. http://docs.python-requests.org/en/latest/
The order of evaluation of the arguments is not specified, so the callback for -g may be called before optparse handles the -f option. The only way to do this during the parse is to make them both callbacks that are aware of each other, and only when the second argument is handled does it perform the behaviour you are looking for.
Is there any reason you can't just set a flag and handle it after the parse_args() is complete, then you will be sure all the arguments have been handled.
BTW: optparse is deprecated in favour of argparse.
If you check the documentation, you'll find out that you actually need to use:
parser.values.f_name
Of course you should make sure that you take the precautions for cases where it hasn't been defined yet.
This first link has the same question in the first section, but it is unanswered
(python argparse: parameter=value). And this second question is similar, but I can't seem to get it working for my particular case
( Using argparse to parse arguments of form "arg= val").
So my situation is this -- I am re-writing a Python wrapper which is used by many other scripts (I would prefer not to modify these other scripts). Currently, the Python wrapper is called with command line arguments of the form --key=value for a number of different arguments, but was parsed manually. I would like to parse them with argparse.
N.B. The argument names are unwieldy, so I am renaming using the dest option in add_argument.
parser = argparse.ArgumentParser(description='Wrappin Ronnie Reagan')
parser.add_argument("--veryLongArgName1", nargs=1, dest="arg1", required=True)
parser.add_argument("--veryLongArgName2", nargs=1, dest="arg2")
parser.add_argument("--veryLongArgName3", nargs=1, dest="arg3")
userOpts = vars(parser.parse_args())
Which, while apparently parsing the passed command lines correctly, displays this as the help:
usage: testing_argsparse.py [-h] --veryLongArgName1 ARG1
[--veryLongArgName2 ARG2]
[--veryLongArgName3 ARG3]
testing_argsparse.py: error: argument --veryLongArgName1 is required
But what I want is that all parameters are specified with the --key=value format, not --key value. i.e.
usage: testing_argsparse.py [-h] --veryLongArgName1=ARG1
[--veryLongArgName2=ARG2]
[--veryLongArgName3=ARG3]
testing_argsparse.py: error: argument --veryLongArgName1 is required
testing_argsparse.py --veryLongArgName1=foo
works. argparse module accepts both --veryLongArgName1=foo and --veryLongArgName1 foo formats.
What exact command line arguments are you trying to pass to argparse that's causing it to not work?
A little late but for anyone with a similar request as the OP you could use a custom HelpFormatter.
class ArgFormatter(argparse.HelpFormatter):
def _format_args(self, *args):
result = super(ArgFormatter, self)._format_args(*args)
return result and '%%%' + result
def _format_actions_usage(self, *args):
result = super(ArgFormatter, self)._format_actions_usage(*args)
return result and result.replace(' %%%', '=')
This can then be passed to ArgumentParser to give the wanted behavior.
parser = argparse.ArgumentParser(
description='Wrappin Ronnie Reagan',
formatter_class=ArgFormatter)
This intercepts the args (ARG1, ARG2, ...) and adds a custom prefix which is later replaced (along with the unwanted space) for an = symbol. The and in the return statements makes sure to only modify the result if it's non-empty.