We Keep Coding

Python argparse help-like option

I am writing a python script that takes two arguments, and some options:
scriptname [-h] [-l] [-q|-d] arg1 arg2
The -q (quiet) and -d (debug) options change the verbosity level, and the -h option is the help option automatically created by argparse.
I would like the -l (list) option to behave similarly to the -h option in that it will not require that the (otherwise mandatory) arguments are present and list some useful information (different from the -h option). In practice, this means that the script could be called in the following three ways:
scriptmane [-q|-d] arg1 arg2
scriptname -l
scriptname -h
Two possible ways forward would be to:
Make the arguments optional (with nargs='?') and add code to verify that there are two arguments in all cases where there -l og -h options are not given.
Write a custom action class (not sure about the details).
But I hope there is a more straightforward way to inherit the "this option is all you need" behaviour from the help option.
Solution (based on samwyse's answer):
Based on the _HelpAction() in argparse.py:
class _ListAction(argparse.Action):
def __init__(self,
option_strings,
dest=argparse.SUPPRESS,
default=argparse.SUPPRESS,
help=None):
super(_ListAction, self).__init__(
option_strings=option_strings,
dest=dest,
default=default,
nargs=0,
help=help)
def __call__(self, parser, namespace, values, option_string=None):
print_list()
parser.exit()
and then, during parser setup:
parser.add_argument('-l', '--list', action=_ListAction,
help="List all available cases")

If the option "list" is intended to have different behavior from "help" then you need to write a custom action. The good news is that it is very simple to do this. The main page for argparse gives you hints, you only have to realize that the action is called as soon as the option is seen in the list of arguments. In the new action's call (that should have two underscores at each end but markdown uses those for emphasis) method, do whatever you need to do for your option and then call parser.exit() to short circuit the processing of any more arguments.
Look at the source for _HelpAction and _VersionAction here: https://github.com/ThomasWaldmann/argparse/blob/master/argparse.py (and probably just subclass one of them to skip writing the init code).

How to indicate that at least one parameter is needed?

My script is accepting --full, --last and --check using ArgParse. If no option is provided, it just show the help message. But in that message, the parameters appear as optional.
usage: script.py [-h] [--full] [--last] [--check log_file]
If I use the keyword required, then the script will always expect the parameter, which is not correct.
usage: script.py [-h] --full --last --check log_file
So, how can I show something like:
usage: script.py [-h] (--full |--last |--check log_file)
Indicating that the help is optional but that at least one of those parameters is required.

On the question of customizing the usage:
The parser constructor takes a usage parameter. The immediate effect is to set an attribute:
parser = argparse.ArgumentParser( ... usage=custom_usage...)
print(parser.usage)
# should show None or the custom_usage string
Being a normal Python object attribute, you can change it after the parser was created.
usage_str = parser.format_usage()
The format_usage method ask the parser for create the usage that will be shown in the help (and error messages). If the parser.usage value is None, it formats it from the arguments. If a string, it is used as is (I think it fills in values like %(prog)s).
So you could write a usage string from scratch. Or you could set up the parser, get the current usage string, and edit that to suit your needs. Editing is most likely something you'd do during development, while testing the parser in an IDE. But it could be done on the fly.
A crude example:
In [441]: parser=argparse.ArgumentParser()
In [442]: g=parser.add_mutually_exclusive_group()
In [443]: g.add_argument('--foo')
In [444]: g.add_argument('--bar')
In [445]: ustr = parser.format_usage()
# 'usage: ipython3 [-h] [--foo FOO | --bar BAR]\n'
In [450]: parser.usage = ustr.replace('[','(').replace(']',')')
In [451]: parser.format_usage()
# 'usage: usage: ipython3 (-h) (--foo FOO | --bar BAR)\n'
I've replaced the [] with () (even on the -h :( ).
For now testing logical combinations of the args attributes is the best choice. Inside the parse_args functions the parser maintains a list (set actually) of arguments that it has seen. That is used to test for required arguments, and for mutually_exclusive_arguments, but it is not available outside that code.
For store_true (or false) arguments, just check their truth value. For others I like to test for the default None. If you use other default values, test accordingly. A nice thing about None is that the user cannot give you that value.
Perhaps the most general way to test for arguments is to count the number of attributes which are not None:
In [461]: args=argparse.Namespace(one=None, tow=2, three=None)
In [462]: ll = ['one','tow','three']
In [463]: sum([getattr(args,l,None) is not None for l in ll])
Out[463]: 1
0 means none are found; >0 at least one present; ==len(ll) all found; >1 violates mutually exclusivity; '==1' for required mutually exclusive.

As #doublep explained in his answer, if you want to use more than one option at a time:
Change the usage message manually to the one you want.
Add the following code from Python argparse: Make at least one argument required:
if not (args.full or args.last or args.check):
parse.error('[-] Error: DISPLAY_ERROR_MESSAGE')

You can use add_mutually_exclusive_group():
parser = argparse.ArgumentParser ()
group = parser.add_mutually_exclusive_group (required = True)
group.add_argument ('--foo')
group.add_argument ('--bar')
However, the main effect is that you won't be able to use more than one option at a time.

Allowing same option in different argparser group

Is it possible to allow the same argparse option to be in 2 different argparser groups?
This is actually what I want to achieve:-
#!/usr/bin/env python
import argparse
... ... ...
g1 = parser.add_argument_group('g1')
g2 = parser.add_argument_group('g2')
g1.add_argument('--aa')
g1.add_argument('--common')
g2.add_argument('--bb')
g2.add_argument('--common')
... and the printed out help looks like this:-
usage: ...
... ... ...
g1:
--aa [aa]
--common [common]
g2:
-bb [bb]
--common [common]
But that is not possible, as argparse complains 'conflicting option string'

http://bugs.python.org/issue10984 discusses adding an argument to two different mutually_exclusive_groups. Doing the same with argument_groups is similar, though simpler.
import argparse
parser=argparse.ArgumentParser()
g1=parser.add_argument_group('group1')
g1.add_argument('-a')
caction=g1.add_argument('-c')
g2=parser.add_argument_group('group2')
g2.add_argument('-b')
g2._group_actions.append(caction)
parser.print_help()
It is a kludge, in the sense that it is modifying a 'private' attribute of the group.
The result:
usage: ipython [-h] [-a A] [-c C] [-b B]
optional arguments:
-h, --help show this help message and exit
group1:
-a A
-c C
group2:
-b B
-c C
Here's what's going on. add_argument creates an Action, registers it with the parser, and returns it. That's what caction captures. If added to a group, it also registers the action with the group - by adding it to a _group_actions list.
If you do g2.add_argument('-c') you get an error because the parser already has an action with that option string. The fact that you are 'adding' it to a different group is incidental. The kludge gets around that by adding it to the group's list, without creating a new action.
In case it isn't obvious from the documentation, argument_groups are basically a 'help' convenience. They do not affect parsing at all. There are other ways you could customize the help. For example, add --common to the parser, possibly with a SUPPRESS help line. Then include a mention of it in the description for each group.

In this special case, you could make --aa and --bb mutually exclusive.
import argparse
parser = argparse.ArgumentParser()
mutex = parser.add_mutually_exclusive_group()
mutex.add_argument('--aa', action = 'store_true')
mutex.add_argument('--bb', action = 'store_true')
parser.add_argument('--common', action = 'store_true')
args = parser.parse_args()
which results in
usage: a.py [-h] [--aa | --bb] [--common]
Generally, argparse has the problem that you must create an option when calling add_argument.
Here's a somewhat related topic: Does argparse (python) support mutually exclusive groups of arguments?
There's a patch that allows you to have one argument in more than one mutually exclusive group:
http://bugs.python.org/issue10984
Another thing you can do is fiddle around with the argparse subparsers, or use another commandline parser altogether.

Python argparse with nargs behaviour incorrect

Here is my argparse sample say sample.py
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-p", nargs="+", help="Stuff")
args = parser.parse_args()
print args
Python - 2.7.3
I expect that the user supplies a list of arguments separated by spaces after the -p option. For example, if you run
$ sample.py -p x y
Namespace(p=['x', 'y'])
But my problem is that when you run
$ sample.py -p x -p y
Namespace(p=['y'])
Which is neither here nor there. I would like one of the following
Throw an exception to the user asking him to not use -p twice instead just supply them as one argument
Just assume it is the same option and produce a list of ['x','y'].
I can see that python 2.7 is doing neither of them which confuses me. Can I get python to do one of the two behaviours documented above?

Note: python 3.8 adds an action="extend" which will create the desired list of ['x','y']
To produce a list of ['x','y'] use action='append'. Actually it gives
Namespace(p=[['x'], ['y']])
For each -p it gives a list ['x'] as dictated by nargs='+', but append means, add that value to what the Namespace already has. The default action just sets the value, e.g. NS['p']=['x']. I'd suggest reviewing the action paragraph in the docs.
optionals allow repeated use by design. It enables actions like append and count. Usually users don't expect to use them repeatedly, or are happy with the last value. positionals (without the -flag) cannot be repeated (except as allowed by nargs).
How to add optional or once arguments? has some suggestions on how to create a 'no repeats' argument. One is to create a custom action class.

I ran into the same issue. I decided to go with the custom action route as suggested by mgilson.
import argparse
class ExtendAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
if getattr(namespace, self.dest, None) is None:
setattr(namespace, self.dest, [])
getattr(namespace, self.dest).extend(values)
parser = argparse.ArgumentParser()
parser.add_argument("-p", nargs="+", help="Stuff", action=ExtendAction)
args = parser.parse_args()
print args
This results in
$ ./sample.py -p x -p y -p z w
Namespace(p=['x', 'y', 'z', 'w'])
Still, it would have been much neater if there was an action='extend' option in the library by default.

Using argparse to parse arguments of form "arg= val"

I want to use argparse to parse command lines of form "arg=val"
For example, the usage would be:
script.py conf_dir=/tmp/good_conf
To achieve it, I am doing this:
desc = "details"
parser = argparse.ArgumentParser(description=desc, add_help=False)
args = parser.add_argument("conf_dir")
args = parser.parse_args("conf_dir=FOO".split())
args = parser.parse_args()
print args.conf_dir
But, the problem is that, on invocation of the script with:
python script.py conf_dir=/tmp/good_conf
I get:
conf_dir=/tmp/good_conf
Where as I expect
/tmp/good_conf
So, the question is: Can I use argparse to parse cmd line, which contains name value pairs?
Any hints?
Edit: The reason I want to do this and not some thing like --conf_dir=/tmp/good_dir is because there are other tools (written in other language), which uses conf_dir=/tmp/good_dir style of arguments. To maintain consistency, I was to parse args in this way.

You need a custom action
class StoreNameValuePair(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
n, v = values.split('=', 1)
setattr(namespace, n, v)
args = parser.add_argument("conf_dir", action=StoreNameValuePair)

As per the documentation, argparse doesn't natively let you have unprefixed options like that. If you omit the leading -, it assumes you are describing a positional argument and expects it to be provided as:
python script.py /tmp/good_conf
If you want it to be optional, it needs to be correctly marked as a flag by calling it --conf_dir, and invoking the script like:
python script.py --conf_dir=/tmp/good_conf
However, to accept name-value pairs, you can implement a custom action. In combination with nargs, such an action could accept an arbitrary number of name-value pairs and store them on the argument parsing result object.

#chepner This is great. I improved this to support multiple args as well and store the result as dict:
class StoreDict(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
kv={}
if not isinstance(values, (list,)):
values=(values,)
for value in values:
n, v = value.split('=')
kv[n]=v
setattr(namespace, self.dest, kv)

The usual way to put name value pairs on the command line is with options. I.e. you would use
python script.py --confdir=/tmp/good_conf
argparse can certainly handle that case. See the docs at:
http://docs.python.org/library/argparse.html#option-value-syntax