We Keep Coding

How to make conditional arguments using argparse? [duplicate]

I have done as much research as possible but I haven't found the best way to make certain cmdline arguments necessary only under certain conditions, in this case only if other arguments have been given. Here's what I want to do at a very basic level:
p = argparse.ArgumentParser(description='...')
p.add_argument('--argument', required=False)
p.add_argument('-a', required=False) # only required if --argument is given
p.add_argument('-b', required=False) # only required if --argument is given
From what I have seen, other people seem to just add their own check at the end:
if args.argument and (args.a is None or args.b is None):
# raise argparse error here
Is there a way to do this natively within the argparse package?

I've been searching for a simple answer to this kind of question for some time. All you need to do is check if '--argument' is in sys.argv, so basically for your code sample you could just do:
import argparse
import sys
if __name__ == '__main__':
p = argparse.ArgumentParser(description='...')
p.add_argument('--argument', required=False)
p.add_argument('-a', required='--argument' in sys.argv) #only required if --argument is given
p.add_argument('-b', required='--argument' in sys.argv) #only required if --argument is given
args = p.parse_args()
This way required receives either True or False depending on whether the user as used --argument. Already tested it, seems to work and guarantees that -a and -b have an independent behavior between each other.

You can implement a check by providing a custom action for --argument, which will take an additional keyword argument to specify which other action(s) should become required if --argument is used.
import argparse
class CondAction(argparse.Action):
def __init__(self, option_strings, dest, nargs=None, **kwargs):
x = kwargs.pop('to_be_required', [])
super(CondAction, self).__init__(option_strings, dest, **kwargs)
self.make_required = x
def __call__(self, parser, namespace, values, option_string=None):
for x in self.make_required:
x.required = True
try:
return super(CondAction, self).__call__(parser, namespace, values, option_string)
except NotImplementedError:
pass
p = argparse.ArgumentParser()
x = p.add_argument("--a")
p.add_argument("--argument", action=CondAction, to_be_required=[x])
The exact definition of CondAction will depend on what, exactly, --argument should do. But, for example, if --argument is a regular, take-one-argument-and-save-it type of action, then just inheriting from argparse._StoreAction should be sufficient.
In the example parser, we save a reference to the --a option inside the --argument option, and when --argument is seen on the command line, it sets the required flag on --a to True. Once all the options are processed, argparse verifies that any option marked as required has been set.

Your post parsing test is fine, especially if testing for defaults with is None suits your needs.
http://bugs.python.org/issue11588 'Add "necessarily inclusive" groups to argparse' looks into implementing tests like this using the groups mechanism (a generalization of mutuall_exclusive_groups).
I've written a set of UsageGroups that implement tests like xor (mutually exclusive), and, or, and not. I thought those where comprehensive, but I haven't been able to express your case in terms of those operations. (looks like I need nand - not and, see below)
This script uses a custom Test class, that essentially implements your post-parsing test. seen_actions is a list of Actions that the parse has seen.
class Test(argparse.UsageGroup):
def _add_test(self):
self.usage = '(if --argument then -a and -b are required)'
def testfn(parser, seen_actions, *vargs, **kwargs):
"custom error"
actions = self._group_actions
if actions[0] in seen_actions:
if actions[1] not in seen_actions or actions[2] not in seen_actions:
msg = '%s - 2nd and 3rd required with 1st'
self.raise_error(parser, msg)
return True
self.testfn = testfn
self.dest = 'Test'
p = argparse.ArgumentParser(formatter_class=argparse.UsageGroupHelpFormatter)
g1 = p.add_usage_group(kind=Test)
g1.add_argument('--argument')
g1.add_argument('-a')
g1.add_argument('-b')
print(p.parse_args())
Sample output is:
1646:~/mypy/argdev/usage_groups$ python3 issue25626109.py --arg=1 -a1
usage: issue25626109.py [-h] [--argument ARGUMENT] [-a A] [-b B]
(if --argument then -a and -b are required)
issue25626109.py: error: group Test: argument, a, b - 2nd and 3rd required with 1st
usage and error messages still need work. And it doesn't do anything that post-parsing test can't.
Your test raises an error if (argument & (!a or !b)). Conversely, what is allowed is !(argument & (!a or !b)) = !(argument & !(a and b)). By adding a nand test to my UsageGroup classes, I can implement your case as:
p = argparse.ArgumentParser(formatter_class=argparse.UsageGroupHelpFormatter)
g1 = p.add_usage_group(kind='nand', dest='nand1')
arg = g1.add_argument('--arg', metavar='C')
g11 = g1.add_usage_group(kind='nand', dest='nand2')
g11.add_argument('-a')
g11.add_argument('-b')
The usage is (using !() to mark a 'nand' test):
usage: issue25626109.py [-h] !(--arg C & !(-a A & -b B))
I think this is the shortest and clearest way of expressing this problem using general purpose usage groups.
In my tests, inputs that parse successfully are:
''
'-a1'
'-a1 -b2'
'--arg=3 -a1 -b2'
Ones that are supposed to raise errors are:
'--arg=3'
'--arg=3 -a1'
'--arg=3 -b2'

For arguments I've come up with a quick-n-dirty solution like this.
Assumptions: (1) '--help' should display help and not complain about required argument and (2) we're parsing sys.argv
p = argparse.ArgumentParser(...)
p.add_argument('-required', ..., required = '--help' not in sys.argv )
This can easily be modified to match a specific setting.
For required positionals (which will become unrequired if e.g. '--help' is given on the command line) I've come up with the following: [positionals do not allow for a required=... keyword arg!]
p.add_argument('pattern', ..., narg = '+' if '--help' not in sys.argv else '*' )
basically this turns the number of required occurrences of 'pattern' on the command line from one-or-more into zero-or-more in case '--help' is specified.

Here is a simple and clean solution with these advantages:
No ambiguity and loss of functionality caused by oversimplified parsing using the in sys.argv test.
No need to implement a special argparse.Action or argparse.UsageGroup class.
Simple usage even for multiple and complex deciding arguments.
I noticed just one considerable drawback (which some may find desirable): The help text changes according to the state of the deciding arguments.
The idea is to use argparse twice:
Parse the deciding arguments instead of the oversimplified use of the in sys.argv test. For this we use a short parser not showing help and the method .parse_known_args() which ignores unknown arguments.
Parse everything normally while reusing the parser from the first step as a parent and having the results from the first parser available.
import argparse
# First parse the deciding arguments.
deciding_args_parser = argparse.ArgumentParser(add_help=False)
deciding_args_parser.add_argument(
'--argument', required=False, action='store_true')
deciding_args, _ = deciding_args_parser.parse_known_args()
# Create the main parser with the knowledge of the deciding arguments.
parser = argparse.ArgumentParser(
description='...', parents=[deciding_args_parser])
parser.add_argument('-a', required=deciding_args.argument)
parser.add_argument('-b', required=deciding_args.argument)
arguments = parser.parse_args()
print(arguments)

Until http://bugs.python.org/issue11588 is solved, I'd just use nargs:
p = argparse.ArgumentParser(description='...')
p.add_argument('--arguments', required=False, nargs=2, metavar=('A', 'B'))
This way, if anybody supplies --arguments, it will have 2 values.
Maybe its CLI result is less readable, but code is much smaller. You can fix that with good docs/help.

This is really the same as #Mira 's answer but I wanted to show it for the case where when an option is given that an extra arg is required:
For instance, if --option foo is given then some args are also required that are not required if --option bar is given:
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument('--option', required=True,
help='foo and bar need different args')
if 'foo' in sys.argv:
parser.add_argument('--foo_opt1', required=True,
help='--option foo requires "--foo_opt1"')
parser.add_argument('--foo_opt2', required=True,
help='--option foo requires "--foo_opt2"')
...
if 'bar' in sys.argv:
parser.add_argument('--bar_opt', required=True,
help='--option bar requires "--bar_opt"')
...
It's not perfect - for instance proggy --option foo --foo_opt1 bar is ambiguous but for what I needed to do its ok.

Add additional simple "pre"parser to check --argument, but use parse_known_args() .
pre = argparse.ArgumentParser()
pre.add_argument('--argument', required=False, action='store_true', default=False)
args_pre=pre.parse_known_args()
p = argparse.ArgumentParser()
p.add_argument('--argument', required=False)
p.add_argument('-a', required=args_pre.argument)
p.add_argument('-b', required=not args_pre.argument)

Argparse with spaces

I have the following script:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--arg1', dest='arg1')
parse_results = parser.parse_known_args(['--arg1 = value'])
However I get the result:
(Namespace(arg1=None), ['--arg1 = value'])
Is there a way using argparse to accept this kind of input where there are spaces between argument, = and the value?

The shell processes your argument list before your program ever runs, and that processing is very simple: treat each white-space separated word following the command name as a separate argument. By that logic, --foo = bar is 3 separate arguments, so sys.argv contains ["--foo", "=", "bar"]. There's no avoiding that, since it happens before argparse ever runs.
Given this fact, there's no benefit to writing --foo = bar, since it would be equivalent to --foo=bar or --foo bar, just with extraneous characters (spaces in the first case, = in the second).
As to why --foo=bar is supported at all when it is equivalent to --foo bar, I don't know. It might just be for compatibility with existing practice; it might be useful with different argument parsers, but argparse doesn't seem to need it. At some point in the past, I thought I had come up with a reason why being able to specify the argument to --foo using one shell word instead of two was useful, but I seem to have forgotten it.

It's about how the shell parses arguments:
bruno#bigb:~/Work/playground$ cat argz.py
import sys
print "args:", sys.argv
bruno#bigb:~/Work/playground$ python argz.py --foo = bar --bar=baaz
args: ['argz.py', '--foo', '=', 'bar', '--bar=baaz']

To start, a problem with your code: A program will see space-separated components as separate arguments. Your example should have read
parse_results = parser.parse_known_args(['--arg1', '=', 'value'])
or you can use split() to generate the array on the fly, as the documentation does it:
parse_results = parser.parse_known_args('--arg1 = value'.split())
The above still doesn't give you want you want:
>>> parser.parse_known_args('--arg1 = value'.split())
(Namespace(arg1='='), ['value'])
The equals-sign is parsed as the argument, because long argument syntax is either --option=value or --option value. The = sign is only there to separate the option name from its argument; there is no point in including it if arguments are separated by spaces, so the syntax --option = value is not supported.
Recommendation: Use only standard long-argument syntax, i.e. --arg1 value or --arg=value.
But if you absolutely must support the form --arg1 = value, you can filter the argument list to throw out elements that are a single = sign.:
>>> cleanup = lambda args: (a for a in args if a != '=')
>>> parser.parse_known_args(cleanup('--arg1 = value'.split()))
(Namespace(arg1='value'), [])

When you use
parser.parse_known_args(['--arg1 = value'])
it's equivalent to the commandline:
$myfun.py '--arg1 = value'
In both cases --arg1 = value is treated as one string. Since it does not match '--arg1' or '--arg1=' it goes into the unrecognized strings list.
As others have pointed out, without the quotes $myfun.py --arg1 = value passes to the parser as ['--arg1', '=', 'value']. Which would parse to:
(Namespace(arg1='='), ['value'])
If --arg is defined with nargs='+', then you'd get
(Namespace(arg1=['=', 'value']), [])
and you could ignore the '='.
But for sake of consistency with other commandline practices, I'd suggest sticking with the --arg1=value, and --arg1 value syntax.

How do I configure required arguments while still allowing special arguments like "--version"?

I have a Python script that accepts one or more input files and produces one or more output files (sort of a compiler, translating one syntax into another)
In my argparse section, I have configured so that the list of input files option is "nargs='+'", so that it will show a "too few arguments" error if user provides zero input files.
At the same time, I want to have a "--version" option that will just print the current script version and exit. When this option is provided, everything else (if provided) is irrelevant and should be ignored.
Just like ArgumentParser automatically adds the "--help" option which works like this, how can I add a "--version" option without changing the nargs='+' mechanism?

Try the version action class. From the docs:
'version' - This expects a version= keyword argument in the add_argument() call, and prints version information and exits when invoked:
>>>
>>> import argparse
>>> parser = argparse.ArgumentParser(prog='PROG')
>>> parser.add_argument('--version', action='version', version='%(prog)s 2.0')
>>> parser.parse_args(['--version'])
PROG 2.0
It behaves like the help (-h) except it displays the version parameter that you define with it (or lacking that a version value that you give the parser itself).

Python argparse : how to detect duplicated optional argument?

I'm using argparse with optional parameter, but I want to avoid having something like this : script.py -a 1 -b -a 2
Here we have twice the optional parameter 'a', and only the second parameter is returned. I want either to get both values or get an error message.
How should I define the argument ?
[Edit]
This is the code:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-a', dest='alpha', action='store', nargs='?')
parser.add_argument('-b', dest='beta', action='store', nargs='?')
params, undefParams = self.parser.parse_known_args()

append action will collect the values from repeated use in a list
parser.add_argument('-a', '--alpha', action='append')
producing an args namespace like:
namespace(alpha=['1','3'], b='4')
After parsing you can check args.alpha, and accept or complain about the number of values. parser.error('repeated -a') can be used to issue an argparse style error message.
You could implement similar functionality in a custom Action class, but that requires understanding the basic structure and operation of such a class. I can't think anything that can be done in an Action that can't just as well be done in the appended list after.
https://stackoverflow.com/a/23032953/901925 is an answer with a no-repeats custom Action.
Why are you using nargs='?' with flagged arguments like this? Without a const parameter this is nearly useless (see the nargs=? section in the docs).
Another similar SO: Python argparse with nargs behaviour incorrect

argparse change definition of argument

I set up my argument parser as follows:
parser=argparse.ArgumentParser()
parser.add_argument('--point',help='enter a point (e.g. 2,3,4)')
parser.parse_args('--point=-2,5,6'.split()) #works
parser.parse_args('--point -2,5,6'.split()) #doesn't work :(
Is there any way to tell argparse that strings which match the regular expression r"-\d+.*" are not options but an argument of an option?
Also note that I could do something like this:
parser.add_argument('--point',nargs='*')
parser.parse_args('--point -2 5 6'.split())
but that's not really how I want it to work.

I think preprocessing sys.argv is the most straightforward way here. Consider for example:
import argparse, re
parser=argparse.ArgumentParser()
parser.add_argument('--point',help='enter a point (e.g. 2,3,4)')
args = '--point -2,5,6'.split() # or sys.argv
is_list = re.compile(r'^-?[\d,.]+$')
args = ['"%s"' % x if is_list.match(x) else x for x in args]
print parser.parse_args(args)
This returns Namespace(point='"-2,5,6"') which should be easy to parse.

You could change the prefix char so - is no longer recognized as indicating the start of an argument. It does look a bit weird but it is useful when negative numbers may appear in the arguments.
import argparse
parser=argparse.ArgumentParser(prefix_chars = '#')
parser.add_argument('##point',help='enter a point (e.g. 2,3,4)')
args = parser.parse_args('##point=-2,5,6'.split()) #works
print(args)
# Namespace(point='-2,5,6')
args = parser.parse_args('##point -2,5,6'.split()) #work also
print(args)
# Namespace(point='-2,5,6')

If you don't mind messing around with argparse internals, argparse already does something very similar to what I want to do. One of the classes that ArgumentParser inherits from has this line in __init__
import re as _re
...
self._negative_number_matcher = _re.compile(r'^-\d+$|^-\d*\.\d+$')
So to get my example to work, we just need to swap out an appropriate regex...
parser._negative_number_matcher = re.compile(r'^-\d+|^-\d*\.\d+')
Usually I'm not very much in favor of messing around with the internals of a class when they're prefixed by an underscore (as it is implementation dependent and liable to change) -- However, in this case, I think it's probably ok because:
If argparse changes that variable name, it doesn't hurt, we're just back to the case where "--print -2,3,4" doesn't work again.
I can't think of any better way to determine if something is a number than regex (I suppose they could try to cast to a float and catch the exception, but if they did that, they wouldn't have a variable named _negative_number_matcher anymore and again, argparse would still work fine except in this corner case where it doesn't do what I want)