We Keep Coding

How to make conditional arguments using argparse? [duplicate]

I have done as much research as possible but I haven't found the best way to make certain cmdline arguments necessary only under certain conditions, in this case only if other arguments have been given. Here's what I want to do at a very basic level:
p = argparse.ArgumentParser(description='...')
p.add_argument('--argument', required=False)
p.add_argument('-a', required=False) # only required if --argument is given
p.add_argument('-b', required=False) # only required if --argument is given
From what I have seen, other people seem to just add their own check at the end:
if args.argument and (args.a is None or args.b is None):
# raise argparse error here
Is there a way to do this natively within the argparse package?

I've been searching for a simple answer to this kind of question for some time. All you need to do is check if '--argument' is in sys.argv, so basically for your code sample you could just do:
import argparse
import sys
if __name__ == '__main__':
p = argparse.ArgumentParser(description='...')
p.add_argument('--argument', required=False)
p.add_argument('-a', required='--argument' in sys.argv) #only required if --argument is given
p.add_argument('-b', required='--argument' in sys.argv) #only required if --argument is given
args = p.parse_args()
This way required receives either True or False depending on whether the user as used --argument. Already tested it, seems to work and guarantees that -a and -b have an independent behavior between each other.

You can implement a check by providing a custom action for --argument, which will take an additional keyword argument to specify which other action(s) should become required if --argument is used.
import argparse
class CondAction(argparse.Action):
def __init__(self, option_strings, dest, nargs=None, **kwargs):
x = kwargs.pop('to_be_required', [])
super(CondAction, self).__init__(option_strings, dest, **kwargs)
self.make_required = x
def __call__(self, parser, namespace, values, option_string=None):
for x in self.make_required:
x.required = True
try:
return super(CondAction, self).__call__(parser, namespace, values, option_string)
except NotImplementedError:
pass
p = argparse.ArgumentParser()
x = p.add_argument("--a")
p.add_argument("--argument", action=CondAction, to_be_required=[x])
The exact definition of CondAction will depend on what, exactly, --argument should do. But, for example, if --argument is a regular, take-one-argument-and-save-it type of action, then just inheriting from argparse._StoreAction should be sufficient.
In the example parser, we save a reference to the --a option inside the --argument option, and when --argument is seen on the command line, it sets the required flag on --a to True. Once all the options are processed, argparse verifies that any option marked as required has been set.

Your post parsing test is fine, especially if testing for defaults with is None suits your needs.
http://bugs.python.org/issue11588 'Add "necessarily inclusive" groups to argparse' looks into implementing tests like this using the groups mechanism (a generalization of mutuall_exclusive_groups).
I've written a set of UsageGroups that implement tests like xor (mutually exclusive), and, or, and not. I thought those where comprehensive, but I haven't been able to express your case in terms of those operations. (looks like I need nand - not and, see below)
This script uses a custom Test class, that essentially implements your post-parsing test. seen_actions is a list of Actions that the parse has seen.
class Test(argparse.UsageGroup):
def _add_test(self):
self.usage = '(if --argument then -a and -b are required)'
def testfn(parser, seen_actions, *vargs, **kwargs):
"custom error"
actions = self._group_actions
if actions[0] in seen_actions:
if actions[1] not in seen_actions or actions[2] not in seen_actions:
msg = '%s - 2nd and 3rd required with 1st'
self.raise_error(parser, msg)
return True
self.testfn = testfn
self.dest = 'Test'
p = argparse.ArgumentParser(formatter_class=argparse.UsageGroupHelpFormatter)
g1 = p.add_usage_group(kind=Test)
g1.add_argument('--argument')
g1.add_argument('-a')
g1.add_argument('-b')
print(p.parse_args())
Sample output is:
1646:~/mypy/argdev/usage_groups$ python3 issue25626109.py --arg=1 -a1
usage: issue25626109.py [-h] [--argument ARGUMENT] [-a A] [-b B]
(if --argument then -a and -b are required)
issue25626109.py: error: group Test: argument, a, b - 2nd and 3rd required with 1st
usage and error messages still need work. And it doesn't do anything that post-parsing test can't.
Your test raises an error if (argument & (!a or !b)). Conversely, what is allowed is !(argument & (!a or !b)) = !(argument & !(a and b)). By adding a nand test to my UsageGroup classes, I can implement your case as:
p = argparse.ArgumentParser(formatter_class=argparse.UsageGroupHelpFormatter)
g1 = p.add_usage_group(kind='nand', dest='nand1')
arg = g1.add_argument('--arg', metavar='C')
g11 = g1.add_usage_group(kind='nand', dest='nand2')
g11.add_argument('-a')
g11.add_argument('-b')
The usage is (using !() to mark a 'nand' test):
usage: issue25626109.py [-h] !(--arg C & !(-a A & -b B))
I think this is the shortest and clearest way of expressing this problem using general purpose usage groups.
In my tests, inputs that parse successfully are:
''
'-a1'
'-a1 -b2'
'--arg=3 -a1 -b2'
Ones that are supposed to raise errors are:
'--arg=3'
'--arg=3 -a1'
'--arg=3 -b2'

For arguments I've come up with a quick-n-dirty solution like this.
Assumptions: (1) '--help' should display help and not complain about required argument and (2) we're parsing sys.argv
p = argparse.ArgumentParser(...)
p.add_argument('-required', ..., required = '--help' not in sys.argv )
This can easily be modified to match a specific setting.
For required positionals (which will become unrequired if e.g. '--help' is given on the command line) I've come up with the following: [positionals do not allow for a required=... keyword arg!]
p.add_argument('pattern', ..., narg = '+' if '--help' not in sys.argv else '*' )
basically this turns the number of required occurrences of 'pattern' on the command line from one-or-more into zero-or-more in case '--help' is specified.

Here is a simple and clean solution with these advantages:
No ambiguity and loss of functionality caused by oversimplified parsing using the in sys.argv test.
No need to implement a special argparse.Action or argparse.UsageGroup class.
Simple usage even for multiple and complex deciding arguments.
I noticed just one considerable drawback (which some may find desirable): The help text changes according to the state of the deciding arguments.
The idea is to use argparse twice:
Parse the deciding arguments instead of the oversimplified use of the in sys.argv test. For this we use a short parser not showing help and the method .parse_known_args() which ignores unknown arguments.
Parse everything normally while reusing the parser from the first step as a parent and having the results from the first parser available.
import argparse
# First parse the deciding arguments.
deciding_args_parser = argparse.ArgumentParser(add_help=False)
deciding_args_parser.add_argument(
'--argument', required=False, action='store_true')
deciding_args, _ = deciding_args_parser.parse_known_args()
# Create the main parser with the knowledge of the deciding arguments.
parser = argparse.ArgumentParser(
description='...', parents=[deciding_args_parser])
parser.add_argument('-a', required=deciding_args.argument)
parser.add_argument('-b', required=deciding_args.argument)
arguments = parser.parse_args()
print(arguments)

Until http://bugs.python.org/issue11588 is solved, I'd just use nargs:
p = argparse.ArgumentParser(description='...')
p.add_argument('--arguments', required=False, nargs=2, metavar=('A', 'B'))
This way, if anybody supplies --arguments, it will have 2 values.
Maybe its CLI result is less readable, but code is much smaller. You can fix that with good docs/help.

This is really the same as #Mira 's answer but I wanted to show it for the case where when an option is given that an extra arg is required:
For instance, if --option foo is given then some args are also required that are not required if --option bar is given:
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument('--option', required=True,
help='foo and bar need different args')
if 'foo' in sys.argv:
parser.add_argument('--foo_opt1', required=True,
help='--option foo requires "--foo_opt1"')
parser.add_argument('--foo_opt2', required=True,
help='--option foo requires "--foo_opt2"')
...
if 'bar' in sys.argv:
parser.add_argument('--bar_opt', required=True,
help='--option bar requires "--bar_opt"')
...
It's not perfect - for instance proggy --option foo --foo_opt1 bar is ambiguous but for what I needed to do its ok.

Add additional simple "pre"parser to check --argument, but use parse_known_args() .
pre = argparse.ArgumentParser()
pre.add_argument('--argument', required=False, action='store_true', default=False)
args_pre=pre.parse_known_args()
p = argparse.ArgumentParser()
p.add_argument('--argument', required=False)
p.add_argument('-a', required=args_pre.argument)
p.add_argument('-b', required=not args_pre.argument)

Parsing argparse input bit by bit

I am using Argparse to parse shell input to my Python function.
The tricky part is that this script first reads in a file that partially determines what kind of arguments are available to Argparse (it's a JSON file containing criteria by which the user can specify what data to output).
But before these arguments are added to my parser, I would like to read in some arguments relating to the file reading itself. (e.g. whether to fix the formatting of the input file). Kinda like this:
test.py (fix_formatting=True, **more arguments added later)
When I try to run args = parser.parse_args() twice, after the initial input and after adding more keys, things fall apart: Argparse quite predictably complains that some of the user input are unrecognized arguments:. I thought I might use subparsers to that end.
So I tried variations of (following the example in the docs as best as I could):
def main():
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(help='sub-command help')
settingsparser = subparsers.add_parser('settings') #i want a subparser called 'settings'
settingsparser.add_argument('--fix_formatting', action='store_true') #this subparser shall have a --fix_formatting
Then I try to parse only the "settings" part like so:
settings=parser.parse_args(['settings'])
This seems to work. But then I add my other keys and things break:
keys=['alpha','beta','gamma','delta']
for key in keys:
parser.add_argument("--"+key, type=str, help="X")
args = parser.parse_args()
If I parse any input for any of the arguments from keys, Argparse complains that I make an invalid choice: [...] (choose from 'settings'). Now I don't understand why I have to choose from "settings"; the docs say that the parse
will only contain attributes for the main parser and the subparser that was selected by the command line (and not any other subparsers)
what is my error of understanding here?
and if this is the wrong approach, how would one go about parsing one bit of input before another bit?
Any help is much appreciated!

parse_args calls parse_known_args. This returns the args namesparse along with a list of strings (from sys.argv) that it could not process (extras). parse_args raises this error if this list is not empty.
https://docs.python.org/3/library/argparse.html#partial-parsing
Thus parse_known_args is useful if you want to parse some of the input.
sys.argv remains unchanged. Subsequent calls to a parser (whether it was the original one or not) use that again, unless you pass the extras.
I don't think subparsers help you here. They aren't meant for delayed or two stage parsing. I'd suggest playing with the documentation examples for subparsers first.
To the main parser, the subparsers look like
subparsers = parser.add_argument('cmd', choices=['select',...])
In other words, it adds a positional argument where the choices are the subparser names that you define. That may help you see why it expects you to name select. Positionals are normally required.
(there's an exception to this in recent versions, https://stackoverflow.com/a/22994500/901925)

How ca I get Python ArgParse to stop overwritting positional arguments in child parser

I am attempting to get my script working, but argparse keeps overwriting my positional arguments from the parent parser. How can I get argparse to honor the parent's value for these? It does keep values from optional args.
Here is a very simplified version of what I need. If you run this, you will see that the args are overwritten.
testargs.py
#! /usr/bin/env python3
import argparse
import sys
def main():
preparser = argparse.ArgumentParser(add_help=False)
preparser.add_argument('first',
nargs='?')
preparser.add_argument('outfile',
nargs='?',
type=argparse.FileType('w', encoding='utf-8'),
default=sys.stdout,
help='Output file')
preparser.add_argument(
'--do-something','-d',
action='store_true')
# Parse args with preparser, and find config file
args, remaining_argv = preparser.parse_known_args()
print(args)
parser = argparse.ArgumentParser(
parents=[preparser],
description=__doc__)
parser.add_argument(
'--clear-screen', '-c',
action='store_true')
args = parser.parse_args(args=remaining_argv,namespace=args )
print(args)
if __name__ == '__main__':
main()
And call it with testargs.py something /tmp/test.txt -d -c
You will see it keeps the -d but drops both the positional args and reverts them to defaults.
EDIT: see additional comments in the accepted answer for some caveats.

When you specify parents=[preparser] it means that parser is an extension of preparser, and will parse all arguments relevent to preparser which it is never given.
Lets say the preparser only has one positional argument first and the parser only has one positional argument second, when you make parser a child of preparser it expects both arguments:
import argparse
parser1 = argparse.ArgumentParser(add_help=False)
parser1.add_argument("first")
parser2 = argparse.ArgumentParser(parents=[parser1])
parser2.add_argument("second")
args2 = parser2.parse_args(["arg1","arg2"])
assert args2.first == "arg1" and args2.second == "arg2"
However passing only the remaining arguments that are left over from parser1 would just be ['second'] which is not the correct arguments to parser2:
parser1 = argparse.ArgumentParser(add_help=False)
parser1.add_argument("first")
args1, remaining_args = parser1.parse_known_args(["arg1","arg2"])
parser2 = argparse.ArgumentParser(parents=[parser1])
parser2.add_argument("second")
>>> args1
Namespace(first='arg1')
>>> remaining_args
['arg2']
>>> parser2.parse_args(remaining_args)
usage: test.py [-h] first second
test.py: error: the following arguments are required: second
To only process the arguments that were not handled by the first pass, do not specify it as the parent to the second parser:
parser1 = argparse.ArgumentParser(add_help=False)
parser1.add_argument("first")
args1, remaining_args = parser1.parse_known_args(["arg1","arg2"])
parser2 = argparse.ArgumentParser() #parents=[parser1]) #NO PARENT!
parser2.add_argument("second")
args2 = parser2.parse_args(remaining_args,args1)
assert args2.first == "arg1" and args2.second == "arg2"

The 2 positionals are nargs='?'. A positional like that is always 'seen', since an empty list matches that nargs.
First time through 'text.txt' matches with first and is put in the Namespace. Second time through there isn't any string to match, so the default is used - same as if you had not given that string the first time.
If I change first to have the default nargs, I get
error: the following arguments are required: first
from the 2nd parser. Even though there's a value in the Namespace it still tries to get a value from the argv. (it's like a default, but not quite).
Defaults for positionals with nargs='?' (or *) are tricky. They are optional, but not in quite the same way as optionals. The positional Actions are still called, but with a empty list of values.
I don't think the parents feature does anything for you. preparser already handles that set of arguments; there's no need to handle them again in parser, especially since all the relevant argument strings have been stripped out.
Another option is to leave the parents in, but use the default sys.argv[1:] in the 2nd parser. (but beware of side effects like opening files)
args = parser.parse_args(namespace=args )
A third option is to parse the arguments independently and merge them with a dictionary update.
adict = vars(preparse_args)
adict.update(vars(parser_args))
# taking some care in who overrides who
For more details look in argparse.py file at ArgumentParser._get_values, specifically the not arg_strings cases.
A note about the FileType. That type works nicely for small scripts where you will use the files right away and exit. It isn't so good on large programs where you might want to close the file after use (close stdout???), or use files in a with context.
edit - note on parents
add_argument creates an Action object, and adds it to the parser's list of actions. parse_args basically matches input strings with these actions.
parents just copies those Action objects (by reference) from parent to child. To the child parser it is just as though the actions were created with add_argument directly.
parents is most useful when you are importing a parser and don't have direct access to its definition. If you are defining both parent and child, then parents just saves you some typing/cut-n-paste.
This and other SO questions (mostly triggered the by-reference copy) show that the developers did not intend you to use both the parent and child to do parsing. It can be done, but there are glitches that the they did not consider.
===================
I can imagine defining a custom Action class that would 'behave' in a situation like this. It might, for example, check the namespace for some not default value before adding its own (possibly default) value.
Consider, for example if I changed the action of first to 'append':
preparser.add_argument('first', action='append', nargs='?')
The result is:
1840:~/mypy$ python3 stack37147683.py /tmp/test.txt -d -c
Namespace(do_something=True, first=['/tmp/test.txt'], outfile=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>)
Namespace(clear_screen=True, do_something=True, first=['/tmp/test.txt', None], outfile=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>)
From the first parser, first=['/tmp/test.txt']; from the second, first=['/tmp/test.txt', None].
Because of the append, the item from the first is preserved, and a new default has been added by the second parser.

Python argparse : how to detect duplicated optional argument?

I'm using argparse with optional parameter, but I want to avoid having something like this : script.py -a 1 -b -a 2
Here we have twice the optional parameter 'a', and only the second parameter is returned. I want either to get both values or get an error message.
How should I define the argument ?
[Edit]
This is the code:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-a', dest='alpha', action='store', nargs='?')
parser.add_argument('-b', dest='beta', action='store', nargs='?')
params, undefParams = self.parser.parse_known_args()

append action will collect the values from repeated use in a list
parser.add_argument('-a', '--alpha', action='append')
producing an args namespace like:
namespace(alpha=['1','3'], b='4')
After parsing you can check args.alpha, and accept or complain about the number of values. parser.error('repeated -a') can be used to issue an argparse style error message.
You could implement similar functionality in a custom Action class, but that requires understanding the basic structure and operation of such a class. I can't think anything that can be done in an Action that can't just as well be done in the appended list after.
https://stackoverflow.com/a/23032953/901925 is an answer with a no-repeats custom Action.
Why are you using nargs='?' with flagged arguments like this? Without a const parameter this is nearly useless (see the nargs=? section in the docs).
Another similar SO: Python argparse with nargs behaviour incorrect

Argparse: ignore multiple positional arguments when optional argument is specified

I'm trying to make argparse ignore the fact that two normally required positional arguments shouldn't be evaluated when an optional argument (-l) is specified.
Basically I'm trying to replicate the behavior of --help: when you specify the -h, all missing required arguments are ignored.
Example code:
parser = argparse.ArgumentParser(description="Foo bar baz")
parser.add_argument('arg1', help='arg1 is a positional argument that does this')
parser.add_argument('arg2', help='arg2 is a positional argument that does this')
parser.add_argument('-l', '--list', dest='list', help='this is an optional argument that prints stuff')
options, args = parser.parse_args()
if options.list:
print "I list stuff"
And of course, if I run it now, I get :
error: too few arguments
I tried different things like nargs='?', but couldn't get anything working.
This question is quite similar but wasn't answered.

Unfortunately, argparse isn't quite flexible enough for this. The best you can do is to make arg1 and arg2 optional using nargs="?" and check yourself whether they are given if needed.
The internal help action is implemented by printing the help message and exiting the program as soon as -h or --help are encountered on the command line. You could write a similar action yourself, something like
class MyAction(argparse.Action):
def __call__(self, parser, values, namespace, option_string):
print "Whatever"
parser.exit()
(Warning: untested code!)
There are definite downsides to the latter approac, though. The help message will unconditionally show arg1 and arg2 as compulsory arguments. And parsing the command line simply stops when encountering -l or --list, ignoring any further arguments. This behaviour is quite acceptable for --help, but is less than desirable for other options.

I ran into this issue and decided to use subcommands. Subcommands might be overkill, but if you find your program not using some of the positional arguments in many instances (as I did), then subcommands might be a good solution.
For your given example, I'd use something like the following:
parser = argparse.ArgumentParser(description="Foo bar baz")
subparsers = parser.add_subparsers(description='available subcommands')
parser_main = subparsers.add_parser('<main_command_name>')
parser_main.add_argument('arg1', help='arg1 is a positional argument that does this')
parser_main.add_argument('arg2', help='arg2 is a positional argument that does this')
parser_list = subparsers.add_parser('list', help='this is a subcommand that prints stuff')
options, args = parser.parse_args()
I left out some details that you might want to include (like set_defaults(func=list)), which are mentioned in the argparse documentation.

I think the nargs='*' is helpful.
Positional arguments is ignorable, then you can use if to check the positional arguments is true or false.
http://docs.python.org/library/argparse.html#nargs

The cleanest approach I've been able to find so far is to split the parsing into two stages. First check for -l/--list:
parser = argparse.ArgumentParser(description="Foo bar baz")
parser.add_argument('-l', '--list', dest='list', action='store_true',
help='this is an optional argument that prints stuff')
options, remainder = parser.parse_known_args()
Now, since you used parse_known_args, you won't get an error up to here, and you can decide what to do with the remainder of the arguments:
if options.list:
print "I list stuff"
else:
parser = argparse.ArgumentParser(add_help=False)
parser.add_argument('arg1', help='arg1 is a positional argument that does this')
parser.add_argument('arg2', help='arg2 is a positional argument that does this')
options = parser.parse_args(remainder)
You may want to set the usage option in the first parser to make the help string a bit nicer.

I may have found a solution here. True, it is a dirty hack, but it works.
Note: all the following applies to Python 3.3.2.
As per the answer here, parse_args checks which actions are required and throws an error if any of them are missing. I propose to override this behavior.
By subclassing ArgumentParser we can define a new ArgumentParser.error method (original here) that will check whether the error was thrown because some arguments are missing and take necessary action. Code follows:
import argparse
import sys
from gettext import gettext as _
class ArgumentParser(argparse.ArgumentParser):
skip_list = []
def error(self, message):
# Let's see if we are missing arguments
if message.startswith('the following arguments are required:'):
missingArgs = message.split('required: ')[1].split(', ')
newArgs = [] # Creating a list of whatever we should not skip but is missing
for arg in missingArgs:
if arg not in self.skip_list:
newArgs.append(arg)
else:
self.skip_list.remove(arg) # No need for it anymore
if len(newArgs) == 0:
return # WARNING! UNTESTED! MAY LEAD TO SPACETIME MELTDOWN!
else: # Some required stuff is still missing, so we show a corrected error message
message = _('the following arguments are required: %s') % ', '.join(newArgs)
self.print_usage(sys.stderr) # Original method behavior
args = {'prog': self.prog, 'message': message}
self.exit(2, _('%(prog)s: error: %(message)s\n') % args)
The new method first checks whether the error is because arguments are missing from the command line (see here for the code that generates the error). If so, the method gets the names of the arguments from the error message and puts them into a list (missingArgs).
Then, we iterate over this list and check which arguments should be skipped, and which are still required. To determine which arguments to skip, we compare them against skip_list. It is a field in our ArgumentParser subclass that should contain the names of the arguments to skip even when they are required by the parser. Please note that arguments that happen to be in skip_list are removed from it when they are found.
If there are still required arguments that are missing from the command line, the method throws a corrected error message. If all the missing arguments should be skipped, however, the method returns.
WARNING! The original definition of ArgumentParser.error states that if it is overridden in a subclass it should not return, but rather exit or raise an exception. Therefore, what is shown here is potentially unsafe and may cause your program to crash, your computer to catch fire or worse - IT MAY EVAPORATE ALL YOUR TEA. However, it seems like in this particular case (missing required arguments) it is safe to return from the method. But it might not be. You have been warned.
In order to fill skip_list we could use code like this:
class SpecialHelp(argparse._HelpAction):
def __call__(self, parser, namespace, values, option_string=None):
parser.print_help()
print()
for action in parser._actions:
if action != self and action.required:
parser.skip_list.append(argparse._get_action_name(action))
This particular class imitates the built-in help action, but instead of exiting it inserts all the remaining required arguments into skip_list.
Hope my answer helps and best of luck.

It may be ugly, but that is what I normally do.
def print_list():
the_list = ["name1", "name2"]
return "{0}".format(the_list)
...
parser.add_argument("-l", "--list", action='version',
version=print_list(), help="print the list")