We Keep Coding

Python argparse - make arguments required or optional based on another argument

How can a program accept/validate a set of parameters, depending on a previous parameter/option?
e.g:
params:
<action1> -p <path> -name <name> -t <type>
<action2> -v <value> -name <name>
<action3> -p <path> -t <type>
<action4> -m <mode1 | mode2>
--verbose
--test
--..
So if one of the actionX parameters is used (only one can be used), additional parameters might be required.
For instance for action2 the -v and -name are required.
valid input:
python myparser.py action2 -v 11 -name something --test --verbose
python myparser.py action4 -m mode1
python myparser.py --test
invalid input:
python myparser.py action2 -v 11
python myparser.py action4 -n name1
Can the argparse validate this or is it better to add all of them as optional and validate them later on?

You can use subparsers
Simple example for your case
from argparse import ArgumentParser
parser = ArgumentParser()
parser.add_argument("--test", action="store_true")
subparsers = parser.add_subparsers()
parser_action2 = subparsers.add_parser("action2")
parser_action2.add_argument("-v", required=True)
parser_action2.add_argument("-name", type=str, required=True)
parser_action4 = subparsers.add_parser("action4")
parser_action4.add_argument("-m", type=str, required=True)
valid case
parser.parse_args(["action2", "-v", "11", "-name", "something"])
parser.parse_args(["action4", "-m", "mode1"])
parser.parse_args(["--test"])
# Namespace(v='11', name='something')
# Namespace(m='mode1')
# Namespace(test=True)
Invalid case
parser.parse_args(["action2", "-v", "11"])
# action2: error: the following arguments are required: -name
parser.parse_args(["action4", "-n", "name1"])
# action4: error: the following arguments are required: -m
EDIT: And in case you want to use shared argument which is required in one and optional in others, it is better to make them optional and validate them later as you said.

Put subparser command at the beginning of arguments

I wanted to create an interface with argparse for my script with subcommands; so, if my script is script.py, I want to call it like python script.py command --foo bar, where command is one of the N possible custom commands.
The problem is, I already tried looking for a solution here on StackOverflow, but it seems like everything I tried is useless.
What I have currently is this:
parser = argparse.ArgumentParser()
parser.add_argument("-x", required=True)
parser.add_argument("-y", required=True)
parser.add_argument("-f", "--files", nargs="+", required=True)
# subparsers for commands
subparsers = parser.add_subparsers(title="Commands", dest="command")
subparsers.required = True
summary_parser = subparsers.add_parser("summary", help="help of summary command")
If I try to run it with:
args = parser.parse_args("-x 1 -y 2 -f a/path another/path".split())
I got this error, as it should be: script.py: error: the following arguments are required: command.
If, however, I run this command:
args = parser.parse_args("summary -x 1 -y 2 -f a/path another/path".split())
I got this error, that I shouldn't have: script.py: error: the following arguments are required: -x, -y, -f/--files.
If I put the command at the end, changing also the order of arguments because of -f, it works.
args = parser.parse_args("-x 1 -f a/path another/path -y 2 summary".split())
If I add the parents keyword in subparser, so substitute the summary_parser line with summary_parser = subparsers.add_parser("summary", help=HELP_CMD_SUMMARY, parents=[parser], add_help=False), then I got:
script.py summary: error: the following arguments are required: command when summary is in front of every other argument;
script.py summary: error: the following arguments are required: -x, -y, -f/--files, command when summary is at the end of the args.
My question is, how I have to setup the parsers to have the behaviour script.py <command> <args>? Every command shares the same args, because they are needed to create certain objects, but at the same time every command can needs other arguments too.

Creating another parser helped me getting what I wanted.
The root parser should add all the optional arguments - and also have add_help=False, to avoid an help message conflict -, then another parser - parser2, with a lot of fantasy - will be created.
The second parser will have subparsers, and they all needs to specify as parents the root parser.
parser = argparse.ArgumentParser(add_help=False)
parser.add_argument() ...
parser2 = argparse.ArgumentParser()
subparsers = parser2.add_subparsers(title="Commands", dest="command")
subparsers.required = True
summary_parser = subparsers.add_parser("summary", parents=[parser])
summary_parser.add_argument("v", "--verbose", action="store_true")
# parse args
args = parser2.parse_args()
Now the output will be this:
usage: script.py [-h] {summary} ...
optional arguments:
-h, --help show this help message and exit
Commands:
{summary}
summary For each report print its summary, then exit
usage: script.py summary [-h] -x X -y Y -f FILES [FILES ...] [-v]
optional arguments:
-h, --help show this help message and exit
-x X
-y Y
-f FILES [FILES ...], --files FILES [FILES ...]
-v, --verbose

argparse with 2 date arguments

I want to create a program which selects users from a database between 2 dates given on the command line. I have:
import argparse
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument("--date1","-d1",help="Show users between dates",type=str)
group.add_argument("--date2","-d2",help="Show users between dates",type=str)
if args.date1 and args.date2:
DataCalculation.show_users_between_date(args.date1,args.date2)
And in my DataCalculation I have query to get users between 2 dates.
Unfortunately this solution doesnt work and I get error: argument --date2/-d2: not allowed with argument --date1/d1
I was running program like: py main.py -d1 1994-01-01 -d2 1995-12-31
I was thinking that I can split these 2 dates to list in function and give only 1 argument like: py main.py -d 1994-01-01 1995-12-31, but this idea doesn't work too. Is there an easy way to use 2 arguments which have to be given together?

You're looking for inclusivity, not exclusivity. You can accomplish that by using nargs=2 with one option, like your second case.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument(
"--date",
"-d",
nargs=2,
metavar=('start', 'end'), # Describes each argument
help="Show users between start and end dates",
)
args = parser.parse_args()
print(args)
Usage:
$ ./tmp.py -d 1994-01-01 1995-12-31
Namespace(date=['1994-01-01', '1995-12-31'])
$ ./tmp.py -d 1994-01-01
usage: tmp.py [-h] [--date start end]
tmp.py: error: argument --date/-d: expected 2 arguments
$ ./tmp.py -d 1994-01-01 1995-12-31 1998
usage: tmp.py [-h] [--date start end]
tmp.py: error: unrecognized arguments: 1998
$ ./tmp.py -h
usage: tmp.py [-h] [--date start end]
optional arguments:
-h, --help show this help message and exit
--date start end, -d start end
Show users between start and end dates

You could use
import argparse
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument("--daterange","-dr",help="Show users between dates",type=str)
args = parser.parse_args()
date1, date2 = args.daterange.split()
print(date1)
And then quotes around your arguments as in
python test.py -dr "1994-01-01 1995-12-31"
Which yields with the above snippet:
1994-01-01

argparse command line -option after given path

I'm new to python and currently experimenting using argparse to add command line options. However my code is not working, despite looking at various online tutorials and reading up on argparse I still don't fully understand it. My problem is whenever I try to call my -option it gives me a find.py error: argument regex:
Here is my call:
./find.py ../Python -name '[0-9]*\.txt'
../Python is one directory behind my current one and has a list of files/directories. Without the -name option I print out the files with their path (this works fine) but with the -name option I want to print out files matching the regex but it won't work. Here is what I currently have:
#!/usr/bin/python2.7
import os, sys, argparse,re
from stat import *
def regex_type(s, pattern=re.compile(r"[a-f0-9A-F]")):
if not pattern.match(s):
raise argparse.ArgumentTypeError
return s
def main():
direc = sys.argv[1]
for f in os.listdir(direc):
pathname = os.path.join(direc, f)
mode = os.stat(pathname).st_mode
if S_ISREG(mode):
print pathname
parser = argparse.ArgumentParser()
parser.add_argument(
'-name', default=[sys.stdin], nargs="*")
parser.add_argument('regex', type=regex_type)
args = parser.parse_args()
if __name__ == '__main__':
main()

I tweaked your type function to be more informative:
def regex_type(s, pattern=re.compile(r"[a-f0-9A-F]")):
print('regex string', s)
if not pattern.match(s):
raise argparse.ArgumentTypeError('pattern not match')
return s
Called with
2104:~/mypy$ python2 stack50072557.py .
I get:
<director list>
('regex string', '.')
usage: stack50072557.py [-h] [-name [NAME [NAME ...]]] regex
stack50072557.py: error: argument regex: pattern not match
So it tries to pass sys.argv[1], the first string after the script name, to the regex_type function. If it fails it issues the error message and usage.
OK, the problem was the ..; I'll make a directory:
2108:~/mypy$ mkdir foo
2136:~/mypy$ python2 stack50072557.py foo
('regex string', 'foo')
Namespace(name=[<open file '<stdin>', mode 'r' at 0x7f3bea2370c0>], regex='foo')
2138:~/mypy$ python2 stack50072557.py foo -name a b c
('regex string', 'foo')
Namespace(name=['a', 'b', 'c'], regex='foo')
The strings following '-name' are allocated to that attribute. There's nothing in your code that will test them or pass them through the regex_type function. Only the first non-flag string does that.
Reading sys.argv[1] initially does not remove it from the list. It's still there for use by the parser.
I would set up a parser that uses a store_true --name argument, and 2 positionals - one for the dir and the other for regex.
After parsing check args.name. If false print the contents of args.dir. If true, perform your args.regex filter on those contents. glob might be useful.
The parser finds out what your user wants. Your own code acts on it. Especially as a beginner, it is easier and cleaner to separate the two steps.
With:
def parse(argv=None):
parser = argparse.ArgumentParser()
parser.add_argument('-n', '--name', action='store_true')
parser.add_argument('--dir', default='.')
parser.add_argument('--regex', default=r"[a-f0-9A-F]")
args = parser.parse_args(argv)
print(args)
return args
def main(argv=None):
args = parse(argv)
dirls = os.listdir(args.dir)
if args.name:
dirls = [f for f in dirls if re.match(args.regex, f)]
print(dirls)
else:
print(dirls)
I get runs like:
1005:~/mypy$ python stack50072557.py
Namespace(dir='.', name=False, regex='[a-f0-9A-F]')
['test.npz', 'stack49909128.txt', 'stack49969840.txt', 'stack49824248.py', 'test.h5', 'stack50072557.py', 'stack49963862.npy', 'Mcoo.npz', 'test_attribute.h5', 'stack49969861.py', 'stack49969605.py', 'stack49454474.py', 'Mcsr.npz', 'Mdense.npy', 'stack49859957.txt', 'stack49408644.py', 'Mdok', 'test.mat5', 'stack50012754.py', 'foo', 'test']
1007:~/mypy$ python stack50072557.py -n
Namespace(dir='.', name=True, regex='[a-f0-9A-F]')
['foo']
1007:~/mypy$ python stack50072557.py -n --regex='.*\.txt'
Namespace(dir='.', name=True, regex='.*\\.txt')
['stack49909128.txt', 'stack49969840.txt', 'stack49859957.txt']
and help:
1007:~/mypy$ python stack50072557.py -h
usage: stack50072557.py [-h] [-n] [--dir DIR] [--regex REGEX]
optional arguments:
-h, --help show this help message and exit
-n, --name
--dir DIR
--regex REGEX
If I change the dir line to:
parser.add_argument('dir', default='.')
help is now
1553:~/mypy$ python stack50072557.py -h
usage: stack50072557.py [-h] [-n] [--regex REGEX] dir
positional arguments:
dir
optional arguments:
-h, --help show this help message and exit
-n, --name
--regex REGEX
and runs are:
1704:~/mypy$ python stack50072557.py -n
usage: stack50072557.py [-h] [-n] [--regex REGEX] dir
stack50072557.py: error: too few arguments
1705:~/mypy$ python stack50072557.py . -n
Namespace(dir='.', name=True, regex='[a-f0-9A-F]')
['foo']
1705:~/mypy$ python stack50072557.py ../mypy -n --regex='.*\.txt'
Namespace(dir='../mypy', name=True, regex='.*\\.txt')
['stack49909128.txt', 'stack49969840.txt', 'stack49859957.txt']
I get the error because it now requires a directory, even it is '.'.
Note that the script still uses:
if __name__ == '__main__':
main()
My main loads the dir, and applies the regex filter to that list of names. My args.dir replaces your direc.

Using stdin in a mutually exclusive argument group

I'm trying to use argparse in a script such that I could pass a string either as an argument or via standard input. I figured I could use add_mutually_exclusive_group for that, and set required=True to enforce that just one argument is required. So, I created the following script (my_script.py):
#!/usr/bin/env python
import argparse
from sys import stdin
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('str_arg', nargs='?', type=str)
group.add_argument('str_in', nargs='?', type=argparse.FileType('r'), default=stdin)
args = parser.parse_args()
print args.str_arg or args.str_in.readline()
Passing the string as a parameter works fine. However, when I try to pipe the string from standard input like so:
$ echo Hello | python my_script.py
Python complains that one of the arguments str_arg str_in is required. What am I doing wrong? Is there a better way of achieving this?

It doesn't seem like you should try doing this with fancy argparse features, just some simple logic:
import argparse
from sys import stdin
parser = argparse.ArgumentParser()
parser.add_argument('str_arg', nargs='?', type=str)
args = parser.parse_args()
print(args.str_arg or stdin.readline())

2 positionals in a mutually exclusive group does not work.
import argparse
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('str_arg', nargs='?')
group.add_argument('str_in', nargs='?')
args = parser.parse_args()
print(args)
sample runs:
0217:~/mypy$ python3 stack49705916.py
usage: stack49705916.py [-h] (str_arg | str_in)
stack49705916.py: error: one of the arguments str_arg str_in is required
0904:~/mypy$ python3 stack49705916.py foo
Namespace(str_arg='foo', str_in=None)
0904:~/mypy$ python3 stack49705916.py foo bar
usage: stack49705916.py [-h] (str_arg | str_in)
stack49705916.py: error: argument str_in: not allowed with argument str_arg
piping is not a substitute for a commandline argument. stdin has to be read separately.
import argparse, sys
print(sys.argv)
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('str_arg', nargs='?')
group.add_argument('str_in', nargs='?')
args = parser.parse_args()
print(args)
print(sys.stdin.read())
0909:~/mypy$ echo 'hello' | python3 stack49705916.py foo
['stack49705916.py', 'foo']
Namespace(str_arg='foo', str_in=None)
hello
argparse.FileType('r') recognizes - as stdin.
print(sys.argv)
parser = argparse.ArgumentParser()
parser.add_argument('-i', '--str_in', type=argparse.FileType('r'))
args = parser.parse_args()
print(args)
print(args.str_in.read())
runs
0945:~/mypy$ python3 stack49705916.py -i test.txt
['stack49705916.py', '-i', 'test.txt']
Namespace(str_in=<_io.TextIOWrapper name='test.txt' mode='r' encoding='UTF-8'>)
2000+0
2001+2
2002+1
0946:~/mypy$ python3 stack49705916.py -i -
['stack49705916.py', '-i', '-']
Namespace(str_in=<_io.TextIOWrapper name='<stdin>' mode='r' encoding='UTF-8'>)
typing hello on input line
typing hello on input line
0947:~/mypy$ echo Hello | python3 stack49705916.py -i -
['stack49705916.py', '-i', '-']
Namespace(str_in=<_io.TextIOWrapper name='<stdin>' mode='r' encoding='UTF-8'>)
Hello
0947:~/mypy$

In a shell you can pass the output of echo Whatever as an argument using back quotes and not a pipe:
$ python my_script.py `echo whatever`