I have a requirement as follows:
./xyifier --prox --lport lport --rport rport
for the argument prox , I use action='store_true' to check if it is present or not.
I do not require any of the arguments. But, if --prox is set I require rport and lport as well. Is there an easy way of doing this with argparse without writing custom conditional coding.
More Code:
non_int.add_argument('--prox', action='store_true', help='Flag to turn on proxy')
non_int.add_argument('--lport', type=int, help='Listen Port.')
non_int.add_argument('--rport', type=int, help='Proxy port.')
No, there isn't any option in argparse to make mutually inclusive sets of options.
The simplest way to deal with this would be:
if args.prox and (args.lport is None or args.rport is None):
parser.error("--prox requires --lport and --rport.")
Actually there's already an open PR with an enhancement proposal :
https://github.com/python/cpython/issues/55797
You're talking about having conditionally required arguments. Like #borntyping said you could check for the error and do parser.error(), or you could just apply a requirement related to --prox when you add a new argument.
A simple solution for your example could be:
non_int.add_argument('--prox', action='store_true', help='Flag to turn on proxy')
non_int.add_argument('--lport', required='--prox' in sys.argv, type=int)
non_int.add_argument('--rport', required='--prox' in sys.argv, type=int)
This way required receives either True or False depending on whether the user as used --prox. This also guarantees that -lport and -rport have an independent behavior between each other.
How about using parser.parse_known_args() method and then adding the --lport and --rport args as required args if --prox is present.
# just add --prox arg now
non_int = argparse.ArgumentParser(description="stackoverflow question",
usage="%(prog)s [-h] [--prox --lport port --rport port]")
non_int.add_argument('--prox', action='store_true',
help='Flag to turn on proxy, requires additional args lport and rport')
opts, rem_args = non_int.parse_known_args()
if opts.prox:
non_int.add_argument('--lport', required=True, type=int, help='Listen Port.')
non_int.add_argument('--rport', required=True, type=int, help='Proxy port.')
# use options and namespace from first parsing
non_int.parse_args(rem_args, namespace = opts)
Also keep in mind that you can supply the namespace opts generated after the first parsing while parsing the remaining arguments the second time. That way, in the the end, after all the parsing is done, you'll have a single namespace with all the options.
Drawbacks:
If --prox is not present the other two dependent options aren't even present in the namespace. Although based on your use-case, if --prox is not present, what happens to the other options is irrelevant.
Need to modify usage message as parser doesn't know full structure
--lport and --rport don't show up in help message
Do you use lport when prox is not set. If not, why not make lport and rport arguments of prox? e.g.
parser.add_argument('--prox', nargs=2, type=int, help='Prox: listen and proxy ports')
That saves your users typing. It is just as easy to test if args.prox is not None: as if args.prox:.
The accepted answer worked great for me! Since all code is broken without tests here is how I tested the accepted answer. parser.error() does not raise an argparse.ArgumentError error it instead exits the process. You have to test for SystemExit.
with pytest
import pytest
from . import parse_arguments # code that rasises parse.error()
def test_args_parsed_raises_error():
with pytest.raises(SystemExit):
parse_arguments(["argument that raises error"])
with unittests
from unittest import TestCase
from . import parse_arguments # code that rasises parse.error()
class TestArgs(TestCase):
def test_args_parsed_raises_error():
with self.assertRaises(SystemExit) as cm:
parse_arguments(["argument that raises error"])
inspired from: Using unittest to test argparse - exit errors
I did it like this:
if t or x or y:
assert t and x and y, f"args: -t, -x and -y should be given together"
Related
Code:
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Build dataset')
parser.add_argument('jpeg_dir', type=str, help='path to jpeg images')
parser.add_argument('nb_channels', type=int, help='number of image channels')
parser.add_argument('--img_size', default=256, type=int,
help='Desired Width == Height')
parser.add_argument('--do_plot', action="store_true",
help='Plot the images to make sure the data processing went OK')
args = parser.parse_args()
Error:
$ python make_dataset.py /home/abhishek/Lectures/columbia/deep_learning/project/DeepLearningImplementations/pix2pix/data/pix2pix/datasets 3 --img_size 256 --do_plot True
usage: make_dataset.py [-h] [--img_size IMG_SIZE] [--do_plot]
jpeg_dir nb_channels
make_dataset.py: error: unrecognized arguments: True
I am using a bash shell here. I am passing as mentioned in the docs https://github.com/tdeboissiere/DeepLearningImplementations/tree/master/pix2pix/src/data
As you've configured it, the --do_plot option does not take any arguments. A store_true argument in argparse indicates that the very presence of the option will automatically store True in the corresponding variable.
So, to prevent your problem, just stop passing True to --do_plot.
You do not need to indicate True as far as I can tell, by just including --do_plot, it is telling it that you wanted to do plot. And plus, you did not configure it to take any arguments.
In the following line of the source code:
if args.do_plot:
If you actually included --do_plot in the command lines, it will be evaluated as True, if not, it will be evaluated as False.
The problem is in the specification here:
parser.add_argument('--do_plot', action="store_true",
help='Plot ...')
You've declared do_plot as an option without an argument; the True afterward has no purpose in your argument protocol. This is an option that's off by omission, on when present.
Just one of reason (which I faced) and hope my hypothesis helps your problem is that on Ubuntu (on Windows, IDK but it's fine),
When you imported a function from a .py file (let say A.py) which having args (people create __main__ to test a feature function, let call A function ). The .py importing/using A could be confusedly parsing arguments because A.py also parse arguments and so on.
So, you could solve by refactoring, or just (temporarily) comment out them to run first.
I have a requirement as follows:
./xyifier --prox --lport lport --rport rport
for the argument prox , I use action='store_true' to check if it is present or not.
I do not require any of the arguments. But, if --prox is set I require rport and lport as well. Is there an easy way of doing this with argparse without writing custom conditional coding.
More Code:
non_int.add_argument('--prox', action='store_true', help='Flag to turn on proxy')
non_int.add_argument('--lport', type=int, help='Listen Port.')
non_int.add_argument('--rport', type=int, help='Proxy port.')
No, there isn't any option in argparse to make mutually inclusive sets of options.
The simplest way to deal with this would be:
if args.prox and (args.lport is None or args.rport is None):
parser.error("--prox requires --lport and --rport.")
Actually there's already an open PR with an enhancement proposal :
https://github.com/python/cpython/issues/55797
You're talking about having conditionally required arguments. Like #borntyping said you could check for the error and do parser.error(), or you could just apply a requirement related to --prox when you add a new argument.
A simple solution for your example could be:
non_int.add_argument('--prox', action='store_true', help='Flag to turn on proxy')
non_int.add_argument('--lport', required='--prox' in sys.argv, type=int)
non_int.add_argument('--rport', required='--prox' in sys.argv, type=int)
This way required receives either True or False depending on whether the user as used --prox. This also guarantees that -lport and -rport have an independent behavior between each other.
How about using parser.parse_known_args() method and then adding the --lport and --rport args as required args if --prox is present.
# just add --prox arg now
non_int = argparse.ArgumentParser(description="stackoverflow question",
usage="%(prog)s [-h] [--prox --lport port --rport port]")
non_int.add_argument('--prox', action='store_true',
help='Flag to turn on proxy, requires additional args lport and rport')
opts, rem_args = non_int.parse_known_args()
if opts.prox:
non_int.add_argument('--lport', required=True, type=int, help='Listen Port.')
non_int.add_argument('--rport', required=True, type=int, help='Proxy port.')
# use options and namespace from first parsing
non_int.parse_args(rem_args, namespace = opts)
Also keep in mind that you can supply the namespace opts generated after the first parsing while parsing the remaining arguments the second time. That way, in the the end, after all the parsing is done, you'll have a single namespace with all the options.
Drawbacks:
If --prox is not present the other two dependent options aren't even present in the namespace. Although based on your use-case, if --prox is not present, what happens to the other options is irrelevant.
Need to modify usage message as parser doesn't know full structure
--lport and --rport don't show up in help message
Do you use lport when prox is not set. If not, why not make lport and rport arguments of prox? e.g.
parser.add_argument('--prox', nargs=2, type=int, help='Prox: listen and proxy ports')
That saves your users typing. It is just as easy to test if args.prox is not None: as if args.prox:.
The accepted answer worked great for me! Since all code is broken without tests here is how I tested the accepted answer. parser.error() does not raise an argparse.ArgumentError error it instead exits the process. You have to test for SystemExit.
with pytest
import pytest
from . import parse_arguments # code that rasises parse.error()
def test_args_parsed_raises_error():
with pytest.raises(SystemExit):
parse_arguments(["argument that raises error"])
with unittests
from unittest import TestCase
from . import parse_arguments # code that rasises parse.error()
class TestArgs(TestCase):
def test_args_parsed_raises_error():
with self.assertRaises(SystemExit) as cm:
parse_arguments(["argument that raises error"])
inspired from: Using unittest to test argparse - exit errors
I did it like this:
if t or x or y:
assert t and x and y, f"args: -t, -x and -y should be given together"
I am using Argparse on python to do a script on command line. I have this for my script:
parser = argparse.ArgumentParser(prog = 'manageAdam')
parser.add_argument("-s", action='store_true', default=False, help='Shows configuration file')
parser.add_argument("d", type=str, help="device")
parser.add_argument("o", type=str, help="operation")
parser.add_argument("-v", "--value", type=int, nargs='*', help="value or list to send in the operation")
I am looking that if I call manageAdam -s it would work and don't ask for the positional arguments, something like the -h, which can be called without any other positional argument that is defined. Is it possible?
There is no built-in way to do this. You might be able to achieve something by writing some custom Action classes that keep track on the parser about their state, but I believe it will become quite messy and buggy.
I believe the best bet is to simply improve your UI. The -s is not an option. It's a separate command that completely alters how your script executes. In such cases you should use the subparsers functionality which allows to introduce sub-commands. This is a better interface then the one you thought, and is used by a lot of other tools (e.g. Git/mercurial).
In this case you'd have a config command to handle the configuration and a run (or how you want to call it) command to perform the operations on the device:
subparsers = parser.add_subparsers(dest='command')
parser_config = subparsers.add_parser('config', help='Configuration')
parser_run = subparsers.add_parser('run', help='Execute operation on device')
parser_run.add_argument('d', type=str, ...)
parser_run.add_argument('o', type=str, ...)
parser_run.add_argument('-v', type=int, nargs='*', ...)
# later:
args = parser.parse_args()
if args.command == 'config':
print('Configuration')
else:
print('Run operation')
Used from the command line as:
$ manageAdam config
# or
$ manageAdam run <device> <operation> <values...>
No, there are no such way.
You can make all arguments optional and set default value to None then perform check that all of them aren't None otherwise raise argparse.ArgumentError, if manageAdam provided skip check for other arguments.
In Python, how can I parse the command line, edit the resulting parsed arguments object and generate a valid command line back with the updated values?
For instance, I would like python cmd.py --foo=bar --step=0 call python cmd.py --foo=bar --step=1 with all the original --foo=bar arguments, potentially without extra arguments added when default value is used.
Is it possible with argparse?
You can use argparse to parse the command-line arguments, and then modify those as desired. At the moment however, argparse lacks the functionality to work in reverse and convert those values back into a command-line string. There is however a package for doing precisely that, called argunparse. For example, the following code in cmd.py
import sys
import argparse
import argunparse
parser = argparse.ArgumentParser()
unparser = argunparse.ArgumentUnparser()
parser.add_argument('--foo')
parser.add_argument('--step', type=int)
kwargs = vars(parser.parse_args())
kwargs['step'] += 1
prefix = f'python {sys.argv[0]} '
arg_string = unparser.unparse(**kwargs)
print(prefix + arg_string)
will print the desired command line:
python cmd.py --foo=bar --step=1
argparse is clearly designed to go one way, from sys.argv to the args namespace. No thought has been given to preserving information that would let you map things back the other way, much less do the mapping itself.
In general, multiple sys.argv could produce the same args. You could, for example, have several arguments that have the same dest. Or you can repeat 'optionals'. But for a restricted 'parser' setup there may be enough information to recreate a usable argv.
Try something like:
parser = argparser.ArgumentParser()
arg1 = parser.add_argument('--foo', default='default')
arg2 = parser.add_argument('bar', nargs=2)
and then examine the arg1 and arg2 objects. They contain all the information that you supplied to the add_argument method. Of course you could have defined those values in your own data structures before hand, e.g.
{'option_string':'--foo', 'default':'default'}
{'dest':'bar', 'nargs':2}
and used those as input to add_argument.
While the parser may have enough information to recreate a useable sys.argv, you have to figure out how to do that yourself.
default=argparse.SUPPRESS may be handy. It keeps the parser from adding a default entry to the namespace. So if the option isn't used, it won't appear in the namespace.
This isn't possible in any easy way that I know of, then again I've never needed to do this.
But with the lack of information in the question in regards to how you call your script, I'll assume the following:
python test.py cmd --foo=bar --step=0
And what you could do is do:
from sys import argv
for index in range(1, len(argv)): # the first object is the script itself
if '=' in argv[index]:
param, value = argv[index].split('=', 1)
if param == '--step':
value = '1'
argv[index] = param + '=' + value
print(argv)
Note that this is very specific to --step and may be what you've already thought of and just wanted a "better way", but again, I don't think there is.
depending on the scope, this works at the same module at least:
pprint(argparse._sys.argv)
Per the other answers, rebuilding is imperfect, but if you aren't doing anything too fancy and are okay with imperfect, something like this could work for you as a starting point:
def unparse_args(parser, parsed_args):
"""Unparse argparsed args"""
positional_args = [action.dest
for action in parser._actions
if not action.option_strings]
optionals = []
positionals = []
for key, value in vars(parsed_args).items():
if not value:
# none and false flags go away
continue
elif key in positional_args:
positionals.append(value)
elif value is True:
optionals.append(f"--{key}")
else:
optionals.append(f"--{key}={value}")
return " ".join(optionals + positionals)
Here's an example using this with a git clone clone:
parser = argparse.ArgumentParser(description='A sample git clone wrapper')
# options
parser.add_argument("-v", "--verbose", action="store_true",
help="be more verbose")
parser.add_argument("-q", "--quiet", action="store_true",
help="be more quiet")
parser.add_argument("--recurse-submodules", nargs='?',
help="initialize submodules in the clone")
parser.add_argument("--recursive", nargs='?',
help="alias of --recurse-submodules")
parser.add_argument("-b", "--branch",
help=" checkout <branch> instead of the remote's HEAD")
parser.add_argument("--depth", type=int,
help="create a shallow clone of that depth")
parser.add_argument("--shallow-submodules", action="store_true",
help="any cloned submodules will be shallow")
# positional
parser.add_argument("repo", help="The git repo to clone")
parser.add_argument("dir", nargs='?',help="The location to clone the repo")
# make a fake call to your git clone clone and parse the args
cmdargs = ["--depth=1", "-q", "ohmyzsh/ohmyzsh"]
parsedargs = parser.parse_args(cmdargs)
# now unparse them
unparsed = unparse_args(parser, parsedargs)
print(unparsed)
Let's say I have a script that does some work on a file. It takes this file's name on the command line, but if it's not provided, it defaults to a known filename (content.txt, say). With python's argparse, I use the following:
parser = argparse.ArgumentParser(description='my illustrative example')
parser.add_argument('--content', metavar='file',
default='content.txt', type=argparse.FileType('r'),
help='file to process (defaults to content.txt)')
args = parser.parse_args()
# do some work on args.content, which is a file-like object
This works great. The only problem is that if I run python myscript --help, I get an ArgumentError if the file isn't there (which I guess makes sense), and the help text is not shown. I'd rather it not try to open the file if the user just wants --help. Is there any way to do this? I know I could make the argument a string and take care of opening the file myself later (and I've been doing that), but it would be convenient to have argparse take care of it.
You could subclass argparse.FileType:
import argparse
import warnings
class ForgivingFileType(argparse.FileType):
def __call__(self, string):
try:
super(ForgivingFileType,self).__call__(string)
except IOError as err:
warnings.warn(err)
parser = argparse.ArgumentParser(description='my illustrative example')
parser.add_argument('--content', metavar='file',
default='content.txt', type=ForgivingFileType('r'),
help='file to process (defaults to content.txt)')
args = parser.parse_args()
This works without having to touch private methods like ArgumentParser._parse_known_args.
Looking at the argparse code, I see:
ArgumentParser.parse_args calls parse_known_args and makes sure that there isn't any pending argument to be parsed.
ArgumentParser.parse_known_args sets default values and calls ArgumentParser._parse_known_args
Hence, the workaround would be to use ArgumentParser._parse_known_args directly to detect -h and, after that, use ArgumentParser.parse_args as usual.
import sys, argparse
parser = argparse.ArgumentParser(description='my illustrative example', argument_default=argparse.SUPPRESS)
parser.add_argument('--content', metavar='file',
default='content.txt', type=argparse.FileType('r'),
help='file to process (defaults to content.txt)')
parser._parse_known_args(sys.argv[1:], argparse.Namespace())
args = parser.parse_args()
Note that ArgumentParser._parse_known_args needs a couple of parameters: the arguments from the command line and the namespace.
Of course, I wouldn't recommend this approach since it takes advantage of the internal argparse implementation and that might change in the future. However, I don't find it too messy, so you still might want to use it if you think maintenance risks pay off.
Use stdin as default:
parser.add_argument('file', default='-', nargs='?', type=argparse.FileType('r'))
Perhaps you could define your own type or action in the add_argument call that checks if the file exists, and returns a file handle if it does and None (or something else) otherwise.
This would require you to write some code of yourself as well though, but if the default value can not always be used you probably have to do some checking sooner or later. Like Manny D argues you might want to reconsider your default value.