Code:
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Build dataset')
parser.add_argument('jpeg_dir', type=str, help='path to jpeg images')
parser.add_argument('nb_channels', type=int, help='number of image channels')
parser.add_argument('--img_size', default=256, type=int,
help='Desired Width == Height')
parser.add_argument('--do_plot', action="store_true",
help='Plot the images to make sure the data processing went OK')
args = parser.parse_args()
Error:
$ python make_dataset.py /home/abhishek/Lectures/columbia/deep_learning/project/DeepLearningImplementations/pix2pix/data/pix2pix/datasets 3 --img_size 256 --do_plot True
usage: make_dataset.py [-h] [--img_size IMG_SIZE] [--do_plot]
jpeg_dir nb_channels
make_dataset.py: error: unrecognized arguments: True
I am using a bash shell here. I am passing as mentioned in the docs https://github.com/tdeboissiere/DeepLearningImplementations/tree/master/pix2pix/src/data
As you've configured it, the --do_plot option does not take any arguments. A store_true argument in argparse indicates that the very presence of the option will automatically store True in the corresponding variable.
So, to prevent your problem, just stop passing True to --do_plot.
You do not need to indicate True as far as I can tell, by just including --do_plot, it is telling it that you wanted to do plot. And plus, you did not configure it to take any arguments.
In the following line of the source code:
if args.do_plot:
If you actually included --do_plot in the command lines, it will be evaluated as True, if not, it will be evaluated as False.
The problem is in the specification here:
parser.add_argument('--do_plot', action="store_true",
help='Plot ...')
You've declared do_plot as an option without an argument; the True afterward has no purpose in your argument protocol. This is an option that's off by omission, on when present.
Just one of reason (which I faced) and hope my hypothesis helps your problem is that on Ubuntu (on Windows, IDK but it's fine),
When you imported a function from a .py file (let say A.py) which having args (people create __main__ to test a feature function, let call A function ). The .py importing/using A could be confusedly parsing arguments because A.py also parse arguments and so on.
So, you could solve by refactoring, or just (temporarily) comment out them to run first.
Related
I want to provide the command with 3 arguments: <version>, <input file> and <output file> under normal usage. Except there's a specific flag --init, which will basically run the program without any input and output file specifications required.
I would therefore ideally have a command which usage is:
py program.py --init
OR
py program.py <version> <input file> <output file>
However, since positional arguments are always required (because all 3 are required in any other circumstance than --init), there seems to be no way to cleanly get this syntax and all I could think of would be to make the 3 positional arguments into an optional flag, raise an exception if the optional flag isn't there when --init is not called. And that all seems ugly.
My code so far:
def get_args():
parser = argparse.ArgumentParser(description="Tool for FOO-ing a BAR.")
parser.add_argument(dest="version", help="The version.")
parser.add_argument(dest="input", help="The input file.")
parser.add_argument(dest="output", help="The output file.")
parser.add_argument("-i", "--init", dest="init", action="store_true", help="Foo Init.")
return parser.parse_args()
To Clarify:
Either all 3 arguments (<version> <input> <output>) MUST be specified.
OR
The program is only ran with --init flag and 0 arguments should be specified.
The program should NOT accept with 0, 1 or 2 arguments specified (without the --init flag).
You can define your own action class:
class init_action(argparse.Action):
def __init__(self, option_strings, dest, **kwargs):
return super().__init__(option_strings, dest, nargs=0, default=argparse.SUPPRESS, **kwargs)
def __call__(self, parser, namespace, values, option_string, **kwargs):
# Do whatever should be done here
parser.exit()
def get_args():
parser = argparse.ArgumentParser(description="Tool for FOO-ing a BAR.")
parser.add_argument(dest="version", help="The version.")
parser.add_argument(dest="input", help="The input file.")
parser.add_argument(dest="output", help="The output file.")
parser.add_argument("-i", "--init", action=init_action, help="Foo Init.")
return parser.parse_args()
I had the exact same problem, and fixed it by adding nargs="?" to all my positional arguments
def get_args():
parser = argparse.ArgumentParser(description="Tool for FOO-ing a BAR.")
parser.add_argument(dest="version", nargs="?", help="The version.")
parser.add_argument(dest="input", nargs="?", help="The input file.")
parser.add_argument(dest="output", nargs="?", help="The output file.")
parser.add_argument("-i", "--init", dest="init", action="store_true", help="Foo Init.")
return parser.parse_args()
but I'm not completely satisfied with it since when you run
py program.py -h
the synopsys is printed as
usage: program.py [-h] [--init] [version] [input] [output]
while I would want it to be
usage: program.py [-h] [--init] version input output
since version, input and output are required, just that they are not required when you use the --init flag.
Another downside is that you do not get the help text printed if you do not provide the positional arguments correctly.
I think it is sad that argparse have no way to set ignore_positional_args=True or something on flags. Especially since this property already is true for the -h flag. At least I'm not able to find a way to do it.
EDIT:
I actually found a way to better way to do it, but put it in a separate answer because I think this can be a useful approach as well.
parser.add_argument has a default parameter (docs) which can be used here for version, input and output parameters. Now you won't need third init param
One possible solution is to use sys.argv to determine whether the flag is enabled.
Using your example,
import sys
if '--init' not in sys.argv:
parser.add_argument(dest="version", help="The version.")
parser.add_argument(dest="input", help="The input file.")
parser.add_argument(dest="output", help="The output file.")
Argparse will allow you to run the program with either --init flag without any arguments or force you to include other arguments if --init flag is not present.
One downside to this approach is that you will see the following output, when you run py program.py -h:
usage: program.py [--init] version input output
positional arguments:
version - The version.
input - The input file.
output - The output file.
optional arguments:
--init - ...
which makes it hard for the user to understand the logic unless you clarify it in the description in the help section.
I have a requirement as follows:
./xyifier --prox --lport lport --rport rport
for the argument prox , I use action='store_true' to check if it is present or not.
I do not require any of the arguments. But, if --prox is set I require rport and lport as well. Is there an easy way of doing this with argparse without writing custom conditional coding.
More Code:
non_int.add_argument('--prox', action='store_true', help='Flag to turn on proxy')
non_int.add_argument('--lport', type=int, help='Listen Port.')
non_int.add_argument('--rport', type=int, help='Proxy port.')
No, there isn't any option in argparse to make mutually inclusive sets of options.
The simplest way to deal with this would be:
if args.prox and (args.lport is None or args.rport is None):
parser.error("--prox requires --lport and --rport.")
Actually there's already an open PR with an enhancement proposal :
https://github.com/python/cpython/issues/55797
You're talking about having conditionally required arguments. Like #borntyping said you could check for the error and do parser.error(), or you could just apply a requirement related to --prox when you add a new argument.
A simple solution for your example could be:
non_int.add_argument('--prox', action='store_true', help='Flag to turn on proxy')
non_int.add_argument('--lport', required='--prox' in sys.argv, type=int)
non_int.add_argument('--rport', required='--prox' in sys.argv, type=int)
This way required receives either True or False depending on whether the user as used --prox. This also guarantees that -lport and -rport have an independent behavior between each other.
How about using parser.parse_known_args() method and then adding the --lport and --rport args as required args if --prox is present.
# just add --prox arg now
non_int = argparse.ArgumentParser(description="stackoverflow question",
usage="%(prog)s [-h] [--prox --lport port --rport port]")
non_int.add_argument('--prox', action='store_true',
help='Flag to turn on proxy, requires additional args lport and rport')
opts, rem_args = non_int.parse_known_args()
if opts.prox:
non_int.add_argument('--lport', required=True, type=int, help='Listen Port.')
non_int.add_argument('--rport', required=True, type=int, help='Proxy port.')
# use options and namespace from first parsing
non_int.parse_args(rem_args, namespace = opts)
Also keep in mind that you can supply the namespace opts generated after the first parsing while parsing the remaining arguments the second time. That way, in the the end, after all the parsing is done, you'll have a single namespace with all the options.
Drawbacks:
If --prox is not present the other two dependent options aren't even present in the namespace. Although based on your use-case, if --prox is not present, what happens to the other options is irrelevant.
Need to modify usage message as parser doesn't know full structure
--lport and --rport don't show up in help message
Do you use lport when prox is not set. If not, why not make lport and rport arguments of prox? e.g.
parser.add_argument('--prox', nargs=2, type=int, help='Prox: listen and proxy ports')
That saves your users typing. It is just as easy to test if args.prox is not None: as if args.prox:.
The accepted answer worked great for me! Since all code is broken without tests here is how I tested the accepted answer. parser.error() does not raise an argparse.ArgumentError error it instead exits the process. You have to test for SystemExit.
with pytest
import pytest
from . import parse_arguments # code that rasises parse.error()
def test_args_parsed_raises_error():
with pytest.raises(SystemExit):
parse_arguments(["argument that raises error"])
with unittests
from unittest import TestCase
from . import parse_arguments # code that rasises parse.error()
class TestArgs(TestCase):
def test_args_parsed_raises_error():
with self.assertRaises(SystemExit) as cm:
parse_arguments(["argument that raises error"])
inspired from: Using unittest to test argparse - exit errors
I did it like this:
if t or x or y:
assert t and x and y, f"args: -t, -x and -y should be given together"
I am using Argparse on python to do a script on command line. I have this for my script:
parser = argparse.ArgumentParser(prog = 'manageAdam')
parser.add_argument("-s", action='store_true', default=False, help='Shows configuration file')
parser.add_argument("d", type=str, help="device")
parser.add_argument("o", type=str, help="operation")
parser.add_argument("-v", "--value", type=int, nargs='*', help="value or list to send in the operation")
I am looking that if I call manageAdam -s it would work and don't ask for the positional arguments, something like the -h, which can be called without any other positional argument that is defined. Is it possible?
There is no built-in way to do this. You might be able to achieve something by writing some custom Action classes that keep track on the parser about their state, but I believe it will become quite messy and buggy.
I believe the best bet is to simply improve your UI. The -s is not an option. It's a separate command that completely alters how your script executes. In such cases you should use the subparsers functionality which allows to introduce sub-commands. This is a better interface then the one you thought, and is used by a lot of other tools (e.g. Git/mercurial).
In this case you'd have a config command to handle the configuration and a run (or how you want to call it) command to perform the operations on the device:
subparsers = parser.add_subparsers(dest='command')
parser_config = subparsers.add_parser('config', help='Configuration')
parser_run = subparsers.add_parser('run', help='Execute operation on device')
parser_run.add_argument('d', type=str, ...)
parser_run.add_argument('o', type=str, ...)
parser_run.add_argument('-v', type=int, nargs='*', ...)
# later:
args = parser.parse_args()
if args.command == 'config':
print('Configuration')
else:
print('Run operation')
Used from the command line as:
$ manageAdam config
# or
$ manageAdam run <device> <operation> <values...>
No, there are no such way.
You can make all arguments optional and set default value to None then perform check that all of them aren't None otherwise raise argparse.ArgumentError, if manageAdam provided skip check for other arguments.
I have a requirement as follows:
./xyifier --prox --lport lport --rport rport
for the argument prox , I use action='store_true' to check if it is present or not.
I do not require any of the arguments. But, if --prox is set I require rport and lport as well. Is there an easy way of doing this with argparse without writing custom conditional coding.
More Code:
non_int.add_argument('--prox', action='store_true', help='Flag to turn on proxy')
non_int.add_argument('--lport', type=int, help='Listen Port.')
non_int.add_argument('--rport', type=int, help='Proxy port.')
No, there isn't any option in argparse to make mutually inclusive sets of options.
The simplest way to deal with this would be:
if args.prox and (args.lport is None or args.rport is None):
parser.error("--prox requires --lport and --rport.")
Actually there's already an open PR with an enhancement proposal :
https://github.com/python/cpython/issues/55797
You're talking about having conditionally required arguments. Like #borntyping said you could check for the error and do parser.error(), or you could just apply a requirement related to --prox when you add a new argument.
A simple solution for your example could be:
non_int.add_argument('--prox', action='store_true', help='Flag to turn on proxy')
non_int.add_argument('--lport', required='--prox' in sys.argv, type=int)
non_int.add_argument('--rport', required='--prox' in sys.argv, type=int)
This way required receives either True or False depending on whether the user as used --prox. This also guarantees that -lport and -rport have an independent behavior between each other.
How about using parser.parse_known_args() method and then adding the --lport and --rport args as required args if --prox is present.
# just add --prox arg now
non_int = argparse.ArgumentParser(description="stackoverflow question",
usage="%(prog)s [-h] [--prox --lport port --rport port]")
non_int.add_argument('--prox', action='store_true',
help='Flag to turn on proxy, requires additional args lport and rport')
opts, rem_args = non_int.parse_known_args()
if opts.prox:
non_int.add_argument('--lport', required=True, type=int, help='Listen Port.')
non_int.add_argument('--rport', required=True, type=int, help='Proxy port.')
# use options and namespace from first parsing
non_int.parse_args(rem_args, namespace = opts)
Also keep in mind that you can supply the namespace opts generated after the first parsing while parsing the remaining arguments the second time. That way, in the the end, after all the parsing is done, you'll have a single namespace with all the options.
Drawbacks:
If --prox is not present the other two dependent options aren't even present in the namespace. Although based on your use-case, if --prox is not present, what happens to the other options is irrelevant.
Need to modify usage message as parser doesn't know full structure
--lport and --rport don't show up in help message
Do you use lport when prox is not set. If not, why not make lport and rport arguments of prox? e.g.
parser.add_argument('--prox', nargs=2, type=int, help='Prox: listen and proxy ports')
That saves your users typing. It is just as easy to test if args.prox is not None: as if args.prox:.
The accepted answer worked great for me! Since all code is broken without tests here is how I tested the accepted answer. parser.error() does not raise an argparse.ArgumentError error it instead exits the process. You have to test for SystemExit.
with pytest
import pytest
from . import parse_arguments # code that rasises parse.error()
def test_args_parsed_raises_error():
with pytest.raises(SystemExit):
parse_arguments(["argument that raises error"])
with unittests
from unittest import TestCase
from . import parse_arguments # code that rasises parse.error()
class TestArgs(TestCase):
def test_args_parsed_raises_error():
with self.assertRaises(SystemExit) as cm:
parse_arguments(["argument that raises error"])
inspired from: Using unittest to test argparse - exit errors
I did it like this:
if t or x or y:
assert t and x and y, f"args: -t, -x and -y should be given together"
Let's say I have a script that does some work on a file. It takes this file's name on the command line, but if it's not provided, it defaults to a known filename (content.txt, say). With python's argparse, I use the following:
parser = argparse.ArgumentParser(description='my illustrative example')
parser.add_argument('--content', metavar='file',
default='content.txt', type=argparse.FileType('r'),
help='file to process (defaults to content.txt)')
args = parser.parse_args()
# do some work on args.content, which is a file-like object
This works great. The only problem is that if I run python myscript --help, I get an ArgumentError if the file isn't there (which I guess makes sense), and the help text is not shown. I'd rather it not try to open the file if the user just wants --help. Is there any way to do this? I know I could make the argument a string and take care of opening the file myself later (and I've been doing that), but it would be convenient to have argparse take care of it.
You could subclass argparse.FileType:
import argparse
import warnings
class ForgivingFileType(argparse.FileType):
def __call__(self, string):
try:
super(ForgivingFileType,self).__call__(string)
except IOError as err:
warnings.warn(err)
parser = argparse.ArgumentParser(description='my illustrative example')
parser.add_argument('--content', metavar='file',
default='content.txt', type=ForgivingFileType('r'),
help='file to process (defaults to content.txt)')
args = parser.parse_args()
This works without having to touch private methods like ArgumentParser._parse_known_args.
Looking at the argparse code, I see:
ArgumentParser.parse_args calls parse_known_args and makes sure that there isn't any pending argument to be parsed.
ArgumentParser.parse_known_args sets default values and calls ArgumentParser._parse_known_args
Hence, the workaround would be to use ArgumentParser._parse_known_args directly to detect -h and, after that, use ArgumentParser.parse_args as usual.
import sys, argparse
parser = argparse.ArgumentParser(description='my illustrative example', argument_default=argparse.SUPPRESS)
parser.add_argument('--content', metavar='file',
default='content.txt', type=argparse.FileType('r'),
help='file to process (defaults to content.txt)')
parser._parse_known_args(sys.argv[1:], argparse.Namespace())
args = parser.parse_args()
Note that ArgumentParser._parse_known_args needs a couple of parameters: the arguments from the command line and the namespace.
Of course, I wouldn't recommend this approach since it takes advantage of the internal argparse implementation and that might change in the future. However, I don't find it too messy, so you still might want to use it if you think maintenance risks pay off.
Use stdin as default:
parser.add_argument('file', default='-', nargs='?', type=argparse.FileType('r'))
Perhaps you could define your own type or action in the add_argument call that checks if the file exists, and returns a file handle if it does and None (or something else) otherwise.
This would require you to write some code of yourself as well though, but if the default value can not always be used you probably have to do some checking sooner or later. Like Manny D argues you might want to reconsider your default value.