I have a small CLI app (myscript.py) that is defined like so.
import sys
import argparse
class MyParser(argparse.ArgumentParser):
'''
Overriden to show help on default.
'''
def error(self, message):
print(f'error: {message}')
self.print_help()
sys.exit(2)
def myfunc(args):
'''
Some function.
'''
print(args.input_int**2)
def main():
# Define Main Parser
main_par = MyParser(
prog='myapp',
description='main help')
# Define Command Parser
cmd_par = main_par.add_subparsers(
dest='command',
required=True)
# Add Subcommand Parser
subcmd_par = cmd_par.add_parser(
'subcmd',
description='subcmd help')
# Add Subcommand Argument
subcmd_par.add_argument(
'-i', '--input-int',
type=int,
help='some integer',
required=True)
# Add FromName Dispatcher
subcmd_par.set_defaults(
func=myfunc)
# Parse Arguments
args = main_par.parse_args()
# Call Method
args.func(args)
if __name__ == '__main__':
main()
The MyParser class simply overrides the error() method in argparse.ArgumentParser class to print help on error.
When I execute
$ python myscript.py
I see the default / main help. Expected.
When I execute
$ python myscript.py subcmd
I see the subcmd help. Expected.
When I execute
$ python myscript.py subcmd -i ClearlyWrongValue
I also see the subcmd help. Expected.
However, very annoyingly if I do the following
$ python myscript.py subcmd -i 2 --non-existent-argument WhateverValue
I see the default / main help and not subcmd help.
What can I do, to ensure that this last case shows me the subcmd help and not the main help? I thought the subparser structure would automatically procure the help from subcmd as found in the third case, but it is not so? Why?
The unrecognized args error is raised by parse_args
def parse_args(self, args=None, namespace=None):
args, argv = self.parse_known_args(args, namespace)
if argv:
msg = _('unrecognized arguments: %s')
self.error(msg % ' '.join(argv))
return args
The subparser is called via the cmd_par.__call__ with:
subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
for key, value in vars(subnamespace).items():
setattr(namespace, key, value)
if arg_strings:
vars(namespace).setdefault(_UNRECOGNIZED_ARGS_ATTR, [])
getattr(namespace, _UNRECOGNIZED_ARGS_ATTR).extend(arg_strings)
That is it is called with parse_known_args, and it's extras are returned to the main as UNRECOGNIZED. So it's the main than handles these, not the subparser.
In the $ python myscript.py subcmd -i ClearlyWrongValue case, the subparser raises a ArgumentError which is caught and converted into a self.error call.
Similarly, the newish exit_on_error parameter handles this kind of ArgumentError, but does not handle the urecognized error. There was some discussion of this in the bug/issues.
If you used parse_known_args, the extras would be ['--non-existent-argument', 'WhateverValue'], without distinguishing which parser initially classified them as such.
I am able to pass command line arguments when running
python <filename>.py arg1
But when am trying to pass the command line arguments for running pytest it fails and gives error as below. Can you please advise.
pytest <filename>.py arg1
ERROR: file not found: arg1
EDIT:
For example am thinking of using it this way assuming I have passed an argument and am reading it via sys.argv:
import sys
arg = sys.argv[3]
def f():
return 3
def test_function():
assert f() == arg
Your pytest <filename>.py arg1 command is trying to call pytest on two modules <filename>.py and arg1 , But there is no module arg1.
If you want to pass some argument before running pytest then run the pytest from a python script after extracting your variable.
As others suggested though you would probably want to parameterize your tests in some other way, Try:Parameterized pytest.
# run.py
import pytest
import sys
def main():
# extract your arg here
print('Extracted arg is ==> %s' % sys.argv[2])
pytest.main([sys.argv[1]])
if __name__ == '__main__':
main()
call this using python run.py filename.py arg1
Here's the method I just cooked up from reading the parameterized pytest docs and hacking for a while... I don't know how stable or good it is going to be overall since I just got it working.
I did however check that HTML coverage generation works with this method.
add a file to your test directory for configuring the command-line args you want to pass:
tests\conftest.py
# this is just so we can pass --server and --port from the pytest command-line
def pytest_addoption(parser):
''' attaches optional cmd-line args to the pytest machinery '''
parser.addoption("--server", action="append", default=[], help="real server hostname/IP")
parser.addoption("--port", action="append", default=[], help="real server port number")
and then add a test file, with this special pytest_generate_tests function which is called when collecting a test function
tests\test_junk.py
def pytest_generate_tests(metafunc):
''' just to attach the cmd-line args to a test-class that needs them '''
server_from_cmd_line = metafunc.config.getoption("server")
port_from_cmd_line = metafunc.config.getoption("port")
print('command line passed for --server ({})'.format(server_from_cmd_line))
print('command line passed for --port ({})'.format(port_from_cmd_line))
# check if this function is in a test-class that needs the cmd-line args
if server_from_cmd_line and port_from_cmd_line and hasattr(metafunc.cls, 'real_server'):
# now set the cmd-line args to the test class
metafunc.cls.real_server = server_from_cmd_line[0]
metafunc.cls.real_port = int(port_from_cmd_line[0])
class TestServerCode(object):
''' test-class that might benefit from optional cmd-line args '''
real_server=None
real_port = None
def test_valid_string(self):
assert self.real_server!=None
assert self.real_port!=None
def test_other(self):
from mypackage import my_server_code
if self.real_server != None:
assert "couldn\'t find host" not in my_server_code.version(self.real_server, self.real_port)
then run (with HTML coverage, for example) with:
pytest tests\test_junk.py --server="abc" --port=123 --cov-report html --cov=mypackage
It seems monkeypatch also works.
Example:
import sys
def test_example(monkeypatch):
monkeypatch.setattr(sys, 'argv', ['/path/to/binary', 'opt1', '...'])
assert f() == '...'
def test_another():
# sys.argv is not modified here
assert f() != '...'
Going off of Greg Haskin's answer in this question, I tried to make a unittest to check that argparse is giving the appropriate error when I pass it some args that are not present in the choices. However, unittest generates a false positive using the try/except statement below.
In addition, when I make a test using just a with assertRaises statement, argparse forces the system exit and the program does not execute any more tests.
I would like to be able to have a test for this, but maybe it's redundant given that argparse exits upon error?
#!/usr/bin/env python3
import argparse
import unittest
class sweep_test_case(unittest.TestCase):
"""Tests that the merParse class works correctly"""
def setUp(self):
self.parser=argparse.ArgumentParser()
self.parser.add_argument(
"-c", "--color",
type=str,
choices=["yellow", "blue"],
required=True)
def test_required_unknown_TE(self):
"""Try to perform sweep on something that isn't an option.
Should return an attribute error if it fails.
This test incorrectly shows that the test passed, even though that must
not be true."""
args = ["--color", "NADA"]
try:
self.assertRaises(argparse.ArgumentError, self.parser.parse_args(args))
except SystemExit:
print("should give a false positive pass")
def test_required_unknown(self):
"""Try to perform sweep on something that isn't an option.
Should return an attribute error if it fails.
This test incorrectly shows that the test passed, even though that must
not be true."""
args = ["--color", "NADA"]
with self.assertRaises(argparse.ArgumentError):
self.parser.parse_args(args)
if __name__ == '__main__':
unittest.main()
Errors:
Usage: temp.py [-h] -c {yellow,blue}
temp.py: error: argument -c/--color: invalid choice: 'NADA' (choose from 'yellow', 'blue')
E
usage: temp.py [-h] -c {yellow,blue}
temp.py: error: argument -c/--color: invalid choice: 'NADA' (choose from 'yellow', 'blue')
should give a false positive pass
.
======================================================================
ERROR: test_required_unknown (__main__.sweep_test_case)
Try to perform sweep on something that isn't an option.
----------------------------------------------------------------------
Traceback (most recent call last): #(I deleted some lines)
File "/Users/darrin/anaconda/lib/python3.5/argparse.py", line 2310, in _check_value
raise ArgumentError(action, msg % args)
argparse.ArgumentError: argument -c/--color: invalid choice: 'NADA' (choose from 'yellow', 'blue')
During handling of the above exception, another exception occurred:
Traceback (most recent call last): #(I deleted some lines)
File "/anaconda/lib/python3.5/argparse.py", line 2372, in exit
_sys.exit(status)
SystemExit: 2
The trick here is to catch SystemExit instead of ArgumentError. Here's your test rewritten to catch SystemExit:
#!/usr/bin/env python3
import argparse
import unittest
class SweepTestCase(unittest.TestCase):
"""Tests that the merParse class works correctly"""
def setUp(self):
self.parser=argparse.ArgumentParser()
self.parser.add_argument(
"-c", "--color",
type=str,
choices=["yellow", "blue"],
required=True)
def test_required_unknown(self):
""" Try to perform sweep on something that isn't an option. """
args = ["--color", "NADA"]
with self.assertRaises(SystemExit):
self.parser.parse_args(args)
if __name__ == '__main__':
unittest.main()
That now runs correctly, and the test passes:
$ python scratch.py
usage: scratch.py [-h] -c {yellow,blue}
scratch.py: error: argument -c/--color: invalid choice: 'NADA' (choose from 'yellow', 'blue')
.
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
However, you can see that the usage message is getting printed, so your test output is a bit messed up. It might also be nice to check that the usage message contains "invalid choice".
You can do that by patching sys.stderr:
#!/usr/bin/env python3
import argparse
import unittest
from io import StringIO
from unittest.mock import patch
class SweepTestCase(unittest.TestCase):
"""Tests that the merParse class works correctly"""
def setUp(self):
self.parser=argparse.ArgumentParser()
self.parser.add_argument(
"-c", "--color",
type=str,
choices=["yellow", "blue"],
required=True)
#patch('sys.stderr', new_callable=StringIO)
def test_required_unknown(self, mock_stderr):
""" Try to perform sweep on something that isn't an option. """
args = ["--color", "NADA"]
with self.assertRaises(SystemExit):
self.parser.parse_args(args)
self.assertRegexpMatches(mock_stderr.getvalue(), r"invalid choice")
if __name__ == '__main__':
unittest.main()
Now you only see the regular test report:
$ python scratch.py
.
----------------------------------------------------------------------
Ran 1 test in 0.002s
OK
For pytest users, here's the equivalent that doesn't check the message.
import argparse
import pytest
def test_required_unknown():
""" Try to perform sweep on something that isn't an option. """
parser=argparse.ArgumentParser()
parser.add_argument(
"-c", "--color",
type=str,
choices=["yellow", "blue"],
required=True)
args = ["--color", "NADA"]
with pytest.raises(SystemExit):
parser.parse_args(args)
Pytest captures stdout/stderr by default, so it doesn't pollute the test report.
$ pytest scratch.py
================================== test session starts ===================================
platform linux -- Python 3.6.7, pytest-3.5.0, py-1.7.0, pluggy-0.6.0
rootdir: /home/don/.PyCharm2018.3/config/scratches, inifile:
collected 1 item
scratch.py . [100%]
================================ 1 passed in 0.01 seconds ================================
You can also check the stdout/stderr contents with pytest:
import argparse
import pytest
def test_required_unknown(capsys):
""" Try to perform sweep on something that isn't an option. """
parser=argparse.ArgumentParser()
parser.add_argument(
"-c", "--color",
type=str,
choices=["yellow", "blue"],
required=True)
args = ["--color", "NADA"]
with pytest.raises(SystemExit):
parser.parse_args(args)
stderr = capsys.readouterr().err
assert 'invalid choice' in stderr
As usual, I find pytest easier to use, but you can make it work in either one.
While the parser may raise an ArgumentError during parsing a specific argument, that is normally trapped, and passed to parser.error and parse.exit. The result is that the usage is printed, along with an error message, and then sys.exit(2).
So asssertRaises is not a good way of testing for this kind of error in argparse. The unittest file for the module, test/test_argparse.py has an elaborate way of getting around this, the involves subclassing the ArgumentParser, redefining its error method, and redirecting output.
parser.parse_known_args (which is called by parse_args) ends with:
try:
namespace, args = self._parse_known_args(args, namespace)
if hasattr(namespace, _UNRECOGNIZED_ARGS_ATTR):
args.extend(getattr(namespace, _UNRECOGNIZED_ARGS_ATTR))
delattr(namespace, _UNRECOGNIZED_ARGS_ATTR)
return namespace, args
except ArgumentError:
err = _sys.exc_info()[1]
self.error(str(err))
=================
How about this test (I've borrowed several ideas from test_argparse.py:
import argparse
import unittest
class ErrorRaisingArgumentParser(argparse.ArgumentParser):
def error(self, message):
#print(message)
raise ValueError(message) # reraise an error
class sweep_test_case(unittest.TestCase):
"""Tests that the Parse class works correctly"""
def setUp(self):
self.parser=ErrorRaisingArgumentParser()
self.parser.add_argument(
"-c", "--color",
type=str,
choices=["yellow", "blue"],
required=True)
def test_required_unknown(self):
"""Try to perform sweep on something that isn't an option.
Should pass"""
args = ["--color", "NADA"]
with self.assertRaises(ValueError) as cm:
self.parser.parse_args(args)
print('msg:',cm.exception)
self.assertIn('invalid choice', str(cm.exception))
if __name__ == '__main__':
unittest.main()
with a run:
1931:~/mypy$ python3 stack39028204.py
msg: argument -c/--color: invalid choice: 'NADA' (choose from 'yellow', 'blue')
.
----------------------------------------------------------------------
Ran 1 test in 0.002s
OK
With many of the great answers above, I see that in the setUp method a parser instance is created inside our test and an argument is added to it, effectively causing the test to be of argparse's implementation. This, of course, could be a valid test/use case but wouldn't necessarily test a script's or application's specific use of argparse.
I think Yauhen Yakimovich's answer gives good insight into how to make use of argparse in a pragmatic way. While I haven't embraced it fully, I thought a simplified test method is possible via a parser generator and an override.
I've opted for testing my code rather than argparse's implementation. To achieve this we'll want to utilize a factory to create the parser in our code that holds all the argument definitions. This facilitates testing our own parser in setUp.
// my_class.py
import argparse
class MyClass:
def __init__(self):
self.parser = self._create_args_parser()
def _create_args_parser():
parser = argparse.ArgumentParser()
parser.add_argument('--kind',
action='store',
dest='kind',
choices=['type1', 'type2'],
help='kind can be any of: type1, type2')
return parser
In our test, we can generate our parser and test against it. We will override the error method to ensure we don't get trapped in argparse's ArgumentError evaluation.
import unittest
from my_class import MyClass
class MyClassTest(unittest.TestCase):
def _redefine_parser_error_method(self, message):
raise ValueError
def setUp(self):
parser = MyClass._create_args_parser()
parser.error = self._redefine_parser_error_func
self.parser = parser
def test_override_certificate_kind_arguments(self):
args = ['--kind', 'not-supported']
expected_message = "argument --kind: invalid choice: 'not-supported'.*$"
with self.assertRaisesRegex(ValueError, expected_message):
self.parser.parse_args(args)
This might not be the absolute best answer but I find it nice to use our own parser's arguments and test that part by simply testing against an exception we know should only happen in the test itself.
If you look in the error log, you can see that a argparse.ArgumentError is raised and not an AttributeError. your code should look like this:
#!/usr/bin/env python3
import argparse
import unittest
from argparse import ArgumentError
class sweep_test_case(unittest.TestCase):
"""Tests that the merParse class works correctly"""
def setUp(self):
self.parser=argparse.ArgumentParser()
self.parser.add_argument(
"-c", "--color",
type=str,
choices=["yellow", "blue"],
required=True)
def test_required_unknown_TE(self):
"""Try to perform sweep on something that isn't an option.
Should return an attribute error if it fails.
This test incorrectly shows that the test passed, even though that must
not be true."""
args = ["--color", "NADA"]
try:
self.assertRaises(ArgumentError, self.parser.parse_args(args))
except SystemExit:
print("should give a false positive pass")
def test_required_unknown(self):
"""Try to perform sweep on something that isn't an option.
Should return an attribute error if it fails.
This test incorrectly shows that the test passed, even though that must
not be true."""
args = ["--color", "NADA"]
with self.assertRaises(ArgumentError):
self.parser.parse_args(args)
if __name__ == '__main__':
unittest.main()
If you look into the source code of argparse, in argparse.py, around line 1732 (my python version is 3.5.1), there is a method of ArgumentParser called parse_known_args. The code is:
# parse the arguments and exit if there are any errors
try:
namespace, args = self._parse_known_args(args, namespace)
if hasattr(namespace, _UNRECOGNIZED_ARGS_ATTR):
args.extend(getattr(namespace, _UNRECOGNIZED_ARGS_ATTR))
delattr(namespace, _UNRECOGNIZED_ARGS_ATTR)
return namespace, args
except ArgumentError:
err = _sys.exc_info()[1]
self.error(str(err))
So, the ArgumentError will be swallowed by argparse, and exit with an error code. If you want to test this anyway, the only way I could think out is mocking sys.exc_info.
I know this is an old question but just to expand on #don-kirkby's answer of looking for SystemExit – but without having to use pytest or patching – you can wrap the testcode in contextlib.redirect_stderr, if you want to assert something about the error message:
import contextlib
from io import StringIO
import unittest
class MyTest(unittest.TestCase):
def test_foo(self):
ioerr = StringIO()
with contextlib.redirect_stderr(ioerr):
with self.assertRaises(SystemExit) as err:
foo('bad')
self.assertEqual(err.exception.code, 2)
self.assertIn("That is a 'bad' thing", ioerr.getvalue())
I had a similar problem with the same error of argparse (exit 2) and corrected it capturing the first element of tuple that parse_known_args() return, an argparse.Namespace object.
def test_basics_options_of_parser(self):
parser = w2ptdd.get_parser()
# unpacking tuple
parser_name_space,__ = parser.parse_known_args()
args = vars(parser_name_space)
self.assertFalse(args['unit'])
self.assertFalse(args['functional'])
I have a unittest that wants to call an imported module to do both parse_os based on the unittest's command-line option but it seems unittest does not recognize the option, any ideas:
./python testParser.py --mac
option --mac not recognized
Usage: testParser.py [options] [test] [...]
Options:
-h, --help Show this message
-v, --verbose Verbose output
-q, --quiet Minimal output
-f, --failfast Stop on first failure
-c, --catch Catch control-C and display results
-b, --buffer Buffer stdout and stderr during test runs
Examples:
testParser.py - run default set of tests
testParser.py MyTestSuite - run suite 'MyTestSuite'
testParser.py MyTestCase.testSomething - run MyTestCase.testSomething
testParser.py MyTestCase - run all 'test*' test methods
in MyTestCase
I want to run my unittest program like this: python testParser.py --mac
EDITTED: Works now by changing 'unittest.main()' to:
runner = unittest.TextTestRunner(stream=stderr_file)
itersuite = unittest.TestLoader().loadTestsFromTestCase(TT28046_ForensicSearchSmokeTest)
runner.run(itersuite)
Unittest program:
import logging
import unittest
from myargparse import *
class MyTest(unittest.TestCase):
def test_parse_os(self):
## Parse the args:
self.install = install_sw(parse_os(arg=""))
print 'Which os? %s' % self.install
if __name__ == '__main__':
logging.basicConfig(level=logging.INFO)
# get the default logger
logger = logging.getLogger()
# add a file handler
logger.addHandler(logging.FileHandler('stdout.txt', mode='w'))
# set up a stream for all stderr output
stderr_file = open('stderr.txt', 'w')
# attach that stream to the testRunner
unittest.main(testRunner=unittest.TextTestRunner(stream=stderr_file))
My imported module:
import argparse
import os
import sys
def parse_os(arg):
my_os = ''
parser = argparse.ArgumentParser()
parser.add_argument("-m", "--mac",
action="store_true")
parser.add_argument("-w", "--win",
action="store_true")
args = parser.parse_args()
if args.mac:
print 'Mac'
my_os = "Mac"
if args.win:
print 'Windows'
my_os = "Windows"
return my_os
def install_sw(my_os):
installed_os = None
if my_os == 'Mac':
print 'Installing Mac...'
installed_os = 'Mac'
if my_os == 'Windows':
print 'Installing Windows...'
installed_os = 'Windows'
return installed_os
The sys.argv variable is a simple list so you can modify/replace it at your wish.
I'd consider using a context manager in this case, on the lines of:
class SysArgv(object):
def __init__(self, argv):
self._old_argv = sys.argv
sys.argv = argv
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, tb):
sys.argv = self._old_argv
return False
And used as:
In [4]: with SysArgv(['a', 'b', 'c']):
...: print(sys.argv)
...:
['a', 'b', 'c']
In your case simple wrap the test code like:
with SysArgv(['the_module_name.py', '--mac']):
# test code goes here
and the argparse module will see the arguments you wants.
As for passing the arguments to the unittest module when running the tests, it's possible passing the argv argument to unittest.main. From the documentation:
The `argv` argument can be a list of options passed to the program, with the first element being the program name. If not specified or `None`, the values of `sys.argv` are used.
However in this case you should modify the sys.argv variable before calling unittest.main:
if __name__ == '__main__':
options = ['name_of_module.py'] + sys.argv[-1:] # last argument as option for the test
with SysArgv(sys.argv[:-1]): # or modify how you want
unittest.main(argv=options)
Have you tried using just '-m' instead of '--mac'?
You may also try:
import optparse
parser = optparse.OptionParser()
parser.add_option("-m", "--mac",
dest="mac",
action="store_true",
help="Run as Mac")
parser.add_option("-w", "--win",
dest="win",
action="store_true",
help="Run as Win")
(options, args) = parser.parse_args()
Thank you all for your suggestions but I decided to go with this to limit the changes to my program.
Instead of calling 'unittest.main()', I just changed to call the following:
runner = unittest.TextTestRunner(stream=stderr_file)
itersuite = unittest.TestLoader().loadTestsFromTestCase(MyTest)
runner.run(itersuite)
Based on all the answers here, I originally did this simple hack, and it worked:
# Change sys.argv before unittest tries to parse our args
sys.argv = [sys.argv[0]] # Replace with only the first arg
unittest.main()
The I realized I could still use all the unitest command line args, and in my case I was just passing a bunch of paths, so anything starting with a "-" could just be passed on and there is no need to hack anything since unittest.main() has an argv argument
# Pass on options, and more importantly, don't pass on ALL args
options = [sys.argv[0]] + [a for a in sys.argv if a.startswith("-")]
unittest.main(argv=options)
Just filter out all argparse and not arparse parameters and send then ones not being argparse arguments to sys.argv which is the one unittest uses:
args, notknownargs = parser.parse_known_args()
sys.argv[1:] = notknownargs
I've written a small test suite in Python's unittest:
class TestRepos(unittest.TestCase):
#classmethod
def setUpClass(cls):
"""Get repo lists from the svn server."""
...
def test_repo_list_not_empty(self):
"""Assert the the repo list is not empty"""
self.assertTrue(len(TestRepoLists.all_repos)>0)
def test_include_list_not_empty(self):
"""Assert the the include list is not empty"""
self.assertTrue(len(TestRepoLists.svn_dirs)>0)
...
if __name__ == '__main__':
unittest.main(testRunner=xmlrunner.XMLTestRunner(output='tests',
descriptions=True))
The output is formatted as Junit test using the xmlrunner pacakge.
I've added a command line argument for toggling JUnit output:
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Validate repo lists.')
parser.add_argument('--junit', action='store_true')
args=parser.parse_args()
print args
if (args.junit):
unittest.main(testRunner=xmlrunner.XMLTestRunner(output='tests',
descriptions=True))
else:
unittest.main(TestRepoLists)
The problem is that running the script without --junit works, but calling it with --junit clashes with unittest's arguments:
option --junit not recognized
Usage: test_lists_of_repos_to_branch.py [options] [test] [...]
Options:
-h, --help Show this message
-v, --verbose Verbose output
...
How can I run a unittest.TestCase without calling unittest.main()?
You really should use a proper test runner (such as nose or zope.testing). In your specific case, I'd use argparser.parse_known_args() instead:
if __name__ == '__main__':
parser = argparse.ArgumentParser(add_help=False)
parser.add_argument('--junit', action='store_true')
options, args = parser.parse_known_args()
testrunner = None
if (options.junit):
testrunner = xmlrunner.XMLTestRunner(output='tests', descriptions=True)
unittest.main(testRunner=testrunner, argv=sys.argv[:1] + args)
Note that I removed --help from your argument parser, so the --junit option becomes hidden, but it will no longer interfere with unittest.main. I also pass the remaining arguments on to unittest.main().