I want to use pytest to check if the argparse.ArgumentTypeError exception is raised for an incorrect argument:
import argparse
import os
import pytest
def main(argsIn):
def configFile_validation(configFile):
if not os.path.exists(configFile):
msg = 'Configuration file "{}" not found!'.format(configFile)
raise argparse.ArgumentTypeError(msg)
return configFile
parser = argparse.ArgumentParser()
parser.add_argument('-c', '--configFile', help='Path to configuration file', dest='configFile', required=True, type=configFile_validation)
args = parser.parse_args(argsIn)
def test_non_existing_config_file():
with pytest.raises(argparse.ArgumentTypeError):
main(['--configFile', 'non_existing_config_file.json'])
However, running pytest says During handling of the above exception, another exception occurred: and consequently the test fails. What am I doing wrong?
The problem is that if argument's type converter raises exception ArgumentTypeError agrparse exits with error code 2, and exiting means raising builtin exception SystemExit. So you have to catch that exception and verify that the original exception is of a proper type:
def test_non_existing_config_file():
try:
main(['--configFile', 'non_existing_config_file.json'])
except SystemExit as e:
assert isinstance(e.__context__, argparse.ArgumentError)
else:
raise ValueError("Exception not raised")
Here's the ArgumentTypeError test in the test_argparse.py file (found in the development repository)
ErrorRaisingAgumentParser is a subclass defined at the start of the file, which redefines the parser.error method, so it doesn't exit, and puts the error message on stderr. That part's a bit complicated.
Because of the redirection I described the comment, it can't directly test for ArgumentTypeError. Instead it has to test for its message.
# =======================
# ArgumentTypeError tests
# =======================
class TestArgumentTypeError(TestCase):
def test_argument_type_error(self):
def spam(string):
raise argparse.ArgumentTypeError('spam!')
parser = ErrorRaisingArgumentParser(prog='PROG', add_help=False)
parser.add_argument('x', type=spam)
with self.assertRaises(ArgumentParserError) as cm:
parser.parse_args(['XXX'])
self.assertEqual('usage: PROG x\nPROG: error: argument x: spam!\n',
cm.exception.stderr)
Using pytest you can do the following in order to check that argparse.ArugmentError is raised. Additionally, you can check the error message.
with pytest.raises(SystemExit) as e:
main(['--configFile', 'non_existing_config_file.json'])
assert isinstance(e.value.__context__, argparse.ArgumentError)
assert 'expected err msg' in e.value.__context__.message
Inspired by #Giorgos's answer, here is a small context manager that makes the message extraction a bit more re-usable. I'm defining the following in a common place:
import argparse
import pytest
from typing import Generator, Optional
class ArgparseErrorWrapper:
def __init__(self):
self._error: Optional[argparse.ArgumentError] = None
#property
def error(self):
assert self._error is not None
return self._error
#error.setter
def error(self, value: object):
assert isinstance(value, argparse.ArgumentError)
self._error = value
#contextmanager
def argparse_error() -> Generator[ArgparseErrorWrapper, None, None]:
wrapper = ArgparseErrorWrapper()
with pytest.raises(SystemExit) as e:
yield wrapper
wrapper.error = e.value.__context__
This allows to test for parser errors concisely:
def test_something():
with argparse_error() as e:
# some parse_args call here
...
assert "Expected error message" == str(e.error)
Related
I'm using the Python (version 3.9.4) library argparse to parse a small number of option flags. For a number of reasons, I'd like to handle errors in my code rather than argparse.ArgumentParser. Namely, how an unrecognized argument is handled. For example, if I ran my-python-program --foobar, I'd like to be able to catch an exception and perform work there. This testcase disables almost all of the errors I've tried, except for an invalid argument:
import argparse
import sys
try:
parser = argparse.ArgumentParser(add_help=False, exit_on_error=False, usage=None)
parser.add_argument("--help", action="store_true", default=False)
parser.add_argument("--hello", default="Hello, world!")
args = parser.parse_args()
print(args.help, args.hello)
except Exception as err:
print("a problem occurred!", file=sys.stderr)
print(f"error: {err}", file=sys.stderr)
Instead, running my-python-program --foobar gives me:
usage: my-python-program [--help] [--hello HELLO]
my-python-program: error: unrecognized arguments: --foobar
If you look at the Python argparse source code, you can see that it calls self.error on an error. This function (at the bottom of the file), by default, prints the error message and quits. You can override this method in a subclass to raise an error instead.
import argparse
import sys
class MyArgumentParser(argparse.ArgumentParser):
"""An argument parser that raises an error, instead of quits"""
def error(self, message):
raise ValueError(message)
try:
parser = MyArgumentParser(add_help=False, exit_on_error=False, usage=None)
parser.add_argument("--help", action="store_true", default=False)
parser.add_argument("--hello", default="Hello, world!")
args = parser.parse_args()
print(args.help, args.hello)
except Exception as err:
print("a problem occurred!", file=sys.stderr)
print(f"error: {err}", file=sys.stderr)
Output:
$ python3 test.py --foobar
a problem occurred!
error: unrecognized arguments: --foobar
When trying to Unittest validations of arguments in argparse the following works:
mymodule:
def validate_mac_addr(mac_addr):
regex = re.compile(r'^((([a-f0-9]{2}:){5})|(([a-f0-9]{2}-){5}))[a-f0-9]{2}$', re.IGNORECASE)
if re.match(regex, mac_addr) is not None:
return mac_addr
msg = f"[-] Invalid MAC address: '{mac_addr}'"
raise argparse.ArgumentTypeError(msg)
test:
import mymodule
import unittest
def test_mac_address_false(self):
self.assertRaises(Exception, mymodule.validate_mac_addr,"n0:ma:ca:dd:re:ss:here")
But I wanted to catch a the more specific 'ArgumentTypeError' but this is apparently not possible with arssertRaises() in this example!? What is going on with the general usage of Exception in assertRaises()?
BTW
isinstance(argparse.ArgumentTypeError, Exception)
Returns False?!
Ref.: class ArgumentTypeError(Exception):
argparse.ArgumentTypeError is a subclass, not an instance, of Exception, and is the type of exception you should be asserting gets raised.
import argparse
def test_mac_address_false(self):
self.assertRaises(argparse.ArgumentTypeError, mymodule.validate_mac_addr, "n0:ma:ca:dd:re:ss:here")
What I have done :
I have a function def get_holidays(): which raises a Timeout error. My test function test_get_holidays_raises_ioerror(): first sets requests.get.side_effect = IOError and then uses pytest.raises(IOError) to assert if that function raises an IOError.
What the issue is :
Ideally this should fail, since my actual get_holidays() does not raise an IOError. But the test passes.
Possible reason :
This might be because Timeout is inherited from the IOError class.
What I want :
Want to assert specifically if IOError is raised.
Code :
from mock import Mock
import requests
from requests import Timeout
import pytest
requests = Mock()
# Actual function to test
def get_holidays():
try:
r = requests.get('http://localhost/api/holidays')
if r.status_code == 200:
return r.json()
except Timeout:
raise Timeout
return None
# Actual function that tests the above function
def test_get_holidays_raises_ioerror():
requests.get.side_effect = IOError
with pytest.raises(IOError):
get_holidays()
pytest captures the exception in an ExceptionInfo object. You can compare the exact type after the exception.
def test_get_holidays_raises_ioerror():
requests.get.side_effect = IOError
with pytest.raises(IOError) as excinfo:
get_holidays()
assert type(excinfo.value) is IOError
So, I have the most trivial in the world example. This is my class to be tested:
# My_Class.py
class My_Class(object):
#staticmethod
def doit(name, params):
try:
raise Exception("This is my error message")
except Exception:
print("Exception: I raised Exception")
And this is the tester itself:
# test.py
import unittest
from My_Class import My_Class
class Test_MyClass(unittest.TestCase):
def setUp(self):
self.my_class = My_Class()
def test_my_class(self):
name = "Abrakadabra"
params = {}
self.assertRaises(Exception, self.my_class.doit, name, params)
And this is what I see in the console, when I'm running my test.py:
$ nosetests test.py
F
======================================================================
FAIL: test_my_class (test.Test_MyClass)
----------------------------------------------------------------------
Traceback (most recent call last):
File ....
nose.proxy.AssertionError: Exception not raised by doit
-------------------- >> begin captured stdout << ---------------------
Exception: I raised Exception
--------------------- >> end captured stdout << ----------------------
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1)
It is reaaly iteresting, because it is controversial. On the one hand the test says that "Exception not raised by doit", but one line below it clearly prints out a message from the Exception block. So, what I'm doing wrong here??? Thanks!
To directly answer your question, the reason why you are getting that message is because with this assertion:
self.assertRaises(Exception, self.my_class.doit, name, params)
You are testing to make sure an exception was raised. But your try/except suppresses this. If you actually remove your try/except your test will in fact pass, because now your method will raise.
Since you do not want to do this, what you should be doing instead is testing the behaviour of your method when an exception is raised. Ultimately, you want to make sure that your print method is called in your except. I have put together an example below to help understand this.
Keeping in mind what #user2357112 mentioned, which is very important to keep in mind when unittesting, here is an example to help expand on that to provide a practical use for what you are trying to do:
Let us just put together some method:
def some_method():
pass
We will now put this in to your staticmethod you defined as such:
# My_Class.py
class My_Class(object):
#staticmethod
def doit(name, params):
try:
some_method()
except Exception:
print("Exception: I raised Exception")
So now, when it comes to your unittesting, you want to test the behaviour of your method doit. With that in mind, what you will do in this case, is test that some_method will raise an exception and you will validate how your doit method behaves to that exception being raised.
At this point, I suggest taking a look at the documentation behind unittest and mock to get more familiar with what you can do with your testing, but here is an example using mock patching to test the behaviour of your code if an exception is being raised:
#patch('builtins.print')
#patch('__main__.some_method')
def test_my_class(self, m_some_method, m_print):
name = "Abrakadabra"
params = {}
# have the side_effect raise the exception when some_method is called in doit
m_some_method.side_effect = Exception()
self.my_class.doit(name, params)
# check to make sure you caught the exception by checking print was called
self.assertEqual(m_print.call_count, 1)
When you put it all together, the following is functional code that I ran on my end that you can play around with to understand what is happening:
def some_method():
pass
# My_Class.py
class My_Class(object):
#staticmethod
def doit(name, params):
try:
some_method()
except Exception:
print("Exception: I raised Exception")
# test.py
import unittest
from mock import patch
class Test_MyClass(unittest.TestCase):
def setUp(self):
self.my_class = My_Class()
#patch('builtins.print')
#patch('__main__.some_method')
def test_my_class(self, m_some_method, m_print):
name = "Abrakadabra"
params = {}
m_some_method.side_effect = Exception()
self.my_class.doit(name, params)
self.assertEqual(m_print.call_count, 1)
if __name__ == '__main__':
unittest.main()
assertRaises is an assertion about the function's visible behavior, not its internals. It asserts that the stated exception passes out of the function. Any exceptions that are handled inside the function are not assertRaises's concern.
assertRaises failed since there was actually no exception raised. Well, it was raised but handled with except inside the doit() method. The problem is here:
try:
raise Exception("This is my error message")
except Exception:
print("Exception: I raised Exception")
You are raising an exception and then catching it without re-raising. From a caller (assertRaises is the caller in your case) perspective, no errors were thrown during the function call. Re-raising an exception allows a caller to handle an exception as well. Put a raise after the print:
try:
raise Exception("This is my error message")
except Exception:
print("Exception: I raised Exception")
raise # re-raising
Also see Handling Exceptions.
Apart from tinkering with the argparse source, is there any way to control the exit status code should there be a problem when parse_args() is called, for example, a missing required switch?
I'm not aware of any mechanism to specify an exit code on a per-argument basis. You can catch the SystemExit exception raised on .parse_args() but I'm not sure how you would then ascertain what specifically caused the error.
EDIT: For anyone coming to this looking for a practical solution, the following is the situation:
ArgumentError() is raised appropriately when arg parsing fails. It is passed the argument instance and a message
ArgumentError() does not store the argument as an instance attribute, despite being passed (which would be convenient)
It is possible to re-raise the ArgumentError exception by subclassing ArgumentParser, overriding .error() and getting hold of the exception from sys.exc_info()
All that means the following code - whilst ugly - allows us to catch the ArgumentError exception, get hold of the offending argument and error message, and do as we see fit:
import argparse
import sys
class ArgumentParser(argparse.ArgumentParser):
def _get_action_from_name(self, name):
"""Given a name, get the Action instance registered with this parser.
If only it were made available in the ArgumentError object. It is
passed as it's first arg...
"""
container = self._actions
if name is None:
return None
for action in container:
if '/'.join(action.option_strings) == name:
return action
elif action.metavar == name:
return action
elif action.dest == name:
return action
def error(self, message):
exc = sys.exc_info()[1]
if exc:
exc.argument = self._get_action_from_name(exc.argument_name)
raise exc
super(ArgumentParser, self).error(message)
## usage:
parser = ArgumentParser()
parser.add_argument('--foo', type=int)
try:
parser.parse_args(['--foo=d'])
except argparse.ArgumentError, exc:
print exc.message, '\n', exc.argument
Not tested in any useful way. The usual don't-blame-me-if-it-breaks indemnity applies.
All the answers nicely explain the details of argparse implementation.
Indeed, as proposed in PEP (and pointed by Rob Cowie) one should inherit ArgumentParser and override the behavior of error or exit methods.
In my case I just wanted to replace usage print with full help print in case of the error:
class ArgumentParser(argparse.ArgumentParser):
def error(self, message):
self.print_help(sys.stderr)
self.exit(2, '%s: error: %s\n' % (self.prog, message))
In case of override main code will continue to contain the minimalistic..
# Parse arguments.
args = parser.parse_args()
# On error this will print help and cause exit with explanation message.
Perhaps catching the SystemExit exception would be a simple workaround:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('foo')
try:
args = parser.parse_args()
except SystemExit:
print("do something else")
Works for me, even in an interactive session.
Edit: Looks like #Rob Cowie beat me to the switch. Like he said, this doesn't have very much diagnostic potential, unless you want get silly and try to glean info from the traceback.
As of Python 3.9, this is no longer so painful. You can now handle this via the new argparse.ArgumentParser exit_on_error instantiation argument. Here is an example (slightly modified from the python docs: argparse#exit_on_error):
parser = argparse.ArgumentParser(exit_on_error=False)
parser.add_argument('--integers', type=int)
try:
parser.parse_args('--integers a'.split())
except argparse.ArgumentError:
print('Catching an argumentError')
exit(-1)
You'd have to tinker. Look at argparse.ArgumentParser.error, which is what gets called internally. Or you could make the arguments non-mandatory, then check and exit outside argparse.
You can use one of the exiting methods: http://docs.python.org/library/argparse.html#exiting-methods. It should already handle situations where the arguments are invalid, however (assuming you have defined your arguments properly).
Using invalid arguments:
% [ $(./test_argparse.py> /dev/null 2>&1) ] || { echo error }
error # exited with status code 2
I needed a simple method to catch an argparse error at application start and pass the error to a wxPython form. Combining the best answers from above resulted in the following small solution:
import argparse
# sub class ArgumentParser to catch an error message and prevent application closing
class MyArgumentParser(argparse.ArgumentParser):
def __init__(self, *args, **kwargs):
super(MyArgumentParser, self).__init__(*args, **kwargs)
self.error_message = ''
def error(self, message):
self.error_message = message
def parse_args(self, *args, **kwargs):
# catch SystemExit exception to prevent closing the application
result = None
try:
result = super().parse_args(*args, **kwargs)
except SystemExit:
pass
return result
# testing -------
my_parser = MyArgumentParser()
my_parser.add_argument('arg1')
my_parser.parse_args()
# check for an error
if my_parser.error_message:
print(my_parser.error_message)
running it:
>python test.py
the following arguments are required: arg1
While argparse.error is a method and not a class its not possible to "try", "except" all "unrecognized arguments" errors. If you want to do so you need to override the error function from argparse:
def print_help(errmsg):
print(errmsg.split(' ')[0])
parser.error = print_help
args = parser.parse_args()
on an invalid input it will now print:
unrecognised