This question already has answers here:
How do you test that a Python function throws an exception?
(19 answers)
Closed 4 years ago.
Can somebody tell me why the following unit-test is failing on the
ValueError in test_bad, rather than catching it with assertRaises
and succeeding? I think I'm using the correct procedure and syntax,
but the ValueError is not getting caught.
I'm using Python 2.7.5 on a linux box.
Here is the code …
import unittest
class IsOne(object):
def __init__(self):
pass
def is_one(self, i):
if (i != 1):
raise ValueError
class IsOne_test(unittest.TestCase):
def setUp(self):
self.isone = IsOne()
def test_good(self):
self.isone.is_one(1)
self.assertTrue(True)
def test_bad(self):
self.assertRaises(ValueError, self.isone.is_one(2))
if __name__ == "__main__":
unittest.main()
and here is the output of the unit-test:
======================================================================
ERROR: test_bad (__main__.IsOne_test)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test/raises.py", line 20, in test_bad
self.assertRaises(ValueError, self.isone.is_one(2))
File "test/raises.py", line 8, in is_one
raise ValueError
ValueError
----------------------------------------------------------------------
Ran 2 tests in 0.008s
FAILED (errors=1)
Unittest's assertRaises takes a callable and arguments, so in your case, you'd call it like:
self.assertRaises(ValueError, self.isone.is_one, 2)
If you prefer, as of Python2.7, you could also use it as a context manager like:
with self.assertRaises(ValueError):
self.isone.is_one(2)
Related
How to disable decorator during project unittests in Python?
When running the unittests, I would like to disable the selected decorator for the duration of the tests.
import unittest
import functools
def handle_value_error(func):
#functools.wraps(func)
def wrapper_handle_value_error():
try:
return func()
except ValueError as e:
print(f"During calling {func.__name__!r} an error was raised: {str(e)}")
return wrapper_handle_value_error
#handle_value_error
def func_that_raise_value_error():
# do something
raise ValueError
def disable_decorator_for(func, decorator):
# do something that disable handle_value_error decorator
pass
class TestFunc(unittest.TestCase):
def test_func_that_raise_value_error(self):
disable_decorator_for(func=func_thar_raise_value_error, decorator=handle_value_error)
with self.assertRaises(ValueError):
func_thar_raise_value_error()
When function disable_decorator_for() is called, the tests should pass, but the output is:
During calling 'func_thar_raise_value_error' an error was raised:
Failure
Traceback (most recent call last):
File "...\disable_decorator_for_unittests.py", line 29, in test_func_thar_raise_value_error
with self.assertRaises(ValueError):
AssertionError: ValueError not raised
Ran 1 test in 0.003s
FAILED (failures=1)
Process finished with exit code 1
I have a dummy funtion to try exceptions:
def fun(n):
try:
if n <1:
raise ValueError
return 1
except:
pass
In my unit test I use:
import unittest
class TestFibonnacci(unittest.TestCase):
def test_values(self):
self.assertEqual(fun(1),1)
self.assertRaises(ValueError, fun(-1))
However I'm unable to get an answer I actually get:
An exception occurred
E....
ERROR: test_values (main.TestFibonnacci)
Traceback (most recent call last):
The traceback here
TypeError: 'NoneType' object is not callable
Ran 1 tests in 0.001s
What I'm doing wrong?
You are calling fun(-1) immediately, rather than letting self.assertRaises call it, so that it can catch the exception.
You have to pass the function and its arguments separately.
self.assertRaises(ValueError, fun, -1)
Alternatively, you can use assertRaises as a context manager.
with self.assertRaises(ValueError):
fun(-1)
The with statement captures the exception raised by fun and provides it to the __exit__ method of the value returned by assertRaises.
I am still learning unittest and therefore, am unable to tell if there's a something missing in my test case in test_iterators.py below. Can someone help me to understand why the ValueError failed to be raised within unittest? Here are the scripts:
iterators.py
"""
Simple class to count from zero to N
"""
class count_to(object):
def __init__(self, nber):
self.nber = nber
def __iter__(self):
return count_to_iter(self.nber)
class count_to_iter(object):
def __init__(self, nber):
self.stopat = nber
self.current_nber = 0
def __next__(self):
if self.stopat < 0:
raise ValueError
elif self.current_nber > self.stopat:
raise StopIteration
self.current_nber += 1
return self.current_nber - 1
if __name__ == '__main__':
for x in count_to(-1):
print(x)
tests/test_iterators.py
import unittest
import iterators
class TestBaseIterators(unittest.TestCase):
def setUp(self):
pass
# Can't get the negative test right yet. It returns None instead of raising a ValueError
# Calling iterators.py directly and execute main successfully raised a ValueError however
def test_negative(self):
with self.assertRaises(ValueError): iterators.count_to(-1)
if __name__ == '__main__':
unittest.main()
I have used a similar approach to test raised errors previously and it worked. However, for this particular test case, here's what I get from the test.
test_negative (test_iterators.TestBaseIterators) ... FAIL
NoneType: None
======================================================================
FAIL: test_negative (test_iterators.TestBaseIterators)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/kerwei/Git/Concepts/tests/test_iterators.py", line 19, in test_negative
with self.assertRaises(ValueError): iterators.count_to(-1)
AssertionError: ValueError not raised
----------------------------------------------------------------------
Ran 1 test in 0.004s
FAILED (failures=1)
If I were to call iterators directly from __main__, I can then successfully receive the ValueError.
(py36) Kers-MacBook-Air:Concepts kerwei$ python iterators.py
Traceback (most recent call last):
File "iterators.py", line 29, in <module>
for x in count_to(-1):
File "iterators.py", line 19, in __next__
raise ValueError
ValueError
count_to(-1) creates a new count_to instance, it does not iterate over it, but you placed the test on self.stop_at value and raise the ValueError in the count_to_iter.__next__ method, so you will obviously not get a ValueError until you iterate on the count_to instance.
The naive fix would be to force iteration, ie:
def test_negative(self):
with self.assertRaises(ValueError):
# passing the iterable to `list` will force iteration
list(iterators.count_to(-1))
But the root problem is actually more of a design issue: raising a ValueError at this point is far from optimal since it will only happen when actually consuming the iterable, so you will have to inspect the call stack up until you find where count_to has been passed a wrong value. A much better solution is to check the value and eventually raise directly at the point where count_to is instanciated so it breaks always and immediatly (instead of "eventually, when you try to use the iterator in some possibly remote part of the code):
class count_to(object):
def __init__(self, nber):
if nber < 0:
raise ValueError("count_to argument must be a positive integer")
self.nber = nber
def __iter__(self):
return count_to_iter(self.nber)
And then your current test code will work as intended.
What is wrong with my code below?
I'm expecting assert call_func_once_with("b") to throw an error as call_func was passed 'a'. I confimed that the function was indeed called once and with argument 'a'.
from unittest.mock import Mock, patch
def call_func(x):
pass
#patch("__main__.call_func")
def test_call_func(call_func):
call_func("a")
assert call_func.called_once_with("b")
assert call_func.called == 1
print(call_func.call_args)
test_call_func()
Output:
call('a')
You're not the first person to notice strange things with these types of assertions (see Magic mock assert_called_once vs assert_called_once_with weird behaviour)
For what it's worth, I can only advise that you try to create a test class which inherits from unittest.TestCase and then use the assertEqual method to get more consistent test behaviour:
import unittest
from unittest.mock import patch, call
def call_func(x):
pass
class MyTests(unittest.TestCase):
#patch("__main__.call_func")
def test_call_func(self, call_func_mock):
call_func_mock("a")
# assert call_func_mock.called == 1
# assert call_func_mock.called_once_with("b")
self.assertEqual(call_func_mock.call_count, 1)
self.assertEqual(call_func_mock.call_args_list[0], call("b"))
print(call_func_mock.call_args)
unittest.main()
This gives the following (expected) results:
F
======================================================================
FAIL: test_call_func (__main__.MyTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Python36\lib\unittest\mock.py", line 1179, in patched
return func(*args, **keywargs)
File "C:/scratch.py", line 16, in test_call_func
self.assertEquals(call_func_mock.call_args_list[0], call("b"))
AssertionError: call('a') != call('b')
----------------------------------------------------------------------
Ran 1 test in 0.003s
FAILED (failures=1)
Process finished with exit code 1
I'm rewriting a afl-fuzz (a C application) to Python. Since I don't have enough understanding of its inner workings, I would like to replicate its functionality as close as possible.
I'm trying to run a functional test of a routine that forks Python interpreter, runs execve and if it fails, reports failure to its caller by returning 42. The test runs nice outside of unittest, but fails when put into it:
#!/usr/bin/env python
import os
import sys
import unittest
def run_test():
x = os.fork()
if not x:
sys.exit(42)
waitpid_result, status = os.waitpid(x, os.WUNTRACED)
print(os.WEXITSTATUS(status))
class ForkFunctionalTest(unittest.TestCase):
def test_exercise_fork(self):
run_test()
if __name__ == '__main__':
print('Expecting "42" as output:')
run_test()
print('\nAnd here goes unexpected SystemExit error:')
unittest.main()
Here's how it fails:
Expecting "42" as output:
42
And here goes unexpected SystemExit error:
E
======================================================================
ERROR: test_exercise_fork (__main__.ForkFunctionalTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "afl-fuzz2.py", line 23, in test_exercise_fork
run_test()
File "afl-fuzz2.py", line 15, in run_test
sys.exit(42)
SystemExit: 42
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (errors=1)
1
.
----------------------------------------------------------------------
Ran 1 test in 0.014s
OK
Is there a way to make unittest work with this function without changing run_test? I tried os._exit instead of sys.exit(), but it made the program die in both processes.
It turns out that os._exit actually worked, but in my unit tests I needed to mock it out given that I mocked out os.fork. Silly mistake.
sys.exit() raises a SystemExit class exception, which, if not caught, quits the program. You can try to catch the exception:
def text_exercise_fork(self):
try:
run_test()
except SystemExit as e:
print(e.args[0])