I'm a Python instructor and I wanted to give my students a task: write a function that computes the average of a list using a for loop and the range object.
I wanted to run a test on their function to see whether it actually uses the range object. How can I do that?
It should be something like this:
def avg(L):
Pass
def test_range(avg):
...
If avg contains range, then test_range should return True.
I tried solutions that utilize func_code, but apparantly range doesn't have that.
You can use Python's unittest.mock module to wrap around the range function from the builtins module, and then let your test assert that the wrapped range was indeed called.
For example, using Python's unittest framework for writing the test:
import builtins
import unittest
from unittest.mock import patch
# I don't know what L is supposed to be, and I know that there
# are better ways to compute average of a list, but the code
# for calculating the average is not important for this question.
def avg(L):
total = 0
for index in range(len(L)):
total += L[index]
return total / len(L)
class TestAverage(unittest.TestCase):
def test_avg(self):
with patch("builtins.range", wraps=builtins.range) as wrapped_patch:
expected = 47
actual = avg([1,49,91])
self.assertEqual(expected, actual)
wrapped_patch.assert_called()
if __name__ == '__main__':
unittest.main()
$ python -m unittest -v main.py
test_avg (main.TestAverage) ... ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
It uses unittest.mock's patch targetting the builtins.range function. Normally, patch is used in tests to replace the behavior and/or return value of the target, but in this case, you can pass wraps=builtins.range (which gets passed to the underlying Mock object), which means "I just want to spy on the call, but not modify its behavior":
wraps: Item for the mock object to wrap. If wraps is not None then calling the Mock will pass the call through to the wrapped object (returning the real result).
By wrapping it in a Mock object, you can use any of Mock's assert functions to check calls to range, like assert_called which checks whether the target was called at least once. You can be more specific by asserting the number of times range was called:
self.assertTrue(wrapped_patch.call_count == 1)
The assertion would fail if it wasn't called at all:
# Here, `range` wasn't used at all.
def avg(L):
return sum(L) / len(L)
class TestAverage(unittest.TestCase):
# same as the code above
$ python -m unittest -v main.py
test_avg (main.TestAverage) ... FAIL
======================================================================
FAIL: test_avg (main.TestAverage)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/path/to/main.py", line 15, in test_avg
wrapped_patch.assert_called()
File "/usr/local/Cellar/python#3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/mock.py", line 888, in assert_called
raise AssertionError(msg)
AssertionError: Expected 'range' to have been called.
The most important thing to note when using patch, is to know exactly where to patch. In this case, you can check the docs or use __module__ to know range's module:
>>> range
<class 'range'>
>>> range.__module__
'builtins'
But the test is a bit naive, because it can still pass even though avg didn't really use range:
def avg(L):
range(len(L)) # Called but really unused. Sneaky!
return sum(L) / len(L)
class TestAverage(unittest.TestCase):
# same as the code above
$ python -m unittest -v main.py
test_avg (main.TestAverage) ... ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
A slightly confusing workaround would be a test that "breaks" range such that, if the function was really using range, then it wouldn't work anymore:
def avg(L):
range(len(L)) # Called but really unused. Sneaky!
return sum(L) / len(L)
class TestAverage(unittest.TestCase):
def test_avg(self):
# same as above
def test_avg_is_really_using_range(self):
L = [10,20,90]
# Is it returning the correct result?
self.assertEqual(avg(L), 40)
# OK, but did it really use `range`?
# Let's try breaking `range` so it always yields 0,
# so we expect the return value to be *different*
with patch("builtins.range", return_value=[0,0,0]):
self.assertNotEqual(avg(L), 40)
So, if avg was sneakily calling but not really using range, then test_avg_is_really_using_range would now fail because avg still yields the correct value even with a broken range:
$ python -m unittest -v main.py
test_avg (main.TestAverage) ... ok
test_avg_really_using_range (main.TestAverage) ... FAIL
======================================================================
FAIL: test_avg_really_using_range (main.TestAverage)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/path/to/main.py", line 19, in test_avg_really_using_range
self.assertNotEqual(avg(L), 40)
AssertionError: 40.0 == 40
Lastly, as a side note, I'm using assertEqual here in all the example because the test for the return value is not the focus, but do read up on proper ways to assert possible float values, ex. How to perform unittest for floating point outputs? - python
Related
I was wondering how can I unit test if a recursive function has been called correctly. For example this function:
def test01(number):
if(len(number) == 1):
return 1
else:
return 1+test01(number[1:])
It counts recursvely how many digits a number has (assuming the number type is string)
So, I want to test if the function test01 has been called recursively. It would be ok if it is implemented just like that, but not if it is implemented as:
def test01(number):
return len(number)
EDIT:
The recursive approach is mandatory for educational purposes, so the UnitTest process will automate programming exercises checking. Is there a way to check if the function was called more than once? If that is possible, I can have 2 tests, one asserting the correct output and one to check if the function was called more than once for the same input.
Thank you in advance for your help
Guessing by the tags I assume you want to use unittest to test for the recursive call. Here is an example for such a check:
from unittest import TestCase
import my_module
class RecursionTest(TestCase):
def setUp(self):
self.counter = 0 # counts the number of calls
def checked_fct(self, fct): # wrapper function that increases a counter on each call
def wrapped(*args, **kwargs):
self.counter += 1
return fct(*args, **kwargs)
return wrapped
def test_recursion(self):
# replace your function with the checked version
with mock.patch('my_module.test01',
self.checked_fct(my_module.test01)): # assuming test01 lives in my_module.py
result = my_module.test01('444') # call the function
self.assertEqual(result, 3) # check for the correct result
self.assertGreater(self.counter, 1) # ensure the function has been called more than once
Note: I used import my_module instead of from my_module import test01 so that the first call is also mocked - otherwise the number of calls would be one too low.
Depending on how your setup looks like, you may add further tests manually, or auto-generate the test code for each test, or use parametrization with pytest, or do something else to automate the tests.
Normally a unit test should check at least that your function works and try to test all code paths in it
Your unit test should therefore try to take the main path several times, and then find the exit path, attaining full coverage
You can use the 3rd-party coverage module to see if all your code paths are being taken
pip install coverage
python -m coverage erase # coverage is additive, so clear out old runs
python -m coverage run -m unittest discover tests/unit_tests
python -m coverage report -m # report, showing missed lines
Curtis Schlak taught me this strategy recently.
It utilizes Abstract Syntax Trees and the inspect module.
All my best,
Shawn
import unittest
import ast
import inspect
from so import test01
class Test(unittest.TestCase):
# Check to see if function calls itself recursively
def test_has_recursive_call(self):
# Boolean switch
has_recursive_call = False
# converts function into a string
src = inspect.getsource(test01)
# splits the source code into tokens
# based on the grammar
# transformed into an Abstract Syntax Tree
tree = ast.parse(src)
# walk tree
for node in ast.walk(tree):
# check for function call
# and if the func called was "test01"
if (
type(node) is ast.Call
and node.func.id == "test01"
):
# flip Boolean switch to true
has_recursive_call = True
# assert: has_recursive_call should be true
self.assertTrue(
has_recursive_call,
msg="The function does not "
"make a recursive call",
)
print("\nThe function makes a recursive call")
if __name__ == "__main__":
unittest.main()
I'm trying to set up a grading script for an intro CS class using unittest. Essentially, students submit a python file student.py, which has some number of functions in it that are generally interdependent (meaning that func3() may use func1() in it's computations).
I'm writing unit tests for each method by comparing the output of student.func1 to the output of correct.func1, a method that's known to be the correct implementation (from a file correct.py).
As an example, say that func2 uses func1 in it's computation.
So, either by default or upon student.func1 failing some test, I want to overwrite student.func1 with correct.func1, so student.func2 uses a known correct implementation (and therefore isn't just wrong by default). How could I go about doing that? It seems like the setUp() and tearDown() are similar to what I want, but I don't know how to "unimport" modules in python, and haven't found any resources about it so far.
I'm interested in both the case where student.py contains classes, and func1,func2 are methods of a specific class, and when func1 and func2 are just defined generically within student.py.
The easiest way would be to import student into your module, then catch an AssertionError if a test fails, and replace the bad code inside the student module with your own good code:
import student
import unittest
def safe_f1():
print("Safe f1")
return 1
class TestSomething(unittest.TestCase):
def test_f1(self):
try:
self.assertEqual(student.func1(), 1)
except AssertionError:
student.func1 = safe_f1
raise
def test_f2(self):
self.assertEqual(student.func2(), 2)
Here's a dummy student.py that fails/works:
def func1():
print("Bad f1")
return 2
def func2():
return func1() + 1
return 2
When I run this, I get:
$ python -m unittest test.py
Bad f1
FSafe f1
.
======================================================================
FAIL: test_f1 (test.TestSomething)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/austin/Code/so/test.py", line 13, in test_f1
self.assertEqual(student.func1(), 1)
AssertionError: 2 != 1
----------------------------------------------------------------------
Ran 2 tests in 0.001s
FAILED (failures=1)
Here's the __repr__ method inside a class called Grid
def __repr__(self):
return 'Grid(%r, %r)' % (self.rows, self.cols)
and I've put some basic tests inside a unittest module to check if eval can do an equality test without failure, and that looks like this:
# One of the tests inside a unittest.TestCase subclass
def test_grid(self):
grid = Grid(3, 4)
self.assertEqual(eval(repr(grid)), grid)
Now, this is the test report (I've isolated this test from others):
======================================================================
FAIL: test_grid (tests.test_core.TestCore)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/user/Desktop/sample/tests/test_core.py", line 14, in test_grid
self.assertEqual(eval(repr(grid)), grid)
AssertionError: Grid(3, 4) != Grid(3, 4)
----------------------------------------------------------------------
Ran 1 test in 0.003s
FAILED (failures=1)
The Assertion exception message is even more confusing to me. Isn't Grid(3, 4) != Grid(3, 4) supposed to be False?
I think the core of the issue is that you are creating a new object, and even though the values are the same inside - python cannot tell that they are, so it compares object by references. And they are different. I believe you need to override python's magic comparison operator to be able to compare by internal values.
The docs say:
If you would rather define test functions directly at module level you can also use the following functions to implement fixtures:
def setup_function(function):
""" setup any state tied to the execution of the given function.
Invoked for every test function in the module.
"""
def teardown_function(function):
""" teardown any state that was previously setup with a setup_function
call.
"""
but actually it's unclear how you're supposed to use them.
I tried putting them in my test file exactly as shown above, but they don't get called. Looking closer at them, with that function arg they look like decorators. So I tried to find a way to do:
#pytest.setup_function
def my_setup():
# not called
I couldn't find anywhere to import one from as a decorator though, so that can't be right.
I found this thread on grokbase where someone explains you can do:
#pytest.fixture(autouse=True)
def setup_function(request):
# this one gets called
It works, but it doesn't seem related to the docs any more... there's no reason to call it setup_function here.
You just implement the body of setup_function(), that function is called for each function whose name starts with test_ and the function is handed in as a parameter:
def setup_function(fun):
print ("in setup function: " + fun.__name__)
def test_test():
assert False
This will give as output, when run with py.test:
============================= test session starts ==============================
platform linux2 -- Python 2.7.6 -- pytest-2.3.5
collected 1 items
test/test_test.py F
=================================== FAILURES ===================================
__________________________________ test_test ___________________________________
def test_test():
> assert False
E assert False
test/test_test.py:7: AssertionError
------------------------------- Captured stdout --------------------------------
in setup function: test_test
=========================== 1 failed in 0.01 seconds ===========================
The line before the last one shows the output from the actual call to setup_function()
A slightly more useful example, actually doing something that influences the test function:
def setup_function(function):
function.answer = 17
def teardown_function(function):
del function.answer
def test_modlevel():
assert modlevel[0] == 42
assert test_modlevel.answer == 17
This was taken from py.test's own tests, always a good (and hopefully complete) set of examples of all the features of py.test.
I'm relatively new to Python and unit testing in Python. From the Java world I know the concept of mocking but it seem to be much different from what I can see in Python.
I found this guide, which I found very helpful: http://www.voidspace.org.uk/python/mock/index.html
But as I wrote my (a bit more complex) tests with mocked out dependencies I noticed a strage behavior.
I decided to create a reduced, simple example which also does not work as I expect it.
Take a look at this, the result and my expectation I have added as comments:
import unittest
from mock import patch, Mock, MagicMock
class BasicTest(unittest.TestCase):
#patch("StringIO.StringIO")
def testSomethingNotWorkingAsExpected(self, StringIOMock):
StringIOMock.assert_called_once() # asserts, but why?
#patch("StringIO.StringIO")
def testSomethingSomehowWorking(self, StringIOMock):
# self.instantiateStringIO() # intentionally commented out
assert StringIOMock.called # does not assert (leading to failure of this test); as expected. If the above line is not commented, this asserts as expected.
def instantiateStringIO(self):
import StringIO
StringIO.StringIO()
Why is assert_called_once() asserting the instantiation of StringIO even it has not been instantiated yet?
And why does assert ClassMock.called bring the expected results?
Using assert not ... to assert a method has not been called I found here: Assert a function/method was not called using Mock. I inverted this pattern in my case by omitting the not.
Somewhere I found the pattern ClassMock.return_value to reference an instance. But I understand this as a way to manupulate the instance of a Mock before it will be called, not as a way to access the instance that might an underliing code have internally created. Or am I wrong?
My environment:
Python 2.7.3
mock 0.8.8
Fedora 19
Probably my understanding of the mock/patch thing is wrong. Could please someone aditionally explain what a class mock does and how it works?
Edit1: Added output
... and added paraphrase in parens to comment in testSomethingSomehowWorking
This is the output:
.F
======================================================================
FAIL: testSomethingSomehowWorking (test_test.BasicTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/mock.py", line 1224, in patched
return func(*args, **keywargs)
File "test_test.py", line 15, in testSomethingSomehowWorking
assert StringIOMock.called # does not assert; as expected
AssertionError
----------------------------------------------------------------------
Ran 2 tests in 0.001s
FAILED (failures=1)
The method assert_called_once does not exist and it does not perform an assertion. It's no different from writing StringIOMock.assert_foo_bar_does_not_exist() or any other method. The mock library doesn't check whether the method called on the mock actually exists.
If you use assert_called_once_with then it fails as expected.
You can use the spec parameter to raise an error when you call a non-existent method:
#patch("StringIO.StringIO", spec=StringIO.StringIO)
def testSomethingNotWorkingAsExpected(self, StringIOMock):
StringIOMock.assert_called_once() # will fail as the method doesn't exist