Example
def main(p):
if foo_a(p):
return False
return p**2
def foo_a(p):
return p % 11 == 0
Now you can get 100% test coverage by
import unittest
from script import main
class Foobar(unittest.TestCase):
def test_main(self):
self.assertEquals(main(3), 9)
But maybe one wanted foo_a to be p % 2 == 0 instead.
The question
Branch coverage would shed a light on it, but I would also like to know if a function was never called "directly" by a test (such as main is in the example), but only indirectly (such as foo_a in the example).
Is this possible with pytest?
First of all just general line of thought is to unittest foo_a as well
import unittest
from script import main, foo_a
class Foobar(unittest.TestCase):
def test_main(self):
self.assertEquals(main(3), 9)
def test_foo_a(self):
self.assertEquals(foo_a(11), True)
You are probably looking for https://coverage.readthedocs.io/en/coverage-4.5.1/ which can be used with pytest https://pypi.org/project/pytest-cov/, this tool can show you exactly which lines of code had been called during testing
But I think there is another way to check your problem it is called mutation testing, here are some libraries that could help you with it
https://github.com/sixty-north/cosmic-ray
https://github.com/mutpy/mutpy
And also look into property based testing libraries like https://github.com/HypothesisWorks/hypothesis/tree/master/hypothesis-python
Related
I would like to programmatically add or mark a function as a test-case in pytest, so instead of writing
def test_my_function():
pass
I would like to do something like (pseudo-api, I know neither pytest.add_test nor pytest.testcase exist by that identifier).
def a_function_specification():
pass
pytest.add_test(a_function_specification)
or
I would like to do something like
#pytest.testcase
def a_function_specification():
pass
Basically I would like to write some test-case-generating decorator that isn't exactly working like pytest.mark/parametrizing which is why I started to dig into the internals but I haven't found an obvious way how this can be done for python code.
The YAML example in the pytest docs seem to use pytest.Item but I have a hard time mapping this to something that would work within python and not as part of a non-Python-file test collection.
Starting from version 2.6, pytest support:
nose-style __test__ attribute on modules, classes and functions, including unittest-style Classes. If set to False, the test will not be collected.
So, you need to add this attribute.
One approach is:
def not_a_test1():
assert 1 + 2 == 3
not_a_test1.__test__ = True
Another is:
def make_test(func):
func.__test__ = True
return func
#make_test
def not_a_test2():
assert 1 + 2 == 3
I was wondering how can I unit test if a recursive function has been called correctly. For example this function:
def test01(number):
if(len(number) == 1):
return 1
else:
return 1+test01(number[1:])
It counts recursvely how many digits a number has (assuming the number type is string)
So, I want to test if the function test01 has been called recursively. It would be ok if it is implemented just like that, but not if it is implemented as:
def test01(number):
return len(number)
EDIT:
The recursive approach is mandatory for educational purposes, so the UnitTest process will automate programming exercises checking. Is there a way to check if the function was called more than once? If that is possible, I can have 2 tests, one asserting the correct output and one to check if the function was called more than once for the same input.
Thank you in advance for your help
Guessing by the tags I assume you want to use unittest to test for the recursive call. Here is an example for such a check:
from unittest import TestCase
import my_module
class RecursionTest(TestCase):
def setUp(self):
self.counter = 0 # counts the number of calls
def checked_fct(self, fct): # wrapper function that increases a counter on each call
def wrapped(*args, **kwargs):
self.counter += 1
return fct(*args, **kwargs)
return wrapped
def test_recursion(self):
# replace your function with the checked version
with mock.patch('my_module.test01',
self.checked_fct(my_module.test01)): # assuming test01 lives in my_module.py
result = my_module.test01('444') # call the function
self.assertEqual(result, 3) # check for the correct result
self.assertGreater(self.counter, 1) # ensure the function has been called more than once
Note: I used import my_module instead of from my_module import test01 so that the first call is also mocked - otherwise the number of calls would be one too low.
Depending on how your setup looks like, you may add further tests manually, or auto-generate the test code for each test, or use parametrization with pytest, or do something else to automate the tests.
Normally a unit test should check at least that your function works and try to test all code paths in it
Your unit test should therefore try to take the main path several times, and then find the exit path, attaining full coverage
You can use the 3rd-party coverage module to see if all your code paths are being taken
pip install coverage
python -m coverage erase # coverage is additive, so clear out old runs
python -m coverage run -m unittest discover tests/unit_tests
python -m coverage report -m # report, showing missed lines
Curtis Schlak taught me this strategy recently.
It utilizes Abstract Syntax Trees and the inspect module.
All my best,
Shawn
import unittest
import ast
import inspect
from so import test01
class Test(unittest.TestCase):
# Check to see if function calls itself recursively
def test_has_recursive_call(self):
# Boolean switch
has_recursive_call = False
# converts function into a string
src = inspect.getsource(test01)
# splits the source code into tokens
# based on the grammar
# transformed into an Abstract Syntax Tree
tree = ast.parse(src)
# walk tree
for node in ast.walk(tree):
# check for function call
# and if the func called was "test01"
if (
type(node) is ast.Call
and node.func.id == "test01"
):
# flip Boolean switch to true
has_recursive_call = True
# assert: has_recursive_call should be true
self.assertTrue(
has_recursive_call,
msg="The function does not "
"make a recursive call",
)
print("\nThe function makes a recursive call")
if __name__ == "__main__":
unittest.main()
I'm trying to figure out the best structure to write unittests for a file parser. My parser.py file looks like this:
import re
import sys
import glob
from datetime import datetime
import csv
def main():
# does something
def normalizeGender(gender):
# does something
def normalizeDate(date):
# does something
def parseLine(record, delimiter):
# does something
def formatRecords(record_list):
# does something
etc...
I have tests dir, and inside a parser_test.py file that looks like this:
import unittest
class ParserTest(unittest.TestCase):
# Returns True or False.
def test(self):
self.assertTrue(True)
if __name__ == '__main__':
unittest.main()
Now, if I want to test all the methods from my parser, should import them all at once? Something tells me they should be wrapped as a module of some sort.
The following answer is based on my tool preferences, and the practices that I usually follow:
I would use pytest to implement the tests
I would implement at least 1 test for every function: test_normalizeGender, test_normalizeDate, test_ normalizeDate, test_parseLine, test_formatRecords. However, in case you have if statement with multiple branching inside your code, make sure you try to cover the possible cases of those branches. In addition, if you have for loops, I will implement a test for no elements, one element, and multiple elements.
I will put all the test in the same file since they are related.
Keep in mind that when it comes to implementing unit tests, you are testing a unit, so you don't need to verify the functionality of all your parser in the same test. In addition, you should mock/patch objects, methods, or functions to facilitate testing.
I hope this helps.
I'm struggling with django mock; I have even simplified an unit test but the test is still failing. I want to verify that a method is called (even with any parameter), but the "assert_called_once_with" always returns False.
Currently I'm trying:
#patch('utils.make_reset_password')
def test_shouldHaveCalledMakeResetToken(self, mocked):
user = User.get(...)
make_reset_password(user)
mocked.assert_called_once_with(user)
Even this simple example is failing with:
AssertionError: Expected 'make_reset_password' to be called once. Called 0 times
How this is possible? What am I doing wrong?
Thanks in advance
You have to use full path to utils, e.g. #patch('my_app.utils.make_reset_password') and then in the test call a function that calls make_reset_password.
#patch('my_app.utils.make_reset_password')
def test_shouldHaveCalledMakeResetToken(self, mock_make_reset_password):
user = User.get(...)
function_under_test(user)
mock_make_reset_password.assert_called_once_with(user)
EDIT
The other thing that comes to my mind is you are not mocking the correct function. If make_reset_password is imported from utils in another module then you need to change the path in the #patch decorator.
For example
# my_module.py
from my_app.utils import make_reset_password
def run_make_reset_password(user):
make_reset_password(user)
# tests.py
#patch('my_app.my_module.make_reset_password')
def test_shouldHaveCalledMakeResetToken(self, mock_make_reset_password):
user = User.get(...)
run_make_reset_password(user)
mock_make_reset_password.assert_called_once_with(user)
I have an application that imports a module from PyPI.
I want to write unittests for that application's source code, but I do not want to use the module from PyPI in those tests.
I want to mock it entirely (the testing machine will not contain that PyPI module, so any import will fail).
Currently, each time I try to load the class I want to test in the unittests, I immediately get an import error. so I thought about maybe using
try:
except ImportError:
and catch that import error, then use command_module.run().
This seems pretty risky/ugly and I was wondering if there's another way.
Another idea was writing an adapter to wrap that PyPI module, but I'm still working on that.
If you know any way I can mock an entire python package, I would appreciate it very much.
Thanks.
If you want to dig into the Python import system, I highly recommend David Beazley's talk.
As for your specific question, here is an example that tests a module when its dependency is missing.
bar.py - the module you want to test when my_bogus_module is missing
from my_bogus_module import foo
def bar(x):
return foo(x) + 1
mock_bogus.py - a file in with your tests that will load a mock module
from mock import Mock
import sys
import types
module_name = 'my_bogus_module'
bogus_module = types.ModuleType(module_name)
sys.modules[module_name] = bogus_module
bogus_module.foo = Mock(name=module_name+'.foo')
test_bar.py - tests bar.py when my_bogus_module is not available
import unittest
from mock_bogus import bogus_module # must import before bar module
from bar import bar
class TestBar(unittest.TestCase):
def test_bar(self):
bogus_module.foo.return_value = 99
x = bar(42)
self.assertEqual(100, x)
You should probably make that a little safer by checking that my_bogus_module isn't actually available when you run your test. You could also look at the pydoc.locate() method that will try to import something, and return None if it fails. It seems to be a public method, but it isn't really documented.
While #Don Kirkby's answer is correct, you might want to look at the bigger picture. I borrowed the example from the accepted answer:
import pypilib
def bar(x):
return pypilib.foo(x) + 1
Since pypilib is only available in production, it is not suprising that you have some trouble when you try to unit test bar. The function requires the external library to run, therefore it has to be tested with this library. What you need is an integration test.
That said, you might want to force unit testing, and that's generally a good idea because it will improve the confidence you (and others) have in the quality of your code. To widen the unit test area, you have to inject dependencies. Nothing prevents you (in Python!) from passing a module as a parameter (the type is types.ModuleType):
try:
import pypilib # production
except ImportError:
pypilib = object() # testing
def bar(x, external_lib = pypilib):
return external_lib.foo(x) + 1
Now, you can unit test the function:
import unittest
from unittest.mock import Mock
class Test(unittest.TestCase):
def test_bar(self):
external_lib = Mock(foo = lambda x: 3*x)
self.assertEqual(10, bar(3, external_lib))
if __name__ == "__main__":
unittest.main()
You might disapprove the design. The try/except part is a bit cumbersome, especially if you use the pypilib module in several modules of your application. And you have to add a parameter to each function that relies on the external library.
However, the idea to inject a dependency to the external library is useful, because you can control the input and test the output of your class methods, even if the external library is not within your control. Especially if the imported module is stateful, the state might be difficult to reproduce in a unit test. In this case, passing the module as a parameter may be a solution.
But the usual way to deal with this situation is called dependency inversion principle (the D of SOLID): you should define the (abstract) boundaries of your application, ie what you need from the outside world. Here, this is bar and other functions, preferably grouped in one or many classes:
import pypilib
import other_pypilib
class MyUtil:
"""
All I need from outside world
"""
#staticmethod
def bar(x):
return pypilib.foo(x) + 1
#staticmethod
def baz(x, y):
return other_pypilib.foo(x, y) * 10.0
...
# not every method has to be static
Each time you need one of these functions, just inject an instance of the class in your code:
class Application:
def __init__(self, util: MyUtil):
self._util = util
def something(self, x, y):
return self._util.baz(self._util.bar(x), y)
The MyUtil class must be as slim as possible, but must remain abstract from the underlying library. It is a tradeoff. Obviously, Application can be unit tested (just inject a Mock instead of an instance of MyUtil) while, under some circumstances (like a PyPi library not available during tests, a module that runs inside a framework only, etc.), MyUtil can be only tested within an integration test. If you need to unit test the boundaries of your application, you can use #Don Kirkby's method.
Note that the second benefit, after unit testing, is that if you change the libraries you are using (deprecation, license issue, cost, ...), you just have to rewrite the MyUtil class, using some other libraries or coding it from scratch. Your application is protected from the wild outside world.
Clean Code by Robert C. Martin has a full chapter on the boundaries.
Summary Before using #Don Kirkby's method or any other method, be sure to define the boundaries of your application irrespective of the specific libraries you are using. This, of course, does not apply to the Python standard library...
For a more explicit and granular approach:
import unittest
from unittest.mock import MagicMock, patch
try:
import bogus_module
except ModuleNotFoundError:
bogus_module = MagicMock()
#patch.dict('sys.modules', bogus_module=bogus_module)
class PlatformTests(unittest.TestCase):
...
Using the patch.dict decorator gives you granular control: it only applies to the class / method it is applied to.