We Keep Coding

How to use same unit test for different implementations in python?

I am developing multiple functions that answer a same problem but using different algorithm.
So the same input for all functions should generate the same output, that's why I wnted to use the same unit tests instead of having to create multiple tests with the same logic.
I was using the Python unittest framework, and I wanted to use an abstract test class to have the generic tests defined with a function variable so that I could just instantiate that generic function with the one I want to test in another normal test class. But it seems I can't instantiate the function variable in the child class.
So here is an example abstract class with generic tests for multiple functions.
class AbstractTestCase():
def test_generic_input_one(self):
result = self.function("input 1")
self.assertFalse(result)
def test_generic_input_two(self):
result = self.function("input 2")
self.assertTrue(result)
And here you would have a specific test class for the function_a that inherits the generic tests from the AbstractTestCase class and that implements its own.
class TestsFunctionA(AbstractTestCase, unittest.TestCase):
def setUp(self):
self.function = function_a
def test_specific_input(self):
result = self.assertTrue(self.function("specific input"))
self.assertTrue(result)
I am pretty sure it can be done, but I can't seem to find an example to see how to implement it. I would like to avoid code duplication.
What should be the simplest and best way to do it ?

I have been looking for it and got a couple of example like:
Eli Bendersky's Python unit testing: parametrized test cases
But what helped me the most was vegard's answer about making a class factory which would take parameters and create the TestCase accordingly
The function takes the parameters of the parameterised test case and the actual TestCase class can refer to them without any problems.
Here is an example, take a foo.py file with:
import unittest
def make_test_case(x):
class MyTestCase(unittest.TestCase):
def test_foo(self):
self.assertEquals(x, 1)
return MyTestCase
class ConcreteTestCase(make_test_case(1)):
pass
Then run the test(s):
python -m unittest -v foo
Basically this is very flexible and adapted really well to my usecase.

Basically you need to parametrized you tests with function.
For unittest you can use ddt
#ddt
class ProblemTestCase(unittest.TestCase):
def test_specific_input(self):
self.assertTrue(function_a("specific input"))
#data(function_a, function_b)
def test_generic_input_one(self, function):
result = function("input 1")
self.assertFalse(result)
#data(function_a, function_b)
def test_generic_input_two(self, function):
result = function("input 2")
self.assertTrue(result)
Alternatively you can use just plain OOP:
class AbstractTestCase(object):
def test_generic_input_one(self):
result = self.function("input 1")
self.assertFalse(result)
def test_generic_input_two(self):
result = self.function("input 2")
self.assertTrue(result)
class TestsFunctionA(AbstractTestCase, unittest.TestCase):
def function(self, param):
return function_a(param)
def test_specific_input(self):
self.assertTrue(self.function("specific input"))
class TestsFunctionB(AbstractTestCase, unittest.TestCase):
def function(self, param):
return function_b(param)
def test_another_specific_input(self):
self.assertTrue(self.function("another specific input"))

I came here searching for a way to test multiple implementations of the same function. My use case is testing student's submissions of different search algorithms that are all fed the same test data and should return the same results.
Sylhare's answer was easy to adopt but it does not hurt to share so here is how:
import unittest
def function_a():
result = ...
return result
def function_b():
result = ...
return result
def make_test(function):
class TestImplementation(unittest.TestCase):
def test_foo(self):
self.assertEquals(function, 1)
return TestImplementation
class TestImplementationA(make_test(function_a)):
pass
class TestImplementationB(make_test(function_b)):
pass

Given a structure
├── README.md
├── requirements.txt
├── test
│ ├── __init__.py
│ └── two_sum
│ ├── __init__.py
│ ├── base_test_suite.py
│ ├── test_brute_force.py
│ └── test_two_pass_hash_table.py
└── two_sum
├── __init__.py
├── brute_force.py
└── two_pass_hash_table.py
And there are brute-force and two-pass hash-table solutions (functions called two_sum) in the corresponding files in two_sum module.
And base_test_suite.py
class TestTwoSum:
def __init__(self, unittest, two_sum_func):
self.two_sum = two_sum_func
self.unittest = unittest
def test_it_returns_indices_of_two_numbers_that_add_up_to_target(self):
# given
numbers = [2, 7, 11, 15]
target = 9
# when
result = self.two_sum(numbers, target)
# then
self.unittest.assertEqual(result, [0, 1])
And test_brute_force.py
from unittest import TestCase
from test.two_sum.base_test_suite import TestTwoSum
from two_sum.brute_force import two_sum
class Test(TestCase):
def test(self):
case = TestTwoSum(self, two_sum)
case.test_it_returns_indices_of_two_numbers_that_add_up_to_target()
And test_two_pass_hash_table.py
from unittest import TestCase
from test.two_sum.base_test_suite import TestTwoSum
from two_sum.two_pass_hash_table import two_sum
class Test(TestCase):
def test(self):
case = TestTwoSum(self, two_sum)
case.test_it_returns_indices_of_two_numbers_that_add_up_to_target()
Then one can run python -m unittest which would run the same unit test for different solutions of the two sum problem.

Function patch not being picked up by imported module

I feel like this should be an easy mock, but I have not gotten it to work yet.
I am working off of the following directory structure:
module
├── utilities.py
├── order.py
├── test
│ ├── test_order.py
The relevant code is as follows:
-- utilities.py --
def get_file_path(order_number, file_extension):
# this is what I want to mock out
-- order.py --
from module.utilities import get_file_path
class Order():
# ...
#classmethod
def load_order(order_number, extension):
file_path = get_file_path(order_number, extension)
-- test_order.py --
import unittest
from unittest.mock import patch
from module.order import order
#patch('order.get_file_path')
def mock_file(_, extension):
if extension == 'json':
return static_file_path
class TestOrder(unittest.TestCase):
def test_load_order_by_number(self):
my_order = order.load_order(number, extension)
This is the first time that I have tried mocking in Python. From what I can tell, what I have should work, but whenever an Order calls get_file_path, it always uses the one in utilities.py.
I have tried:
decorating test_load_order_by_number
patching with module.order.get_file_path
I tried looking on SO but none of the solutions that I found helped, so I thought that I was just doing something obviously wrong that someone can point out.

It does not look like creating the patch outside of the class was getting picked up. It started working when I pulled the patch in as a decorator for the specific test.
class TestOrder(unittest.TestCase):
#patch('utilities.order.get_file_path')
def test_load_order_by_number(self, file_mock):
def mock_get_file_path(*args, **kwargs):
if kwargs.get('extension', None) == 'json':
return static_file_path
return None
file_mock.side_effect = mock_get_file_path
my_order = order.load_order(number, extension)

python mock patch decorator behaves different for class methods and individual functions

Several times I've ran into a problem with unittest.mock.patch decorator. When I tried to mock individual functions from included module, patch didn't work. However if functions from included module are collected as class methods, patch works perfectly.
Partially this question intersects with mine. But there is no good answer for this issue as well.
Here is an example of what I'm trying to describe:
|-- __init__.py
|-- helpers.py
|-- main_module.py
|-- tests.py
I wrote one function in helpers.py as a class method, and another one as an individual function:
# helpers.py
class HelperClass():
def method_a(self):
return "a"
def function_a():
return "a"
I've included both of them in the main module:
# main_module.py
from helpers import HelperClass, function_a
def function_which_uses_helper_function():
a_val = function_a()
return a_val
def function_which_uses_helper_class_method():
a_val = HelperClass().method_a()
return a_val
And finally tests:
# tests.py
from unittest import TestCase
from unittest.mock import patch
from main_module import function_which_uses_helper_function, function_which_uses_helper_class_method
class TestClass(TestCase):
#patch('helpers.function_a')
def test_function_which_uses_helper_function(self, mock_function_a):
mock_function_a.return_value = "c"
self.assertEqual(function_which_uses_helper_function(), "c")
#patch('helpers.HelperClass.method_a')
def test_function_which_uses_helper_class_method(self, mock_method_a):
mock_method_a.return_value = "c"
self.assertEqual(function_which_uses_helper_class_method(), "c")
Which gives me these results:
$ py.test tests.py
<...>
tests.py .F
<...>
tests.py:11: in test_function_which_uses_helper_function
self.assertEqual(function_which_uses_helper_function(), "c")
E AssertionError: 'a' != 'c'
E - a
E + c
============ 1 failed, 1 passed in 0.14 seconds ============
I'll appreciate any help. Hope this also helps someone :)

After a while I finally understood why my example for function didn't work. The explanations is here. And this article was very useful as well. Considering all of that solution will be:
# main_module.py
import helpers # <- I had to change my import statement
def function_which_uses_helper_function():
a_val = helpers.function_a()
return a_val
def function_which_uses_helper_class_method():
a_val = helpers.HelperClass().method_a()
return a_val
Another solution is to change the way how I mock my function_a, which is:
from unittest import TestCase
from unittest.mock import patch
from main_module import function_which_uses_helper_function, function_which_uses_helper_class
class TestClass(TestCase):
#patch('main_module.function_a') # <-- !!! I need to mock function in the`main_module`, not in `helpers` !!!
def test_function_which_uses_helper_function(self, mock_function_a):
mock_function_a.return_value = "c"
self.assertEqual(function_which_uses_helper_function(), "c")
#patch('helpers.HelperClass.function_a')
def test_function_which_uses_helper_class(self, mock_function_a):
mock_function_a.return_value = "c"
self.assertEqual(function_which_uses_helper_class(), "c")
I'm sad that I've realized all of that just recently. Hope this helps somebody :)

Mock function called on import

I have a module I need to test that calls a function on import but I cannot call this function for various reasons. So I am mocking this function but even mocking it calls import.
For example I am testing mod1.py that looks like this:
import os
def bar():
return 'foo'
def dont_call():
os.listdir("C:\\tmp")
dont_call()
And my test looks something like this:
import mock
#mock.patch("mod1.dont_call")
def test_mod1(mock_dont_call):
import mod1
assert mod1.bar()=='foo'
if __name__=="__main__":
test_mod1()
The problem is os.listdir is called.
I cannot change mod1 so what can I do?
I am using python2.7.
To put this in context I am testing a module that opens a database connection on import which I do not agree with but I can see the reasoning behind it. Unfortunately I cannot access this database on my QA machine.

If you want code to 'not' be executed on import put them inside the following condition:
In mod1.py, do the following:
if __name__=="__main__":
dont_call()
This is because, by default when you import a python module, all the code in it gets executed. By adding the above condition, you are explicitly stating that dont_call() is to be called only when the file it run as a script and not when it is imported in other modules.

The workaround I found was to mock what dont_call was calling giving me something like this:
import mock
#mock.patch("os.listdir")
def test_mod1(mock_dont_call):
import mod1
assert mod1.bar()=='foo'
if __name__=="__main__":
test_mod1()

Check your dir
$tree.
test_shot/
├── mod1.py
├── __pycache__
│ └── mod1.cpython-310.pyc
└── test.py
Below code works fine for me.
mod1.py
import os
def bar():
return 'foo'
def dont_call():
os.listdir(".")
def call_this():
print('called this')
call_this()
dont_call()
test.py
import mock
#mock.patch("mod1.dont_call")
def test_mod1(mock_dont_call):
import mod1
assert mod1.bar()=='foo'
if __name__=="__main__":
test_mod1()
Here is output:
$cd test_shot
$python3 test.py
called this

How to call setup once for all tests and teardown after all are finished

I have a bunch of tests written using pytest. There are all under a directory dir. For example:
dir/test_base.py
dir/test_something.py
dir/test_something2.py
...
The simplified version of code in them is as follows:
test_base.py
import pytest
class TestBase:
def setup_module(module):
assert False
def teardown_module(module):
assert False
test_something.py
import pytest
from test_base import TestBase
class TestSomething(TestBase):
def test_dummy():
pass
test_something2.py
import pytest
from test_base import TestBase
class TestSomethingElse(TestBase):
def test_dummy2():
pass
All my test_something*.py files extend the base class in test_base.py. Now I wrote setup_module(module) and teardown_module(module) methods in test_base.py. I was expecting the setup_module to be called once for all tests, and teardown_module() to be called at the end, once all tests are finished.
But the functions don’t seem to be getting called? Do I need some decorators for this to work?

The OP's requirement was for setup and teardown each to execute only once, not one time per module. This can be accomplished with a combination of a conftest.py file, #pytest.fixture(scope="session") and passing the fixture name to each test function.
These are described in the Pytest fixtures documentation
Here's an example:
conftest.py
import pytest
#pytest.fixture(scope="session")
def my_setup(request):
print '\nDoing setup'
def fin():
print ("\nDoing teardown")
request.addfinalizer(fin)
test_something.py
def test_dummy(my_setup):
print '\ntest_dummy'
test_something2.py
def test_dummy2(my_setup):
print '\ntest_dummy2'
def test_dummy3(my_setup):
print '\ntest_dummy3'
The output when you run py.test -s:
collected 3 items
test_something.py
Doing setup
test_dummy
.
test_something2.py
test_dummy2
.
test_dummy3
.
Doing teardown
The name conftest.py matters: you can't give this file a different name and expect Pytest to find it as a source of fixtures.
Setting scope="session" is important. Otherwise setup and teardown will be repeated for each test module.
If you'd prefer not to pass the fixture name my_setup as an argument to each test function, you can place test functions inside a class and apply the pytest.mark.usefixtures decorator to the class.

Put setup_module and teardown_module outside of a class on module level. Then add your class with your tests.
def setup_module(module):
"""..."""
def teardown_module(module):
"""..."""
class TestSomething:
def test_dummy(self):
"""do some tests"""
For more info refer to this article.

setup_module/teardown_module are called for the module where the eventual (derived) tests are defined. This also allows to customize the setup. If you only ever have one setup_module you can put it to test_base.py and import it from the other places. HTH.

First of all it is a good practice to put all tests in a module called "tests":
<product>
...
tests/
__init__.py
test_something.py
Secondly I think you should use setup_class/teardown_class methods in your base class:
import unittest
class MyBase(unittest.TestCase):
#classmethod
def setup_class(cls):
...
#classmethod
def teardown_class(cls):
...
More info: http://pytest.org/latest/xunit_setup.html