I have several classes that share some invariants and have a common interface, and I would like to run automatically the same test for each of them.
As an example, suppose I have several classes that implement different approaches for partitioning a data-set. The common invariant here would be, that for all of these classes the union over all partitions should equal the original data-set.
What I currently have looks something like this:
class PartitionerInvariantsTests(unittest.TestCase):
def setUp(self):
self.testDataSet = range(100) # create test-data-set
def impl(self, partitioner):
self.assertEqual(self.testDataSet,
chain.from_iterable(partitioner(self.testDataSet))
Then I add a different function that calls impl for each of the classes I want to test with an instance of that class. The problem with this becomes apparent when doing this for more than one test-function. Suppose I have 5 test-functions and 5 classes I want to test. This would make 25 functions that look almost identical for invoking all the tests.
Another approach I was thinking about was to implement the template as a super-class, and then create a sub-class for each of the classes I want to test. The sub-classes could provide a function for instantiating the class. The problem with that is that the default test-loader would consider the (unusable) base-class a valid test-case and try to run it, which would fail.
So, what are your suggestions?
P.S.: I am using Python 2.6
You could use multiple inheritance.
class PartitionerInvariantsFixture(object):
def setUp(self):
self.testDataSet = range(100) # create test-data-set
super(PartitionInvariantsFixture, self).setUp()
def test_partitioner(self):
TestCase.assertEqual(self.testDataSet,
chain.from_iterable(self.partitioner(self.testDataSet))
class MyClassTests(TestCase, PartitionerInvariantsFixture):
partitioner = Partitioner
Subclass PartitionerInvariantsTests:
class PartitionerInvariantsTests(unittest.TestCase):
def test_impl(self):
self.assertEqual(self.testDataSet,
chain.from_iterable(self.partitioner(self.testDataSet))
class PartitionerATests(PartitionerInvariantsTests):
for each Partitioner class you wish to test. Then test_impl would be run for each Partitioner class, by virtue of inheritance.
Following up on Nathon's comment, you can prevent the base class from being tested by having it inherit only from object:
import unittest
class Test(object):
def test_impl(self):
print('Hi')
class TestA(Test,unittest.TestCase):
pass
class TestB(Test,unittest.TestCase):
pass
if __name__ == '__main__':
unittest.sys.argv.insert(1,'--verbose')
unittest.main(argv = unittest.sys.argv)
Running test.py yields
test_impl (__main__.TestA) ... Hi
ok
test_impl (__main__.TestB) ... Hi
ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
Related
I'm trying to create mocks from scratch that can pass the test issubclass(class_mock, base_class) where the base class is an abstract class derived from abc.ABC. Before you ask the question, I will answer why I'm trying to do it.
I have an internal package containing a base class and a collection of sub-classes that properly implement the abstract interface. Besides, I have a factory class that can instantiate the sub-classes. The factory is built is such a way that it can inspect its own package and have access to the existing sub-classes. The factory is meant to be always in the same package as the derived and base class (constraint). I think you guessed that I'm actually testing the factory... However, since the sub-classes can change in number, their name or their package name, etc., I cannot implement a correct unit test that directly refers to the actual cub-classes (because it introduces a coupling) and I need mocks.
The problem is that I didn't succeed to create a mock that satisfies the above conditions for a class derived from an abstract class. What I was able to achieve is for a class derived from another non-abstract class.
Here is the code that illustrates the problem more concretely.
import unittest.mock
import inspect
import abc
class A:
pass
class B(A):
pass
class TestSubClass(unittest.TestCase):
def test_sub_class(self):
b_class_mock = self._create_class_mock("B", A)
print(isinstance(b_class_mock, type))
print(inspect.isclass(b_class_mock))
print(issubclass(b_class_mock, A))
#staticmethod
def _create_class_mock(mock_name, base_class):
class_mock = unittest.mock.MagicMock(spec=type(base_class), name=mock_name)
class_mock.__bases__ = (base_class,)
return class_mock
So, for this code, everything is ok. It prints 3 True as wanted.
But as long as the class A is defined as abstract (class A(abc.ABC)), the last test is failing with an error saying that the mock is not a class even if the 2 previous tests are saying the opposite.
I dived a bit into the implementation of abc.ABCMeta and found out that __subclasscheck__ is overridden. I tried to know the process behind it but when I reached the C code and everything became a way more complicated, I tried to rather track when the error message is generated. Unfortunately, I didn't succeed to understand why it is actually not working.
My automation framework uses pytest setup/teardown type of testing instead of fixtures. I also have several levels of classes:
BaseClass - highest, all tests inhriet from it
FeatureClass - medium, all tests related to the programs feature inherit from it
TestClass - hold the actual tests
edit, for examples sake, i change the DB calls to a simple print
I want to add DB report in all setup/teardowns. i.e. i want that the general BaseClass setup_method will create a DB entry for the test and teardown_method will alter the entry with the results. i have tried but i can't seem to get out of method the values of currently running test during run time. is it possible even? and if not, how could i do it otherwise?
samples:
(in base.py)
class Base(object):
test_number = 0
def setup_method(self, method):
Base.test_number += 1
self.logger.info(color.Blue("STARTING TEST"))
self.logger.info(color.Blue("Current Test: {}".format(method.__name__)))
self.logger.info(color.Blue("Test Number: {}".format(self.test_number)))
# --->here i'd like to do something with the actual test parameters<---
self.logger.info("print parameters here")
def teardown_method(self, method):
self.logger.info(color.Blue("Current Test: {}".format(method.__name__)))
self.logger.info(color.Blue("Test Number: {}".format(self.test_number)))
self.logger.info(color.Blue("END OF TEST"))
(in my_feature.py)
class MyFeature(base.Base):
def setup_method(self, method):
# enable this feature in program
return True
(in test_my_feature.py)
class TestClass(my_feature.MyFeature):
#pytest.mark.parametrize("fragment_length", [1,5,10])
def test_my_first_test(self):
# do stuff that is changed based on fragment_length
assert verify_stuff(fragment_length)
so how can i get the parameters in setup_method, of the basic parent class of the testing framework?
The brief answer: NO, you cannot do this. And YES, you can work around it.
A bit longer: these unittest-style setups & teardowns are done only for compatibility with the unittest-style tests. They do not support the pytest's fixture, which make pytest nice.
Due to this, neither pytest nor pytest's unittest plugin provide the context for these setup/teardown methods. If you would have a request, function or some other contextual objects, you could get the fixture's values dynamically via request.getfuncargvalue('my_fixture_name').
However, all you have is self/cls, and method as the test method object itself (i.e. not the pytest's node).
If you look inside of the _pytest/unittest.py plugin, you will find this code:
class TestCaseFunction(Function):
_excinfo = None
def setup(self):
self._testcase = self.parent.obj(self.name)
self._fix_unittest_skip_decorator()
self._obj = getattr(self._testcase, self.name)
if hasattr(self._testcase, 'setup_method'):
self._testcase.setup_method(self._obj)
if hasattr(self, "_request"):
self._request._fillfixtures()
First, note that the setup_method() is called fully isolated from the pytest's object (e.g. self as the test node).
Second, note that the fixtures are prepared after the setup_method() is called. So even if you could access them, they will not be ready.
So, generally, you cannot do this without some trickery.
For the trickery, you have to define a pytest hook/hookwrapper once, and remember the pytest node being executed:
conftest.py or any other plugin:
import pytest
#pytest.hookimpl(hookwrapper=True)
def pytest_runtest_protocol(item, nextitem):
item.cls._item = item
yield
test_me.py:
import pytest
class Base(object):
def setup_method(self, method):
length = self._item.callspec.getparam('fragment_length')
print(length)
class MyFeature(Base):
def setup_method(self, method):
super().setup_method(method)
class TestClass(MyFeature):
#pytest.mark.parametrize("fragment_length", [1,5,10])
def test_my_first_test(self, fragment_length):
# do stuff that is changed based on fragment_length
assert True # verify_stuff(fragment_length)
Also note that MyFeature.setup_method() must call the parent's super(...).setup_method() for obvious reasons.
The cls._item will be set on each callspec (i.e. each function call with each parameter). You can also put the item or the specific parameters into some other global state, if you wish.
Also be carefull not to save the field in the item.instance. The instance of the class will be created later, and you have to use setup_instance/teardown_instance method for that. Otherwise, the saved instance's field is not preserved and is not available as self._item in setup_method().
Here is the execution:
============ test session starts ============
......
collected 3 items
test_me.py::TestClass::test_my_first_test[1] 1
PASSED
test_me.py::TestClass::test_my_first_test[5] 5
PASSED
test_me.py::TestClass::test_my_first_test[10] 10
PASSED
============ 3 passed in 0.04 seconds ============
Suppose that I have an interface defined like this:
class MyInterface(abc.ABCMeta):
#abstractmethod
def my_method(self, *params):
pass
And several implementations of it.
class Implentation1(MyInterface):
pass
class Implentation2(MyInterface):
pass
All of these classes must be tested under the same class. I was wondering it there is any way to specify that you want a test case to be run with different classes. I am using nosetests and unittest.
I've written some tests using unittest as below and I want to reuse them in another class where I'm stuck and need help..
Code snippets are as below.
MyTestClass.py
Class MyTestClass(unittest.TestCase):
#classmethod
def test_TC01_set(self):
self.devAddr = "127.0.0.0"
self.teststoSkip = 'TC02'
def skip(type):
if type in self.teststoSkip:
self.skipTest('skipped!!') #unittest.Testcase method
def test_TC02(self):
self.skip('TC02')
print 'test_TC02 will do other tasks'
def test_TC03(self):
self.skip('TC03')
print 'test_TC03 will do other tasks'
This will work fine. Now I want to reuse the same testcases in another class. say,
RegressionMyTest.py
from MyTestClass import MyTestClass
Class RegressionMyTest(MyTestClass):
#classmethod
def setupmytest(self):
self.test_TC01_set(self)#this will work fine since it is accessing classmethod
self.tes_TC02(self)#cant access like this since it is not a class method
self.tes_TC03(self)#cant access like this since it is not a class method
How can I reuse the tests in MyTestClass in RegressionMyTest so that both MyTestClass and RegressionMyTest should work if they are run individually using nosetests/unittest.
Usually tests are supposed to assert code is functioning in a certain way, so I'm not sure if it would make sense to actually share tests between testsuites, (I don't think it would be very explicit)
Python tests cases are just python classes, that are introspected by the test runner for methods beginning in test_. Because of this you can you use inheritance in the same way you would with normal classes.
If you need shared functionality, you could create a base class with shared initialization methods/ helper methods. Or create testing mixins with utility functions that are needed across tests.
class BaseTestCase(unittest.TestCase):
def setUp(self):
# ran by all subclasses
def helper(self):
# help
class TestCaseOne(BaseTestCase):
def setUp(self):
# additional setup
super(TestCaseOne, self).setUp()
def test_something(self):
self.helper() # <- from base
Perhaps you don't want to define a setup method on the base class and just define on child classes using some helper methods defined in the base class? Lots of options!
I don't think your title describe your question correctly. Your code mistake is:
Calling a parent class "object method" in the child "class method"(#classmethod), because an "object method" must have one class instance(object), so in the child "class method", the system could find any object instance for its parent class.
You just need review the concepts of "class methods" and "object methods"(or instance methods) in programming language.
Not sure if this is a dupe or not. Here it goes.
I need to write some Python code that looks like:
class TestClass:
def test_case(self):
def get_categories(self):
return [“abc”,”bcd”]
# do the test here
and then have a test engine class that scans all these test classes, loads all the test_case functions and for each invokes get_categories to find out if the test belongs t the group of interest for the specific run.
The problem is that get_categories is not seen as an attribute of test_case, and even if I manually assign it
class TestClass:
def test_case(self):
def get_categories(self):
return [“abc”,”bcd”]
# do the test here
test_case.get_categories = get_categories
this is only going to happen when test_case first runs, too late for me.
The reason why this function can’t go on the class (or at least why I want it to be also available at the per-function level) is that a TestClass can have multiple test cases.
Since this is an already existing testing infrastructure, and the categories mechanism works (other than the categories-on-function scenario, which is of lesser importance), a rewrite is not in the plans.
Language tricks dearly appreciated.
Nested functions don't become attributes any more than any other assignment.
I suspect your test infrastructure is doing some severely weird things if this isn't supported (and uses old-style classes!), but you could just do this:
class TestClass:
def test_case(self):
# ...
def _get_categories(self):
return [...]
test_case.get_categories = _get_categories
del _get_categories
Class bodies are executable code like any other block.
What you need is nested classes. Functions aren't made to do what you are trying to do, so you have to move up a notch. Function attributes are mainly used as markup, whereas classes can have anything you want.
class TestClass(object):
class TestCase(object):
#classmethod
def get_categories(cls):
return ['abc', 'efg']
Note that I used #classmethod so that you could use it without instantiating TestCase(); modify if you want to do test_case = TestCase().