Let's say that, for whatever reason, I want to write a set of test cases that start out from the premise of failing code.
Maybe because setting up a failure is really complicated, while the demonstrating a fixed state is simple. In my case, it's not even that, I wanted to start with a failed test and then show how to fix it, for documentation purposes.
I can decorate #unittest.expectedFailure on the base class.
But the the fixed subclasses blow up with unexpected success because the decoration is inherited.
Can I remove the expectedFailure somehow?
In the code itself, not in a command line argument?
While I use and appreciate pytest this is a question for regular unittest.
unittest.skipXXX is not what I want, I do want to run Test_ThisShouldFail's test.
import sys
import unittest
#unittest.expectedFailure
class Test_ThisShouldFail(unittest.TestCase):
""" for argument's sake, let's say
the configuration and testing is very complicated
and I want it fail by default.
Subclasses fix the issue but re-use the test code
"""
data = dict(a=1, b=2)
def test_evens(self):
for key, v in self.data.items():
self.assertFalse(v % 2, f"{key}:Odd on {v}")
## 👉???unittest.expectedSuccess????
class Test_ThisShouldWork(Test_ThisShouldFail):
""" how do I turn off the expected failure? """
def setUp(self):
self.data.update(a=10)
if __name__ == "__main__":
sys.exit(unittest.main())
output:
FAILED (expected failures=1, unexpected successes=1)
(venv) #explore$ py test_expectedfail.py -v
test_evens (__main__.Test_ThisShouldFail) ... expected failure
test_evens (__main__.Test_ThisShouldWork) ... unexpected success
----------------------------------------------------------------------
Ran 2 tests in 0.000s
FAILED (expected failures=1, unexpected successes=1)
this didn't work:
I was hoping the MRO would look at TurnItOff's blank unittest settings and use them. No such luck.
class TurnItOff(unittest.TestCase):
pass
class Test_ThisShouldWork(TurnItOff, Test_ThisShouldFail):
....
This relies on the internal implementation of unittest.expectedFailure, but works for a start:
def expectedSuccess(test_item):
test_item.__unittest_expecting_failure__ = False
return test_item
#unittest.expectedFailure
class TestFailure(unittest.TestCase):
def test_something(self):
self.assertTrue(False)
#expectedSuccess
class TestSuccess(TestFailure):
def test_something(self):
self.assertTrue(True)
Note that test_item can be both a class or a function, depending on where you put the decorator.
Related
I have following unittest code running via py.test.
Mere presence of the constructor make the entire class skip when running
py.test -v -s
collected 0 items / 1 skipped
Can anyone please explain to me this behaviour of py.test?
I am interested in understanding py.test behaviour, I know the constructor is not needed.
Thanks,
Zdenek
class TestClassName(object):
def __init__(self):
pass
def setup_method(self, method):
print "setup_method called"
def teardown_method(self, method):
print "teardown_method called"
def test_a(self):
print "test_a called"
assert 1 == 1
def test_b(self):
print "test_b called"
assert 1 == 1
The documentation for py.test says that py.test implements the following standard test discovery:
collection starts from the initial command line arguments which may be directories, filenames or test ids.
recurse into directories, unless they match norecursedirs
test_*.py or *_test.py files, imported by their package name.
Test prefixed test classes (without an __init__ method) [<-- notice this one here]
test_ prefixed test functions or methods are test items
So it's not that the constructor isn't needed, py.test just ignores classes that have a constructor. There is also a guide for changing the standard test discovery.
As already mentioned in the answer by Matti Lyra py.test purposely skips classes which have a constructor. The reason for this is that classes are only used for structural reasons in py.test and do not have any inherent behaviour, while when actually writing code it is the opposite and much rarer to not have an .__init__() method for a class. So in practice skipping a class with a constructor will likely be what was desired, usually it is just a class which happens to have a conflicting name.
Lastly py.test needs to instantiate the class in order to execute the tests. If the constructor takes any arguments it can't instantiate it, so again skipping is the right thing to do.
All the above answers clearly explain the underlying cause, I just thought to share my experience and workaround the warnings.
I got my test to work without the warnings by aliasing the imported Class
from app.core.utils import model_from_meta
from app.core.models import Panel, TestType as _TestType
from app.core.serializers import PanelSerializer, TestType as _TestTypeSerializer
def test_model_from_meta():
assert (Panel is model_from_meta(PanelSerializer))
assert (_TestType is model_from_meta(_TestTypeSerializer))
After importing the class using aliases the warnings no longer get printed
I hope this helps someone.
In my case, I just so happened to have a parameter's class names TestParams, which conflicts with pytest looking for classes beginning with the name test....
Solution: rename your own class
Source
I am doing some unittests with python and some pre-test checks in setUpClass. How can I throw a unitest-fail within the setUpClass, as the following simple example:
class MyTests(unittest.TestCase):
#classmethod
def setUpClass(cls):
unittest.TestCase.fail("Test")
def test1(self):
pass
if __name__ == '__main__':
unittest.main()
gives the error TypeError: unbound method fail() must be called with TestCase instance as first argument (got str instance instead).
I understand the error I get as fail is a instance method, and I don't have an instance of MyClass yet. Using an instance on-the-fly like
unittest.TestCase().fail("Test")
also does not work, as unittest.TestCase itself has no tests. Any ideas how to fail all tests in MyClass, when some condition in setUpClass is not met?
Followup question: Is there a way to see the tests in setUpClass?
self.fail("test") put into your setUp instance method fails all the tests
I think the easiest way to do this at the class level is to make a class variable so something like:
#classmethod
def setUpClass(cls):
cls.flag = False
def setUp(self):
if self.flag:
self.fail("conditions not met")
Hope this is what you want.
Using a simple assert should work
assert False, "I mean for this to fail"
I'm not an expert in python but with same problem, I've resolved adding cls param:
...
#classmethod
def setUpClass(cls):
cls.fail(cls, "Test")
...
I think is strange but clearer.
When you pass cls, warning appear (Expected type 'TestCase', got 'Type[MyTests]' instead) and MyTests inherits from TestCase thus can
ignoring adding: # noinspection PyTypeChecker
I have a data structure that is created in one function and passed onto other functions. I am trying to unit test all those functions.
Do I need to re-create that data structure (the environment) at each function? I tried using a global variable but I cannot guarantee which test case will run before the other.
I know I cannot override __init__ of unittest.TestCase without much headache.
How else can I achieve that? Passing a parameter or somehow making it a variable and avoiding a race condition?
It sounds like you do not want to redefine the data structure before each test. As long as the tests do not modify the data, I don't think there is any problem with defining the data structure in __init__:
import unittest
class Test(unittest.TestCase):
def __init__(self, methodName = 'runTest'):
unittest.TestCase.__init__(self, methodName)
self.data = range(5)
def test_getitem(self):
self.assertEqual(self.data[1],1)
def test_reversed(self):
self.assertEqual(list(reversed(self.data)),[4,3,2,1,0])
if __name__ == '__main__':
import sys
sys.argv.insert(1,'--verbose')
unittest.main(argv = sys.argv)
yields
% test.py
test_getitem (__main__.Test) ... ok
test_reversed (__main__.Test) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
I am trying to get started with unittest, but I am having a problem getting setUpClass() to work. Here is my test code...
import unittest
class TestRepGen(unittest.TestCase):
""" Contains methods for training data testing """
testvar = None
#classmethod
def setUpClass(cls):
cls.testvar = 6
def test_method(self):
""" Ensure data is selected """
self.assertIsNotNone(self.testvar,"self.testvar is None!")
# run tests
if __name__ == '__main__':
unittest.main()
The assert error message is displayed, indicating that self.testvar == None, and has not been changed by setUpClass(). Is there something wrong with my code?
I get the same result if I run the code from within my IDE (Wing), or directly from the command line. For the record, I am using Python 3.2.1 under Windows7.
setUpClass is new to the unittest framework as of 2.7 and 3.2. If you want to use it with an older version you will need to use nose as your test runner instead of unittest.
I'm going to guess you aren't calling this test class directly, but a derived class.
If that's the case, you have to call up to setUpClass() manually -- it's not automatically called.
class TestB(TestRepGen):
#classmethod
def setUpClass(cls):
super(TestB, cls).setUpClass()
Also, accoding to the docs class-level fixtures are implemented by the test suite. So, if you're calling TestRepGen or test_method some weird way you didn't post, setUpClass might not get run.
Ok - it's hands-up time to admit that it was my mistake. I was using 3.1.2 when I should have been (and thought I was) using 3.2.1. I have now changed to the correct version, all is well and the test is passing. Many thanks to all who replied (and sorry for wasting your time :-( ).
When using nosetests for Python it is possible to disable a unit test by setting the test function's __test__ attribute to false. I have implemented this using the following decorator:
def unit_test_disabled():
def wrapper(func):
func.__test__ = False
return func
return wrapper
#unit_test_disabled
def test_my_sample_test()
#code here ...
However, this has the side effect of calling wrapper as the unit test. Wrapper will always pass but it is included in nosetests output. Is there another way of structuring the decorator so that the test will not run and does not appear in nosetests output.
Nose already has a builtin decorator for this:
from nose.tools import nottest
#nottest
def test_my_sample_test()
#code here ...
Also check out the other goodies that nose provides: https://nose.readthedocs.org/en/latest/testing_tools.html
You can also use unittest.skip decorator:
import unittest
#unittest.skip("temporarily disabled")
class MyTestCase(unittest.TestCase):
...
There also is a skiptest plugin for nosetest, which will cause the test show in test output as skipped. Here is a decorator for that:
def skipped(func):
from nose.plugins.skip import SkipTest
def _():
raise SkipTest("Test %s is skipped" % func.__name__)
_.__name__ = func.__name__
return _
Example output:
$ nosetests tests
..........................................................................
..................................S.............
----------------------------------------------------------------------
Ran 122 tests in 2.160s
OK (SKIP=1)
You can just start the class, method or function name with an underscore and nose will ignore it.
#nottest has its uses but I find that it does not work well when classes derive from one another and some base classes must be ignored by nose. This happens often when I have a series of similar Django views to test. They often share characteristics that need testing. For instance, they are accessible only to users with certain permissions. Rather than write the same permission check for all of them, I put such shared test in an initial class from which the other classes derive. The problem though is that the base class is there only to be derived by the later classes and is not meant to be run on its own. Here's an example of the problem:
from unittest import TestCase
class Base(TestCase):
def test_something(self):
print "Testing something in " + self.__class__.__name__
class Derived(Base):
def test_something_else(self):
print "Testing something else in " + self.__class__.__name__
And the output from running nose on it:
$ nosetests test.py -s
Testing something in Base
.Testing something in Derived
.Testing something else in Derived
.
----------------------------------------------------------------------
Ran 3 tests in 0.000s
OK
The Base class is included in the tests.
I cannot just slap #nottest on Base because it will mark the entire hierarchy. Indeed if you just add #nottest to the code above in front of class Base, then nose won't run any tests.
What I do is add an underscore in front of the base class:
from unittest import TestCase
class _Base(TestCase):
def test_something(self):
print "Testing something in " + self.__class__.__name__
class Derived(_Base):
def test_something_else(self):
print "Testing something else in " + self.__class__.__name__
And when running it _Base is ignored:
$ nosetests test3.py -s
Testing something in Derived
.Testing something else in Derived
.
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
This behavior is not well documented but the code that selects tests explicitly checks for an underscore at the start of class names.
A similar test is performed by nose on function and method names so it is possible to exclude them by adding an underscore at the start of the name.
I think you will also need to rename your decorator to something that has not got test in. The below only fails on the second test for me and the first does not show up in the test suite.
def unit_disabled(func):
def wrapper(func):
func.__test__ = False
return func
return wrapper
#unit_disabled
def test_my_sample_test():
assert 1 <> 1
def test2_my_sample_test():
assert 1 <> 1