I'm testing a class, with many test methods. However, each method has a unique context. I then write my code as following:
class TestSomeClass(unittest.TestCase):
def test_a():
with a_context() as c:
pass
def test_b():
with b_context() as c:
pass
def test_c():
with c_context() as c:
pass
However, the context managers are irrelevant to the test case, and produce temporary files. So as to not pollute the file system when the test fails, I would like to use each context manager in a setup/teardown scenario.
I've looked at nose's with_setup, but the docs say that is meant for functions only, not methods. Another way is to move the test methods to separate classes each with a setup/teardown function. What's a good way to do this?
First of all, I'm not sure why what you have isn't working. I wrote some test code, and it shows that the exit code always gets called, under the unittest.main() execution environment. (Note, I did not test nose, so maybe that's why I couldn't replicate your failure.) Maybe your context manager is broken?
Here's my test:
import unittest
import contextlib
import sys
#contextlib.contextmanager
def context_mgr():
print "setting up context"
try:
yield
finally:
print "tearing down context"
class TestSomeClass(unittest.TestCase):
def test_normal(self):
with context_mgr() as c:
print "normal task"
def test_raise(self):
with context_mgr() as c:
print "raise task"
raise RuntimeError
def test_exit(self):
with context_mgr() as c:
print "exit task"
sys.exit(1)
if __name__ == '__main__':
unittest.main()
By running that with $ python test_test.py I see tearing down context for all 3 tests.
Anyway, to answer your question, if you want a separate setup and teardown for each test, then you need to put each test in its own class. You can set up a parent class to do most of the work for you, so there isn't too much extra boilerplate:
class TestClassParent(unittest.TestCase):
context_guard = context_mgr()
def setUp(self):
#do common setup tasks here
self.c = self.context_guard.__enter__()
def tearDown(self):
#do common teardown tasks here
self.context_guard.__exit__(None,None,None)
class TestA(TestClassParent):
context_guard = context_mgr('A')
def test_normal(self):
print "task A"
class TestB(TestClassParent):
context_guard = context_mgr('B')
def test_normal(self):
print "task B"
This produces the output:
$ python test_test.py
setting up context: A
task A
tearing down context: A
.setting up context: B
task B
tearing down context: B
.
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
Related
I was amazed to see something like this in a code review request today:
import unittest
class SomeTestClass(unittest.TestCase):
#classmethod
def setUpClass(cls):
...
cls.assertTrue(some_condition, "a message")
This sparked my interest as I know in python2.7 you could not call assertXXX methods in a classmethod or a staticmethod and it would have failed there. I quickly put up some test code to check:
import unittest
class TestClass(unittest.TestCase):
#classmethod
def setUpClass(cls):
var = 34
cls.assertTrue(var == 34)
cls.assertTrue(var == 33)
def test_123(self):
self.assertFalse(False)
if __name__ == '__main__':
unittest.main()
Hoping for python language to fail on calling assertTrue which is an instance method in a static context and if it works (I thought may be in python3 it changed) then cls.assertTrue(var == 33) to raise an AssertionError. But to much of my surprize None of the above happened and I see:
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
I am using python3.7. My questions are:
shouldn't this fail as assertTrue is an instance method? note that assertFalse and other such method do cause a failure
If it does work shouldn't assertTrue fail when the condition evaluate to False ?
This has to do with how functions are bound in 3.x. Consider a simpler example:
class Example:
def method(self):
print(f'Example.method called on {self!r}')
Example.method('a string')
I need to test smth on python via ssh. I don't want to make ssh connection for every test, because it is to long, I have written this:
class TestCase(unittest.TestCase):
client = None
def setUp(self):
if not hasattr(self.__class__, 'client') or self.__class__.client is None:
self.__class__.client = paramiko.SSHClient()
self.__class__.client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
self.__class__.client.connect(hostname=consts.get_host(), port=consts.get_port(), username=consts.get_user(),
password=consts.get_password())
def test_a(self):
pass
def test_b(self):
pass
def test_c(self):
pass
def disconnect(self):
self.__class__.client.close()
and my runner
if __name__ == '__main__':
suite = unittest.TestSuite((
unittest.makeSuite(TestCase),
))
result = unittest.TextTestRunner().run(suite)
TestCase.disconnect()
sys.exit(not result.wasSuccessful())
In this version I get error TypeError: unbound method disconnect() must be called with TestCase instance as first argument (got nothing instead). So how i can call disconnect after all tests pass?
With best regards.
You should use setUpClass and tearDownClass instead, if you want to keep the same connection for all tests. You'll also need to make the disconnect method static, so it belongs to the class and not an instance of the class.
class TestCase(unittest.TestCase):
def setUpClass(cls):
cls.connection = <your connection setup>
#staticmethod
def disconnect():
... disconnect TestCase.connection
def tearDownClass(cls):
cls.disconnect()
you can do it by defining startTestRun,stopTestRun of unittest.TestResult class. setUpClass and tearDownClass are running per test class(per test file) so if you have multiple files this methods will run for each one.
by adding following code to my tests/__init__.py i managed to achieve it. this code runs only once for all tests(regardless of number of test classes and test files).
def startTestRun(self):
"""
https://docs.python.org/3/library/unittest.html#unittest.TestResult.startTestRun
Called once before any tests are executed.
:return:
"""
DockerCompose().start()
setattr(unittest.TestResult, 'startTestRun', startTestRun)
def stopTestRun(self):
"""
https://docs.python.org/3/library/unittest.html#unittest.TestResult.stopTestRun
Called once after all tests are executed.
:return:
"""
DockerCompose().compose.stop()
setattr(unittest.TestResult, 'stopTestRun', stopTestRun)
There seems to be a simple solution for the basic (beginner's) case:
def tmain():
setup()
unittest.main(verbosity=1, exit=False)
clean()
The trick is "exit=False" which lets the function tmain run until the end.
setup() and clean() can be any functions to do what the name implies.
I am able to setup nose tests to run with the #attr tag. I am now interested in know if I can append to the end of the test name, the #attr tag? What we are trying to do is add a tag if our tests run into an issue and we write up a defect for it, we would then put the defect number as an #attr tag. Then when we run we could easily identify which tests have open defects against them.
Just wondering if this is even possible, and where to go to see how to set it up?
EDIT RESULTS RUNNING WITH ANSWER:
Test Results:
So I sort of know what is going on, if I have the #fancyattr() at the class level it picks it up and changes the name of the class. When I put the #fancyattr() at the test level it is not changing the name of the test, which is what I need for it to do.
For example - Changes the name of the class:
#dms_attr('DMSTEST')
#attr('smoke_login', 'smoketest', priority=1)
class TestLogins(BaseSmoke):
"""
Just logs into the system and then logs off
"""
def setUp(self):
BaseSmoke.setUp(self)
def test_login(self):
print u"I can login -- taking a nap now"
sleep(5)
print u"Getting off now"
def tearDown(self):
BaseSmoke.tearDown(self)
This is what I need and it isn't working:
#attr('smoke_login', 'smoketest', priority=1)
class TestLogins(BaseSmoke):
"""
Just logs into the system and then logs off
"""
def setUp(self):
BaseSmoke.setUp(self)
#dms_attr('DMSTEST')
def test_login(self):
print u"I can login -- taking a nap now"
sleep(5)
print u"Getting off now"
def tearDown(self):
BaseSmoke.tearDown(self)
Updated screenshot with what I am seeing with __doc__:
Here is how to do it with args type attributes:
rename_test.py:
import unittest
from nose.tools import set_trace
def fancy_attr(*args, **kwargs):
"""Decorator that adds attributes to classes or functions
for use with the Attribute (-a) plugin. It also renames functions!
"""
def wrap_ob(ob):
for name in args:
setattr(ob, name, True)
#using __doc__ instead of __name__ works for class methods tests
ob.__doc__ = '_'.join([ob.__name__, name])
#ob.__name__ = '_'.join([ob.__name__, name])
return ob
return wrap_ob
class TestLogins(unittest.TestCase):
#fancy_attr('slow')
def test_method():
assert True
#fancy_attr('slow')
def test_func():
assert True
Running test:
$ nosetests rename_test.py -v
test_method_slow ... ok
test_func_slow ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.003s
OK
EDIT: For xunit reporting to work, test renaming should take place before running the test. You can do it on import, here is untested hack showing how to do it:
rename_test.py:
import unittest
def fancy_attr(*args, **kwargs):
"""Decorator that adds attributes to classes or functions
for use with the Attribute (-a) plugin. It also renames functions!
"""
def wrap_ob(ob):
for name in args:
setattr(ob, name, True)
ob.__doc__ = '_'.join([ob.__name__, name])
return ob
return wrap_ob
class TestLogins(unittest.TestCase):
#fancy_attr('slow')
def test_method(self):
assert True
def make_name(orig, attrib):
return '_'.join([orig, attrib])
def rename(cls):
methods = []
for key in cls.__dict__:
method = getattr(cls, key)
if method:
if hasattr(cls.__dict__[key], '__dict__'):
if 'slow' in cls.__dict__[key].__dict__:
methods.append(key)
print methods
for method in methods:
setattr(cls, make_name(method, 'slow'), cls.__dict__[key])
delattr(cls, method)
rename(TestLogins)
#fancy_attr('slow')
def test_func():
assert True
import unittest
import HTMLTestRunner
class TestClass1(unittest.TestCase):
def setUp(self):
pass
def case1(self):
assert 4 == 3
def case2(self):
assert 4 == 4
def tearDown(self):
pass
class TestClass2(unittest.TestCase):
def setUp(self):
pass
def case3(self):
assert 1 == 2
def tearDown(self):
pass
def suite():
suite = unittest.TestSuite()
suite.addTest(TestClass1(['case1','case2']))
suite.addTest(TestClass2('case4'))
return suite
test_suite = suite()
unittest.TextTestRunner(verbosity=2).run(test_suite)
fp = file('my_report.html', 'wb')
runner = HTMLTestRunner.HTMLTestRunner(
stream=fp,
title='My unit test',
description='This demonstrates the report output by HTMLTestRunner.'
)
runner.run(test_suite)
I am trying to run all the methods in both the classes in a single run. However, the code above did not do so. In the suite function, I tried to add multiple tests from the classes but that also did not work and was giving an error.
From this answer at the question "Is test suite deprecated in PyUnit?":
"unittest.TestSuite is not necessary if you want to run all the tests in a single module as unittest.main() will dynamically examine the module it is called from and find all classes that derive from unittest.TestCase."
There's more in that answer about when unittest.TestSuite is useful.
That said, I needed to make some changes to get these tests to work. Firstly, unittest looks for functions with "test_" at their start. Also, unittest's assertEqual and similar methods should be used, instead of just Python's assert statement. Doing that and eliminating some unneeded code led to:
import unittest
class TestClass1(unittest.TestCase):
def test_case1(self):
self.assertEqual(4, 3)
def test_case2(self):
self.assertEqual(4, 4)
class TestClass2(unittest.TestCase):
def test_case3(self):
self.assertEqual(1, 2)
unittest.main()
This produced appropriate output (3 tests run with 2 failures), which I won't reproduce here in the interest of space.
Can I call a test method from within the test class in python? For example:
class Test(unittest.TestCase):
def setUp(self):
#do stuff
def test1(self):
self.test2()
def test2(self):
#do stuff
update: I forgot the other half of my question. Will setup or teardown be called only after the method that the tester calls? Or will it get called between test1 entering and after calling test2 from test1?
This is pretty much a Do Not Do That. If you want tests run in a specific order define a runTest method and do not name your methods test....
class Test_Some_Condition( unittest.TestCase ):
def setUp( self ):
...
def runTest( self ):
step1()
step2()
step3()
def tearDown( self ):
...
This will run the steps in order with one (1) setUp and one (1) tearDown. No mystery.
Try running the following code:
import unittest
class Test(unittest.TestCase):
def setUp(self):
print 'Setting Up'
def test1(self):
print 'In test1'
self.test2()
def test2(self):
print 'In test2'
def tearDown(self):
print 'Tearing Down'
if __name__ == '__main__':
unittest.main()
And the results is:
Setting Up
In test1
In test2
Tearing Down
.Setting Up
In test2
Tearing Down
.
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
Now you see, setUp get called before a test method get called by unittest, and tearDown is called after the call.
All methods whose name begins with the string 'test' are considered unit tests (i.e. they get run when you call unittest.main()). So you can call methods from within the Test class, but you should name it something that does not start with the string 'test' unless you want it to be also run as a unit test.
sure, why not -- however that means test2 will be called twice -- once by test1 and then again as its own test, since all functions named test will be called.
Yes to both:
setUp will be called between each test
test2 will be called twice.
If you would like to call a function inside a test, then omit the test prefix.