We Keep Coding

How to run unittest tests using coverage API

I am trying to generate a coverage report using the coverage-API (https://coverage.readthedocs.io/en/6.3.2/api.html#api).
The simple use-case described on the linked page tells me to wrap my executed code inside the snippet they provide. I am using unittest.main() to execute tests. The below code runs without an error but neither is any report information created nor is print("Done.") executed.
I guess unittest.main() calls a sys.exit() somewhere along the way? How does one use the API to execute all unittest-tests?
Example
import coverage
import unittest
def func(input):
return input
class testInput(unittest.TestCase):
def test_func(self):
self.assertEqual(func(1), 1)
if __name__ == '__main__':
cov = coverage.Coverage()
cov.start()
unittest.main()
cov.stop()
cov.save()
cov.html_report()
print("Done.")

Yes, it looks like unittest.main() is calling sys.exit(). Are you sure you need to use the coverage API? You can skip coverage and most of unittest:
# foo.py
import unittest
def func(input):
return input
class testInput(unittest.TestCase):
def test_func(self):
self.assertEqual(func(1), 1)
Then:
$ python -m coverage run -m unittest foo.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
$ python -m coverage report -m
Name Stmts Miss Cover Missing
--------------------------------------
foo.py 6 0 100%
--------------------------------------
TOTAL 6 0 100%

To get the example to work you can simply put the call to unittest.main() in a try/except statement so the cov.stop() will be called once the unit test has completed.
# testInput.py
import coverage
import unittest
def func(input):
return input
class testInput(unittest.TestCase):
def test_func(self):
self.assertEqual(func(1), 1)
if __name__ == '__main__':
cov = coverage.Coverage()
cov.start()
try:
unittest.main()
except: # catch-all except clause
pass
cov.stop()
cov.save()
cov.html_report()
print("Done.")
This will run as desired, including printing Done. at the end:
python3 testInput.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
Done.
The HTML coverage report will have been created in the htmlcov directory in the current working directory.

How to use unittest.TestSuite in VS Code?

In the future, I'll need to add many identical tests with different parameters. Now I am making a sample test suite:
import unittest
class TestCase(unittest.TestCase):
def __init__(self, methodName='runTest', param=None):
super(TestCase, self).__init__(methodName)
self.param = param
def test_something(self):
print '\n>>>>>> test_something: param =', self.param
self.assertEqual(1, 1)
if __name__ == "__main__":
suite = unittest.TestSuite()
testloader = unittest.TestLoader()
testnames = testloader.getTestCaseNames(TestCase)
for name in testnames:
suite.addTest(TestCase(name, param=42))
unittest.TextTestRunner(verbosity=2).run(suite)
It gets discovered by VS Code:
start
test.test_navigator.TestCase.test_something
When I run the tests, I don't receive the parameter:
test_something (test.test_navigator.TestCase) ...
>>>>>> test_something: param = None
ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
If I run this file directly, everything works as expected (note param = 42 part)
test_something (__main__.TestCase) ...
>>>>>> test_something: param = 42
ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
So it looks like VS Code is running the tests on its own just by using the discovered classes and ignoring TestSuite completely?
What am I doing wrong?
Thanks.

The problem is your code is in a if __name__ == "__main__" block which is only executed when you point Python directly at the file. So when the extension asks unittest to get all the tests and then run them for us it doesn't run the code in your if __name__ == "__main__" block (which is why it can find it but it doesn't do anything magical).
If you can get it to work using unittest's command-line interface then the extension should run it as you want it to.

The key is to implement the load_tests function:
def load_tests(loader, tests, pattern):
suite = unittest.TestSuite()
testnames = loader.getTestCaseNames(TestCase)
for name in testnames:
suite.addTest(TestCase(name, param=42))
suite.addTest(TestCase(name, param=84))
return suite
The documentation says:
If load_tests exists then discovery does not recurse into the package, load_tests is responsible for loading all tests in the package.
Now my tests run as expected.
P.S. Thanks to Brett Cannon for pointing me to Unit testing framework documentation

Why does python only run a single test_method in my TestCase?

I have a set of unit tests that look basically like this:
import unittest
class MyTestCase(unittest.TestCase):
def test_one(self):
self.assertEqual(1,1)
def test_two(self):
self.assertEqual(2,2)
if __name__ == '__main__':
unittest.main()
When I run the tests, I would expect to see results from both test_methods (test_one and test_two), but I only get results for one:
$ python -m unittest -v tests.test_dummy
test_one (tests.test_dummy.MyTestCase) ... ok
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
What's going on here?

Oh, this is mortifying.
It was an indentation problem. The test file had a combination of tabs and spaces, so the second test wasn't being identified.
I copied the invisible glitch from my real code, to the simplified MyTestCase class I used to replicate the error. But stackoverflow stripped out the tabs, so no one else was able to replicate.
Here's the bug that was killing my code. Tabs are shown with hyphens: ----
import unittest
class MyTestCase(unittest.TestCase):
def test_one(self):
self.assertEqual(1,1)
----def test_two(self):
--------self.assertEqual(2,2)
if __name__ == '__main__':
unittest.main()

Python doesn't detect my unittest

I have following code snippet -
import unittest
class SimpleWidgetTestCase(unittest.TestCase):
def setUp(self):
print 'setup'
def method_test(self):
print 'test method'
def tearDown(self):
print 'tear down'
if __name__ == "__main__":
unittest.main()
Output -
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK

test method name should start with test. Replace method_test to test_method, and try again.
From unittest documentation:
A testcase is created by subclassing unittest.TestCase. The three
individual tests are defined with methods whose names start with the
letters test. This naming convention informs the test runner about
which methods represent tests.

Python suite test doesn't run tests

I am trying to code a suite test, I have one module which runs unit test correctly but I intend to add more modules and test them at once, so I have coded the following code:
#main.py
import unittest
from test.Services import TestOS
if __name__ == '__main__':
suite = unittest.TestSuite()
suite.addTests( TestOS.TestOS() )
unittest.TextTestRunner().run(suite)
TestOS.py
import unittest
from app.Services.OS import OS
class TestOS(unittest.TestCase):
os = OS()
def setUp(self):
pass
def tearDown(self):
pass
def testOSName(self):
self.assertEquals(self.os.getPlatform(), 'Windows')
def testOSVersion(self):
self.assertEquals(self.os.getVersion(), '7')
if __name__ == "__main__":
#import sys;sys.argv = ['', 'Test.testName']
unittest.main()
After running it, I get this output:
Finding files... done.
Importing test modules ... done.
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
It didn't find any test, What's wrong with my code?

suite.addTest( TestOS.TestOS() ) works only if your testcase contains a runTest() function.
Otherwise you need a "TestLoader" to detect the functions of TestOS that start with "test*".
#main.py
import unittest
from test.Services import TestOS
if __name__ == '__main__':
suite = unittest.TestSuite()
tests = unittest.defaultTestLoader.loadTestsFromTestCase(TestOS)
suite.addTests(tests)
unittest.TextTestRunner().run(suite)

modify your setUp method as follows
def setUp(self):
self.os = OS()
pass