I have a lot of legacy test like this. Tests are skipped with unittest but are run with nosetests.
import unittest
import nose
import time
class BaseTestClass(unittest.TestCase):
#unittest.skip("Skipped")
def test_add(self):
"""Test 1"""
print "Execute test 1"
time.sleep(5)
def test_sub(self):
"""Test 2"""
print "Execute test 2"
time.sleep(5)
if __name__ == '__main__':
nose.run(argv=["nose", ".", "--verbosity=2", "--nocapture", '--no-skip'])
I want to run all skipped tests.
Looks like "no-skip" - option not works, become I have output
Execute test 2
Test 1 ... Test 2 ... ok
----------------------------------------------------------------------
Ran 2 tests in 5.001s
OK
looks like test is present in output but doesn't execute code inside.
I expected to see:
Test 1 ... ok
Execute test 1
Test 2 ... ok
Execute test 2
----------------------------------------------------------------------
Ran 2 tests in 10 s
OK
I got a similar problem, and after some research the issue seems to be that --no-skip does not have anything to do with unittests.skip but with the handling the exception nose.SkipTest.
I ended up with a hacky solution that does the job in my case. Maybe it can help somebody. It consists of using your own custom skip function so that it "piggybacks" the --no-skip flag and reacts accordingly.
import sys
import unittest
no_skip_mode = '--no-skip' in sys.argv
def skip(reason):
if no_skip_mode:
# Don't decorate the test function. Just return as is.
def dummy_wrapper(fn):
return fn
return dummy_wrapper
return unittest.skip(reason)
I guess you could try to monkeypatch unittest.skip.
Related
I am trying to generate a coverage report using the coverage-API (https://coverage.readthedocs.io/en/6.3.2/api.html#api).
The simple use-case described on the linked page tells me to wrap my executed code inside the snippet they provide. I am using unittest.main() to execute tests. The below code runs without an error but neither is any report information created nor is print("Done.") executed.
I guess unittest.main() calls a sys.exit() somewhere along the way? How does one use the API to execute all unittest-tests?
Example
import coverage
import unittest
def func(input):
return input
class testInput(unittest.TestCase):
def test_func(self):
self.assertEqual(func(1), 1)
if __name__ == '__main__':
cov = coverage.Coverage()
cov.start()
unittest.main()
cov.stop()
cov.save()
cov.html_report()
print("Done.")
Yes, it looks like unittest.main() is calling sys.exit(). Are you sure you need to use the coverage API? You can skip coverage and most of unittest:
# foo.py
import unittest
def func(input):
return input
class testInput(unittest.TestCase):
def test_func(self):
self.assertEqual(func(1), 1)
Then:
$ python -m coverage run -m unittest foo.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
$ python -m coverage report -m
Name Stmts Miss Cover Missing
--------------------------------------
foo.py 6 0 100%
--------------------------------------
TOTAL 6 0 100%
To get the example to work you can simply put the call to unittest.main() in a try/except statement so the cov.stop() will be called once the unit test has completed.
# testInput.py
import coverage
import unittest
def func(input):
return input
class testInput(unittest.TestCase):
def test_func(self):
self.assertEqual(func(1), 1)
if __name__ == '__main__':
cov = coverage.Coverage()
cov.start()
try:
unittest.main()
except: # catch-all except clause
pass
cov.stop()
cov.save()
cov.html_report()
print("Done.")
This will run as desired, including printing Done. at the end:
python3 testInput.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
Done.
The HTML coverage report will have been created in the htmlcov directory in the current working directory.
In the future, I'll need to add many identical tests with different parameters. Now I am making a sample test suite:
import unittest
class TestCase(unittest.TestCase):
def __init__(self, methodName='runTest', param=None):
super(TestCase, self).__init__(methodName)
self.param = param
def test_something(self):
print '\n>>>>>> test_something: param =', self.param
self.assertEqual(1, 1)
if __name__ == "__main__":
suite = unittest.TestSuite()
testloader = unittest.TestLoader()
testnames = testloader.getTestCaseNames(TestCase)
for name in testnames:
suite.addTest(TestCase(name, param=42))
unittest.TextTestRunner(verbosity=2).run(suite)
It gets discovered by VS Code:
start
test.test_navigator.TestCase.test_something
When I run the tests, I don't receive the parameter:
test_something (test.test_navigator.TestCase) ...
>>>>>> test_something: param = None
ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
If I run this file directly, everything works as expected (note param = 42 part)
test_something (__main__.TestCase) ...
>>>>>> test_something: param = 42
ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
So it looks like VS Code is running the tests on its own just by using the discovered classes and ignoring TestSuite completely?
What am I doing wrong?
Thanks.
The problem is your code is in a if __name__ == "__main__" block which is only executed when you point Python directly at the file. So when the extension asks unittest to get all the tests and then run them for us it doesn't run the code in your if __name__ == "__main__" block (which is why it can find it but it doesn't do anything magical).
If you can get it to work using unittest's command-line interface then the extension should run it as you want it to.
The key is to implement the load_tests function:
def load_tests(loader, tests, pattern):
suite = unittest.TestSuite()
testnames = loader.getTestCaseNames(TestCase)
for name in testnames:
suite.addTest(TestCase(name, param=42))
suite.addTest(TestCase(name, param=84))
return suite
The documentation says:
If load_tests exists then discovery does not recurse into the package, load_tests is responsible for loading all tests in the package.
Now my tests run as expected.
P.S. Thanks to Brett Cannon for pointing me to Unit testing framework documentation
I have a set of unit tests that look basically like this:
import unittest
class MyTestCase(unittest.TestCase):
def test_one(self):
self.assertEqual(1,1)
def test_two(self):
self.assertEqual(2,2)
if __name__ == '__main__':
unittest.main()
When I run the tests, I would expect to see results from both test_methods (test_one and test_two), but I only get results for one:
$ python -m unittest -v tests.test_dummy
test_one (tests.test_dummy.MyTestCase) ... ok
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
What's going on here?
Oh, this is mortifying.
It was an indentation problem. The test file had a combination of tabs and spaces, so the second test wasn't being identified.
I copied the invisible glitch from my real code, to the simplified MyTestCase class I used to replicate the error. But stackoverflow stripped out the tabs, so no one else was able to replicate.
Here's the bug that was killing my code. Tabs are shown with hyphens: ----
import unittest
class MyTestCase(unittest.TestCase):
def test_one(self):
self.assertEqual(1,1)
----def test_two(self):
--------self.assertEqual(2,2)
if __name__ == '__main__':
unittest.main()
My code:
import unittest
from mylib import myfunc
class test_mylib(unittest.TestCase):
def test_myfunc(self):
self.assertTrue(myfunc(10)==50)
if __name__ == '__main__':
unittest.main()
It shows the test as OK, but then returns False at the same time. Is there a way for it to return True if the tests are OK?
Output:
OK
Exit code: False
On Unix, the process exit code 0 means success. I guess whatever tool you're using to run the tests is interpreting that as False.
Using Python Unittest, here's an example of a test suite :
import unittest
# Here's our "unit".
def IsOdd(n):
return n % 2 == 1
# Here's our "unit tests".
class IsOddTests(unittest.TestCase):
def testOne(self):
self.failUnless(IsOdd(1))
def testTwo(self):
self.failIf(IsOdd(2))
def main():
unittest.main(verbosity=2)
if __name__ == '__main__':
main()
And the result :
testOne (__main__.IsOddTests) ... ok
testTwo (__main__.IsOddTests) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
Is it possible to enhance the display of the tests, something like :
Testing ODD method
Testing with value is 1 (__main__.IsOddTests) ... ok
Testing with value is 2 (__main__.IsOddTests) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
What I'm trying to do, is in case of a lot of tests, displaying a Group name for each TestCase (that contains multiple tests), and a name for each tests (that should be more explicit than just the function name).
To do this simply set a docstring for your tests:
def testOne(self):
"""Test IsOdd(1)"""
self.failUnless(IsOdd(1))
def testTwo(self):
"""Test IsOdd(2)"""
self.failIf(IsOdd(2))
There's a bit of an art to picking docstrings for your tests that will make sense later. Don't be afraid to go back and refactor your things.