Since version 3.4, Python supports a simple subtest syntax when writing unittests. A simple example could look like this:
import unittest
class NumbersTest(unittest.TestCase):
def test_successful(self):
"""A test with subtests that will all succeed."""
for i in range(0, 6):
with self.subTest(i=i):
self.assertEqual(i, i)
if __name__ == '__main__':
unittest.main()
When running the tests, the output will be
python3 test_foo.py --verbose
test_successful (__main__.NumbersTest)
A test with subtests that will all succeed. ... ok
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
However, in my real world use cases, the subtests will depend on a more complex iterable and check something which is very different for each subtest. Consequently, I would rather have each subtest counted and listed as a separated test case in the output (Ran 6 tests in ... in this example) to get the full picture.
Is this somehow possible with the plain unittest module in Python? The nose test generator feature would output each test separately but I would like to stay compatible with the standard library if possible.
You could subclass unittest.TestResult:
class NumbersTestResult(unittest.TestResult):
def addSubTest(self, test, subtest, outcome):
# handle failures calling base class
super(NumbersTestResult, self).addSubTest(test, subtest, outcome)
# add to total number of tests run
self.testsRun += 1
Then in NumbersTest override the run function:
def run(self, test_result=None):
return super(NumbersTest, self).run(NumbersTestResult())
Sorry I cannot test this in a fully working environment right now, but this should do the trick.
Using python 3.5.2, themiurge's answer didn't work out-of-the-box for me but a little tweaking got it to do what I wanted.
I had to specifically get the test runner to use this new class as follows:
if __name__ == '__main__':
unittest.main(testRunner=unittest.TextTestRunner(resultclass=NumbersTestResult))
However this didn't print the details of the test failures to the console as in the default case. To restore this behaviour I had to change the class NumbersTestResult inherited from to unittest.TextTestResult.
class NumbersTestResult(unittest.TextTestResult):
def addSubTest(self, test, subtest, outcome):
# handle failures calling base class
super(NumbersTestResult, self).addSubTest(test, subtest, outcome)
# add to total number of tests run
self.testsRun += 1
Related
I'd like to prevent/skip some tests cases during runinng others in python. I couldn't achive to use #unittest.skip(reason) on my case. It always generates a Script Error in python unittest.
My code;
import unittest
#unittest.skip("something")
def main():
try:
something = []
for _ in range(4):
test.log("something happened")
The result is;
Error Script Error
Detail: SkipTest: something
Do you have any idea about the issue?
Unittest is a module that act as testing framework. You should use it according to the framework practices (taken from tutorialspoint.com):
import unittest
def add(x,y):
return x+y
class SimpleTest(unittest.TestCase):
#unittest.skip("demonstrating skipping")
def testadd1(self):
self.assertEquals(add(4,5),9)
if __name__ == '__main__':
unittest.main()
Squish test scripts written in Python are unrelated to the unittest module. I cannot tell how the two could be used together.
Squish offers the test.skip() function for skipping test cases:
test.skip(message);
test.skip(message, detail);
This function skips further execution of the current script test case, or the current BDD scenario (or BDD scenario outline iteration), by throwing an exception and adding a SKIPPED entry to Squish's test log with the given message string, and with the given detail, if provided. If the test case logged verification failures before the skip, it is still considered failed.
Please see the code below. I was trying to implement a test suite for my project
import unittest
class TestClass1(unittest.TestCase):
def test_1_first(self):
print("First test case")
def test_2_second(self):
print("Second test case")
class TestClass2(unittest.TestCase):
def test_3_third(self):
print("Third test case")
def test_4_fourth(self):
print("Fourth test case")
if __name__ == "__main__":
# val = 1 <-- getting from user
# if val == 1:
# Execute test cases in TestClass2
# else
# Execute test cases in TestClass1\
unittest.main()
I will get the class name as command line argument and I need to run each class of test according to the argument. Means I need to select and run some classes of test cases on runtime. So can any one help me to prevent a class from unit test execution at runtime? The problem is It's not allowed to use the method like passing class name while execution
unittest.main() already parses sys.argv to allow the running of specific classes or individual tests.
For example, if your script is named test.py, you can run the following to run just TestClass1:
python test.py __main__.TestClass1
Or the following to run just TestClass1.test_1_first:
python test.py __main__.TestClass1.test_1_first
If you wish to do it in your script, you can pass the name of test you want to run as the defaultTest argument:
unittest.main(defaultTest='__main__.TestClass1')
That is already built-in. You can run specific class via command:
python -m unittest test_module.TestClass1
If you really need to do it in your scpript, you can pass your classes to unittest.TestLoader().loadTestsFromName and then run test suite with unittest.TextTestRunner().run(suite). It will look something like this:
test_suite = unittest.TestLoader().loadTestsFromName('__main__.TestClass1')
unittest.TextTestRunner().run(test_suite)
I have a problem running unittests in pycharm. The first class 'KnownValues' runs but the other class doesn't get checked at all.
import roman
import unittest
class KnownValues(unittest.TestCase):
def test_too_large(self):
'''to_roman should fail with large input'''
self.assertRaises(roman.OutOfRangeError, roman.to_roman, 4000)
def test_too_small(self):
ls = [0,-1,-25,-60]
for x in ls:
self.assertRaises(roman.OutOfRangeError, roman.to_roman, x)
def test_non_int(self):
ls = [1.5, -6.5, 6.8,12.9, "hello wold", "nigga123"]
for x in ls:
self.assertRaises(roman.TypeError, roman.to_roman, x)
class Test2(unittest.TestCase):
def test1(self):
assert 1 == 1
if __name__ == '__main__':
unittest.main()
Start all of your test functions with test. Many people use underscores to separate words, so a lot of people end up with tests starting with test_, but test is all that is required.
When having trouble in the GUI, you can check how your tests are running from the command line.
python test.py
or
python -m test
One problem that you might run into is that you have defined your tests within classes, and when running them through the GUI, the GUI has automatically discovered them for you. Be sure to include the lines at the end of your test file directing the interpreter to use the main function built into unittest.
if __name__ == '__main__':
unittest.main()
Keep in mind, you can optionally run the tests in only one of your classes at a time:
python tests.py KnownValues
python tests.py Test2
In PyCharm, it should automatically discover all the test classes. You still have the option of running only one class at a time. Choose Run->Edit Configurations to see the options that you are currently running under. Using command line parameters you can control running fewer or more tests.
As you can see, you can choose to run a script, a class, or a method. Be sure to set the name of your run configuration such that it reflects the scope of what you are running.
I'm considering converting some unittest.TestCase tests into Pytest ones to take advantage of Pytest's fixtures. One feature of unittest that I wasn't able to easily find the equivalent of in Pytest, however, is the ability to create testing suites and run them. I currently often do something like this:
import unittest
class TestSomething(unittest.TestCase):
def test_1(self):
self.assertEqual("hello".upper(), "HELLO")
def test_2(self):
self.assertEqual(1+1, 2)
if __name__ == "__main__":
suite = unittest.TestSuite()
# suite.addTest(TestSomething('test_1'))
suite.addTest(TestSomething('test_2'))
runner = unittest.TextTestRunner()
runner.run(suite)
By commenting in and out the lines with addTest, I can easily select which tests to run. How would I do something similar with Pytest?
You can use the -k argument to run specific tests. For example
# put this in test.py
import unittest
class TestSomething(unittest.TestCase):
def test_1(self):
self.assertEqual("hello".upper(), "HELLO")
def test_2(self):
self.assertEqual(1+1, 2)
Running all tests in the class TestSomething can be done like this:
py.test test.py -k TestSomething
Running only test_2:
py.test test.py -k "TestSomething and test_2"
More examples in the documentation
Another way to go is to use special test names. These can be configures in the pytest.ini file.
# content of pytest.ini
# can also be defined in tox.ini or setup.cfg file, although the section
# name in setup.cfg files should be "tool:pytest"
[pytest]
python_files=check_*.py
python_classes=Check
python_functions=*_check
Another way is to take action in conftest.py. In this example the collect_ignore config variable is used. It is a list of test paths that are to be ignored. In this example test_somthing.py is always ignored for collection. test_other_module_py2.py is ignored if we are testing with a python 3.
# content of conftest.py
import sys
collect_ignore = ["test_something/test_something.py"]
if sys.version_info[0] > 2:
collect_ignore.append("test_other/test_other_module_py2.py")
Since pytest 2.6 it is also possible to omit classes from test registration like this:
# Will not be discovered as a test
class TestClass:
__test__ = False
These examples were loosely taken from the documentation of pytest chapter Changing standard (Python) test discovery
In addition to using -k filters, you can name specific test classes or cases you want to run,
py.test test.py::TestSomething::test_2
Would run just test_2
Think the best way to do this is to use custom pytest markers.
You should mark specific tests (which you want to run) with
#pytest.mark.mymarkername
And run only tests with the custom marker using command:
py.test -v -m mymarkername
Here you can find more info regarding markers:
http://doc.pytest.org/en/latest/example/markers.html
Building on mbatchkarov's answer, since the names of my tests can get quite lengthy, I would like to still be able to select tests by commenting in and out lines and hitting "Cntrl+B" in Sublime (or "Cntrl+R" using the Atom Runner). One way to do this is as follows:
import unittest
import pytest
class TestSomething(unittest.TestCase):
def test_1(self):
self.assertEqual("hello".upper(), "HELLO")
def test_2(self):
self.assertEqual(1+1, 2)
if __name__ == "__main__":
tests_to_run = []
# tests_to_run.append('TestSomething and test_1')
tests_to_run.append('TestSomething and test_2')
tests_to_run = " or ".join(tests_to_run)
args = [__file__, '-k', tests_to_run]
pytest.main(args)
The idea behind this is that because Pytest accepts a string expression to match tests (rather than just a list of tests), one must generate a list of expressions matching one test only, and concatenate them using or.
Hello I have a test module like the following under "test.py":
class TestBasic(unittest.TestCase):
def setUp(self):
# set up in here
class TestA(TestBasic):
def test_one(self):
self.assertEqual(1,1)
def test_two(self):
self.assertEqual(2,1)
if __name__ == "__main__":
unittest.main()
And this works pretty good, but I need a way to print which test passed, for example I could print the output to the console:
test_one: PASSED
test_two: FAILED
Now the twist, I could add a print statement right after the self.assertEqual() and that would be a passed test and I could just print it, but I need to run the test from a different module, let's say "test_reporter.py" where I have something like this:
import test
suite = unittest.TestLoader().loadTestsFromModule(test)
results = unittest.TextTestRunner(verbosity=0).run(suite)
at this point with results is when I build a report.
So please any suggestion is welcome
Thanks !!
Like Corey's comment mentioned, if you set verbosity=2 unittest will print the result of each test run.
results = unittest.TextTestRunner(verbosity=2).run(suite)
If you want a little more flexibility - and you might since you are creating suites and using test runners - I recommend that you take a look at Twisted Trial. It extends Python's unittest module and provides a few more assertions and reporting features.
Writing your tests will be exactly the same (besides subclassing twisted.trial.unittest.TestCase vs python's unittest) so your workflow won't change. You can still use your TestLoader but you'll have the options of many more TestReporters http://twistedmatrix.com/documents/11.1.0/api/twisted.trial.reporter.html.
For example, the default TestReporter is TreeReporter which returns the following output: