I would like to run a function at the end of all the tests.
A kind of global teardown function.
I found an example here and some clues here but it doesn't match my need. It runs the function at the beginning of the tests. I also saw the function pytest_runtest_teardown(), but it is called after each test.
Plus: if the function could be called only if all the tests passed, it would be great.
I found:
def pytest_sessionfinish(session, exitstatus):
""" whole test run finishes. """
exitstatus can be used to define which action to run. pytest docs about this
To run a function at the end of all the tests, use a pytest fixture with a "session" scope. Here is an example:
#pytest.fixture(scope="session", autouse=True)
def cleanup(request):
"""Cleanup a testing directory once we are finished."""
def remove_test_dir():
shutil.rmtree(TESTING_DIR)
request.addfinalizer(remove_test_dir)
The #pytest.fixture(scope="session", autouse=True) bit adds a pytest fixture which will run once every test session (which gets run every time you use pytest). The autouse=True tells pytest to run this fixture automatically (without being called anywhere else).
Within the cleanup function, we define the remove_test_dir and use the request.addfinalizer(remove_test_dir) line to tell pytest to run the remove_test_dir function once it is done (because we set the scope to "session", this will run once the entire testing session is done).
You can use the atexit module. For instance, if you want to report something at the end of all the test you need to add a report funtion as follows:
def report(report_dict=report_dict):
print("THIS IS AFTER TEST...")
for k, v in report_dict.items():
print(f"item for report: {k, v}")
then at the end of the module, you call atexit as follows:
atexit.register(report)
hoop this helps!
Related
I'm using Pytest to run test on one file that goes like this:
def function():...
def test_function():...
def main():...
main()
I want Pytest to exclude the line that calls main, is it possible?
Pytest will execute only those functions with test as a prefix. Since you have already have a function def function() at the start there can be some issues running the entire test case.
As #hoefling commented, pytest can not skip the code.
Try adding the following at the end of you code:
if __name__=='__main__':
main()
I would like to perform some sanity check when I run tests using pytest. Typically, I want to check that some executables are accessible to the tests, and that the options provided by the user on the command-line are valid.
The closest thing I found was to use a fixture such as:
#pytest.fixture(scope="session", autouse=True)
def sanity_check(request):
if not good:
sys.exit(0)
But this still runs all the tests. I'd like for the script to fail before attempting to run the tests.
You shouldn't need to validate the command line options explicitly; this will be done by the arg parser which will abort the execution early if necessary. As for conditions checking, you are not far from the solution. Use
pytest.exit to to an immediate abort
pytest.skip to skip all tests
pytest.xfail to fail all tests (this is an expected failure though, so it won't mark the whole execution as failed)
Example fixture:
#pytest.fixture(scope='session', autouse=True)
def precondition():
if not shutil.which('spam'):
# immediate shutdown
pytest.exit('Install spam before running this test suite.')
# or skip each test
# pytest.skip('Install spam before running this test suite.')
# or make it an expected failure
# pytest.xfail('Install spam before running this test suite.')
xdist compatibility
Invoking pytest.exit() in the test run with xdist will only crash the current worker and will not abort the main process. You have to move the check to a hook that is invoked before the runtestloop starts (so anything before the pytest_runtestloop hook), for example:
# conftest.py
def pytest_sessionstart(session):
if not shutil.which('spam'):
# immediate shutdown
pytest.exit('Install spam before running this test suite.')
If you want to run sanity check before whole test scenario then you can use conftest.py file - https://docs.pytest.org/en/2.7.3/plugins.html?highlight=re
Just add your function with the same scope and autouse option to conftest.py:
#pytest.fixture(scope="session", autouse=True)
def sanity_check(request):
if not good:
pytest.exit("Error message here")
I have the testcases structured in the following way.
app
test_login.py
class1
test_11
test_12
test_function1.py
class2
test_21
test_22
When I run "pytest.exe app", pytest is able to identify all the testcases, but it executes in random order. For example, test11, test22, test12 and so on
Is there any way I can change this and execute all testcases in a file::class first and then move on to another file::class?
it executes in random order
By default, tests are sorted by modules; inside the modules, tests are executed in the order they are specified. So you should get the rough order like this:
$ pytest --collect-only -q
test_function1.py::class2::test_21
test_function1.py::class2::test_22
test_login.py::class1::test_11
test_login.py::class1::test_12
Is there any way I can change this and execute all testcases in a file::class first and then move on to another file::class?
If you want to change the default execution order, you can do it in the pytest_collection_modifyitems hook. Example that reorders the collected tests by class name, then by test name:
# conftest.py
import operator
def pytest_collection_modifyitems(items):
items.sort(key=operator.attrgetter('cls.__name__', 'name'))
Now tests in test_login will be executed before those in test_function1 because the module names are not counted in the ordering anymore:
$ pytest --collect-only -q
test_login.py::class1::test_11
test_login.py::class1::test_12
test_function1.py::class2::test_21
test_function1.py::class2::test_22
Following code block solved my issue. Thanks #hoefling for your time and help.
# conftest.py
#pytest.hookimpl(hookwrapper=True)
def pytest_collection_modifyitems(items):
yield
items.sort(key=operator.attrgetter('cls.__name__'))
I have unit tests that runs with py.test for python 2.7 and py.test 3.0. My test directory is like this:
tests
---dir1
test1.py
-------sub-dir1-1
test-1-1.py
-------sub-dir1-2
test-1-2.py
---dir2
test2.py
-------sub-dir2-1
test-2-1.py
-------sub-dir2-2
test-2-2.py
I want all my tests to run a common setup and tear down function before and after test. I would like to do it with the least modification of all the test code.
Thanks
If I understand correctly, you can write a fixture that is session scoped (scope param) and make it used automatically (autouse param). I used a "yield fixture" as an example. Please note, that pytest.yield_fixture is deprecated since pytest 3.0 and pytest.fixture allows use of yield.
import pytest
#pytest.fixture(scope="session", autouse=True)
def callattr_ahead_of_alltests(request):
print 'run_pre_start'
yield
print 'run_after_finish'
It will run before first test (printing "run_pre_start") and part after yield will run after all the tests (printing "run_after_finish").
I am using unittest to test my Flask application, and nose to actually run the tests.
My first set of tests is to ensure the testing environment is clean and prevent running the tests on the Flask app's configured database. I'm confident that I've set up the test environment cleanly, but I'd like some assurance of that without running all the tests.
import unittest
class MyTestCase(unittest.TestCase):
def setUp(self):
# set some stuff up
pass
def tearDown(self):
# do the teardown
pass
class TestEnvironmentTest(MyTestCase):
def test_environment_is_clean(self):
# A failing test
assert 0 == 1
class SomeOtherTest(MyTestCase):
def test_foo(self):
# A passing test
assert 1 == 1
I'd like the TestEnvironmentTest to cause unittest or noseto bail if it fails, and prevent SomeOtherTest and any further tests from running. Is there some built-in method of doing so in either unittest (preferred) or nose that allows for that?
In order to get one test to execute first and only halt execution of the other tests in case of an error with that test, you'll need to put a call to the test in setUp() (because python does not guarantee test order) and then fail or skip the rest on failure.
I like skipTest() because it actually doesn't run the other tests whereas raising an exception seems to still attempt to run the tests.
def setUp(self):
# set some stuff up
self.environment_is_clean()
def environment_is_clean(self):
try:
# A failing test
assert 0 == 1
except AssertionError:
self.skipTest("Test environment is not clean!")
For your use case there's setUpModule() function:
If an exception is raised in a setUpModule then none of the tests in
the module will be run and the tearDownModule will not be run. If
the exception is a SkipTest exception then the module will be
reported as having been skipped instead of as an error.
Test your environment inside this function.
You can skip entire test cases by calling skipTest() in setUp(). This is a new feature in Python 2.7. Instead of failing the tests, it will simply skip them all.
I'm not quite sure whether it fits your needs, but you can make the execution of a second suite of unittests conditional on the result of a first suite of unittests:
envsuite = unittest.TestSuite()
moretests = unittest.TestSuite()
# fill suites with test cases ...
envresult = unittest.TextTestRunner().run(envsuite)
if envresult.wasSuccessful():
unittest.TextTestRunner().run(moretests)