Is it possible to run tests from code using pytest? I did find pytest.main, but it's just a command line interface available from code. I would like to pass a test class / function from the code.
In unittest it's possible this way:
from unittest import TestLoader, TestCase, TestResult
class TestMy(TestCase):
def test_silly(self):
assert False
runner = TestLoader()
test_suite = runner.loadTestsFromTestCase(TestMy)
test_result = TestResult()
test_suite.run(test_result)
print(test_result)
Yes it's possible, that way for instance:
from pytest import main
class TestMy:
def test_silly(self):
assert False
main(['{}::{}'.format(__file__, TestMy.__name__)])
You can pass any argument to main as if called from command line.
Related
How can I get test_greet to run in the below; note: test_one(when uncommented) is seen and run by the test runner; to be specific, I want the line unittest.main() to correctly pick up the module level test (test_greet).
import unittest
#class MyTests(unittest.TestCase):
# def test_one(self):
# assert 1==2
def test_greet():
assert 1==3
if __name__=="__main__":
unittest.main()
Let's say i have a file called MyTests.py as below:
import unittest
class MyTests(unittest.TestCase):
def test_greet(self):
self.assertEqual(1,3)
Then:
Open a CMD in the folder that MyTests.py exists
Run python -m unittest MyTests
Please note that, all your tests must have the test_ otherwise, it will not be run.
How to pass custom arguments to fixture inside test method of unittest.TestCase derived class using pytest?
After searching for definitely too long time I managed to invent solution with usage of doc and threads about wrapping fixtures. I hope somebody will find it useful.
conftest.py
import pytest
#pytest.fixture()
def add(request):
def wrapped(a=10, b=5):
return a + b
request.cls.add = wrapped
add_test.py
import pytest
from unittest import TestCase
#pytest.mark.usefixtures('add')
class AddTestCase(TestCase):
def test_add(self):
# parameters can be passed inside test method
result = self.add(2, 2)
assert result == 4
I'm currently running my tests like this:
tests = unittest.TestLoader().discover('tests')
unittest.TextTestRunner().run(tests)
Now I want to run a specific test knowing his name (like test_valid_user) but not knowing his class. If there is more than one test with such name than I would like to run all such tests. Is there any way to filter tests after discover?
Or maybe there are other solutions to this problem (please note that it shouldn't be done from command line)?
You can use the unittest.loader.TestLoader.testMethodPrefix instance variable to change the test methods filter according to a different prefix than "test".
Say you have a tests directory with this king of unit tests:
import unittest
class MyTest(unittest.TestCase):
def test_suite_1(self):
self.assertFalse("test_suite_1")
def test_suite_2(self):
self.assertFalse("test_suite_2")
def test_other(self):
self.assertFalse("test_other")
You can write your own discover function to discover only test functions starting with "test_suite_", for instance:
import unittest
def run_suite():
loader = unittest.TestLoader()
loader.testMethodPrefix = "test_suite_"
suite = loader.discover("tests")
result = unittest.TestResult()
suite.run(result)
for test, info in result.failures:
print(info)
if __name__ == '__main__':
run_suite()
remark: the argument "tests" in the discover method is a directory path, so you may need to write a fullpath.
As a result, you'll get:
Traceback (most recent call last):
File "/path/to/tests/test_my_module.py", line 8, in test_suite_1
self.assertFalse("test_suite_1")
AssertionError: 'test_suite_1' is not false
Traceback (most recent call last):
File "/path/to/tests/test_my_module.py", line 11, in test_suite_2
self.assertFalse("test_suite_2")
AssertionError: 'test_suite_2' is not false
Another simpler way, would be to use py.test with the -k option which does a test name keyword scan. It will run any tests whose name matches the keyword expression.
Although that is using the command-line which you didn't want, please not that you can call the command-line from your code using subprocess.call to pass any arguments you want dynamically.
E.g.: Assuming you have the following tests:
def test_user_gets_saved(self): pass
def test_user_gets_deleted(self): pass
def test_user_can_cancel(self): pass
You can call py.test from cli:
$ py.test -k "test_user"
Or from code:
return_code = subprocess.call('py.test -k "test_user"', shell=True)
There are two ways to run a single test method:
Command line:
$ python -m unittest test_module.TestClass.test_method
Using Python script:
import unittest
class TestMyCode(unittest.TestCase):
def setUp(self):
pass
def test_1(self):
self.assertTrue(True)
def test_2(self):
self.assertTrue(True)
if __name__ == '__main__':
testSuite = unittest.TestSuite()
testSuite.addTest(TestMyCode('test_1'))
runner=unittest.TextTestRunner()
runner.run(testSuite)
Output:
------------------------------------------------------------
Ran 1 test in 0.000s
OK
import unittest
import HTMLTestRunner
class TestClass1(unittest.TestCase):
def setUp(self):
pass
def case1(self):
assert 4 == 3
def case2(self):
assert 4 == 4
def tearDown(self):
pass
class TestClass2(unittest.TestCase):
def setUp(self):
pass
def case3(self):
assert 1 == 2
def tearDown(self):
pass
def suite():
suite = unittest.TestSuite()
suite.addTest(TestClass1(['case1','case2']))
suite.addTest(TestClass2('case4'))
return suite
test_suite = suite()
unittest.TextTestRunner(verbosity=2).run(test_suite)
fp = file('my_report.html', 'wb')
runner = HTMLTestRunner.HTMLTestRunner(
stream=fp,
title='My unit test',
description='This demonstrates the report output by HTMLTestRunner.'
)
runner.run(test_suite)
I am trying to run all the methods in both the classes in a single run. However, the code above did not do so. In the suite function, I tried to add multiple tests from the classes but that also did not work and was giving an error.
From this answer at the question "Is test suite deprecated in PyUnit?":
"unittest.TestSuite is not necessary if you want to run all the tests in a single module as unittest.main() will dynamically examine the module it is called from and find all classes that derive from unittest.TestCase."
There's more in that answer about when unittest.TestSuite is useful.
That said, I needed to make some changes to get these tests to work. Firstly, unittest looks for functions with "test_" at their start. Also, unittest's assertEqual and similar methods should be used, instead of just Python's assert statement. Doing that and eliminating some unneeded code led to:
import unittest
class TestClass1(unittest.TestCase):
def test_case1(self):
self.assertEqual(4, 3)
def test_case2(self):
self.assertEqual(4, 4)
class TestClass2(unittest.TestCase):
def test_case3(self):
self.assertEqual(1, 2)
unittest.main()
This produced appropriate output (3 tests run with 2 failures), which I won't reproduce here in the interest of space.
I have a bunch of tests written using pytest. There are all under a directory dir. For example:
dir/test_base.py
dir/test_something.py
dir/test_something2.py
...
The simplified version of code in them is as follows:
test_base.py
import pytest
class TestBase:
def setup_module(module):
assert False
def teardown_module(module):
assert False
test_something.py
import pytest
from test_base import TestBase
class TestSomething(TestBase):
def test_dummy():
pass
test_something2.py
import pytest
from test_base import TestBase
class TestSomethingElse(TestBase):
def test_dummy2():
pass
All my test_something*.py files extend the base class in test_base.py. Now I wrote setup_module(module) and teardown_module(module) methods in test_base.py. I was expecting the setup_module to be called once for all tests, and teardown_module() to be called at the end, once all tests are finished.
But the functions don’t seem to be getting called? Do I need some decorators for this to work?
The OP's requirement was for setup and teardown each to execute only once, not one time per module. This can be accomplished with a combination of a conftest.py file, #pytest.fixture(scope="session") and passing the fixture name to each test function.
These are described in the Pytest fixtures documentation
Here's an example:
conftest.py
import pytest
#pytest.fixture(scope="session")
def my_setup(request):
print '\nDoing setup'
def fin():
print ("\nDoing teardown")
request.addfinalizer(fin)
test_something.py
def test_dummy(my_setup):
print '\ntest_dummy'
test_something2.py
def test_dummy2(my_setup):
print '\ntest_dummy2'
def test_dummy3(my_setup):
print '\ntest_dummy3'
The output when you run py.test -s:
collected 3 items
test_something.py
Doing setup
test_dummy
.
test_something2.py
test_dummy2
.
test_dummy3
.
Doing teardown
The name conftest.py matters: you can't give this file a different name and expect Pytest to find it as a source of fixtures.
Setting scope="session" is important. Otherwise setup and teardown will be repeated for each test module.
If you'd prefer not to pass the fixture name my_setup as an argument to each test function, you can place test functions inside a class and apply the pytest.mark.usefixtures decorator to the class.
Put setup_module and teardown_module outside of a class on module level. Then add your class with your tests.
def setup_module(module):
"""..."""
def teardown_module(module):
"""..."""
class TestSomething:
def test_dummy(self):
"""do some tests"""
For more info refer to this article.
setup_module/teardown_module are called for the module where the eventual (derived) tests are defined. This also allows to customize the setup. If you only ever have one setup_module you can put it to test_base.py and import it from the other places. HTH.
First of all it is a good practice to put all tests in a module called "tests":
<product>
...
tests/
__init__.py
test_something.py
Secondly I think you should use setup_class/teardown_class methods in your base class:
import unittest
class MyBase(unittest.TestCase):
#classmethod
def setup_class(cls):
...
#classmethod
def teardown_class(cls):
...
More info: http://pytest.org/latest/xunit_setup.html