I am working with a code base which which is mostly written with unittest. The tests are run with pytest. I am trying to simplify the tests by parameterizing some of them. To work towards this goal, I am trying to learn how to use pytest fixtures and refactor some of the existing unittest-based tests. The current tests have a global fixture in conftest.py:
#pytest.fixture(autouse=True)
def register_cleanup():
yield True
print("cleaning up database")
Now I want to add a fixture which is specific to one of my test modules, something along the lines of
#pytest.fixture()
def foo_fixture():
print("setup fixture")
yield
print("tear down fixture")
class Foo(unittest.TestCase):
def setUp(self):
print('unittest setUp()')
def test(self):
print('test')
However, the print() statements in this module fixture never execute. As an intermediate step, I am keeping the unittest-based structure and adding to it. Is it possible to get this working the way I want or do I need to scrap unittest altogether and go directly to pytest.
You'll want to mark it:
#pytest.mark.usefixtures("foo_fixture")
class Foo(unittest.TestCase):
...
Note: you may have more than one conftest.py. Put the more specific fixture into this test subdirectory's conftest.py.
Where would you expect this fixture to be used?
Did you want to turn on autouse=True here as well? In general, fixtures are only executed when they are a parameter to the test, and are less of generic cleanup mechanisms than scoped assumptions. By turning on autouse for this fixture you will essentially add an extra setup-cleanup step for the current module:
https://docs.pytest.org/en/latest/fixture.html#autouse-fixtures-xunit-setup-on-steroids
Here is how autouse fixtures work in other scopes:
autouse fixtures obey the scope= keyword-argument: if an autouse
fixture has scope='session' it will only be run once, no matter where
it is defined. scope='class' means it will be run once per class, etc.
if an autouse fixture is defined in a test module, all its test
functions automatically use it. if an autouse fixture is defined in a
conftest.py file then all tests in all test modules below its
directory will invoke the fixture. lastly, and please use that with
care: if you define an autouse fixture in a plugin, it will be invoked
for all tests in all projects where the plugin is installed. This can
be useful if a fixture only anyway works in the presence of certain
settings e. g. in the ini-file. Such a global fixture should always
quickly determine if it should do any work and avoid otherwise
expensive imports or computation.
[Adaptied comment into an answer]
Related
I know what pytest fixtures do and how they work, but they do things before and after the test. How do you do things inside the test in a composable fashion? I need to run tests composed of various smaller functions:
#pytest.mark.django_db
def my_example_test():
# Check if user does not exist
assert not Users.objects.filter(email='foo#bar.com').exists()
# Do a bunch of things to sign up and commit a user to the db
sign_up_routine()
# Check if user exists.
assert Users.objects.filter(email='foo#bar.com').exists()
# Checkout a shopping cart
checkout_shopping_cart(item="toothpaste", qty=10)
# do some more checks
...
Now, in my case, fixture doesn't work because it runs even before the test case starts. In general, I want to compose hundreds of tests like this:
Run a bunch of assert statements
Run a composable routine <--- how? function? fixture?
Assert more conditions
Run a different routine
What is a good way to structure composable tests like this in pytest? I am thinking of just writing a bunch of functions and give them database access?
I am sorry if this is just an obvious solution to run functions but I thought there was a pytest way to do this.
I think the short answer you're looking for is just use plain old functions! There's nothing wrong with that. If you want these reusable chunks of code to have access to other fixtures, just pass them through on invocation.
#pytest.mark.fixture
def db_session():
...
def create_some_users(session):
...
def my_test(db_session):
expected = ...
create_some_users(db_session)
actual = do_thing()
assert actual == expected
I like to think of tests with the AAA pattern - Arrange, Act, & Assert. First we get the universe in order, then we fire our cannon off at it, and finally we check to see that everything is how we'd expect it to be at the end. It's an ideal world if all tests are kept simple like this. Fixtures are pytest's way of managing and sharing sets of resources and the instructions to arrange them in some way. This is why they always run at start and -b/c we often want to do some related disposal afterwards- at end. And a nice side-effect is you can more explicitly state the dependencies of a given test within its declaration and can move common surrounding (beginning & end) "Arrange" code so that the test is more easily parsed as "do this X and expect Y".
For what you're looking for, you'd have to have some way to tell pytest when to run your reusable thing as it could be at any midpoint within the test function and at that point, mind as well just use a normal function. For example, you could write a fixture that returns a callable(function) and then just invoke the fixture with (..), but there's not a ton of difference. You could also have fixtures return classes and encapsulate reusable logic in that way. Any of these things would work fine.
I restructured the test with fixtures as follows. Instead of running one test with steps in a linear fashion, I read thoroughly through the fixtures documentation to end up with this:
#pytest.fixture(scope="function")
def sign_up_user(db)
# Check if user does not exist
assert not Users.objects.filter(email='foo#bar.com').exists()
# Do a bunch of things to sign up and commit a user to the db
# that were part of the sign_up_routine() here. Just showing an example below:
client = Client()
resp = client.post('url', kwargs={form:form})
#pytest.fixture(scope="function")
def assert_user_exists(db, sign_up_user):
# Check if user exists. You can imagine a lot more things to assert here, just example from my original post here.
assert Users.objects.filter(email='foo#bar.com').exists()
#pytest.fixture(scope="function")
def checkout_shopping_cart(db, assert_user_exists):
# Checkout shopping cart with 10 quantity of toothpaste
...
def test_order_in_database(db, checkout_shopping_cart):
# Check if order exists in the database
assert Orders.objects.filter(order__user__email='foo#bar.com').exists()
# This is the final test that calls all previous fixtures.
# Now, fixtures can be used to compose tests in various ways. For example, repeat 'sign_up_user' but checkout toothbrush instead of toothpaste.
I think this is pretty clean, not sure if it this is the intended way to use pytest. I welcome feedback. I can now compose smaller bits of tests that can run as fixtures by calling other fixtures in a long chain.
This is a toy example but you can imagine testing a lot of conditions in the database in each of these fixtures. Please note db fixture is needed for django-pytest package for database to work properly in fixtures. Otherwise, you'll get errors that won't be obvious ("use django_db mark" which doesn't fix the problem), see here: pytest django: unable to access db in fixture teardown
Also, the pytest.fixture scope must be function so each fixture runs again instead of caching.
More reading here: https://docs.pytest.org/en/6.2.x/fixture.html#running-multiple-assert-statements-safely
I want to create fixtures as library components.
A standard test database config is useful for several projects in different repos. It is currently copy/pasted into each independent project as they can't share a config.py.
I refactored the code into a pip installable library but can't work out an elegant way to use it in each project. This doesn't work:
import my_db_fixture
#pytest.fixture
def adapted_db_fixture(my_db_fixture):
# adapt the test setup
For the real code, the fixture I want to re-use is built from other fixtures. The best work-around I can find so far is to create a local conftest.py as copy/paste code but limited to importing functions and calling them in local fixture functions. I don't like copy/paste and unnecessarily exposes the inner workings of the fixtures.
It is possible to re-use fixtures from an installed library.
Define the fixtures as usual in the installable package. Then import them into a conftest.py local to the project. You need to import not just the fixture you want but also all fixtures it depends on and (if used) pytest_addoption
from my.package import (
the_fixture_i_want,
all_fixtures_it_uses,
pytest_addopt
)
I also discovered you can't un-decorate a library function with a teardown and call it in the local conftest.py:
# This doesn't work
# pip installed my_fixture.py
def my_fixture(dependencies)
# setup code
yield fixture_object
# teardown code
# local conftest.py
import pytest
import my_fixture # NB: the module
#pytest.fixture
def my_fixture(dependencies):
my_fixture.my_fixture()
# teardown code isn't called: pytest knows the function has no yield
# but doesn't realise it's returning a generator none the less
This article helped me:
peterhurford/pytest-fixture-modularization.md
I reckoned pytest should recognise something returning a generator as a generator so logged it as a bug. I imagine comments responding to it could be useful:
call_fixture_func should test the return value not the function
Lets assume I have:
pytest-3rd-party-plugin that provides a fixture cool_fixture
pytest-own-plugin that provides the fixture cool_fixture but with "better" functionality than cool_fixture
I want my own cool_fixture plugin to be the one that is selected. I know the following:
If I would have put that fixture in conftest.py, there wouldn't have been any problem, that would have been selected by the pytest runner. But I don't want to do that, as I am reusing this fixture in many projects.
I know that I can just call the fixture better_cool_fixture, and all will be fine :) But the fixture name is so pretty, so I kind of wanna keep it that way.
As I have understood pytest, there is no defined structure, what fixture that will be used. With exception of fixtures in module, conftest.py etc...
So anyway around this, so I don't need to change the name of this beautiful fixture?
I'm trying to understand what conftest.py files are meant to be used for.
In my (currently small) test suite I have one conftest.py file at the project root. I use it to define the fixtures that I inject into my tests.
I have two questions:
Is this the correct use of conftest.py? Does it have other uses?
Can I have more than one conftest.py file? When would I want to do that? Examples will be appreciated.
More generally, how would you define the purpose and correct use of conftest.py file(s) in a pytest test suite?
Is this the correct use of conftest.py?
Yes it is. Fixtures are a potential and common use of conftest.py. The
fixtures that you will define will be shared among all tests in your test suite. However, defining fixtures in the root conftest.py might be useless and it would slow down testing if such fixtures are not used by all tests.
Does it have other uses?
Yes it does.
Fixtures: Define fixtures for static data used by tests. This data can be accessed by all tests in the suite unless specified otherwise. This could be data as well as helpers of modules which will be passed to all tests.
External plugin loading: conftest.py is used to import external plugins or modules. By defining the following global variable, pytest will load the module and make it available for its test. Plugins are generally files defined in your project or other modules which might be needed in your tests. You can also load a set of predefined plugins as explained here.
pytest_plugins = "someapp.someplugin"
Hooks: You can specify hooks such as setup and teardown methods and much more to improve your tests. For a set of available hooks, read Hooks link. Example:
def pytest_runtest_setup(item):
""" called before ``pytest_runtest_call(item). """
#do some stuff`
Test root path: This is a bit of a hidden feature. By defining conftest.py in your root path, you will have pytest recognizing your application modules without specifying PYTHONPATH. In the background, py.test modifies your sys.path by including all submodules which are found from the root path.
Can I have more than one conftest.py file?
Yes you can and it is strongly recommended if your test structure is somewhat complex. conftest.py files have directory scope. Therefore, creating targeted fixtures and helpers is good practice.
When would I want to do that? Examples will be appreciated.
Several cases could fit:
Creating a set of tools or hooks for a particular group of tests.
root/mod/conftest.py
def pytest_runtest_setup(item):
print("I am mod")
#do some stuff
test root/mod2/test.py will NOT produce "I am mod"
Loading a set of fixtures for some tests but not for others.
root/mod/conftest.py
#pytest.fixture()
def fixture():
return "some stuff"
root/mod2/conftest.py
#pytest.fixture()
def fixture():
return "some other stuff"
root/mod2/test.py
def test(fixture):
print(fixture)
Will print "some other stuff".
Overriding hooks inherited from the root conftest.py.
root/mod/conftest.py
def pytest_runtest_setup(item):
print("I am mod")
#do some stuff
root/conftest.py
def pytest_runtest_setup(item):
print("I am root")
#do some stuff
By running any test inside root/mod, only "I am mod" is printed.
You can read more about conftest.py here.
EDIT:
What if I need plain-old helper functions to be called from a number
of tests in different modules - will they be available to me if I put
them in a conftest.py? Or should I simply put them in a helpers.py
module and import and use it in my test modules?
You can use conftest.py to define your helpers. However, you should follow common practice. Helpers can be used as fixtures at least in pytest. For example in my tests I have a mock redis helper which I inject into my tests this way.
root/helper/redis/redis.py
#pytest.fixture
def mock_redis():
return MockRedis()
root/tests/stuff/conftest.py
pytest_plugin="helper.redis.redis"
root/tests/stuff/test.py
def test(mock_redis):
print(mock_redis.get('stuff'))
This will be a test module that you can freely import in your tests. NOTE that you could potentially name redis.py as conftest.py if your module redis contains more tests. However, that practice is discouraged because of ambiguity.
If you want to use conftest.py, you can simply put that helper in your root conftest.py and inject it when needed.
root/tests/conftest.py
#pytest.fixture
def mock_redis():
return MockRedis()
root/tests/stuff/test.py
def test(mock_redis):
print(mock_redis.get(stuff))
Another thing you can do is to write an installable plugin. In that case your helper can be written anywhere but it needs to define an entry point to be installed in your and other potential test frameworks. See this.
If you don't want to use fixtures, you could of course define a simple helper and just use the plain old import wherever it is needed.
root/tests/helper/redis.py
class MockRedis():
# stuff
root/tests/stuff/test.py
from helper.redis import MockRedis
def test():
print(MockRedis().get(stuff))
However, here you might have problems with the path since the module is not in a child folder of the test. You should be able to overcome this (not tested) by adding an __init__.py to your helper
root/tests/helper/init.py
from .redis import MockRedis
Or simply adding the helper module to your PYTHONPATH.
In a wide meaning conftest.py is a local per-directory plugin. Here you define directory-specific hooks and fixtures. In my case a have a root directory containing project specific tests directories. Some common magic is stationed in 'root' conftest.py. Project specific - in their own ones. Can't see anything bad in storing fixtures in conftest.py unless they are not used widely (In that case I prefer to define them in test files directly)
I use the conftest.py file to define the fixtures that I inject into my tests, is this the correct use of conftest.py?
Yes, a fixture is usually used to get data ready for multiple tests.
Does it have other uses?
Yes, a fixture is a function that is run by pytest before, and sometimes
after, the actual test functions. The code in the fixture can do whatever you
want it to. For instance, a fixture can be used to get a data set for the tests to work on, or a fixture can also be used to get a system into a known state before running a test.
Can I have more than one conftest.py file? When would I want to do that?
First, it is possible to put fixtures into individual test files. However, to share fixtures among multiple test files, you need to use a conftest.py file somewhere centrally located for all of the tests. Fixtures can be shared by any test. They can be put in individual test files if you want the fixture to only be used by tests in that file.
Second, yes, you can have other conftest.py files in subdirectories of the top tests directory. If you do, fixtures defined in these lower-level conftest.py files will be available to tests in that directory and subdirectories.
Finally, putting fixtures in the conftest.py file at the test root will make them available in all test files.
Here are the official docs about using conftest.py to share fixtures:
conftest.py: sharing fixtures across multiple files
The conftest.py file serves as a means of providing fixtures for an entire directory. Fixtures defined in a conftest.py can be used by any test in that package without needing to import them (pytest will automatically discover them).
You can have multiple nested directories/packages containing your tests, and each directory can have its own conftest.py with its own fixtures, adding on to the ones provided by the conftest.py files in parent directories.
Ok, as Google search isn't helping me in a while (even when using the correct keywords).
I have a class extending from TestCase in which I want to have some auxiliary methods that are not going to be executed as part of the test, they'll be used to generate some mocked objects, etc, auxiliary things for almost any test.
I know I could use the #skip decorator so unittest doesn't run a particular test method, but I think that's an ugly hack to use for my purpose, any tips?
Thanks in advance, community :D
I believe that you don't have to do anything. Your helper methods should just not start with test_.
The only methods that unittest will execute [1] are setUp, anything that starts with test, and tearDown [2], in that order. You can make helper methods and call them anything except for those three things, and they will not be executed by unittest.
You can think of setUp as __init__: if you're generating mock objects that are used by multiple tests, create them in setUp.
def setUp(self):
self.mock_obj = MockObj()
[1]: This is not entirely true, but these are the main 3 groups of methods that you can concentrate on when writing tests.
[2]: For legacy reasons, unittest will execute both test_foo and testFoo, but test_foo is the preferred style these days. setUp and tearDown should appear as such.
The test runner will only directly execute methods beginning with test, so just make sure your helper methods' names don't begin with test.