We Keep Coding

How to write clean tests on model with database access

I'm using SQLAlchemy + Ormar and I want to write clean tests as it possible to write with pytest-django:
import pytest
#pytest.mark.django_db
def test_user_count():
assert User.objects.count() == 0
I'm using FastAPI and not using Django at all so decorator as above isn't possible to use.
How to write clean tests on model with Database access as above but not with Django. It would be great to have that infrastructure for SQLAlchemy + Ormar but changing ORM is an option too.
Example of model to test:
class User(ormar.Model):
class Meta:
metadata = metadata
database = database
id: int = ormar.BigInteger(primary_key=True)
phone: str = ormar.String(max_length=100)
account: str = ormar.String(max_length=100)

I think this discussion can be useful for you https://github.com/collerek/ormar/discussions/136
Using an autouse fixture should help you:
# fixture
#pytest.fixture(autouse=True, scope="module") # adjust your scope
def create_test_database():
engine = sqlalchemy.create_engine(DATABASE_URL)
metadata.drop_all(engine) # i like to drop also before - even if test crash in the middle we start clean
metadata.create_all(engine)
yield
metadata.drop_all(engine)
# actual test - note to test async you need pytest-asyncio and mark test as asyncio
#pytest.mark.asyncio
async def test_actual_logic():
async with database: # <= note this is the same database that used in ormar Models
... (logic)

This is what I use for my standalone script (a notebook) in the root directory of the project, where manage.py resides;
import sys, os, django
# append your project to your path
sys.path.append("./<your-project>")
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "<your-project>.settings")
os.environ["DJANGO_ALLOW_ASYNC_UNSAFE"] = "true" # for notebooks only
django.setup()
# import the model
from listings.models import Listing
However, it should be noted that Django comes with it's own unit testing. Have a look here. This will enable you to run tests with python3 manage.py test <your-test>.

There is a bit of magic happening here (but this is generally true within pytest). The #pytest.mark.django_db fixture simply marks the test, but doesn't do much else on its own. The heavy lifting happens later on inside of pytest-django, where the plugin will filter/scan for tests with that mark and add appropriate fixtures to them.
We can replicate this behavior:
# conftest.py (or inside a dedicated plugin, if you fancy)
import pytest
# register the custom marker called my_orm
def pytest_configure(config):
config.addinivalue_line(
"markers", "my_orm: This test uses my ORM to connect to my DB."
)
#pytest.fixture()
def setup_my_orm():
print("TODO: set up DB and connect.")
yield
print("TODO: tear down DB and disconnect.")
# this is where the magic happens
def pytest_runtest_setup(item):
needs_my_orm = len([marker for marker in item.iter_markers(name="my_orm")]) > 0
if needs_my_orm and "setup_my_orm" not in item.fixturenames:
item.fixturenames.append("setup_my_orm")
# test_mymodule.py
#pytest.mark.my_orm
def test_foo():
assert 0 == 0
You can check that the test indeed prints the above TODO statements via pytest -s.
Of course, you can customize this further using parameters for the marker, more sophisticated fixture scoping, etc. This should, however, put you on the right track :)

Pytest fixture inside class to execute only once

I need to create a class that uses a fixture from conftest.py, and this class has a fixture that I need to use only once per test session.
I have two test classes that depend on this class that has a fixture.
Sample code in test_app.py:
#pytest.mark.usefixtures("driver_get")
class TestBase:
#pytest.fixture(scope="module", autouse=True)
def set_up(self):
# set up code.
# web page will load if this fixture is called
# uses self.driver, where driver was set in driver_get
class TestOne(TestBase):
def test_1(self):
# test code
# uses self.driver also in the test
class TestTwo(TestBase):
def test_2(self):
# test code
# uses self.driver also in the test
Sample code in conftest.py (follows https://dzone.com/articles/improve-your-selenium-webdriver-tests-with-pytest):
#pytest.fixture(scope="session")
def driver_get(request):
from selenium import webdriver
web_driver = webdriver.Chrome()
session = request.node
for item in session.items:
cls = item.getparent(pytest.Class)
setattr(cls.obj,"driver",web_driver)
yield
web_driver.close()
As you can see, conftest.py sets the driver as a class attribute, which is why I am applying the driver_get fixture to class TestBase, since I need to use the driver inside the class.
The problem is, once TestOne finishes, the web page will load again, and execute TestTwo, which means that the fixture set_up was executed again, which is not what I want (since I set set_up scope to module, so it should only really happen once).
I know there is a similar question asked here (py.test method to be executed only once per run), but the asker didn't have the constraint of needing TestBase to have a fixture applied to it as well.
I have thought of putting the fixture inside conftest.py, but I am not sure if it is possible given my constraint of the fixture needing to be inside a class, and executed only once.
Any help will be appreciated. Thank you!

In your code, your set_up fixture which is module scope is called before your TestBase resolves the driver_get fixture. Because of this, at set_up trying self.driver will raise an AttributeError: object has no attribute 'driver'.
One quick way to fix this in your example code is to refer to the driver_get fixture in your module set_up fixture like so:
class TestBase:
#pytest.fixture(scope="module", autouse=True)
def set_up(self, driver_get): # <-------------
self.driver.get("https://www.yahoo.com")
Another way to refer to fixtures is just by including the fixture name as an argument.
Personally I am not a huge fan of the approach you copied from that blog of setting a class attribute on the request node. You'd get an IDE warning about referring to self.driver. To me, it would be clearer to yield the driver out of driver_get and then within the test classes either set it to self in a setup style fixture or use it directly. Similar to something below.
#pytest.fixture(scope="session")
def driver_get():
from selenium import webdriver
web_driver = webdriver.Chrome()
yield web_driver
web_driver.close()
class TestClass:
#pytest.fixture(autouse=True)
def setup(self, driver_get):
self.driver = driver_get
def test_something(self):
self.driver.get("https://www.google.com")
But depending on what else needs to be setup, if you want to control it happening just once per session or module you would need to modify the approach a bit.

How to dynamically add new fixtures to a test based on the fixture signature of a test

So what I would like to achieve is mocking functions in various modules automatically with pytest. So I defined this in my conftest.py:
import sys
import __builtin__
from itertools import chain
# Fixture factory magic START
NORMAL_MOCKS = [
"logger", "error", "logging", "base_error", "partial"]
BUILTIN_MOCKS = ["exit"]
def _mock_factory(name, builtin):
def _mock(monkeypatch, request):
module = __builtin__ if builtin else request.node.module.MODULE
ret = Mock()
monkeypatch.setattr(module, name, ret)
return ret
return _mock
iterable = chain(
((el, False) for el in NORMAL_MOCKS),
((el, True) for el in BUILTIN_MOCKS))
for name, builtin in iterable:
fname = "mock_{name}".format(name=name)
_tmp_fn = pytest.fixture(name=fname)(_mock_factory(name, builtin))
_tmp_fn.__name__ = fname
setattr(
sys.modules[__name__],
"mock_{name}".format(name=name), _tmp_fn)
# Fixture normal factory magic END
This works and all, but I would like to omit the usage of the NORMAL_MOCKS and BUILTIN_MOCKS lists. So basically in a pytest hook I should be able to see that say there is a mock_foo fixture, but it's not registered yet, so I create a mock for it with the factory and register it. I just couldn't figure out how to do this. Basically I was looking into the pytest_runtest_setup function, but could not figure out how to do the actual fixture registration. So basically I would like to know with which hook/call can I register new fixture functions programatically from this hook.

One of the ways is to parameterize the tests at the collection/generation stage, i.e. before the test execution begins: https://docs.pytest.org/en/latest/example/parametrize.html
# conftest.py
import pytest
def mock_factory(name):
return name
def pytest_generate_tests(metafunc):
for name in metafunc.fixturenames:
if name.startswith('mock_'):
metafunc.parametrize(name, [mock_factory(name[5:])])
# test_me.py
def test_me(request, mock_it):
print(mock_it)
A very simple solution. But the downside is that the test is reported as parametrized when it actually is not:
$ pytest -s -v -ra
====== test session starts ======
test_me.py::test_me[it] PASSED
====== 1 passed in 0.01 seconds ======
To fully simulate the function args without the parametrization, you can make a less obvious trick:
# conftest.py
import pytest
def mock_factory(name):
return name
#pytest.hookimpl(hookwrapper=True)
def pytest_runtest_protocol(item, nextitem):
for name in item.fixturenames:
if name.startswith('mock_') and name not in item.funcargs:
item.funcargs[name] = mock_factory(name[5:])
yield
The pytest_runtest_setup hook is also a good place for this, as long as I've just tried.
Note that you do not register the fixture in that case. It is too late for the fixture registration, as all the fixtures are gathered and prepared much earlier at the collection/parametrization stages. In this stage, you can only execute the tests and provide the values. It is your responsibility to calculate the fixture values and to destroy them afterward.

The snippet below is a pragmatic solution to "how to dynamically add fixtures".
Disclaimer: I don't have expertise on pytest. I'm not saying this is what pytest was designed for, I just looked at the source code and came up with this and it seems to work. The fact that I use "private" attributes means it might not work with all versions (currently I'm on pytest 7.1.3)
from _pytest.fixtures import FixtureDef
from _pytest.fixtures import SubRequest
import pytest
#pytest.fixture(autouse=True) # autouse is relevant, as then the fixture registration happens in-time. It's too late if requiring the fixture without autouse e.g. like `#pytest.mark.usefixtures("add_fixture_dynamically")`
def add_fixture_dynamically(request: SubRequest):
"""
Conditionally and dynamically adds another fixture. It's conditional on the presence of:
#pytest.mark.my_mark()
"""
marker = request.node.get_closest_marker("my_mark")
# don't register fixture if marker is not present:
if marker is None:
return
def your_fixture(): # the name of the fixture must match the parameter name, like other fixtures
return "hello"
# register the fixture just-in-time
request._fixturemanager._arg2fixturedefs[your_fixture.__name__] = [
FixtureDef(
argname=your_fixture.__name__,
func=your_fixture,
scope="function",
fixturemanager=request._fixturemanager,
baseid=None,
params=None,
),
]
yield # runs the test. Could be wrapped in try/except/finally
# suppress warning (works if this and `add_fixture_dynamically` are in `conftest.py`)
def pytest_configure(config):
"""Prevents printing of the warning 'PytestUnknownMarkWarning: Unknown pytest.mark.<fixture_name>'"""
config.addinivalue_line("markers", "my_mark")
#pytest.mark.my_mark()
def test_adding_fixture_dynamically(your_fixture):
assert your_fixture == "hello"

Set up and tear down functions for pytest bdd

I'm trying to do setup and teardown modules using pytest-bdd.
I know with behave you can do a environment.py file with before_all and after_all modules. How do I do this in pytest-bdd
I have looked into "classic xunit-style setup" plugin and it didn't work when I tried it. (I know thats more related to py-test and not py-test bdd).

You could just declare a pytest.fixture with autouse=true and whatever scope you want. You can then use the request fixture to specify the teardown. E.g.:
#pytest.fixture(autouse=True, scope='module')
def setup(request):
# Setup code
def fin():
# Teardown code
request.addfinalizer(fin)

A simple-ish approach for me is to use a trivial fixture.
# This declaration can go in the project's confest.py:
#pytest.fixture
def context():
class Context(object):
pass
return Context()
#given('some given step')
def some_when_step(context):
context.uut = ...
#when('some when step')
def some_when_step(context):
context.result = context.uut...
Note: confest.py allows you to share fixtures between codes, and putting everything in one file gives me a static analysis warning anyway.

"pytest supports execution of fixture specific finalization code when the fixture goes out of scope. By using a yield statement instead of return, all the code after the yield statement serves as the teardown code:"
See: https://docs.pytest.org/en/latest/fixture.html
eg.
#pytest.fixture(scope="module")
def smtp_connection():
smtp_connection = smtplib.SMTP("smtp.gmail.com", 587, timeout=5)
yield smtp_connection # provide the fixture value
print("teardown smtp")
smtp_connection.close()

How to call setup once for all tests and teardown after all are finished

I have a bunch of tests written using pytest. There are all under a directory dir. For example:
dir/test_base.py
dir/test_something.py
dir/test_something2.py
...
The simplified version of code in them is as follows:
test_base.py
import pytest
class TestBase:
def setup_module(module):
assert False
def teardown_module(module):
assert False
test_something.py
import pytest
from test_base import TestBase
class TestSomething(TestBase):
def test_dummy():
pass
test_something2.py
import pytest
from test_base import TestBase
class TestSomethingElse(TestBase):
def test_dummy2():
pass
All my test_something*.py files extend the base class in test_base.py. Now I wrote setup_module(module) and teardown_module(module) methods in test_base.py. I was expecting the setup_module to be called once for all tests, and teardown_module() to be called at the end, once all tests are finished.
But the functions don’t seem to be getting called? Do I need some decorators for this to work?

The OP's requirement was for setup and teardown each to execute only once, not one time per module. This can be accomplished with a combination of a conftest.py file, #pytest.fixture(scope="session") and passing the fixture name to each test function.
These are described in the Pytest fixtures documentation
Here's an example:
conftest.py
import pytest
#pytest.fixture(scope="session")
def my_setup(request):
print '\nDoing setup'
def fin():
print ("\nDoing teardown")
request.addfinalizer(fin)
test_something.py
def test_dummy(my_setup):
print '\ntest_dummy'
test_something2.py
def test_dummy2(my_setup):
print '\ntest_dummy2'
def test_dummy3(my_setup):
print '\ntest_dummy3'
The output when you run py.test -s:
collected 3 items
test_something.py
Doing setup
test_dummy
.
test_something2.py
test_dummy2
.
test_dummy3
.
Doing teardown
The name conftest.py matters: you can't give this file a different name and expect Pytest to find it as a source of fixtures.
Setting scope="session" is important. Otherwise setup and teardown will be repeated for each test module.
If you'd prefer not to pass the fixture name my_setup as an argument to each test function, you can place test functions inside a class and apply the pytest.mark.usefixtures decorator to the class.

Put setup_module and teardown_module outside of a class on module level. Then add your class with your tests.
def setup_module(module):
"""..."""
def teardown_module(module):
"""..."""
class TestSomething:
def test_dummy(self):
"""do some tests"""
For more info refer to this article.

setup_module/teardown_module are called for the module where the eventual (derived) tests are defined. This also allows to customize the setup. If you only ever have one setup_module you can put it to test_base.py and import it from the other places. HTH.

First of all it is a good practice to put all tests in a module called "tests":
<product>
...
tests/
__init__.py
test_something.py
Secondly I think you should use setup_class/teardown_class methods in your base class:
import unittest
class MyBase(unittest.TestCase):
#classmethod
def setup_class(cls):
...
#classmethod
def teardown_class(cls):
...
More info: http://pytest.org/latest/xunit_setup.html