How to patch my custom decorator with monkeypatch or pytest.mock? i manage to mock it by doing (answer to this question):
package.decorator = mytestdecorator
The problem is that it breaks some other tests where i actually need that decorator to work.
You have to control complete lifecycle of your mocked decorator and revert the decorator back to original state.
It can be done in few different ways:
context manager which builds the mocked decorator, and reverts it back by __exit__.
setup and teardown functions for your test, teardown must revert the decorator.
pytest fixture with finalizer
pytest fixture with yield expression.
Personally I like the #pytest.yield_fixture as it keeps the code short and as soon as you realize, that all what comes after yield statement in the fixture code is cleanup code, things are very clear to do.
Related
I have an asynchronous library asynpg which makes requests to an external database.
I want to have a test which mocks syncpg.connect to simulate a database connection, even when there is none:
#pytest.fixture
def mock_asyncpg(monkeypatch, mock_data):
""" Patch asyncpg.connect """
monkeypatch.setattr(asyncpg, "connect", mock_asyncio_connect(mock_data))
If I want to test using the mock I can include it:
#pytest.mark.asyncio
async def test_with_mock(mock_asyncpg):
But I'd also like to test with the real, unmocked, version of asyncpg, but asyncpg has been monkey-patched everywhere, and all tests see the monkey-patched version.
I assume this is happening because my tests are running asynchronously so they all see the same version of asyncpg
I see that there is a library for asynchronous monkey patching, but I wonder if there's a better pattern I should be using. Maybe something to do with CoroutineMock?
I’m developing database based flask application (using flask-sqlalchemy). I use fixtures to define individual pieces of test data.
#pytest.fixture(scope='class')
def model_a(db):
a = ModelA()
db.session.add(a)
db.session.commit()
return a
#pytest.fixture(scope='class')
def model_b(db, a):
b = ModelB(a=a)
db.session.add(b)
db.session.commit()
return b
# …
While it works to call db.session.commit() for each and every test object it would be more efficient to call it only once right before executing the actual tests.
Is there a way to run db.session.commit() before every test, after all fixtures are loaded, but only if the test directly or indirectly needs db?
Things that I don’t think that they will work:
A python_runtest_setup-hook doesn’t seem to be able to access fixtures or to determine whether the db fixture is loaded for/required by the test.
A autouse fixture would need to depend on db and thus make all tests use the db fixture. Also I couldn’t find a way to make it executed last.
You can't specify fixtures ordering other than indirectly (fixtures depending on other fixtures), see discussion in issue #1216. You can access both fixture names and fixture values in hooks though, so using a hook is actually a good idea. However, pytest_runtest_setup is too early for all fixtures to be executed; use pytest_pyfunc_call instead. Example:
from _pytest.fixtures import FixtureRequest
def pytest_pyfunc_call(pyfuncitem):
if 'db' in pyfuncitem.fixturenames:
db = FixtureRequest(pyfuncitem, _ispytest=True).getfixturevalue('db')
db.session.commit()
# ensure this hook returns None, or your underlying test function won't be executed.
Warning: pytest.FixtureRequest() is considered non-public by the pytest maintainers for anything except type-hinting. That’s why its use issues a deprecation warning without the _ispytest=True flag. The API might change without a major release. There is no public API yet that supports this use-case. (Last updated for pytest 6.2.5)
For an application I'm testing I'd like to create an autouse=True fixture which monkeypatches smtplib.SMTP.connect to fail tests if they try to send an email unexpectedly.
However, in cases where I do expect tests to send emails, I want to use a different fixture logging those emails instead (most likely by using the smtpserver fixture from pytest-localserver and monkeypatching the connect method to use the host/port returned by that fixture)
Of course that can only work if the autouse fixture is executed before the other fixture (loaded as funcarg). Is there any specific order in which fixtures are executed and/or is there a way to guarantee the execution order?
The easiest way to control the order in which fixtures are executed, is to just request the previous fixture in the later fixture. So to make sure b runs before a:
#pytest.fixture(autouse=True, scope="function")
def b():
pass
#pytest.fixture(scope="function")
def a(b):
pass
For details on the general fixture resolution order, see Maxim's excellent answer below or have a look at the documentation.
TL;DR
There are 3 aspects being considered together when building fixture evaluation order, aspects themselves are placed in order of priority:
Fixture dependencies - fixtures are evaluated from rootest required back to one required in test function.
Fixture scope - fixtures are evaluated from session through module to function scoped. On the same scope autouse fixtures are evaluated before non-autouse ones.
Fixture position in test function arguments - fixtures are evaluated in order of presence in test function arguments, from leftest to rightest.
Official explanation with code example by link below
https://docs.pytest.org/en/stable/fixture.html#fixture-instantiation-order
UPDATE
Current documentation mention 3 factors, as only considered, while discouraging from relying on another factors (like order in test function arguments):
scope
dependencies
autouse
Link above is outdated. See updated link
I was just having this problem with two function-scoped autouse fixtures. I wanted fixture b to run before fixture a, but every time, a ran first. I figured maybe it was alphabetical order, so I renamed a to c, and now b runs first. Pytest doesn't seem to have this documented. It was just a lucky guess. :-)
That's for autouse fixtures. Considering broader scopes (eg. module, session), a fixture is executed when pytest encounters a test that needs it. So if there are two tests, and the first test uses a session-scoped fixture named sb and not the one named sa, then sb will get executed first. When the next test runs, it will kick off sa, assuming it requires sa.
IIRC you can rely on higher scoped fixtures to be executed first. So if you created a session scoped autouse fixture to monkeypatch smtplib.SMTP.connect then you could create a function-scoped fixture which undoes this monkeypatching for one test, restoring it afterwards. I assume the easiest way to do this is create your own smtpserver fixture which depends on both the disallow_smtp fixture as well as the smtpserver fixture from pytest-localserver and then handles all setup and teardown required to make these two work together.
This is vaguely how pytest-django handles it's database access btw, you could try and look at the code there but it is far from a simple example and has many of it's own weird things.
With the help of below code we can easily set execution order of fixtures / functions
e.g:-
execution order first
#pytest.mark.order(1)
execution order second
#pytest.mark.order(2)
I'm developing a test suite using pytest for a project of mine. Because of the nature of the project, I need to create a Pytest plugin that controls how the tests are being run; they are not run locally, but sent to a different process to run. (I know about xdist but I think it doesn't solve my problem.)
I've been writing my own Pytest plugin by overriding the various pytest_runtest_* methods. So far it's been progressing well. Here is where I've hit a wall: I want my implementations of pytest_runtest_setup, pytest_runtest_call and pytest_runtest_teardown to actually be responsible for doing the setup, call and teardown. They're going to do it in a different process. My problem is: After Pytest calls my pytest_runtest_setup, it also calls all the other pytest_runtest_setup down the line of plugins. This is because the hook specification for pytest_runtest_setup has firstresult=False.
I don't want this, because I don't want pytest_runtest_setup to actually run on the current process. I want to be responsible for running it on my own. I want to override how it's being run, not add to it. I want the other implementations of pytest_runtest_setup below my own to not be run.
How can I do this?
Generic “runtest” hooks
All runtest related hooks receive a pytest.Item object.
pytest_runtest_protocol(item, nextitem)[source]
implements the runtest_setup/call/teardown protocol for the given test item, including capturing exceptions and calling reporting hooks.
Parameters:
item – test item for which the runtest protocol is performed.
nextitem – the scheduled-to-be-next test item (or None if this is the end my friend). This argument is passed on to pytest_runtest_teardown().
Return boolean:
True if no further hook implementations should be invoked.
pytest_runtest_setup(item)[source]
called before pytest_runtest_call(item).
pytest_runtest_call(item)[source]
called to execute the test item.
pytest_runtest_teardown(item, nextitem)[source]
called after pytest_runtest_call.
Parameters: nextitem – the scheduled-to-be-next test item (None if no further test item is scheduled). This argument can be used to perform exact teardowns, i.e. calling just enough finalizers so that nextitem only needs to call setup-functions.
pytest_runtest_makereport(item, call)[source]
return a _pytest.runner.TestReport object for the given pytest.Item and _pytest.runner.CallInfo.
For deeper understanding you may look at the default implementation of these hooks in _pytest.runner and maybe also in _pytest.pdb which interacts with _pytest.capture and its input/output capturing in order to immediately drop into interactive debugging when a test failure occurs.
The _pytest.terminal reported specifically uses the reporting hook to print information about a test run.
For an application I'm testing I'd like to create an autouse=True fixture which monkeypatches smtplib.SMTP.connect to fail tests if they try to send an email unexpectedly.
However, in cases where I do expect tests to send emails, I want to use a different fixture logging those emails instead (most likely by using the smtpserver fixture from pytest-localserver and monkeypatching the connect method to use the host/port returned by that fixture)
Of course that can only work if the autouse fixture is executed before the other fixture (loaded as funcarg). Is there any specific order in which fixtures are executed and/or is there a way to guarantee the execution order?
The easiest way to control the order in which fixtures are executed, is to just request the previous fixture in the later fixture. So to make sure b runs before a:
#pytest.fixture(autouse=True, scope="function")
def b():
pass
#pytest.fixture(scope="function")
def a(b):
pass
For details on the general fixture resolution order, see Maxim's excellent answer below or have a look at the documentation.
TL;DR
There are 3 aspects being considered together when building fixture evaluation order, aspects themselves are placed in order of priority:
Fixture dependencies - fixtures are evaluated from rootest required back to one required in test function.
Fixture scope - fixtures are evaluated from session through module to function scoped. On the same scope autouse fixtures are evaluated before non-autouse ones.
Fixture position in test function arguments - fixtures are evaluated in order of presence in test function arguments, from leftest to rightest.
Official explanation with code example by link below
https://docs.pytest.org/en/stable/fixture.html#fixture-instantiation-order
UPDATE
Current documentation mention 3 factors, as only considered, while discouraging from relying on another factors (like order in test function arguments):
scope
dependencies
autouse
Link above is outdated. See updated link
I was just having this problem with two function-scoped autouse fixtures. I wanted fixture b to run before fixture a, but every time, a ran first. I figured maybe it was alphabetical order, so I renamed a to c, and now b runs first. Pytest doesn't seem to have this documented. It was just a lucky guess. :-)
That's for autouse fixtures. Considering broader scopes (eg. module, session), a fixture is executed when pytest encounters a test that needs it. So if there are two tests, and the first test uses a session-scoped fixture named sb and not the one named sa, then sb will get executed first. When the next test runs, it will kick off sa, assuming it requires sa.
IIRC you can rely on higher scoped fixtures to be executed first. So if you created a session scoped autouse fixture to monkeypatch smtplib.SMTP.connect then you could create a function-scoped fixture which undoes this monkeypatching for one test, restoring it afterwards. I assume the easiest way to do this is create your own smtpserver fixture which depends on both the disallow_smtp fixture as well as the smtpserver fixture from pytest-localserver and then handles all setup and teardown required to make these two work together.
This is vaguely how pytest-django handles it's database access btw, you could try and look at the code there but it is far from a simple example and has many of it's own weird things.
With the help of below code we can easily set execution order of fixtures / functions
e.g:-
execution order first
#pytest.mark.order(1)
execution order second
#pytest.mark.order(2)