I need to import some functions locally within my tests (yes, the code base can be designed better to avoid this necessity, but let's assume we can't do that).
That means the first line of all my tests within a module looks like in this example:
def test_something():
from worker import process_message
process_message()
Now I wanted to make this more DRY by creating the following fixture:
#pytest.fixture(scope="module", autouse=True)
def process_message():
from worker import process_message
return process_message
But I always get the error
Fixture "process_message" called directly. Fixtures are not meant to
be called directly, but are created automatically when test functions
request them as parameters. See
https://docs.pytest.org/en/stable/explanation/fixtures.html for more
information about fixtures, and
https://docs.pytest.org/en/stable/deprecations.html#calling-fixtures-directly
about how to update your code.
The linked documentation doesn't help me much.
How can I achieve what I want? I'd like to return the function handle obviously.
The reason this happens is that you are calling the fixture directly from one of your tests. I assume that your test code with the fixture looks something like this:
import pytest
#pytest.fixture(scope="module", autouse=True)
def process_message():
from worker import process_message
return process_message
def test_something():
process_message()
And then test_something fails with the specified exception.
The way to fix it is to add process_message as an argument to the test function, indicating that you are using it as a fixture:
def test_something(process_message):
process_message()
btw, since you have to specify process_message fixture in every test in order to call it, means there is no point in using the autouse=True and it can be removed.
Related
I am following this mini-tutorial/blog on pytest-mock. I can not understand how the mocker is working since there is no import for it - in particular the function declaration def test_mocking_constant_a(mocker):
import mock_examples.functions
from mock_examples.functions import double
def test_mocking_constant_a(mocker):
mocker.patch.object(mock_examples.functions, 'CONSTANT_A', 2)
expected = 4
actual = double() # now it returns 4, not 2
assert expected == actual
Somehow the mocker has the attributes/functions of pytest-mocker.mocker: in particular mocker.patch.object . But how can that be without the import statement?
The mocker variable is a Pytest fixture. Rather than using imports, fixtures are supplied using dependency injection - that is, Pytest takes care of creating the mocker object for you and supplies it to the test function when it runs the test.
Pytest-mock defines the "mocker" fixture here, using the Pytest fixture decorator. Here, the fixture decorator is used as a regular function, which is a slightly unusual way of doing it. A more typical way of using the fixture decorator would look something like this:
#pytest.fixture()
def mocker(pytestconfig: Any) -> Generator[MockerFixture, None, None]:
"""
Return an object that has the same interface to the `mock` module, but
takes care of automatically undoing all patches after each test method.
"""
result = MockerFixture(pytestconfig)
yield result
result.stopall()
The fixture decorator registers the "mocker" function with Pytest, and when Pytest runs a test with a parameter called "mocker", it inserts the result of the "mocker" function for you.
Pytest can do this because it uses Python's introspection features to view the list of arguments, complete with names, before calling the test function. It compares the names of the arguments with names of fixtures that have been registered, and if the names match, it supplies the corresponding object to that parameter of the test function.
I have the following code:
import asyncio
import pytest
from mymodule import myasyncfunction
from unittest import TestCase
class TestDummy(TestCase):
def setUp(self):
await myasyncfunction()
#pytest.mark.asyncio
async def test_dummy(self):
assert False
The test passes because it doesn't enter the test at all. It only says:
RuntimeWarning: coroutine 'TestDummy.setUp' was never awaited
How to make the setUp function async?
Observation: If I remove the inheritance from TestCase the test runs but it won't enter the setUp function before, which is needed.
The solution is to define a method as a fixture instead of using the traditional setUp() method.
import pytest
class TestClass:
#pytest.fixture
def setup(self):
pass
#pytest.mark.asyncio
async def test_some_stuff(setup):
pass
As you discovered, with pytest-asyncio the setUp() method doesn't work when a class inherits from Unitest.Testcase:
TestPersonClass is not a child class of unittest.TestCase. If it was, the test would still succeed – but the success would be a false positive because code after the await expression would not run.
Why is this happening? The answer is complex enough that it deserves a
separate post, but the tl;dr version is that on line 93 of
pytest-asyncio’s source the author is expecting the event loop to be
passed into the test from a pytest fixture, while unittest.TestCase
methods cannot directly receive fixture function arguments.
For the above explanation see end of this blog post:
https://jacobbridges.github.io/post/unit-testing-with-asyncio/
For some decent tutorials on testing with pytest-asyncio see:
1)https://www.roguelynn.com/words/asyncio-testing/
2)https://medium.com/ideas-at-igenius/testing-asyncio-python-code-with-pytest-a2f3628f82bc
I encountered this issue today. In my case, it was an easy fix using this pattern:
class TestDummy:
def setup_class(self) -> None:
asyncio.run(do_server_registration_setup_stuff())
# other sync setup logic...
#pytest.mark.asyncio
async def test_some_stuff(setup):
pass
This may not suit your needs but for me, I am doing functional testing and so just needed to use aiohttp to run some http POST requests to register account settings etc. before the tests are run...
If you need to pre-populate some in-memory caches on the TestDummy instance you could pass self to do_server_registration_setup_stuff and set instance attributes in there.
There are probably better ways, but maybe this can help somebody to just solve the immediate problem and keep it moving!
in the case of unit testing a wrapper library, testing the wrapper without depending/exercising the upstream library is a goal; In a known case, all calls to the upstream library can be mocked and that's what I've done, but I've been frustrated by changes to the wrapper that introduce more calls to the upstream library being missed by the mock tools;
How can I best fail any test that tries to use a given namespace?
My idea currently is to change all the unittest methods to have a monkey patch like
#unittest.mock.patch('wrapper_namespace.upsteam_namespace')
and reply the upstream library with a mock that can be asserted untouched; I'm hoping for an option that works globally, so that I
don't have to add a monkeypatch to every test method, though this level of granularity is acceptable; but also don't have to perform the assertion that the mock was never used in the test methods (or make a decorator to do all that either)
prohibits access to the upstream library from any part of the software
(e.g, Wrapper calls B calls Upstream, B's call to upstream might not be caught)
You don't have to patch every test method. You can easily patch over the class if you're using unittest, or just assign the module to whatever you want to patch over it with. Here's a workable example:
A fake lib in some_lib.py:
def some_lib_func():
raise ValueError("I've been called.")
def some_other_lib_func():
raise ValueError("I've been called.")
class SomeClass:
def __init__(self):
raise ValueError("I've been constructed.")
wrapper.py:
import some_lib
def wrapper1():
some_lib.some_lib_func()
def wrapper2():
some_lib.some_other_lib_func()
def wrapper3():
x = some_lib.SomeClass()
test.py:
from unittest.mock import patch, MagicMock
import unittest
import wrapper
# Alternative:
# wrapper.some_lib = MagicMock()
# Can patch an entire class
#patch('wrapper.some_lib', MagicMock())
class TestWrapper(unittest.TestCase):
def test_wrapper1(self):
wrapper.wrapper1()
def test_wrapper2(self):
wrapper.wrapper2()
def test_wrapper3(self):
wrapper.wrapper3()
if __name__ == "__main__":
unittest.main()
We would explode if the functions/classes in some_lib were called, but they aren't:
Matthews-MacBook-Pro:stackoverflow matt$ python test.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.001s
OK
Feel free to comment out the patch and comment in wrapper.some_lib = MagicMock(). You'll get the same result in this toy example, but there is a major difference between the two approaches:
When using #patch('wrapper.some_lib', MagicMock()) the patch is only live for that Test Case class.
When using wrapper.some_lib = MagicMock(), however, that patch will stay live for the entire length of your python program, unless you save off the original module and patch it back manually at some point. Everything that is using the wrapper module will get the mocked version.
So you could so something like:
original_lib = wrapper.some_lib
wrapper.some_lib = MagicMock()
...
# call some test suite, every call to the wrapper module will be mocked out
...
wrapper.some_lib = original_lib
...
# call some other test suite that actually needs the real thing
...
HTH.
EDIT: Misread your question slightly, but you can inspect MagicMock objects to see if they've been called, and if so, fail the test. Or just patch over with something that fails when called (instead of MagicMock). I can provide code to do this if requested (just leave a comment), but hopefully the above can get you started. I think the crux of the question was really about the global patching. Cheers!
My current script calls an external script to perform some task. I want to check the code up to that point in a unittest, but not actually run the external script. Is there some way I can tell the script to effectively skip the following block IF the code is being run as part of a unit test?
The unittest package has extensive support for "mocking" functions and methods. Encapsulate the call to an external program in a simple function that your unit tests can override ("mock out") without modifying the structure of your program. Example:
Here is part of your program, in the module realcode.py
def internal_function_calling_exec(arg1):
"""The real thing"""
print("I am executing an external program")
def bigger_function_being_tested(arg1, arg2):
"""
A complex function with one or more calls to `internal_function_calling_exec`
"""
print("I will now call `internal_function_calling_exec()`")
internal_function_calling_exec(42)
Your unit test can then look like this:
import unittest
from unittest.mock import patch
import realcode
class MyTest(unittest.TestCase):
#patch("realcode.internal_function_calling_exec")
def test_something(self, mocked_func):
realcode.bigger_function_being_tested(1, 2)
mocked_func.assert_called_with(42)
This will never call the original internal_function_calling_exec(). Instead, this will trigger a call to the mock object; your test can then query the object to confirm that it was called properly.
There are ways to mock class methods etc., so you could mock subprocess.call instead, for example. But I think the above is the better pattern.
One possible approach is to set an environment variable in the unit test, and check for that environment variable in the script being tested.
For example, in unittest.py:
os.environ["testing"] = "1"
And in script-to-be-tested.py:
testing = os.environ["testing"]
... do stuff based on the testing variable
Since script-to-be-tested.py will be called from unittest.py, it should inherit the environment variables.
Possibly not the cleanest solution, but it should work.
Is it possible to prevent the execution of "function scoped" fixtures with autouse=True on specific marks only?
I have the following fixture set to autouse so that all outgoing requests are automatically mocked out:
#pytest.fixture(autouse=True)
def no_requests(monkeypatch):
monkeypatch.setattr("requests.sessions.Session.request", MagicMock())
But I have a mark called endtoend that I use to define a series of tests that are allowed to make external requests for more robust end to end testing. I would like to inject no_requests in all tests (the vast majority), but not in tests like the following:
#pytest.mark.endtoend
def test_api_returns_ok():
assert make_request().status_code == 200
Is this possible?
You can also use the request object in your fixture to check the markers used on the test, and don't do anything if a specific marker is set:
import pytest
#pytest.fixture(autouse=True)
def autofixt(request):
if 'noautofixt' in request.keywords:
return
print("patching stuff")
def test1():
pass
#pytest.mark.noautofixt
def test2():
pass
Output with -vs:
x.py::test1 patching stuff
PASSED
x.py::test2 PASSED
In case you have your endtoend tests in specific modules or classes you could also just override the no_requests fixture locally, for example assuming you group all your integration tests in a file called end_to_end.py:
# test_end_to_end.py
#pytest.fixture(autouse=True)
def no_requests():
return
def test_api_returns_ok():
# Should make a real request.
assert make_request().status_code == 200
I wasn't able to find a way to disable fixtures with autouse=True, but I did find a way to revert the changes made in my no_requests fixture. monkeypatch has a method undo that reverts all patches made on the stack, so I was able to call it in my endtoend tests like so:
#pytest.mark.endtoend
def test_api_returns_ok(monkeypatch):
monkeypatch.undo()
assert make_request().status_code == 200
It would be difficult and probably not possible to cancel or change the autouse
You can't canel an autouse, being as it's autouse. Maybe you could do something to change the autouse fixture based on a mark's condition. But this would be hackish and difficult.
possibly with:
import pytest
from _pytest.mark import MarkInfo
I couldn't find a way to do this, but maybe the #pytest.fixture(autouse=True) could get the MarkInfo and if it came back 'endtoend' the fixture wouldn't set the attribute. But you would also have to set a condition in the fixture parameters.
i.e.: #pytest.fixture(True=MarkInfo, autouse=True). Something like that. But I couldn't find a way.
It's recommended that you organize tests to prevent this
You could just separate the no_requests from the endtoend tests by either:
limit the scope of your autouse fixture
put the no_requests into a class
Not make it an auto use, just pass it into the params of each def you need it
Like so:
class NoRequests:
#pytest.fixture(scope='module', autouse=True)
def no_requests(monkeypatch):
monkeypatch.setattr("requests.sessions.Session.request", MagicMock())
def test_no_request1(self):
# do stuff here
# and so on
This is good practice. Maybe a different organization could help
But in your case, it's probably easiest to monkeypatch.undo()