I'm new to FastAPI, I have implemented everything but when it comes to testing the API I can't override a dependency.
Here is my code:
test_controller.py
import pytest
from starlette.testclient import TestClient
from app.main import app
from app.core.manager_imp import ManagerImp
#pytest.fixture()
def client():
with TestClient(app) as test_client:
yield test_client
async def over_create_record():
return {"msg": "inserted successfully"}
app.dependency_overrides[ManagerImp.create_record] = over_create_record
def test_post(client):
data = {"name": "John", "email": "john#abc.com"}
response = client.post("/person/", json=data)
assert response.status_code == 200
assert response.json() == {"msg": "inserted successfully"}
controller.py
from app.controllers.v1.controller import Controller
from fastapi import status, HTTPException
from app.models.taxslip import Person
from app.core.manager_imp import ManagerImp
from app.core.duplicate_exception import DuplicateException
from fastapi_utils.cbv import cbv
from fastapi_utils.inferring_router import InferringRouter
router = InferringRouter(tags=["Person"])
#cbv(router)
class ControllerImp(Controller):
manager = ManagerImp()
#router.post("/person/")
async def create_record(self, person: Person):
"""
Person: A person object
returns response if the person was inserted into the database
"""
try:
response = await self.manager.create_record(person.dict())
return response
except DuplicateException as e:
return e
manager_imp.py
from fastapi import HTTPException, status
from app.database.database_imp import DatabaseImp
from app.core.manager import Manager
from app.core.duplicate_exception import DuplicateException
class ManagerImp(Manager):
database = DatabaseImp()
async def create_record(self, taxslip: dict):
try:
response = await self.database.add(taxslip)
return response
except DuplicateException:
raise HTTPException(409, "Duplicate data")
In testing I want to override create_record function from ManagerImp class so that I could get this response {"msg": "inserted successfully"}. Basically, I want to mock ManagerImp create_record function. I have tried as you can see in test_controller.py but I still get the original response.
You're not using the dependency injection system to get the ManagerImp.create_record function, so there is nothing to override.
Since you're not using FastAPI's Depends to get your dependency - FastAPI has no way of returning the alternative function.
In your case you'll need to use a regular mocking library instead, such as unittest.mock or pytest-mock.
I'd also like to point out that initializing a shared dependency as in you've done here by default will share the same instance across all instances of ControllerImp instead of being re-created for each instance of ControllerImp.
The cbv decorator changes things a bit, and as mentioned in the documentation:
For each shared dependency, add a class attribute with a value of type Depends
So to get this to match the FastAPI way of doing things and make the cbv decorator work as you want to:
def get_manager():
return ManagerImp()
#cbv(router)
class ControllerImp(Controller):
manager = Depends(get_manager)
And when you do it this way, you can use dependency_overrides as you planned:
app.dependency_overrides[get_manager] = lambda: return MyFakeManager()
If you only want to replace the create_record function, you'll still have to use regular mocking.
You'll also have to remove the dependency override after the test has finished unless you want it to apply to all tests, so use yield inside your fixture and then remove the override when the fixture starts executing again.
I think you should put your app.dependency_overrides inside the function with #pytest.fixture. Try to put it inside your client().
#pytest.fixture()
def client():
app.dependency_overrides[ManagerImp.create_record] = over_create_record
with TestClient(app) as test_client:
yield test_client
because every test will run the fresh app, meaning it will reset everything from one to another test and only related things bind with the pytest will effect the test.
Related
I have a FastAPI application which, in several different occasions, needs to call external APIs. I use httpx.AsyncClient for these calls. The point is that I don't fully understand how I shoud use it.
From httpx' documentation I should use context managers,
async def foo():
""""
I need to call foo quite often from different
parts of my application
"""
async with httpx.AsyncClient() as aclient:
# make some http requests, e.g.,
await aclient.get("http://example.it")
However, I understand that in this way a new client is spawned each time I call foo(), and is precisely what we want to avoid by using a client in the first place.
I suppose an alternative would be to have some global client defined somewhere, and just import it whenever I need it like so
aclient = httpx.AsyncClient()
async def bar():
# make some http requests using the global aclient, e.g.,
await aclient.get("http://example.it")
This second option looks somewhat fishy, though, as nobody is taking care of closing the session and the like.
So the question is: how do I properly (re)use httpx.AsyncClient() within a FastAPI application?
You can have a global client that is closed in the FastApi shutdown event.
import logging
from fastapi import FastAPI
import httpx
logging.basicConfig(level=logging.INFO, format="%(levelname)-9s %(asctime)s - %(name)s - %(message)s")
LOGGER = logging.getLogger(__name__)
class HTTPXClientWrapper:
async_client = None
def start(self):
""" Instantiate the client. Call from the FastAPI startup hook."""
self.async_client = httpx.AsyncClient()
LOGGER.info(f'httpx AsyncClient instantiated. Id {id(self.async_client)}')
async def stop(self):
""" Gracefully shutdown. Call from FastAPI shutdown hook."""
LOGGER.info(f'httpx async_client.is_closed(): {self.async_client.is_closed} - Now close it. Id (will be unchanged): {id(self.async_client)}')
await self.async_client.aclose()
LOGGER.info(f'httpx async_client.is_closed(): {self.async_client.is_closed}. Id (will be unchanged): {id(self.async_client)}')
self.async_client = None
LOGGER.info('httpx AsyncClient closed')
def __call__(self):
""" Calling the instantiated HTTPXClientWrapper returns the wrapped singleton."""
# Ensure we don't use it if not started / running
assert self.async_client is not None
LOGGER.info(f'httpx async_client.is_closed(): {self.async_client.is_closed}. Id (will be unchanged): {id(self.async_client)}')
return self.async_client
httpx_client_wrapper = HTTPXClientWrapper()
app = FastAPI()
#app.get('/test-call-external')
async def call_external_api(url: str = 'https://stackoverflow.com'):
async_client = httpx_client_wrapper()
res = await async_client.get(url)
result = res.text
return {
'result': result,
'status': res.status_code
}
#app.on_event("startup")
async def startup_event():
httpx_client_wrapper.start()
#app.on_event("shutdown")
async def shutdown_event():
await httpx_client_wrapper.stop()
if __name__ == '__main__':
import uvicorn
LOGGER.info(f'starting...')
uvicorn.run(f"{__name__}:app", host="127.0.0.1", port=8000)
Note - this answer was inspired by a similar answer I saw elsewhere a long time ago for aiohttp, I can't find the reference but thanks to whoever that was!
EDIT
I've added uvicorn bootstrapping in the example so that it's now fully functional. I've also added logging to show what's going on on startup and shutdown, and you can visit localhost:8000/docs to trigger the endpoint and see what happens (via the logs).
The reason for calling the start() method from the startup hook is that by the time the hook is called the eventloop has already started, so we know we will be instantiating the httpx client in an async context.
Also I was missing the async on the stop() method, and had a self.async_client = None instead of just async_client = None, so I have fixed those errors in the example.
The answer to this question depends on how you structure your FastAPI application and how you manage your dependencies. One possible way to use httpx.AsyncClient() is to create a custom dependency function that returns an instance of the client and closes it when the request is finished. For example:
from fastapi import FastAPI, Depends
import httpx
app = FastAPI()
async def get_client():
# create a new client for each request
async with httpx.AsyncClient() as client:
# yield the client to the endpoint function
yield client
# close the client when the request is done
#app.get("/foo")
async def foo(client: httpx.AsyncClient = Depends(get_client)):
# use the client to make some http requests, e.g.,
response = await client.get("http://example.it")
return response.json()
This way, you don't need to create a global client or worry about closing it manually. FastAPI will handle the dependency injection and the context management for you. You can also use the same dependency function for other endpoints that need to use the client.
Alternatively, you can create a global client and close it when the application shuts down. For example:
from fastapi import FastAPI, Depends
import httpx
import atexit
app = FastAPI()
# create a global client
client = httpx.AsyncClient()
# register a function to close the client when the app exits
atexit.register(client.aclose)
#app.get("/bar")
async def bar():
# use the global client to make some http requests, e.g.,
response = await client.get("http://example.it")
return response.json()
This way, you don't need to create a new client for each request, but you need to make sure that the client is closed properly when the application stops. You can use the atexit module to register a function that will be called when the app exits, or you can use other methods such as signal handlers or event hooks.
Both methods have their pros and cons, and you should choose the one that suits your needs and preferences. You can also check out the FastAPI documentation on dependencies and testing for more examples and best practices.
Is there anyway for a FastAPI "dependency" to interpret Path parameters?
I have a lot of functions of the form:
#app.post("/item/{item_id}/process", response_class=ProcessResponse)
async def process_item(item_id: UUID, session: UserSession = Depends(security.user_session)) -> ProcessResponse:
item = await get_item(client_id=session.client_id, item_id=item_id)
await item.process()
Over and over, I need to pass in [multiple] arguments to fetch the required item before doing something with it. This is very repetitive and makes the code very verbose. What I'd really like to do is pass the item in as an argument to the method.
Ideally I'd like to make get_item a dependency or embed it somehow in the router. This would dramatically reduce the repetitive logic and excessively verbose function arguments. The problem is that some critical arguments are passed by the client in the Path.
Is it possible to pass Path arguments into a dependency or perhaps execute the dependency in the router and pass the result?
A FastAPI dependency function can take any of the arguments that a normal endpoint function can take.
So in a normal endpoint you might define a path parameter like so:
from fastapi import FastAPI
app = FastAPI()
#app.get("/items/{item_id}")
async def read_item(item_id):
return {"item_id": item_id}
Now if you want to use that parameter in a dependency, you can simply do:
from fastapi import Depends, FastAPI
app = FastAPI()
async def my_dependency_function(item_id: int):
return {"item_id": item_id}
#app.get("/items/{item_id}")
async def read_item(item_id: int, my_dependency: dict = Depends(my_dependency_function)):
return my_dependency
The parameters will simply be passed on through to the dependency function if they are present there. You can also use things like Path and Query within the dependency function to define where these are coming from.
It will just analyze the request object to pull these values.
Here is an example using the Path function from FastAPI:
from fastapi import Depends, FastAPI, Path
app = FastAPI()
async def my_dependency_function(item_id: int = Path(...)):
return {"item_id": item_id}
#app.get("/items/{item_id}")
async def read_item(my_dependency: dict = Depends(my_dependency_function)):
return my_dependency
As for your concern of implementing it as a dependency in the router, you can do something like this when creating the router:
items_router = APIRouter(
prefix="/items",
tags=["items"],
dependencies=[Depends(my_dependency_function)],
)
Or you can do it when you run include_router on the app like:
app.include_router(
items_router,
prefix="/items",
dependencies=[Depends(my_dependency_function)],
)
For more on dependencies and more examples like this see https://fastapi.tiangolo.com/tutorial/dependencies/
How does pytest-django know whether to inject a test function with a RequestFactory or Client instance?
def test_with_client(client):
response = client.get('/')
assert response.content == 'Foobar'
def test_details(rf):
request = rf.get('/customer/details')
response = my_view(request)
assert response.status_code == 200
In other words: how can you make sure the input fixture is of a certain type?
pytest doesn't inject based on type but on name. The name of the input parameter is matched to registered fixtures.
See the docs here, but in short
import pytest
#pytest.fixture
def connection():
return Connection()
def test_my_object(connection):
target = MyObject(connection)
assert ...
You can use type annotations to help PyCharm etc infer the correct type, but these are not used by pytest.
Short answer: you shouldn't be running these checks for each test. Using test argument names to determine which fixtures are injected is a core component of pytest, and littering each test which uses fixtures with assert isinstance(my_fixture, MyFixtureType) for each fixture is redundant.
pytest-django is already testing that the client and rf fixtures are of the correct type:
def test_client(client):
assert isinstance(client, Client)
...
def test_rf(rf):
assert isinstance(rf, RequestFactory)
Edit:
As you've passed the fixture as param to the test method, if you got the fixture name correctly, you don't have to check anything.
Here's an example:
#pytest.fixture(scope='session')
def factory():
return RequestFactory(HTTP_X_REQUESTED_WITH='XMLHttpRequest')
#pytest.fixture(scope='session')
def client():
return Client(HTTP_X_REQUESTED_WITH='XMLHttpRequest')
Now, in your test method you can take one or both of the fixtures and work with them e.g.:
def test_foo(client):
# Do stuff
def test_bar(factory):
# Do stuff
Original answer:
You can check for the type of the input fixture using isinstance.
from django.test import RequestFactory, Client
Inside the test method test for Client:
if isinstance(client, Client):
# This is a Client instance
Similarly for RequestFactory:
if instance(rf, RequestFactory):
# This is a RequestFactory instance
I've written a simple HTTP client using aiohttp and I'm trying to test it by patching aiohttp.ClientSession and aiohttp.ClientResponse. However, it appears as though the unittest.mock.patch decorator is not respecting my asynchronous code. At a guess, I would say it's some kind of namespacing mismatch.
Here's a minimal example:
from aiohttp import ClientSession
async def is_ok(url:str) -> bool:
async with ClientSession() as session:
async with session.request("GET", url) as response:
return (response.status == 200)
I'm using an asynchronous decorator for testing, as described in this answer. So here's my attempted test:
import unittest
from unittest.mock import MagicMock, patch
from aiohttp import ClientResponse
from my.original.module import is_ok
class TestClient(unittest.TestCase):
#async_test
#patch("my.original.module.ClientSession", spec=True)
async def test_client(self, mock_client):
mock_response = MagicMock(spec=ClientResponse)
mock_response.status = 200
async def _mock_request(*args, **kwargs):
return mock_response
mock_client.request = mock_response
status = await is_ok("foo")
self.assertTrue(status)
My is_ok coroutine works fine when it's used in, say, __main__, but when I run the test, it gives me an error that indicates that the session.request function has not been mocked per my patch call. (Specifically it says "Could not parse hostname from URL 'foo'", which it should if it weren't mocked.)
I am unable to escape this behaviour. I have tried:
Importing is_ok after the mocking is done.
Various combinations of assigning mocks to mock_client and mock_client.__aenter__, setting mock_client.request to MagicMock(return_value=mock_response), or using mock_client().request, etc.
Writing a mock ClientSession with specific __aenter__ and __aexit__ methods and using it in the new argument to patch.
None of these appear to make a difference. If I put assertions into is_ok to test that ClientSession is an instance of MagicMock, then these assertions fail when I run the test (as, again, they would when the code is not patched). That leads me to my namespacing mismatch theory: That is, the event loop is running in a different namespace to which patch is targeting.
Either that, or I'm doing something stupid!
Mocking ClientSession is discouraged.
Recommended way is creation fake server and sending real requests to it.
Take a look on aiohttp example.
I'm looking for a way to set request level context in Tornado.
This is useful for logging purpose, to print some request attributes with every log line (like user_id).
I'd like to populate the context in web.RequestHandler and then access it in other coroutines that this request called.
class WebRequestHandler(web.RequestHandler):
#gen.coroutine
def post(self):
RequestContext.test_mode = self.application.settings.get('test_mode', False)
RequestContext.corr_id = self.request.header.get('X-Request-ID')
result = yield some_func()
self.write(result)
#gen.coroutine
def some_func()
if RequestContext.test_mode:
print "In test mode"
do more async calls
Currently I pass context object (dict with values) to every async function call down stream, this way every part of the code can do monitoring and logging with right context.
I'm looking for a cleaner/simpler solution.
Thanks
Alex
The concept of request context doesn't really hold well in async frameworks (especially if you have high volume traffic) for the simple fact that there could potentially be hundreds of concurrent requests and it becomes difficult to determine which "context" to use. This works for sequential frameworks like Flask, Falcon, Django, etc. because requests are handled one by one and it's simple to determine which request you're dealing with.
The preferred method of handling functionality between a request start and end is to override prepare and on_finish respectively.
class WebRequestHandler(web.RequestHandler):
def prepare(self):
print('Logging...prepare')
if self.application.settings.get('test_mode', False):
print("In test mode")
print('X-Request-ID: {0}'.format(self.request.header.get('X-Request-ID')))
#gen.coroutine
def post(self):
result = yield some_func()
self.write(result)
def on_finish(self):
print('Logging...on_finish')
The simple solution would be to create an object that represents the context of your request and pass that into your log function. Example:
class RequestContext(object):
"""
Hold request context
"""
class WebRequestHandler(web.RequestHandler):
#gen.coroutine
def post(self):
# create new context obj and fill w/ necessary parameters
request_context = RequestContext()
request_context.test_mode = self.application.settings.get('test_mode', False)
request_context.corr_id = self.request.header.get('X-Request-ID')
# pass context objects into coroutine
result = yield some_func(request_context)
self.write(result)
#gen.coroutine
def some_func(request_context)
if request_context.test_mode:
print "In test mode"
# do more async calls