I am trying to find out if it is possible to use the FastAPI dependency injection commonly used to get repository instances as a generic callable, rather than having to get every repo independently in the router and then passing them through to services used later on.
An example of how it is currently done:
Dependencies
from app.db.repositories.base import BaseRepository
def get_database(request: Request) -> Database:
return request.app.state._db
def get_repository(Repo_type: Type[BaseRepository]) -> Callable:
def get_repo(db: Database = Depends(get_database)) -> Type[BaseRepository]:
return Repo_type(db)
return get_repo
Router
#router.get("/", response_model=List[TodoPublic], name="todo:get-all-todos")
async def get_all_todos(
todos_repo: TodoRepository = Depends(get_repository(TodoRepository))
) -> List[TodoPublic]:
return await todos_repo.get_all_todos()
What would be ideal is to be able to pass a generic get_repo object through into a service and within that service be able to get any repo that i need
example
Router
#router.get("/", response_model=List[TodoPublic], name="todo:get-all-todos")
async def get_all_todos(
get_repo = Depends(get_repository())
) -> List[TodoPublic]:
return await app.services.todo_service.get_todos(get_repo)
Service
async def get_all_todos(get_repository) -> List[TodoPublic]:
todos_repo: TodoRepository = get_repository(TodoRepository)
return await todos_repo.get_all_todos()
This is a bad example because there is only 1 repo needed, but if i were dependent on many repos, i would have to get them all in the router and pass each one through to the service, rather than just passing the single get_repo callable into the service and handling it in the service.
Thanks in advance
Related
I'm building an API with FastAPI. I want the API to be type-safe, but running mypy on a route returns this error: "Untyped decorator males function 'get_overview' untyped". Here's my route:
router = APIRouter()
#router.post("/overview/{location_id}")
async def get_overview(location_id: str) -> object:
if 4 not in fake_items_db:
raise HTTPException(status_code=404, detail="Item not found")
return overview
This post provides a solution to the error, but doesn't work with the router API.
Is there a FastAPI-specific solution to this error?
Introduction:
In our FastAPI app, we implemented dependency which is being used to commit changes in the database (according to https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/).
Issue:
The issue we are facing (which is also partially mentioned on the website provided above) is that the router response is 200 even though the commit is not succeeded. This is simply because in our case commit or rollback functions are being called after the response is sent to the requestor.
Example:
Database dependency:
def __with_db(request: Request):
db = Session()
try:
yield db
db.commit()
except Exception as e:
db.rollback()
raise e
finally:
db.close()
As an endpoint example, we import csv file with records, then create db model instances and then add them to the db session (for simplicity, irrelevant things deleted).
from models import Movies
...
#router.post("/import")
async def upload_movies(file: UploadFile, db: DbType = db_dependency):
df = await read_csv(file)
new_records = [Movies(**item) for item in df.to_dict("records")]
db.add_all(new_records) # this is still valid operation, no error here
return "OK"
Everything within the endpoint doesn't raise an error, so the endpoint returns a positive response, however, once the rest of the dependency code is being executed, then it throws an error (ie. whenever one of the records has a null value).
Question:
Is there any solution, to how to actually get an error when the database failed to commit the changes?
Of course, the simplest one would be to add db.commit() or even db.flush() to each endpoint but because of the fact we have a lot of endpoints, we want to avoid this repetition in each of them (if it is even possible).
Best regards,
This is the solution we have implemented for this individual use case.
As a reminder, the main purpose was to catch a database error and react to it by sending proper response to the client. The tricky part was that we wanted to omit the scenario of adding the same line of code to every endpoint as we have plenty of them.
We managed to solve it with middleware.
Updated dependency.py
def __with_db(request: Request):
db = Session()
#assign db to request state for middleware to be able to acces it
request.state.db = db
yield db
Added one line to the app.py
# fastAPI version: 0.79.0
from starlette.middleware.base import BaseHTTPMiddleware
from middlewares import unit_of_work_middleware
...
app = FastAPI()
...
app.add_middleware(BaseHTTPMiddleware, dispatch=unit_of_work_middleware) #new line
...
And created main middleware logic in middlewares.py
from fastapi import Request, Response
async def unit_of_work_middleware(request: Request, call_next) -> Response:
try:
response = await call_next(request)
# Committing the DB transaction after the API endpoint has finished successfully
# So that all the changes made as part of the router are written into the database all together
# This is an implementation of the Unit of Work pattern https://martinfowler.com/eaaCatalog/unitOfWork.html
if "db" in request.state._state:
request.state.db.commit()
return response
except:
# Rolling back the database state to the version before the API endpoint call
# As the exception happened, all the database changes made as part of the API call
# should be reverted to keep data consistency
if "db" in request.state._state:
request.state.db.rollback()
raise
finally:
if "db" in request.state._state:
request.state.db.close()
The middleware logic is applied to every endpoint so each request that is coming is going through it.
I think it's relatively easy way to implement it and get this case resolved.
I don't know your FastAPI version. But as i know, from 0.74.0, dependencies with yield can catch HTTPException and custom exceptions before response was sent(i test 0.80.0 is okay):
async def get_database():
with Session() as session:
try:
yield session
except Exception as e:
# rollback or other operation.
raise e
finally:
session.close()
If one HTTPException raised, the flow is:
endpoint -> dependency catch exception -> ExceptionMiddleware catch exception -> respond
Get more info https://fastapi.tiangolo.com/release-notes/?h=asyncexi#breaking-changes_1
Additional, about commit only in one code block,
Solution 1, can use decorator:
def decorator(func):
#wraps(func)
async def wrapper(*arg, **kwargs):
rsp = await func(*arg, **kwargs)
if 'db' in kwargs:
kwargs['db'].commit()
return rsp
return wrapper
#router.post("/import")
#decorator
async def upload_movies(file: UploadFile, db: DbType = db_dependency):
df = await read_csv(file)
new_records = [Movies(**item) for item in df.to_dict("records")]
db.add_all(new_records) # this is still valid operation, no error here
return "OK"
#Barosh I think decorator is the easiest way. I also thought about middleware but it's not possible.
Solution 2, just an idea:
save session in request
async def get_database(request: Request):
with Session() as session:
request.state.session = session
try:
yield session
except Exception as e:
# rollback or other operation.
raise e
finally:
session.close()
custom starlette.request_response
def custom_request_response(func: typing.Callable) -> ASGIApp:
"""
Takes a function or coroutine `func(request) -> response`,
and returns an ASGI application.
"""
is_coroutine = iscoroutinefunction_or_partial(func)
async def app(scope: Scope, receive: Receive, send: Send) -> None:
request = Request(scope, receive=receive, send=send)
if is_coroutine:
response = await func(request)
else:
response = await run_in_threadpool(func, request)
request.state.session.commit() # or other operation
await response(scope, receive, send)
return app
custom FastAPI.APIRoute.app
class CustomRoute(APIRoute):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.app = custom_request_response(self.get_route_handler())
create router with CustomRoute
router = APIRouter(route_class=CustomRoute)
I think this is a executable idea. You can test it.
Hope this is useful.
I'm new to FastAPI, I have implemented everything but when it comes to testing the API I can't override a dependency.
Here is my code:
test_controller.py
import pytest
from starlette.testclient import TestClient
from app.main import app
from app.core.manager_imp import ManagerImp
#pytest.fixture()
def client():
with TestClient(app) as test_client:
yield test_client
async def over_create_record():
return {"msg": "inserted successfully"}
app.dependency_overrides[ManagerImp.create_record] = over_create_record
def test_post(client):
data = {"name": "John", "email": "john#abc.com"}
response = client.post("/person/", json=data)
assert response.status_code == 200
assert response.json() == {"msg": "inserted successfully"}
controller.py
from app.controllers.v1.controller import Controller
from fastapi import status, HTTPException
from app.models.taxslip import Person
from app.core.manager_imp import ManagerImp
from app.core.duplicate_exception import DuplicateException
from fastapi_utils.cbv import cbv
from fastapi_utils.inferring_router import InferringRouter
router = InferringRouter(tags=["Person"])
#cbv(router)
class ControllerImp(Controller):
manager = ManagerImp()
#router.post("/person/")
async def create_record(self, person: Person):
"""
Person: A person object
returns response if the person was inserted into the database
"""
try:
response = await self.manager.create_record(person.dict())
return response
except DuplicateException as e:
return e
manager_imp.py
from fastapi import HTTPException, status
from app.database.database_imp import DatabaseImp
from app.core.manager import Manager
from app.core.duplicate_exception import DuplicateException
class ManagerImp(Manager):
database = DatabaseImp()
async def create_record(self, taxslip: dict):
try:
response = await self.database.add(taxslip)
return response
except DuplicateException:
raise HTTPException(409, "Duplicate data")
In testing I want to override create_record function from ManagerImp class so that I could get this response {"msg": "inserted successfully"}. Basically, I want to mock ManagerImp create_record function. I have tried as you can see in test_controller.py but I still get the original response.
You're not using the dependency injection system to get the ManagerImp.create_record function, so there is nothing to override.
Since you're not using FastAPI's Depends to get your dependency - FastAPI has no way of returning the alternative function.
In your case you'll need to use a regular mocking library instead, such as unittest.mock or pytest-mock.
I'd also like to point out that initializing a shared dependency as in you've done here by default will share the same instance across all instances of ControllerImp instead of being re-created for each instance of ControllerImp.
The cbv decorator changes things a bit, and as mentioned in the documentation:
For each shared dependency, add a class attribute with a value of type Depends
So to get this to match the FastAPI way of doing things and make the cbv decorator work as you want to:
def get_manager():
return ManagerImp()
#cbv(router)
class ControllerImp(Controller):
manager = Depends(get_manager)
And when you do it this way, you can use dependency_overrides as you planned:
app.dependency_overrides[get_manager] = lambda: return MyFakeManager()
If you only want to replace the create_record function, you'll still have to use regular mocking.
You'll also have to remove the dependency override after the test has finished unless you want it to apply to all tests, so use yield inside your fixture and then remove the override when the fixture starts executing again.
I think you should put your app.dependency_overrides inside the function with #pytest.fixture. Try to put it inside your client().
#pytest.fixture()
def client():
app.dependency_overrides[ManagerImp.create_record] = over_create_record
with TestClient(app) as test_client:
yield test_client
because every test will run the fresh app, meaning it will reset everything from one to another test and only related things bind with the pytest will effect the test.
Consider the following example class containing attributes that require running a coroutine for initialization:
class Example:
def __init__(self) -> None:
self._connection: Optional[Connection] = None
async def connect() -> None:
self._connection = await connect_somewhere(...)
async def send(data: bytes) -> None:
self._connection.send(data)
If I run mypy (perhaps with strict-optional enabled) on this example, it will complain that _connection can be None in send method and the code is not type-safe. I can't initialize the _connection variable in __init__, as it needs to be run asynchronously in a coroutine. It's probably a bad idea to declare the variable outside __init__ too. Is there any way to solve this? Or do you recommend another (OOP) design that would solve the issue?
Currently, I either ignore the mypy complaints, prepend assert self._connection before each usage or append # type: ignore after the usage.
It is generally not good design to have classes in an unusable state unless some method is called on them. An alternative is dependency injection and an alternative constructor:
from typing import TypeVar, Type
# not strictly needed – one can also use just 'Example'
# if inheritance is not needed
T = TypeVar('T')
class Example:
# class always receives a fully functioning connection
def __init__(self, connection: Connection) -> None:
self._connection = connection
# class can construct itself asynchronously without a connection
#classmethod
async def connect(cls: Type[T]) -> T:
return cls(await connect_somewhere(...))
async def send(self, data: bytes) -> None:
self._connection.send(data)
This frees __init__ from relying on some other initialiser to be called later on; as a bonus, it is possible to provide a different connection, e.g. for testing.
The alternative constructor, here connect, still allows to create the object in a self-contained way (without the callee knowing how to connect) but with full async support.
async def example():
# create instance asynchronously
sender = await Example.connect()
await sender.send(b"Hello ")
await sender.send(b"World!")
To get the full life-cycle of opening and closing, supporting async with is the most straightforward approach. This can be supported in a similar way to the alternative constructor – by providing an alternative construct as a context manager:
from typing import TypeVar, Type, AsyncIterable
from contextlib import asynccontextmanager
T = TypeVar('T')
class Example:
def __init__(self, connection: Connection) -> None:
self._connection = connection
#asynccontextmanager
#classmethod
async def scope(cls: Type[T]) -> AsyncIterable[T]:
connection = await connect_somewhere(...) # use `async with` if possible!
try:
yield cls(connection)
finally:
connection.close()
async def send(self, data: bytes) -> None:
self._connection.send(data)
Alternative connect constructor omitted for brevity. For Python 3.6, asynccontextmanager can be fetched from the asyncstdlib (Disclaimer: I maintain this library).
There is a general caveat: closing does leave objects in an unusable – thus inconsistent – state practically by definition. Python's type system has no way to separate "open Connection" from "closed Connection", and especially not to detect that .close or the end of a context transitions from one to the other.
By using async with one partially side-steps this issue, since context managers are generally understood not to be useable after their block by convention.
async def example():
async with Example.scope() as sender:
await sender.send(b"Hello ")
await sender.send(b"World!")
It's probably a bad idea to declare the variable outside __init__ too
This is close. You have to annotate it outside of __init__.
class Example:
_connection: Connection
async def connect(self) -> None:
self._connection = await connect_somewhere(…)
Is there anyway for a FastAPI "dependency" to interpret Path parameters?
I have a lot of functions of the form:
#app.post("/item/{item_id}/process", response_class=ProcessResponse)
async def process_item(item_id: UUID, session: UserSession = Depends(security.user_session)) -> ProcessResponse:
item = await get_item(client_id=session.client_id, item_id=item_id)
await item.process()
Over and over, I need to pass in [multiple] arguments to fetch the required item before doing something with it. This is very repetitive and makes the code very verbose. What I'd really like to do is pass the item in as an argument to the method.
Ideally I'd like to make get_item a dependency or embed it somehow in the router. This would dramatically reduce the repetitive logic and excessively verbose function arguments. The problem is that some critical arguments are passed by the client in the Path.
Is it possible to pass Path arguments into a dependency or perhaps execute the dependency in the router and pass the result?
A FastAPI dependency function can take any of the arguments that a normal endpoint function can take.
So in a normal endpoint you might define a path parameter like so:
from fastapi import FastAPI
app = FastAPI()
#app.get("/items/{item_id}")
async def read_item(item_id):
return {"item_id": item_id}
Now if you want to use that parameter in a dependency, you can simply do:
from fastapi import Depends, FastAPI
app = FastAPI()
async def my_dependency_function(item_id: int):
return {"item_id": item_id}
#app.get("/items/{item_id}")
async def read_item(item_id: int, my_dependency: dict = Depends(my_dependency_function)):
return my_dependency
The parameters will simply be passed on through to the dependency function if they are present there. You can also use things like Path and Query within the dependency function to define where these are coming from.
It will just analyze the request object to pull these values.
Here is an example using the Path function from FastAPI:
from fastapi import Depends, FastAPI, Path
app = FastAPI()
async def my_dependency_function(item_id: int = Path(...)):
return {"item_id": item_id}
#app.get("/items/{item_id}")
async def read_item(my_dependency: dict = Depends(my_dependency_function)):
return my_dependency
As for your concern of implementing it as a dependency in the router, you can do something like this when creating the router:
items_router = APIRouter(
prefix="/items",
tags=["items"],
dependencies=[Depends(my_dependency_function)],
)
Or you can do it when you run include_router on the app like:
app.include_router(
items_router,
prefix="/items",
dependencies=[Depends(my_dependency_function)],
)
For more on dependencies and more examples like this see https://fastapi.tiangolo.com/tutorial/dependencies/