We Keep Coding

How to add depedency overriding in FastAPI testing

I'm new to FastAPI, I have implemented everything but when it comes to testing the API I can't override a dependency.
Here is my code:
test_controller.py
import pytest
from starlette.testclient import TestClient
from app.main import app
from app.core.manager_imp import ManagerImp
#pytest.fixture()
def client():
with TestClient(app) as test_client:
yield test_client
async def over_create_record():
return {"msg": "inserted successfully"}
app.dependency_overrides[ManagerImp.create_record] = over_create_record
def test_post(client):
data = {"name": "John", "email": "john#abc.com"}
response = client.post("/person/", json=data)
assert response.status_code == 200
assert response.json() == {"msg": "inserted successfully"}
controller.py
from app.controllers.v1.controller import Controller
from fastapi import status, HTTPException
from app.models.taxslip import Person
from app.core.manager_imp import ManagerImp
from app.core.duplicate_exception import DuplicateException
from fastapi_utils.cbv import cbv
from fastapi_utils.inferring_router import InferringRouter
router = InferringRouter(tags=["Person"])
#cbv(router)
class ControllerImp(Controller):
manager = ManagerImp()
#router.post("/person/")
async def create_record(self, person: Person):
"""
Person: A person object
returns response if the person was inserted into the database
"""
try:
response = await self.manager.create_record(person.dict())
return response
except DuplicateException as e:
return e
manager_imp.py
from fastapi import HTTPException, status
from app.database.database_imp import DatabaseImp
from app.core.manager import Manager
from app.core.duplicate_exception import DuplicateException
class ManagerImp(Manager):
database = DatabaseImp()
async def create_record(self, taxslip: dict):
try:
response = await self.database.add(taxslip)
return response
except DuplicateException:
raise HTTPException(409, "Duplicate data")
In testing I want to override create_record function from ManagerImp class so that I could get this response {"msg": "inserted successfully"}. Basically, I want to mock ManagerImp create_record function. I have tried as you can see in test_controller.py but I still get the original response.

You're not using the dependency injection system to get the ManagerImp.create_record function, so there is nothing to override.
Since you're not using FastAPI's Depends to get your dependency - FastAPI has no way of returning the alternative function.
In your case you'll need to use a regular mocking library instead, such as unittest.mock or pytest-mock.
I'd also like to point out that initializing a shared dependency as in you've done here by default will share the same instance across all instances of ControllerImp instead of being re-created for each instance of ControllerImp.
The cbv decorator changes things a bit, and as mentioned in the documentation:
For each shared dependency, add a class attribute with a value of type Depends
So to get this to match the FastAPI way of doing things and make the cbv decorator work as you want to:
def get_manager():
return ManagerImp()
#cbv(router)
class ControllerImp(Controller):
manager = Depends(get_manager)
And when you do it this way, you can use dependency_overrides as you planned:
app.dependency_overrides[get_manager] = lambda: return MyFakeManager()
If you only want to replace the create_record function, you'll still have to use regular mocking.
You'll also have to remove the dependency override after the test has finished unless you want it to apply to all tests, so use yield inside your fixture and then remove the override when the fixture starts executing again.

I think you should put your app.dependency_overrides inside the function with #pytest.fixture. Try to put it inside your client().
#pytest.fixture()
def client():
app.dependency_overrides[ManagerImp.create_record] = over_create_record
with TestClient(app) as test_client:
yield test_client
because every test will run the fresh app, meaning it will reset everything from one to another test and only related things bind with the pytest will effect the test.

How do I mock a session? When mocking a user, it needs to set session.permanent but is outside of context

I have a user profile function I want to test. Inside of it, it carries out flask's login_use functionality, but then references session. I am trying to mock a session to verify what the function overall is doing. It seems like this is a bit more tedious than I thought.
from flask import session
from flask_login import user_login
# Base function to mock.
def set_user_session(token):
user=User(token['id'], token['perms'])
login_user(user, remember=True)
session.permanent = True
session["id"] = user.id
session["perms"] = user.perms
returns True
import unittest
from unittest.mock import patch
class TestSessionSet(unittest.TestCase):
def testSuccessfulFn(self):
with patch("login_user") as loginuser:
loginuser.return_value = True
sampleToken = { "id": 5, "perms": []}
rslt = set_user_session(sampleToken)
self.assertTrue(rslt) # It will be more complex because the result from the function is a bit more complex.
In the sample above, when running i will get an error on line sesssion.permanent = True saying:
RuntimeError: Working outside of request context.
E
E This typically means that you attempted to use functionality that needed
E an active HTTP request. Consult the documentation on testing for
E information about how to avoid this problem.
I thought there would be a fairly simple way to do this, but I was hoping to mock a session to allow this to work. It seems though that a session needs a flask app of sorts running as well, which makes me think: Is there fairly simple and straight forward way to mock this out just to validate the function?

HTTPServer.run requires a BaseHTTPRequestHandler class instead an object

I have an own class derived from BaseHTTPRequestHandler, which implements my specific GET method. This works quite fine:
from http.server import BaseHTTPRequestHandler, HTTPServer
class MyHTTPRequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
""" my implementation of the GET method """
myServer = HTTPServer(("127.0.0.1", 8099), MyHTTPRequestHandler)
myServer.handle_request()
But why do I need to pass my class MyHTTPRequestHandler to the HTTPServer? I know that it is required by documentation:
class http.server.BaseHTTPRequestHandler(request, client_address, server)
This class is used to handle the HTTP requests that arrive at the server. By itself, it cannot respond to any actual HTTP requests; it
must be subclassed to handle each request method (e.g. GET or POST).
BaseHTTPRequestHandler provides a number of class and instance
variables, and methods for use by subclasses.
The handler will parse the request and the headers, then call a method specific to the request type. The method name is constructed
from the request. For example, for the request method SPAM, the
do_SPAM() method will be called with no arguments. All of the relevant
information is stored in instance variables of the handler. Subclasses
should not need to override or extend the init() method.
But I do want to pass an instantiated object of my subclass instead. I don't understand why this has been designed like that and it looks like design failure to me. The purpose of object oriented programming with polymorphy is that I can subclass to implement a specific behavior with the same interfaces, so this seems to me as an unnecessary restriction.
That is what I want:
from http.server import BaseHTTPRequestHandler, HTTPServer
class MyHTTPRequestHandler(BaseHTTPRequestHandler):
def __init__(self, myAdditionalArg):
self.myArg = myAdditionalArg
def do_GET(self):
""" my implementation of the GET method """
self.wfile(bytes(self.myArg, "utf-8"))
# ...
myReqHandler = MyHTTPRequestHandler("mySpecificString")
myServer = HTTPServer(("127.0.0.1", 8099), myReqHandler)
myServer.handle_request()
But if I do that, evidently I receive the expected error message:
TypeError: 'MyHTTPRequestHandler' object is not callable
How can I workaround this so that I can still use print a specific string?
There is also a 2nd reason why I need this: I want that MyHTTPRequestHandler provides also more information about the client, which uses the GET method to retrieve data from the server (I want to retrieve the HTTP-Header of the client browser).
I just have one client which starts a single request to the server. If a solution would work in a more general context, I'll be happy, but I won't
need it for my current project.
Somebody any idea to do that?

A server needs to create request handlers as needed to handle all the requests coming in. It would be bad design to only have one request handler. If you passed in an instance, the server could only ever handle one request at a time and if there were any side effects, it would go very very badly. Any sort of change of state is beyond the scope of what a request handler should do.
BaseHTTPRequestHandler has a method to handle message logging, and an attribute self.headers containing all the header information. It defaults to logging messages to sys.stderr, so you could do $ python -m my_server.py 2> log_file.txt to capture the log messages. or, you could write to file in your own handler.
class MyHTTPRequestHandler(BaseHTTPRequestHandler):
log_file = os.path.join(os.path.abspath(''), 'log_file.txt') # saves in directory where server is
myArg = 'some fancy thing'
def do_GET(self):
# do things
self.wfile.write(bytes(self.myArg,'utf-8'))
# do more
msg_format = 'start header:\n%s\nend header\n'
self.log_message(msg_format, self.headers)
def log_message(self, format_str, *args): # This is basically a copy of original method, just changing the destination
with open(self.log_file, 'a') as logger:
logger.write("%s - - [%s] %s\n" %
self.log_date_time_string(),
format%args))
handler = MyHTTPRequestHandler
myServer = HTTPServer(("127.0.0.1", 8099), handler)

It is possible to derive a specific HTTPServer class (MyHttpServer), which has the following attributes:
myArg: the specific "message text" which shall be printed by the HTTP
request handler
headers: a dictionary storing the headers set by a
HTTP request handler
The server class must be packed together with MyHTTPRequestHandler. Furthermore the implementation is only working properly under the following conditions:
only one HTTP request handler requests an answer from the server at the same time (otherwise data kept by the attributes are corrupted)
MyHTTPRequestHandler is only used with MyHttpServer and vice versa (otherwise unknown side effects like exceptions or data corruption can occur)
That's why both classes must be packed and shipped together in a way like this:
from http.server import BaseHTTPRequestHandler, HTTPServer
class MyHTTPRequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
self.wfile.write(bytes(self.server.myArg, 'utf-8'))
#...
self.server.headers = self.headers
class MyHttpServer(HTTPServer):
def __init__(self, server_address, myArg, handler_class=MyHttpRequestHandler):
super().__init__(server_address, handler_class)
self.myArg = myArg
self.headers = dict()
The usage of these classes could look like this, whereas only one request of a client (i.e. Web-Browser) is answered by the server:
def get_header():
httpd = MyHttpServer(("127.0.0.1", 8099), "MyFancyText", MyHttpRequestHandler)
httpd.handle_request()
return httpd.headers

Accessing the API logger in Flask-Restful's resources

I'm currently using flask-restful (http://flask-restful.readthedocs.io/en/0.3.5/index.html) to deploy resources as endpoints and I'm wondering if there's a way to access the API logger from within then resources classes. I've skimmed through the docs and couldn't find the appropriate answer.
Basically I want to do that :
from flask_restful import Resource
class SomeEndpoint(Resource):
def get(self):
try:
... something throws an exception
except SomeException as se:
... send custom message to API logger <----- Here!
return response
What I though of doing was passing the logger from the API through the constructor of the Resource like that :
App = Flask(__name__)
api = Api(App)
api.add_resource(SomeEndpoint, '/', resource_class_kwargs={'logger': App.logger})
Is this the most appropriate way to access the logger inside flask-restful resource endpoints ?
Thanks a lot

I know the answer has been chosen already, but there is a slightly different approach that also works.
First, import
from flask import current_app as app
in the resource file, and when calling the logger, do:
app.logger.info("This is an info message")

You need to define constructor of Resource. Here an example:
import logging
class SomeEndpoint(Resource):
def __init__(self, **kwargs):
self.logger = kwargs.get('logger')
def get(self):
# self.logger - 'logger' from resource_class_kwargs
return self.logger.name
api.add_resource(SomeEndpoint, '/', resource_class_kwargs={
# any logger here...
'logger': logging.getLogger('my_custom_logger')
})
Open your endpoint. You will see my_custom_logger.
Hope this helps.

Setting debug=False temporarily fixes the problem.
But I dont really know what is the issue when debug is set to True

Reusing connections in Django with Python Requests

What's the correct way of reusing Python Requests connections in Django across multiple HTTP requests. That's what I'm doing currently:
import requests
def do_request(data):
return requests.get('http://foo.bar/', data=data, timeout=4)
def my_view1(request)
req = do_request({x: 1})
...
def my_view2(request)
req = do_request({y: 2})
...
So, I have one function that makes the request. This function is called in various Django views. The views get called in separate HTTP requests by users. My question is: Does Python Requests automatically reuse the same connections (via urllib3 connection pooling)?
Or do I have to first create a Requests session object to work with?
s = requests.Session()
def do_request(data):
return s.get('http://foo.bar/', data=data, auth=('user', 'pass'), timeout=4).text
And if so, does the session object have to be created in global scope or should it be inside the function?
def do_request(data):
s = requests.Session()
return s.get('http://foo.bar/', data=data, auth=('user', 'pass'), timeout=4).text
I can have multiple HTTP requests at the same time, so the solution needs to e thread safe ... I'm a newbie to connection pooling, so I really am not sure and the Requests docs aren't that extensive here.

Create a session, keep the session maintained by passing it through functions and returning it, or create the session object at global level or class level, so the latest state is maintained whenever it is referenced. And it will work like a charm.

For thread-safety, do not hold a global Session object. They should be mostly safe, but there are unresolved discussions about it. See Is the Session object from Python's Requests library thread safe? and Document threading contract for Session class #2766.
Yet, for the purpose of reusing connections, it should be safe to hold a global HTTPAdapter instance, the class actually operating the underlying urllib3 PoolManager and ConnectionPool.
Mount the adapter in your sessions and maybe use the sessions from a service class for easier access and setup.
import atexit
import requests.adapters
adapter = requests.adapters.HTTPAdapter()
# Fix ResourceWarning, but at interpreter exit
# the connections are about to be closed anyway
atexit.register(adapter.close)
class FooService():
URL = "http://foo.bar/"
def __init__(self):
self.session = requests.Session()
self.session.mount("http://", adapter) # <------ THIS
def get_bar(self, data):
return self.session.get(self.URL, data)
def my_view1(request)
foo = FooService()
foo.get_bar({"y": 1})
# ...
def my_view2(request)
foo = FooService()
for y in range(1234):
foo.get_bar({"y": y})
# ...