I have a FastAPI application which, in several different occasions, needs to call external APIs. I use httpx.AsyncClient for these calls. The point is that I don't fully understand how I shoud use it.
From httpx' documentation I should use context managers,
async def foo():
""""
I need to call foo quite often from different
parts of my application
"""
async with httpx.AsyncClient() as aclient:
# make some http requests, e.g.,
await aclient.get("http://example.it")
However, I understand that in this way a new client is spawned each time I call foo(), and is precisely what we want to avoid by using a client in the first place.
I suppose an alternative would be to have some global client defined somewhere, and just import it whenever I need it like so
aclient = httpx.AsyncClient()
async def bar():
# make some http requests using the global aclient, e.g.,
await aclient.get("http://example.it")
This second option looks somewhat fishy, though, as nobody is taking care of closing the session and the like.
So the question is: how do I properly (re)use httpx.AsyncClient() within a FastAPI application?
You can have a global client that is closed in the FastApi shutdown event.
import logging
from fastapi import FastAPI
import httpx
logging.basicConfig(level=logging.INFO, format="%(levelname)-9s %(asctime)s - %(name)s - %(message)s")
LOGGER = logging.getLogger(__name__)
class HTTPXClientWrapper:
async_client = None
def start(self):
""" Instantiate the client. Call from the FastAPI startup hook."""
self.async_client = httpx.AsyncClient()
LOGGER.info(f'httpx AsyncClient instantiated. Id {id(self.async_client)}')
async def stop(self):
""" Gracefully shutdown. Call from FastAPI shutdown hook."""
LOGGER.info(f'httpx async_client.is_closed(): {self.async_client.is_closed} - Now close it. Id (will be unchanged): {id(self.async_client)}')
await self.async_client.aclose()
LOGGER.info(f'httpx async_client.is_closed(): {self.async_client.is_closed}. Id (will be unchanged): {id(self.async_client)}')
self.async_client = None
LOGGER.info('httpx AsyncClient closed')
def __call__(self):
""" Calling the instantiated HTTPXClientWrapper returns the wrapped singleton."""
# Ensure we don't use it if not started / running
assert self.async_client is not None
LOGGER.info(f'httpx async_client.is_closed(): {self.async_client.is_closed}. Id (will be unchanged): {id(self.async_client)}')
return self.async_client
httpx_client_wrapper = HTTPXClientWrapper()
app = FastAPI()
#app.get('/test-call-external')
async def call_external_api(url: str = 'https://stackoverflow.com'):
async_client = httpx_client_wrapper()
res = await async_client.get(url)
result = res.text
return {
'result': result,
'status': res.status_code
}
#app.on_event("startup")
async def startup_event():
httpx_client_wrapper.start()
#app.on_event("shutdown")
async def shutdown_event():
await httpx_client_wrapper.stop()
if __name__ == '__main__':
import uvicorn
LOGGER.info(f'starting...')
uvicorn.run(f"{__name__}:app", host="127.0.0.1", port=8000)
Note - this answer was inspired by a similar answer I saw elsewhere a long time ago for aiohttp, I can't find the reference but thanks to whoever that was!
EDIT
I've added uvicorn bootstrapping in the example so that it's now fully functional. I've also added logging to show what's going on on startup and shutdown, and you can visit localhost:8000/docs to trigger the endpoint and see what happens (via the logs).
The reason for calling the start() method from the startup hook is that by the time the hook is called the eventloop has already started, so we know we will be instantiating the httpx client in an async context.
Also I was missing the async on the stop() method, and had a self.async_client = None instead of just async_client = None, so I have fixed those errors in the example.
The answer to this question depends on how you structure your FastAPI application and how you manage your dependencies. One possible way to use httpx.AsyncClient() is to create a custom dependency function that returns an instance of the client and closes it when the request is finished. For example:
from fastapi import FastAPI, Depends
import httpx
app = FastAPI()
async def get_client():
# create a new client for each request
async with httpx.AsyncClient() as client:
# yield the client to the endpoint function
yield client
# close the client when the request is done
#app.get("/foo")
async def foo(client: httpx.AsyncClient = Depends(get_client)):
# use the client to make some http requests, e.g.,
response = await client.get("http://example.it")
return response.json()
This way, you don't need to create a global client or worry about closing it manually. FastAPI will handle the dependency injection and the context management for you. You can also use the same dependency function for other endpoints that need to use the client.
Alternatively, you can create a global client and close it when the application shuts down. For example:
from fastapi import FastAPI, Depends
import httpx
import atexit
app = FastAPI()
# create a global client
client = httpx.AsyncClient()
# register a function to close the client when the app exits
atexit.register(client.aclose)
#app.get("/bar")
async def bar():
# use the global client to make some http requests, e.g.,
response = await client.get("http://example.it")
return response.json()
This way, you don't need to create a new client for each request, but you need to make sure that the client is closed properly when the application stops. You can use the atexit module to register a function that will be called when the app exits, or you can use other methods such as signal handlers or event hooks.
Both methods have their pros and cons, and you should choose the one that suits your needs and preferences. You can also check out the FastAPI documentation on dependencies and testing for more examples and best practices.
Related
I do not want to await my function. Look at this code:
loop = asyncio.new_event_loop() # Needed because of flask server
asyncio.set_event_loop(loop) # Needed because of flask server
task = loop.create_task(mainHandler(code, send, text)) #Start mainHandler
return formResponse(True, "mainHandler Started") # Run this without awaiting the task
This is part of a function called on a flask server endpoint. I don't really care how mainHandler is started, all I want is that it starts and the function (not mainHandler) immediately returns without awaiting. I have tried scheduling tasks, using futures and just running it but all to no avail. Even this question describing exactly what I need did not help: How to run an Asyncio task without awaiting? Does anyone have experience with this?
When trying to use asyncio.create_task in this example:
from flask import Flask
from flask import request
import asyncio
app = Flask(__name__)
async def mainHandler(code: str, sends: int, sendText: str):
await asyncio.sleep(60) # simulate the time the function takes
#app.route('/endpoint')
def index():
code = request.args.get('code', default = "NOTSET", type = str)
send = request.args.get('send', default = 0, type = int)
text = request.args.get('text', default = "NOTSET", type = str).replace("+", " ")
if (code != "NOTSET" and send > 0 and text != "NOTSET"):
asyncio.create_task(mainHandler(code, send, text))
return {"error":False}
else:
return {"error":True}
if __name__ == '__main__':
app.debug = True
app.run(host="0.0.0.0", port=80)
It throws:
RuntimeError: no running event loop
when calling asyncio.create_task(mainHandler(code, send, text)).
To sum this up for everyone coming here in the future:
The version of Flask I used was sync. I just switched to Sanic which is similar in Syntax (only minor differences), faster and supports async via asyncio.ensure_future(mainHandler(code, send, text)).
According to gre_gor (Thank you!) you should also be able to use pip install flask[async] to install async flask.
Using FastAPI in a sync, not async mode, I would like to be able to receive the raw, unchanged body of a POST request.
All examples I can find show async code, when I try it in a normal sync way, the request.body() shows up as a coroutine object.
When I test it by posting some XML to this endpoint, I get a 500 "Internal Server Error".
from fastapi import FastAPI, Response, Request, Body
app = FastAPI()
#app.get("/")
def read_root():
return {"Hello": "World"}
#app.post("/input")
def input_request(request: Request):
# how can I access the RAW request body here?
body = request.body()
# do stuff with the body here
return Response(content=body, media_type="application/xml")
Is this not possible with FastAPI?
Note: a simplified input request would look like:
POST http://127.0.0.1:1083/input
Content-Type: application/xml
<XML>
<BODY>TEST</BODY>
</XML>
and I have no control over how input requests are sent, because I need to replace an existing SOAP API.
Using async def endpoint
If an object is a co-routine, it needs to be awaited. FastAPI is actually Starlette underneath, and Starlette methods for returning the request body are async methods (see the source code here as well); thus, one needs to await them (inside an async def endpoint). For example:
from fastapi import Request
#app.post("/input")
async def input_request(request: Request):
return await request.body()
Update 1 - Using def endpoint
Alternatively, if you are confident that the incoming data is a valid JSON, you can define your endpoint with def instead, and use the Body field, as shown below (for more options on how to post JSON data, see this answer):
from fastapi import Body
#app.post("/input")
def input_request(payload: dict = Body(...)):
return payload
If, however, the incoming data are in XML format, as in the example you provided, one option is to pass them using Files instead, as shown below—as long as you have control over how client data are sent to the server (have a look here as well). Example:
from fastapi import File
#app.post("/input")
def input_request(contents: bytes = File(...)):
return contents
Update 2 - Using def endpoint and async dependency
As described in this post, you can use an async dependency function to pull out the body from the request. You can use async dependencies on non-async (i.e., def) endpoints as well. Hence, if there is some sort of blocking code in this endpoint that prevents you from using async/await—as I am guessing this might be the reason in your case—this is the way to go.
Note: I should also mention that this answer—which explains the difference between def and async def endpoints (that you might be aware of)—also provides solutions when you are required to use async def (as you might need to await for coroutines inside a route), but also have some synchronous expensive CPU-bound operation that might be blocking the server. Please have a look.
Example of the approach described earlier can be found below. You can uncomment the time.sleep() line, if you would like to confirm yourself that a request won't be blocking other requests from going through, as when you declare an endpoint with normal def instead of async def, it is run in an external threadpool (regardless of the async def dependency function).
from fastapi import FastAPI, Depends, Request
import time
app = FastAPI()
async def get_body(request: Request):
return await request.body()
#app.post("/input")
def input_request(body: bytes = Depends(get_body)):
print("New request arrived.")
#time.sleep(5)
return body
For convenience, you can simply use asgiref, this package supports async_to_sync and sync_to_async:
from asgiref.sync import async_to_sync
sync_body_func = async_to_sync(request.body)
print(sync_body_func())
async_to_sync execute an async function in a eventloop, sync_to_async execute a sync function in a threadpool.
I want to create a REST service using FastAPI and aio-pika while working asynchronously. For other async database drivers, I could create clients on startup, when get them in route handlers. For example, with motor I would declare simple connection manager:
from motor.motor_asyncio import AsyncIOMotorClient
class Database:
client: AsyncIOMotorClient = None
db = Database()
async def connect_to_mongo():
db.client = AsyncIOMotorClient("mongo:27017")
async def close_mongo_connection():
db.client.close()
async def get_mongo_client() -> AsyncIOMotorClient:
return db.client
Then add couple of handlers:
app.add_event_handler("startup", connect_to_mongo)
app.add_event_handler("shutdown", close_mongo_connection)
and then just use get_mongo_client to get one to my handler.
Problem here is that aio-pika needs asyncio loop to function. Here is an example from the docs:
connection = await aio_pika.connect_robust(
"amqp://guest:guest#127.0.0.1/", loop=loop
)
And with FastAPI I don't have asyncio loop. Are there any way to use it with interface like in example? Can I just create new loop using asyncio.get_event_loop() and pass it to the connect_robust without really using it anywhere? Like this:
connection = await aio_pika.connect_robust(
"amqp://guest:guest#127.0.0.1/", loop=asyncio.get_event_loop()
)
Ok, so, according to the docs, I can just use connect instead of connect_robust:
connection = await aio_pika.connect(
"amqp://guest:guest#127.0.0.1/"
)
I would like to log all HTTP requests sent by an aiohttp ClientSession. The docs provide a list of available loggers. So I tried the following:
import asyncio
import logging
import aiohttp
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
http_logger = logging.getLogger("aiohttp.client")
http_logger.setLevel(logging.DEBUG)
http_logger.propagate = True
async def make_request():
async with aiohttp.ClientSession() as session:
async with session.get('https://httpbin.org/get') as resp:
return await resp.text()
loop = asyncio.get_event_loop()
response_text = loop.run_until_complete(make_request())
print(response_text)
But this only produced the following output:
DEBUG:asyncio:Using selector: EpollSelector
// response text print here
I tried all the loggers from that list in the docs and then searched for questions. This one is similar: specify log request format in aiohttp 2
The answer describes how to set up logging for an aiohttp server. Interestingly, they have to explicitly register the logger:
app = web.Application(loop=loop)
app.router.add_get('/', handle)
app.router.add_get('/{name}', handle)
loop.run_until_complete(
loop.create_server(
app.make_handler(access_log=mylogger, #<--------------- HERE
access_log_format='%r %s %b'), '0.0.0.0', 8080))
Is this necessary for the client, too?
The only way where I could reasonably inject my logger would probably be the session. But the API reference for the ClientSession doesn't specify any logger argument.
All that was necessary for me was to import logging and add
logging.basicConfig(level=logging.DEBUG)
in my make_app().
The default access.log format is quite wordy so I calmed it down a bit with the access_log_format argument to web.run_app
web.run_app(app, access_log_format=" :: %r %s %T %t")
Client request-response lifecycle is pretty complex, that's why aiohttp doesn't log it.
Please use client tracing to register callbacks for all steps. After that, you can log all events by any convenient method (print, logging, whatever).
I'm working on an application that will have to consult multiple APIs for information and after processing the data, will output the answer to a client. The client uses a browser to connect to a web server to forward the request, afterwards, the web server will look for the information needed from the multiple APIs and after joining the responses from those APIs will then give an answer to the client.
The web server was built using Flask and a module that extracts the needed information for each API was also implemented (Python). Since the consulting process for each API takes time, I would like to give the web server a timeout for responding, therefore, after the requests are sent only those that are below the time buffer will be used.
My proposed solution:
Use a Redis Queue and an RQ worker to enqueue the requests for each API and store the responses on the Queue then wait for the timeout and collect the responses that were able to respond in the allowed time. Afterwards, process the information and give the response to the user.
The flask web server is setup something like this:
#app.route('/result',methods=["POST"])
def show_result():
inputText = request.form["question"]
tweetModule = Twitter()
tweeterResponse = tweetModule.ask(params=inputText)
redditObject = RedditModule()
redditResponse = redditObject.ask(params=inputText)
edmunds = Edmunds()
edmundsJson = edmunds.ask(params=inputText)
# More APIs could be consulted here
# Send each request async and the synchronize the responses from the queue
template = env.get_template('templates/result.html')
return render_template(template,resp=resp)
The worker:
conn = redis.from_url(redis_url)
if __name__ == '__main__':
with Connection(conn):
worker = Worker(map(Queue, listen))
worker.work()
And lets assume each Module handles its own queueing process.
I can see some problems ahead:
What happens to the information stored on the queue that did not make it to the timeout?
How can I make Flask wait and then extract the responses from the Queue?
Is it possible that information could get mixed if two clients ask in the same time-frame?
Is there a better way to handle the async requests and then synchronize the response?
Thanks!
In such cases I prefer a combination of HTTPX and flask[async]
First - HTTPX
HTTPX offers a standard synchronous API by default, but also gives you the option of an async client if you need it.
Async is a concurrency model that is far more efficient than multi-threading, and can provide significant performance benefits and enable the use of long-lived network connections such as WebSockets.
If you're working with an async web framework then you'll also want to use an async client for sending outgoing HTTP requests.
>>> async with httpx.AsyncClient() as client:
... r = await client.get('https://www.example.com/')
...
>>> r
<Response [200 OK]>
Second - Using async and await in a flask
Routes, error handlers, before request, after request, and teardown functions can all be coroutine functions if Flask is installed with the async extra (pip install flask[async]). It requires Python 3.7+ where contextvars.ContextVar is available. This allows views to be defined with async def and use await.
For example, you should do something like this:
import asyncio
import httpx
from flask import Flask, render_template, request
app = Flask(__name__)
#app.route('/async', methods=['GET', 'POST'])
async def async_form():
if request.method == 'POST':
...
async with httpx.AsyncClient() as client:
tweeterResponse, redditResponse, edmundsJson = await asyncio.gather(
client.get(f'https://api.tweeter....../id?id={request.form["tweeter_id"]}', timeout=None),
client.get(f'https://api.redditResponse.....?key={APIKEY}&reddit={request.form["reddit_id"]}'),
client.post(f'https://api.edmundsJson.......', data=inputText)
)
...
resp = {
"tweeter_response" : tweeterResponse,
"reddit_response": redditResponse,
"edmunds_json" : edmundsJson
}
template = env.get_template('templates/result.html')
return render_template(template, resp=resp)