I'm working on an application that will have to consult multiple APIs for information and after processing the data, will output the answer to a client. The client uses a browser to connect to a web server to forward the request, afterwards, the web server will look for the information needed from the multiple APIs and after joining the responses from those APIs will then give an answer to the client.
The web server was built using Flask and a module that extracts the needed information for each API was also implemented (Python). Since the consulting process for each API takes time, I would like to give the web server a timeout for responding, therefore, after the requests are sent only those that are below the time buffer will be used.
My proposed solution:
Use a Redis Queue and an RQ worker to enqueue the requests for each API and store the responses on the Queue then wait for the timeout and collect the responses that were able to respond in the allowed time. Afterwards, process the information and give the response to the user.
The flask web server is setup something like this:
#app.route('/result',methods=["POST"])
def show_result():
inputText = request.form["question"]
tweetModule = Twitter()
tweeterResponse = tweetModule.ask(params=inputText)
redditObject = RedditModule()
redditResponse = redditObject.ask(params=inputText)
edmunds = Edmunds()
edmundsJson = edmunds.ask(params=inputText)
# More APIs could be consulted here
# Send each request async and the synchronize the responses from the queue
template = env.get_template('templates/result.html')
return render_template(template,resp=resp)
The worker:
conn = redis.from_url(redis_url)
if __name__ == '__main__':
with Connection(conn):
worker = Worker(map(Queue, listen))
worker.work()
And lets assume each Module handles its own queueing process.
I can see some problems ahead:
What happens to the information stored on the queue that did not make it to the timeout?
How can I make Flask wait and then extract the responses from the Queue?
Is it possible that information could get mixed if two clients ask in the same time-frame?
Is there a better way to handle the async requests and then synchronize the response?
Thanks!
In such cases I prefer a combination of HTTPX and flask[async]
First - HTTPX
HTTPX offers a standard synchronous API by default, but also gives you the option of an async client if you need it.
Async is a concurrency model that is far more efficient than multi-threading, and can provide significant performance benefits and enable the use of long-lived network connections such as WebSockets.
If you're working with an async web framework then you'll also want to use an async client for sending outgoing HTTP requests.
>>> async with httpx.AsyncClient() as client:
... r = await client.get('https://www.example.com/')
...
>>> r
<Response [200 OK]>
Second - Using async and await in a flask
Routes, error handlers, before request, after request, and teardown functions can all be coroutine functions if Flask is installed with the async extra (pip install flask[async]). It requires Python 3.7+ where contextvars.ContextVar is available. This allows views to be defined with async def and use await.
For example, you should do something like this:
import asyncio
import httpx
from flask import Flask, render_template, request
app = Flask(__name__)
#app.route('/async', methods=['GET', 'POST'])
async def async_form():
if request.method == 'POST':
...
async with httpx.AsyncClient() as client:
tweeterResponse, redditResponse, edmundsJson = await asyncio.gather(
client.get(f'https://api.tweeter....../id?id={request.form["tweeter_id"]}', timeout=None),
client.get(f'https://api.redditResponse.....?key={APIKEY}&reddit={request.form["reddit_id"]}'),
client.post(f'https://api.edmundsJson.......', data=inputText)
)
...
resp = {
"tweeter_response" : tweeterResponse,
"reddit_response": redditResponse,
"edmunds_json" : edmundsJson
}
template = env.get_template('templates/result.html')
return render_template(template, resp=resp)
Related
I have a FastAPI application for testing/development purposes. What I want is that any request that arrives to my app to automatically be sent, as is, to another app on another server, with exactly the same parameters and same endpoint. This is not a redirect, because I still want the app to process the request and return values as usual. I just want to initiate a similar request to a different version of the app on a different server, without waiting for the answer from the other server, so that the other app gets the request as if the original request was sent to it.
How can I achieve that? Below is a sample code that I use for handling the request:
#app.post("/my_endpoint/some_parameters")
def process_request(
params: MyParamsClass,
pwd: str = Depends(authenticate),
):
# send the same request to http://my_other_url/my_endpoint/
return_value = process_the_request(params)
return return_value.as_json()
You could use the AsyncClient() from the httpx library, as described in this answer, as well as this answer and this answer (have a look at those answers for more details on the approach demonstrated below). You can spawn a Client inside the startup event handler, store it on the app instance—as described here, as well as here and here—and reuse it every time you need it. You can explicitly close the Client once you are done with it, using the shutdown event handler.
Working Example
Main Server
When building the request that is about to be forwarded to the other server, the main server uses request.stream() to read the request body from the client's request, which provides an async iterator, so that if the client sent a request with some large body (for instance, the client uploads a large file), the main server would not have to wait for the entire body to be received and loaded into memory before forwarding the request, something that would happen in case you used await request.body() instead, which would likely cause server issues if the body could not fit into RAM.
You can add multiple routes in the same way the /upload one has been defined below, specifying the path, as well as the HTTP method for the endpoint. Note that the /upload route below uses Starlette's path convertor to capture arbitrary paths, as demonstrated here and here. You could also specify the exact path parameters if you wish, but the below provides a more convenient way if there are too many of them. Regardless, the path will be evaluated against the endpoint in the other server below, where you can explicitly specify the path parameters.
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse
from starlette.background import BackgroundTask
import httpx
app = FastAPI()
#app.on_event('startup')
async def startup_event():
client = httpx.AsyncClient(base_url='http://127.0.0.1:8001/') # this is the other server
app.state.client = client
#app.on_event('shutdown')
async def shutdown_event():
client = app.state.client
await client.aclose()
async def _reverse_proxy(request: Request):
client = request.app.state.client
url = httpx.URL(path=request.url.path, query=request.url.query.encode('utf-8'))
req = client.build_request(
request.method, url, headers=request.headers.raw, content=request.stream()
)
r = await client.send(req, stream=True)
return StreamingResponse(
r.aiter_raw(),
status_code=r.status_code,
headers=r.headers,
background=BackgroundTask(r.aclose)
)
app.add_route('/upload/{path:path}', _reverse_proxy, ['POST'])
if __name__ == '__main__':
import uvicorn
uvicorn.run(app, host='0.0.0.0', port=8000)
The Other Server
Again, for simplicity, the Request object is used to read the body, but you can isntead define UploadFile, Form and other parameters as usual. The below is listenning on port 8001.
from fastapi import FastAPI, Request
app = FastAPI()
#app.post('/upload/{p1}/{p2}')
async def upload(p1: str, p2: str, q1: str, request: Request):
return {'p1': p1, 'p2': p2, 'q1': q1, 'body': await request.body()}
if __name__ == '__main__':
import uvicorn
uvicorn.run(app, host='0.0.0.0', port=8001)
Test the example above
import httpx
url = 'http://127.0.0.1:8000/upload/hello/world'
files = {'file': open('file.txt', 'rb')}
params = {'q1': 'This is a query param'}
r = httpx.post(url, params=params, files=files)
print(r.content)
I have a FastAPI application which, in several different occasions, needs to call external APIs. I use httpx.AsyncClient for these calls. The point is that I don't fully understand how I shoud use it.
From httpx' documentation I should use context managers,
async def foo():
""""
I need to call foo quite often from different
parts of my application
"""
async with httpx.AsyncClient() as aclient:
# make some http requests, e.g.,
await aclient.get("http://example.it")
However, I understand that in this way a new client is spawned each time I call foo(), and is precisely what we want to avoid by using a client in the first place.
I suppose an alternative would be to have some global client defined somewhere, and just import it whenever I need it like so
aclient = httpx.AsyncClient()
async def bar():
# make some http requests using the global aclient, e.g.,
await aclient.get("http://example.it")
This second option looks somewhat fishy, though, as nobody is taking care of closing the session and the like.
So the question is: how do I properly (re)use httpx.AsyncClient() within a FastAPI application?
You can have a global client that is closed in the FastApi shutdown event.
import logging
from fastapi import FastAPI
import httpx
logging.basicConfig(level=logging.INFO, format="%(levelname)-9s %(asctime)s - %(name)s - %(message)s")
LOGGER = logging.getLogger(__name__)
class HTTPXClientWrapper:
async_client = None
def start(self):
""" Instantiate the client. Call from the FastAPI startup hook."""
self.async_client = httpx.AsyncClient()
LOGGER.info(f'httpx AsyncClient instantiated. Id {id(self.async_client)}')
async def stop(self):
""" Gracefully shutdown. Call from FastAPI shutdown hook."""
LOGGER.info(f'httpx async_client.is_closed(): {self.async_client.is_closed} - Now close it. Id (will be unchanged): {id(self.async_client)}')
await self.async_client.aclose()
LOGGER.info(f'httpx async_client.is_closed(): {self.async_client.is_closed}. Id (will be unchanged): {id(self.async_client)}')
self.async_client = None
LOGGER.info('httpx AsyncClient closed')
def __call__(self):
""" Calling the instantiated HTTPXClientWrapper returns the wrapped singleton."""
# Ensure we don't use it if not started / running
assert self.async_client is not None
LOGGER.info(f'httpx async_client.is_closed(): {self.async_client.is_closed}. Id (will be unchanged): {id(self.async_client)}')
return self.async_client
httpx_client_wrapper = HTTPXClientWrapper()
app = FastAPI()
#app.get('/test-call-external')
async def call_external_api(url: str = 'https://stackoverflow.com'):
async_client = httpx_client_wrapper()
res = await async_client.get(url)
result = res.text
return {
'result': result,
'status': res.status_code
}
#app.on_event("startup")
async def startup_event():
httpx_client_wrapper.start()
#app.on_event("shutdown")
async def shutdown_event():
await httpx_client_wrapper.stop()
if __name__ == '__main__':
import uvicorn
LOGGER.info(f'starting...')
uvicorn.run(f"{__name__}:app", host="127.0.0.1", port=8000)
Note - this answer was inspired by a similar answer I saw elsewhere a long time ago for aiohttp, I can't find the reference but thanks to whoever that was!
EDIT
I've added uvicorn bootstrapping in the example so that it's now fully functional. I've also added logging to show what's going on on startup and shutdown, and you can visit localhost:8000/docs to trigger the endpoint and see what happens (via the logs).
The reason for calling the start() method from the startup hook is that by the time the hook is called the eventloop has already started, so we know we will be instantiating the httpx client in an async context.
Also I was missing the async on the stop() method, and had a self.async_client = None instead of just async_client = None, so I have fixed those errors in the example.
The answer to this question depends on how you structure your FastAPI application and how you manage your dependencies. One possible way to use httpx.AsyncClient() is to create a custom dependency function that returns an instance of the client and closes it when the request is finished. For example:
from fastapi import FastAPI, Depends
import httpx
app = FastAPI()
async def get_client():
# create a new client for each request
async with httpx.AsyncClient() as client:
# yield the client to the endpoint function
yield client
# close the client when the request is done
#app.get("/foo")
async def foo(client: httpx.AsyncClient = Depends(get_client)):
# use the client to make some http requests, e.g.,
response = await client.get("http://example.it")
return response.json()
This way, you don't need to create a global client or worry about closing it manually. FastAPI will handle the dependency injection and the context management for you. You can also use the same dependency function for other endpoints that need to use the client.
Alternatively, you can create a global client and close it when the application shuts down. For example:
from fastapi import FastAPI, Depends
import httpx
import atexit
app = FastAPI()
# create a global client
client = httpx.AsyncClient()
# register a function to close the client when the app exits
atexit.register(client.aclose)
#app.get("/bar")
async def bar():
# use the global client to make some http requests, e.g.,
response = await client.get("http://example.it")
return response.json()
This way, you don't need to create a new client for each request, but you need to make sure that the client is closed properly when the application stops. You can use the atexit module to register a function that will be called when the app exits, or you can use other methods such as signal handlers or event hooks.
Both methods have their pros and cons, and you should choose the one that suits your needs and preferences. You can also check out the FastAPI documentation on dependencies and testing for more examples and best practices.
I am a beginner in flask. I am making several api using flask.
Below is an example of an api.
#app.route('/alarm', method = [POST])
def add_alarm():
insert_task_alarmDB()
requests.post("another server", data)
response = requests.post("another server", data)
requests.post("another server", data) ...
return response['content']
I would like to process the request asynchronously in the above code.
How can i do that?
..The method of official documentation of flask 2.0 does not work well for me.
You have to define the method as async: async def add_alarm(): and then you have to use an async library for requests for example grequests
I'd like to be able to trigger a long-running python script via a web request, in bare-bones fashion. Also, I'd like to be able to trigger other copies of the script with different parameters while initial copies are still running.
I've looked at flask, aiohttp, and queueing possibilities. Flask and aiohttp seem to have the least overhead to set up. I plan on executing the existing python script via subprocess.run (however, I did consider refactoring the script into libraries that could be used in the web response function).
With aiohttp, I'm trying something like:
ingestion_service.py:
from aiohttp import web
from pprint import pprint
routes = web.RouteTableDef()
#routes.get("/ingest_pipeline")
async def test_ingest_pipeline(request):
'''
Get the job_conf specified from the request and activate the script
'''
#subprocess.run the command with lookup of job conf file
response = web.Response(text=f"Received data ingestion request")
await response.prepare(request)
await response.write_eof()
#eventually this would be subprocess.run call
time.sleep(80)
return response
def init_func(argv):
app = web.Application()
app.add_routes(routes)
return app
But though the initial request returns immediately, subsequent requests block until the initial request is complete. I'm running a server via:
python -m aiohttp.web -H localhost -P 8080 ingestion_service:init_func
I know that multithreading and concurrency may provide better solutions than asyncio. In this case, I'm not looking for a robust solution, just something that will allow me to run multiple scripts at once via http request, ideally with minimal memory costs.
OK, there were a couple of issues with what I was doing. Namely, time.sleep() is blocking, so asyncio.sleep() should be used. However, since I'm interested in spawning a subprocess, I can use asyncio.subprocess to do that in a non-blocking fashion.
nb:
asyncio: run one function threaded with multiple requests from websocket clients
https://docs.python.org/3/library/asyncio-subprocess.html.
Using these help, but there's still an issue with the webhandler terminating the subprocess. Luckily, there's a solution here:
https://docs.aiohttp.org/en/stable/web_advanced.html
aiojobs has a decorator "atomic" that will protect the process until it is complete. So, code along these lines will function:
from aiojobs.aiohttp import setup, atomic
import asyncio
import os
from aiohttp import web
#atomic
async def ingest_pipeline(request):
#be careful what you pass through to shell, lest you
#give away the keys to the kingdom
shell_command = "[your command here]"
response_text = f"running {shell_command}"
response_code = 200
response = web.Response(text=response_text, status=response_code)
await response.prepare(request)
await response.write_eof()
ingestion_process = await asyncio.create_subprocess_shell(shell_command,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE)
stdout, stderr = await ingestion_process.communicate()
return response
def init_func(argv):
app = web.Application()
setup(app)
app.router.add_get('/ingest_pipeline', ingest_pipeline)
return app
This is very bare bones, but might help others looking for a quick skeleton for a temporary internal solution.
Below is a simple app to send mesg to the browser. if there is a new mesg from the redis channel it will be sent other wise send the last know value in a non-blocking way.
But i am doing something wrong. can someone please help me understand
from gevent import monkey, Greenlet
monkey.patch_all()
from flask import Flask,render_template,request,redirect,url_for,abort,session,Response,jsonify
app = Flask(__name__)
myglobaldict = {'somedata':''}
class RedisLiveData:
def __init__(self, channel_name):
self.channel_name = channel_name
self.redis_conn = redis.Redis(host='localhost', port=6379, db=0)
pubsub = self.redis_conn.pubsub()
gevent.spawn(self.sub, pubsub)
def sub(self,pubsub):
pubsub.subscribe(self.channel_name)
for message in pubsub.listen():
gevent.spawn(process_rcvd_mesg, message['data'])
def process_rcvd_mesg(mesg):
print "Received new message %s " % mesg
myglobaldict['somedata'] = mesg
g = RedisLiveData("test_channel")
#app.route('/latestmessage')
def latestmessage():
return Response(myglobaldict,mimetype="application/json")
if __name__ == '__main__':
app.run()
on the javascript side i am just using a simple $.ajax get to view the messages.
but the client http://localhost:5000/latestmessage has the old message even after the redis update.
It should be the cache issue.
You can add a timestamp or a random number to the request http://localhost:5000/latestmessage?t=timestamp sent from the ajax.
I suggest you to use POST instead of GET as http method, you eliminate the caching problem and some annoying behaviour from browsers like chrome where the requests after the first will wait for the first to complete before being sent to the webserver.
If you want to keep the GET method then you can ask jquery to make the request non cache-able by the browser with the setting parameter cache
$.ajax(..., {cache:false})