We Keep Coding

What is limiting the speed of HTTP requests over localhost?

When I try sending a 10MB HTTP request from my server to my client both running on the same machine, it takes ~2s to complete. I would expect it to be faster since all the data is just moving over localhost loopback. I followed this article to test my localhost speed and it showed ~2000MB/s, at that speed a 10MB request should take 5ms.
Why does my HTTP request take so long and how can I speed it up?
The context for why I'm asking is that I am trying to make a web based GUI for my Python project and it needs to display large data tables. I thought a web-based GUI could be just as fast as native frameworks like tkinter since over localhost the HTTP request time should be negligible. I tried FastAPI, Flask, Tornado and ExpressJS, but they all had similar slow performance.
server.py
from fastapi import FastAPI
from starlette.middleware.cors import CORSMiddleware
import uvicorn
from fastapi.responses import PlainTextResponse
app = FastAPI()
app.add_middleware(CORSMiddleware,
allow_methods=["*"],
allow_headers=["*"],
allow_origins=["*"])
DATA = 'x' * 10 ** 7
#app.get("/api/data", response_class=PlainTextResponse)
def _():
return DATA
uvicorn.run(app, debug=False, port=8000)
client.py
import requests
import time
start = time.time()
response = requests.get("http://localhost:8000/api/data")
end = time.time()
duration = (end - start)
print(duration)
print(f"Response size: {len(response.content) / 1048576} MB")

Returning 'still loading' response with Flask API

I have a scikit-learn classifier running as a Dockerised Flask app, launched with gunicorn. It receives input data in JSON format as a POST request, and responds with a JSON object of results.
When the app is first launched with gunicorn, a large model (serialised with joblib) is read from a database, and loaded into memory before the app is ready for requests. This can take 10-15 minutes.
A reproducible example isn't feasible, but the basic structure is illustrated below:
from flask import Flask, jsonify, request, Response
import joblib
import json
def classifier_app(model_name):
# Line below takes 10-15 mins to complete
classifier = _load_model(model_name)
app = Flask(__name__)
#app.route('/classify_invoice', methods=['POST'])
def apicall():
query = request.get_json()
results = _build_results(query['data'])
return Response(response=results,
status=200,
mimetype='application/json')
print('App loaded!')
return app
How do I configure Flask or gunicorn to return a 'still loading' response (or suitable error message) to any incoming http requests while _load_model is still running?

Basically, you want to return two responses for one request. So there are two different possibilities.
First one is to run time-consuming task in background and ping server with simple ajax requests every two seconds to check if task is completed or not. If task is completed, return result, if not, return "Please standby" string or something.
Second one is to use websockets and flask-socketio extension.
Basic server code would be something like this:
from threading import Thread
from flask import Flask
app = Flask(__name__)
socketio = SocketIO(app)
def do_work():
result = your_heavy_function()
socketio.emit("result", {"result": result}, namespace="/test/")
#app.route("/api/", methods=["POST"])
def start():
socketio.start_background_task(target=do_work)
# return intermediate response
return Response()
On the client side you should do something like this
var socket = io.connect('http://' + document.domain + ':' + location.port + '/test/');
socket.on('result', function(msg) {
// Process your request here
});
For further details, visit this blog post, flask-socketio documentation for server-side reference and socketio documentation for client-side reference.
PS Using web-sockets this you can make progress-bar too.

Why am I not getting DeadlineExceededError with this GCP App Engine app?

I'm trying to better understand Google App Engine, specifically the request processing time limits. On the documentation it states:
A request handler has a limited amount of time to generate and return
a response to a request, typically around 60 seconds. Once the
deadline has been reached, the request handler is interrupted.
I understood that to mean that a call to the sleep function below (via the [project-id].appspot.com/sleep url) would result in an error, but that is not the case. I've ran it for 600 seconds and it still returned the expected response.
import time
from flask import Flask
# If `entrypoint` is not defined in app.yaml, App Engine will look for an app
# called `app` in `main.py`.
app = Flask(__name__)
#app.route("/")
def hello():
"""Return a friendly HTTP greeting."""
return "Hello World!"
#app.route("/sleep")
def sleep():
sleep_time = request.args.get("time", default=120, type=int)
time.sleep(sleep_time)
return f"slept {sleep_time} seconds"

Two things are happening: App Engine is being generous here, and you haven't quite exceeded the limit. I tried the following app:
import time
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
"""Return a friendly HTTP greeting."""
time.sleep(1000000)
return "Hello World!"
And it timed out after 601.9 seconds. I'm sure you'll get similar behavior if you try longer times, but generally you shouldn't depend on the extra runtime being available.

Running two process on Flask with common variables

I want to build a Webapp with Flask where some data is printed on a dynamic page in real time.
The data is taken from a Python script which connects to a Websocket, then it's printed on the frontend with Flask.
I have two problems:
1) I can't run both the scripts together
2) I don't know how to call parsed from test to yield
Here is the code:
from time import sleep
from flask import Flask, render_template
import websocket
from bitmex_websocket import Instrument
from bitmex_websocket.constants import InstrumentChannels
from bitmex_websocket.constants import Channels
import json
from threading import Thread, Event
app = Flask(__name__)
websocket.enableTrace(True)
channels = [
InstrumentChannels.trade,
]
XBTUSD = Instrument(symbol='XBTUSD',
channels=channels)
XBTUSD.on('action', lambda msg: test(msg))
def test(msg):
parsed = json.loads(json.dumps(msg))
print(parsed)
#app.route('/')
def index():
# render the template (below) that will use JavaScript to read the stream
return render_template('index.html')
#app.route('/stream_sqrt')
def stream():
def generate():
yield '{}\n'.format('test')
return app.response_class(generate(), mimetype='text/plain')
if __name__ == '__main__':
XBTUSD.run_forever()
app.run()
If i put XBTUSD.run_forever() before app.run() i will start the part supposed to retrieve the data but the Flask app won't start. If i do the opposite, the Flask app will run but not the other part. How can i run together the whole app? How could i "share" variables between test and generate?

An easier way to go, please use flask-socketio instead flask.
https://flask-socketio.readthedocs.io/en/latest/
Sample for sending messages using flask-socketio
https://flask-socketio.readthedocs.io/en/latest/#sending-messages

How to use grequests in flask?

I am experimenting with gunicorn with gevent workers and Flask. In a view function, I need to make a GET request to an url. Previously, I'm using requests library to do that. But since I want to use gevent workers, I believe I have to make it async, so I use grequests library.
This is my code:
from gevent import monkey
monkey.patch_all()
import grequests
from flask import Flask
from flask import jsonify
pool = grequests.Pool()
#app.route('/<search_term>')
def find_result(search_term):
return get_result(search_term) or ''
def get_result(search_term):
request = grequests.get('https://www.google.com/search?q=' + search_term)
job = grequests.send(request, pool=pool)
async_resp = job.get() # this blocks
resp = async_resp.response
if resp.status_code == 200:
return resp.text
Is this the correct way to use grequests, since I use blocking job.get()? I want to exploit the fact that the problem I have is IO bound.