I’ve started working a lot with Flask SocketIO in Python with Eventlet and are looking for a solution to handle concurrent requests/threading. I’ve seen that it is possible with gevent, but how can I do it if I use eventlet?
The eventlet web server supports concurrency through greenlets, same as gevent. No need for you to do anything, concurrency is always enabled.
You could use gunicorn or its analogs to launch the app in production mode with several workers.
As said here:
gunicorn --worker-class eventlet -w 5 module:app
Where the number after -w is the number of workers, module is your flask-socketio server module, and app is the flask app (app = flask.Flask(__name__)). Each worker is a process busy with handling incoming requests, so you will have concurrency. If the tasks your app does take significant time, the worker doing that task will be irresponsive while doing it.
Note: if you launch your app this way, the if __name__ == '__main__': part will be ignored, it seems that your module will be imported. And you don't need to call app.run yourself in the module in this case
Related
A simple flask app accepts requests and then makes calls to https endpoints. Using gunicorn with multiple worker processes leads to ssl failures.
Using flask run works perfectly, albeit slowly.
Using gunicorn --preload --workers 1 also works perfectly, albeit slowly.
Changing to gunicorn --preload --workers 10 very frequently fails with [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] which leads me to think that there's some per-connection state that is being messed up. But, gunicorn is supposed to fork before beginning service of requests.
Ideas?
I was using --preload to avoid having each worker retrieve initial oauth context for use in some of the https webapi calls. Rule of thumb should be that when doing fork() (in gunicorn), you really need to understand what is happening with the ssl state.
Solution was to disable the preload and do the oauth individually in each worker.
I have a flask app which i'm trying to front with gunicorn. I want to use the preload flag since my application has some scheduled jobs using apscheduler which i want only to run in the master and not the workers.
I also want to use the ThreadPoolExecutor in python to delegate jobs to the background triggered by a route on my app.
when I use the --preload flag with gunicorn any calls to my threadpoolexecutor (using executor.submit) seem to fail. The same seems to happen when i programatically trigger a job through the apscheduler.
When i don't use the --preload flag everything runs smoothly.
Is there some config i can change to get this working or would this not work with the --preload flag?
I've got a web app developed in Flask. The setup is simple. The app is running on Gunicorn. All requests are proxied through the nginx. The Flask app itself makes HTTP requests to external API. The HTTP requests from the flask app to the external API are initiated by AJAX calls from the javascript code in the frontend. The external API returns data in JSON format to the Flask app and the back to the frontend.
The problem is that when I run this app in development mode with the option multithreaded = True I can see that the JSON data get returned asynchronously to the server and I can see the result on the frontend page very quickly.
However, when I try to run the app in production mode with nginx and gunicorn I see that the JSON data get returned sequentially - quit slowly, one by one. It seems that due to some reason the HTTP requests to the external API get blocked.
I use supervisor on linux Ubuntu Server 16.04. This is how I start gunicorn through supervisor:
command = /path/to/project/env/bin/gunicorn -k gevent --worker-connections 1000 wsgi:app -b localhost:8500
It seems that gunicorn does not handle the requests asynchronously, although it should.
As experiment I ran the Flask app using it's built in wsgi server (NOT gunicorn) in development mode, with debug=True and multithreaded=True. All requests were still proxied through the nginx. The JSON data returned much quicker, i.e. asynchronously (seems the calls did not block).
I read gunicorn's documentation. It says if I need to make calls to external API, then I should use async workers. I use them but it doesn't work.
All the caching stuff was taken into account. I may assume that I don't use any cache. I cleared it all when I checked the server setups.
What am I missing? How can I make gunicorn run as expected?
Thanks.
I actually solved this problem quite quickly and forgot to post the answer right away. The reason why the gunicorn server did not process the requests acynchronously as I would expect was very simple and stupid. Since I was managing gunicorn through the supervisor after I had changed the config to:
command = /path/to/project/env/bin/gunicorn -k gevent --worker-connections 1000 wsgi:app -b localhost:8500
I forgot to run:
sudo supervisorctl reread
sudo supervisorctl update
It's simple but not obvious though. My mistake was that I expected the config to update automatically after I restart my app on gunicorn using this command:
sudo supervisorctl restart my_app
Yes it restart the app, but not the config of gunicorn.
I have a WSGI application (it's a Flask app, but that should be irrelevant, I think) running under a Gunicorn server at port 9077. The app has a /status endpoint, which is supposed to report 'OK' if the app is running. If it fails to report OK within a reasonable time, the whole container gets killed (by Kubernetes).
The problem is this: when the app is under very heavy load (which does happen occasionally), the /status endpoint can take a while to respond and the container sometimes gets killed prematurely. Is there a way to configure Gunicorn to always serve the /status endpoint in a separate thread? Perhaps even on a different port? I would appreciate any hints or ideas for dealing with this situation.
never worked with Gunicorn, and im not sure if it supports this feature.
But with uWSGI, when i know that the app is going to be under a heavy load,
i run uwsgi with --processes (can also run in multithread mode or both)
uWSGI just spins up multiple instances of the flask app and act as a load balancer, no need for different ports, uwsgi takes care of everything.
You are not bound by GIL anymore and your app uses all the resources available on the machine.
documentation about uWSGI concurrency
a quick tutorial on how to setup a flask app, uWSGI and nginx (you can skip the nginx part)
here is an example of the config file i provide.
[uwsgi]
module = WSGI:app
master = true
processes = 16
die-on-term = true
socket = 0.0.0.0:8808
protocol = http
uwsgi --daemonize --ini my_uwsgi_conf.ini
I can easily achieve 1000 calls/sec when its running that way.
hope that helps.
ps: Another solution for you, just spin up more containers that are running your app.
And put them behind nginx to load-balance
I am using a flask application running under uwsgi. Now I haven't used any processes, threads or workers in uwsgi and all I want to do is to handle concurrent requests. So shall I use multi threading in my flask application to handle concurrent requests?
p.s - I expect at least 500 requests/sec on my app.
...
//Flask Application
def app():
start_thread()
...