I have a web app setup like this:
nginx <--> gunicorn <--> flask
I believe nginx can serve a lot of concurrent connection. But I heard that from the WSGI gateway to the flask app, it's blocking. I.e. only a single request can be served at a time. I read it here. My question is: why couldn't nginx invoke another instance (not sure if this is the right term) of gunicorn and handle multiple request in parallel?
This is simply not true, Gunicorn (and all other WSGI app servers) can (and should) be configured to use multiple threads, processes, or eventlets depending on the specific WSGI server's concurrency model. Each thread (or eventlet) in each process dispatches one request at a time to the app it is running.
Nginx does nothing to launch the first, or any subsequent, WSGI processes. You start the WSGI server, configured correctly, and it handles the concurrency. Nginx dispatches requests as concurrently as it can to whatever app it is configured to proxy to.
Related
Setting up Flask with uWSGI and Nginx can be difficult. I tried following this DigitalOcean tutorial and still had trouble. Even with buildout scripts it takes time, and I need to write instructions to follow next time.
If I don't expect a lot of traffic, or the app is private, does it make sense to run it without uWSGI? Flask can listen to a port. Can Nginx just forward requests?
Does it make sense to not use Nginx either, just running bare Flask app on a port?
When you "run Flask" you are actually running Werkzeug's development WSGI server, and passing your Flask app as the WSGI callable.
The development server is not intended for use in production. It is not designed to be particularly efficient, stable, or secure. It does not support all the possible features of a HTTP server.
Replace the Werkzeug dev server with a production-ready WSGI server such as Gunicorn or uWSGI when moving to production, no matter where the app will be available.
The answer is similar for "should I use a web server". WSGI servers happen to have HTTP servers but they will not be as good as a dedicated production HTTP server (Nginx, Apache, etc.).
Flask documents how to deploy in various ways. Many hosting providers also have documentation about deploying Python or Flask.
First create the app:
import flask
app = flask.Flask(__name__)
Then set up the routes, and then when you want to start the app:
import gevent.pywsgi
app_server = gevent.pywsgi.WSGIServer((host, port), app)
app_server.serve_forever()
Call this script to run the application rather than having to tell gunicorn or uWSGI to run it.
I wanted the utility of Flask to build a web application, but had trouble composing it with other elements. I eventually found that gevent.pywsgi.WSGIServer was what I needed. After the call to app_server.serve_forever(), call app_server.stop() when to exit the application.
In my deployment, my application is listening on localhost:port using Flask and gevent, and then I have Nginx reverse-proxying HTTPS requests to it.
You definitely need something like a production WSGI server such as Gunicorn, because the development server of Flask is meant for ease of development without much configuration for fine-tuning and optimization.
Eg. Gunicorn has a variety of configurations depending on the use case you are trying to solve. But the development flask server does not have these capabilities. In addition, these development servers show their limitations as soon as you try to scale and handle more requests.
With respect to needing a reverse proxy server such as Nginx is concerned it depends on your use case.
If you are deploying your application behind the latest load balancer in AWS such as an application load balancer(NOT classic load balancer), that itself will suffice for most use cases. No need to take effort into setting up NGINX if you have that option.
The purpose of a reverse proxy is to handle slow clients, meaning clients which take time to send the request. These reverse load balancers buffer the requests till the entire request is got from the clients and send them async to Gunicorn. This improves the performance of your application considerably.
I intend to use Kubernetes and Ingress for load balancing. I'm trying to learn how to set up Flask, uWSGI and Nginx.
I see this tutorial that has all three installed in the same container, and I'm wondering whether I should use it or not.
https://ianlondon.github.io/blog/deploy-flask-docker-nginx/
I'm guessing the benefit of having them as separate containers and separate pods is that they can then all scale individually?
But also, should Flask and uwsgi even be in separate containers? (or Flask and Gunicorn, since uwsgi seems to be very similar to Gunicorn)
Flask is a web framework, any application written with it needs a WSGI server to host it. Although you could use the Flask builtin developer server, you shouldn't as that isn't suitable for production systems. You therefore need to use a WSGI server such as uWSGI, gunicorn or mod_wsgi (mod_wsgi-express). Since the web application is hosted by the WSGI server, it can only be in the same container, but there isn't a separate process for Flask, it runs in the web server process.
Whether you need a separate web server such as nginx then depends. In the case of mod_wsgi you don't as it uses the Apache web server and so draws direct benefits from that. When using mod_wsgi-express it also is already setup to run in an optimal base configuration and how it does that avoids the need to have a separate front facing web server like people often do with nginx when using uWSGI or gunicorn.
For containerised systems, where the platform already provides a routing layer for load balancing, as is the case for ingress in Kubernetes, using nginx in the mix could just add extra complexity you don't need and could reduce performance. This is because you either have to run nginx in the same container, or create a separate container in the same pod and use shared emptyDir volume type to allow them to communicate via a UNIX socket still. If you don't use a UNIX socket, and use INET socket, or run nginx in a completely different pod, then it is sort of pointless as you are introducing an additional hop for traffic which is going to be more expensive than having it closely bound using a UNIX socket. The uWSGI server doesn't perform as well when accepting requests over INET when coupled with nginx, and having nginx in a separate pod, potentially on different host, can make that worse.
Part of the reason for using nginx in front is that it can protect you from slow clients due to request buffering, as well as other potential issues. When using ingress though, you already have a haproxy or nginx front end load balancer that can to a degree protect you from that. So it is really going to depend on what you are doing as to whether there is a point in introducing an additional nginx proxy in the mix. It can be simpler to just put gunicorn or uWSGI directly behind the load balancer.
Suggestions are as follows.
Also look at mod_wsgi-express. It was specifically developed with containerised systems in mind to make it easier, and can be a better choice than uWSGI and gunicorn.
Test different WSGI servers and configurations with your actual application with real world traffic profiles, not benchmarks which just overload it. This is important as the dynamics of a Kubernetes based system, along with how its routing may be implemented, means it all could behave a lot differently to more traditional systems you may be used to.
Setting up Flask with uWSGI and Nginx can be difficult. I tried following this DigitalOcean tutorial and still had trouble. Even with buildout scripts it takes time, and I need to write instructions to follow next time.
If I don't expect a lot of traffic, or the app is private, does it make sense to run it without uWSGI? Flask can listen to a port. Can Nginx just forward requests?
Does it make sense to not use Nginx either, just running bare Flask app on a port?
When you "run Flask" you are actually running Werkzeug's development WSGI server, and passing your Flask app as the WSGI callable.
The development server is not intended for use in production. It is not designed to be particularly efficient, stable, or secure. It does not support all the possible features of a HTTP server.
Replace the Werkzeug dev server with a production-ready WSGI server such as Gunicorn or uWSGI when moving to production, no matter where the app will be available.
The answer is similar for "should I use a web server". WSGI servers happen to have HTTP servers but they will not be as good as a dedicated production HTTP server (Nginx, Apache, etc.).
Flask documents how to deploy in various ways. Many hosting providers also have documentation about deploying Python or Flask.
First create the app:
import flask
app = flask.Flask(__name__)
Then set up the routes, and then when you want to start the app:
import gevent.pywsgi
app_server = gevent.pywsgi.WSGIServer((host, port), app)
app_server.serve_forever()
Call this script to run the application rather than having to tell gunicorn or uWSGI to run it.
I wanted the utility of Flask to build a web application, but had trouble composing it with other elements. I eventually found that gevent.pywsgi.WSGIServer was what I needed. After the call to app_server.serve_forever(), call app_server.stop() when to exit the application.
In my deployment, my application is listening on localhost:port using Flask and gevent, and then I have Nginx reverse-proxying HTTPS requests to it.
You definitely need something like a production WSGI server such as Gunicorn, because the development server of Flask is meant for ease of development without much configuration for fine-tuning and optimization.
Eg. Gunicorn has a variety of configurations depending on the use case you are trying to solve. But the development flask server does not have these capabilities. In addition, these development servers show their limitations as soon as you try to scale and handle more requests.
With respect to needing a reverse proxy server such as Nginx is concerned it depends on your use case.
If you are deploying your application behind the latest load balancer in AWS such as an application load balancer(NOT classic load balancer), that itself will suffice for most use cases. No need to take effort into setting up NGINX if you have that option.
The purpose of a reverse proxy is to handle slow clients, meaning clients which take time to send the request. These reverse load balancers buffer the requests till the entire request is got from the clients and send them async to Gunicorn. This improves the performance of your application considerably.
So from the Python/Flask docs, they both recommend not running the Flask web server as the production web server which makes sense. My question is, am I then able to run my Flask application on top of an Nginx server? Why do all the guides on the internet recommend wrapping Flask around uWSGI, Tornado, or some other WSGI server? What does it mean for something to be WSGI? Isn't Flask WGSI compliant?
I am particularly lost because here, the first response states:
Apache and Nginx are both HTTP servers.They can serve static files like
(.jpg and .html files) or dynamic pages (like a Wordpress blog or forum written in a language like PHP or Python).
However this post states:
Nginx is a web server. It serves static files, however it cannot execute and host
Python application. uWSGI fills that gap.
It just seems inefficient for my application to be handled by a server (ex: uWSGI) and then another server (ex: Nginx).
Nginx is a web server and is concerned with web server stuff, not with how to run Python programs. uWSGI is an application server and knows how to speak WSGI with Python (and other languages now). Both Nginx and uWSGI speak the uWSGI protocol, which is an efficient protocol over UNIX sockets.
Nginx deals with http requests from/responses to the outside world (possibly load balancing, caching, etc.). Your Flask application deals with WSGI requests/responses. uWSGI knows how to start your application (possibly with multiprocessing and/or threading) and bridge the gap between HTTP and WSGI.
There are other HTTP servers besides Nginx, and other WSGI servers besides uWSGI, but they all use the same workflow: the HTTP server passes to the WSGI server, which manages your application process and passes back to the HTTP server.
This setup is known as a reverse proxy. It allows each tool to do what it's good at and not be concerned about the other parts of the process. There is nothing particularly inefficient about it, until you get to truly massive scales.
Well, WSGI is a specification for interface between Python applications and web servers. uWSGI is (simply put) a realization of this specification written in C/C++. You can run almost any application on a "serious" webserver (like nginx) by just providing an entry point:
def application(env, start_response):
start_response('200 OK', [('Content-Type','text/html')])
return ["Hello World"]
and then running it like this:
uwsgi --http :9090 --wsgi-file file_with_the_code_above.py
You can return anything you want instead of ["Hello world"], of course. See http://uwsgi-docs.readthedocs.org/en/latest/WSGIquickstart.html for more.
When you use Django with mod_wsgi, what exactly happens when a user makes a request to the server from a browser? Does apache load up your Django app when it starts and have it running in a separate process? Does it create a new Python process for every HTTP request?
In embedded mode, the Django app is part of the httpd worker. In daemon mode, the Django app is a separate process and the httpd worker communicates with it over a socket. In either case, the WSGI interface is the same.