Is using Django with gunicorn is considered to be a replacement for using evented/async servers like Tornado, Node.js, and similar ? Additionally, Will that be helpful in handling long-polling/cometed services?
Finally, is Gunicorn only replacing the memory consuming Apache threads (in case of Apache/mod-wsgi) with lightweight threads, or there are an additional benefits?
Gunicorn by default will spawn regular synchronous WSGI processes. You can however tell it to spawn processes that use gevent, eventlet or tornado instead. I am only familiar with gevent which can certainly be used instead of Node.js for long polling.
The memory footprint per process is about the same for mod_wsgi and gunicorn (in my limited experience), but you get more bells-and-whistles with gunicorn. If you change the default worker class to gevent (or eventlet or tornado) you also get a LOT more performance out of each process.
Related
Does Gunicorn have a separate worker like the uwsgi mules that I can use to offload tasks? I need at least 4 workers dedicated to certain logics. I searched the documentation but I can't find anything that resembles them
I love uwsgi but I found out the hard way that you can bring it down with only one request that has multiple headers(can't limit the size too much bc of my program) so i need to migrate to Gunicorn
My setup is flask-socketio with a flask-restful webserver.
Eventlet is installed, so in production mode, eventlet webserver is used.
I understand flask-socketio and eventlet webserver themselves are event-loop based.
Does flask-socketio and eventlet webserver runs on the same eventloop (same thread) or in two different threads?
I think you are confusing the terminology.
The event loop is the task scheduler. This is provided by eventlet, and a single event loop is used for the whole application, including the Flask and the Flask-SocketIO parts.
Each time a request arrives to the eventlet web server, it will allocate a new task for it. So basically each request (be it Flask or Flask-SocketIO, HTTP or WebSocket) will get its own task. Tasks are constantly being created and destroyed as requests are handled.
When you use eventlet, tasks are not threads, they are greenlets, that is why I avoided calling them threads above and used the more generic "task" term. They behave like threads in many ways, but they are not.
I have a Python web application (using WSGI) deployed on Openshift. The application is quite memory greedy. What I have noticed is that there are several instance of Apache httpd service deployed at all times. That means the memory usage of my gear is multiplied by the number of these processes and the application crashes pretty often.
I don't have lots of traffic yet, so there is no need to have multiple httpd running.
Is there any way to configure Python cartridge to limit it to a single httpd process?
If you are using the OpenShift Python cartridge and its default setup, only two of those processes should actually have copies of your application running in it. The other httpd processes are the parent monitor process and the Apache child worker processes which will proxy requests to the processes which are actually running your web application.
If you need control to reduce it down to one process, then you would need to follow:
http://blog.dscpl.com.au/2015/01/using-alternative-wsgi-servers-with.html
to override the standard setup and use mod_wsgi-express instead. This will default to using one process for your application and allow you to control both number of processes and threads for the application processes.
If you are seeing lots of memory use, then it could just be your application code, or there is an outside chance you are seeing memory issues due to use of older mod_wsgi as there are some odd corner cases which can cause extra memory usage because of how Apache works. If you use mod_wsgi-express it will use the latest and avoid those problems.
So try mod_wsgi-express and if still have memory issues, suggest you get on the mod_wsgi mailing list to get help debugging it.
I've been able to deploy a test application by using pyramid with pserve and running pceleryd (I just send an email without blocking while it is sent).
But there's one point that I don't understand: I want to run my application with mod_wsgi, and I don't understand if I can can do it without having to run pceleryd from a shell, but if I can do something in the virtualhost configuration.
Is it possible? How?
There are technically ways you could use Apache/mod_wsgi to manage a process distinct from that handling web requests, but the pain point is that Celery will want to fork off further worker processes. Forking further processes from a process managed by Apache can cause problems at times and so is not recommended.
You are thus better of starting up Celery process separately. One option is to use supervisord to start it up and manage it.
Can someone explain the difference between apache mod_wsgi in daemon mode and django fastcgi in threaded mode. They both use threads for concurrency I think.
Supposing that I'm using nginx as front end to apache mod_wsgi.
UPDATE:
I'm comparing django built in fastcgi(./manage.py method=threaded maxchildren=15) and mod_wsgi in 'daemon' mode(WSGIDaemonProcess example threads=15). They both use threads and acquire GIL, am I right?
UPDATAE 2:
So if they both are similar, is there any benefits of apache mod_wsgi against fastcgi. I see such pros of fastcgi:
we don't need apache
we consume less RAM
I noticed that fastcgi has lesser overhead
UPDATAE 3:
I'm now happy with nginx + uwsgi.
UPDATAE 4:
I'm now happy with nginx + gunicorn :)
Neither have to use threads to be able to handle concurrent requests. It depends on how you configure them. You can use multiple processes where each is single threaded if you want.
For more background on mod_wsgi process/threading models see:
http://code.google.com/p/modwsgi/wiki/ProcessesAndThreading
The models are similar albeit that mod_wsgi handles process management itself. What happens in FASTCGI as far as process management depends on what FASTCGI hosting mechanism you are using and you don't say what that is.
Another difference is that FASTCGI still needs a separate FASTCGI to WSGI bridge such as flup where as mod_wsgi doesn't need any sort of bridge as implements WSGI interface natively.
Finally, FASTCGI process are an exec/fork of some supervisor process or the web server, dependent on hosting mechanism. In mod_wsgi the processes are a fork only of Apache parent process. In general this doesn't matter too much but does have some implications.
There are other differences but they arise more because mod_wsgi offers a lot more functionality and configurability than a FASTCGI hosting mechanism does.
Anyway, the question is a bit vague, can you be more specific about what it is you are wanting to know or contrast between the two and why? Answer can then perhaps be targeted better.