Everywhere i see that uWSGI and Gunicorn are recommended for production mode from everyone. However, there is a lot more suffering to operate with it, the python manage.py runserver is more faster, simpler, and the logging is also more visible if something goes wrong. Still, why not recommend the "python manage.py runserver" command for live production?
The runserver management command is optimized for different things from a web-server. Here are some things it does that are great for local development but would add unnecessary overhead in a production environment (source):
The development server automatically reloads Python code for each request, as needed
When you start the server, and each time you change Python code while the server is running, the system check framework will check your entire Django project for some common errors
Serves static files if the staticfiles contrib app is enabled (in a manner the docs describe as "grossly inefficient and probably insecure")
Meanwhile, production web-servers are designed to handle massively parallel workloads and are also under much higher security standards as they are the entry-point for all port 80/443 traffic to the server
Related
./manage.py runserver 0.0.0.0:8000
I am using the line above as part of the code I borrowed from github (https://github.com/ribeiroit/boh-puppet) to run bag of holding installation.
So far so good on http but not https. How do I modify the line above to incorporate https? I have already obtained ssl certificate from Comodo and updated my nginx conf.d file but the website won't display in https. Any ideas please shoot my way.
Thank you
While cezar's recommendation of django-extensions is valid to run a server with https, neither runserver or runserver_plus should ever be used in a production setting.
Quoting Django's documentation:
DO NOT USE THIS SERVER IN A PRODUCTION SETTING. It has not gone through security audits or performance tests. (And that’s how it’s gonna stay. We’re in the business of making Web frameworks, not Web servers, so improving this server to be able to handle a production environment is outside the scope of Django.)
Quoting django-extensions' documentation:
WARNING: This should never be used in any kind of production environment. Not even for a quick problem check. I cannot emphasize this enough. The interactive debugger allows you to evaluate python code right against the server. You’ve been warned.
Now yes, this warning from the django-extensions documentation is in reference to a single feature, the interactive console, which will theoretically not be exposed if DEBUG is set to False. But this is precisely how Patreon got hacked, and probably others as well. Why even risk it?
Instead, it would be far better to deploy your application using one of the officially recommended WSGI servers such as gunicorn or uWSGI.
For this purpose I use the the third-party app django-extensions, which offers some nice additional functionalities.
One of those extras is RunServerPlus.
You can then start the server like this:
python manage.py runserver_plus --cert-file /path/to/your/certificate
and open https://localhost:8000 in your browser.
./manage.py runserver 0.0.0.0:8000
I am using the line above as part of the code I borrowed from github (https://github.com/ribeiroit/boh-puppet) to run bag of holding installation.
So far so good on http but not https. How do I modify the line above to incorporate https? I have already obtained ssl certificate from Comodo and updated my nginx conf.d file but the website won't display in https. Any ideas please shoot my way.
Thank you
While cezar's recommendation of django-extensions is valid to run a server with https, neither runserver or runserver_plus should ever be used in a production setting.
Quoting Django's documentation:
DO NOT USE THIS SERVER IN A PRODUCTION SETTING. It has not gone through security audits or performance tests. (And that’s how it’s gonna stay. We’re in the business of making Web frameworks, not Web servers, so improving this server to be able to handle a production environment is outside the scope of Django.)
Quoting django-extensions' documentation:
WARNING: This should never be used in any kind of production environment. Not even for a quick problem check. I cannot emphasize this enough. The interactive debugger allows you to evaluate python code right against the server. You’ve been warned.
Now yes, this warning from the django-extensions documentation is in reference to a single feature, the interactive console, which will theoretically not be exposed if DEBUG is set to False. But this is precisely how Patreon got hacked, and probably others as well. Why even risk it?
Instead, it would be far better to deploy your application using one of the officially recommended WSGI servers such as gunicorn or uWSGI.
For this purpose I use the the third-party app django-extensions, which offers some nice additional functionalities.
One of those extras is RunServerPlus.
You can then start the server like this:
python manage.py runserver_plus --cert-file /path/to/your/certificate
and open https://localhost:8000 in your browser.
I have an Linux instance running on Google Compute Engine. I installed pip and django on it and cloned a Django project that I worked on locally. Like I would on localhost I ran my app like so: python3 manage.py runserver 0.0.0.0:8080, and my server was up and running with no problems. I read online on how WSGI servers are required for python apps to run well on servers however I don't see why I would need something like gunicorn to run my app
Here's what the documentation for runserver says:
DO NOT USE THIS SERVER IN A PRODUCTION SETTING. It has not gone through security audits or performance tests. (And that’s how it’s gonna stay. We’re in the business of making Web frameworks, not Web servers, so improving this server to be able to handle a production environment is outside the scope of Django.)
Django's runserver is itself a WSGI server, but it's aimed at being easy for developers to use.
Production WSGI servers like uWSGI and Gunicorn have performance and production environments in mind. They handle concurrency better, they are faster, and are built to withstand malicious users, not just developers.
I'm currently using screen and doing
sudo python manage.py runserver 0.0.0.0:80
Then closing the terminal. Seems like a bit of a hack. What is the correct way to do it?
runserver is a development server. You shouldn't use it in production, as explained at https://docs.djangoproject.com/en/1.4/ref/django-admin/#runserver-port-or-address-port where it says:
DO NOT USE THIS SERVER IN A PRODUCTION SETTING. It has not gone through security audits or performance tests. (And that's how it's gonna stay. We're in the business of making Web frameworks, not Web servers, so improving this server to be able to handle a production environment is outside the scope of Django.)
You should use one of the methods given in https://docs.djangoproject.com/en/1.4/howto/deployment/ for deploying a Django project in production. I have used mod_wsgi with Apache, gunicorn with nginx - the precise solution is up to you and the requirements of your project, but the deployment section of the Django manual goes through various options.
This really isn't a good idea; the built in development server shouldn't be used outside of your local development machine. Look at the docs:
DO NOT USE THIS SERVER IN A PRODUCTION SETTING. It has not gone through security audits or performance tests. (And that's how it's gonna stay. We're in the business of making Web frameworks, not Web servers, so improving this server to be able to handle a production environment is outside the scope of Django.)
Instead you should set up nginx+gunicorn/uwsgi or just apache+mod_wsgi.
If you ec2 instance is totally blocked from all possible communication from the outside world (which is unlikely) you can use the screen command
Good morning.
As the title indicates, I've got some questions about using python for web development.
What is the best setup for a development environment, more specifically, what webserver to use, how to bind python with it. Preferably, I'd like it to be implementable in both, *nix and win environment.
My major concern when I last tried apache + mod_python + CherryPy was having to reload webserver to see the changes. Is it considered normal? For some reason cherrypy's autoreload didn't work at all.
What is the best setup to deploy a working Python app to production and why? I'm now using lighttpd for my PHP web apps, but how would it do for python compared to nginx for example?
Is it worth diving straight with a framework or to roll something simple of my own? I see that Django has got quite a lot of fans, but I'm thinking it would be overkill for my needs, so I've started looking into CherryPy.
How exactly are Python apps served if I have to reload httpd to see the changes? Something like a permanent process spawning child processes, with all the major file includes happening on server start and then just lazy loading needed resources?
Python supports multithreading, do I need to look into using that for a benefit when developing web apps? What would be that benefit and in what situations?
Big thanks!
What is the best setup for a development environment?
Doesn't much matter. We use Django, which runs in Windows and Unix nicely. For production, we use Apache in Red Hat.
Is having to reload webserver to see the changes considered normal?
Yes. Not clear why you'd want anything different. Web application software shouldn't be dynamic. Content yes. Software no.
In Django, we develop without using a web server of any kind on our desktop. The Django "runserver" command reloads the application under most circumstances. For development, this works great. The times when it won't reload are when we've damaged things so badly that the app doesn't properly.
What is the best setup to deploy a working Python app to production and why?
"Best" is undefined in this context. Therefore, please provide some qualification for "nest" (e.g., "fastest", "cheapest", "bluest")
Is it worth diving straight with a framework or to roll something simple of my own?
Don't waste time rolling your own. We use Django because of the built-in admin page that we don't have to write or maintain. Saves mountains of work.
How exactly are Python apps served if I have to reload httpd to see the changes?
Two methods:
Daemon - mod_wsgi or mod_fastcgi have a Python daemon process to which they connect. Change your software. Restart the daemon.
Embedded - mod_wsgi or mod_python have an embedded mode in which the Python interpreter is inside the mod, inside Apache. You have to restart httpd to restart that embedded interpreter.
Do I need to look into using multi-threaded?
Yes and no. Yes you do need to be aware of this. No, you don't need to do very much. Apache and mod_wsgi and Django should handle this for you.
So here are my thoughts about it:
I am using Python Paste for developing my app and eventually also running it (or any other python web server). I am usually not using mod_python or mod_wsgi as it makes development setup more complex.
I am using zc.buildout for managing my development environment and all dependencies together with virtualenv. This gives me an isolated sandbox which does not interfere with any Python modules installed system wide.
For deployment I am also using buildout/virtualenv, eventually with a different buildout.cfg. I am also using Paste Deploy and it's configuration mechanism where I have different config files for development and deployment.
As I am usually running paste/cherrypy etc. standalone I am using Apache, NGINX or maybe just a Varnish alone in front of it. It depends on what configuration options you need. E.g. if no virtual hosting, rewrite rules etc. are needed, then I don't need a full featured web server in front. When using a web server I usually use ProxyPass or some more complex rewriting using mod_rewrite.
The Python web framework I use at the moment is repoze.bfg right now btw.
As for your questions about reloading I know about these problems when running it with e.g. mod_python but when using a standalone "paster serve ... -reload" etc. it so far works really well. repoze.bfg additionally has some setting for automatically reloading templates when they change. If the framework you use has that should be documented.
As for multithreading that's usually used then inside the python web server. As CherryPy supports this I guess you don't have to worry about that, it should be used automatically. You should just eventually make some benchmarks to find out under what number of threads your application performs the best.
Hope that helps.
+1 to MrTopf's answer, but I'll add some additional opinions.
Webserver
Apache is the webserver that will give you the most configurability. Avoid mod_python because it is basically unsupported. On the other hand, mod_wsgi is very well supported and gives you better stability (in other words, easier to configure for cpu/memory usage to be stable as opposed to spikey and unpredictable).
Another huge benefit, you can configure mod_wsgi to reload your application if the wsgi application script is touched, no need to restart Apache. For development/testing servers you can even configure mod_wsgi to reload when any file in your application is changed. This is so helpful I even run Apache+mod_wsgi on my laptop during development.
Nginx and lighttpd are commonly used for webservers, either by serving Python apps directly through a fastCGI interface (don't bother with any WSGI interfaces on these servers yet) or by using them as a front end in front of Apache. Calls into the app get passed through (by proxy) to Apache+mod_wsgi and then nginx/lighttpd serve the static content directly.
Nginx has the added advantage of being able to serve content directly from memcached if you want to get that sophisticated. I've heard disparaging comments about lighttpd and it does seem to have some development problems, but there are certainly some big companies using it successfully.
Python stack
At the lowest level you can program to WSGI directly for the best performance. There are lots of helpful WSGI modules out there to help you in areas you don't want to develop yourself. At this level you'll probably want to pick third-party WSGI components to do things like URL resolving and HTTP request/response handling. A great request/response component is WebOb.
If you look at Pylons you can see their idea of "best-of-breed" WSGI components and a framework that makes it easier than Django to choose your own components like templating engine.
Django might be overkill but I don't think that's a really good argument against. Django makes the easy stuff easier. When you start to get into very complicated applications is where you really need to look at moving to lower level frameworks.
Look at Google App Engine. From their website:
Google App Engine lets you run your
web applications on Google's
infrastructure. App Engine
applications are easy to build, easy
to maintain, and easy to scale as your
traffic and data storage needs grow.
With App Engine, there are no servers
to maintain: You just upload your
application, and it's ready to serve
your users.
You can serve your app using a free
domain name on the appspot.com domain,
or use Google Apps to serve it from
your own domain. You can share your
application with the world, or limit
access to members of your
organization.
App Engine costs nothing to get
started. Sign up for a free account,
and you can develop and publish your
application for the world to see, at
no charge and with no obligation. A
free account can use up to 500MB of
persistent storage and enough CPU and
bandwidth for about 5 million page
views a month.
Best part of all: It includes Python support, including Django. Go to http://code.google.com/appengine/docs/whatisgoogleappengine.html
When you use mod_python on a threaded Apache server (the default on Windows), CherryPy runs in the same process as Apache. In that case, you almost certainly don't want CP to restart the process.
Solution: use mod_rewrite or mod_proxy so that CherryPy runs in its own process. Then you can autoreload to your heart's content. :)