I'm working with web2py and for some reason web2py seems to fail to notice when code has changed in certain cases. I can't really narrow it down, but from time to time changes in the code are not reflected, web2py obviously has the old version cached somewhere.
The only thing that helps is quitting web2py and restarting it (i'm using the internal server).
Any hints ? Thank you !
web2py does cache your code, except for Google App Engine (for speed). That is not the problem. If you you edit code in models, views or controllers, you see the effect immediately.
The problem may be modules; if you edit code in modules you will not see the effect immediately, unless you import them with local_import('module', reload=True), or by restarting web2py.
Is that is also not your problem, then your browser is caching something. Please bring up this question to the web2py mailing list as we can help more.
P.S. If you are using the latest web2py it no longer comes with cherrypy. The built-in web server is called Rocket.
web2py itself shouldn't "cache" your code, but whatever app server you're using it on surely might. But web2py can be deployed on such a huge variety of app servers that it's impossible to give completely general suggestions.
If you're using the popular cherrypy WSGI server that I believe comes bundled with web2py, for example, see, in cherrypy's own docs, the AutoReload feature. Such features are not recommended in a production deployment (they can require very significant resources), but they sure come in handy when you're just developing!-)
Related
I have been looking at setting up a web server to use Python and I have installed Apache 2.2.22 on Debian 7 Wheezy with mod_wsgi. I have gotten the initial page up and going and the Apache will display the contents of the wsgi file that I have in my directory.
However, I have been researching on how to deploy a Python application and I have to admin, I find some of it a little confusing. I am coming from a background in PHP where it is literally install what you need and you are up and running and PHP is processing the way it should be.
Is this the same with Python? I can't seem to get anything to process outside of the wsgi file that I have setup. I can't import anything from other files without the server throwing a "500" error. I have looked on Google and Bing to try to find an answer to this, but I can't seem to find anything, or don't know that what I have been looking at is the answer.
I really appreciate any help that you guys can offer.
Thanks in advance! (If I need to post any coding, I can do that, I just don't know what you guys would need, if anything, as far as coding examples for this...)
Python is different from PHP in that PHP executes your entire program separately for each hit to your website, whereas Python runs "worker processes" that stay resident in memory.
You need some sort of web framework to do this work for you (you could write your own, but using someone else's framework makes it much easier). Flask is an example of a light one; Django is an example of a very heavy one. Pick one and follow that framework's instructions, or look for tutorials for that framework. Since the frameworks differ, most practical documentation on handling web services with Python are focused around a framework instead of just around the language itself.
Nearly any python web framework will have a development server that you can run locally, so you don't need to worry about deploying yet. When you are ready to deploy, Apache will work, although it's usually easier and better to use Gunicorn or another python-specific webserver, and then if you need more webserver functionality, set up nginx or Apache as a reverse proxy. Apache is a very heavy application to use for nothing but wsgi functionality. You also have the option of deploying to a PaaS service like Heroku (free for development work, costs money for production applications) which will handle a lot of sysadmin work for you.
As an aside, if you're not using virtualenv to set up your Python environment, you should look into it. It will make it much easier to keep track of what you have installed, to install new packages, and to isolate an environment so you can work on multiple projects on the same computer.
I never dug into how server tech like WSGI really worked underneath and thought I had a basic understanding until now...
What's the explanation for this type of behavior?: On an Apache2/WSGI/Django setup, after getting the new code on the dev server and "reloading" it by doing the prescribed touch myapp.wsgi, things started getting weird. On successive browser refreshes, I get either the old version of the app (from before pushing the new code) or the new one, RANDOMLY! It's like some threads/processes are still serving the old code while some have loaded the new code from the disk... What would be a simple explanation for this and how can I properly "reload" my app without restarting apache? Or where can I find a simple (better still, graphical/schematical) explanations of how things like WSGI, FCGI etc. work.
Note: I'm not a devops guy, but I've been forced into wrangling with things like this and I'm looking for any "condensed", "crash course type" knowledge on this, not the full fledged documentations for all the components...
You likely aren't using daemon mode. Read:
http://blog.dscpl.com.au/2012/10/why-are-you-using-embedded-mode-of.html
http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode
The Django documentation on mod_wsgi setup wasn't clear enough about what you had to do to use daemon mode. That has been fixed now.
https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/modwsgi/
I don’t want to burden you all with the details, but basically I’m a 2nd year compsci student with no Web dev experience.
Basically I want to create a small “web app” that takes in input from a html form, have a python script perform some calculations, and re-display those results in your browser.
As of right now, I have the form and script built. However, when I try to test the form, instead of running the script, my browser tries to download it. To my understanding, this is a cgi script problem, and that I must create a web server in order to test this script.
And heres were I’m stuck. I know little to nothing about web servers and how to set them up. On top of that I’ve heard that GCI scripting is a thing of the past, and requires major overhead in order to run properly.
This leads to my questions. How do I go about completing my app and testing my cgi script? Do I install apache and mess around with it or should I be looking into something like google app engine? Are there other ways to complete this task without cgi scripts? Where do frameworks like Django fit into this?
Django, while being nice, all-encompassing and well-supported, is sometimes too much for a small web application. Django wants you to play by its rules from the beginning, you'll have to avoid things like the database and admin panels if you don't need them. It's also easier, with Django, to follow its project layout, even when it's too complex for a simple app.
The so-called micro frameworks might suit you better for your small app. They are built upon the opposite principle: use the bare minimum of features now, add more as you need them.
Flask is based on Werkzeug WSGI library and Jinja2 templating (the latter is switchable), is extensively documented (with notes concerning virtualenv and stuff) and well-suited for small and larger apps alike. It comes bundled with an auto-reloading dev server (no need for Apache on your dev machine) and Werkzeug-powered interactive debugger. There are extensions for things like HTML forms and database ORM.
Bottle is as small as a microframework can get, consisting of 1 (one) file, dev server included. Drop it into your project folder and start hacking. The built-in SimpleTemplate templating engine is switchable, but the dev server is flakier in comparison to Flask's. The documentation is less complete, and, in my opinion, the whole thing is less polished and convenient as Flask.
In both cases, you use dev server locally, and then deploy using WSGI, the server interface for Python web apps which both frameworks support. There are many ways to deploy a WSGI app, Apache mod_wsgi being one of the popular ones.
I'd totally go with Flask unless one dependency (Bottle) is better than three (Flask, Jinja2 and Werkzeug).
(There are many other frameworks as well, so wait for their users to come and tell about them. I'd suggest to avoid web.py: it works, but is full of magic, and is inelegant compared to Flask or Bottle.)
One way of getting to working webapp quickly is by first understanding, and then modifying, something like the App Engine "guestbook" example. This has the benefit that much of the otherwise necessary tedium of running a web server and setting up a database server (assuming you need persistence) is done for you. App Engine also provides a fairly flexible development environment. It's certainly not the only way to go, and I'll admit to bias in recommending it, but it's fairly low friction.
GCI scripting is hardly a thing of the past, though it's not what the cool kids are doing. CGI has the benefit, and the curse, of exposing more of the raw plumbing. It forces you to understand a lot about primitive (in the low-level sense) web architecture, but it's also a bit of a large bite to chew on if you have an immediate problem to solve that can solved by simpler means.
It appears most python web development seems to be done by frameworks these days. There are a couple reasons for this:
a plethora of mature tools. Django has built in user auth, built in database management, built in sessions, built in just about everything ORM which lets you seamlessly supports a couple databases.
Built in webservers. The larger python frameworks like django and pylons have built in webservers. Django has a very simple webserver python manage.py startserver (that simple) That makes it extremely easy to create and debug applications. It is single threaded so dropping a debugger into it is painless
Huge communities. If you have a django question it will be answered very quickly the so community is huge.
The django tutorial will introduce you to all the major aspects of development. It is only 4 pages and you will be able to get your app going a lot simpler than having to read, learn and fiddle with an apache setup.
https://docs.djangoproject.com/en/dev/intro/tutorial01/
Although django for right now might be overkill if your app is just going to be 1 form and a script to process it. Because of its seamless testing framework it is quite easy to grow any project. I have never used flask or bottle or the other microframeworks, but I would keep in mind where your project will be in the future.
As for where django fits into this, it is a full stack framework encompassing presentation (templates), data management (server orm), authentication, middleware, forms ... everything necessary to create a completely inclusive web application. Django and almost all other python frameworks implement the wsgi standard. It is an interface that allows for interoperation between webservers. http://en.wikipedia.org/wiki/Web_Server_Gateway_Interface it is pretty dry and you will never have to interface it directly. That is what these frameworks do under the hood.
Why setup and maintain your own webserver if you can use app engine. It has an excellent SDK for testing your code. Here is an example https://developers.google.com/appengine/docs/python/gettingstarted/handlingforms
And Django you will find here : https://developers.google.com/appengine/docs/python/gettingstarted/templates
I prefer to use Jinja for templating.
Django comes with its own server, but in your case i would recommend apache and mod_python since it seems to be a rather simple site you're building.
Setting up Apache is a breeze and a simple search on the web should give you all you need.
You can find more information on mod_python here read up a little bit on it and then google after a tutorial that fits your needs.
Is it possible to eliminate the desktop and fully code and deploy a python/django application to the cloud from within a browser? I think Heroku makes it possible to do that with Ruby on Rails but I have not come across a Python/Django equivalent.
I think there are huge advantages of developing code in the cloud. No deployment overhead, no code versioning headaches (because the cloud will track all changes anyway), access your code from anywhere, even an iPad, if you want to make a small change to your production code. I think Heroku has already proven that this is the future of web development and browser based IDEs like Mozilla Bespin are already getting some traction.
I'm not really sure that Heroku does that, it just a cloud space for Rails.....like phpfog (php), dotcloud (many), appengine (java, python)....
I think http://kodingen.com/ is something more like what you ask, code in the web and deploy on it.....
Haven't tried Kodingen though....
As an updated answer to #jjchiw, Koding provides a you with Python2 and Python3, and gives you root access to a full Ubuntu VM, so everything will work just like you expect :)
I say updated, because Kodingen turned into Koding a year ago, and is under very active development.
Let me know if you have any questions :)
I am deploying a WSGI application. There are many ways to skin this cat. I am currently using apache2 with mod-wsgi, but I can see some potential problems with this.
So how can it be done?
Apache Mod-wsgi (the other mod-wsgi's seem to not be worth it)
Pure Python web server eg paste, cherrypy, Spawning, Twisted.web
as 2 but with reverse proxy from nginx, apache2 etc, with good static file handling
Conversion to other protocol such as FCGI with a bridge (eg Flup) and running in a conventional web server.
More?
I want to know how you do it, and why it is the best way to do it. I would absolutely love you to bore me with details about the whats and the whys, application specific stuff, etc.
As always: It depends ;-)
When I don't need any apache features I am going with a pure python webserver like paste etc. Which one exactly depends on your application I guess and can be decided by doing some benchmarks. I always wanted to do some but never came to it. I guess Spawning might have some advantages in using non blocking IO out of the box but I had sometimes problems with it because of the patching it's doing.
You are always free to put a varnish in front as well of course.
If an Apache is required I am usually going with solution 3 so that I can keep processes separate. You can also more easily move processes to other servers etc. I simply like to keep things separate.
For static files I am using right now a separate server for a project which just serves static images/css/js. I am using lighttpd as webserver which has great performance (in this case I don't have a varnish in front anymore).
Another useful tool is supervisord for controlling and monitoring these services.
I am additionally using buildout for managing my deployments and development sandboxes (together with virtualenv).
I would absolutely love you to bore me with details about the whats and the whys, application specific stuff, etc
Ho. Well you asked for it!
Like Daniel I personally use Apache with mod_wsgi. It is still new enough that deploying it in some environments can be a struggle, but if you're compiling everything yourself anyway it's pretty easy. I've found it very reliable, even the early versions. Props to Graham Dumpleton for keeping control of it pretty much by himself.
However for me it's essential that WSGI applications work across all possible servers. There is a bit of a hole at the moment in this area: you have the WSGI standard telling you what a WSGI callable (application) does, but there's no standardisation of deployment; no single way to tell the web server where to find the application. There's also no standardised way to make the server reload the application when you've updated it.
The approach I've adopted is to put:
all application logic in modules/packages, preferably in classes
all website-specific customisations to be done by subclassing the main Application and overriding members
all server-specific deployment settings (eg. database connection factory, mail relay settings) as class __init__() parameters
one top-level ‘application.py’ script that initialises the Application class with the correct deployment settings for the current server, then runs the application in such a way that it can work deployed as a CGI script, a mod_wsgi WSGIScriptAlias (or Passenger, which apparently works the same way), or can be interacted with from the command line
a helper module that takes care of above deployment issues and allows the application to be reloaded when the modules the application is relying on change
So what the application.py looks like in the end is something like:
#!/usr/bin/env python
import os.path
basedir= os.path.dirname(__file__)
import MySQLdb
def dbfactory():
return MySQLdb.connect(db= 'myappdb', unix_socket= '/var/mysql/socket', user= 'u', passwd= 'p')
def appfactory():
import myapplication
return myapplication.Application(basedir, dbfactory, debug= False)
import wsgiwrap
ismain= __name__=='__main__'
libdir= os.path.join(basedir, 'system', 'lib')
application= wsgiwrap.Wrapper(appfactory, libdir, 10, ismain)
The wsgiwrap.Wrapper checks every 10 seconds to see if any of the application modules in libdir have been updated, and if so does some nasty sys.modules magic to unload them all reliably. Then appfactory() will be called again to get a new instance of the updated application.
(You can also use command line tools such as
./application.py setup
./application.py daemon
to run any setup and background-tasks hooks provided by the application callable — a bit like how distutils works. It also responds to start/stop/restart like an init script.)
Another trick I use is to put the deployment settings for multiple servers (development/testing/production) in the same application.py script, and sniff ‘socket.gethostname()’ to decide which server-specific bunch of settings to use.
At some point I might package wsgiwrap up and release it properly (possibly under a different name). In the meantime if you're interested, you can see a dogfood-development version at http://www.doxdesk.com/file/software/py/v/wsgiwrap-0.5.py.
The absolute easiest thing to deploy is CherryPy. Your web application can also become a standalone webserver. CherryPy is also a fairly fast server considering that it's written in pure Python. With that said, it's not Apache. Thus, I find that CherryPy is a good choice for lower volume webapps.
Other than that, I don't think there's any right or wrong answer to this question. Lots of high-volume websites have been built on the technologies you talk about, and I don't think you can go too wrong any of those ways (although I will say that I agree with mod-wsgi not being up to snuff on every non-apache server).
Also, I've been using isapi_wsgi to deploy python apps under IIS. It's a less than ideal setup, but it works and you don't always get to choose otherwise when you live in a windows-centric world.
Nginx reverse proxy and static file sharing + XSendfile + uploadprogress_module. Nothing beats it for the purpose.
On the WSGI side either Apache + mod_wsgi or cherrypy server. I like to use cherrypy wsgi server for applications on servers with less memory and less requests.
Reasoning:
I've done benchmarks with different tools for different popular solutions.
I have more experience with lower level TCP/IP than web development, especially http implementations. I'm more confident that I can recognize a good http server than I can recognize a good web framework.
I know Twisted much more than Django or Pylons. The http stack in Twisted is still not up to this but it will be there.
I'm using Google App Engine for an application I'm developing. It runs WSGI applications.
Here's a couple bits of info on it.
This is the first web-app I've ever really worked on, so I don't have a basis for comparison, but if you're a Google fan, you might want to look into it. I've had a lot of fun using it as my framework for learning.
TurboGears (2.0)
TurboGears 2.0 is leaving Beta within the next month (has been in it for plenty of time). 2.0 improves upon 1.0 series and attempts to give you best-of-breed WSGI stack, so it makes some default choices for you, if you want the least fuss.
it has the tg* tools for testing and deployment in 1.x series, but now transformed to paster equivalents in 2.0 series, which shoud seem familiar if you've expermiented with pylons.
tg-admin quickstart —> paster quickstart
tg-admin info —> paster tginfo
tg-admin toolbox –> paster toolbox
tg-admin shell –> paster shell
tg-admin sql create –> paster setup-app development.ini
Pylons
It you'd like to be more flexible in your WSGI stack (choice of ORM, choice of templater, choice of form-ing), Pylons is becoming the consolidated choice. This would be my recommended choice, since it offers excellent documentation and allows you to experiment with different components.
It is a pleasure to work with as a result, and works on under Apache (production deployment) or stand-alone (helpful for testing and experimenting stage).
so it follows, you can do both with Pylons:
2 option for testing stage (python standalone)
4 for scalable production purposes (FastCGI, assuming the database you choose can keep up)
The Pylons admin interface is very similar to TurboGears. Here's a toy standalone example:
$ paster create -t pylons helloworld
$ cd helloworld
$ paster serve --reload development.ini
for production-class deployment, you could refer to the setup guide of Apache + FastCGI + mod_rewrite is available here. this would scale up to most needs.
Apache httpd + mod_fcgid using web.py (which is a wsgi application).
Works like a charm.
We are using pure Paste for some of our web services. It is easy to deploy (with our internal deployment mechanism; we're not using Paste Deploy or anything like that) and it is nice to minimize the difference between production systems and what's running on developers' workstations. Caveat: we don't expect low latency out of Paste itself because of the heavyweight nature of our requests. In some crude benchmarking we did we weren't getting fantastic results; it just ended up being moot due to the expense of our typical request handler. So far it has worked fine.
Static data has been handled by completely separate (and somewhat "organically" grown) stacks, including the use of S3, Akamai, Apache and IIS, in various ways.
Apache+mod_wsgi,
Simple, clean. (only four lines of webserver config), easy for other sysadimns to get their head around.