I've pretty much tried every Python web framework that exists, and it took me a long time to realize there wasn't a silver bullet framework, each had its own advantages and disadvantages. I started out with Snakelets and heartily enjoyed being able to control almost everything at a lower level without much fuss, but then I discovered TurboGears and I have been using it (1.x) ever since. Tools like Catwalk and the web console are invaluable to me.
But with TurboGears 2 coming out which brings WSGI support, and after reading up on the religious debates between the Django and WSGI camps, I'm really torn between "doing it the right way", e.g., learning WSGI, spending valuable time writing functionality that already exists in Django and other full-stack frameworks, as opposed to using Django or some high-level framework that does everything for me. The downsides with the latter that I can see are pretty obvious:
I'm not learning anything in the process
If I ever need to do anything lower level it's going to be a pain
The overhead required for just a basic site which uses authentication is insane. (IMO)
So, I guess my question is, which is the better choice, or is it just a matter of opinion, and should I suck it up and use Django if it achieves what I want with minimal fuss (I want authentication and a CRUD interface to my database)? I tried Werkzeug, Glashammer, and friends, but AuthKit and Repoze scared me off, as well as the number of steps involved to just setup basic authentication. I looked at Pylons, but the documentation seems lacking, and when referencing simple features like authentication or a CRUD interface, various wiki pages and documentation seemed to contradict each other, with different hacks for versions and such.
Thanks to S. Lott for pointing out that I wasn't clear enough. My question is: which of the following is worthwhile in the long run, but not painful in the short (e.g., some sort of middle ground, anyone?) - Learn WSGI, or stick with a "batteries-included" framework? If the latter, I would appreciate a suggestion as to whether I should give Django another try, stick with TurboGears 1.x, or venture into some other framework.
Also, I have tried CherryPy, but couldn't seem to find a good enough CRUD application that I could plop in and use right away.
the religious debates between the Django and WSGI camps
It would seem as though you're a tad bit confused about what WSGI is and what Django is. Saying that Django and WSGI are competing is a bit like saying that C and SQL are competing: you're comparing apples and oranges.
Django is a framework, WSGI is a protocol (which is supported by Django) for how the server interacts with the framework. Most importantly, learning to use WSGI directly is a bit like learning assembly. It's a great learning experience, but it's not really something you should do for production code (nor was it intended to be).
At any rate, my advice is to figure it out for yourself. Most frameworks have a "make a wiki/blog/poll in an hour" type exercise. Spend a little time with each one and figure out which one you like best. After all, how can you decide between different frameworks if you're not willing to try them out?
I'd say you're being a bit too pessimistic about "not learning anything" using Django or a similar full-stack framework, and underestimating the value of documentation and a large community. Even with Django there's still a considerable learning curve; and if it doesn't do everything you want, it's not like the framework code is impenetrable.
Some personal experience: I spent years, on and off, messing around with Twisted/Nevow, TurboGears and a few other Python web frameworks. I never finished anything because the framework code was perpetually unfinished and being rewritten underneath me, the documentation was often nonexistent or wrong and the only viable support was via IRC (where I often got great advice, but felt like I was imposing if I asked too many questions).
By comparison, in the past couple of years I've knocked off a few sites with Django. Unlike my previous experience, they're actually deployed and running. The Django development process may be slow and careful, but it results in much less bitrot and deprecation, and documentation that is actually helpful.
HTTP authentication support for Django finally went in a few weeks ago, if that's what you're referring to in #3.
I suggest taking another look at TG2. I think people have failed to notice some of the strides that have been made since the last version. Aside from the growing WSGI stack of utilities available there are quite a few TG2-specific items to consider. Here are a couple of highlights:
TurboGears Administration System - This CRUD interface to your database is fully customizable using a declarative config class. It is also integrated with Dojo to give you infinitely scrollable tables. Server side validation is also automated. The admin interface uses RESTful urls and HTTP verbs which means it would be easy to connect to programatically using industry standards.
CrudRestController/RestController - TurboGears provides a structured way to handle services in your controller. Providing you the ability to use standardized HTTP verbs simply by extending our RestController. Combine Sprox with CrudRestController, and you can put crud anywhere in your application with fully-customizable autogenerated forms.
TurboGears now supports mime-types as file extensions in the url, so you can have your controller render .json and .xml with the same interface it uses to render html (returning a dictionary from a controller)
If you click the links you will see that we have a new set of documentation built with sphinx which is more extensive than the docs of the past.
With the best web server, ORM, and template system(s) (pick your own) under the hood, it's easy to see why TG makes sense for people who want to get going quickly, and still have scalability as their site grows.
TurboGears is often seen as trying to hit a moving target, but we are consistent about releases, which means you won't have to worry about working out of the trunk to get the latest features you need. Coming to the future: more TurboGears extensions that will allow your application to grow functionality with the ease of paster commands.
Your question seems to be "is it worth learning WSGI and doing everything yourself," or using a "full stack framework that does everything for you."
I'd say that's a false dichotomy and there's an obvious third way. TurboGears 2 tries to provide a smooth path from a "do everything for you" style framework up to an understanding of WSGI middleware, and an ability to customize almost every aspect of the framework to suit your application's needs.
We may not be successful in every place at every level, but particularly if you've already got some TurboGears 1 experience I think the TG2 learning curve will be very, very easy at first and you'll have the ability to go deeper exactly when you need it.
To address your particular issues:
We provide an authorization system out of the box that matches the one you're used to from TG1.
We provide an out of the box "django admin" like interface called the tgext.admin, which works great with dojo to make a fancy spreadsheet like interface the default.
I'd also like to address a couple of the other options that are out there and talk a little bit about the benifits.
CherryPy. I think CherryPy is a great webserver and a nice minimalistic web-framework. It's not based on WSGI internally but has good WSGI support although it will not provide you with the "full stack" experience. But for custom setups that need to be both fast and aren't particularly suited to the defaults provided by Django or TurboGears, it's a great solution.
Django. I think Django is a very nice, tigtly integrated system for developing websites. If your application and style of working fits well within it's standard setup it can be fantastic. If however you need to tune your DB usage, replace the template language, use a different user authorization model or otherwise do things differently you may very likely find yourself fighting the framework.
Pylons Pylons like CherryPy is a great minimalistic web-framework. Unlike CherryPy it's WSGI enabled through the whole system and provides some sane defaults like SQLAlchemy and Mako that can help you scale well. The new official docs are of much better quality than the old wiki docs which are what you seem to have looked at.
Have you taken a look at CherryPy. It is minimalistic, yet efficient and simple. It is low level enough for not it to get in they way, but high enough to hide complexity. If I remember well, TurboGears was built on it.
With CherryPy, you have the choice of much everything. (Template framework, ORM if wanted, back-end, etc.)
Learn WSGI
WSGI is absurdly simple.. It's basically a function that looks like..
def application(environ, start_response) pass
The function is called when an HTTP request is received. environ contains various data (like the request URI etc etc), start_response is a callable function, used to set headers.
The returned value is the body of the website.
def application(environ, start_response):
start_response("200 OK", [])
return "..."
That's all there is to it, really.. It's not a framework, but more a protocol for web-frameworks to use..
For creating sites, using WSGI is not the "right way" - using existing frameworks is.. but, if you are writing a Python web-framework then using WSGI is absolutely the right way..
Which framework you use (CherryPy, Django, TurboGears etc) is basically personal preference.. Play around in each, see which you like the most, then use it.. There is a StackOverflow question (with a great answer) about this, "Recommendation for straight-forward python frameworks"
Have you checked out web2py? After recently evaluating many Python web frameworks recently I've decided to adopt this one. Also check out Google App Engine if you haven't already.
I'd say the correct answer depends on what you actually want and need, as what will be worthwhile in the long run depends on what you'll need in the long run. If your goal is to get applications deployed ASAP then the 'simpler' route, ie. Django, is surely the way to go. The value of a well-tested and well-documented system that exactly what you want can't be underestimated.
On the other hand if you have time to learn a variety of new things which may apply in other domains and want to have the widest scope for customisation then something like Turbogears is superior. Turbogears gives you maximum flexibility but you will have to spend a lot of time reading external docs for things like Repoze, SQLAlchemy, and Genshi to get anything useful done with it. The TG2 docs are deliberately less detailed than the TG1 docs in some cases because it's considered that the external docs are better than they used to be. Whether this sort of thing is an obstacle or an investment depends on your own requirements.
Django is definitely worth learning, and sounds like it will fit your purposes. The admin interface it comes with is easy to get up and running, and it does use authentication.
As for "anything lower level", if you mean sql, it is entirely possible to shove sql into you queries with the extra keyword. Stylistically, you always try to avoid that as much as possible.
As for "not learning anything"...the real question is whether your preference is to be primarily learning something lower-level or higher-level, which is hardly a question anyone here can answer for you.
Pylons seems a great tool for me:
a real web framework (CherryPy is just a web server),
small code base - reuse of other projects,
written entirely with WSGI in mind, based on Paste,
allows you to code the app right away and touch the low level bits if it's necessary,
I've used CherryPy and TurboGears and look at many other frameworks but none of them were so light and productive as Pylons is. Check the presentation at Google.
I'm a TurboGears fan, and this is exactly the reason why: a very nice trade-off between control and doing things right vs. easy.
You'll have to make up your own mind of course. Maybe you'd prefer to learn less, maybe more. Maybe the areas that I like knowledge/control (database for example), you couldn't care less about. And don't misunderstand. I'm not characterizing any frameworks as necessarily hard or wrong. It's just my subjective judgment.
Also I would recommend TurboGears 2 if at all possible. When it comes out, I think it will be much better than 1.0 in terms of what it has selected for defaults (genshi, pylons, SqlAlchemy)
I would suggest for TurboGears 2. They have done a fantastic job of integrating best of Python world.
WSGI: Assuming you are developing moderately complex projects/ business solutions in TG2 or some other framework say Grok. Even though these frameworks supports WSGI does that mean one who is using these frameworks have to learn WSGI? In most cases answer is No. I mean it's good have this knowledge no doubt.
WSGI knowledge is probably is more useful in cases like
you want to use some middleware or some other component which is not provided as part of the standard stack for eg. Authkit with TG or Grok without ZODB.
you are doing some integration.
CherryPy is good but think of handling your database commits/rollbacks at the end of transactions, exposing json, validations in such cases TG, Django like frameworks do it all for you.
Web2py is the secret sauce here. Don't miss checking it out.
Related
I have no background in web applications, but have a fairly experience background in C++, and a quick learner.
I have spent some time learning Python and reading through SQLAlchemy. I kind of like the idea of coding in pure Python OO, and then use a nice SQLAlchemy mapper to persist everything. I like this decoupled approach (using pure Python classes along a mapper function to talk to DB) better than the ActiveRecord idea of Rails. I think eventually I would have more control over connecting the DB to the app. (I need to work with a DB that is updated by a background process. Something like a web crawler that fills the DB.)
At the same time, some stuff makes me think again about Rails. Like streamlined Email and Ajax handling in Rails.
Am I thinking the right way, that Rails is less flexible for Form Validation Manipulations, and working with DB? And is it harder in Pylons to handle Email (notifications), RSS, Ajax?
What would you suggest?
Thanks
Have a look at Django. It sounds like this exactely what you are looking for :-).
Have a look at these Python frameworks:
Django: Probably the single most popular Python framework, but for better (and worse) is very-much a full-stack solution.
Pylons: As reaction to the Django One Way of working, Pylons, for better (and worse), uses a much looser binding of the modules that make up your framework.
TurboGears: As an attempt at a happy medium between Django and Pylons, TurboGears is based on Pylons, but comes ready made with certain component choices, and the glue to hold it together.
Zope: Zope is more of an application server and framework than a "web framework". It just happens to be web based.
The first three are all inspired by Ruby on Rails, but each has it's own ideas for improvement. Zope predates Rails, and is it's own world.
I've used TurboGears to develop a few small apps. Kinda nice. At the time, their docs were kinda bad. I hope that's changed.
I've also directly used Python Paste a few times. Paste is the HTTP server base upon which Pylons, and in turn TurboGears, is based. Again very nice.
Also: When given the choice, I've always used SQLAlchemy as an ORM. It's truly an impressive piece of software that I've used for even non-web projects.
Hope this helps. Let us know what path you take. :-)
Rails is written in Ruby, not Python. If you have your heart set on Python, then go with Django. But please give Rails a fair shake; ActiveRecord is not the only ORM available either. I use DataMapper for some apps too. I may be biased, but I'm inclined to think the Rails ecosystem is bigger than that of Django too.
If you are looking for a project similar to rails, you should check out Masonite, A modern and developer-centric web framework in Python
You should also checkout web2py instead of Django. Just an alternative you might consider.
Here's two "A vs. B" articles and discussions regarding the two:
Django vs. web2py
Reddit.com community discussion
I wrote a few websites in Pylons over the years and I like it a lot. The great things about Pylons is that it consists mostly of third party libraries. That means that you're learning many useful libraries that can be used in you other projects, for example SQLAlchemy, WebOb, FormEncode, Beaker, Mako and so on ... Especially SQLAlchemy and Beaker are extremely useful in pretty much any context.
You could also have a look to Nagare, another full stack framework.
Some of Nagare based projects already in production can be found on the Nagare web site.
I used Web2Py for many small projects, including many goodies such as the "Workers" & Scheduler concepts, some event-driven updates in web page through the short tornado example in websocket_messaging.py. If you're looking for a small but powerful development framework that includes a small DB and display tables, it's just amazing. You even do not need to write a single HTML line. I do not see any competitor in this area. In my opinion it's far easier and faster than django, but django might provide much freedom in complex apps.
I have to develop a site which has to accomodate around 2000 users a day and speed is a criterion for it. Moreover, the site is a user oriented one where the user will be able to log in and check his profile, register for specific events he/she wants to participate in. The site is to be hosted on a VPS server.Although I have pretty good experience with python and PHP but I have no idea how to use either of the framework. We have plenty of time to experiment and learn one of the above frameworks.Could you please specify which one would be preferred for such a scenario considering speed, features, and security of the site.
Thanks,
niting
This is a very subjective question but personally I'd recommend Django. Python is a very nice language to use and the Django framework is small, easy to use, well documented and also has a pretty active community.
This choice was made partly because of my dislike for PHP though, so take the recommendation with a pinch of salt.
Most of the frameworks out there nowadays are fast enough to serve whatever needs you will have. It really depends on in which environment you feel most comfortable. Though there are nuances here and there, MVC frameworks share a lot of the same principles, so whichever you choose to use is really a matter of which you most enjoy using.
So, if you like Python more, there's your answer. Use a Python framework, and Django is the best. If you like PHP more (which I personally don't), you've got some more decisions to make. But any of the PHP frameworks are fine. They really are. Just pick one that looks nice with comprehensive documentation and get to work.
I've worked with CakePHP and Django and I really recommend Django. I don't know too much about CodeIgniter, but I remember ruling it out when I was evaluating frameworks myself about a year ago. CakePHP seemed much more developed at the time.
First of all, the Django community is much bigger and has spent a lot of time focusing on reusable apps. This means that you get a lot of functionality for free. Pair this with the django admin, and you have a lot of things already done for you. I haven't kept up with the PHP frameworks much, but I'm pretty sure Django is also more developed.
This is more of a personal thing, but I just like Python over PHP. Compare the way models are done in CakePHP and Django: http://book.cakephp.org/view/67/Understanding-Models, http://docs.djangoproject.com/en/dev/topics/db/models/#topics-db-models. The python is clearly more readable.
Keep in mind that Django gives you an awesome ORM and builds your schema for you, i.e. you never have to touch the database if you don't want to. With the PHP frameworks, you have to do your own db design, which just slows me down at this point. You can always go in and add indexes for speed later.
This is probably the most biased, but if you are starting a new application - seriously - just stick with Django or Ruby on Rails. There is a reason everyone talks about them and they have the biggest communities and best developers behind them.
You can also check out Pinax for a lot of Django goodies.
Codeigniter it's fast and very documented also has a large community to and finaly friendly with the programmer.
CodeIgniter is a great PHP framework that is fast and has excellent documentation. Start reading through their user guide and it will give you a good idea how to work with the framework.
Extending Matchu:
Or, -If you like PHP more- its time to learn/growup about other things like Python. Its not hard to learn, and when you get started it gets very enjoyable.
Many people has done the PHPtoPython/Django port, like Mozilla, Netgeo, Nasa, TheOnion, etc.
If for the PHP part I would choose CodeIgniter - it doesn't get too much into your way. But it doesn't have any code/view/model generators out of the box, you need to type a bit.
But languages other than PHP appear to be more sexy.
I am using CodeIgniter 1.7.2 and for complex websites it's very good and powerfull, but it definitely is missing some kind of code generator which will allow for example to build an IT application in one click.
I had the impression (from watching a tutorial) that Django has it.
Django is my favorite python web framework. I've tried out others like pylons, web2py, nevow and others.
But I've never looked into TurboGears with much enthusiasm.
Now with TG2 out of beta I may give it a try. I'd like to know what are some of the pros and cons compared to Django.
TG2 has several advantages that I think are important:
Multi-database support
sharding/data partitioning support
longstanding support for aggregates, multi-column primary keys
a transaction system that handles multi-database transactions for you
an admin system that works with all of the above
out of the box support for reusable template snipits
an easy method for creating reusable template tag-libraries
more flexibility in using non-standard components
There are more, but I think it's also important to know that Django has some advantages over TG2:
Larger, community, more active IRC channel
more re-usable app-components
a bit more developed documentation
All of this means that it's a bit easier to get started in Django than TG2, but I personally think the added power and flexibility that you get is worth it. But your needs may always be different.
TG2 takes Pylons and changes some defaults - object dispatching instead of Routes, and Genshi instead of Mako. They believe there's only one way to do it, so apps can rely on the same API for any TurboGears website.
Similarities
TG2 and Django both distinguish between websites and components, so you'll eventually see reusable building blocks for TurboGears, too.
Differences
Django uses its own handlers for HTTP, routing, templating, and persistence. Django also has stellar documentation and an established community.
TurboGears defaults to best-of-breed libraries, which apparently are Paste, object dispatching, Genshi, and SqlAlchemy. This philosophy produces a better all-round toolset, but at the risk of instability - because it means throwing away backwards compatibility if and when better libraries appear.
Pros.
SQLAlchemy > django ORM
Multiple template languages out of the box (genshi,mako,jinja2)
more WSGI friendly
Object Dispatch > routes > regexp routing. You can get the first 2 with TG2
Almost all components are optional you can keep the core and use any ORM, template, auth library, etc.
Sprox > django forms
Cons.
- Admin is more basic (no inline objects yet!)
- less third party apps
- "app" system still in the making.
- given it's modularity you need to read documentation from different sources (SQLAlchemy, Genshi or Mako, repoze.who, Pylons, etc.)
I was struggling with the same question months ago and decided for Turbogears 2, and my reasoning was simple. "I'm new to python, I want to learn it not just for web-projects but as a substitute to php for scripting small helpers"
What I didn't like about Django, to me looks like a "close platform". ORM, Template system, sessions, etc they all are Django's
On the other hand, Turbogears 2 uses already known open platforms and just glued them, just like Appfuse does it for Java
With TurboGears 2 I learn SQLAlchemy that I can use later for small python scripts, or from the python shell to solve common tasks.
Main drawbacks are the lack of complete documentation and error messages.
Sometimes you have to search very deep to find simple solutions, the learning curve is steep, but it pays long term. The error messages where to me very confusing (coming from more than 10 years in Java development). I had lost many hours trying to find an "ascii encode error" when the real problem was a module not being imported.
That's my opinion, just remember I'm new to python and I could be wrong about many things stated here.
Besides what Nikhil gave in his answer, I think another minor difference is that Turbogears provdes some support for javascript widgets and integration with Mochikit.
Whereas Django steadfastly remains javascript framework neutral.
(At least this was true with older versions of Turbogears... this might have changed with TG2)
Edit: I just went over TG2 documentation and see that it did indeed change. Turbogears now uses ToscaWidgets which can use jQuery, ExtJS, Dojo, etc. underneath. This nicely makes it more framework neutral while still providing nice javascript widgets.
This strikes me as a pro for Turbogears if you don't have any javascript experience and a pro for Django if you are writing a lot of specialized javascript.
One of the most important questions is not just what technical features this platform provides or that platform provides, but the driving philosophy of the open source project and the nature of the community supporting it.
I've got no dog in this fight myself, but I found Mark Ramm's talk at DjangoCon 2008 to be very interesting on this point (Google will yield no end of subsequent discussion, no doubt).
Because Django uses its own ORM it limits you to learn that ORM for that specific web framework. I think using an web framework with a more popular ORM (like SqlAlchemy which TG uses) increases your employability chances. Just my 2 cents ..
Last I checked, django has a very poor data implementation. And that's a huge weakness in my book. Django's orm doesn't allow me to use the power of the underlying database. For example I can't use compound primary keys, which are important to good db design. It also doesn't support more than a single database, which is not a big deal until you really need it and find that you can't do it without resorting to doing it manually. Lastly if you have to make changes to your database structure in a team-friendly way, you have to try to choose between a set of 3rd party migration tools.
Turbogears seems to be more architecturally sound, doing its best to integrate individual tools that are awesome in their own right. And because TG is more of an integrator, you're able to switch out pieces to suit your preferences. Don't like SQL Alchemy? You can use SQLObject. Don't like Genshi templates? You can use Mako or even django's, although you're not exactly stuck with the default on django either.
Time for tg2's cons:
TG has a much smaller community, and community usually has its benefit.
Django has a much better name. I really like that name ;-)
Django seems simpler for the beginning web developer, with pretty cool admin tools.
TG has decent documentation, but you also need to go to Genshi's site to learn Genshi, SQL Alchemy's site to learn that, etc. Django has great docs.
My 2 cents.
I know the various frameworks have their benefits, but I personally want my web development in python to be as straight-forward as possible: less writing to the framework, more writing python.
The only thing I have found so far that lets me do this in the most obvious way possible is web.py but I have slight concerns on its performance.
For those of you using nginx(or another flavour)+mod_wsgi+web.py... how's performance? Can it be improved further?
For those of you who have used web.py, liked the idea and went on to write something better or found something better... care to point me to the source?
I'd like to hear about all the conspicuous, minimal yet powerful approaches.
The way to go is wsgi.
WSGI is the Web Server Gateway Interface. It is a specification for web servers and application servers to communicate with web applications (though it can also be used for more than that). It is a Python standard, described in detail in PEP 333.
All current frameworks support wsgi. A lot of webservers support it also (apache included, through mod_wsgi). It is the way to go if you want to write your own framework.
Here is hello world, written to wsgi directly:
def application(environ, start_response):
status = '200 OK'
response_headers = [('Content-type','text/plain')]
start_response(status, response_headers)
return ['Hello world!\n']
Put this in a file.py, point your mod_wsgi apache configuration to it, and it will run. Pure python. No imports. Just a python function.
If you are really writing your own framework, you could check werkzeug. It is not a framework, but a simple collection of various utilities for WSGI applications and has become one of the most advanced WSGI utility modules. It includes a powerful debugger, full featured request and response objects, HTTP utilities to handle entity tags, cache control headers, HTTP dates, cookie handling, file uploads, a powerful URL routing system and a bunch of community contributed addon modules. Takes the boring part out of your hands.
It's hilarious how, even prompted with a question asking how to write without a framework, everyone still piles in to promote their favourite framework. The OP whinges about not wanting a “heavyweight framework”, and the replies mention Twisted, of all things?! Come now, really.
Yes, it is perfectly possible to write straight WSGI apps, and grab bits of wanted functionality from standalone modules, instead of fitting your code into one particular framework's view of the world.
To take this route you'll generally want to be familiar with the basics of HTTP and CGI (since WSGI inherits an awful lot from that earlier specification). It's not necessarily an approach to recommend to beginners, but it's quite doable.
I'd like to hear about all the conspicuous, minimal yet powerful approaches
You won't hear about them, because no-one has a tribal interest in promoting “do it yourself” as a methodology. Me, I use a particular standalone templating package, a particular standalone form-reading package, a particular data access layer, and a few home-brew utility modules. I'm not writing to one particular philosophy I can proselytise about, they're all just boring tools that could be swapped out and replaced with something else just as good.
You could also check cherrypy. The focus of cherrypy is in being a framework that lets you write python. Cherrypy has its own pretty good webserver, but it is wsgi-compatible so you can run cherrypy applications in apache via mod_wsgi. Here is hello world in cherrypy:
import cherrypy
class HelloWorld(object):
def index(self):
return "Hello World!"
index.exposed = True
cherrypy.quickstart(HelloWorld())
+1 to all the answers with WSGI.
Eric Florenzo wrote a great blog post lately you should read: Writing Blazing Fast, Infinitely Scalable, Pure-WSGI Utilities. This will give you a better idea of pure WSGI beyond Hello World. Also pay attention to the comments, especially the first comment by Kevin Dangoor where he recommends at least adding WebOb to your toolset.
For what it's worth, I wrote my website in mod_python without any intervening framework like Django. I didn't really have any reason to complain. (Well maybe a little, mod_python is kind of quirky in a few ways but not in the common use cases) One thing's for sure, it will definitely let you write Python ;-)
Why do you have concerns about web.py's performance? As I mentioned here, we use CherryPy (the web server "built into" web.py) behind nginx to serve most of the HTML at Oyster.com -- nginx splits the traffic across 2 or 3 web servers each running 4 Python processes, and we can easily handle 100s of requests per second.
Oyster.com is a high-volume website averaging 200,000 dynamically-generated pageviews/day, and peaking to much higher numbers than that. However, we do use a content delivery network (CDN) for our static resources like images and CSS.
We definitely care about performance (most of our pages render in less than 25ms), but web.py isn't the bottleneck. Our bottlenecks are template rendering (we use Cheetah, which is fast enough but not other-worldly fast) and database queries (we cache heavily and keep the number of database queries per page to 0 or 1) and accessing our 3rd-party hotel pricing providers (these are accessed when you do a search with dates we don't already have cached).
Remember, premature optimization is the root of all evil -- unless you're serving google.com, web.py will probably work for you.
I've written a few small web applications using mod-python and PSP -- mod-python's equivalent to php.
In one case, I wrote a web page that generates release notes by inspecting our source code repository. I rewrote it into PHP, and was surprised to discover that the PSP version was about 20% faster, as well as being about half as many lines of code.
So, for small problems at least, psp has worked well for me.
I think it depends on the definition of what a framework is and what it should do for you.
As pointed out, a very minimal "framework" would be WSGI as it only defines one small interface for interfacing with a web server. But it's a powerful approach because of the middleware you can put between your app and the server.
If you want more slightly more, like some URL to function mapping, then you have some choices, some of which have been already mentioned.
If you go further you might come to Pylons or Turbogears or Django, after that maybe Zope but it grows bigger and maybe the pain as well as you always buy into the opinions of that framework.
What I recently use more and more (coming from Zope/Plone) is repoze.bfg. It's very small, does not come with a ORM bundled (so you can use SQLAlchemy, Storm or simply go to an object database like the ZODB). What it does is basically handling how you come from a URL to a view which is a function. It supports both URL Mapping (a la Routes) or object traversal, which IMHO is very powerful in some circumstances esp. if you have a not so strict mapping. The good thing is that it directly comes with an ACL based security framework which can use if you want to which IMHO is very practical to have. That way you don't need decorators which seem to be used mostly for such things.
And of course it's based on WSGI. Also look for the repoze subversion repository for quite a lot of middleware and the Paste stuff is also very useful for WSGI related tasks.
What's wrong with Django? It doesn't force you to use it's ORM and controllers are just plain Python functions instead of Rails-like class methods. Also, url routing is done with regular expressions instead of another framework-invented syntax. If django seems like too much for you anyway, i recommend taking a look at Werkzeug
I'm pretty fond of Google AppEngine. I use the ORM and templating system, but otherwise follow a REST-patterned design and just implement Python methods for the corresponding HTTP ones. It makes the raw HTTP interaction central, and optionally gives you other things to use. Plus no more configuring and managing your deployment environment!
I'd like to do some server-side scripting using Python. But I'm kind of lost with the number of ways to do that.
It starts with the do-it-yourself CGI approach and it seems to end with some pretty robust frameworks that would basically do all the job themselves. And a huge lot of stuff in between, like web.py, Pyroxide and Django.
What are the pros and cons of the frameworks or approaches that you've worked on?
What trade-offs are there?
For what kind of projects they do well and for what they don't?
Edit: I haven't got much experience with web programing yet.
I would like to avoid the basic and tedious things like parsing the URL for parameters, etc.
On the other hand, while the video of blog created in 15 minutes with Ruby on Rails left me impressed, I realized that there were hundreds of things hidden from me - which is cool if you need to write a working webapp in no time, but not that great for really understanding the magic - and that's what I seek now.
CGI is great for low-traffic websites, but it has some performance problems for anything else. This is because every time a request comes in, the server starts the CGI application in its own process. This is bad for two reasons: 1) Starting and stopping a process can take time and 2) you can't cache anything in memory. You can go with FastCGI, but I would argue that you'd be better off just writing a straight WSGI app if you're going to go that route (the way WSGI works really isn't a whole heck of a lot different from CGI).
Other than that, your choices are for the most part how much you want the framework to do. You can go with an all singing, all dancing framework like Django or Pylons. Or you can go with a mix-and-match approach (use something like CherryPy for the HTTP stuff, SQLAlchemy for the database stuff, paste for deployment, etc). I should also point out that most frameworks will also let you switch different components out for others, so these two approaches aren't necessarily mutually exclusive.
Personally, I dislike frameworks that do too much magic for me and prefer the mix-and-match technique, but I've been told that I'm also completely insane. :)
How much web programming experience do you have? If you're a beginner, I say go with Django. If you're more experienced, I say to play around with the different approaches and techniques until you find the right one.
The simplest web program is a CGI script, which is basically just a program whose standard output is redirected to the web browser making the request. In this approach, every page has its own executable file, which must be loaded and parsed on every request. This makes it really simple to get something up and running, but scales badly both in terms of performance and organization. So when I need a very dynamic page very quickly that won't grow into a larger system, I use a CGI script.
One step up from this is embedding your Python code in your HTML code, such as with PSP. I don't think many people use this nowadays, since modern template systems have made this pretty obsolete. I worked with PSP for awhile and found that it had basically the same organizational limits as CGI scripts (every page has its own file) plus some whitespace-related annoyances from trying to mix whitespace-ignorant HTML with whitespace-sensitive Python.
The next step up is very simple web frameworks such as web.py, which I've also used. Like CGI scripts, it's very simple to get something up and running, and you don't need any complex configuration or automatically generated code. Your own code will be pretty simple to understand, so you can see what's happening. However, it's not as feature-rich as other web frameworks; last time I used it, there was no session tracking, so I had to roll my own. It also has "too much magic behavior" to quote Guido ("upvars(), bah").
Finally, you have feature-rich web frameworks such as Django. These will require a bit of work to get simple Hello World programs working, but every major one has a great, well-written tutorial (especially Django) to walk you through it. I highly recommend using one of these web frameworks for any real project because of the convenience and features and documentation, etc.
Ultimately you'll have to decide what you prefer. For example, frameworks all use template languages (special code/tags) to generate HTML files. Some of them such as Cheetah templates let you write arbitrary Python code so that you can do anything in a template. Others such as Django templates are more restrictive and force you to separate your presentation code from your program logic. It's all about what you personally prefer.
Another example is URL handling; some frameworks such as Django have you define the URLs in your application through regular expressions. Others such as CherryPy automatically map your functions to urls by your function names. Again, this is a personal preference.
I personally use a mix of web frameworks by using CherryPy for my web server stuff (form parameters, session handling, url mapping, etc) and Django for my object-relational mapping and templates. My recommendation is to start with a high level web framework, work your way through its tutorial, then start on a small personal project. I've done this with all of the technologies I've mentioned and it's been really beneficial. Eventually you'll get a feel for what you prefer and become a better web programmer (and a better programmer in general) in the process.
If you decide to go with a framework that is WSGI-based (for instance TurboGears), I would recommend you go through the excellent article Another Do-It-Yourself Framework by Ian Bicking.
In the article, he builds a simple web application framework from scratch.
Also, check out the video Creating a web framework with WSGI by Kevin Dangoor. Dangoor is the founder of the TurboGears project.
If you want to go big, choose Django and you are set. But if you want just to learn, roll your own framework using already mentioned WebOb - this can be really fun and I am sure you'll learn much more (plus you can use components you like: template system, url dispatcher, database layer, sessions, et caetera).
In last 2 years I built few large sites using Django and all I can say, Django will fill 80% of your needs in 20% of time. Remaining 20% of work will take 80% of the time, no matter which framework you'd use.
It's always worth doing something the hard way - once - as a learning exercise. Once you understand how it works, pick a framework that suits your application, and use that. You don't need to reinvent the wheel once you understand angular velocity. :-)
It's also worth making sure that you have a fairly robust understanding of the programming language behind the framework before you jump in -- trying to learn both Django and Python at the same time (or Ruby and Rails, or X and Y), can lead to even more confusion. Write some code in the language first, then add the framework.
We learn to develop, not by using tools, but by solving problems. Run into a few walls, climb over, and find some higher walls!
If you've never done any CGI programming before I think it would be worth doing one project - perhaps just a sample play site just for yourself - using the DIY approach. You'll learn a lot more about how all the various parts work than you would by using a framework. This will help in you design and debug and so on all your future web applications however you write them.
Personally I now use Django. The real benefit is very fast application deployment. The object relational mapping gets things moving fast and the template library is a joy to use. Also the admin interface gives you basic CRUD screens for all your objects so you don't need to write any of the "boring" stuff.
The downside of using an ORM based solution is that if you do want to handcraft some SQL, say for performance reasons, it much harder than it would have been otherwise, although still very possible.
If you are using Python you should not start with CGI, instead start with WSGI (and you can use wsgiref.handlers.CGIHandler to run your WSGI script as a CGI script. The result is something that is basically as low-level as CGI (which might be useful in an educational sense, but will also be somewhat annoying), but without having to write to an entirely outdated interface (and binding your application to a single process model).
If you want a less annoying, but similarly low-level interface, using WebOb would provide that. You would be implementing all the logic, and there will be few dark corners that you won't understand, but you won't have to spend time figuring out how to parse HTTP dates (they are weird!) or parse POST bodies. I write applications this way (without any other framework) and it is entirely workable. As a beginner, I'd advise this if you were interested in understanding what frameworks do, because it is inevitable you will be writing your own mini framework. OTOH, a real framework will probably teach you good practices of application design and structure. To be a really good web programmer, I believe you need to try both seriously; you should understand everything a framework does and not be afraid of its internals, but you should also spend time in a thoughtful environment someone else designed (i.e., an existing framework) and understand how that structure helps you.
OK, rails is actually pretty good, but there is just a little bit too much magic going on in there (from the Ruby world I would much prefer merb to rails). I personally use Pylons, and am pretty darn happy. I'd say (compared to django), that pylons allows you to interchange ints internal parts easier than django does. The downside is that you will have to write more stuff all by youself (like the basic CRUD).
Pros of using a framework:
get stuff done quickly (and I mean lighning fast once you know the framework)
everything is compying to standards (which is probably not that easy to achieve when rolling your own)
easier to get something working (lots of tutorials) without reading gazillion articles and docs
Cons:
you learn less
harder to replace parts (not that much of an issue in pylons, more so with django)
harder to tweak some low-level stuff (like the above mentioned SQLs)
From that you can probably devise what they are good for :-) Since you get all the code it is possible to tweak it to fit even the most bizzare situations (pylons supposedly work on the Google app engine now...).
For smaller projects, rolling your own is fairly easy. Especially as you can simply import a templating engine like Genshi and get alot happening quite quickly and easily. Sometimes it's just quicker to use a screwdriver than to go looking for the power drill.
Full blown frameworks provide alot more power, but do have to be installed and setup first before you can leverage that power. For larger projects, this is a negligible concern, but for smaller projects this might wind up taking most of your time - especially if the framework is unfamiliar.