Related
After much research, I've come up with a list of what I think might be the best way of putting together a Python based social network/cms, but have some questions about how some of these components fit together.
Before I ask about the particular components, here are some of the key features of the site to be built:
a modern almost desktop-like gui
future ability to host an advanced html5 sub-application (ex.http://www.lucidchart.com)
high scalability both for functionality and user load
user ability to password protect and permission manage content on per item/group basis
typical social network features
ability to build a scaled down mobile version in the future
Here's the list of tools I'm considering using:
Google App Engine
Python
Django
Pinax
Pyjamas
wxPython
And the questions:
Google App Engine -- this is an attempt to cut to the chase as many pieces of the puzzle seem to be in place.
Question: Am I limiting my options with this choice? Example: datastore not being relational? Should I wait
for SQL support under the Business version?
Python -- I considered 'drupal' at first, but in the end decided that being dependent on modules that may or
may not exist tomorrow + limitations of its templating system are a no-no. Learning its API, too, would be useless elsewhere
whereas Python seems like a swiss army knife of languages -- good for almost anything.
Question: v.2.5.2 is required by GAE, but python.org recommends 2.5.5. Which do I install?
Django -- v.0.96 is built into GAE. You seem to be able to upgrade it.
Questions: Any reason not to upgrade to the latest version? Ways to get around the lack of HTML5 support?
Pinax (http://pinaxproject.com) Rides on top of Django and appears to provide most of the social network functionality
anyone would want.
Question: Reasons NOT to use it? Alternatives?
Pyjamas and wxPython -- this is the part that gets a little confusing. The basic idea behind these is the ability
to build a GUI. I've considered Silverlight and Flash, before the GAE/Python route, but a few working versions of
HTML5 apps convinced me that enough of it ALREADY runs on the latest batch of browsers to chose the HTML5/Javascript
route instead.
Question: How do I extend/supplement Python/Django to build an app-like HTML5 interface? Are Pyjamas and wxPython
the way to go? Or should I change my thinking completely?
Answers to some/any of these questions would be of great help. Please excuse my ignorance if any of this doesn't make much sense.
My last venture into web programming was a decent sized LAMP website some 5-6 years ago. On the desktop side of things,
my programming experience boils down to very high level scripting languages that I keep on learning to accomplish very specific
tasks :)
As someone who has deployed a Django site to GAE, I can tell you that you are not going to reach the ideal solution. Django on GAE misses some of the best aspects of Django because the ORM doesn't work right. The best compromise may be to use Django-nonrel to add the features back in.
This introduces it's own problems though: because of the large number of files and memory used by a Django app you're code will be unloaded from memory quickly after the app becomes idle. That means that visitors will frequently hit an approximately 6 second delay on the first page view after the site's code has been unloaded from memory while GAE uncompresses the zipped modules. Once your site is busy this won't be a problem, but while your site is still young and unknown it will cause the appearance of performance problems. :-(
Second, I've also worked for a company that built a custom CMS and can tell you that the first 80% is pretty easy, especially with modern frameworks. However, the rest can be quite challenging. For example, user roles and custom content types are two challenging aspects. Therefore strongly consider standing on the backs of giants and finding a CMS or CMS framework that almost perfectly meets your needs and then extend it to do that extra bit you need.
So, that said, answering your points:
Yes, you're limiting your options but that may be OK. Most developers are more comfortable with the relational model than the nosql model. Therefore more open source software is built with it in mind. Also, GAE is a closed source platform which is also a deterrent to open source developers. App Engine Oil is a CMS framework that may suit you well and is optimized for App Engine. Also look at web2py which has support for GAE.
I've found myself to be extremely productive with Python. I used to write a lot of PHP now I find it ugly. That said, think about the total line count of code you'll have to write. If you can make Drupal work with high quality pre-made modules you may find yourself only needing 1/10th of the code. By the way, the trick with Drupal is to mainly use only high quality modules. Look at the history, make sure not to use development versions. Try to contact the authors on IRC. I'm not saying you should use Drupal but it is possible to have a reliable site with it (for example, whitehouse.gov)
You're in the classic GAE/Django problem. If you use 0.96 you get great performance but you miss a lot of the great 1.0+ features and you don't get the ORM and all of it's benefits. If you use a newer version of Django you get the performance/memory problems mentioned above.
I'm about to investigate pinax for my company. I've done a very cursor glance at it. I don't know if it has good support for non relational model backends. You'll probably need to look at django-nonrel. However know that you're going to be investing in relatively untried solutions here. A small percentage of Django users use Pinax and an even smaller percentage, if any, use it on a nonrelational backend. Therefore you're going to be in the highly experimental scenario you mentioned in point 2 above.
I can't offer personal experience on it. I've investigated pyjamas a few times. However I like writing HTML CSS and JS. I like to have control. I like progressive enhancement and knowing what users will see if they don't have the full capabilities. Also, I think any new app that doesn't explicitly address mobile clients is implicitly shooting themself in the foot. As many as 15% of Internet users only use the Internet via their smart phone. What kind of experience will they get with pyjamas?
You didn't mention this, but one thing I consider when choosing a platform is vendor lockin and portability. If you develop your solution for GAE and find that you're not able to do what you want, will you be able to port it to another solution elsewhere? How much work will it take? If you code heavily for GAE or make commitments to its architecture, you're stuck with it or with rewriting to move. Using Django or Web2py can help mitigate this.
That said, the big benefit of Python GAE is that you get to be very productive, see your results instantly, get hosting for free while your site is small and get excellent scalability. These are not small things. There is great value there.
Note: I use TRAC integrated with SVN, framework testing tools, an excellent mixture of staging servers, development servers, and other tools to speed development and keep track of tasks.
I am asking about the specific process of design, and even more specifically, the design of functionality and flow in a web application.
--- Original question ---
So far I spend a lot of time with my text editor open basically talking to myself, then coding for a while, then talking to myself again. When there is more than just me we do some whiteboarding, but that's about it.
What do you find works well, specifically for projects for very small teams or one-developer shows.
BTW I usually develop with Django, this last project also involves RabbitMQ and Orbited, plus a fair chunk of jquery-assisted JavaScript.
Pencils!
Seriously, we use JIRA to track work/issue, Confluence to track requirements before they're baked enough to put in JIRA.. Anything to scratch together wire-frames, etc. (Including OmniGraffle, and Pencils).
I find the combination of JIRA and Confluence for tracking chunks of work and longer-lived concepts and standards pretty darned effective.
Another mainly solo developer here. I recommend using a bug/issue tracker. I use TRAC (although I'm looking at alternatives). It might seem strange, since you're the one creating, assigning, and closing all tickets, but it really helps to plan out/prioritize development. I also use the wiki to organize my thoughts on the roadmap, main goals of each release, etc.
+1 for a good question.
Web development is not much different to any other development.
P.20 of that venerable classic breaks a typical project down into 1/3 planning, 1/6 coding, 1/4 component test and 1/4 system test. I might catch some flak for that, YMMV and all, but that looks about right to me.
Whether or not you agree with the proportions, the message is "don't jump right in and code; think about it first (measure twice, cut once). How often have you jumped in and coded only to get near to "then end" and discover that you have a fundamental design flaw and have to through away much of your code & rewrite chunks more?
You need methodologies (Processes) and Tools, and each can necessitate the other.
Methodologies
Design it first! Gather requirements, make a system level design document then detailed design. If you are more than one, have these reviewed by someone (client can maybe review the high level docs?). If if you are alone, the simple act of writing it down forces you to slow down and think and will uncover problems. A good idea is to have a requirements traceability matrix to ensure that each requirement gets designed, implemented and tested somewhere.
After you review the high level design, you can being the detailed design, and after you review that, you can begin to implement. When the high level design is reviewed you can, in parallel, or later, produce a high level test spec. When the detailed level design is reviewed you can, in parallel, or later, produce unit test specs.
Note that test cases should be automated and should require no human interaction. Get into the habit of running regression tests after every code change - automate this if you can, with nightly build and or coupled with check-in to your version control system.
When everything is thoroughly unit tested, you can begin your system level testing.
Tools
At the very least you ought to be considering these:
A good IDE (WYSIWYG for web design), preferably with debugging capabilities, and it would be nice if it interfaced with your version control system, bug tracker, etc. A spill chuker is useful for websites ;-)
A project management tool to plan the project (Open Workbench does some nice Gantt charts)
A version control system.
A change request and bug tracker.
An automated test system.
An automated build system like Hudson (it may not seem relevant if you don't compile and link, but at least it can verify that all files exist and can schedule regression testing for you)
A backup system in case of disk crash, laptop loss, etc.
And if all that seems like too much "extra" to do, I was sceptical too once, until I saw that it actually saves time because you discover problems earlier when they cost less to fix. In fact, I am so sure of this that I do all of my personal one-man hobby projects this way. All of them.
I've pretty much tried every Python web framework that exists, and it took me a long time to realize there wasn't a silver bullet framework, each had its own advantages and disadvantages. I started out with Snakelets and heartily enjoyed being able to control almost everything at a lower level without much fuss, but then I discovered TurboGears and I have been using it (1.x) ever since. Tools like Catwalk and the web console are invaluable to me.
But with TurboGears 2 coming out which brings WSGI support, and after reading up on the religious debates between the Django and WSGI camps, I'm really torn between "doing it the right way", e.g., learning WSGI, spending valuable time writing functionality that already exists in Django and other full-stack frameworks, as opposed to using Django or some high-level framework that does everything for me. The downsides with the latter that I can see are pretty obvious:
I'm not learning anything in the process
If I ever need to do anything lower level it's going to be a pain
The overhead required for just a basic site which uses authentication is insane. (IMO)
So, I guess my question is, which is the better choice, or is it just a matter of opinion, and should I suck it up and use Django if it achieves what I want with minimal fuss (I want authentication and a CRUD interface to my database)? I tried Werkzeug, Glashammer, and friends, but AuthKit and Repoze scared me off, as well as the number of steps involved to just setup basic authentication. I looked at Pylons, but the documentation seems lacking, and when referencing simple features like authentication or a CRUD interface, various wiki pages and documentation seemed to contradict each other, with different hacks for versions and such.
Thanks to S. Lott for pointing out that I wasn't clear enough. My question is: which of the following is worthwhile in the long run, but not painful in the short (e.g., some sort of middle ground, anyone?) - Learn WSGI, or stick with a "batteries-included" framework? If the latter, I would appreciate a suggestion as to whether I should give Django another try, stick with TurboGears 1.x, or venture into some other framework.
Also, I have tried CherryPy, but couldn't seem to find a good enough CRUD application that I could plop in and use right away.
the religious debates between the Django and WSGI camps
It would seem as though you're a tad bit confused about what WSGI is and what Django is. Saying that Django and WSGI are competing is a bit like saying that C and SQL are competing: you're comparing apples and oranges.
Django is a framework, WSGI is a protocol (which is supported by Django) for how the server interacts with the framework. Most importantly, learning to use WSGI directly is a bit like learning assembly. It's a great learning experience, but it's not really something you should do for production code (nor was it intended to be).
At any rate, my advice is to figure it out for yourself. Most frameworks have a "make a wiki/blog/poll in an hour" type exercise. Spend a little time with each one and figure out which one you like best. After all, how can you decide between different frameworks if you're not willing to try them out?
I'd say you're being a bit too pessimistic about "not learning anything" using Django or a similar full-stack framework, and underestimating the value of documentation and a large community. Even with Django there's still a considerable learning curve; and if it doesn't do everything you want, it's not like the framework code is impenetrable.
Some personal experience: I spent years, on and off, messing around with Twisted/Nevow, TurboGears and a few other Python web frameworks. I never finished anything because the framework code was perpetually unfinished and being rewritten underneath me, the documentation was often nonexistent or wrong and the only viable support was via IRC (where I often got great advice, but felt like I was imposing if I asked too many questions).
By comparison, in the past couple of years I've knocked off a few sites with Django. Unlike my previous experience, they're actually deployed and running. The Django development process may be slow and careful, but it results in much less bitrot and deprecation, and documentation that is actually helpful.
HTTP authentication support for Django finally went in a few weeks ago, if that's what you're referring to in #3.
I suggest taking another look at TG2. I think people have failed to notice some of the strides that have been made since the last version. Aside from the growing WSGI stack of utilities available there are quite a few TG2-specific items to consider. Here are a couple of highlights:
TurboGears Administration System - This CRUD interface to your database is fully customizable using a declarative config class. It is also integrated with Dojo to give you infinitely scrollable tables. Server side validation is also automated. The admin interface uses RESTful urls and HTTP verbs which means it would be easy to connect to programatically using industry standards.
CrudRestController/RestController - TurboGears provides a structured way to handle services in your controller. Providing you the ability to use standardized HTTP verbs simply by extending our RestController. Combine Sprox with CrudRestController, and you can put crud anywhere in your application with fully-customizable autogenerated forms.
TurboGears now supports mime-types as file extensions in the url, so you can have your controller render .json and .xml with the same interface it uses to render html (returning a dictionary from a controller)
If you click the links you will see that we have a new set of documentation built with sphinx which is more extensive than the docs of the past.
With the best web server, ORM, and template system(s) (pick your own) under the hood, it's easy to see why TG makes sense for people who want to get going quickly, and still have scalability as their site grows.
TurboGears is often seen as trying to hit a moving target, but we are consistent about releases, which means you won't have to worry about working out of the trunk to get the latest features you need. Coming to the future: more TurboGears extensions that will allow your application to grow functionality with the ease of paster commands.
Your question seems to be "is it worth learning WSGI and doing everything yourself," or using a "full stack framework that does everything for you."
I'd say that's a false dichotomy and there's an obvious third way. TurboGears 2 tries to provide a smooth path from a "do everything for you" style framework up to an understanding of WSGI middleware, and an ability to customize almost every aspect of the framework to suit your application's needs.
We may not be successful in every place at every level, but particularly if you've already got some TurboGears 1 experience I think the TG2 learning curve will be very, very easy at first and you'll have the ability to go deeper exactly when you need it.
To address your particular issues:
We provide an authorization system out of the box that matches the one you're used to from TG1.
We provide an out of the box "django admin" like interface called the tgext.admin, which works great with dojo to make a fancy spreadsheet like interface the default.
I'd also like to address a couple of the other options that are out there and talk a little bit about the benifits.
CherryPy. I think CherryPy is a great webserver and a nice minimalistic web-framework. It's not based on WSGI internally but has good WSGI support although it will not provide you with the "full stack" experience. But for custom setups that need to be both fast and aren't particularly suited to the defaults provided by Django or TurboGears, it's a great solution.
Django. I think Django is a very nice, tigtly integrated system for developing websites. If your application and style of working fits well within it's standard setup it can be fantastic. If however you need to tune your DB usage, replace the template language, use a different user authorization model or otherwise do things differently you may very likely find yourself fighting the framework.
Pylons Pylons like CherryPy is a great minimalistic web-framework. Unlike CherryPy it's WSGI enabled through the whole system and provides some sane defaults like SQLAlchemy and Mako that can help you scale well. The new official docs are of much better quality than the old wiki docs which are what you seem to have looked at.
Have you taken a look at CherryPy. It is minimalistic, yet efficient and simple. It is low level enough for not it to get in they way, but high enough to hide complexity. If I remember well, TurboGears was built on it.
With CherryPy, you have the choice of much everything. (Template framework, ORM if wanted, back-end, etc.)
Learn WSGI
WSGI is absurdly simple.. It's basically a function that looks like..
def application(environ, start_response) pass
The function is called when an HTTP request is received. environ contains various data (like the request URI etc etc), start_response is a callable function, used to set headers.
The returned value is the body of the website.
def application(environ, start_response):
start_response("200 OK", [])
return "..."
That's all there is to it, really.. It's not a framework, but more a protocol for web-frameworks to use..
For creating sites, using WSGI is not the "right way" - using existing frameworks is.. but, if you are writing a Python web-framework then using WSGI is absolutely the right way..
Which framework you use (CherryPy, Django, TurboGears etc) is basically personal preference.. Play around in each, see which you like the most, then use it.. There is a StackOverflow question (with a great answer) about this, "Recommendation for straight-forward python frameworks"
Have you checked out web2py? After recently evaluating many Python web frameworks recently I've decided to adopt this one. Also check out Google App Engine if you haven't already.
I'd say the correct answer depends on what you actually want and need, as what will be worthwhile in the long run depends on what you'll need in the long run. If your goal is to get applications deployed ASAP then the 'simpler' route, ie. Django, is surely the way to go. The value of a well-tested and well-documented system that exactly what you want can't be underestimated.
On the other hand if you have time to learn a variety of new things which may apply in other domains and want to have the widest scope for customisation then something like Turbogears is superior. Turbogears gives you maximum flexibility but you will have to spend a lot of time reading external docs for things like Repoze, SQLAlchemy, and Genshi to get anything useful done with it. The TG2 docs are deliberately less detailed than the TG1 docs in some cases because it's considered that the external docs are better than they used to be. Whether this sort of thing is an obstacle or an investment depends on your own requirements.
Django is definitely worth learning, and sounds like it will fit your purposes. The admin interface it comes with is easy to get up and running, and it does use authentication.
As for "anything lower level", if you mean sql, it is entirely possible to shove sql into you queries with the extra keyword. Stylistically, you always try to avoid that as much as possible.
As for "not learning anything"...the real question is whether your preference is to be primarily learning something lower-level or higher-level, which is hardly a question anyone here can answer for you.
Pylons seems a great tool for me:
a real web framework (CherryPy is just a web server),
small code base - reuse of other projects,
written entirely with WSGI in mind, based on Paste,
allows you to code the app right away and touch the low level bits if it's necessary,
I've used CherryPy and TurboGears and look at many other frameworks but none of them were so light and productive as Pylons is. Check the presentation at Google.
I'm a TurboGears fan, and this is exactly the reason why: a very nice trade-off between control and doing things right vs. easy.
You'll have to make up your own mind of course. Maybe you'd prefer to learn less, maybe more. Maybe the areas that I like knowledge/control (database for example), you couldn't care less about. And don't misunderstand. I'm not characterizing any frameworks as necessarily hard or wrong. It's just my subjective judgment.
Also I would recommend TurboGears 2 if at all possible. When it comes out, I think it will be much better than 1.0 in terms of what it has selected for defaults (genshi, pylons, SqlAlchemy)
I would suggest for TurboGears 2. They have done a fantastic job of integrating best of Python world.
WSGI: Assuming you are developing moderately complex projects/ business solutions in TG2 or some other framework say Grok. Even though these frameworks supports WSGI does that mean one who is using these frameworks have to learn WSGI? In most cases answer is No. I mean it's good have this knowledge no doubt.
WSGI knowledge is probably is more useful in cases like
you want to use some middleware or some other component which is not provided as part of the standard stack for eg. Authkit with TG or Grok without ZODB.
you are doing some integration.
CherryPy is good but think of handling your database commits/rollbacks at the end of transactions, exposing json, validations in such cases TG, Django like frameworks do it all for you.
Web2py is the secret sauce here. Don't miss checking it out.
A few weeks ago I asked the question "Is a PHP, Python, PostgreSQL design suitable for a non-web business application?" Is a PHP, Python, PostgreSQL design suitable for a business application?
A lot of the answers recommended skipping the PHP piece and using Django to build the application. As I've explored Django, I've started to question one specific aspect of my goals and how Django comes into play for a non-web business application.
Based on my understanding, Django would manage both the view and controller pieces and PostgreSQL or MySQL would handle the data. But my goal was to clearly separate the layers so that the database, domain logic, and presentation could each be changed without significantly affecting the others. It seems like I'm only separating the M from the VC layers with the Django solution.
So, is it counterproductive for me to build the domain layer in Python with an SQL Alchemy/Elixir ORM tool, PostgreSQL for the database layer, and then still use Django or PHP for the presentation layer? Is this possible or pure insanity?
Basically, I'd be looking at an architecture of Django/PHP > Python/SQLAlchemy > PostgreSQL/MySQL.
Edit: Before the fanboys get mad at me for asking a question about Django, just realize: It's a question, not an accusation. If I knew the answer or had my own opinion, I wouldn't have asked!
You seem to be saying that choosing Django would prevent you from using a more heterogenous solution later. This isn't the case. Django provides a number of interesting connections between the layers, and using Django for all the layers lets you take advantage of those connections. For example, using the Django ORM means that you get the great Django admin app almost for free.
You can choose to use a different ORM within Django, you just won't get the admin app (or generic views, for example) along with it. So a different ORM takes you a step backward from full Django top-to-bottom, but it isn't a step backward from other heterogenous solutions, because those solutions didn't give you intra-layer goodness the admin app in the first place.
Django shouldn't be criticized for not providing a flexible architecture: it's as flexible as any other solution, you just forgo some of the Django benefits if you choose to swap out a layer.
If you choose to start with Django, you can use the Django ORM now, and then later, if you need to switch, you can change over to SQLalchemy. That will be no more difficult than starting with SQLalchemy now and later moving to some other ORM solution.
You haven't said why you anticipate needing to swap out layers. It will be a painful process no matter what, because there is necessarily much code that relies on the behavior of whichever toolset and library you're currently using.
Django will happily let you use whatever libraries you want for whatever you want to use them for -- you want a different ORM, use it, you want a different template engine, use it, and so on -- but is designed to provide a common default stack used by many interoperable applications. In other words, if you swap out an ORM or a template system, you'll lose compatibility with a lot of applications, but the ability to take advantage of a large base of applications typically outweighs this.
In broader terms, however, I'd advise you to spend a bit more time reading up on architectural patterns for web applications, since you seem to have some major conceptual confusion going on. One might just as easily say that, for example, Rails doesn't have a "view" layer since you could use different file systems as the storage location for the view code (in other words: being able to change where and how the data is stored by your model layer doesn't mean you don't have a model layer).
(and it goes without saying that it's also important to know why "strict" or "pure" MVC is an absolutely horrid fit for web applications; MVC in its pure form is useful for applications with many independent ways to initiate interaction, like a word processor with lots of toolbars and input panes, but its benefits quickly start to disappear when you move to the web and have only one way -- an HTTP request -- to interact with the application. This is why there are no "true" MVC web frameworks; they all borrow certain ideas about separation of concerns, but none of them implement the pattern strictly)
You seem to be confusing "separate layers" with "separate languages/technologies." There is no reason you can't separate your concerns appropriately within a single programming language, or within an appropriately modular framework (such as Django). Needlessly multiplying programming languages / frameworks is just needlessly multiplying complexity, which is likely to slow down your initial efforts so much that your project will never reach the point where it needs a technology switch.
You've effectively got a 3 layer architecture whether you use Django's ORM or SQLAlchemy, though your forgo some of the Django's benefits if you choose the latter.
Based on my understanding, Django would manage both the view and controller pieces and PostgreSQL or MySQL would handle the data.
Not really, Django has its own ORM, so it does separate data from view/controller.
here's an entry from the official FAQ about MVC:
Where does the “controller” fit in, then? In Django’s case, it’s probably the framework itself: the machinery that sends a request to the appropriate view, according to the Django URL configuration.
If you’re hungry for acronyms, you might say that Django is a “MTV” framework – that is, “model”, “template”, and “view.” That breakdown makes much more sense.
At the end of the day, of course, it comes down to getting stuff done. And, regardless of how things are named, Django gets stuff done in a way that’s most logical to us.
There's change and there's change. Django utterly seperates domain model, business rules and presentation. You can change (almost) anything with near zero breakage. And by change I am talking about meaningful end-user focused business change.
The technology mix-n-match in this (and the previous) question isn't really very helpful.
Who -- specifically -- benefits from replacing Django templates with PHP. It's not more productive. It's more complex for no benefit. Who benefits from changing ORM's. The users won't and can't tell.
I'd like to do some server-side scripting using Python. But I'm kind of lost with the number of ways to do that.
It starts with the do-it-yourself CGI approach and it seems to end with some pretty robust frameworks that would basically do all the job themselves. And a huge lot of stuff in between, like web.py, Pyroxide and Django.
What are the pros and cons of the frameworks or approaches that you've worked on?
What trade-offs are there?
For what kind of projects they do well and for what they don't?
Edit: I haven't got much experience with web programing yet.
I would like to avoid the basic and tedious things like parsing the URL for parameters, etc.
On the other hand, while the video of blog created in 15 minutes with Ruby on Rails left me impressed, I realized that there were hundreds of things hidden from me - which is cool if you need to write a working webapp in no time, but not that great for really understanding the magic - and that's what I seek now.
CGI is great for low-traffic websites, but it has some performance problems for anything else. This is because every time a request comes in, the server starts the CGI application in its own process. This is bad for two reasons: 1) Starting and stopping a process can take time and 2) you can't cache anything in memory. You can go with FastCGI, but I would argue that you'd be better off just writing a straight WSGI app if you're going to go that route (the way WSGI works really isn't a whole heck of a lot different from CGI).
Other than that, your choices are for the most part how much you want the framework to do. You can go with an all singing, all dancing framework like Django or Pylons. Or you can go with a mix-and-match approach (use something like CherryPy for the HTTP stuff, SQLAlchemy for the database stuff, paste for deployment, etc). I should also point out that most frameworks will also let you switch different components out for others, so these two approaches aren't necessarily mutually exclusive.
Personally, I dislike frameworks that do too much magic for me and prefer the mix-and-match technique, but I've been told that I'm also completely insane. :)
How much web programming experience do you have? If you're a beginner, I say go with Django. If you're more experienced, I say to play around with the different approaches and techniques until you find the right one.
The simplest web program is a CGI script, which is basically just a program whose standard output is redirected to the web browser making the request. In this approach, every page has its own executable file, which must be loaded and parsed on every request. This makes it really simple to get something up and running, but scales badly both in terms of performance and organization. So when I need a very dynamic page very quickly that won't grow into a larger system, I use a CGI script.
One step up from this is embedding your Python code in your HTML code, such as with PSP. I don't think many people use this nowadays, since modern template systems have made this pretty obsolete. I worked with PSP for awhile and found that it had basically the same organizational limits as CGI scripts (every page has its own file) plus some whitespace-related annoyances from trying to mix whitespace-ignorant HTML with whitespace-sensitive Python.
The next step up is very simple web frameworks such as web.py, which I've also used. Like CGI scripts, it's very simple to get something up and running, and you don't need any complex configuration or automatically generated code. Your own code will be pretty simple to understand, so you can see what's happening. However, it's not as feature-rich as other web frameworks; last time I used it, there was no session tracking, so I had to roll my own. It also has "too much magic behavior" to quote Guido ("upvars(), bah").
Finally, you have feature-rich web frameworks such as Django. These will require a bit of work to get simple Hello World programs working, but every major one has a great, well-written tutorial (especially Django) to walk you through it. I highly recommend using one of these web frameworks for any real project because of the convenience and features and documentation, etc.
Ultimately you'll have to decide what you prefer. For example, frameworks all use template languages (special code/tags) to generate HTML files. Some of them such as Cheetah templates let you write arbitrary Python code so that you can do anything in a template. Others such as Django templates are more restrictive and force you to separate your presentation code from your program logic. It's all about what you personally prefer.
Another example is URL handling; some frameworks such as Django have you define the URLs in your application through regular expressions. Others such as CherryPy automatically map your functions to urls by your function names. Again, this is a personal preference.
I personally use a mix of web frameworks by using CherryPy for my web server stuff (form parameters, session handling, url mapping, etc) and Django for my object-relational mapping and templates. My recommendation is to start with a high level web framework, work your way through its tutorial, then start on a small personal project. I've done this with all of the technologies I've mentioned and it's been really beneficial. Eventually you'll get a feel for what you prefer and become a better web programmer (and a better programmer in general) in the process.
If you decide to go with a framework that is WSGI-based (for instance TurboGears), I would recommend you go through the excellent article Another Do-It-Yourself Framework by Ian Bicking.
In the article, he builds a simple web application framework from scratch.
Also, check out the video Creating a web framework with WSGI by Kevin Dangoor. Dangoor is the founder of the TurboGears project.
If you want to go big, choose Django and you are set. But if you want just to learn, roll your own framework using already mentioned WebOb - this can be really fun and I am sure you'll learn much more (plus you can use components you like: template system, url dispatcher, database layer, sessions, et caetera).
In last 2 years I built few large sites using Django and all I can say, Django will fill 80% of your needs in 20% of time. Remaining 20% of work will take 80% of the time, no matter which framework you'd use.
It's always worth doing something the hard way - once - as a learning exercise. Once you understand how it works, pick a framework that suits your application, and use that. You don't need to reinvent the wheel once you understand angular velocity. :-)
It's also worth making sure that you have a fairly robust understanding of the programming language behind the framework before you jump in -- trying to learn both Django and Python at the same time (or Ruby and Rails, or X and Y), can lead to even more confusion. Write some code in the language first, then add the framework.
We learn to develop, not by using tools, but by solving problems. Run into a few walls, climb over, and find some higher walls!
If you've never done any CGI programming before I think it would be worth doing one project - perhaps just a sample play site just for yourself - using the DIY approach. You'll learn a lot more about how all the various parts work than you would by using a framework. This will help in you design and debug and so on all your future web applications however you write them.
Personally I now use Django. The real benefit is very fast application deployment. The object relational mapping gets things moving fast and the template library is a joy to use. Also the admin interface gives you basic CRUD screens for all your objects so you don't need to write any of the "boring" stuff.
The downside of using an ORM based solution is that if you do want to handcraft some SQL, say for performance reasons, it much harder than it would have been otherwise, although still very possible.
If you are using Python you should not start with CGI, instead start with WSGI (and you can use wsgiref.handlers.CGIHandler to run your WSGI script as a CGI script. The result is something that is basically as low-level as CGI (which might be useful in an educational sense, but will also be somewhat annoying), but without having to write to an entirely outdated interface (and binding your application to a single process model).
If you want a less annoying, but similarly low-level interface, using WebOb would provide that. You would be implementing all the logic, and there will be few dark corners that you won't understand, but you won't have to spend time figuring out how to parse HTTP dates (they are weird!) or parse POST bodies. I write applications this way (without any other framework) and it is entirely workable. As a beginner, I'd advise this if you were interested in understanding what frameworks do, because it is inevitable you will be writing your own mini framework. OTOH, a real framework will probably teach you good practices of application design and structure. To be a really good web programmer, I believe you need to try both seriously; you should understand everything a framework does and not be afraid of its internals, but you should also spend time in a thoughtful environment someone else designed (i.e., an existing framework) and understand how that structure helps you.
OK, rails is actually pretty good, but there is just a little bit too much magic going on in there (from the Ruby world I would much prefer merb to rails). I personally use Pylons, and am pretty darn happy. I'd say (compared to django), that pylons allows you to interchange ints internal parts easier than django does. The downside is that you will have to write more stuff all by youself (like the basic CRUD).
Pros of using a framework:
get stuff done quickly (and I mean lighning fast once you know the framework)
everything is compying to standards (which is probably not that easy to achieve when rolling your own)
easier to get something working (lots of tutorials) without reading gazillion articles and docs
Cons:
you learn less
harder to replace parts (not that much of an issue in pylons, more so with django)
harder to tweak some low-level stuff (like the above mentioned SQLs)
From that you can probably devise what they are good for :-) Since you get all the code it is possible to tweak it to fit even the most bizzare situations (pylons supposedly work on the Google app engine now...).
For smaller projects, rolling your own is fairly easy. Especially as you can simply import a templating engine like Genshi and get alot happening quite quickly and easily. Sometimes it's just quicker to use a screwdriver than to go looking for the power drill.
Full blown frameworks provide alot more power, but do have to be installed and setup first before you can leverage that power. For larger projects, this is a negligible concern, but for smaller projects this might wind up taking most of your time - especially if the framework is unfamiliar.