I am a newbie to zope and I previously worked on Django for about 2.5 years. So when I first jumped into Zope(v2) (only because my new company is using it since 7 years), I faced these questions. Please help me in understanding them.
What is the "real" purpose of zodb as such? I know what it does, but tell me one great thing that zodb does and a framework like Django (which doesn't have zodb) misses.
Update: Based on the answers, Zodb replaces the need for ORM. You can directly store the object inside the db(zodb itself).
It is said one of the zope's killer feature is the TTW(Through the Web or Developing using ZMI) philosophy. But I(and any developer) prefers File-System based development(using Version control, using Eclipse, using any favorite tool outside Zope). Then where is this TTW actually used?
This is the big one. What "EXTRA Stuff" does Zope's Acquistion gain when compared to Python/Django Inheritance.
Is it really a good move to come to Zope, from Django ?
Any site like djangosnippets.org for Zope(v2)?
First things first: current zope2 versions include all of zope3, too. And if you look at modern zope2 applications like Plone, you'll see that it uses a lot of "zope 3" (now called the "zope tool kit", ZTK) under the hood.
The real purpose of the ZODB: it is one of the few object databases (as opposed to relational SQL databases) that sees real widespread use. You can "just" store all your python objects in there without needing to use an object-relational mapper. No "select * from xyz" under the hood. And adding a new attribute on a zodb object "just" persists that change. Luxurious! Especially handy when your data cannot be handily mapped to a strict relational database. If you can map it easily: just use such a database, I've used sqlalchemy a few times in zope projects.
TTW: we've come back from that. At least, the zope2 way of TTW indeed has all the drawbacks that you fear. No version control, no outside tools, etc. Plone is experimenting (google for "dexterity") with nice explicit zope 3 ways of doing TTW development that can still be mapped back to the filesystem.
TTW: the zodb makes it easy and cheap to store all sorts of config settings in the database, so you can typically adjust a lot of things through the browser. This doesn't really count as typical TTW development, though.
Acquisition: handy trick, though it leads to a huge namespace polution. Double edged sword. To improve debuggability and maintenance we try to do without in most of the cases. The acquisition happens inside the "object graph", so think "folder structure inside the zope site". A call to "contact_form" three folders down can still find the "contact_form" on the root of the site if it isn't found somewhere in between. Double edged sword!
(And regular python object oriented inheritance happens all over the place of course).
Moving from django to zope: a really good idea for certain problems and nonsensical for other problems :-) Quite a lot of zope2/plone companies have actually done some django projects for specific projects, typically those that have 99% percent of their content in a relatively straightforward SQL database. If you're more into content management, zope (and plone) is probably better.
Additional tip: don't focus only on zope2. Zope3's "component architecture" has lots of functionality for creating bigger applications (also non-web). Look at grok (http://grok.zope.org) for a friendly packaged zope, for instance. The pure component architecture is also usable inside django projects.
On the ZODB:
Another way to ask "What is the real purpose of the ZODB?" is to ask, "Why was the ZODB originally created?"
The answer to that is the project was started very early on, around 1996. This was before the existance of MySQL or PostgreSQL, when miniSQL (a free-to-use but not free software) database was still in common use, or big money databases such as Oracle. Python provided the pickle module to serialize Python objects to disk - but serialization is lower level, it doesn't allow for features such as transactions, concurrent writes, and replication. This is what the ZODB provides.
It's still in use today in Zope because it works well. If you have no existing skillset in realational databases, it's easier to learn to use the ZODB than a relational database. It's also usable simpler use-cases, for example if you have a command-line script that needs to store some configuration information, using a relational database means having to run a database server just to store a little bit of configuraiton. You could use a config file, but the ZODB also works quite nicely because it's an embedable database. That means that the database is running in the same process as the rest of your Python code.
It's also worth noting that the API used to store objects inside containers is different between Zope 2 and Zope 3. In Zope 2, containers are stored as attributes:
root.mycontainer.myattr
In Zope 3, they use the same interface as Python standard dictionary type:
root['mycontainer']myattr
This is another reason why it can be easier to learn to use the ZODB than the Django ORM, since Django has it's own interface for it's ORM which is distinct from Python's existing interfaces.
Through-the-web (TTW):
Again, understanding the reason for TTW goes back when Zope was developed. While it seems silly to break with well known developer tools such Subversion or Mercurial, Zope was developed in the late 90s when the only free version control system was CVS. Zope 2 had it's own simple version control capabilites, and they were as good as CVS (which is to say, "they were limited and sucky."). UNIX workstations cost a lot more money back then, and had far fewer resources, so System Administrators were much more guarded and careful about how servers were managed. TTW allowed people who might not normally be able to upload code to the server with sysadmin intervation a way to do that.
With text editors, emacs and vi have had ftp-modes, and Zope 2 can listen on an FTP port. This would allow you to develop so that code was stored in the ZODB (editable TTW), but it was common to edit this code using a emacs or vi.
Today in Zope, TTW is more rarely used or promoted since it no longer makes sense to do this. Disk space is cheap, servers are (relatively) cheap, and there are lots of developer tools which expect to interact with the standard filesystem.
Acquisition:
It was a mistake. It was a very confusing feature that caused lots of unexpected things to happen. In theory there are some interesting ideas to acquisition, but in practice it's best tossed in the bin and has little practical use.
Moving from Django to Zope:
Work started on Zope 3 in 2001. This fixed a lot of the problems with Zope 2. It's a testament to the Zope community that Zope 2 is still actively and well maintained, but it's hardly state-of-the-art. Zope 2 is really only interesting to learn from a historical perspective.
Zope 3 ended up getting evolved in a few different directions, and so modern incarnations of Zope are best expressed in the form of Grok, BFG or Bobo.
Grok is closest to Zope 3, and as such is a pretty large framework - it can be rather overwhelming at times when delving through it's code base. However, just like Django, or any other full-stack framework you don't need to use every part of Grok, it can be quite easy to learn the basic and create web applications with it. It's convention-over-configuration is second to none, and it's class-based Views give it a much tighter, arguably cleaner code base than a Django web application. It's URL routing system is extremely flexible, but also arguably over-engineered.
BFG is a "pay for only what you eat" framework written by long time Zope developer Chris McDonough. As such, it's closer to Pylons in spirit, where only the parts deemed core or essential to a framework are included. It also plays very well with WSGI. It only uses a few core Zope packages.
Bobo is a "micro-framework". It's just a way to route URLs and serve up an app. It doesn't use any Zope packages, so isn't strictly in the Zope family of web frameworks. But it was written by Zope's creator, Jim Fulton, who originally called the publishing part of Zope, "Bobo". The original Bobo, written in the early 90's, mapped URLs to packages and modules, so if your source code was layed out as:
mypackage.mymodule.MyClass
You could have a URL such as:
/mypackage/mymodule/MyClass
Which was very inflexible, and was replaced with URL Traversel in Zope 2, which is fairly complex. Bobo uses Routes, so it's a middle ground between dead-simple URL resolution and complex URL resolution - about the same in complexity as Django's URL resolution machinery.
I answer without much experience on both, but I had the chance to manipulate both, so I can tell you my opinion on some of your questions.
1)What is the "real" purpose of zodb
as such? Meaning I know what it does,
but tell me one great thing that zodb
does and a framework like django(which
doesn't have zodb) misses
Load distribution via ZEO and search via ZCatalog. Django is very low level on this point of view. To achieve the same, you would have to reimplement a lot of wheels, triangular.
Something I learned quite soon is: don't mess with low level database issues. You will screw them up. It's a can of worms, Dune sized.
So why choose django ORM ? You should also consider if YAGNI. django is easy and self contained, documentation is premium, and when (if) your site will grow that much, you will do the switch to a better ORM (or to a pure OODB, in case of ZODB) later on.
2)It is said one of the zope's killer
feature is the TTW(Through the Web or
Developing using ZMI) philosophy. But
I(and any developer) prefers
File-System based development(using
Version control, using Eclipse, using
any favorite tool outside Zope). Then
where is this TTW actually used?
I cannot answer properly to this question, but I would not say that it's fundamentally bad to develop with such approach. Of course it's a change of mindset, and I tend to prefer filesystem based development as well.
4)Is it really a good move to work on
Zope, from Django ?
Zope 3 is very modular, so you are free to use many of its components from django. I would advise against it though. You can, of course, but what I found most problematic is the lack of help. There are not many people using zope components and django at the same time. Sooner or later, you will have a problem and google won't help. At that point, you will realize that if your life was a videogame, you are definitely playing it at level difficult (maybe extreme, if you will have to put your nose into the zope code).
A very good reference on ZODB is ZODB/ZEO programmer's guide. ZODB is not an ORM. Its a true object database. Python objects are persisted inside the database transparently without any worries about how to transform them into a representation suitable for database. Any pickleable Python object can be saved inside the ZODB. Relational databases are suitable for large amount of flat data (like employee records) while ZODB is best for hierarchical data (typically found in web applications). I personally use Zope 3 for my applications. I never did TTW type of work. Best part of using ZODB was the fact that I never had to worry at all about how I am going to save data and how things would change when I upgrade my software from one version to next one. For example, if I add a new attribute to a Python class, all I have to do is provide a default value as a class attribute. It then becomes automatically available to all objects created with the previous version of the same class. Removing an attribute is a simple del operation on existing objects. BTW, ZODB can be used independently in any kind of Python application and isn't coupled with just ZOPE platform. I love the fact that I don't have to worry about the nitty gritties of SQL while working on Python applications thanx to ZODB. And off course if you need a database server so that you can run multiple copies of your application backed by the same server ZEO comes to your rescue on top of ZODB.
Zope started with the idea of being an Object Publishing Environment. From that perspective mapping the URL directly to the object hierarchy in ZODB was great. The URLs simply reflect the hierarchy of objects. Now so far as figuring out the URL is considered, there is always the Rotterdam debugging interface for help. For development work, I keep the development flags on in the zope configuration and look at the contents of ZODB through the Rotterdam interface. Rotterdam skin provide a great way of introspecting the Python objects stored inside the ZODB and figuring out the URLs is much more interactive. Moreover, for major containers inside my ZODB, I register them as persistent utilities inside the site manager (Zope 3 sites and site managers). Anywhere in my code, whenever I need access to such containers, all I do is getUtility(IMyContainerType). I don't even have to remember the detailed locations of those containers inside the code base. They are once registered with the site manager and going forward available anywhere inside the code base through getUtility() calls.
And the URLs also support namespaces. For example using the ++skin++ namespace, you can anytime change the skin of your web application. Using the ++language++ namespace, you can any time change the preferred language of your user interface. Using the ++attributes++ namespace you can access individual attributes of an object. URLs are simply much more powerful and much more customizable. And you can write traversal adapters, define your own namespaces, to enhance the capabilities of your URLs. To give an example, all pages which are directly accessible from the web interface, are part of my default skin. While all pages which are invoked through background AJAX calls, are under a different skin. This way, one can implement different ways of authentication mechanisms in different skins. In main skin, one is redirected to a different login page in case of authentication failure. For AJAX pages, one could simply receive an HTTP error. This could be centrally done. Zope 3 objects have interfaces and one view can be defined for multiple interfaces. Wherever you have an object which supports the given interface, all associated views become automatically available and all such URLs are automatically valid. If you think about it, its a much more powerful than a single python file or XML file where the URLs are hard-coded. I don't really know much about DJango and J2EE so cannot say if they have equivalent capability.
ZODB is a OO-style database that doesn't need a schema definition. You can simply create (nearly) all kinds of objects, and persist them.
The TTW is sometimes annoying, but you can mount the ZOPE-object-tree using webdav. Then you can edit the templates and scripts using your favorite editor.
ZOPE is especially powerful for creating CMS-like systems, IMHO there it is still unmatched - you'd have to go through a lot to make it work equally well in Django.
And through the TTW, actually non-developers like designers have a good chance of developing e.g. templates and CSS without need for developer interaction.
+1 on Wheat's answer, above: "Zope 2 is really only interesting to learn from a historical perspective". I did Zope dev for a large site for a couple of years, 50% zope 2, 50% zope 3. Even then (this was 2 years ago) we were working to migrate everything off of zope 2. Unless you already have a lot invested in an existing Zope 2 project, there's no reason to use it; there's just not much of future there. And if you do have a big existing zope 2 project, I'd suggest taking a look at a product caled Five (a joke: 2 + 3 = 5) that aims to
allow you to integrate Zope 3
technologies into Zope 2. Among
others, it allows you to use Zope 3
interfaces, ZCML-based configuration,
adapters, browser pages (including
skins, layers, and resources),
automated add and edit forms based on
schemas, object events, as well as
Zope 3-style i18n message catalogs.
When all is said and done, Zope 3 is a very different framework from 2, and IMHO, a much better (albeit more complicated) one. TTW is optional, and not recommended for most cases. Implicit acquisition is gone.
Looks like people here have covered why you might want to use the ZODB, so I thought I'd mention one other thing about Zope 3 (or Zope 2 using Five) that's good. Zope has a very powerful system for wiring together different application components called the Zope Component Architecture (ZCA). It allows you to write components that are more or less autonomous and reusable, and which can be plugged together in a standardized way. I mostly do Django development now and I sometimes find myself missing the ZCA. In Django, the ability to write reusable components is limited and kind of ad-hoc. But, like Reinout says zope.component (like most zope packages, including the ZODB) works outside of the zope framework and could be used in a Django project.
That said, the ZCA has its drawbacks, one of which is the tedious process of registering your components in XML files; it always felt a little Java-esqe to me. One reason I really like Grok http://grok.zope.org/ is that it sits on top of zope.component and does much of that grunt work for you.
So bottom line: Zope 2 is mostly a dead end. If your employer is amenable to it, start looking at Zope 3, or at least Five. I think you'll find Zope 3 has a steep learning curve compared to Django, so it might be a good idea to come at it via Grok, which smooths out a lot of Zope 3's rougher edges. But, I think for a really large or complex web application with lots of moving parts, I'd go for Zope over Django (and I say this as someone who really likes Django a lot). For smaller projects, Django would probably be faster. Quantifying "large" and "small" in this context is hard though, and would probably require a couple of thousand more words. If you really are interested in Zope 3, the book by Philipp von Weitershausen is definitely the place to start.
Related
After much research, I've come up with a list of what I think might be the best way of putting together a Python based social network/cms, but have some questions about how some of these components fit together.
Before I ask about the particular components, here are some of the key features of the site to be built:
a modern almost desktop-like gui
future ability to host an advanced html5 sub-application (ex.http://www.lucidchart.com)
high scalability both for functionality and user load
user ability to password protect and permission manage content on per item/group basis
typical social network features
ability to build a scaled down mobile version in the future
Here's the list of tools I'm considering using:
Google App Engine
Python
Django
Pinax
Pyjamas
wxPython
And the questions:
Google App Engine -- this is an attempt to cut to the chase as many pieces of the puzzle seem to be in place.
Question: Am I limiting my options with this choice? Example: datastore not being relational? Should I wait
for SQL support under the Business version?
Python -- I considered 'drupal' at first, but in the end decided that being dependent on modules that may or
may not exist tomorrow + limitations of its templating system are a no-no. Learning its API, too, would be useless elsewhere
whereas Python seems like a swiss army knife of languages -- good for almost anything.
Question: v.2.5.2 is required by GAE, but python.org recommends 2.5.5. Which do I install?
Django -- v.0.96 is built into GAE. You seem to be able to upgrade it.
Questions: Any reason not to upgrade to the latest version? Ways to get around the lack of HTML5 support?
Pinax (http://pinaxproject.com) Rides on top of Django and appears to provide most of the social network functionality
anyone would want.
Question: Reasons NOT to use it? Alternatives?
Pyjamas and wxPython -- this is the part that gets a little confusing. The basic idea behind these is the ability
to build a GUI. I've considered Silverlight and Flash, before the GAE/Python route, but a few working versions of
HTML5 apps convinced me that enough of it ALREADY runs on the latest batch of browsers to chose the HTML5/Javascript
route instead.
Question: How do I extend/supplement Python/Django to build an app-like HTML5 interface? Are Pyjamas and wxPython
the way to go? Or should I change my thinking completely?
Answers to some/any of these questions would be of great help. Please excuse my ignorance if any of this doesn't make much sense.
My last venture into web programming was a decent sized LAMP website some 5-6 years ago. On the desktop side of things,
my programming experience boils down to very high level scripting languages that I keep on learning to accomplish very specific
tasks :)
As someone who has deployed a Django site to GAE, I can tell you that you are not going to reach the ideal solution. Django on GAE misses some of the best aspects of Django because the ORM doesn't work right. The best compromise may be to use Django-nonrel to add the features back in.
This introduces it's own problems though: because of the large number of files and memory used by a Django app you're code will be unloaded from memory quickly after the app becomes idle. That means that visitors will frequently hit an approximately 6 second delay on the first page view after the site's code has been unloaded from memory while GAE uncompresses the zipped modules. Once your site is busy this won't be a problem, but while your site is still young and unknown it will cause the appearance of performance problems. :-(
Second, I've also worked for a company that built a custom CMS and can tell you that the first 80% is pretty easy, especially with modern frameworks. However, the rest can be quite challenging. For example, user roles and custom content types are two challenging aspects. Therefore strongly consider standing on the backs of giants and finding a CMS or CMS framework that almost perfectly meets your needs and then extend it to do that extra bit you need.
So, that said, answering your points:
Yes, you're limiting your options but that may be OK. Most developers are more comfortable with the relational model than the nosql model. Therefore more open source software is built with it in mind. Also, GAE is a closed source platform which is also a deterrent to open source developers. App Engine Oil is a CMS framework that may suit you well and is optimized for App Engine. Also look at web2py which has support for GAE.
I've found myself to be extremely productive with Python. I used to write a lot of PHP now I find it ugly. That said, think about the total line count of code you'll have to write. If you can make Drupal work with high quality pre-made modules you may find yourself only needing 1/10th of the code. By the way, the trick with Drupal is to mainly use only high quality modules. Look at the history, make sure not to use development versions. Try to contact the authors on IRC. I'm not saying you should use Drupal but it is possible to have a reliable site with it (for example, whitehouse.gov)
You're in the classic GAE/Django problem. If you use 0.96 you get great performance but you miss a lot of the great 1.0+ features and you don't get the ORM and all of it's benefits. If you use a newer version of Django you get the performance/memory problems mentioned above.
I'm about to investigate pinax for my company. I've done a very cursor glance at it. I don't know if it has good support for non relational model backends. You'll probably need to look at django-nonrel. However know that you're going to be investing in relatively untried solutions here. A small percentage of Django users use Pinax and an even smaller percentage, if any, use it on a nonrelational backend. Therefore you're going to be in the highly experimental scenario you mentioned in point 2 above.
I can't offer personal experience on it. I've investigated pyjamas a few times. However I like writing HTML CSS and JS. I like to have control. I like progressive enhancement and knowing what users will see if they don't have the full capabilities. Also, I think any new app that doesn't explicitly address mobile clients is implicitly shooting themself in the foot. As many as 15% of Internet users only use the Internet via their smart phone. What kind of experience will they get with pyjamas?
You didn't mention this, but one thing I consider when choosing a platform is vendor lockin and portability. If you develop your solution for GAE and find that you're not able to do what you want, will you be able to port it to another solution elsewhere? How much work will it take? If you code heavily for GAE or make commitments to its architecture, you're stuck with it or with rewriting to move. Using Django or Web2py can help mitigate this.
That said, the big benefit of Python GAE is that you get to be very productive, see your results instantly, get hosting for free while your site is small and get excellent scalability. These are not small things. There is great value there.
I need your advices to choose a Python Web Framework for developing a large project:
Database (Postgresql)will have at least 500 tables, most of them with a composite primary
key, lots of constraints, indexes & queries. About 1,500 views for starting. The project belongs to the financial area. Alwasy new requirements are coming.
Will a ORM be helpful?
Django has been used by many large organizations (Washington Post, etc.) and can connect with Postgresql easily enough. I use it fairly often and have had no trouble.
Yes. An ORM is essential for mapping SQL stuff to objects.
You have three choices.
Use someone else's ORM
Roll your own.
Try to execute low-level SQL queries and pick out the fields they want from the result set. This is -- actually -- a kind of ORM with the mappings scattered throughout the applications. It may be fast to execute and appear easy to develop, but it is a maintenance nightmare.
If you're designing the tables first, any ORM will be painful. For example, "composite primary key" is generally a bad idea, and with an ORM it's almost always a bad idea. You'll need to have a surrogate primary key. Then you can have all the composite keys with indexes you want. They just won't be "primary".
If you design the objects first, then work out tables that will implement the objects, the ORM will be pleasant, simple and will run quickly, also.
Since most of your tables have composite primary keys, you'll want an ORM that supports that functionality. Django's default ORM does not support composite primary keys. SQLAlchemy does have that support (http://www.sqlalchemy.org/features.html).
The TurboGears framework uses SQLAlchemy as its default ORM. Pylons lets you use SQLAlchemy as well.
There are also ways to get Django to use SQLAlchemy, though I've not tried them myself. I prefer to use Django myself, but given your needs, I'd go with Pylons or TurboGears rather then shoe-horning a different ORM into the system.
For such horrid data-layer complexity as 500 tables with 1500 views, differently from most answers, I would personally prefer to stick with SQL (PostgreSQL offers a really excellent implementation thereof, expecially in the new 8.4 version which you should really lobby for if you have any chance); the only ORM I would [grudgingly] accept is SQLAlchemy (one of the few ORBs I don't really mind -- but the main added value is portability to different DBMS: if you're committed to just one, and in a project of this DB complexity you'd better be, then my personal opinion is that any ORM is just overhead, as the data-access layer developers will need deep familiarity with the specific DBMS to crawl towards acceptable performance).
Having picked "raw psycopg2" or SQLAlchemy as the technology for my data-access layer, that would rule out Django (which in my experience only works well with its own ORM -- but that's not suitable for a project of such DB complexity, IMNSHO). I'd go with Werkzeug, personally, as the framework most suitable for highly complex projects requiring ridiculous amounts of flexibility and power -- though Pylons and Turbogears 2 on top of it may be acceptable as a fall-back if the team just doesn't have the web app experience and skill it takes to make truly beautiful music with a flexible framework such as Werkzeug.
Last but not least, I'd strongly lobby for Dojo for the presentation layer on the client -- a rich and strongly structured Javascript framework, offering superbly designed functionality for "local data", host access, &c, optimized for the best that each of several browsers (and plug-ins such as Gears) can offer, as well as advanced UI functionality, seems likeliest to lighten the heavy development burden on the back-end team (in fact, I'd strongly recommend looking at offering an essentially RESTful interface on the server side, and delegate all presentation work to Dojo on the client -- see this site for more, except I'd be thinking of JSON rather than XML as the preferred interchange format). But, I'll readily admit to knowing far less about the UI side of things than about back-ends, business logic and overall architecture, so take this last paragraph for what it's worth!-)
Depending on what you want to do, you actually have a few possible frameworks :
[Django] Big, strong (to the limit of what a python framework can be), and the older in the race. Used by a few 'big' sites around the world ([Django sites]). Still is a bit of an overkill for almost everything and with a deprecated coding approach.
[Turbogears] is a recent framework based on Pylons. Don't know much about it, but got many good feedbacks from friends who tried it.
[Pylons] ( which Turbogears2 is based on ). Often saw at the "PHP of Python" , it allow very quick developements from scratch. Even if it can seem inappropriate for big projects, it's often the faster and easier way to go.
The last option is [Zope] ( with or without Plone ), but Plone is way to slow, and Zope learning curve is way too long ( not even speaking in replacing the ZODB with an SQL connector ) so if you don't know the framework yet, just forget about it.
And yes, An ORM seem mandatory for a project of this size. For Django, you'll have to handle migration to their database models (don't know how hard it is to plug SQLAlchemy in Django). For turbogears and Pylons, the most suitable solution is [SQLAlchemy], which is actually the most complete ( and rising ) ORM for python. For zope ... well, nevermind
Last but not least, I'm not sure you're starting on a good basis for your project. 500 tables on any python framework would scare me to death. A boring but rigid language such as java (hibernate+spring+tapestry or so) seem really more appropriate.
I would absolutely recommend Repoze.bfg with SQLAlchemy for what you describe. I've done projects now in Django, TurboGears 1, TurboGears 2, Pylons, and dabbled in pure Zope3. BFG is far and away the framework most designed to accomodate a project growing in ways you don't anticipate at the beginning, but is far more lightweight and pared down than Grok or Zope 3. Also, the docs are the best technical docs of all of them, not the easiest, but the ones that answer the hard questions you're going to encounter the best. I'm currently doing a similar thing where we are overhauling a bunch of legacy databases into a new web deliverable app and we're using BFG, some Pylons, Zope 3 adapters, Genshi for templating, SQLAlchemy, and Dojo for the front end. We couldn't be happier with BFG, and it's working out great. BFGs classes as views that are actually zope multi-adapters is absolutely perfect for being able to override only very specific bits for certain domain resources. And the complete lack of magic globals anywhere makes testing and packaging the easiest we've had with any framework.
ymmv!
Alwasy new requirements are coming.
So what you really need is a framework that will allow you to adapt rapidly to changing specs.
From personal experience, I can only discuss django, which is great because it allows you to get up and running quickly.
If you stick to its ORM, you will have a pretty easy time getting your models fleshed out and connected in useful ways. You will need to familiarize yourself with a database migration tool, because Django does not have one built in. dmigrations seems to be a leading tool for this.
Another choice for ORM's is SQLAlchemy, which appears to be a bit more mature out of the box.
I've pretty much tried every Python web framework that exists, and it took me a long time to realize there wasn't a silver bullet framework, each had its own advantages and disadvantages. I started out with Snakelets and heartily enjoyed being able to control almost everything at a lower level without much fuss, but then I discovered TurboGears and I have been using it (1.x) ever since. Tools like Catwalk and the web console are invaluable to me.
But with TurboGears 2 coming out which brings WSGI support, and after reading up on the religious debates between the Django and WSGI camps, I'm really torn between "doing it the right way", e.g., learning WSGI, spending valuable time writing functionality that already exists in Django and other full-stack frameworks, as opposed to using Django or some high-level framework that does everything for me. The downsides with the latter that I can see are pretty obvious:
I'm not learning anything in the process
If I ever need to do anything lower level it's going to be a pain
The overhead required for just a basic site which uses authentication is insane. (IMO)
So, I guess my question is, which is the better choice, or is it just a matter of opinion, and should I suck it up and use Django if it achieves what I want with minimal fuss (I want authentication and a CRUD interface to my database)? I tried Werkzeug, Glashammer, and friends, but AuthKit and Repoze scared me off, as well as the number of steps involved to just setup basic authentication. I looked at Pylons, but the documentation seems lacking, and when referencing simple features like authentication or a CRUD interface, various wiki pages and documentation seemed to contradict each other, with different hacks for versions and such.
Thanks to S. Lott for pointing out that I wasn't clear enough. My question is: which of the following is worthwhile in the long run, but not painful in the short (e.g., some sort of middle ground, anyone?) - Learn WSGI, or stick with a "batteries-included" framework? If the latter, I would appreciate a suggestion as to whether I should give Django another try, stick with TurboGears 1.x, or venture into some other framework.
Also, I have tried CherryPy, but couldn't seem to find a good enough CRUD application that I could plop in and use right away.
the religious debates between the Django and WSGI camps
It would seem as though you're a tad bit confused about what WSGI is and what Django is. Saying that Django and WSGI are competing is a bit like saying that C and SQL are competing: you're comparing apples and oranges.
Django is a framework, WSGI is a protocol (which is supported by Django) for how the server interacts with the framework. Most importantly, learning to use WSGI directly is a bit like learning assembly. It's a great learning experience, but it's not really something you should do for production code (nor was it intended to be).
At any rate, my advice is to figure it out for yourself. Most frameworks have a "make a wiki/blog/poll in an hour" type exercise. Spend a little time with each one and figure out which one you like best. After all, how can you decide between different frameworks if you're not willing to try them out?
I'd say you're being a bit too pessimistic about "not learning anything" using Django or a similar full-stack framework, and underestimating the value of documentation and a large community. Even with Django there's still a considerable learning curve; and if it doesn't do everything you want, it's not like the framework code is impenetrable.
Some personal experience: I spent years, on and off, messing around with Twisted/Nevow, TurboGears and a few other Python web frameworks. I never finished anything because the framework code was perpetually unfinished and being rewritten underneath me, the documentation was often nonexistent or wrong and the only viable support was via IRC (where I often got great advice, but felt like I was imposing if I asked too many questions).
By comparison, in the past couple of years I've knocked off a few sites with Django. Unlike my previous experience, they're actually deployed and running. The Django development process may be slow and careful, but it results in much less bitrot and deprecation, and documentation that is actually helpful.
HTTP authentication support for Django finally went in a few weeks ago, if that's what you're referring to in #3.
I suggest taking another look at TG2. I think people have failed to notice some of the strides that have been made since the last version. Aside from the growing WSGI stack of utilities available there are quite a few TG2-specific items to consider. Here are a couple of highlights:
TurboGears Administration System - This CRUD interface to your database is fully customizable using a declarative config class. It is also integrated with Dojo to give you infinitely scrollable tables. Server side validation is also automated. The admin interface uses RESTful urls and HTTP verbs which means it would be easy to connect to programatically using industry standards.
CrudRestController/RestController - TurboGears provides a structured way to handle services in your controller. Providing you the ability to use standardized HTTP verbs simply by extending our RestController. Combine Sprox with CrudRestController, and you can put crud anywhere in your application with fully-customizable autogenerated forms.
TurboGears now supports mime-types as file extensions in the url, so you can have your controller render .json and .xml with the same interface it uses to render html (returning a dictionary from a controller)
If you click the links you will see that we have a new set of documentation built with sphinx which is more extensive than the docs of the past.
With the best web server, ORM, and template system(s) (pick your own) under the hood, it's easy to see why TG makes sense for people who want to get going quickly, and still have scalability as their site grows.
TurboGears is often seen as trying to hit a moving target, but we are consistent about releases, which means you won't have to worry about working out of the trunk to get the latest features you need. Coming to the future: more TurboGears extensions that will allow your application to grow functionality with the ease of paster commands.
Your question seems to be "is it worth learning WSGI and doing everything yourself," or using a "full stack framework that does everything for you."
I'd say that's a false dichotomy and there's an obvious third way. TurboGears 2 tries to provide a smooth path from a "do everything for you" style framework up to an understanding of WSGI middleware, and an ability to customize almost every aspect of the framework to suit your application's needs.
We may not be successful in every place at every level, but particularly if you've already got some TurboGears 1 experience I think the TG2 learning curve will be very, very easy at first and you'll have the ability to go deeper exactly when you need it.
To address your particular issues:
We provide an authorization system out of the box that matches the one you're used to from TG1.
We provide an out of the box "django admin" like interface called the tgext.admin, which works great with dojo to make a fancy spreadsheet like interface the default.
I'd also like to address a couple of the other options that are out there and talk a little bit about the benifits.
CherryPy. I think CherryPy is a great webserver and a nice minimalistic web-framework. It's not based on WSGI internally but has good WSGI support although it will not provide you with the "full stack" experience. But for custom setups that need to be both fast and aren't particularly suited to the defaults provided by Django or TurboGears, it's a great solution.
Django. I think Django is a very nice, tigtly integrated system for developing websites. If your application and style of working fits well within it's standard setup it can be fantastic. If however you need to tune your DB usage, replace the template language, use a different user authorization model or otherwise do things differently you may very likely find yourself fighting the framework.
Pylons Pylons like CherryPy is a great minimalistic web-framework. Unlike CherryPy it's WSGI enabled through the whole system and provides some sane defaults like SQLAlchemy and Mako that can help you scale well. The new official docs are of much better quality than the old wiki docs which are what you seem to have looked at.
Have you taken a look at CherryPy. It is minimalistic, yet efficient and simple. It is low level enough for not it to get in they way, but high enough to hide complexity. If I remember well, TurboGears was built on it.
With CherryPy, you have the choice of much everything. (Template framework, ORM if wanted, back-end, etc.)
Learn WSGI
WSGI is absurdly simple.. It's basically a function that looks like..
def application(environ, start_response) pass
The function is called when an HTTP request is received. environ contains various data (like the request URI etc etc), start_response is a callable function, used to set headers.
The returned value is the body of the website.
def application(environ, start_response):
start_response("200 OK", [])
return "..."
That's all there is to it, really.. It's not a framework, but more a protocol for web-frameworks to use..
For creating sites, using WSGI is not the "right way" - using existing frameworks is.. but, if you are writing a Python web-framework then using WSGI is absolutely the right way..
Which framework you use (CherryPy, Django, TurboGears etc) is basically personal preference.. Play around in each, see which you like the most, then use it.. There is a StackOverflow question (with a great answer) about this, "Recommendation for straight-forward python frameworks"
Have you checked out web2py? After recently evaluating many Python web frameworks recently I've decided to adopt this one. Also check out Google App Engine if you haven't already.
I'd say the correct answer depends on what you actually want and need, as what will be worthwhile in the long run depends on what you'll need in the long run. If your goal is to get applications deployed ASAP then the 'simpler' route, ie. Django, is surely the way to go. The value of a well-tested and well-documented system that exactly what you want can't be underestimated.
On the other hand if you have time to learn a variety of new things which may apply in other domains and want to have the widest scope for customisation then something like Turbogears is superior. Turbogears gives you maximum flexibility but you will have to spend a lot of time reading external docs for things like Repoze, SQLAlchemy, and Genshi to get anything useful done with it. The TG2 docs are deliberately less detailed than the TG1 docs in some cases because it's considered that the external docs are better than they used to be. Whether this sort of thing is an obstacle or an investment depends on your own requirements.
Django is definitely worth learning, and sounds like it will fit your purposes. The admin interface it comes with is easy to get up and running, and it does use authentication.
As for "anything lower level", if you mean sql, it is entirely possible to shove sql into you queries with the extra keyword. Stylistically, you always try to avoid that as much as possible.
As for "not learning anything"...the real question is whether your preference is to be primarily learning something lower-level or higher-level, which is hardly a question anyone here can answer for you.
Pylons seems a great tool for me:
a real web framework (CherryPy is just a web server),
small code base - reuse of other projects,
written entirely with WSGI in mind, based on Paste,
allows you to code the app right away and touch the low level bits if it's necessary,
I've used CherryPy and TurboGears and look at many other frameworks but none of them were so light and productive as Pylons is. Check the presentation at Google.
I'm a TurboGears fan, and this is exactly the reason why: a very nice trade-off between control and doing things right vs. easy.
You'll have to make up your own mind of course. Maybe you'd prefer to learn less, maybe more. Maybe the areas that I like knowledge/control (database for example), you couldn't care less about. And don't misunderstand. I'm not characterizing any frameworks as necessarily hard or wrong. It's just my subjective judgment.
Also I would recommend TurboGears 2 if at all possible. When it comes out, I think it will be much better than 1.0 in terms of what it has selected for defaults (genshi, pylons, SqlAlchemy)
I would suggest for TurboGears 2. They have done a fantastic job of integrating best of Python world.
WSGI: Assuming you are developing moderately complex projects/ business solutions in TG2 or some other framework say Grok. Even though these frameworks supports WSGI does that mean one who is using these frameworks have to learn WSGI? In most cases answer is No. I mean it's good have this knowledge no doubt.
WSGI knowledge is probably is more useful in cases like
you want to use some middleware or some other component which is not provided as part of the standard stack for eg. Authkit with TG or Grok without ZODB.
you are doing some integration.
CherryPy is good but think of handling your database commits/rollbacks at the end of transactions, exposing json, validations in such cases TG, Django like frameworks do it all for you.
Web2py is the secret sauce here. Don't miss checking it out.
Django is my favorite python web framework. I've tried out others like pylons, web2py, nevow and others.
But I've never looked into TurboGears with much enthusiasm.
Now with TG2 out of beta I may give it a try. I'd like to know what are some of the pros and cons compared to Django.
TG2 has several advantages that I think are important:
Multi-database support
sharding/data partitioning support
longstanding support for aggregates, multi-column primary keys
a transaction system that handles multi-database transactions for you
an admin system that works with all of the above
out of the box support for reusable template snipits
an easy method for creating reusable template tag-libraries
more flexibility in using non-standard components
There are more, but I think it's also important to know that Django has some advantages over TG2:
Larger, community, more active IRC channel
more re-usable app-components
a bit more developed documentation
All of this means that it's a bit easier to get started in Django than TG2, but I personally think the added power and flexibility that you get is worth it. But your needs may always be different.
TG2 takes Pylons and changes some defaults - object dispatching instead of Routes, and Genshi instead of Mako. They believe there's only one way to do it, so apps can rely on the same API for any TurboGears website.
Similarities
TG2 and Django both distinguish between websites and components, so you'll eventually see reusable building blocks for TurboGears, too.
Differences
Django uses its own handlers for HTTP, routing, templating, and persistence. Django also has stellar documentation and an established community.
TurboGears defaults to best-of-breed libraries, which apparently are Paste, object dispatching, Genshi, and SqlAlchemy. This philosophy produces a better all-round toolset, but at the risk of instability - because it means throwing away backwards compatibility if and when better libraries appear.
Pros.
SQLAlchemy > django ORM
Multiple template languages out of the box (genshi,mako,jinja2)
more WSGI friendly
Object Dispatch > routes > regexp routing. You can get the first 2 with TG2
Almost all components are optional you can keep the core and use any ORM, template, auth library, etc.
Sprox > django forms
Cons.
- Admin is more basic (no inline objects yet!)
- less third party apps
- "app" system still in the making.
- given it's modularity you need to read documentation from different sources (SQLAlchemy, Genshi or Mako, repoze.who, Pylons, etc.)
I was struggling with the same question months ago and decided for Turbogears 2, and my reasoning was simple. "I'm new to python, I want to learn it not just for web-projects but as a substitute to php for scripting small helpers"
What I didn't like about Django, to me looks like a "close platform". ORM, Template system, sessions, etc they all are Django's
On the other hand, Turbogears 2 uses already known open platforms and just glued them, just like Appfuse does it for Java
With TurboGears 2 I learn SQLAlchemy that I can use later for small python scripts, or from the python shell to solve common tasks.
Main drawbacks are the lack of complete documentation and error messages.
Sometimes you have to search very deep to find simple solutions, the learning curve is steep, but it pays long term. The error messages where to me very confusing (coming from more than 10 years in Java development). I had lost many hours trying to find an "ascii encode error" when the real problem was a module not being imported.
That's my opinion, just remember I'm new to python and I could be wrong about many things stated here.
Besides what Nikhil gave in his answer, I think another minor difference is that Turbogears provdes some support for javascript widgets and integration with Mochikit.
Whereas Django steadfastly remains javascript framework neutral.
(At least this was true with older versions of Turbogears... this might have changed with TG2)
Edit: I just went over TG2 documentation and see that it did indeed change. Turbogears now uses ToscaWidgets which can use jQuery, ExtJS, Dojo, etc. underneath. This nicely makes it more framework neutral while still providing nice javascript widgets.
This strikes me as a pro for Turbogears if you don't have any javascript experience and a pro for Django if you are writing a lot of specialized javascript.
One of the most important questions is not just what technical features this platform provides or that platform provides, but the driving philosophy of the open source project and the nature of the community supporting it.
I've got no dog in this fight myself, but I found Mark Ramm's talk at DjangoCon 2008 to be very interesting on this point (Google will yield no end of subsequent discussion, no doubt).
Because Django uses its own ORM it limits you to learn that ORM for that specific web framework. I think using an web framework with a more popular ORM (like SqlAlchemy which TG uses) increases your employability chances. Just my 2 cents ..
Last I checked, django has a very poor data implementation. And that's a huge weakness in my book. Django's orm doesn't allow me to use the power of the underlying database. For example I can't use compound primary keys, which are important to good db design. It also doesn't support more than a single database, which is not a big deal until you really need it and find that you can't do it without resorting to doing it manually. Lastly if you have to make changes to your database structure in a team-friendly way, you have to try to choose between a set of 3rd party migration tools.
Turbogears seems to be more architecturally sound, doing its best to integrate individual tools that are awesome in their own right. And because TG is more of an integrator, you're able to switch out pieces to suit your preferences. Don't like SQL Alchemy? You can use SQLObject. Don't like Genshi templates? You can use Mako or even django's, although you're not exactly stuck with the default on django either.
Time for tg2's cons:
TG has a much smaller community, and community usually has its benefit.
Django has a much better name. I really like that name ;-)
Django seems simpler for the beginning web developer, with pretty cool admin tools.
TG has decent documentation, but you also need to go to Genshi's site to learn Genshi, SQL Alchemy's site to learn that, etc. Django has great docs.
My 2 cents.
A few weeks ago I asked the question "Is a PHP, Python, PostgreSQL design suitable for a non-web business application?" Is a PHP, Python, PostgreSQL design suitable for a business application?
A lot of the answers recommended skipping the PHP piece and using Django to build the application. As I've explored Django, I've started to question one specific aspect of my goals and how Django comes into play for a non-web business application.
Based on my understanding, Django would manage both the view and controller pieces and PostgreSQL or MySQL would handle the data. But my goal was to clearly separate the layers so that the database, domain logic, and presentation could each be changed without significantly affecting the others. It seems like I'm only separating the M from the VC layers with the Django solution.
So, is it counterproductive for me to build the domain layer in Python with an SQL Alchemy/Elixir ORM tool, PostgreSQL for the database layer, and then still use Django or PHP for the presentation layer? Is this possible or pure insanity?
Basically, I'd be looking at an architecture of Django/PHP > Python/SQLAlchemy > PostgreSQL/MySQL.
Edit: Before the fanboys get mad at me for asking a question about Django, just realize: It's a question, not an accusation. If I knew the answer or had my own opinion, I wouldn't have asked!
You seem to be saying that choosing Django would prevent you from using a more heterogenous solution later. This isn't the case. Django provides a number of interesting connections between the layers, and using Django for all the layers lets you take advantage of those connections. For example, using the Django ORM means that you get the great Django admin app almost for free.
You can choose to use a different ORM within Django, you just won't get the admin app (or generic views, for example) along with it. So a different ORM takes you a step backward from full Django top-to-bottom, but it isn't a step backward from other heterogenous solutions, because those solutions didn't give you intra-layer goodness the admin app in the first place.
Django shouldn't be criticized for not providing a flexible architecture: it's as flexible as any other solution, you just forgo some of the Django benefits if you choose to swap out a layer.
If you choose to start with Django, you can use the Django ORM now, and then later, if you need to switch, you can change over to SQLalchemy. That will be no more difficult than starting with SQLalchemy now and later moving to some other ORM solution.
You haven't said why you anticipate needing to swap out layers. It will be a painful process no matter what, because there is necessarily much code that relies on the behavior of whichever toolset and library you're currently using.
Django will happily let you use whatever libraries you want for whatever you want to use them for -- you want a different ORM, use it, you want a different template engine, use it, and so on -- but is designed to provide a common default stack used by many interoperable applications. In other words, if you swap out an ORM or a template system, you'll lose compatibility with a lot of applications, but the ability to take advantage of a large base of applications typically outweighs this.
In broader terms, however, I'd advise you to spend a bit more time reading up on architectural patterns for web applications, since you seem to have some major conceptual confusion going on. One might just as easily say that, for example, Rails doesn't have a "view" layer since you could use different file systems as the storage location for the view code (in other words: being able to change where and how the data is stored by your model layer doesn't mean you don't have a model layer).
(and it goes without saying that it's also important to know why "strict" or "pure" MVC is an absolutely horrid fit for web applications; MVC in its pure form is useful for applications with many independent ways to initiate interaction, like a word processor with lots of toolbars and input panes, but its benefits quickly start to disappear when you move to the web and have only one way -- an HTTP request -- to interact with the application. This is why there are no "true" MVC web frameworks; they all borrow certain ideas about separation of concerns, but none of them implement the pattern strictly)
You seem to be confusing "separate layers" with "separate languages/technologies." There is no reason you can't separate your concerns appropriately within a single programming language, or within an appropriately modular framework (such as Django). Needlessly multiplying programming languages / frameworks is just needlessly multiplying complexity, which is likely to slow down your initial efforts so much that your project will never reach the point where it needs a technology switch.
You've effectively got a 3 layer architecture whether you use Django's ORM or SQLAlchemy, though your forgo some of the Django's benefits if you choose the latter.
Based on my understanding, Django would manage both the view and controller pieces and PostgreSQL or MySQL would handle the data.
Not really, Django has its own ORM, so it does separate data from view/controller.
here's an entry from the official FAQ about MVC:
Where does the “controller” fit in, then? In Django’s case, it’s probably the framework itself: the machinery that sends a request to the appropriate view, according to the Django URL configuration.
If you’re hungry for acronyms, you might say that Django is a “MTV” framework – that is, “model”, “template”, and “view.” That breakdown makes much more sense.
At the end of the day, of course, it comes down to getting stuff done. And, regardless of how things are named, Django gets stuff done in a way that’s most logical to us.
There's change and there's change. Django utterly seperates domain model, business rules and presentation. You can change (almost) anything with near zero breakage. And by change I am talking about meaningful end-user focused business change.
The technology mix-n-match in this (and the previous) question isn't really very helpful.
Who -- specifically -- benefits from replacing Django templates with PHP. It's not more productive. It's more complex for no benefit. Who benefits from changing ORM's. The users won't and can't tell.