We Keep Coding

What are the limitations of Django's ORM? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
I've heard developers not wanting to use ORM, but don't know why. What are the shortcomings of the ORM?

Let me start by saying that I fully advocate the use of the ORM for most simple cases. It offers a lot of convenience when working with a very straightforward (relational) data model.
But, since you asked for shortcomings...
From a conceptual point of view, an ORM can never be an effective representation of the underlying data model. It will, at best, be an approximation of your data - and most of the time, this is enough.
The problem is that an ORM will map on a "one class -> one table" basis, which doesn't always work.
If you have a very complex data model - one which, ideally, cannot be properly represented by a single DB table - then you may find that you spend a lot of time fighting against the ORM, rather than having it work for you.
On a practical level, you'll find that there is always a workaround; some developers will be partisan in their support for/against an ORM, but I favour a hybrid approach. The Django works well for this, as you can easily drop into raw SQL as needed. Something like:
Model.objects.raw("SELECT ...")
ORMs take a lot of the work out of the 99.99% of other cases, when you're performing simple CRUD operations against your data.
In my experience, the two best reasons to avoid an ORM altogether are:
When you have complex data that is frequently retrieved via multiple joins and aggregations. Often, writing the SQL by hand will be clearer.
Performance. ORMs are pretty good at constructing optimised queries, but nothing can compete with writing a nice, efficient piece of SQL.
But, when all's said and done, after working extensively with Django, I can count on one hand the number of occasions that the ORM hasn't allowed me to do what I want.

creator of SQLAlchemy's response to the question is django considered now pythonic.. This shows a lots of difference and deep understanding of the system.
sqlalchemy_vs_django_db discussion in reddit
Note: Both the links are pretty long, will take time to read. I am not writing gist of them which may lead to misunderstanding.

Another answer from a Django fan, but:
If you use inheritance and query for parent classes, you can't get children (while you can with SQLAlchemy).
Group By and Having clauses are really hard to translate using the aggregate/annotate.
Some queries the ORM make are just ridiculously long, and sometimes you and up with stuff like model.id IN [1, 2, 3... ludicrous long list]
There is a way ask for raw where "stuff is in field" using __contains, but not "field is in stuff". Since there is no portable way to do this accross DBMS, writting raw SQL for it is really annoying. A lot of small edge cases like this one appear if your application starts to be complex, because as #Gary Chambers said, data in the DBMS doesn't always match the OO model.
It's an abstraction, and sometimes, the abstraction leaks.
But more than often, the people I meet that don't want to use an ORM do it for the wrong reason: intellectual laziness. Some people won't make the effort to give a fair try to something because they know something and want to stick to it. And it's scary how many of them you can find in computer science, where a good part of the job is about keeping up with the new stuff.
Of course, in some area it just make sense. But usually someone with good reason not to use it, will use it in other cases. I never met any serious computer scientist saying to it all, just people not using it in some cases, and being able to explain why.
And to be fair, a lot of programmers are not computer scientists, there are biologists, mathematician, teachers or Bob, the guy next door that just wanna help. From their point of view, it's prefectly logical to not spend hours to learn new stuff when you can do what you want with your toolbox.

There are various problems that seem to arise with every Object-Relational Mapping system, about which I think the classic article is by Ted Neward, who described the subject as "The Vietnam of Computer Science". (There's also a followup in response to comments on that post and some comments from Stack Overflow's own Jeff Atwood here.)
In addition, one simple practical problem with ORM systems is they make it hard to see how many queries (and which queries) are actually being run by a given bit of code, which obviously can lead to performance problems. In Django, using the assertNumQueries assertion in your unit tests really helps to avoid this, as does using django-devserver, a replacement for runserver that can output queries as they're being performed.

One of the biggest problem that come to mind is that Building inheritance into Django ORM's is difficult. Essentially this is due to the fact that (Django) ORM layers are trying to bridge the gap by being both relational & OO. Another thing is of course multiple field foreign keys.
One charge leveled at Django ORM is that they abstract away so much of the database engine that writing efficient, scalable applications with them is impossible. For some kinds of applications - those with millions of accesses and highly interrelated models — this assertion is often true.
The vast majority of Web applications never reach such huge audiences and don't achieve that level of complexity. Django ORMs are designed to get projects off the ground quickly and to help developers jump into database-driven projects without requiring a deep knowledge of SQL. As your Web site gets bigger and more popular, you will certainly need to audit performance as described in the first section of this article. Eventually, you may need to start replacing ORM-driven code with raw SQL or stored procedures (read SQLAlchemy etc).
Happily, the capabilities of Django's ORMs continue to evolve. Django V1.1's aggregation library is a major step forward, allowing efficient query generation while still providing a familiar object-oriented syntax. For even greater flexibility, Python developers should also look at SQLAlchemy, especially for Python Web applications that don't rely on Django.

IMHO the bigger issue with Django ORM is the lack of composite primary keys, this prevents me from using some legacy databases with django.contrib.admin.
I do prefer SqlAlchemy over Django ORM, for projects where django.contrib.admin is not important I tend to use Flask instead of Django.
Django 1.4 is adding some nice "batch" tools to the ORM.

Why don't django templates just use python code?

I mean I understand that these templates are aimed at designers and other less code-savvy people, but for developers I feel the template language is just a hassle. I need to re-learn how to do very simple things like iterate through dictionaries or lists that I pass into the template, and it doesn't even seem to work very well. I'm still having trouble getting the whole "dot" notation working as I would expect (for example, {{mydict.dictkey}} inside a for loop doesn't work :S -- I might ask this as a separate question), and I don't see why it wouldn't be possible to just use python code in a template system. In particular, I feel that if templates are meant to be simple, then the level of python code that would need to be employed in these templates would be of a caliber not more complicated than the current templating language. So these designer peeps wouldn't have more trouble learning that much python than they would learning the Django template language (and there's more places you can go with this knowledge of basic python as opposed to DTL) And the added advantage would be that people who already know python would be in familiar territory with all the usual syntax and power available to them and can just get going.
Am I missing something? If so I plead django noob and would love for you to enlighten me on the many merits of the current system. But otherwise, any recommendations on other template systems that may be more what I'm looking for?

The reason that most people give for limited template languages is that they don't want to mix the business logic of their application with its presentation (that wouldn't work well with the MVC philosophy; using Django I'm sure you understand the benefits of this).
Daniel Greenfeld wrote an article a few days ago explaining why he likes "stupid template languages", and many people wrote responses (see the past few days on Planet Python). If you read what Daniel wrote and how others responded to it, you'll get an idea of some of the arguments for and against allowing template languages to use Python.

Don't forget that you aren't limited to Django's template language. You're free to use whatever templating system you like in your view functions. However you want to create the HTML to return from your view function is fine. There are many templating implementations in the Python world: choose one that suits you better, and use it.

Seperation of concerns.
Designer does design. Developer does development. Templates are written by the designers.
Design and development are independent and different areas of work typically handled by different people.
I guess having template code in python would work very well if one is a developer and their spouse is a designer. Otherwise, let each do his job, with least interference.

Django templates don’t just use Python code for the same reason Django uses the MVC paradigm:
No particular reason.
(ie: The same reason anyone utilizes MVC at all, and that reason is just that certain people prefer this rigid philosophical pattern.)
In general I’d suggest you avoid Django if you don’t like things like this, because the Django people won’t be changing this approach. You could also, however, do something silly (in that it’d be contradictory to the philosophy of the chosen software), like put all the markup and anything else you can into the "view" files, turning Django from its "MVC" (or "MTV" ) paradigm into roughly what everything else is (a boring but straightforward lump).

Form generation/validation libraries in Python

I'm a developer in Python coming from a PHP background. In PHP most frameworks included a decent form generation/validation API (Zend and CakePHP come to mind). At my new company we try to stay away from Django and use Werkzeug extensively.
I've looked at FormEncode and Formular. Formular seems better to me, but there must be stuff my noobish brain is not aware of. Please enlighten me.

There's also WTForms, which is a fairly minimal forms library that integrates well with Werkzeug in my experience.

FormEncode is more a validation library and not very good at generating forms.
Personally, I don't like to intermix output generation (forms) and validation. As I see some shortcomings in FormEncode, which tried to remove in my own implementation called pycerberus. However it doesn't do any form generation at all.
The reason why I don't like to have html generation + validation together is that because:
I need validation also in non-ui contexts like server applications, libraries and I don't want to switch libraries but maintain one toolset.
There are very good tools for output generation and I'd like to choose the one that suits me best. In case I need extra functionality there, the choice is really limited.

Python after Ruby on Rails [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I have been working with Ruby on Rails for over a year now and have been offered some development work with Python. I would like know if development with Python is as enjoyable as Ruby in terms of the clarity and ease of use. And how well is Python suited for Web development. I've heard of Pylons being a direct port of the Rails framework but does it provide the same level of comfort and features. Are there any popular websites built using Python and a framework that offers the same level of flexibilty as Rails.
Because Rails doesn't seem like work.

Django is one of the most famous. It follows a different approach to web devlopment then ruby does, but it is just as powerful and feature rich. An example website running Django is lawrence.com
Pylons is another popular one, I don't know why you heard it was a Rails clone, because it is not. It is a lightweight framework that leverages the power of other open-source projects to give you flexibility in implementation. For example, you can choose to use SQLAlchemy, SQLObject or CouchDB for managing your database. Or you can choose between Mako, Genshi, Jinja2, or whatever you like for your templates. I think you get the picture. Some example website running of pylons are: freebase and Charlie Rose
There exist other web framework as well, but they are less popular.
Notably, TurboGears, which is now built upon Pylons. I would say it tries to pack more juice then pylons does, but it also constrain you more as it assumes more decisions for you. Still, you can stay away from them and do as you please, but it starts with a more constrained framework.
The last one I will mention is Zope, which is the big commercially backed one, that has been there for a while now, but I don't have much experience with it. I do believe it is the less "fun" to work with, but that's just my feeling, you can check it out yourself.
All in all, it comes down to your workflow, I personally, do not enjoy Ruby as a language as much as I do Python and it is natural that I thus like to work with python for web development then Ruby. You really need to try them out yourself, at least the first two I mentioned, try to build a small website, just to get a feel for it. All I can say is from my experience, people either like Rails or Python, not both...
Good Luck!

One very good web development framework is Django

The main two frameworks in Python are Pylons (with the coaligned Turbogears framework) and the more popular Django. Django stomps everything for doing content-based sites (CMS etc) because the admin is excellent.
However, your question makes you sound very much enthused with Ruby and I doubt you'll find anything you like as much. It goes both ways: I'm pretty meh on Rails but really like Python and node.js.

I have done a lot of work with Python in the past year, mostly using Django. I enjoy it, and agree with others that it's great for content-heavy sites. Python and all of its frameworks very much follow the mantra of there being one correct way of doing things. I have learned that most of my pain extending Django lies in me approaching a problem wrongly and need to refactor the code. If you are a precise, logically-driven thinker, you'll enjoy Python a lot.
As far as websites that use Python for a code base, the biggest may be reddit and its family of sites. Django's website also lists sites that use it. I haven't had the privilege of using Pylons, but I also hear good things about it.

Clarity and ease of use are some of Pythons biggest selling points. In saying that, the different Python web frameworks cover almost the entire spectrum from small and simple all the way up to large and complex with everything in between.
You should find that most Python web frameworks have less 'magic' than Rails - ie they are a bit more explicit which is arguably better from the clarity point of view.
In my opinion, even if you enjoy Rails and don't ever plan on leaving, you should still try out other languages and frameworks occasionally to give you a broader perspective.
Personally I like Turbogears2, but I think Django would make a good starting point for a Rails developer that wanted to try out something else.

web2py in the future? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Given the size of web2py and the lack of resources and corporate support, do you think it would be advisable to learn web2py as the only web development framework I know. I'm considersing learning Ruby on Rails or web2py for a website I need to create for as a school project.

web2py does have a smaller market share than competitor products but it is also much younger. I have knowledge of at least 13 consulting companies that provide web2py support. Anyway, I do believe web2py is much easier to use than other systems and therefore you will need less support that you may think. Most of the current users get their support via the web2py google group where you can find 29781 messages and almost all questions have been answered within 24 hours by one of the contributors.

Learning is bad. Sherlock Holmes explains:
"You see," he explained, "I consider
that a man's brain originally is like
a little empty attic, and you have to
stock it with such furniture as you
choose. A fool takes in all the lumber
of every sort that he comes across, so
that the knowledge which might be
useful to him gets crowded out, or at
best is jumbled up with a lot of other
things, so that he has a difficulty in
laying his hands upon it. Now the
skilful workman is very careful indeed
as to what he takes into his
brain-attic. He will have nothing but
the tools which may help him in doing
his work, but of these he has a large
assortment, and all in the most
perfect order. It is a mistake to
think that that little room has
elastic walls and can distend to any
extent. Depend upon it there comes a
time when for every addition of
knowledge you forget something that
you knew before. It is of the highest
importance, therefore, not to have
useless facts elbowing out the useful
ones."
I'm sure I'm not the only one who has wasted an inordinate amount of time wading through the many bad and poorly documented Python web frameworks trying to find one I can just use. If I was programming in Ruby or PHP I probably would have spent that time actually writing a web application. This is the curse of web development in Python.
This bit of flamebait may help:
stackoverflow.com tags about web frameworks http://spreadsheets.google.com/pub?key=tZCdBPAkC75t27UzsPdLfMg&oid=2&output=image
Omitted from the chart are the 13,000+ questions tagged [php], but let's not go there.
To be clear, even though choosing a framework for Python web development can be confusing, once you decide on one you get to program in Python. This is the blessing of web development in Python. It can be really nice.
My advice is don't accept anything less than a framework with excellent documentation. With the amount of choices out there there's no need to settle for poor, incomplete docs. Failing that, the simplest frameworks, those lacking room for any magic, are pleasant to work with and quickly learnable.

web2py may be young, but the mailing list has ~2000 messages / month, which is similar to Django and far more than Turbogears. I usually get answers to my questions within a few hours.
There is also an excellent online book, but I find the best source of information is the mailing list.
I have used both RoR, Django, Turbogears, and web2py, and find web2py the most productive.

Learning is good.
Learning something (that eventually goes away) is no loss at all. The basic skills of web development (HTML, CSS, URL-parsing, GET vs. POST) don't ever change.
Frameworks come and go. Learn as many as you can. Learn how to manage your learning so that you (a) get to the important stuff first and (b) leave the other framework stuff behind when tackling a new framework.
Every framework has it's bias (or focus). Once you figure this out, you can make use of them without all the "compare and contrast" that slows some people down. Once you've learned web2py, you have to be careful learning Django that you start fresh, with no translation from old concepts to new.

Web2py is a good one to learn. If this is going to be deployed to a server, double check it supports wsgi. Sometimes php is the way to go because you know it's supported almost anywhere.

Ask yourself what you are looking to gain from the experience. Ie, is it more important to just get the application built and running with a minimum of time and effort, or are you trying to learn about web stack architecture?
If you're just looking for results, obviously you'll have more code and documentation to borrow from if you stick with a more commonly used framework. If you grit your teeth and accept Django's view of the world, you can build very functional applications very quickly. If you can find some pre-made reusable Django apps that handle part of your problem, it'll be even faster.
But if you want to make sure you have a very solid understanding of everything in the request cycle from HTTP request handling to database access and abstraction to form generation and processing and HTML templating, you'll be bettered served with a minimal framework that forces you to think more about the architecture and has a small enough codebase that you can just read it all top to bottom and not really need documentation beyond that. In that case though, I'd advise going even deeper and building your own framework on top of a WSGI library (you don't actually want to waste time learning the intricacies of working around browser quirks if you can help it). Once you've built your own and seen where things get complicated and where the tradeoffs are, you'll be in an excellent position to judge other frameworks and decide if there's one that does things the way you want to work.

This may seem slightly off-topic, but Paul Graham has probably the best essay on this subject that I have seen: The Python Paradox.
Let me put it this way, if you want to work for me, I notice this kind of free thinking and experimentation on a resume, whether the work was commercial, academic, or otherwise. And I'm pretty sure I'm not alone.

Glad I found this thread! Cause some outdated pages and broken external links on Web2Py's website almost scared me off. But at least now I know there's a pretty good community around Web2Py.
I've just been looking through a load of Python web frameworks, and Web2Py's description sounded enticing and managed to make Django sound overly laborious. Pretty sure there are some tangible benefits to Django's design decisions avoiding "too much magic" when it comes to larger projects.
But to just throw something up on the web with err "sane defaults" sounds perfectly good to me. Instead of throwaway scripts, we can make throwaway websites to handle some temporary thing...
There should be room for an appliance style framework with no installation...
Interesting possibilities for some projects. I saw someone already got a python framework + server to work on android phones :))
For me, thanks to this thread, I will just learn both.
Another thought; if Web2Py is open source and you like what it does you might not even mind being the only user at some point in the future, since you can add features to it yourself?
Mind you, I have not used either yet, just read the docs. I think the Web2Py people should put up a blurb on their website to differentiate themselves from Django in more detail, I haven't been able to check off all my question marks for choosing the right one.

I've already used Java EE and Django. The web2py learning curve is so fast! It's incredible! Things that I was getting a time to develop in three days using java, I can do fastly using web2py. Of course, Web2py has not the same ready plugins that RoR, but, doubtless, we can do these things fastly using web2py. Therefore, is a good opportunity to start learning = )

I'm agree with S.Lott saying that:"Learning something (that eventually goes away) is no loss at all."
YEAH It's true but let me suggest that also a scholastic project should be able to reach the better support possible, otherwise could be very frustrating and a waste of time to learn and teach something not well supported, debugged, stable etc.
The time you spent, and maybe your auditors/students, should in some sense projected with an eye to the future...
just for example take a look to turbogears