I am quite new to Python and I have seen that both
Entries.objects.raw("SELECT * FROM register_entries")
and
Entries.objects.filter()
do the same query.
In which cases is better to use one or the other?
It depends on the database backend that you are using. The first assume that you have a SQL-based database engine. That is not always true. At the opposite, the second one will work in any case (if the backend is designed for). There was for instance few years ago a LDAP backend which was designed so, but LDAP queries do not use SQL language at all.
In all cases, I advice you to use the second one. It is the better way to go if you want to make reusable and long-term code.
There are also other ideas to prefer the second one to the first one
avoiding possible SQL injections ;
no need to know about database design (table's name, fields' name) ;
generic code is better than specific one ;
and moreover, it is shorter...
But you sometimes will have to use the first one when you do specific operations (calling specific backend's functions), but avoid them as much as possible.
In a nutshell, use the second one!!!
From django documentation:
When the model query APIs don’t go far enough, you can fall back to writing raw SQL
For all aspects, django queryset api offers you many options to customize your queries. But in some cases, you need to use very specific queries where django api become insufficient. But before you go for raw SQL, it is best to read all Queryset Api docs and learn everything about django queryset api.
Related
I'm building a web app using Django and couchdb 2.0.
The new version of couchdb doesn't support temporary views. They recommend using a Mongo query but I couldn't find any useful documentation.
What is the best approach or library to use couchdb 2.0 with Django?
Temporary views were indeed abandoned in CouchDB 2.0. With mango, you could emulate them using a Hack, but that's just as bad (read: performance-wise). The recommendation is to actually use persistent views. As only the delta of new or updated documents need indexing, this will likely need significantly less resources.
As opposed to relational DBs, the created view (which is a persisted index by keys), is meant to be queried many times with different parameters (there is no such a thing as a query optimizer taking your temp view definition or something). So, when you're built heavily on temporary views, you might consider changing the way you query in the first place. One place to start is thinking about which attribute will collapse the result set most quickly to what you're looking for and build a view for that. Then, go query this view with keys and post-filter for the rest.
The closest thing you can do to a temporary view (when you really, really need it) is creating a design doc (e.g. _design/temp<uuid>) and use it for the one query execution.
Just to add a link (not new - but timeless) on the details: http://guide.couchdb.org/draft/views.html
In MVC frameworks, where do you usually embed SQL Alchemy Code, is it ideal to put the query in Controller Methods or just use the Model Methods?
query = session.query(User, Document, DocumentsPermissions).join(Document).join(DocumentsPermissions).filter_by(Document.name=="Something")
Or I just delegate this to a Model Method which takes a args? What is the preferred way to do this? One of the benefits of the latter is that it can be re-used and it almost presents a view for the API programmers. Another advantage is that I can easily over-ride this if I make it a class method. This is usually helpful in customizations especially in commercial softwares.
#Ctrl.py
self.getdocument("Foo")
#Mdl.py
def getdocument(name):
query = session.query(User, Document, DocumentsPermissions).join(Document).join(DocumentsPermissions).filter_by(Document.name=="Something")
TL;DR: Isn't the concept of "M" in MVC blurred when you use ORM's like SQL Alchemy? I didn't have any problems with Model View Controller design patterns.
[PS: I am not sure if this belongs to Code Review Site, if so please let me know, I can just transfer over.]
I strongly prefer the second approach. It has a few advantages:
Your controller code can be dumb. This is good. Controllers that just fetch data from the backend, possibly reformat it a little bit, and pass it on to views are very easy to reason about.
It's easier to test that method in isolation. You can run getdocument('valid_name'), getdocument(None), getdocument(123), etc. to ensure they all work or fail as expected without dealing with all the surrounding controller code.
It's easier to test the controller. You can write a mock for getdocument() so that it always returns a known value and test that your controller processes it correctly.
I tend to put database query code in the Controller rather than the Model. As my understanding goes, Model methods are used to transform the data of the model into something else.
For example, a UserModel may have a FullName() method to return the concatenation of the user's first and last names.
Whereas, a UserController contains a GetAll() method to get a list of all users, which is where the database query code is found.
This is a bit of a strange question, I know, but bear with me. We've developed a RESTful platform using Python for one of our iPhone apps. The webapp version has been built using Django, which makes use of this API as well. We were thinking it would be a great idea to use Django's built-in control panel capabilities to help manage the data.
This itself isn't the issue. The problem is that everyone has decided it would be best of the admin center was essentially a client that sits on top of the RESTful platform.
So, my question is, is there a way to manipulate the model layer of Django to access our API directly, rather than communicated directly with the database? The model layer would act as the client passing requests and responses to and from the admin center.
I'm sure this is possible, but I'm not so sure as to where I would start. Any input?
I remember I once thought about doing such thing. At the time, I created a custom Manager using a custom QuerySet. And I overrode some methods such as _filter_or_exclude(), count(), exists(), select_related(), ... and added some properties. It took less than a week to become a total mess that had probably no chance to work one day. So I immediately stopped everything and found a more suitable solution.
If I had to do it once again, I would take a long time to consider alternatives. And if it really sounds like the best thing to do, I'd probably create a custom database backend. This backend would, rather than converting Django ORM queries to SQL queries, convert them to HTTP requests.
To do so, I think the best starting point would be to get familiar with django source code concerning database backends.
I also think there are some important things to consider before starting such development:
Is the API able to handle any Django ORM request? Put another way: Will any Django ORM query be translatable to an API request?
If not, may "untranslatable" queries be safely ignored? For instance, an ORDER BY clause might be safe to ignore. While a GROUP BY clause is very unlikely to be safely dismissed.
If some queries can't be neither translated nor ignored, may them be reasonably emulated. For instance, if your API does not support a COUNT() operation, you could emulate it by getting the whole data and count it in python with len(), but is this reasonable?
If they are still some queries that you won't be able to handle (which is more than likely): Are all "common" queries (in this case, all queries potentially used by Django Admin) covered and will it be possible to upgrade the API if an uncovered case is discovered lately or is introduced in a future version of Django?
According to the use case, there are probably tons of other considerations to take, such as:
the integrity of the data
support of transactions
the timing of a query which will be probably much higher than just querying a local (or even remote) database.
I am building a website using Django, and this website uses blocks which are enabled for a certain page.
Right now I use a textfield containing paths were a block is enabled. When a page is requested, Django retrieves all blocks from database and does re.search on the TextField.
However, I was wondering if it is not a better idea to use a separate DB table for block/paths, were each row contains a single path and reference to a block, in terms of overhead.
A seperate DB table is definitely the "right" way to do it, because mysql has to send all the data from your TEXT fields every time you query. As you add more rows and the TEXT fields get bigger, you'll start to notice performance issues and eventually crash the server. Also, you'll be able to use VARCHAR and add a unique index to the paths, making lookups lightning fast.
I am not exactly familiar with Django, but if I am understanding the situation correctly, you should use a table.
In fact this is exactly the kind of use that DB software is designed and optimized for.
No worries. It will actually be faster.
By doing the search yourself, you are trying to implement part of the DB logic on your own. Fun, certainly, but not so fast. :)
Here are some nice links on designing a database:
http://dev.mysql.com/tech-resources/articles/intro-to-normalization.html
http://en.wikipedia.org/wiki/Third_normal_form
Hope this helps. Good luck. :-)
I'm a firm believer of the heretic thought of tight coupling between the backend and frontend: I want existing, implied knowledge about a backend to be automatically made use of when generating user interfaces. E.g., if a VARCHAR column has a maximum with of 20 characters, there GUIs should automatically constrain the user from typing more than 20 characters in a related form field.
And I have strong antipathy to ORMs which want to define my database tables, or are based on some hack where every table needs to have extra numeric ID columns because of the ORM.
I've looked a bit into Python database frameworks and I think I can conclude the SQLAlchemy fits best to my mentality.
Now, I need to find a web application framework which fits naturally with SQLAlchemy (or an equivalent) and perhaps even with my appetite for coupling. With "web application framework", I mean products/project such as Pyhons, Django, TurboGears, web2py, etc.
E.g., it should ideally be able to:
automatically select a suitable form widget for data entering a given column if told to do so; e.g., if the column has a foreign key to a column with 10 different values, widget should display the 10 possible values as a dropdown
auto-generate javascript form validation code which gives the end-user quick error feedback if a string is entered into a field which is about to end up in an INTEGER column, etc
auto-generate a calendar widget for data which will end up in a DATE column
hint NOT NULL constraints as javascript which complains about empty or whitespace-only data in a related input field
generate javascript validation code which matches relevant (simple) CHECK-constraints
make it easy to avoid SQL injection, by using prepared statements and/or validation of externally derived data
make it easy to avoid cross site scripting by automatically escape outgoing strings when appropriate
make use of constraint names to generate somewhat user friendly error messages in case a constrataint is violated
All this should happen dynamically, so table adjustments are automatically reflected on the frontend - probably with a caching mechanism, so that all the model introspection wouldn't kill performance. In other words, I don't want to repeat my model definition in an XML file (or alike) when it has already been carefully been defined in my database.
Does such a framework exist for Python (or for any language, for that matter)? If not: Which of the several Python web application frameworks will be least in the way if I were to add parts of the above features myself?
web2py does most of what you ask:
Based on a field type and its validators it will render the field with the appropriate widget. You can override with
db.table.field.widget=...
and use a third party widget.
web2py has js to blocks the user from entering a non-integer in a integer field or a non-double in a double field. time, date and datetime fields have their own pickers. These js validation work with (not instead) of server side validation.
There is IS_EMPTY_OR(...) validator.
The DAL prevents SQL injections since everthing is escaped when goes in the DB.
web2py prevents XSS because in {{=variable}}, 'variable' is escaped unless specified otherwise {{=XML(variable)}} or {{=XML(variable,sanitize=True)}}
Error messages are arguments of validators for example
db.table.field.requires=IS_NOT_EMPTY(error_message=T('hey! write something in here'))
T is for internationalization.
You should have a look at django and especially its newforms and admin modules. The newforms module provides a nice possibility to do server side validation with automated generation of error messages/pages for the user. Adding ajax validation is also possible
TurboGears currently uses SQLObject by default but you can use it with SQLAlchemy. They are saying that the next major release of TurboGears (1.1) will use SQLAlchemy by default.
I know that you specificity ask for a framework but I thought I would let you know about what I get up to here. I have just undergone converting my company's web application from a custom in-house ORM layer into sqlAlchemy so I am far from an expert but something that occurred to me was that sqlAlchemy has types for all of the attributes it maps from the database so why not use that to help output the right html onto the page. So we use sqlAlchemy for the back end and Cheetah templates for the front end but everything in between is basically our own still.
We have never managed to find a framework that does exactly what we want without compromise and prefer to get all the bits that work right for us and write the glue our selves.
Step 1. For each data type sqlAlchemy.types.INTEGER etc. Add an extra function toHtml (or many maybe toHTMLReadOnly, toHTMLAdminEdit whatever) and just have that return the template for the html, now you don't even have to care what data type your displaying if you just want to spit out a whole table you can just do (as a cheetah template or what ever your templating engine is).
Step 2
<table>
<tr>
#for $field in $dbObject.c:
<th>$field.name</th>
#end for
</tr>
<tr>
#for $field in dbObject.c:
<td>$field.type.toHtml($field.name, $field.value)</td>
#end for
</tr>
</table>
Using this basic method and stretching pythons introspection to its potential, in an afternoon I managed to make create read update and delete code for our whole admin section of out database, not yet with the polish of django but more then good enough for my needs.
Step 3 Discovered the need for a third step just on Friday, wanted to upload files which as you know needs more then just the varchar data types default text box. No sweat, I just overrode the rows class in my table definition from VARCHAR to FilePath(VARCHAR) where the only difference was FilePath had a different toHtml method. Worked flawlessly.
All that said, if there is a shrink wrapped one out there that does just what you want, use that.
Disclaimer: This code was written from memory after midnight and probably wont produce a functioning web page.
I believe that Django models does not support composite primary keys (see documentation). But perhaps you can use SQLAlchemy in Django? A google search indicates that you can. I have not used Django, so I don't know.
I suggest you take a look at:
ToscaWidgets
DBSprockets, including DBMechanic
Catwalk. Catwalk is an application for TurboGears 1.0 that uses SQLObject, not SQLAlchemy. Also check out this blog post and screencast.
FastData. Also uses SQLObject.
formalchemy
Rum
I do not have any deep knowledge of any of the projects above. I am just in the process of trying to add something similar to one of my own applications as what the original question mentions. The above list is simply a list of interesting projects that I have stumbled across.
As to web application frameworks for Python, I recommend TurboGears 2. Not that I have any experience with any of the other frameworks, I just like TurboGears...
If the original question's author finds a solution that works well, please update or answer this thread.