Django: custom serialization options? - python

I'm working on a Django-based web service and I'm trying to figure out what the best way to do my serialization will be.
The tricky requirement, though, is that I'd like to have pretty much full control over format of, and fields contained in, the response.
For example, the Django serializers (which, unfortunately, includes the wadofstuff serializer) automatically wrap the fields in { model: "app.Model", pk: 42, fields: { ... }}, which is great for creating fixtures, but isn't great for me — I'd like full control over the output.
Additionally, I'd like a serializer that is aware of Django's objects so, for example, it will do the Right Thing with a QuerySet or ManyToManyField.
Currently I'm thinking of using django-piston's emitters.py, but my experience with django-piston has only been mediocreª, so I'd like to see if there are other options.
So, are there any other options for customizable Django serializers?
ª: It's sparsely documented and tested, and I've had some problems with the serializer.

Have you looked at django-piston? It should have a bunch of stuff to make this easier.
(Not sure about serialization specifically, but Django RESTy web services.)

When I need some custom serialization really fast and my case doesn't require deserialization I just write django template that can make any format I want from list/queryset/object. You can then call render_to_string with proper context and you have your data serialized.
UPDATE: some short example
Let's say you want to get json format accepted by datatables.net plugin. Since there are some special parameters required serializing queryset using simplejson or sth else is impossible here (or might be not so easy at least). We found that fastest way to provide such structure is to create simple template like this one:
{
"sEcho": {{sEcho}},
"iTotalRecords": {{iTotalRecords}},
"iTotalDisplayRecords": {{iTotalDisplayRecords}},
"aaData":[
{% for obj in querySet %}
[
"{{obj.name}}",
"{{obj.message|truncatewords:20}}",
"{{obj.name}}"
]
{% if not forloop.last %}
,
{% endif %}
{% endfor %}
]
}
which renders to beatiful json that we were looking for. It gives you full control over the format also. Other advantages is the abillity to modify objects fields by using built-in django filters which in our case was really usefull.
I know this is not serialization as described in books but if f you want to convert some object to a custom format this solution might be the fastest one. For some reason django developers allowed to render template to any given format not only html, so why not to use it?
Example shown above is very specific but you can generate any other format. Of course writing deserializer for that could restore object from this format might be painfull but if you don't need it...

EDIT :
Now out at https://bitbucket.org/sebpiq/any2any/
I am currently writing a full-featured serialization framework for Django. The aim is precisely to have full control over the serialization. It would probably fill-in your requirements pretty well ! However, it is not ready yet. I estimate that in 1 or 2 weeks I will be able to release a first version.
You can still check the google code : http://code.google.com/p/django-serializable/ , even give some help if you are interested in it.
There will be a featured download when the first release will be out !

Related

working with serializers in the Django-Rest Framework

I don't know is this a right platform to ask this question or not. I recently came across Django rest framework. I am using django for one of my projects.I want to use DRF as an api client for my project. I came across this term serializing the model instances and serializers. I went through DRF Tutorial , but I still have no idea of how it actually works.My questions are :
1.Why are serializers used? What purpose they serve?
2.What more can be done using serializers?
3.How do serializers work?
There may be lots of references on this present in the internet but I want a simple and easy-to-understand answer to this. Thanks!!
1.Why are serializers used? What purpose they serve?
In a very general way:
Imagine you want to invite your friend for a coffee. To inform your friend, you text "what: coffee, time: 12h, where: The fancy coffee bar on the corner". This text is your serialization of your idea.
The DRF does this for your Django models.
Imagine you have a JS client that doesn't (want to) know about your DB or Django models. You serialize your Django objects into JSON to have that data available in a JS compatible way:
data = {
"address": {
"street": "Fancy Street 1b"
}
};
DRF offers an easy way to translate Django model instances into JSON (or other formats). And that is what serializers do. They also offer to deserialize what your JS frontend sends back to your server.
2.What more can be done using serializers?
You can add more fields that are computed at serialization time. For example, instead of adding simple IDs for related objects you can also serialize the related objects and add them nested into the JSON (or XML or whatever format).
You can also add fields that you only need on the frontend side for some view, like things related to paging etc.
3.How do serializers work?
For python, the simplest serializer from python to JSON would be one that takes a python dictionary and simply creates string keys from the python keys, translates Django boolean into true and false strings, python None into null etc.
For more complex translations, you might want to add your own logic. E.g. like resolving related object references on Django models. Or formatting date objects into ISO standard strings (with timezone), localizing (decimal) numbers, localizing texts in general etc.

How a django form is passed to a html template

I am new in Django framework and trying to understand how it works and what is its structure. I am just curious about how a django form is passed to a html template ? Any help would be appriciated.
I think you are curious in how django converts a python object into HTML, its internal mechanism, so the tutorial might not cover what it is.
I did have the same question before, but if you do look at code in forms.py https://github.com/django/django/blob/master/django/forms/forms.py you will see that internally there are methods which will look at the attributes of the objects that you have declared and generate snipplets of html code which will then be rendered.
Of course I cant tell you exactly how it works, I leave the heavy lifting to django.. Isn't that why we use a modern web framework like django in the first place.
Hope you will find this useful

How can I display multiple django modelformset forms in grouped fieldsets?

I have a problem with needing to provide multiple model backed forms on the same page. I understand how to do this with single forms, i.e. just create both the forms call them something different then use the appropriate names in the template.
Now how exactly do you expand that solution to work with modelformsets? The wrinkle, of course, is that each 'form' must be rendered together in the appropriate fieldset.
For example I want my template to produce something like this:
Home Base Description:
Want ice cream?
Home Base Description:
Want ice cream?
I am using a loop like this to process the results (after form validation)
base_models = base_formset.save(commit=False)
like_models = like_formset.save(commit=False)
for base_model, likes_model in map(None, base_models, likes_models):
which works as I'd expect (I'm using map because the # of forms can be different). The problem is that I can't figure out a way to do the same thing with the templating engine. The system does work if I layout all the base models together then all the likes models after wards, but it doesn't meet the layout requirements.
EDIT: Updated the problem statement to be more clear about what exactly I'm processing (I'm processing models not forms in the for loop)
In the view:
forms = itertools.izip(base_forms, likes_forms)
In the template:
{% for (base_form,like_form) in forms %}
After doing a fair amount of poking around and hack experimenting I've come up with the following solution thanks in huge part to Ignacio Vazquez-Abrams :)
In the view:
forms = itertools.izip_longest((None,),base_formset.forms, likes_formset.forms)
In the template:
{% for (garbage1, base_form, like_form, garbage2) in forms %}
The astute reader might notice that the number of arguments in the unpacking list is one larger than what was given to the izip_longest() method. You might also note that there is an effectively blank list as the first argument.
I could never get the template to display the first argument in the list hence the dummy first argument. Also I found that N-1 list elements were being rendered in the template. I also stumbled upon the fact that the template doesn't much care about a size mismatch so by padding the front and the back I was able to get the forms I actually wanted to display!
NOTE: When processing the POST I simply construct the formsets that I'm working with since all the phantom data isn't being sent back through the POST.
Certainly not the cleanest solution and it is probably extremely vulnerable to upgrade breakage, but it's practical enough to work for me for now.

How to implement Symfony Partials or Components in Django?

I've been developing in the Symfony framework for quite a time, but now I have to work with Django and I'm having problems with doing something like a "component" or "partial" in Symfony.
That said, here is my goal:
I have a webpage with lots of small widgets, all these need their logic - located in a "views.py" I guess. But, how do I tell Django to call all this logic and render it all as one webpage?
It sounds like what you're looking for is something like custom template tags...
You can write your own set of tags that process custom logic and return template chunks that are reusable in a very widget-like way.
Assuming you are going to be using the components in different places on different pages I would suggest trying {% include "foo.html" %}. One of the (several) downsides of the Django templating language is that there is no concept of macros, so you need to be very consistent in the names of values in the context you pass to your main template so that the included template finds things it's looking for.
Alternatively, in the view you can invoke the template engine for each component and save the result in a value passed in the context. Then in the main template simply use the value in the context.
I'm not fond of either of these approaches. The more complex your template needs become the more you may want to look at Jinja2. (And, no, I don't buy the Django Party Line about 'template designers' -- never saw one in my life.)

Rendering JSON objects using a Django template after an Ajax call

I've been trying to understand what's the optimal way to do Ajax in Django. By reading stuff here and there I gathered that the common process is:
formulate your Ajax call using some JavaScript library (e.g., jQuery), set up a URL pattern in Django that catches the call and passes it to a view function
in the Python view function retrieve the objects you are interested in and send them back to the client in JSON format or similar (by using the built in serializer module, or simplejson)
define a callback function in JavaScript that receives the JSON data and parses them, so to create whatever HTML is needed to be displayed. Finally, the JavaScript script puts the HTML wherever it should stay.
Now, what I still don't get is how are Django templates related to all of this? Apparently, we're not making use of the power of templates at all.
Ideally, I thought it'd be nice to pass back a JSON object and a template name, so that the data could be iterated over and an HTML block is created. But maybe I'm totally wrong here...
The only resource I found that goes in this direction is this snippet (769) but I haven't tried it yet.
Obviously, what's going to happen in this case is that all the resulting HTML is created on the server side, then passed to the client. The JavaScript-callback function only has to display it in the right place.
Does this cause performance problems? If not, even without using the snippet above, why not formatting the HTML directly in the backend using Python instead of the front-end?
Many thanks!
UPDATE: please use snippet 942 because it is an enhanced version of the one above! I found that the inheritance support works much better this way..
Hey thanks vikingosegundo!
I like using decorators too :-).
But in the meanwhile I've been following the approach suggested by the snippet I was mentioning above. Only thing, use instead the snippet n. 942 cause it's an improved version of the original one. Here's how it works:
Imagine you have a template (e.g., 'subtemplate.html') of whatever size that contains a useful block you can reuse:
........
<div id="results">
{% block results %}
{% for el in items %}
<li>{{el|capfirst}}</li>
{% endfor %}
{% endblock %}
</div><br />
........
By importing in your view file the snippet above you can easily reference to any block in your templates. A cool feature is that the inheritance relations among templates are taken into consideration, so if you reference to a block that includes another block and so on, everything should work just fine. So, the ajax-view looks like this:
from django.template import loader
# downloaded from djangosnippets.com[942]
from my_project.snippets.template import render_block_to_string
def ajax_view(request):
# some random context
context = Context({'items': range(100)})
# passing the template_name + block_name + context
return_str = render_block_to_string('standard/subtemplate.html', 'results', context)
return HttpResponse(return_str)
Here is how I use the same template for traditional rendering and Ajax-response rendering.
Template:
<div id="sortable">
{% include "admin/app/model/subtemplate.html" %}
</div>
Included template (aka: subtemplate):
<div id="results_listing">
{% if results %}
{% for c in results %}
.....
{% endfor %}
{% else %}
The Ajax-view:
#login_required
#render_to('admin/app/model/subtemplate.html')#annoying-decorator
def ajax_view(request):
.....
return {
"results":Model.objects.all(),
}
Of course you can use render_to_response. But I like those annoying decorators :D
There's no reason you can't return a rendered bit of HTML using Ajax, and insert that into the existing page at the point you want. Obviously you can use Django's templates to render this HTML, if you want.
When you are doing Ajax I don't think you have any use for templates.
Template is there so that you can generate dynamic HTML on the server side easily and hence it provides few programming hooks inside HTML.
In case of Ajax you are passing JSON data and you can format it as you want in Python.
and HTML/document elements will be generated on client side using the JSON by some JavaScript library e.g. jQuery on client side.
Maybe if you have a very specific case of replacing some inner HTML from server side HTML then maybe you can use templates but in that case why you would need JSON?
You can just query the HTML page via Ajax and change inner or outer or whatever HTML.
Templates are for the purpose of presentation. Responding with data in format X (JSON, JSONP, XML, YAML, *ml, etc.) is not presentation, so you don't need templates. Just serialize your data into format X and return it in an HttpResponse.
While templates are indeed just for presentation purposes, it shouldn't matter if you are doing it on the serverside or client side. It all comes down to separating the control logic that is performing an action, from the view logic that is just responsible for creating the markup. If your javascript control logic is having to handle how you are rendering or displaying the HTML, then you might be doing it wrong, but if you isolate that rendering logic to another object or function, and just passing it the data necessary for the render, then you should be fine; it mirrors how we separate our controllers, models and views on the server side.
Take a look at the github project: http://github.com/comolongo/Yz-Javascript-Django-Template-Compiler
It compiles django templates into optimized javascript functions that will generate your presentation html with data that you pass it. The compiled functions are in pure javascript, so there are no dependencies on other libraries. Since the templates are compiled instead of being parsed at runtime, the strings and variables are all already placed into javascript strings that just need to be concatenated, so you get a huge speed increase compared to techniques that require you to do dom manipulation or script parsing to get the final presentation. Right now only the basic tags and filters are there, but should be enough for most things, and more tags will be added as people start making requests for them or start contributing to the project.
You can use jquery.load() or similar, generating the HTML on the server and loading it into the DOM with JavaScript. I think someone has called this AJAH.
Unfortunately, Django templates are designed to be executed server side only. There is at least one project to render Django templates using Javascript, but I haven't used it and so I don't know how fast, well supported or up to date it is. Other than this, you have to either use the Django templates on the server or generate dynamic elements on the client without using templates.

Categories