Weird runtime error when looping over DefaultDict in django template - python

Using Django 1.5.1
I have a simple code here -
count_by_media_type = defaultdict(int)
for user_media in user_media_data:
count_by_media_type[user_media['media_type']] += 1
This is a part of a view, which is rendered in the template using a for iteration loop -
{% for media_type in count_by_media_type %}
..........
{% endfor %}
The value of count_by_media_type just before the render_to_response method in the loop is
defaultdict(<type 'int'>, {u'photo': 1, u'audio': 4, u'video': 3})
But somehow, on rendering , there is a runtime error in iteration
dictionary changed size during iteration
And the value of count_by_media_type as seen in the template context becomes
defaultdict(<type 'int'>, {u'photo': 1, u'audio': 4, u'video': 3, u'media_type': 0})
This is quite weird, as how does a new key media_type is coming in the variable?
Now , when I switch the above code, mentioned in the view with
count_by_media_type = Counter([user_media['media_type'] for user_media in user_media_data])
Everything works fine.
Any clues anyone?

Using defaultdicts causes weird behaviour in Django templates, because of the way template variable lookups work. See the Behind the scenes box of the Django docs.
The Django docs suggest converting the defaultdict to a regular dict before passing to the template.
count_by_media_type = defaultdict(int)
for user_media in user_media_data:
count_by_media_type[user_media['media_type']] += 1
count_by_media_type = dict(count_by_media_type)
Or, as this answer suggests, you can disable the defaulting feature after you have finished inserting values.
count_by_media_type.default_factory = None

Related

Django : how to use modified queryset in templates without saving modification

In my views.py, I do:
context_data = {}
all_answers = Answer.objects.all()
for i in range(0, len(all_answers)) :
all_answers[i].text = modify_text(all_answers[i].text, request)
context_data ["all_answers"] = all_answers
print(context_data ["all_answers"][0].text) #Check that it was modified
return render(request, 'template.html', context_data)
And in my templates I have like :
{% for answer in all_answers.all %}
{{ answer.text}}
{% endfor %}
The check shows that the modification is made, yet it's my template the answer.text is the unmodified data from the database.
I saw that the type of the context_data ["all_answers"] is a queryset, that I guess is triggered during the template rendering, making any unsaved changes useless, how to make my changes show in the template?
I have tried :
context_data ["all_answers"] = list(all_answers)
To load the queryset. The check works but then nothing shows in the template (1)
An error arises during the template render when I used this function to load the queryset into a list of dict.
I also saw a [linked question without answer].3
I don't want to save the changes as they are customized for each request (for each user basically).
** TLDR: How to see my modifications in the templates without saving the changes into the database ?**
PS: Using Python 3.6.8, Django 2.2.3.
Querysets are not data holders they are just lazy references, and get evaluated on runtime. The problem with your code is that you are modifying instances in queryset (which is also wrong way you should iterate using in operator like).
for answer in all_answers:
answer.text = modify_text(answer.text, request)
The real problem is that you are calling all method on queryset in your template again all_answers.all which returns a fresh queryset and without changes you made so you should, in template, do
{% for answer in all_answers %}
{{ answer.text}}
{% endfor %}
In Python you should never iterate through range(len(something)), but always over the thing itself. In the case of Django querysets this is even more important, since accessing an item in an unevaluated queryset via its index ([i]) actually causes a separate request to the database each time.
Do this instead:
for answer in all_answers:
answer.text = modify_text(answer.text, request)
context_data ["all_answers"] = all_answers
print(context_data ["all_answers"][0].text) #Check that it was modified
Note that the loop evaluates the queryset, so the [0] there doesn't cause another db request.

Django's template language can not safely process dict

In django's template language, if a variable called a.x, then django will try in this order to get the right value:
a['x'], dictionary lookup
a.x or a.x(), attribute or method lookup
a[x], list indexing
The result is: any variable that support var['str'] becomes a time bomb. Imagine a is a dict, with a key 'items' whoes value is 'hello world', then a.items will result in 'hello world', not ditc.items(). Here is the code:
from django.template import Engine, Context
template = """
{% for k, v in a.items %}
{{ k }} = {{ v }}
{% endfor %}
"""
e = Engine()
t = e.from_string(template)
a = {'items': 'hello world'}
print(t.render(Context({'a': a})))
While I expect items = hello world as the output, the real output is:
h =
e =
l =
...
For a dict variable, if it has a key with the same name of a dict method, then you will never be able to call that method in the template. It becomes a time bomb because you never know if in the future you will add a key called 'items' into a dict variable.
Such risk is not only for dict but any type which accept var['key'].
So my question is, why the order is designed like this, and how to make it safe.
There is a simple workaround (besides using a better name for your keys):
from django.template import Engine, Context
template = """
{% for k, v in items_func %}
{{ k }} = {{ v }}
{% endfor %}
"""
e = Engine()
t = e.from_string(template)
a = {'items': 'hello world'}
print(t.render(Context({'a': a, 'items_func': a.items})))
>> items = hello world
why the order is designed like this
It's designed like this because the template syntax is supposed to be simple and doing so encourages you to put more logic into the view rather than in the template.
How to make it safe?
Give your keys a descriptive name
"items" doesn't do anything to actually describe what should be inside of the key, so not only will it give django a headache, it will give any other developer that has to debug an issue with this dictionary a headache too.
By using the . for three different lookups, there's always the potential for a problem like this. If the order was changed to try a.x first, then it would cause problems for other users that wanted to access a['x'].
I don't think there's a general way to protect against this behaviour. Once you're burned by it once, you'll remember not to put an items key in your dictionaries in future.
As a last resort, you could switch to Jinja, which supports subscript syntax, {{ foo['bar'] }}. That means that {{ foo.bar }} checks for the attribute foo.bar first. See the Jinja docs for more information.

Django Model.objects.all() does not show in suggestions

I am using Python Tools for Visual Studio (Python 3 and django 1.6) and trying to access data from a table like this:
from django.http import HttpResponse
from django.template import Template, Context
from ticket.models import Task
def ticket_listing(request):
objects = Task.objects.all().order_by('-due_date')
template = Template('{%for elem in objects %} {{elem}}<br/> {% endfor %}')
context = Context({'objects', objects})
return HttpResponse(template.render(context))
The problem is that after Task, objects does not appear in the suggestions and it seems that it is not available. Why? If I run this code I get an empty template... There is entries in the database (3 rows) I have checked.
There is a typo in code but it doesn't make a TypeError... You have created a set instead of a dictionary to pass to the template. Django doesn't complain since it is an iterable and there is no type checking.
>>> {1, 2}
set([1, 2])
>>> {1: 2}
{1: 2}
You just have to replace the wrong line by:
context = Context({'objects': objects})

Django Pagination KeyError 'source'

I installed Django template pagination tag from: https://github.com/jmcclell/django-bootstrap-pagination and while I followed all the instructions, I'm getting 'source' errors.
Apparently I'm doing something wrong.
===========================================
EDIT 3
SETTINGS.py
TEMPLATE_CONTEXT_PROCESSORS = (
"django.contrib.auth.context_processors.auth",
"django.core.context_processors.request",
)
VIEW.py:
def pagination(request):
location = Location.objects.all()
return render_to_response('pagination.html',
location,
context_instance=RequestContext(request))
TEMPLATE
{% load bootstrap_pagination %}
<h1>Location</h1>
{% for location in location %}
<h2>{{ location.name }}</h2>
{% endfor %}
{% bootstrap_paginate location %}
ERROR:
AttributeError at /pagination/
'str' object has no attribute 'paginator'
Request Method: GET
Request URL: http://127.0.0.1:8000/pagination/
Django Version: 1.5.4
Exception Type: AttributeError
Exception Value:
'str' object has no attribute 'paginator'
Error during template rendering
In template /home/jr/Documents/python/amapp/sdr/article/templates/pagination.html, error at line 7
'str' object has no attribute 'paginator'
1 {% load bootstrap_pagination %}
2
3
4
5
6
7 {% bootstrap_paginate location %}
I'm the author of the library, but you must forgive me as I haven't used Python in quite some time and I've core dumped most of my knowledge about this library.
So, #WayBehind was correct, your first mistake was using "page_obj". That was simply an example. In your case, you want to use "location"
However, you never got to see that error because you have a more pressing error which is that the library isn't playing nice with your setup. I wrote this library with Python 2.7 with the request context preprocessor. Please double check that you have the context preprocessor enabled as per the documentation and please be sure you are using Python <3.0. I know for a fact the library does not currently working on 3.x. There is a fork of the library where some other folks have been working to fix that and I am actively keeping an eye on it to pull those changes in when ready, but as of now it just doesn't work.
If you are using Python 2.x and you have the request context preprocessor enabled, I am not sure why you would be getting that error. If you can confirm those two things are true, I'll be happy to take a closer look tomorrow.
Edit:
This may or may not be an issue, but I notice that you loop through your Location object using the same variable name for the instance:
{% for location in location %}
<h2>{{ location.name }}</h2>
{% endfor %}
{% bootstrap_paginate location %}
Is it possible that Django's template scoping is such that the object you are passing to bootstra_paginate is the last instance of "location" rather than the entire set? This is an off the cuff guess as a first stab at this because otherwise things appear to be correct.
Have you followed all the steps ?
Request is in the context_processor (settings.py)?
TEMPLATE_CONTEXT_PROCESSORS = (
....
"django.core.context_processors.request",
....
)
You are using obj_list in the template, but do you have anything inside obj_list ? Maybe you have to use "location" instead of "obj_list" ? Because I think your object list is inside location (Location objects) but you are using obj_list like in the example. In the example obj_list is just a variable example for an object list.
EDIT:
Change this:
def pagination(request):
args = {}
args.update(csrf(request))
args['location'] = Location.objects.all()
return render_to_response('pagination.html', args)
for this:
from django.template import RequestContext
def pagination(request):
location = Location.objects.all()
return render_to_response('pagination.html', 'location':location,context_instance=RequestContext(request))
Using page_obj worked for me, for anyone wondering why this doesn't work when using Django 2.0.2 and django-bootstrap4 0.0.6.
I came across this by digging through the context variables in the error message that showed up from Django's DEBUG mode.

How to parse Django templates for template tags

Situation
I'm writing a checker program that checks Django templates. For example I want to check if all Django templates that use url template tag, use it with quotes on first parameter so that it is Django 1.5 compatible. Also I want to check that they have included {% load url from future %} in their templates.
For example if my program parses the following Django template, I want it to raise an exception.
{% extends 'base.html' %}
<td>
<a href="{% url first second %}">
</a>
</td>
But this template should get parsed without exception.
{% extends 'base.html' %}
{% load url from future %}
<td>
<a href="{% url 'first' second %}">
</a>
</td>
I'm not limited to this simple example. I have other parsings to do. For example I want to check how many load template tags are present in the template.
Question
How can I elegantly solve this parsing problem?
I don't want to use regular expressions.
I this Django it self has some utilities in this regard. I think using them is a good idea, but I don't know how.
I want to run the program separately from Django. So I don't want Django to run the program itself (with render_to_response). (This is important)
Code
Please show me some code that can solve the example I mentioned. I want to detect whether {% load url from future %} is in the code. Also I want to check every url template tag and check if the first argument is quoted.
Bonus:
I want to be able to see the rendered HTML that Django generates from this template, and do my HTML parsing on it. (for example with PyQuery)
You say...
I want to check if all Django templates that use url
template tag, use it with quotes on first parameter so that it is
Django 1.5 compatible.
...and...
I don't want to use regular expressions.
...because...
the result of that might become a huge spaghetti code
...but, frankly, writing a parser from scratch is likely to be even messier than using a regular expression. I don't see what's so messy about a regex as simple as something like...
"{% *url +[^']"
...and I doubt there's a non-regex-based solution that's as terse as that.
With regards to...
Also I want to check that they have included
{% load url from future %} in their templates.
If your intention is to ensure Django 1.5 compatibility, this is pointless. According to the Django 1.5 release notes, the new-style url tag syntax is enabled by default, so the line {% load url from future %} won't have any effect.
And in versions prior to 1.5, it's much simpler just to put...
import django.template
django.template.add_to_builtins('django.templatetags.future')
...at the bottom of your settings.py and be done with it. :-)
You can also use the compile_string method.
>>> from django.template.base import *
>>> settings.configure()
>>> compile_string("<a href='ab'></a>{% cycle 'row1' 'row2' as rowcolors %}", None)
>>> [<Text Node: '<a href='ab'></a>'>, <django.template.defaulttags.CycleNode object at 0x10511b210>]
The compile string method is utilized by the Template class and is the method used to produce the node list.
Tested in Django 1.8 Alpha.
https://github.com/django/django/blob/1f8bb95cc2286a882e0f7a4692f77b285d811d11/django/template/base.py
Next code still uses django, but it can check if syntax is correct:
>>> from django.template import Template
>>> from django.template.defaulttags import URLNode
>>> t = Template("{% load url from future %}\n{% url projects_list company.slug %}")
>>> for node in t.nodelist:
... if isinstance(node, URLNode):
... for arg in node.args: print(arg)
...
company.slug
>>> t2 = Template('{% load url from future %}\n{% url "projects_list" company.slug }')
>>> for node in t2.nodelist:
... print(node)
...
<django.template.defaulttags.LoadNode object at 0x32145d0>
<Text Node: '
{% url "projects_list" c'>
>>>
As you see last node is not URLNode

Categories