I have the following model.py.
from django.db import models
class Address(models.Model):
addr = models.CharField(max_length=150)
def __unicode__(self):
return u'%s' % (self.addr)
class Anniversary(models.Model):
date = models.DateField()
def __unicode__(self):
return u'%s' % (self.date)
class Person(models.Model):
name = models.CharField(max_length=50)
birthday = models.DateField()
anniversary = models.ForeignKey(Anniversary)
address = models.ForeignKey(Address)
def __unicode__(self):
return u'%s %s %s %s' % (self.name, self.birthday, self.anniversary, self.address)
I want to print the contents of all entries into my template. But sorted by date of birth against the current date. i.e most recent first and then name. What is the best way to do this. Should I sort it first and then append it to a list or dict ?
Any pointers would be great.
Thanks,
You can add default ordering in a model's meta class, e.g.
class Person(models.Model):
# fields
class Meta:
ordering = ('anniversary__date',)
then in your template it's as easy as:
<ul>
{% for person in persons %}
<li>{{ person.anniversary.date|date:"M d, Y" }} - {{ person.name }}</li>
{% endfor %}
</ul>
If you need custom ordering within a view (to filter out persons based on the request object):
def myview(request):
context = dict(
persons = Person.objects.exclude(person.id=request.user.id)\
.order_by('anniversary__date')
)
return render_to_response('app/template.html', context)
N.b. Adding a minus before your order_by parameter order_by('-attr') reverses ordering.
As #DanielEriksson mentioned, if your case isn't hypothetic, it seems you should simplify things.
I think you are looking for this :
persons = Person.objects.all().order_by(birthday ,anniversary)
In you view you can get the current date using :
import datetime
now = datetime.datetime.now()
and then :
persons = Person.objects.all().order_by(now, anniversary)
Hope it helps!
I personally like to avoid sorting in the Model, since it's more a presentation/view type thing. I tend to use sorted in render_to_response for single value sorting, or order_by for multi-value sorting. Using sorted in the return statement lets me kind of split the difference between the view/template, since Django views aren't entirely controllers. Anyway, this is just my preference. You can also use dictsort in the template to do single value sorting.
You have indicated you want to sort using multiple values, first by most recent birth date and second by the name. To do this, I would use order_by in a view, then comment my return statement:
from django.shortcuts import render_to_response
def view_persons(request):
persons = Person.objects.all().order_by('-birthday', 'name')
return render_to_response(
'view_persons.html',
{
'persons': persons, # ordered by most recent birthday, then name
},
context_instance=RequestContext(request)
)
In the template, you have little more to do than:
{% if persons %}
<table>
<thead>
<tr>
<th>Birthday</th>
<th>Name</th>
</tr>
</thead>
<tbody>
{% for person in persons %}
<tr>
<td>{{ person.birthday }}</td>
<td>{{ person.name }}</td>
</tr>
{% endfor %}
</tbody>
</table>
{% else %}
<p>There are no people!</p>
{% endif %}
I guess the complexity is that you need to compare the sorted date against the current date, not simply sorted by date.
If you are using Postgres and are happy to stick to raw SQL, you could potentially use their datetime functions, such as age
http://www.postgresql.org/docs/8.1/static/functions-datetime.html
Otherwise, you would need to select where the date of birth/anniversary is greater than now. It might be simpler if you refactored your anniversary model to store the day, month and year a separate fields, so that you can ignore the year when filtering.
I recently posted some code in a question, that might give you some hints - I need to return events closest to now. See:
Is it bad practice to return a tuple from a Django Manager method rather than a queryset?
Related
I am trying to solve performance issues at a Django-based web site, with very little knowledge of Django and Python syntax. I seem to have correctly identified the problem. I also seem to know what to do next, but I can't get a grasp on the Python/Django syntax to make everything work.
class Company(models.Model):
name = models.CharField(max_length=100)
bic = models.CharField(max_length=100, blank=True)
def get_order_count(self):
return self.orders.count()
def get_order_sum(self):
total_sum = 0
for contact in self.contacts.all():
for order in contact.orders.all():
total_sum += order.total
return total_sum
class Contact(models.Model):
company = models.ForeignKey(
Company, related_name="contacts", on_delete=models.DO_NOTHING)
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100, blank=True)
def get_order_count(self):
return self.orders.count()
class Order(models.Model):
order_number = models.CharField(max_length=100)
company = models.ForeignKey(Company, related_name="orders", on_delete=models.DO_NOTHING)
contact = models.ForeignKey(Contact, related_name="orders", on_delete=models.DO_NOTHING)
total = models.DecimalField(max_digits=18, decimal_places=9)
order_date = models.DateTimeField(null=True, blank=True)
def __str__(self):
return "%s" % self.order_number
My hunch is that the performance problems are caused by the nested loop in get_order_sum. And my solution is quite clear: nested "fors" should be replaced by a simple command, that uses aggregation and utilizes the database's own efficient internal SQL functionality. So in my mind, the solution should look something like this:
return self.contacts.all().orders.all().aggregate(Sum('total'))
The problem is that I can't figure out how to properly write what I want Django/Python to do. Please help me!
Or am I mistaken and is the problem (or part of it) in my View-code?
<table>
<tr>
<th>Name</th>
<th>Order Count</th>
<th>Order Sum</th>
<th>Select</th>
</tr>
{% for company in company_list|slice:":100" %}
<tr>
<td>{{ company.name }}</td>
<td>{{ company.orders.count }}</td>
<td>{{ company.get_order_sum|floatformat:2 }}</td>
<td><input type="checkbox" name="select{{company.pk}}" id=""></td>
</tr>
{% for contact in company.contacts.all %}
<tr>
<td>- {{ contact.first_name }} {{ contact.last_name }}</td>
<td>Orders: {{ contact.orders.count }}</td>
<td> </td>
<td> </td>
</tr>
{% endfor %}
{% endfor %}
</table>
I would also appreciate any other hints and opinions on how this code could be further improved (especially from the performance POV).
The key is not to have excess database queries (this is not the same as having as few as possible) and, as you mentioned, to let the database do work it's good at.
In practical terms, a) all data you need should be in company_list by the time it gets to your template, so you don't have database queries sent from within the loops in your template, and b) company_list should be filled with data in an efficient way.
from django.db.models import Prefetch, Sum, Count
contacts_with_orders = Prefetch(
'contacts',
queryset=Contact.objects.annotate(order_count=Count('orders'))
)
company_list = (Company.objects
.prefetch_related(contacts_with_orders)
.annotate(order_sum=Sum('orders__total'),
order_count=Count('orders'))
)
Now you can do your loops and access all required data without any further queries:
for company in company_list:
company.name
company.order_sum
company.order_count
for contact in company.contacts.all():
contact.first_name
contact.order_count
While this should be orders of magnitude faster than previously, it is still quite onerous. There is also some more room for optimization: If needed, you can shave off a bit more by querying only the columns you need instead of complete rows and by returning dictionaries instead of objects. See only(), values() and values_list() and the to_attr parameter in Prefetch.
As you guessed, your performance problem is most likely caused by get_order_sum method. You are running a query to get all contats of a company, then for each contact, you are running a query to get that contact's ordesr. You can find this sum with a single query, like this in Django:
from django.db.models import Sum
def get_order_sum(self):
return self.contacts.aggregate(order_sum=Sum('orders__total')).get('order_sum')
Note that aggregate function returns a dictionary in the following format:
{
'order_sum': 123
}
I have a table inside a form, generated by a formset.
In this case, my problem is to save all the items after one of them is modified, adding a new "virtual" column as the sum of other two (that is only generated when displaying the table, not saved).
I tried different ways, but no one is working.
Issues:
This save is not working at all. It worked when it was only one form, but not for the formset
I tried to generate the column amount as a Sum of box_one and box_two without success. I tried generating the form this way too, but this is not working:
formset = modelformset_factory(
Item, form=ItemForm)(queryset=Item.objects.order_by(
'code__name').annotate(amount=Sum('box_one') + Sum('box_two')))
This issue is related to this previous one, but this new one is simpler:
Pre-populate HTML form table from database using Django
Previous related issues at StackOverflow are very old and not working for me.
I'm using Django 2.0.2
Any help would be appreciated. Thanks in advance.
Current code:
models.py
class Code(models.Model):
name = models.CharField(max_length=6)
description = models.CharField(max_length=100)
def __str__(self):
return self.name
class Item(models.Model):
code = models.ForeignKey(Code, on_delete=models.DO_NOTHING)
box_one = models.IntegerField(default=0)
box_two = models.IntegerField(default=0)
class Meta:
ordering = ["code"]
views.py
class ItemForm(ModelForm):
description = CharField()
class Meta:
model = Item
fields = ['code', 'box_one', 'box_two']
def save(self, commit=True):
item = super(ItemForm, self).save(commit=commit)
item.box_one = self.cleaned_data['box_one']
item.box_two = self.cleaned_data['box_two']
item.code.save()
def get_initial_for_field(self, field, field_name):
if field_name == 'description' and hasattr(self.instance, 'code'):
return self.instance.code.description
else:
return super(ItemForm, self).get_initial_for_field(
field, field_name)
class ItemListView(ListView):
model = Item
def get_context_data(self, **kwargs):
data = super(ItemListView, self).get_context_data()
formset = modelformset_factory(Item, form=ItemForm)()
data['formset'] = formset
return data
urls.py
app_name = 'inventory'
urlpatterns = [
path('', views.ItemListView.as_view(), name='index'),
item_list.html
...
<div>
<form action="" method="post"></form>
<table>
{% csrf_token %}
{{ formset.management_form }}
{% for form in formset %}
<thead>
<tr>
{% if forloop.first %}
<th>{{ form.code.label_tag }} </th>
<th>{{ form.description.label_tag }} </th>
<th> <label>Amount:</label> </th>
<th>{{ form.box_one.label_tag }} </th>
<th>{{ form.box_two.label_tag }} </th>
{% endif %}
</tr>
</thead>
<tbody>
<tr>
<td>{{ form.code }}</td>
<td>{{ form.description }}</td>
<td>{{ form.amount }}</td>
<td>{{ form.box_one }}</td>
<td>{{ form.box_two }}</td>
</tr>
</tbody>
{% endfor %}
<input type="submit" value="Update" />
</table>
</form>
</div>
...
Annotating query with virtual column
Sum is an aggregate expression and is not how you want to be annotating this query in this case. Instead, you should use an F exrepssion to add the value of two numeric fields
qs.annotate(virtual_col=F('field_one') + F('field_two'))
So your corrected queryset would be
Item.objects.order_by('code__name').annotate(amount=F('box_one') + F('box_two'))
The answer provided by cezar works great if intend to use the property only for 'row-level' operations. However, if you intend to make a query based on amount, you need to annotate the query.
Saving the formset
You have not provided a post method in your view class. You'll need to provide one yourself since you're not inheriting from a generic view that provides one for you. See the docs on Handling forms with class-based views. You should also consider inheriting from a generic view that handles forms. For example ListView does not implement a post method, but FormView does.
Note that your template is also not rendering form errors. Since you're rendering the formset manually, you should consider adding the field errors (e.g. {{ form.field.errors}}) so problems with validation will be presented in the HTML. See the docs on rendering fields manually.
Additionally, you can log/print the errors in your post method. For example:
def post(self, request, *args, **kwargs):
formset = MyFormSet(request.POST)
if formset.is_valid():
formset.save()
return SomeResponse
else:
print(formset.errors)
return super().post(request, *args, **kwargs)
Then if the form does not validate you should see the errors in your console/logs.
You're already on the right path. So you say you need a virtual column. You could define a virtual property in your model class, which won't be stored in the database table, nevertheless it will be accessible as any other property of the model class.
This is the code you should add to your model class Item:
class Item(models.Model):
# existing code
#property
def amount(self):
return self.box_one + self.box_one
Now you could do something like:
item = Item.objects.get(pk=1)
print(item.box_one) # return for example 1
print(item.box_two) # return for example 2
print(item.amount) # it will return 3 (1 + 2 = 3)
EDIT:
Through the ModelForm we have access to the model instance and thus to all of its properties. When rendering a model form in a template we can access the properties like this:
{{ form.instance.amount }}
The idea behind the virtual property amount is to place the business logic in the model and follow the approach fat models - thin controllers. The amount as sum of box_one and box_two can be thus reused in different places without code duplication.
As I loop through a Prize.objects.all() set, I am trying to get the quantity value from PrizeItem, but it seems to reference only the GameItem model, not giving access to the intermediate PrizeItem attributes.
How do I get access to the Through model's attributes?
MODELS:
class GameItem(models.Model):
'...'
class Prize(models.Model):
'...'
item = models.ManyToManyField(GameItem, through='PrizeItem')
class PrizeItem(models.Model):
#relations
game_item = models.ForeignKey(GameItem, on_delete=models.CASCADE)
prize = models.ForeignKey(Prize, on_delete=models.CASCADE)
#Item details
quantity = models.IntegerField(null=True, blank=True, default=1)
VIEWS:
def gameprizes(request):
prizes=Prize.objects.all()
context={'prizes':prizes}
return render(request, "the-app/game-prizes.html", context)
TEMPLATE:
{% if prize.item.all %}
<table class="table table-condensed">
<tbody>
{% for prize_item in prize.item.all %}
<tr>
<td>{{ prize_item.type }}</td><td>{{ prize_item.name }} {{ prize_item.quantity }}</td>
</tr>
{% endfor %}
</tbody>
</table>
{% endif %}
It gets a little ugly when it comes to intermediate models in querysets. One way you can address this would be:
class Prize(models.Model):
'...'
item = models.ManyToManyField(GameItem, through='PrizeItem')
def quantity(self):
return self.item.quantity
Note that this does lead to additional queries, and you can use prefetch_related in your query to optimize
Another option would be to use the template
For game item, you would be doing something like:
class GameItem(models.Model):
'...'
def quantity(self):
return self.prizeitem_set.first()
What the hell with the models?
~> Use the #karthikr answer
Just prepare in views.py (overwrite query_set)
Use (_set ~ join) in html
Have a nice day, don't make it hard, give the hardest part for the strongest (server - database)
I'm trying to create a structure to show the opening times from food companies, ordered by "open" status. But I don't know how can I get the informations on my template. For example:
#models.py
class Company(Model):
#... fields ...
def __unicode__(self):
return u'%s'%self.title
Here I'll store all times and days.
class OpeningHours(Model):
class Meta:
verbose_name = _(u"Horário de Abertura")
verbose_name_plural = _(u"Horários de Abertura")
unique_together = ('company', 'weekday')
company = ForeignKey(Company, related_name="opening_times", verbose_name=_(u"Empresa"))
weekday = IntegerField(choices=WEEKDAYS, verbose_name=_(u"Dia da Semana"))
fromHour = TimeField(verbose_name=_(u"Abre ás:"), null=True, blank=True)
toHour = TimeField(verbose_name=_(u"Fecha ás:"), null=True, blank=True)
def __unicode__(self):
return "%s %s (%s - %s)" % (self.company, self.weekday, self.fromHour, self.toHour)
And then I'm catching all the companies just like that on my views:
#views.py - This is how I'm processing the views
companies = sorted(Company.objects.filter(category__food=True).order_by('?')[0:4], key=lambda c: c.show_open(), reverse=True)
So, now the problem are in template, I need someway to catch this informations:
template.html - This is what I need to do on template
{% for company in companies %}
{% if company.open %}
OPEN
{% else %}
CLOSED
{% endif %}
<!-- I need to know when it's today, when it's tomorrow or when it's another day -->
{% ifequal company.open today %}
Next day will open is today at {{ company.next_time_open }}
{% ifequal company.open another_day %}
Next day will open is Sunday at {{ company.next_time_open }}
{% else %}
Wait, it's open right now from 22h till 00h
{% endif %}
{% endfor %}
First off, doing .order_by('?') is randomizing your queryset, which you are then sorting in Python. Ordering randomly causes the query to take more process time on the database side, and then sorting it in Python is its own additional process time. I would consider ordering your queryset using Django's methods, instead, by specifying the field(s) you want to order by in .order_by().
Secondly, one way to reduce database hits is to use .select_related() on the queryset. This will include related models in the queryset under a single SQL statement, so that later calls to the related models in the template don't cause new database hits.
Third, there is a lot of code in your different sections that reference fields and methods that I assume you've defined, but without seeing them directly, I can't make a judgment on what exactly you're doing. It is not possible at this time to give you a more direct answer.
My problem is simple: I have Users who own Assets or Assets which belong to Users If you prefer and I cannot make it to retrieve the number (count) of Assets each User has. I know this might be sound silly to most of you but I am new to python/django (coming from PHP/MySQL) and I do not know how things work here. I do not want to be engaged with raw SQL - this would be my last choice If nothing else works.
(*) I have removed all non-related raws from the code
Users
class Users(models.Model):
firstname = models.CharField(max_length=100)
lastname = models.CharField(max_length=100)
Assets
class Assets(models.Model):
serial = models.CharField(unique=True, max_length=100)
user = models.ForeignKey('Users', blank=True, null=True)
# this is what I am playing with to retrieve the number of assets each user owns
#classmethod
def user_assets(self):
return Assets.objects.filter(user=user).count()
views.py
class UserList(ListView):
model = Users
def get_context_data(self, **kwargs):
context = super(UserList, self).get_context_data(**kwargs)
context['user_assets'] = self.model.user_assets()
return context
template
{% for user in object_list %}
<tr>
<td>{{ user.id }}</td>
<td>
{{ user_assets }}
</td>
</tr>
{% endfor %}
How can I get that number? I have read about aggregations, annotations and filters but can't really get it.
EDIT:
I am looking for a simple solution by using class based views and easily expandable (I may want to add other models later)
In your UserList instead using model, use this queryset:
from django.db.models import Count
class UserList(ListView):
queryset = Users.objects.annotate(num_assets=Count('assets'))
and define your user field like so:
user = models.ForeignKey('Users', blank=True, null=True, related_name='assets')
then from template:
{{ user.num_assets }}
Also please remember, it's a good practice to use singular model names, to avoid confusion with reverse relation names..
You are doing weird things. Use the related managers that django give you instead. I'll write the view as a function based view:
views.py
def users_list(request):
object_list = Users.objects.all()
render(request, 'mytemplate.html', { 'object_list': object_list })
You can get the counts directly in the template via the RelatedManager:
mytemplate.html
{% for user in object_list %}
<tr>
<td>{{ user.id }}</td>
<td>
{{ user.assets_set.count }}
</td>
</tr>
{% endfor %}
You could also annotate with a count. But learn to float before you swim :)
BTW, you should call your models "User" and "Asset", not Users and Assets.
You need to use select_related(), count() and pass user instance as argument to class method like so:
#classmethod
def user_assets(cls,user):
return Assets.objects.select_related('Users').filter(user=user).count()
and then use it like so:
user = Users.objects.all()[0] # some user object (this assumes you have at least one user)
Assets.user_assets(user)
this should work fine, you can try it in the shell.
In your context this will be used like this:
user = self.model.all()[0] # or get or filter just get some particular user
context['user_assets'] = Assets.user_assets(user)
EDIT: added links, and Users.object.all() instead of Users.object.get(), also added example suited to your specific use case.