Suppose I have the following models
class Award(models.Model):
user = models.ForeignKey(User)
class AwardReceived(models.Model):
award = models.ForeignKey(award)
date = models.DateField()
units = models.IntegerField()
class AwardUsed(models.Model):
award = models.ForeignKey(award)
date = models.DateField()
units = models.IntegerField()
Now, suppose I want to get the number of awards for all users and the number of awards used for all users (ie, a queryset containing both). I prefer to do it one query for each calculation - when I combined it in my code I had some unexpected results. Also for some of my queries it won't be possible to do it one query, since the query will get too complex - I'm calculating 8 fields. This is how I solved it so far:
def get_summary(query_date)
summary = (Award.objects.filter(awardreceived__date__lte=query_date))
.annotate(awarded=Sum('awardissuedactivity__units_awarded')))
awards_used = (Award.objects.filter(awardused__date__lte=query_date)
.annotate(used=Sum('awardused__date__lte__units')))
award_used_dict = {}
for award in awards_used:
award_used_dict[award] = award.used
for award in summary:
award.used = award_used_dict.get(award, 0)
return summary
I'm sure there must be a way to solve this without the dictionary approach? For instance, something like this: awards_used.get(award=award), but this causes a db lookup every loop.
Or some other fancy way to join the querysets?
Note this is a simplified example and I know for this example the DB structure can be improved, I'm just trying to illustrate my question.
SOLUTION 1
Just try to concatenate your queryset using |
final_q = q1 | q2
In your example
final_q = summary | awards_used
UPDATED:
| does not works using calculated attributes, so, we can select our queryset first and then mapping our extra attributes
summary = Award.objects.filter(awardreceived__date__lte=query_date)
awards_used = Award.objects.filter(awardused__date__lte=query_date)
final_q = summary | awards_used
final_q = final_q.annotate(used=Sum('awardused__date__lte__units')).annotate(awarded=Sum('awardissuedactivity__units_awarded'))
SOLUTION 2
Using chain built-in function
from itertools import chain
final_list = list(chain(summary, awards_used))
There is an issue with this approach, you won't get a queryset, you will get a list containing instances.
Related
I have two models: ActorModel and FilmModel joined as follows:
FilmModel(models.Model):
actors = models.ManyToManyField(Actor, blank=True, related_name='film_actors')
ActorModel(models.Model):
name = models.CharField(max_length=40)
def __str__(self):
return self.imdb_id
I want to filter my ActorModel for any instance which has more than 5 joins with the FilmModel. I can do this as follows:
actors = ActorModel.objects.all()
more_than_five_films = []
for actor in actors:
actor_film_list = FilmModel.objects.filter(actors__imdb_id=str(name))
if len(actor_film_list)>5:
more_than_five_films.append(actor)
However, using the above code uses lots of processing power. Is there a more efficient way of finding the actors with more than 5 joins? Could I do this at the filtering stage for example?
You could use query like this:
more_than_five_films = ActorModel.objects.annotate(count=Count('film_actors')).filter(count__gt=5)
You access FilmModel objects of ActorModel through related_name field, annotate new field named count by counting number of FilmModel objects related to each ActorModel object and then filter out only objects that have count value greater than 5.
Advice for code you provided is to never use len() on a queryset because it evaluates the whole query which is expensive and not needed since you need only a count value. You should use count() function which returns the number as same as len() does. It looks like this:
FilmModel.objects.filter(actors__imdb_id=str(name)).count()
i have a model with ratings, the results of a filter query must be in a special order for chartit (comparing ratings for trainee's) but I can't find the right way to do it. (ok I'm new to Django and python ;)
class Bewertung(models.Model):
auffassung = models.PositiveSmallIntegerField()
interesse = models.PositiveSmallIntegerField()
arbeitsabw = models.PositiveSmallIntegerField()
aufmerksamkeit = models.PositiveSmallIntegerField()
arbeitsgenauigkeit = models.PositiveSmallIntegerField()
verhalten = models.PositiveSmallIntegerField()
ausb_sach = models.PositiveSmallIntegerField(null=True, blank=True)
ausb_fuehr = models.PositiveSmallIntegerField(null=True, blank=True)
the query:
qs = Bewertung.objects.filter(pk__in=pk_list)
I want to compare the integer values in a multi bar chart e.g.
auffassung_from_pk(1,2,3) interesse_from_pk(1,2,3) .. n
but every try ends in a list with a lot of unordered values
(Auffassung_from_pk(1), interesse_from_pk(1), Auffassung_from_pk(2) ..)
I can't find a way to solve it nice and efficient in an python way.
so I need a little help, can you help?
#Sachin Kukreja correct, separate it and order it. so every field (e.g. auffassung must be one list with every result from the queryset.)
if I have 3 resulting query sets (pk_list=(1,2,3)) I need something like ((1,2,1),(2,3,3)...) ((auffassung),(interesse))
#Rajez there are no multiple filters
i set all my approach's to zero to start new. I have only this (in the Django shell)
for q in qs:
print(q.auffassung)
print(q.interesse)
i am really struggling at this at the moment
try it:
import itertools
qs = Bewertung.objects.filter(pk__in=pk_list)
values = qs.values_list('auffassung', 'interesse')
result = list(itertools.chain(*values))
I'm trying to sort a list of posts where votes have priority over the date.
I have my own app called UserPost and I'm using the django-voting app to do votes.
class UserPost(models.Model):
user = models.ForeignKey(User)
datetime = models.DateTimeField(auto_now_add=True)
text = models.CharField(max_length=255, blank=True)
is_deleted = models.BooleanField(default=False)
vote = models.ForeignKey(Vote)
Right now, I'm sorting without votes taking precedence yet:
posts_list = sorted(posts_list, key=attrgetter('datetime'))
What's the best way to go about this?
Thanks!
Tuples are sorted lexicographically, therefore if you return a tuple for sorted's key= argument you can sort by votes then by dates:
posts_list = sorted(posts_list, key=lambda post: (Vote.objects.get_score(post)['score'], post.datetime))
Alternatively, you might want to also look at the ordering option in a django Model's Meta class or the order_by method on django Queryset. They will do the sorting on the database in one query, so can be much faster. Alternatively, you can try the posts_list.get_score_in_bulk() to reduce the number of queries to two (one for posts_list, and one for get_score_in_bulk), like so:
scores = Vote.objects.get_score_in_bulk(posts_list)
posts_list = sorted(posts_list, key=lambda post: (scores[post.id]['score'], post.datetime))
You can sort it like that:
def posts_sorting(post1, post2):
# put here the way you compare posts
pass # return 1 if post2 before post1, -1 otherwise, 0 if they are "equal"
posts_list = sorted(posts_list, cmp=posts_sorting)
where posts_sorting is the function that compares your posts.
I am working on a membership application. I would like to make a membership reminder. (member during a period of time which is not member for another period of time).
Currently, I am using set for making this calculation. See the code below.
class Member(models.Model):
...
class Membership(models.Model):
member = models.ForeignKey(Member, verbose_name=_("Member"))
start_date = models.DateField(_("Start date"))
end_date = models.DateField(_("End date"))
x = Member.objects.filter(Q(membership__start_date__lte=dt1) & Q(membership__end_date__gte=dt1))
y = Member.objects.filter(Q(membership__start_date__lte=dt2) & Q(membership__end_date__gte=dt2))
result = set(x) - set(y)
I would like to know of I can do it only by using the django ORM (filter, exclude, annotate, distinct ...)?
Thanks in advance for your help
UPDATE
In fact, my model is a bit more complex. I also have newspaper foreign key.
class Member(models.Model):
...
class Newspaper(models.Model):
...
class Membership(models.Model):
member = models.ForeignKey(Member, verbose_name=_("Member"))
start_date = models.DateField(_("Start date"))
end_date = models.DateField(_("End date"))
newspaper = models.ForeignKey(Newspaper)
I want to have the reminder for a given newspaper. In this case, the working query is
sin = models.Membership.objects.filter(start_date__lte=dt1,
end_date__gte=dt1,
newspaper__id=2)
sout = models.Membership.objects.filter(start_date__lte=dt2,
end_date__gte=dt2,
newspaper__id=2)
result = models.Member.objects.filter(membership__in=sin).exclude(membership__in=sout)
I think that this a more verbose version of the answer given Ghislain Leveque which is also working well for me.
Thanks to S.Lott and KillianDS for very valuable answers and sorry for not so clear question :)
Isn't it simply negating the second expression and putting it in the same filter? So you have something like !(a&b), which equals to (!a)|(!b), in this case:
result = Member.objects.filter(membership__start_date__lte=dt1, membership__end_date__gte=dt1, ~Q(membership__start_date__lte=dt2) | ~Q(membership__end_date__gte=dt2))
note by the way that for simple anding and basic lookups you need no Q objects, like I showed with the first two lookup parameters. Anding happens just by passing multiple arguments, Q objects are needed for negating and OR'ing lookups.
A relational database table is a set -- by definition. Set - is where not exists in SQL, which is exclude in Django's ORM.
It seems (without testing) that you're doing this.
result = Member.objects.filter(
Q(membership__start_date__lte=dt1) & Q(membership__end_date__gte=dt1)
).exclude(
Q(membership__start_date__lte=dt2) & Q(membership__end_date__gte=dt2)
)
You should try :
result = Member.objects.\
filter(
membership__start_date__lte = dt1,
membership__end_date__gte=dt1).\
exclude(
pk__in = \
Member.objects.filter(
membership__start_date__lte = dt2,
membership__end_date__gte = dt2).\
values_list('pk')
I've got a model like this:
class Thing(models.Model):
property1 = models.IntegerField()
property2 = models.IntegerField()
property3 = models.IntegerField()
class Subthing(models.Model):
subproperty = models.IntegerField()
thing = modelsForeignkey(Thing)
main = models.BooleanField()
I've got a function that is passed a list of filters where each filter is of the form {'type':something, 'value':x}. This function needs to return a set of results ANDing all the filters together:
final_q = Q()
for filter in filters:
q = None
if filter['type'] =='thing-property1':
q = Q(property1=filter['value'])
elif filter['type'] =='thing-property2':
q = Q(property2=filter['value'])
elif filter['type'] =='thing-property2':
q = Q(property3=filter['value'])
if q:
final_q = final_q & q
return Thing.objects.filter(final_q).distinct()
Each Subthing has a Boolean property 'main'. Every Thing has 1 and only 1 Subthing where main==True.
I now need to add filter that returns all the Things which have a Subthing where main==True and subproperty==filter['value']
Can I do this as part of the Q object I'm constructing? If not how else? The queryset I get before my new filter can be quite large so I would like a method that doesn't involve looping over the results.
It's a bit easier to understand if you explicitly give your Subthings a "related_name" in their relationship to the Thing
class Subthing(models.Model):
...
thing = models.ForeignKey(Thing, related_name='subthings')
...
Now, you use Django join syntax to build your Q object:
Q(subthings__main=True) & Q(subthings__subproperty=filter['value'])
The reverse relationship has the default name 'subthing_set', but I find that it's easier to follow if you give it a better name like 'subthings'.
Using (instead of final_q=Q() in the beginning)
final_q=Q(subthing_set__main=True)
sub_vals = map(lambda v: v['value'], filters)
if sub_vals:
final_q = final_q & Q(subthing_set__subproperty__in=sub_vals)
should get you what you want, you can also adjust your loop to build the sub_vals list and apply it after the loop.
subthing_set is and automatically added related field added to the Thing to access related Subthings.
you can assign another related name, e.g.
thing=models.ForeignKey(Thing,related_name='subthings')