I've got a model like this:
class Thing(models.Model):
property1 = models.IntegerField()
property2 = models.IntegerField()
property3 = models.IntegerField()
class Subthing(models.Model):
subproperty = models.IntegerField()
thing = modelsForeignkey(Thing)
main = models.BooleanField()
I've got a function that is passed a list of filters where each filter is of the form {'type':something, 'value':x}. This function needs to return a set of results ANDing all the filters together:
final_q = Q()
for filter in filters:
q = None
if filter['type'] =='thing-property1':
q = Q(property1=filter['value'])
elif filter['type'] =='thing-property2':
q = Q(property2=filter['value'])
elif filter['type'] =='thing-property2':
q = Q(property3=filter['value'])
if q:
final_q = final_q & q
return Thing.objects.filter(final_q).distinct()
Each Subthing has a Boolean property 'main'. Every Thing has 1 and only 1 Subthing where main==True.
I now need to add filter that returns all the Things which have a Subthing where main==True and subproperty==filter['value']
Can I do this as part of the Q object I'm constructing? If not how else? The queryset I get before my new filter can be quite large so I would like a method that doesn't involve looping over the results.
It's a bit easier to understand if you explicitly give your Subthings a "related_name" in their relationship to the Thing
class Subthing(models.Model):
...
thing = models.ForeignKey(Thing, related_name='subthings')
...
Now, you use Django join syntax to build your Q object:
Q(subthings__main=True) & Q(subthings__subproperty=filter['value'])
The reverse relationship has the default name 'subthing_set', but I find that it's easier to follow if you give it a better name like 'subthings'.
Using (instead of final_q=Q() in the beginning)
final_q=Q(subthing_set__main=True)
sub_vals = map(lambda v: v['value'], filters)
if sub_vals:
final_q = final_q & Q(subthing_set__subproperty__in=sub_vals)
should get you what you want, you can also adjust your loop to build the sub_vals list and apply it after the loop.
subthing_set is and automatically added related field added to the Thing to access related Subthings.
you can assign another related name, e.g.
thing=models.ForeignKey(Thing,related_name='subthings')
Related
I have a model called Service, which has a field url of type str. I have a function f that returns the hostname of an url:
def f(url):
return urllib.parse.urlparse(url).hostname
I want to get all the objects whose f(url) equals a value target.
One way to achieve this would be by doing the following:
[x for x in Service.objects.all() if(f(x.url) == target)]
But in that case, I'll get a list, not a QuerySet.
Is there a way to filter the objects and get a QuerySet satisfying the above criteria?
Can you try sthg like this instead of looping through, we are changing target:
from django.db.models import Q
target_not_safe = 'http://'+target
target_safe = 'https://'+target
queryset = Service.objects.filter(Q(url=target_not_safe) | Q(url=target_safe))
Q objects
EDIT
How about using _istartwith:
queryset = Service.objects.filter(Q(url__istartswith=target_not_safe) | Q(url__istartswith=target_safe))
Edit 2
Another trick could be to check inside the list using __in. So:
query_list = [x.id for x in Service.objects.all() if(f(x.url) == target)]
queryset = Service.objects.filter(id__in=query_list)
I need to filter for objects where the number of elements in a ManyToMany relationship matches a condition. Here's some simplified models:
Place(models.Model):
name = models.CharField(max_length=100)
Person(models.Model):
type = models.CharField(max_length=1)
place = models.ManyToManyField(Place, related_name="people")
I tried to do this:
c = Count(Q(people__type='V'))
p = Places.objects.annotate(v_people=c)
But this just makes the .v_people attribute count the number of People.
Since python-2.0, you can use the filter=... parameter of the Count(..) function [Django-doc] for this:
Place.objects.annotate(
v_people=Count('people', filter=Q(people__type='V'))
)
So this will assign to v_people the number of people with type='V' for that specific Place object.
An alternative is to .filter(..) the relation first:
Place.objects.filter(
Q(people__type='V') | Q(people__isnull=True)
).annotate(
v_people=Count('people')
)
Here we thus filter the relation such that we allow people that either have type='V', or with no people at all (since it is possible that the Place has no people. We then count the related model.
This generates a query like:
SELECT `place`.*, COUNT(`person_place`.`person_id`) AS `v_people`
FROM `place`
LEFT OUTER JOIN `person_place` ON `place`.`id` = `person_place`.`place_id`
LEFT OUTER JOIN `person` ON `person_place`.`person_id` = `person`.`id`
WHERE `person`.`type` = V OR `person_place`.`person_id` IS NULL
My objects are set up similar to this:
class Event(Model):
pass
class Inventory(Model):
event = OneToOneField(Event)
def has_altered_item_counts(self):
return any(obj.field_one is not None or obj.field_two is not None for obj in self.itemcounts_set.all())
class ItemCounts(Model):
inventory = ForeignKey(Inventory)
field_one = IntegerField(blank=True, null=True)
field_two = IntegerField(blank=True, null=True)
Basically, I'd like to filter uniquely on Event where inventory.has_altered_item_counts would return False
I have
Q(inventory__itemcounts__field_one__isnull=True) & \
Q(inventory__itemcounts__field_two__isnull=True)
but that returns the event every time it meets those conditions. Given that result, I want to exclude that event if the number of times it appears is less than the total number of item counts. Does any of this make sense? Is this possible with filter? I really sort of need it to be, this is part of a programmatically built batch update
I ended up doing it in a "get the PKs I'm trying to exclude first, then exclude those PKs"
Example in case anyone else is interested:
counts = Q(inventory__itemcounts__field_one__isnull=False) | \
Q(inventory__itemcounts__field_two__isnull=False)
bad_pks = set(Event.objects.filter(counts).values_list("pk", flat=True))
Event.objects.exclude(pk__in=bad_pks).update(stuff)
Works like a charm.
Suppose I have the following models
class Award(models.Model):
user = models.ForeignKey(User)
class AwardReceived(models.Model):
award = models.ForeignKey(award)
date = models.DateField()
units = models.IntegerField()
class AwardUsed(models.Model):
award = models.ForeignKey(award)
date = models.DateField()
units = models.IntegerField()
Now, suppose I want to get the number of awards for all users and the number of awards used for all users (ie, a queryset containing both). I prefer to do it one query for each calculation - when I combined it in my code I had some unexpected results. Also for some of my queries it won't be possible to do it one query, since the query will get too complex - I'm calculating 8 fields. This is how I solved it so far:
def get_summary(query_date)
summary = (Award.objects.filter(awardreceived__date__lte=query_date))
.annotate(awarded=Sum('awardissuedactivity__units_awarded')))
awards_used = (Award.objects.filter(awardused__date__lte=query_date)
.annotate(used=Sum('awardused__date__lte__units')))
award_used_dict = {}
for award in awards_used:
award_used_dict[award] = award.used
for award in summary:
award.used = award_used_dict.get(award, 0)
return summary
I'm sure there must be a way to solve this without the dictionary approach? For instance, something like this: awards_used.get(award=award), but this causes a db lookup every loop.
Or some other fancy way to join the querysets?
Note this is a simplified example and I know for this example the DB structure can be improved, I'm just trying to illustrate my question.
SOLUTION 1
Just try to concatenate your queryset using |
final_q = q1 | q2
In your example
final_q = summary | awards_used
UPDATED:
| does not works using calculated attributes, so, we can select our queryset first and then mapping our extra attributes
summary = Award.objects.filter(awardreceived__date__lte=query_date)
awards_used = Award.objects.filter(awardused__date__lte=query_date)
final_q = summary | awards_used
final_q = final_q.annotate(used=Sum('awardused__date__lte__units')).annotate(awarded=Sum('awardissuedactivity__units_awarded'))
SOLUTION 2
Using chain built-in function
from itertools import chain
final_list = list(chain(summary, awards_used))
There is an issue with this approach, you won't get a queryset, you will get a list containing instances.
I am working on a membership application. I would like to make a membership reminder. (member during a period of time which is not member for another period of time).
Currently, I am using set for making this calculation. See the code below.
class Member(models.Model):
...
class Membership(models.Model):
member = models.ForeignKey(Member, verbose_name=_("Member"))
start_date = models.DateField(_("Start date"))
end_date = models.DateField(_("End date"))
x = Member.objects.filter(Q(membership__start_date__lte=dt1) & Q(membership__end_date__gte=dt1))
y = Member.objects.filter(Q(membership__start_date__lte=dt2) & Q(membership__end_date__gte=dt2))
result = set(x) - set(y)
I would like to know of I can do it only by using the django ORM (filter, exclude, annotate, distinct ...)?
Thanks in advance for your help
UPDATE
In fact, my model is a bit more complex. I also have newspaper foreign key.
class Member(models.Model):
...
class Newspaper(models.Model):
...
class Membership(models.Model):
member = models.ForeignKey(Member, verbose_name=_("Member"))
start_date = models.DateField(_("Start date"))
end_date = models.DateField(_("End date"))
newspaper = models.ForeignKey(Newspaper)
I want to have the reminder for a given newspaper. In this case, the working query is
sin = models.Membership.objects.filter(start_date__lte=dt1,
end_date__gte=dt1,
newspaper__id=2)
sout = models.Membership.objects.filter(start_date__lte=dt2,
end_date__gte=dt2,
newspaper__id=2)
result = models.Member.objects.filter(membership__in=sin).exclude(membership__in=sout)
I think that this a more verbose version of the answer given Ghislain Leveque which is also working well for me.
Thanks to S.Lott and KillianDS for very valuable answers and sorry for not so clear question :)
Isn't it simply negating the second expression and putting it in the same filter? So you have something like !(a&b), which equals to (!a)|(!b), in this case:
result = Member.objects.filter(membership__start_date__lte=dt1, membership__end_date__gte=dt1, ~Q(membership__start_date__lte=dt2) | ~Q(membership__end_date__gte=dt2))
note by the way that for simple anding and basic lookups you need no Q objects, like I showed with the first two lookup parameters. Anding happens just by passing multiple arguments, Q objects are needed for negating and OR'ing lookups.
A relational database table is a set -- by definition. Set - is where not exists in SQL, which is exclude in Django's ORM.
It seems (without testing) that you're doing this.
result = Member.objects.filter(
Q(membership__start_date__lte=dt1) & Q(membership__end_date__gte=dt1)
).exclude(
Q(membership__start_date__lte=dt2) & Q(membership__end_date__gte=dt2)
)
You should try :
result = Member.objects.\
filter(
membership__start_date__lte = dt1,
membership__end_date__gte=dt1).\
exclude(
pk__in = \
Member.objects.filter(
membership__start_date__lte = dt2,
membership__end_date__gte = dt2).\
values_list('pk')