With a method to get all related connections, is it better to simply return a queryset, or iterate through the queryset to return a list of IDs?
example code:
class Foo(models.Model):
bar = models.ForeignKey(Bar, related_name="foos")
class Bar(models.Model):
is_active = models.BooleanField(default=True)
def get_foos(self):
#this is the method to focus on for the question
to return a queryset all that would need to be in get_foos is the following:
return self.foos.all()
but to get a list of ids, it would have to look as such:
foos = self.foos.all()
ids = []
for foo in foos:
ids.append(foo.pk)
return ids
I was looking through an API, and they used the latter, but I don't really understand why you would ever actually do that if you could just do a one-liner like the former! Can someone please explain the benefits of these methods and if there are specific cases in which one is better than the other?
You can get Foo ids in one line:
ids = self.foos.values_list('pk', flat=True)
Using id list is more efficient if used in other expressions, e.g.:
my_query.filter(foo__pk__in=ids)
If you really saw something like your second snippet in some published app, then you can send them a patch (and probably more than one, since chances are you'll find a lot of bad code in this app). Point is : there's a lot of very poorly written Django apps around. As Niekas posted, you can get the same result with whay better performances in a no-brainer one-liner.
Now wrt/ why return a list of pks instead of a full queryset, well, sometimes that's really just what you need, and sometimes that's just silly - depends on the context, really.
I don't think this is a better vs worse scenario. The ID list can also be generated with a one-liner:
list(self.foos.values_list('pk', flat=True))
It's sometimes good practice for a function to return the minimal or least powerful representation of the data requested. Maybe you don't want callers to have access to the queryset object.
Maybe those IDs are going to the template rendering engine, and you'd rather not have templates with write access to querysets, for example.
Related
I 've got a real problem that i can't fix. I'm trying to order a transaction by time but everything I do for ordering the queryset.... nothing work.
here is my model
class Transaction(models.Model):
collocDonneur=models.ForeignKey(Person,related_name="collocDonneur",on_delete=models.SET_NULL,null=True, blank=True)
collocReceveur=models.ForeignKey(Person,related_name="collocReceveur",on_delete=models.SET_NULL,null=True, blank=True)
point=models.IntegerField(null=False)
raison=models.TextField(null=True,blank=True)
date=models.DateTimeField(default=timezone.now)
class Meta:
verbose_name="Transaction"
and here is my view :
def vote(request):
person=Person.objects.all()
transactions=Transaction.objects.all()
transactions.order_by('date')
return render(request,"liste.html",locals())
even if I replace 'date' by '-date', nothing work, even if i do a reverse() on the queryset...
If you call .order_by(..) you do not sort the given queryset, you construct a new one that is a copy of the old one, but where the items are ordered. You could think of these in a similar way as strings: you can not alter a string, you can only make a (modified) copy of the string. Strictly speaking, you can alter the state of a QuerySet, but that is usually not a good idea, by changing values hold by the QuerySet (and the underlying .query), it is likely that you will break some assumption, and thus construct an invalid query, or defeat the caching mechanism.
You thus should construct a queryset like:
def vote(request):
persons = Person.objects.all()
transactions = Transaction.objects.order_by('date')
return render(
request,
'liste.html',
{'persons': persons, 'transactions': transaction}
)
Extra notes:
since person is a collection of Persons, it is better to name it persons, not person.
please do not use locals() this is an anti-pattern, since you make it unclear what you pass to the context. If you later aim to optimize the view for example, you might remove some variable by mistake. You furthermore easily start passing too much variables than the ones you really need.
A have piece of code, which fetches some QuerySet from DB and then appends new calculated field to every object in the Query Set. It's not an option to add this field via annotation (because it's legacy and because this calculation based on another already pre-fetched data).
Like this:
from django.db import models
class Human(models.Model):
name = models.CharField()
surname = models.CharField()
def calculate_new_field(s):
return len(s.name)*42
people = Human.objects.filter(id__in=[1,2,3,4,5])
for s in people:
s.new_column = calculate_new_field(s)
# people.somehow_reorder(new_order_by=new_column)
So now all people in QuerySet have a new column. And I want order these objects by new_column field. order_by() will not work obviously, since it is a database option. I understand thatI can pass them as a sorted list, but there is a lot of templates and other logic, which expect from this object QuerySet-like inteface with it's methods and so on.
So question is: is there some not very bad and dirty way to reorder existing QuerySet by dinamically added field or create new QuerySet-like object with this data? I believe I'm not the only one who faced this problem and it's already solved with django. But I can't find anything (except for adding third-party libs, and this is not an option too).
Conceptually, the QuerySet is not a list of results, but the "instructions to get those results". It's lazily evaluated and also cached. The internal attribute of the QuerySet that keeps the cached results is qs._result_cache
So, the for s in people sentence is forcing the evaluation of the query and caching the results.
You could, after that, sort the results by doing:
people._result_cache.sort(key=attrgetter('new_column'))
But, after evaluating a QuerySet, it makes little sense (in my opinion) to keep the QuerySet interface, as many of the operations will cause a reevaluation of the query. From this point on you should be dealing with a list of Models
Can you try it functions.Length:
from django.db.models.functions import Length
qs = Human.objects.filter(id__in=[1,2,3,4,5])
qs.annotate(reorder=Length('name') * 42).order_by('reorder')
I commonly find myself writing the same criteria in my Django application(s) more than once. I'll usually encapsulate it in a function that returns a Django Q() object, so that I can maintain the criteria in just one place.
I will do something like this in my code:
def CurrentAgentAgreementCriteria(useraccountid):
'''Returns Q that finds agent agreements that gives the useraccountid account current delegated permissions.'''
AgentAccountMatch = Q(agent__account__id=useraccountid)
StartBeforeNow = Q(start__lte=timezone.now())
EndAfterNow = Q(end__gte=timezone.now())
NoEnd = Q(end=None)
# Now put the criteria together
AgentAgreementCriteria = AgentAccountMatch & StartBeforeNow & (NoEnd | EndAfterNow)
return AgentAgreementCriteria
This makes it so that I don't have to think through the DB model more than once, and I can combine the return values from these functions to build more complex criterion. That works well so far, and has saved me time already when the DB model changes.
Something I have realized as I start to combine the criterion from these functions that is that a Q() object is inherently tied to the type of object .filter() is being called on. That is what I would expect.
I occasionally find myself wanting to use a Q() object from one of my functions to construct another Q object that is designed to filter a different, but related, model's instances.
Let's use a simple/contrived example to show what I mean. (It's simple enough that normally this would not be worth the overhead, but remember that I'm using a simple example here to illustrate what is more complicated in my app.)
Say I have a function that returns a Q() object that finds all Django users, whose username starts with an 'a':
def UsernameStartsWithAaccount():
return Q(username__startswith='a')
Say that I have a related model that is a user profile with settings including whether they want emails from us:
class UserProfile(models.Model):
account = models.OneToOneField(User, unique=True, related_name='azendalesappprofile')
emailMe = models.BooleanField(default=False)
Say I want to find all UserProfiles which have a username starting with 'a' AND want use to send them some email newsletter. I can easily write a Q() object for the latter:
wantsEmails = Q(emailMe=True)
but find myself wanting to something to do something like this for the former:
startsWithA = Q(account=UsernameStartsWithAaccount())
# And then
UserProfile.objects.filter(startsWithA & wantsEmails)
Unfortunately, that doesn't work (it generates invalid PSQL syntax when I tried it).
To put it another way, I'm looking for a syntax along the lines of Q(account=Q(id=9)) that would return the same results as Q(account__id=9).
So, a few questions arise from this:
Is there a syntax with Django Q() objects that allows you to add "context" to them to allow them to cross relational boundaries from the model you are running .filter() on?
If not, is this logically possible? (Since I can write Q(account__id=9) when I want to do something like Q(account=Q(id=9)) it seems like it would).
Maybe someone suggests something better, but I ended up passing the context manually to such functions. I don't think there is an easy solution, as you might need to call a whole chain of related tables to get to your field, like table1__table2__table3__profile__user__username, how would you guess that? User table could be linked to table2 too, but you don't need it in this case, so I think you can't avoid setting the path manually.
Also you can pass a dictionary to Q() and a list or a dictionary to filter() functions which is much easier to work with than using keyword parameters and applying &.
def UsernameStartsWithAaccount(context=''):
field = 'username__startswith'
if context:
field = context + '__' + field
return Q(**{field: 'a'})
Then if you simply need to AND your conditions you can combine them into a list and pass to filter:
UserProfile.objects.filter(*[startsWithA, wantsEmails])
I'm trying to filter a queryset by checking that the object is in a list of those objects.
employee_list = [<Employee: A>, <Employee: B>, <Employee: C>]
qs = Employee.objects.filter(id__in=employee_list, [other_filters])
After running above, qs is an empty list. I was thinking that I could make a new list such as
employee_ids = [emp.id for emp in employee_list]
qs = Employee.objects.filter(id__in=employee_ids, [other_filters])
I haven't done benchmarking on this method, but I imagine performance would probably take a hit. Alternatively I could intersect the lists afterwards like:
qs = Employee.objects.filter([other_filters])
filtered_qs = [emp for emp in employee_lids if emp in qs]
However, I think the performance hit would be even worse this way.
What's the best/fastest way to do this? Thanks.
Rule of thumb is to filter as much as possible through SQL, so I would go for
qs = Employee.objects.filter(id__in=[emp.id for emp in employee_list], [other_filters])
I do not have any performance testing to back this up with though.
As Martol1ni noted you want to filter at the SQL level whenever possible so I think your methods do get progressively slower, but another issue...
Based on the Django Docs: https://docs.djangoproject.com/en/dev/ref/models/querysets/ I think yourid__in should be a list of integer ids, not a list of models.
Edit:
Oh I see he covered this in his answer, but it was not explicit that it was incorrect in your question.
Edit2:
But yes, if you want to know for sure, what really matters is real world performance, which you can do with django-debug-toolbar. It seems to me though the real issue was the id__in misuse which lead you to find trickier ways to do what you wanted to do.
I am trying to test my Django views. This view passes a QuerySet to the template:
def merchant_home(request, slug):
merchant = Merchant.objects.get(slug=slug)
product_list = merchant.products.all()
return render_to_response('merchant_home.html',
{'merchant': merchant,
'product_list': product_list},
context_instance=RequestContext(request))
and test:
def test(self):
"Merchant home view should send merchant and merchant products to the template"
merchant = Merchant.objects.create(name='test merchant')
product = Product.objects.create(name='test product', price=100.00)
merchant.products.add(product)
test_client = Client()
response = test_client.get('/' + merchant.slug)
# self.assertListEqual(response.context['product_list'], merchant.products.all())
self.assertQuerysetEqual(response.context['product_list'], merchant.products.all())
EDIT
I am using self.assertQuerysetEqual instead of self.assertListEqual. Unfortunately this still doesn't work, and the terminal displays this:
['<Product: Product object>'] != [<Product: Product object>]
assertListEqual raises: 'QuerySet' object has no attribute 'difference' and
assertEqual does not work either, although self.assertSetEqual(response.context['product_list'][0], merchant.products.all()[0]) does pass.
I assume this is because the QuerySets are different objects even though they contain the same model instances.
How do I test that two QuerySets contain the same data? I am even testing this correctly? This is my 4th day learning Django so I would like to know best practices, if possible. Thanks.
By default assertQuerysetEqual uses repr() on the first argument. This is why you were having issues with the strings in the queryset comparison.
To work around this you can override the transform argument with a lambda function that doesn't use repr():
self.assertQuerysetEqual(queryset_1, queryset_2, transform=lambda x: x)
Use assertQuerysetEqual, which is built to compare the two querysets for you. You will need to subclass Django's django.test.TestCase for it to be available in your tests.
I just had the same problem. The second argument of assertQuerysetEqual needs to be a list of the expected repr()s as strings. Here is an example from the Django test suite:
self.assertQuerysetEqual(c1.tags.all(), ["<Tag: t1>", "<Tag: t2>"], ordered=False)
I ended up solving this issue using map to repr() each entry in the queryset inside the self.assertQuerysetEqual call, e.g.
self.assertQuerysetEqual(queryset_1, map(repr, queryset_2))
An alternative, but not necessarily better, method might look like this (testing context in a view, for example) when using pytest:
all_the_things = Things.objects.all()
assert set(response.context_data['all_the_things']) == set(all_the_things)
This converts it to a set, which is directly comparable with another set. Be careful with the behaviour of set though, it might not be exactly what you want since it will remove duplicates and ignore the order of objects.
I found that using self.assertCountEqual(queryset1, queryset2) also solves the issue.