I 've got a real problem that i can't fix. I'm trying to order a transaction by time but everything I do for ordering the queryset.... nothing work.
here is my model
class Transaction(models.Model):
collocDonneur=models.ForeignKey(Person,related_name="collocDonneur",on_delete=models.SET_NULL,null=True, blank=True)
collocReceveur=models.ForeignKey(Person,related_name="collocReceveur",on_delete=models.SET_NULL,null=True, blank=True)
point=models.IntegerField(null=False)
raison=models.TextField(null=True,blank=True)
date=models.DateTimeField(default=timezone.now)
class Meta:
verbose_name="Transaction"
and here is my view :
def vote(request):
person=Person.objects.all()
transactions=Transaction.objects.all()
transactions.order_by('date')
return render(request,"liste.html",locals())
even if I replace 'date' by '-date', nothing work, even if i do a reverse() on the queryset...
If you call .order_by(..) you do not sort the given queryset, you construct a new one that is a copy of the old one, but where the items are ordered. You could think of these in a similar way as strings: you can not alter a string, you can only make a (modified) copy of the string. Strictly speaking, you can alter the state of a QuerySet, but that is usually not a good idea, by changing values hold by the QuerySet (and the underlying .query), it is likely that you will break some assumption, and thus construct an invalid query, or defeat the caching mechanism.
You thus should construct a queryset like:
def vote(request):
persons = Person.objects.all()
transactions = Transaction.objects.order_by('date')
return render(
request,
'liste.html',
{'persons': persons, 'transactions': transaction}
)
Extra notes:
since person is a collection of Persons, it is better to name it persons, not person.
please do not use locals() this is an anti-pattern, since you make it unclear what you pass to the context. If you later aim to optimize the view for example, you might remove some variable by mistake. You furthermore easily start passing too much variables than the ones you really need.
Related
A have piece of code, which fetches some QuerySet from DB and then appends new calculated field to every object in the Query Set. It's not an option to add this field via annotation (because it's legacy and because this calculation based on another already pre-fetched data).
Like this:
from django.db import models
class Human(models.Model):
name = models.CharField()
surname = models.CharField()
def calculate_new_field(s):
return len(s.name)*42
people = Human.objects.filter(id__in=[1,2,3,4,5])
for s in people:
s.new_column = calculate_new_field(s)
# people.somehow_reorder(new_order_by=new_column)
So now all people in QuerySet have a new column. And I want order these objects by new_column field. order_by() will not work obviously, since it is a database option. I understand thatI can pass them as a sorted list, but there is a lot of templates and other logic, which expect from this object QuerySet-like inteface with it's methods and so on.
So question is: is there some not very bad and dirty way to reorder existing QuerySet by dinamically added field or create new QuerySet-like object with this data? I believe I'm not the only one who faced this problem and it's already solved with django. But I can't find anything (except for adding third-party libs, and this is not an option too).
Conceptually, the QuerySet is not a list of results, but the "instructions to get those results". It's lazily evaluated and also cached. The internal attribute of the QuerySet that keeps the cached results is qs._result_cache
So, the for s in people sentence is forcing the evaluation of the query and caching the results.
You could, after that, sort the results by doing:
people._result_cache.sort(key=attrgetter('new_column'))
But, after evaluating a QuerySet, it makes little sense (in my opinion) to keep the QuerySet interface, as many of the operations will cause a reevaluation of the query. From this point on you should be dealing with a list of Models
Can you try it functions.Length:
from django.db.models.functions import Length
qs = Human.objects.filter(id__in=[1,2,3,4,5])
qs.annotate(reorder=Length('name') * 42).order_by('reorder')
With a method to get all related connections, is it better to simply return a queryset, or iterate through the queryset to return a list of IDs?
example code:
class Foo(models.Model):
bar = models.ForeignKey(Bar, related_name="foos")
class Bar(models.Model):
is_active = models.BooleanField(default=True)
def get_foos(self):
#this is the method to focus on for the question
to return a queryset all that would need to be in get_foos is the following:
return self.foos.all()
but to get a list of ids, it would have to look as such:
foos = self.foos.all()
ids = []
for foo in foos:
ids.append(foo.pk)
return ids
I was looking through an API, and they used the latter, but I don't really understand why you would ever actually do that if you could just do a one-liner like the former! Can someone please explain the benefits of these methods and if there are specific cases in which one is better than the other?
You can get Foo ids in one line:
ids = self.foos.values_list('pk', flat=True)
Using id list is more efficient if used in other expressions, e.g.:
my_query.filter(foo__pk__in=ids)
If you really saw something like your second snippet in some published app, then you can send them a patch (and probably more than one, since chances are you'll find a lot of bad code in this app). Point is : there's a lot of very poorly written Django apps around. As Niekas posted, you can get the same result with whay better performances in a no-brainer one-liner.
Now wrt/ why return a list of pks instead of a full queryset, well, sometimes that's really just what you need, and sometimes that's just silly - depends on the context, really.
I don't think this is a better vs worse scenario. The ID list can also be generated with a one-liner:
list(self.foos.values_list('pk', flat=True))
It's sometimes good practice for a function to return the minimal or least powerful representation of the data requested. Maybe you don't want callers to have access to the queryset object.
Maybe those IDs are going to the template rendering engine, and you'd rather not have templates with write access to querysets, for example.
So QuerySets are "lazy" and only run in certain instances (repr,list,etc.). I have created a custom QuerySet for doing many queries, but these could end up having millions of items!! That is way more than I want to return.
When returning the evaluated QuerySet, it should not have more than 25 results! Now I know I could do the following:
first = example.objects.filter(...)
last = first.filter(...)
result = last[:25]
#do stuff with result
but I will be doing so many queries with example objects that I feel it unnecessary to have the line result = last[:25]. Is there a way to specify how a QuerySet is returned?
If there is, how can I change it so that whenever the QuerySet would be evaluated it only returns the first x items in the QuerySet where, in this case, x = 25
Important note:
slicing must be on evaluation because that way I can chain queries without limited results, but when I return a result upon evaluation, it would have a max of x
I'm not sure I understand what your issue with slicing the queryset is. If it's the extra line of code or the hardcoded number that's bothering you, you can run
example.objects.filter(**filters)[:x]
and pass x into whatever method you're using.
You can write a custom Manager:
class LimitedNumberOfResultsManager(models.Manager):
def get_queryset(self):
return super(LimitedNumberOfResultsManager, self).get_queryset()[:25]
Note: You may think that adding a slice here will immediately evaluate the queryset. It won't. Instead information about the query limit will be saved to an underlying Query object and used later, during the final evaluation - as long as it is not overwritten by an another slice in the meantime.
Then add the manager to your model:
class YourModel(models.Model):
# ...
objects = LimitedNumberOfResultsManager()
After setting this up YourModel.objects.all() and other operations on your queryset will always return only up to 25 results. You can still overwrite this any time using slicing. For example:
This will return up to 25 resuls:
YourModel.objects.filter(lorem='ipsum')
but this will return up to 100 resuls:
YourModel.objects.filter(lorem='ipsum')[:100]
One more point. Overwriting the default manager may be confusing to other people reading your code. So I think it would be better to leave the default manager alone and use the custom one as an optional alternative:
class YourModel(models.Model):
# ...
objects = models.Manager()
limited = LimitedNumberOfResultsManager()
With this set up this will return all results:
YourModel.objects.all()
and this will return only up to 25 results:
YourModel.limited.all()
Depending on your exact use case you man also want to look at pagination in Django.
I have a simple model which includes a product and category table. The Product model has a foreign key Category.
When I make a tastypie API call that returns a list of categories /api/vi/categories/
I would like to add a field that determines the "product count" / the number of products that have a giving category. The result would be something like:
category_objects[
{
id: 53
name: Laptops
product_count: 7
},
...
]
The following code is working but the hit on my DB is heavy
def dehydrate(self, bundle):
category = Category.objects.get(pk=bundle.obj.id)
products = Product.objects.filter(category=category)
bundle.data['product_count'] = products.count()
return bundle
Is there a more efficient way to build this query? Perhaps with annotate ?
You can use prefetch_related method of QuerSet to reverse select_related.
Asper documentation,
prefetch_related(*lookups)
Returns a QuerySet that will automatically
retrieve, in a single batch, related objects for each of the specified
lookups.
This has a similar purpose to select_related, in that both are
designed to stop the deluge of database queries that is caused by
accessing related objects, but the strategy is quite different.
If you change your dehydrate function to following then database will be hit single time.
def dehydrate(self, bundle):
category = Category.objects.prefetch_related("product_set").get(pk=bundle.obj.id)
bundle.data['product_count'] = category.product_set.count()
return bundle
UPDATE 1
You should not initialize queryset inside dehydrate function. queryset should be always set in Meta class only. Please have a look at following example from django-tastypie documentation.
class MyResource(ModelResource):
class Meta:
queryset = User.objects.all()
excludes = ['email', 'password', 'is_staff', 'is_superuser']
def dehydrate(self, bundle):
# If they're requesting their own record, add in their email address.
if bundle.request.user.pk == bundle.obj.pk:
# Note that there isn't an ``email`` field on the ``Resource``.
# By this time, it doesn't matter, as the built data will no
# longer be checked against the fields on the ``Resource``.
bundle.data['email'] = bundle.obj.email
return bundle
As per official django-tastypie documentation on dehydrate() function,
dehydrate
The dehydrate method takes a now fully-populated bundle.data & make
any last alterations to it. This is useful for when a piece of data
might depend on more than one field, if you want to shove in extra
data that isn’t worth having its own field or if you want to
dynamically remove things from the data to be returned.
dehydrate() is only meant to make any last alterations to bundle.data.
Your code does additional count query for each category. You're right about annotate being helpfull in this kind of a problem.
Django will include all queryset's fields in GROUP BY statement. Notice .values() and empty .group_by() serve limiting field set to required fields.
cat_to_prod_count = dict(Product.objects
.values('category_id')
.order_by()
.annotate(product_count=Count('id'))
.values_list('category_id', 'product_count'))
The above dict object is a map [category_id -> product_count].
It can be used in dehydrate method:
bundle.data['product_count'] = cat_to_prod_count[bundle.obj.id]
If that doesn't help, try to keep similar counter on category records and use singals to keep it up to date.
Note categories are usually a tree-like beings and you probably want to keep count of all subcategories as well.
In that case look at the package django-mptt.
I've looked through Tastypie's documentation and searched for a while, but can't seem to find an answer to this.
Let's say that we've got two models: Student and Assignment, with a one-to-many relationship between them. The Assignment model includes an assignment_date field. Basically, I'd like to build an API using Tastypie that returns Student objects sorted by most recent assignment date. Whether the sorting is done on the server or in the client side doesn't matter - but wherever the sorting is done, the assignment_date is needed to sort by.
Idea #1: just return the assignments along with the students.
class StudentResource(ModelResource):
assignments = fields.OneToManyField(
AssignmentResource, 'assignments', full=True)
class Meta:
queryset = models.Student.objects.all()
resource_name = 'student'
Unfortunately, each student may have tens or hundreds of assignments, so this is bloated and unnecessary.
Idea #2: augment the data during the dehydrate cycle.
class StudentResource(ModelResource):
class Meta:
queryset = models.Student.objects.all()
resource_name = 'student'
def dehydrate(self, bundle):
bundle.data['last_assignment_date'] = (models.Assignment
.filter(student=bundle.data['id'])
.order_by('assignment_date')[0].assignment_date)
This is not ideal, since it'll be performing a separate database roundtrip for each student record. It's also not very declarative, nor elegant.
So, is there a good way to get this kind of functionality with Tastypie? Or is there a better way to do what I'm trying to achieve?
You can sort a ModelResource by a field name. Check out this part of the documentation http://django-tastypie.readthedocs.org/en/latest/resources.html#ordering
You could also set this ordering by default in the Model: https://docs.djangoproject.com/en/dev/ref/models/options/#ordering