This question already has an answer here:
Sorting a Django QuerySet by a property (not a field) of the Model
(1 answer)
Closed 8 years ago.
I'm trying to order a query set by a property I defined in the model, but not sure the best way to do this. Here's the property:
#property
def name(self):
if self.custom_name:
return self.custom_name
else:
return self.module_object.name
Essentially, I'd like to do a:
things = Thing.objects.all().order_by('-name')
but of course getting a Caught FieldError while rendering: Cannot resolve keyword 'name' into field.
Any ideas?
EDIT: I understand that I can't sort this way because the #property isn't a database field. My question is how to sort given that #property isn't a database field.
You can't do that because that property is not in MySQL, but in your python code. If you really want to do this, you can on the client-side(though it will be very slow):
sorted(Thing.objects.all(), key=lambda t: t.name)
order_by happens on the sql level, so it can't make use of properties, only field data.
have a look at the queryset api, you may be able to make use of e.g. extra to annotate your query and sort on that
Have a look at django-denorm. It lets you maintain calculated values in the database (which you can then use to sort efficiently) for the cost of a single method decorator.
You can only order by database fields. You can pull all records, turn them into a list, and sort them, although that may be inefficient and slow.
Or you can add a database field called name and fill it in via a post-save hook; then you can sort it using order_by.
Related
I have a list of object ID's that I am getting from a query in an model's method, then I'm using that list to delete objects from a different model:
class SomeObject(models.Model):
# [...]
def do_stuff(self, some_param):
# [...]
ids_to_delete = {item.id for item in self.items.all()}
other_object = OtherObject.objects.get_or_create(some_param=some_param)
other_object.items.filter(item_id__in=ids_to_delete).delete()
What I don't like is that this takes 2 queries (well, technically 3 for the get_or_create() but in the real code it's actually .filter(some_param=some_param).first() instead of the .get(), so I don't think there's any easy way around that).
How do I pass in an unevaluated queryset as the argument to an __in lookup?
I would like to do something like:
ids_to_delete = self.items.all().values("id")
other_object.items.filter(item_id__in=ids_to_delete).delete()
You can, pass a QuerySet to the query:
other_object.items.filter(id__in=self.items.all()).delete()
this will transform it in a subquery. But not all databases, especially MySQL ones, are good with such subqueries. Furthermore Django handles .delete() manually. It will thus make a query to fetch the primary keys of the items, and then trigger the delete logic (and also remove items that have a CASCADE dependency). So .delete() is not done as one query, but at least two queries, and often a larger amount due to ForeignKeys with an on_delete trigger.
Note however that you here remove Item objects, not "unlink" this from the other_object. For this .remove(…) [Django-doc] can be used.
I should've tried the code sample I posted, you can in fact do this. It's given as an example in the documentation, but it says "be cautious about using nested queries and understand your database server’s performance characteristics" and recommends against doing this, casting the subquery into a list:
values = Blog.objects.filter(
name__contains='Cheddar').values_list('pk', flat=True)
entries = Entry.objects.filter(blog__in=list(values))
A have piece of code, which fetches some QuerySet from DB and then appends new calculated field to every object in the Query Set. It's not an option to add this field via annotation (because it's legacy and because this calculation based on another already pre-fetched data).
Like this:
from django.db import models
class Human(models.Model):
name = models.CharField()
surname = models.CharField()
def calculate_new_field(s):
return len(s.name)*42
people = Human.objects.filter(id__in=[1,2,3,4,5])
for s in people:
s.new_column = calculate_new_field(s)
# people.somehow_reorder(new_order_by=new_column)
So now all people in QuerySet have a new column. And I want order these objects by new_column field. order_by() will not work obviously, since it is a database option. I understand thatI can pass them as a sorted list, but there is a lot of templates and other logic, which expect from this object QuerySet-like inteface with it's methods and so on.
So question is: is there some not very bad and dirty way to reorder existing QuerySet by dinamically added field or create new QuerySet-like object with this data? I believe I'm not the only one who faced this problem and it's already solved with django. But I can't find anything (except for adding third-party libs, and this is not an option too).
Conceptually, the QuerySet is not a list of results, but the "instructions to get those results". It's lazily evaluated and also cached. The internal attribute of the QuerySet that keeps the cached results is qs._result_cache
So, the for s in people sentence is forcing the evaluation of the query and caching the results.
You could, after that, sort the results by doing:
people._result_cache.sort(key=attrgetter('new_column'))
But, after evaluating a QuerySet, it makes little sense (in my opinion) to keep the QuerySet interface, as many of the operations will cause a reevaluation of the query. From this point on you should be dealing with a list of Models
Can you try it functions.Length:
from django.db.models.functions import Length
qs = Human.objects.filter(id__in=[1,2,3,4,5])
qs.annotate(reorder=Length('name') * 42).order_by('reorder')
How can I apply annotations and filters from a custom manager queryset when filtering via a related field? Here's some code to demonstrate what I mean.
Manager and models
from django.db.models import Value, BooleanField
class OtherModelManager(Manager):
def get_queryset(self):
return super(OtherModelManager, self).get_queryset().annotate(
some_flag=Value(True, output_field=BooleanField())
).filter(
disabled=False
)
class MyModel(Model):
other_model = ForeignKey(OtherModel)
class OtherModel(Model):
disabled = BooleanField()
objects = OtherModelManager()
Attempting to filter the related field using the manager
# This should only give me MyModel objects with related
# OtherModel objects that have the some_flag annotation
# set to True and disabled=False
my_model = MyModel.objects.filter(some_flag=True)
If you try the above code you will get the following error:
TypeError: Related Field got invalid lookup: some_flag
To further clarify, essentially the same question was reported as a bug with no response on how to actually achieve this: https://code.djangoproject.com/ticket/26393.
I'm aware that this can be achieved by simply using the filter and annotation from the manager directly in the MyModel filter, however the point is to keep this DRY and ensure this behaviour is repeated everywhere this model is accessed (unless explicitly instructed not to).
How about running nested queries (or two queries, in case your backend is MySQL; performance).
The first to fetch the pk of the related OtherModel objects.
The second to filter the Model objects on the fetched pks.
other_model_pks = OtherModel.objects.filter(some_flag=...).values_list('pk', flat=True)
my_model = MyModel.objects.filter(other_model__in=other_model_pks)
# use (...__in=list(other_model_pks)) for MySQL to avoid a nested query.
I don't think what you want is possible.
1) I think you are miss-understanding what annotations do.
Generating aggregates for each item in a QuerySet
The second way to generate summary values is to generate an
independent summary for each object in a QuerySet. For example, if you
are retrieving a list of books, you may want to know how many authors
contributed to each book. Each Book has a many-to-many relationship
with the Author; we want to summarize this relationship for each book
in the QuerySet.
Per-object summaries can be generated using the annotate() clause.
When an annotate() clause is specified, each object in the QuerySet
will be annotated with the specified values.
The syntax for these annotations is identical to that used for the
aggregate() clause. Each argument to annotate() describes an aggregate
that is to be calculated.
So when you say:
MyModel.objects.annotate(other_model__some_flag=Value(True, output_field=BooleanField()))
You are not annotation some_flag over other_model.
i.e. you won't have: mymodel.other_model.some_flag
You are annotating other_model__some_flag over mymodel.
i.e. you will have: mymodel.other_model__some_flag
2) I'm not sure how familiar SQL is for you, but in order to preserve MyModel.objects.filter(other_model__some_flag=True) possible, i.e. to keep the annotation when doing JOINS, the ORM would have to do a JOIN over subquery, something like:
INNER JOIN
(
SELECT other_model.id, /* more fields,*/ 1 as some_flag
FROM other_model
) as sub on mymodel.other_model_id = sub.id
which would be super slow and I'm not surprised they are not doing it.
Possible solution
don't annotate your field, but add it as a regular field in your model.
The simplified answer is that models are authoritative on the field collection and Managers are authoritative on collections of models. In your efforts to make it DRY you made it WET, cause you alter the field collection in your manager.
In order to fix it, you would have to teach the model about the lookup and need to do that using the Lookup API.
Now I'm assuming that you're not actually annotating with a fixed value, so if that annotation is in fact reducible to fields, then you may just get it done, because in the end it needs to be mapped to database representation.
The below link specifies how to create a list-field in Django.
How to create list field in django
My question; how to add, remove items to this field? if in case that's possible. I want to be able to add to and remove from this field of a Model's instance afterwards. If not possible please someone suggest me an alternative.
in ListField:
to_python function is used to convert database string to python object(here is list)
get_prep_value is used to handle python object to string(to store into database next step).
when you call your obj.listfield, such as a.list, you'll get a python list, so you could use a.list.append(val), a.list.insert(0, val) method to operate, and at last call a.save(), listfield will do the conversion for you between database and python.
class Dummy(models.Model):
mylist = ListField()
def insert_to_mylist(self, index, value):
self.mylist.insert(index, value)
#self.save()
def remove_from_mylist(self, value):
self.mylist.remove(value)
#self.save()
for example:
d = Dummy()
d.mylist.insert(0, 'tu')
d.save()
d.mylist.remove('tu')
d.save()
d.insert_to_mylist(0, "tu")
d.save()
d.remove_from_mylist('tu')
d.save()
if you want to insert or remove without .save(), just add .save() to insert_to_mylist and remove_from_mylist function.
ListField is just used to conversion between database and python, so add a remove or insert to it may be difficult. But you can do it in your Model, if many Models need to use this method, just write an abstract model.
My way of storing lists in models is by converting them to a string and storing them as TextField.
This question already has answers here:
Convert Django Model object to dict with all of the fields intact
(17 answers)
Closed 6 years ago.
In my Django app, how can I turn objects from Models into a dictionary that includes the foreign-key references of the Model object?
When I try this:
from django.forms.models import model_to_dict
model_to_dict(instance, fields=[], exclude=[])
The resulting dictionary only has the direct fields. I would like to also get the foreign keys related to the Model object. How can I do this?
obj = get_object_or_404(CustomModel,id=some_id)
my_dict = obj.__dict__
How about:
model_to_dict(instance, fields=[field.name for field in instance._meta.fields])
By explicitly naming all of the fields, it should give you the foreign key ids at least (although I haven't verified).
Note that this function does not return fields with editable=False (since it is intended for forms).
I think I had the same need as you -- I wanted a plain-and-simple dict representation of an object. the other answers posted (at the time I write) wouldn't give me that, and the serializers, while useful for their purpose, produce output with extra info I don't want, and an inconvenient structure.
Though admittedly a hack, this is giving me good mileage:
from django.core import serializers
def obj_to_dict(model_instance):
serial_obj = serializers.serialize('json', [model_instance])
obj_as_dict = json.loads(serial_obj)[0]['fields']
obj_as_dict['pk'] = model_instance.pk
return obj_as_dict
wrapping the django_model_object in a list, then accessing item 0 after parsing the json is needed because, for some reason, serializers can only serialize iterables of model objects (weird).
You'd need some extra gears to handle any kind of foreign key fields, I may post back if I end up needing to write that (unless someone else edits it in first!).
HTH for now.
You may want to have look at Django serialization if you want to work with related models.
None of the other answers quite worked for me. This, however, seemed to do the trick.
def model_to_dict(instance, include=None, exclude=None):
fields = instance._meta.concrete_fields
if include is not None:
return {f.attname: getattr(instance, f.attname) for f in fields if f.name in include}
if exclude is not None:
return {f.attname: getattr(instance, f.attname) for f in fields if f.name not in exclude}
return {f.attname: getattr(instance, f.attname) for f in fields}
I imagine that you've already solved this problem for your case, but I found some info as I was searching for similar solutions. I wanted to be able to access objects and their fields/attributes in a template. I am just learning django.
What I finally read-up on was the https://docs.djangoproject.com/en/3.0/ref/class-based-views/generic-display/, which seem to be able to send an object instance or a queryset of objects along to a template. Then you can access the objects like the model.field, or even if it's a foreignkey field, model.foreignmodel.foreignmodelfield.
So it's been pretty useful for me. This was after I built a bunch of custom 'to-dict' methods in different class managers. This helped me a lot. But you'll have to read up on the Generic Views and how to customize them to really get the details you want.
-Matt