I have three different models that I want to gather in a feed type page. They do all contain different types of things but for simplicity, the models are the same in this instance.
class ObjectA(models.Model):
text = models.TextField()
pub_date = models.DateTimeField('date published',auto_now_add=True)
...
class ObjectB(models.Model):
text = models.TextField()
pub_date = models.DateTimeField('date published',auto_now_add=True)
...
class ObjectC(models.Model):
text = models.TextField()
pub_date = models.DateTimeField('date published',auto_now_add=True)
...
What would be the general idea to serialize lists of all three objects into one list ordered by pub_date using the Django REST Framework. I just have experience using the meta version below but it can only deal with one model I am assuming. Thanks in advance.
class ObjectAListSerializer(serializers.ModelSerializer):
class Meta:
model = ObjectA
fields = [
'text',
'pub_date'
]
Pretty much trying to create something that would work like this:
class AllObjectsListSerializer(serializers.ModelSerializer):
The most important thing here is that you shouldn't be having three different models here. If they are storing the same data, there should be only one model. To have three models means every time you need to execute a statement you need to precede that with a IF ELIF ELSE which is far from DRY. Not to mention the fact that you need to do a UNION as suggested by Wtower in his answer
Merge the models.
You need to make a union on the querysets:
query_union = query1 | query 2
And you need a custom serializer for the fields of that union. Or if the union fields are all the same from any of the models, possibly use those model's modelserializer, but haven't tested that one.
dataA = ASerializer(queryset_A, many=True).data
dataB = BSerializer(queryset_B, many=True).data
dataC = CSerializer(queryset_C, many=True).data
just
return Response(data={'dataA ': dataA , 'dataB ': dataB ,'dataC ': dataC })
If you want return a list,the item is {'a':a,''b':'b,'c':c},you should declare you relation between a,b,c and can filter b and c with a.If so,i will write an example for you.
Related
I have multiple models which has fields like "created_at" "updated_at" which I don't want to get with objects.values().
Does Django has any way to exclude fields in values()?
I know people refer to defer(), but it doesn't return QuerySet<Dict> like values() instead returns QuerySet<Model>.
I tried objects.defer("created_at", "updated_at").values(), but it includes those 2 deferred fields in the resulting Dict.
I see defer().query only selecting the non-exluded fields in the SQL, but using defer(..).values() resets the deferred fields and selects all fields.
I cannot specify which field I want, since different model has different fields, I can only specity which fields I don't want. So I cannot use values('name', 'age', ...)
I'm planning to use a CustomeManager, which I can use in all model.
Example:
class CustomManager(models.Manager):
def values_excluded(self):
return self.values() # somehow exlude the fields and return QuerySet<Dict>
class ExampleModel(models.Model):
name = models.CharField(max_length=20)
age = models.IntegerField()
created_at = models.DateTimeField()
updated_at = models.DateTimeField()
objects = CustomManager()
ExampleModel.objects.values_excluded()
Is there any way in Django or do I have to manually delete those keys from the resulting Dict from values()?
esclude_fields = ['created_at', 'updated_at']
keys = [f.name for f in Model._meta.local_fields if f.name not in esclude_fields]
queryset.values(*keys)
This should work:
class CustomManager(models.Manager):
def values_excluded(self, *excluded_fields):
included_fields = [f.name for f in self.model._meta.fields if f.name not in excluded_fields]
return self.values(*included_fields)
Here are my simplified models:
class MasterFood(models.Model):
class Meta:
verbose_name_plural = 'Master Foods'
FOOD_TYPE_CHOICES = (
('F', 'Fats'),
('C', 'Carbohydrates'),
('P', 'Proteins'),
('O', 'Others'),
)
food_name = models.CharField(max_length=255)
food_type = models.CharField(max_length=20,choices=FOOD_TYPE_CHOICES)
def __str__(self):
return self.food_name
class MasterMeal(models.Model):
class Meta:
verbose_name_plural = 'Master Meal Plan Meals'
proteins = models.ManyToManyField(to="MasterFood", limit_choices_to={'food_type': 'P'},blank=True,related_name='food_proteins')
carbohydrates = models.ManyToManyField(to="MasterFood", limit_choices_to={'food_type': 'C'}, blank=True, related_name='food_carbohydrates')
fats = models.ManyToManyField(to="MasterFood", limit_choices_to={'food_type': 'F'}, blank=True, related_name='food_fats')
def all_foods(self):
return list(self.proteins.all())+list(self.carbohydrates.all())+list(self.fats.all())
def __str__(self):
return ', '.join(map(lambda x: x.food_name, self.all_foods()))+f' ({len(self.all_foods())})'
and my modelAdmin object in admin.py:
class MealAdmin(admin.ModelAdmin):
model = MasterMeal
save_as = True
ordering = ('proteins__food_name','carbohydrates__food_name','fats__food_name',)
What I would like is in the Django admin page for MasterMeals is to have each object named after the __str__ method given in my MasterMeal model, but in a different ordering than the default.
Specifically, I would like the objects to be sorted in alphabetical order according to the __str__ method definition, and if possible, I don't want to modify my MasterMeal model to add another field to achieve this. I have tried several things such as the ordering = ('proteins__food_name','carbohydrates__food_name','fats__food_name',) in my MealAdmin object as well as ordering = ('proteins','carbohydrates','fats',). I have also tried overwriting the queryset of the ModelAdmin with an annotate function like:
class MealAdmin(admin.ModelAdmin):
model = MasterMeal
save_as = True
def get_queryset(self,request):
qs = super(MealAdmin,self).get_queryset(request)
return qs.distinct() \
.annotate(protein_order=F('proteins__food_name'),carb_order = F('carbohydrates__food_name'),fat_order = F('fats__food_name')) \
.order_by('protein_order', 'carb_order','fat_order')
but I am clearly not annotating the objects in the way that they need to be in order to get the ordering that I am after. All 3 of these examples produce this exact same ordering:
Incorrect Ordering,
You can see an object with the exact same name has several other entries in between it. The objects are not sorted in alphabetical order according to the __str__ method, and are instead, (I believe?) being sorted on keys that have been generated for each of the groups of proteins/carbs/fats.
I did read here in the Django docs in regard to ModelAdmin.ordering that:
'To ensure a deterministic ordering of results, the changelist adds pk to the ordering if it can’t find a single or unique together set of fields that provide total ordering.' But I do not know how to modify my code to get it to order these objects in the order I DO want.
I believe I could add a non-editable field to my MasterMeal model, and overwrite the save() method for the model as shown here, in order to auto-populate the field with my desired string, which I believe I could then order on, but I don't want to have to change my model and have to update all of my objects on all of my servers to add this field.
Can anyone tell me if there is a way I can achieve this ordering?
I am very new to Django, and have limited experience with Python, so thank you very much for any help you can provide.
You can do something like this:
class MasterMeal(models.Model):
class Meta:
verbose_name_plural = 'Master Meal Plan Meals'
proteins = models.ManyToManyField(to="MasterFood", limit_choices_to={'food_type': 'P'},blank=True,related_name='food_proteins')
carbohydrates = models.ManyToManyField(to="MasterFood", limit_choices_to={'food_type': 'C'}, blank=True, related_name='food_carbohydrates')
fats = models.ManyToManyField(to="MasterFood", limit_choices_to={'food_type': 'F'}, blank=True, related_name='food_fats')
def all_foods_string(self):
proteins = self.proteins.all().values_list('food_name', flat=True)
carbs = self.carbohydrates.all().values_list('food_name', flat=True)
fats = self.fats.all().values_list('food_name', flat=True)
return sorted(proteins + carbs + fats)
def __str__(self):
all_foods = self.all_foods_string()
return ', '.join(all_foods)) + f' ({len(all_foods)})'
I changed the query for each food to only return the string value of the food_name field to avoid making full query of the object if we only need one field (this is very useful).
I found that it wasn't possible to do what I wanted since I did not have a field to order on and I was not able to use a query expression to do what I needed. I chose instead to create a new CharField in my model, name, which I auto populated with my __str__ method when left blank. See the solution here.
There are dozens of posts about n+1 queries in nested relations in Django, but I can't seem to find the answer to my question. Here's the context:
The Models
class Book(models.Model):
title = models.CharField(max_length=255)
class Tag(models.Model):
book = models.ForeignKey('app.Book', on_delete=models.CASCADE, related_name='tags')
category = models.ForeignKey('app.TagCategory', on_delete=models.PROTECT)
page = models.PositiveIntegerField()
class TagCategory(models.Model):
title = models.CharField(max_length=255)
key = models.CharField(max_length=255)
A book has many tags, each tag belongs to a tag category.
The Serializers
class TagSerializer(serializers.ModelSerializer):
class Meta:
model = Tag
exclude = ['id', 'book']
class BookSerializer(serializers.ModelSerializer):
tags = TagSerializer(many=True, required=False)
class Meta:
model = Book
fields = ['title', 'tags']
def create(self, validated_data):
with transaction.atomic():
tags = validated_data.pop('tags')
book = Book.objects.create(**validated_data)
Tag.objects.bulk_create([Tag(book=book, **tag) for tag in tags])
return book
The Problem
I am trying to POST to the BookViewSet with the following example data:
{
"title": "The Jungle Book"
"tags": [
{ "page": 1, "category": 36 }, // plot intro
{ "page": 2, "category": 37 }, // character intro
{ "page": 4, "category": 37 }, // character intro
// ... up to 1000 tags
]
}
This all works, however, during the post, the serializer proceeds to make a call for each tag to check if the category_id is a valid one:
With up to 1000 nested tags in a call, I can't afford this.
How do I "prefetch" for the validation?
If this is impossible, how do I turn off the validation that checks if a foreign_key id is in the database?
EDIT: Additional Info
Here is the view:
class BookViewSet(views.APIView):
queryset = Book.objects.all().select_related('tags', 'tags__category')
permission_classes = [IsAdminUser]
def post(self, request, format=None):
serializer = BookSerializer(data=request.data)
if serializer.is_valid():
serializer.save()
return Response(serializer.data, status=status.HTTP_201_CREATED)
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
The DRF serializer is not the place (in my own opinion) to optimize a DB query. Serializer has 2 jobs:
Serialize and check the validity of input data.
Serialize output data.
Therefore the correct place to optimize your query is the corresponding view.
We will use the select_related method that:
Returns a QuerySet that will “follow” foreign-key relationships, selecting additional related-object data when it executes its query. This is a performance booster which results in a single more complex query but means later use of foreign-key relationships won’t require database queries.
to avoid the N+1 database queries.
You will need to modify the part of your view code that creates the corresponding queryset, in order to include a select_related call.
You will also need to add a related_name to the Tag.category field definition.
Example:
# In your Tag model:
category = models.ForeignKey(
'app.TagCategory', on_delete=models.PROTECT, related_name='categories'
)
# In your queryset defining part of your View:
class BookViewSet(views.APIView):
queryset = Book.objects.all().select_related(
'tags', 'tags__categories'
) # We are using the related_name of the ForeignKey relationships.
If you want to test something different that uses also the serializer to cut down the number of queries, you can check this article.
I think the issue here is that the Tag constructor is automatically converting the category id that you pass in as category into a TagCategory instance by looking it up from the database. The way to avoid that is by doing something like the following if you know that all of the category ids are valid:
def create(self, validated_data):
with transaction.atomic():
tags = validated_data.pop('tags')
book = Book.objects.create(**validated_data)
tag_instances = [ Tag(book_id=book.id, page=x['page'], category_id=x['category']) for x in tags ]
Tag.objects.bulk_create(tag_instances)
return book
I've come up with an answer that gets things working (but that I'm not thrilled about): Modify the Tag Serializer like this:
class TagSerializer(serializers.ModelSerializer):
category_id = serializers.IntegerField()
class Meta:
model = Tag
exclude = ['id', 'book', 'category']
This allows me to read/write a category_id without having the overhead of validations. Adding category to exclude does mean that the serializer will ignore category if it's set on the instance.
Problem is that you don't set created tags to the book instance so serializer try to get this while returning.
You need to set it to the book as a list:
def create(self, validated_data):
with transaction.atomic():
book = Book.objects.create(**validated_data)
# Add None as a default and check that tags are provided
# If you don't do that, serializer will raise error if request don't have 'tags'
tags = validated_data.pop('tags', None)
tags_to_create = []
if tags:
tags_to_create = [Tag(book=book, **tag) for tag in tags]
Tag.objects.bulk_create(tags_to_create)
# Here I set tags to the book instance
setattr(book, 'tags', tags_to_create)
return book
Provide Meta.fields tuple for TagSerializer (it's weird that this serializer don't raise error saying that fields tuple is required)
class TagSerializer(serializers.ModelSerializer):
class Meta:
model = Tag
fields = ('category', 'page',)
Prefetching tag.category should be NOT necessary in this case because it's just id.
You will need prefetching Book.tags for GET method. The simplest solution is to create static method for serializer and use it in viewset get_queryset method like this:
class BookSerializer(serializers.ModelSerializer):
...
#staticmethod
def setup_eager_loading(queryset): # It can be named any name you like
queryset = queryset.prefetch_related('tags')
return queryset
class BookViewSet(views.APIView):
...
def get_queryset(self):
self.queryset = BookSerializer.setup_eager_loading(self.queryset)
# Every GET request will prefetch 'tags' for every book by default
return super(BookViewSet, self).get_queryset()
select_related function will check ForeignKey in the first time.
Actually,this is a ForeignKey check in the relational database and you can use SET FOREIGN_KEY_CHECKS=0; in database to close inspection.
So let's say I have the following models in Django:
class Review(models.Model):
datetime = models.DateTimeField(("Date"), default=timezone.now)
class Rating(models.Model):
datetime = models.DateTimeField(("Date"), default=timezone.now)
And my view looks like this:
def get_page(request, id_slug):
context_dict = {}
reviews = Review.objects.filter(attribute=id_slug).order_by('-datetime')
ratings = Rating.objects.filter(attribute=id_slug).order_by('-datetime')
context_dict['reviews'] = reviews
context_dict['ratings'] = ratings
return render(request, 'page.html', context_dict)
What I want to do is make a query set that would include both Review and Rating objects, ordered by the datetime attribute, so something like:
activity = reviewsANDratings.order_by('-datetime')
context_dict['activity'] = activity
What would be the best way to achieve this?
Thank you!
If your models are basically the same thing, or parent/child of one another, consider using Django polymorphic, which will let you get all the Reviews and Ratings in one PolymorphicQuerySet.
For more disparate models, (or for simpler implementation), just do two queries, like you are now, and merge the items in Python like so:
reviews_and_ratings = sum([list(reviews), list(ratings)], [])
reviews_and_ratings.sort(key=lambda item: item.datetime, reverse=True)
context_dict['reviews_and_ratings'] = reviews_and_ratings
I am trying to show a M2M field in a django-table2 as seen in Django-tables2: How to use accessor to bring in foreign columns? and Accessing related models with django-tables2
Using: foreigncolumn = tables.Column(accessor='foreignmodel.foreigncolumnname'), I only see a '--'...
# The models:
class Organism(models.Model):
species_name = models.CharField(max_length=200)
strain_name = models.CharField(max_length=200)
eukaryotic = models.BooleanField(default=True)
lipids = models.ManyToManyField('Lipid',blank=True)
class Lipid(models.Model):
lm_id = models.CharField(max_length=100)
common_name = models.CharField(max_length=100,blank=True)
category = models.CharField(max_length=100,blank=True)
#The tables
class OrganismTable(tables.Table):
name = tables.LinkColumn('catalog:organism-detail', text=lambda record: record.species_name, args=[A('pk')])
lp = tables.Column(accessor='Lipid.common_name')
class Meta:
model = Organism
sequence = ['name','lp']
exclude = ['id','species_name']
Any idea what I'm doing wrong?
This does not work so easily for ManyToManyFields because of the simple way Accessor works. You could display the repr of the related QuerySet via 'lipids.all' but that does not seem sufficient here. You can, however, add a property (or method) to your Organism model and use it in the accessor. This way, you can display any custom information related to the instance:
class Organism(models.Model):
# ...
#property
def lipid_names(self):
return ', '.join(l.common_name for l in self.lipids.all()) # or similar
class OrganismTable(tables.Table):
# ...
lp = tables.Column(accessor='lipid_names')
I would recommend then to add a prefetch_related('lipids') to the Organism QuerySet that you pass to the table for better performance.