I'm trying to better understand how to write some more advanced queries. I'm used to the basic Model.objects.all() and .filter(), etc, but I'm having trouble writing out queries that are annotated.
Here is my model setup (I have removed any additional fields that are not necessary for this question):
from django.db import models
class Client(models.Model):
name = models.CharField(max_length=100)
class Project(models.Model):
client = models.ForeignKey(Client, on_delete=models.CASCADE)
class Campaign(models.Model):
project = models.ForeignKey(Project, on_delete=models.CASCADE, related_name="campaigns")
class Advertisement(models.Model):
campaign = models.ForeignKey(Campaign, on_delete=models.CASCADE, related_name='advertisements')
What I need is a query that shows how many Advertisements each client has, ordered by most to least.
I'm trying to follow the docs from here: https://docs.djangoproject.com/en/4.0/topics/db/aggregation/ but their annotation example isn't as nested, the publisher model example is a foreign key on the book object, where as with mine the client is a foreign key of a project, which is a foreign key of a campaign, which is a foreign key of advertisements.
Any advice on how to drill down through the layers of the models to get the info I need?
On Django ORM, the joins that navigate the relationships are done with __.
You can keep on nesting the relationship traversal.
This would make your query something like the following.
Client.objects.annotate(num_advertisements=Count('project__campaigns__advertisements'))
The names campaigns and advertisements have to be used because of the related_name you set.
Related
I'm try to decreasing queries to single query but queryset returns 2 queries. I tried some method but nothing changed
Models.py
class Match(models.Model):
...
participant = models.ManyToManyField(TournamentTeam, through='MatchParticipant')
...
class MatchParticipant(models.Model):
match = models.ForeignKey(Match, on_delete=models.SET_NULL, null=True, blank=True)
team = models.ForeignKey(TournamentTeam, on_delete=models.SET_NULL, null=True, blank=True)
views.py
queryset = Match.objects.prefetch_related(
Prefetch(
'participant',
queryset=MatchParticipant.objects.select_related(
'match','team'
),
),
).select_related('some_foreignkey',).get(slug=slug)
From django official documentation:
prefetch_related, on the other hand, does a separate lookup for each
relationship, and does the ‘joining’ in Python. This allows it to
prefetch many-to-many and many-to-one objects, which cannot be done
using select_related, in addition to the foreign key and one-to-one
relationships that are supported by select_related.
That's why you get 2 queries being made to your database. That being said, doing so makes you sure you won't get into a N+1 query problem for that query.
Going further
Some profiling/debug tools (ex: Django debug toolbar) can help you troubleshoot complicated queries on your page before it's too late. Also, if you're a pytest-django user, you might want to have a look at django_assert_num_queries. With "native" django test case, have a look at assertNumQueries.
I have a simple notification model:
class Notification(models.Model):
user = models.ForeignKey(User)
sender = models.ForeignKey(User)
model = '''What to put here?'''
comment = models.CharField(max_length=200)
created = models.DateTimeField(auto_now=False,auto_now_add=True)
I need the notification to relate to several different models, for example; posts, user follows, etc
Is there anyway in django you can relate to several models instead of creating a notification model for each one?
I want to avoid models like this:
PostLikeNotification, UserFollowNotification, etc.
So does django have this functionality? I couldn't find it anywhere in the docs.
You could use Content Types/Generic Relations
class Notification(models.Model):
user = models.ForeignKey(User)
sender = models.ForeignKey(User)
object_id = models.PositiveIntegerField(default=None, null=True)
content_type = models.ForeignKey(ContentType, default=None, null=True)
comment = models.CharField(max_length=200)
created = models.DateTimeField(auto_now=False,auto_now_add=True)
#property
def model_object(self):
content_type = self.content_type
object_id = self.object_id
if content_type is not None and object_id is not None:
MyClass = content_type.model_class()
model_object = MyClass.objects.filter(pk=object_id)
if model_object.exists():
return model_object.first()
return None
Here we are storing the Model (Using the Content Types framework) and Primary Key (must be an Integer in this example) of the related object in the Notification model, then adding a property method to fetch the related object.
With this you can relate your notifications to any other model. You could also use the ForeignKey.limit_choices_to argument on the content_type field to validate that it only accepts certain models.
Django need to know the model before creating a relation, you can store the model in char field like post:23 user_follow:41 and define a get_model method that will parse that field and return the right model object
All depends on your design, you have several options. Different options depend on the size of your database:
How many notifications are there?
Do you need to update the notifications often?
Or most of the notifications are inserted once and then read often?
Use an abstract model
Use an abstract model and actually create the PostLikeNotification and UserFollowNotification and other models of such a kind.
class Notification(models.Model):
# ...
class Meta:
abstract = True
class PostLikeNotification(Notification):
model = models.ForeignKey(SomePost)
class UserFollowNotification(Notifcation):
model = models.ForeignKey(Follower)
# ...
This has several advantages:
You keep your relations in your (relational) database.
You have strong foreign keys to prevent inconsistent data.
It is "Djangoic": relations in the database, starting with a normalised database, and no early optimisations are django's way of doing things.
And, of course, this has some disadvantages:
If you need to search all notifications for something the query will be complex.
Moreover, a query over all notifications will be slow, since it filters several tables.
Use a CharField
You can use a simple CharField and store in it the model name and id. Or two fields one for the name and another for the id.
class Notification(models.Model):
model_type = models.CharField(max_len=48)
model_id = models.PositiveInteger()
Advantages:
You have a single table, querying is faster if you have the right indexes.
You can get one of the types of notifications with a simple comparison (index model_type for extra speed).
Disadvantages:
Inconsistent data may appear.
You will need to add extra code at a higher level to deal with possible inconsistent data.
Parallel writes (that may need to lock the entire table) may be a problem.
The middle ground, use several foreign keys
This is just one way of implementing a middle ground between the two options below: You add several nullable foreign keys. Other ways of achieving middle ground exist.
class Notification(models.Model):
model_post = models.ForeignKey(SomePost, null=True, blank=True)
model_follow = models.ForeignKey(Follower, null=True, blank=True)
Advantage:
Verification of inconsistent data can be made without searching other tables (foreign keys are foreign keys, the database takes care of their consistency).
Disadvantage:
It has most of the disadvantages of the other two methods but to a lesser extent (at least in most of them).
Conclusion
If you're just starting a project, and you do not know (or are not worried) about the volume of data then do create several tables. Abstract models were created for this purpose.
On the other hand if you have a lot of notifications to be read and filtered (by a lot, I mean millions) then you have good reasons to create a single notification table and process the relations at a higher level. Note that this incurs locking problems, you shall (almost) never lock notifications if you have a single table.
I've read many posts on what OneToOneField, ManyToManyField, and Foreign Key are but they aren't very clear. I am very new to Django and python programming, currently trying to develop models. Can someone explain to me in simple language, preferably with example, what they each are?
Imagine a database, which stores your book collection:
from django.db import models
class Place(models.Model):
address = models.CharField(max_length=50)
country = models.CharField(max_length=50)
class Publisher(models.Model):
name = models.CharField(max_length=30)
place = models.OneToOneField(Place, primary_key=True)
class Author(models.Model):
first_name = models.CharField(max_length=30)
last_name = models.CharField(max_length=40)
class Book(models.Model):
title = models.CharField(max_length=100)
publisher = models.ForeignKey(Publisher)
authors = models.ManyToManyField(Author)
One-to-many/Foreign Key
Every Book has one Publisher, but a Publisher might have published multiple books. Therefore they are in a one-to-many (book-to-publisher) relationship.
One-to-one
Every Publisher is located in one Place, and every Place can only hold one Publisher. Therefore they are in a one-to-one relationship. You could just have well have put the Place information (address and country) with the Publisher model in one table, but sometimes it is preferred to have seperate models. For example, if you do not know the Place for every Publisher, you don't need to take up a lot of space with empty rows.
Many-to-many
Every Book also has one or more Authors. However, an Author might have written multiple Books, so they are in a many-to-many relationship.
If you still need some guidance, I suggest taking a look at the model chapter of the Django Book.
I have this following (simplified) model in Django which is very similar to the Pinterest data model:
class UserProfile(models.Model):
user = models.OneToOneField(User)
class Collection(models.Model):
owner = models.ForeignKey(User,related_name='collection_owner')
followers = models.ManyToManyField(User, related_name='collection_followers', null=True, blank=True, default = None)
class Item(models.Model):
collections = models.ManyToManyField(Collection,blank=True,null=True)
I have a User model, a UserProfile model that maps 1-1 with a user, a Collection model which has an owner and followers and items that can be part of multiple collections. I'm struggling with determining how to execute the following queries in Django:
Get all the followers of a given user. The definition of a follower is one that follows at least one collection that is owned by that particular user.
Get all the distinct items of the collection a user follows.
I'm not sure if I can do these in a single query or do I have to break it up in several queries? What would be the best approach and are there any trade offs?
Thanks for any help.
The first would be something like this I think:
User.objects.filter(collection_owner__owner='the user')
The latter should be something like this:
Item.objects.filter(collections__followers='the user').distinct()
You should be aware however that these type of queries do not scale to large amounts of data. Doing that will require quite a bit of hacking...
I have a model Model with a m2m field :
user = .. fk user
...
watchers = models.ManyToManyField(User, related_name="boardShot_watchers", null=True)
How do i select all distinct Users involved in this watchers relationship for all my entries of type Model ?
I dont think there is an ORM way to access to intermediary M2M table.
Greg
Not in your current model. If you want to have explicit access to the joining table, you need to make it part of the Django object model. The docs explain how to do this:
http://www.djangoproject.com/documentation/models/m2m_intermediary/
The admin and other django.contrib* components can be configured to treat most fields the same as if they were just model.ManyToMany's. But it will take a little config.