I have two models:
class Account(models.Model):
...
class Transaction(models.Model):
....
account = models.ForeignKey(Account)
source_account = models.ForeignKey(Account, null=True)
I need to display the number of transactions for each of a user's accounts. Django's annotate seemed like the proper tool for this task. I did:
queryset = models.Account.objects.filter(user=self.request.user)
queryset.annotate(transactions_count=Count('transaction'))
This gives the correct number for transactions with account field set to the predicate account but leaves out transactions where source_account is set to the predicate account.
Using the Django shell I am able to do something like:
accounts_count = user_transactions.filter(Q(account=account)|Q(source_account=account)).count()
This gives the correct answer. Is there something I am doing wrong? Can someone point me in the correct direction. Any assistance is highly appreciated.
I would set related_name to your ForeignKey fields. Then it's a bit easier to work with them. So for example in your models let's set:
class Transaction(models.Model):
...
account = models.ForeignKey(Account, related_name='transactions')
source_account = models.ForeignKey(Account, null=True, related_name='source_transactions')
then can do something like:
queryset = models.Account.objects.filter(user=self.request.user).annotate(transactions_count=(Count('transactions')+Count('source_transactions'))
it would work without the naming too, it's just more readable and easier. The main point is adding the two Count as one field in annotate.
The best approach for these types of problems is to imagine them in raw SQL and then try to mimic it in Django ORM.
(in raw sql you would also simply just add two columns like SELECT (a.col + a.col2) AS count
The problem is that your transaction has to ForgeinKeys to Account. I would suggest trying something like this
class Transaction(models.Model):
....
account = models.ForeignKey(Account, related_name="transaction_account")
source_account = models.ForeignKey(Account, null=True, related_name="transaction_source_account")
Then in your query:
queryset.annotate(transactions_count=((Count('transaction_account') + Count('transaction_source_account'))
Related
Django newbie here!
I am coming from .NET background I am frustrated as to how to do the following simple thing:
My simplified models are as follows
class Circle(BaseClass):
name = models.CharField("Name", max_length=2048, blank=False, null=False)
active = models.BooleanField(default=False)
...
class CircleParticipant(BaseClass):
circle = models.ForeignKey(Circle, on_delete=models.CASCADE, null=True, blank=True)
user = models.ForeignKey(User, on_delete=models.SET_NULL, null=True, blank=True)
status = models.CharField("Status", max_length=256, blank=False, null=False)
...
class User(AbstractBaseUser, PermissionsMixin):
email = models.EmailField(verbose_name="Email", unique=True, max_length=255, validators=[email_validator])
first_name = models.CharField(verbose_name="First name", max_length=30, default="first")
last_name = models.CharField(verbose_name="Last name", max_length=30, default="last")
...
My goal is to get a single circle with participants that include the users as well. With the extra requirement to do all that in a single DB trip.
in SQL terms I want to accomplish this:
SELECT circle.name, circle.active, circle_participant.status, user.email. user.first_name. user.last_name
FROM circle
JOIN circle_participant on circle.id = circle_participant.id
JOIN user on user.id = circle_participant.id
WHERE circle.id = 43
I've tried the following:
Circle.objects.filter(id=43) \
.prefetch_related(Prefetch('circleparticipant_set', queryset=CircleParticipant.objects.prefetch_related('user')))
This is supposed to be working but when I check the query property on that statement it returns
SELECT "circle"."id", "circle"."created", "circle"."updated", "circle"."name", "circle"."active", FROM "circle" WHERE "circle"."id" = 43
(additional fields omitted for brevity.)
Am I missing something or is the query property incorrect?
More importantly how can I achieve fetching all that data with a single DB trip.
For reference here's how to do it in .NET Entity Framework
dbContext.Circle
.Filter(x => x.id == 43)
.Include(x => x.CircleParticipants) // This will exist in the entity/model
.ThenInclude(x => x.User)
.prefetch_related will use a second query to reduce the bandwidth, otherwise it will repeat data for the same Circle and CircleParticipants multiple times. Your CircleParticipant however acts as a junction table, so you can use:
Circle.objects.filter(id=43).prefetch_related(
Prefetch('circleparticipant_set', queryset=CircleParticipant.objects.select_related('user')
)
)
Am I missing something or is the query property incorrect?
There are two ways that Django gives you to solve the SELECT N+1 problem. The first is prefetch_related(), which creates two queries, and joins the result in memory. The second is select_related(), which creates a join, but has a few more restrictions. (You also haven't set related_name on any of your foriegn keys. IIRC that is required before using select_related().)
More importantly how can I achieve fetching all that data with a single DB trip.
I would suggest that you not worry too much about doing it all in one query. One of the downsides of doing this in one query as you suggest is that lots of the data that comes back will be redundant. For example, the circle.name column will be the same for every row in the table which is returned.
You should absolutely care about how many queries you do - but only to the extent that you avoid a SELECT N+1 problem. If you're doing one query for each model class involved, that's pretty good.
If you care strongly about SQL performance, I also recommend the tool Django Debug Toolbar, which can show you the number of queries, the exact SQL, and the time taken by each.
in SQL terms I want to accomplish this:
There are a few ways you could accomplish that.
Use many-to-many
Django has a field which can be used to create a many-to-many relationship. It's called ManyToManyField. It will implicitly create a many-to-many table to represent the relationship, and some helper methods to allow you to easily query for all circles a user is in, or all users that a circle has.
You're also attaching some metadata to each user/circle relationship. That means you'll need to define an explicit table using ManyToManyField.through.
There are examples in the docs here.
Use a related model query
If I specifically wanted a join, and not a subquery, I would query the users like this:
Users.objects.filter(circleparticipant_set__circle_id=43)
Use a subquery
This also creates only one query, but it uses a subquery instead.
Users.objects.filter(circleparticipant_set=CircleParticipant.objects.filter(circle_id=43))
Suppose I have the two models below and I want to get a queryset of all developers that have games where the platform field matches a certain value. How would I go about that?
class Developer(models.Model):
name = models.CharField(max_length=100, default="Unknown")
class Game(models.Model):
name = models.CharField(max_length=300)
developer = models.ForeignKey(Developer, related_name="games", on_delete=models.CASCADE)
platform = models.CharField(max_length=40)
I tried a few approached but can't seem to figure anything out that works.
You can query this with:
Developer.objects.filter(games__platform='name-of-platform').distinct()
Without the .distinct() [Django-doc], the same developer will be returned multiple times, if they developed multiple Games for the same platform. If that is not a problem, you can of course omit the .distinct().
I have models defined as follow:
class Employee(models.Model):
...
user = models.OneToOneField(User, on_delete=models.CASCADE, primary_key=True)
class Project(models.Model):
...
employees = models.ManyToManyField(Employee, null=True, blank=True)
I'm trying to retrieve all the projects that have at least one employee assigned to them, but I don't know how. I tried the following things:
projects.filter(employees__gt=0)
where projects = Project.objects.all() but I don't think this is the right query, because if I do projects.filter(employees_lte=0) it returns nothing, even if I have projects with no employees assigned. How can I retrieve what I'm looking for? Could you point to a page where I can find all the lookups I can use?
Thanks!
You can try like this using isnull:
Project.objects.filter(employees__isnull=False)
Update
If you want to check specific number of employees, maybe try like this
from django.db.models import Count
Project.objects.annotate(employee_count=Count('employees')).filter(employee_count__gt=5)
In my project I have to deal with different type of users e.g. costumers and employers. Each type of user have its own fields and permissions: costumer can buy things whereas employer can not.
I have read the Django docs and it looks like there are two options:
Costumize the AbstractUser class and adding all the fields of costumers and employers. Then use the permission system to grant/revoke permission or create a group for each type of user. The downside here is that there unused fields.
Adopt the proxy model:
from django.contrib.auth.models import User
from django.db import models
class Costumer(models.Model):
user = models.OneToOneField(User, on_delete=models.CASCADE)
class Meta:
db_table = 'costumers'
permissions = (
("access_this", "User may access this"),
)
ordering = []
class Employee(models.Model):
user = models.OneToOneField(User, on_delete=models.CASCADE)
class Meta:
db_table = 'employers'
permissions = (
("access_that", "User may access that"),
)
ordering = []
This case seems more reasonable but I don't know how to deal with the permissions. Consider I'd like to use #permission_required instead of checking the type (if the user has a specific field) because it seems more legit for Django system.
So in the end, what is the best way to approach such scenario?
The first solution is much better.
The downside here is that there unused fields.
I disagree with this, you don't have to store all the fields within the User model. Also, if you're talking about 5 fields for example, that doesn't really matter.
You can extend AbtractUser and also use some composition; you don't have to put all the fields there:
class User(AbstractUser):
email = ...
...
# for the employer, let's say you want to save the company details
company = models.ForeignKey('myapp.Company', null=True, blank=True)
...
# for the customer
customer_details = models.ForeignKey('...')
This way you could record a user_type if you want or deduce the type from the foreign key (if there is a company, it's an employer).
To help you more with the model, I need to know what differentiate an employer from a customer in your application. Note that with that solution, an instance of User could be both.
Concerning the permissions, I feel like it's a separate problem; I'd recommend you to sort it last. If the design you pick is close to the reality and you get the features working; adding custom permissions will be really easy.
So, I have the following models:
class Band(models.Model):
name = models.CharField(max_length=50)
class Contract(models.Model):
band = models.ForeignKey(Band)
when = models.DateTimeField(auto_now_add=True)
salary = models.IntegerField()
class Musician(models.Model):
first_name = models.CharField(max_length=50)
last_name = models.CharField(max_length=50)
instrument = models.CharField(max_length=100)
bands = models.ManyToManyField(Band, through=Contract)
class Album(models.Model):
artist = models.ForeignKey(Musician)
name = models.CharField(max_length=100)
release_date = models.DateField()
num_stars = models.IntegerField()
So, I wanted to expose that on the admin page. So far, so good.
Note that our musicians here keep jumping in and out from bands. Some say one of them even had been on over 2 millions bands in his life-time. I don't know, maybe the bands are Whitesnake, Metallica or something.
How should we do that on the Django Admin Page?
I tried using raw_id_fields and apart the fact I didn't like the effect, it didn't work so well. It took a lot of time to load and it didn't let me add more ids. Weird.
I've used admin.StackedInline with no luck cause it will try to load every contract in a which, well, it's gonna take only 2 thousand years.
When Musician had a direct relation to Band it worked just fine with this library. But now that the relation isn't an straight one. Looks like autocomplete doesn't support it(it was getting slow anyway).
So, with all of this, I ask you lord SO members. What's the best way to do this? Is it autocomplete? Someone must have had to come across this issue!
Thanks in advance.
To avoid loading every bands in your admin page use autocomplete_fields Django doc.
Just use it like that in your admin.py.
autocomplete_fields = ('bands',)
Then no bands will be pulled from DB to front, but you will be able to select it through a Select2 search field and it will be printed as "tags".
I found this solution and hope it will help somebody in the same situation:
I have many to many relations between the Product and Characteristic model.
So, in the admin.py I am setting a form for a Product like the following where catch/get all the Characteristics and make the "prefecth_related" for Characteristic, as well the "select_related" could be done there:
class ProductAdminForm(forms.ModelForm):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.fields['characteristics'].queryset = Characteristic.objects.prefetch_related('category').all()