Django admin runs query multiple times - python

I have a function defined in list_items in django admin as follows:
def won(self):
result = (Result.objects.filter(user=self)
.select_related('user')
.aggregate(Sum('prize'))['prize__sum'])
return result if result else 0
A result contains a foreign key to user, as well as a 'prize', defining the amount of points a user earned.
Using Django Debug Toolbar, I've discovered that each user being listed results in 40 identical queries being run to calculate the 'won' value for each user.
SELECT SUM("games_result"."prize") AS "prize__sum" FROM "games_result" WHERE "games_result"."user_id" = 28297
Duplicated 40 times.
Am I doing something really obviously wrong to cause this problem? Is there a better way of getting the Result objects with a link to this user?

Considering your user has a field named results, which represents all of the results from the games for a certain user you can use annotate aggregation function to add a field to each user and then on every user you can simpy select the prize_sum as in this example:
class UserAdmin(admin.ModelAdmin):
list_display = ('won', )
def won(self):
return self.prize_sum
def get_queryset(self, request):
return super(UserAdmin, self).get_queryset(
request).select_related('results').annotate(prize_sum=Sum('results__prize'))

Related

DJANGO:How to perform AND operation for my query?

There are two models .I want to make query to extract only the app exact app related Adspaces .
models.py
class Appname(models.Model):
user=models.ForeignKey(User,related_name='appname', null=True, default=None,on_delete=models.CASCADE)
name=models.CharField(max_length=150,blank=False,null=False,help_text='Add your new App')
def __str__(self):
return self.name
def get_absolute_url(self):
return reverse("dashapp:space",kwargs={'pk':self.pk})
class Adspace(models.Model):
user=models.ForeignKey(User,related_name='adspace', null=True, default=None,on_delete=models.CASCADE)
ad_space=models.CharField(max_length=150,blank=False,null=False)
app=models.ForeignKey('Appname', related_name='appnames',default=None, on_delete=models.CASCADE)
PID_TYPE = (
('FN','FORMAT_NATIVE'),
('FNB','FORMAT_NATIVE_BANNER'),
('FI','FORMAT_INTERSTITIAL'),
('FB','FORMAT_BANNER'),
('FMR','FORMAT_MEDIUM,RECT'),
('FRV','FORMAT_REWARDED_VIDEO'),
)
format_type=models.CharField(max_length=3,choices=PID_TYPE,default='FN',blank=False, null=False)
def __str__(self):
return self.ad_space
def get_absolute_url(self):
return reverse("dashapp:create",kwargs={'pk':self.pk})
Views.py
SHowing the one where i need to the query
class spacelist(LoginRequiredMixin,ListView):
model=Adspace
template_name='adspace_list.html'
def get_queryset(self):
query_set=super().get_queryset()
return query_set.filter(user=self.request.user)
Here I need to perform One more query so that EACH APP show their own adspaces when clicked right now every app show every show adspaces.
I have the idea what to do as if i compare app_id then it'll show the exact app related adspaces, but i dont know how to write query for the same as i already have one query present.???
You could try using a Q objects: https://docs.djangoproject.com/en/2.1/topics/db/queries/#complex-lookups-with-q-objects
From what I understand you are trying to filter both on the app_id and the request user at the same time, so you could try look something like this:
from django.db.models import Q
...
def get_queryset(self):
query_set=super().get_queryset()
return query_set.filter(Q(user=self.request.user) & Q(app_id=app_id))
...
This lets you do a single filter with both your requirements at the same time (i.e. retrieve the Adspace instances for a specific user with a specific Appname).
You chain another filter at the end like this:
class spacelist(LoginRequiredMixin,ListView):
model=Adspace
template_name='adspace_list.html'
def get_queryset(self):
query_set = super().get_queryset()
query_set = query_set.filter(user=self.request.user)
app_id = [...]
return query_set.filter(app_id=app_id)
The problem left is to find out what is the app_id coming from. How do you know what is the current app? Several options here.
Option 1: From the request
It can come from the current user: self.request.user.appname.all() but that will give you multiple apps, if the user can only have one app, you should change your model Appname.user to a OneToOneField.
Otherwise, I suggest changing your related_name='appnames' to reflect the multiplicity in the reverse relationship.
Option 2: From the URL
It can come from the URL, your space list view should extract an app_id parameter from the URL where it's defined:
url(r'^(?P<app_id>[0-9]+)/spaces/$', spacelist.as_view(), name='space_list'),
And then in the spacelist view, you would get this parameter like this:
app_id = self.kwargs['app_id']
return query_set.filter(app_id=app_id)
Hope that helps
UPDATE:
Also worth noting that QuerySets are lazy, meaning the result will get evaluated as late as possible by Django. Therefore, when you call:
query_set = query_set.filter(user=self.request.user)
The Django ORM doesn't execute any DB queries yet, and you can chain more filters after that:
query_set = query_set.filter(user=self.request.user)
query_set = query_set.filter(app_id=app_id)
Which behind the scenes is extending the query that will be executed when required. But at this point, no query is actually run. To see the query that will get executed you can print out the query attribute of the QuerySet:
print(query_set.query)
Which should log something like:
SELECT "app_adspace"."user_id" ...
FROM
"app_adspace"
WHERE
"app_adspace"."user_id" = 1234 AND "app_adspace"."app_id" = 5678

How to handle concurrency with django queryset get method?

I'm using django (1.5 with mysql) select_for_update method for fetching data from one model and serve this data to user upon request, but when two user request at simultaneously it returns same data for both of the user, see the sample code below
models.py
class SaveAccessCode(models.Model):
code = models.CharField(max_length=10)
class AccessCode(models.Model):
code = models.CharField(max_length=10)
state = models.CharField(max_length=10, default='OPEN')
views.py
def view(request, code):
# for example code = 234567
acccess_code = AccessCode.objects.select_for_update().filter(
code=code, state='OPEN')
acccess_code.delete()
SaveAccessCode.objects.create(code=code)
return
Concurrent request will generate two records of SaveAccessCode with same code, Please guide me how to handle this scenario in better way
You need to set some flag on the model when doing select_for_update, something like:
qs.first().update(is_locked=True)`
and before that should do select like
qs = self.select_for_update().filter(state='OPEN', is_locked=False).order_by('id')
Then after the user, I presume, has done something with it and saved, set the flag is_locked=False and save.
Also make the fetch_available_code as a #staticmethod.

Can not add a group to a auth user

I want create a user with custom group in django admin.
so I write below code:
from django.contrib.auth.models import User as AuthUser
from django.contrib.auth.models import Group
# these groups have already been created.
class TestGroup(object):
Admin = 'Admin'
Merchant = 'Merchant'
User = 'User'
class Merchant(AuthUser):
def save(self, **kwargs):
super(Merchant, self).save(**kwargs)
for group in Group.objects.all():
print group.name
# way 1
if not self.groups.filter(name=TestGroup.Merchant).exists():
print self.groups.filter(name='Merchant').exists()
g = Group.objects.get(name=TestGroup.Merchant)
g.user_set.add(self)
print self.groups.filter(name='Merchant').exists()
# way 2
if not self.groups.filter(name=TestGroup.Merchant).exists():
g = Group.objects.get(name=TestGroup.Merchant)
self.groups.add(g)
# way 3
if not self.groups.filter(name=TestGroup.Merchant).exists():
g = Group.objects.get(name=TestGroup.Merchant)
self.groups.add(g)
self.save()
I have tried three ways to add a group to a user.But none of them could work.
UPDATE:
You can test by following this steps:
create a group named 'Merchant' at django admin
add my code (add print in way 1 to test), syncdb and so on.
create a Merchant at django admin.
you can see log:
u'Merchant'
False
True
enter the merchant you just created, you can see, the group merchant is not selected(means this user do not beyond to this group).
click save again,
you would still see
u'Merchant'
False
True
add group to merchant fail, very strange.
Thanks for your clarifications. From what I can see your problem is the way you're trying to filter and get the Group. You're checking whether the name equals an object instance: TestGroup.Merchant when it expects a string. Try:
if not self.groups.filter(name='Merchant'):
g = Group.objects.get(name='Merchant')
g.user_set.add(self)
I've also removed the exists part of your if statement because an empty queryset will evaluate to Boolean False anyway so exists is redundant.
EDIT: Following your comment I'm back to the comment I put on your question. You are assigning strings to variables, not creating groups. If the groups are already created the TestGroup class is completely redundant and should be removed.
At your command prompt import django's Group class and try various things until you can select your groups. Change your code to reflect whatever worked at the command line. For example:
for group in Group.objects.all():
print group.name
Then change your code to look up Group.objects.get(name='relevant_name').
I debug into save method and find:
ValueError: "<Merchant: m23533>" needs to have a value for field "user" before this many-to-many relationship can be used.
It is saying this user have not saved to db yet, that's why adding groups fails!
solution:
class Merchant(AuthUser):
def save(self, **kwargs):
super(Merchant, self).save(force_insert=True)
if not self.groups.filter(name=TestGroup.Merchant).exists():
g = Group.objects.get(name=TestGroup.Merchant)
g.user_set.add(self)

How to select and limit the related_name connection in the Peewee ORM?

I'm using Flask with the Peewee ORM in which I have defined two tables like so:
class Ticket(db.Model):
created = DateTimeField(default=datetime.now)
customer_uuid = CharField() # the customer's UUID gotten from App. More info comes from bunq API.
ticket_type = ForeignKeyField(TicketType, related_name='tickets')
active = BooleanField(default=True)
class Assign(db.Model):
created = DateTimeField(default=datetime.now)
ticket = ForeignKeyField(Ticket, related_name='assigned_to')
user = ForeignKeyField(User, related_name='assigned_tickets')
In the Assign table, several users can be assigned to a ticket, but only the last one counts (i.e., if a new user gets assigned, the previous ones should be disregarded). So I select the active tickets using the following:
open_tickets = Ticket.select().where(Ticket.active == True)
I now want to use this loop in my template. With every iteration however, I also want to display the assigned user. But open_ticket[0].assigned_to obviously returns several assignments, and with it several users.
Would anybody know how I can get the latest assigned user for every ticket within a loop?
This worked for me in Sqlite:
q = (Ticket
.select(Ticket, Assign, User)
.join(Assign)
.join(User)
.group_by(Ticket)
.order_by(Ticket.id, Assign.id.desc()))
for ticket in q:
print ticket.id, ticket.assign.user.username

Django admin filter list with custom model manager extra methods

I have an Address model that contains two float fields, lat and lng. I have written a custom model manager with a nearby(lat, lng, distance) method that uses raw SQL to return only Addresses which lie within a certain radius. (GeoDjango appeared to be overkill).
Example call:
Address.objects.nearby(53.3, 13.4, 10) (returns QuerySet)
Now I want to dynamically filter Address objects in the Django admin using this method (ideally letting the user pick a location on a Google map and a max distance using a slider). I have no idea how to achieve that. Can anyone point me in the right direction?
CLARIFICATION
You don't need to write any JavaScript for me, I just want to know how to make Django admin evaluate extra Query parameters in the URL, such that I can do queries like /admin/appname/address/?lat=53&long=13&dist=10. I can then probably figure out how to stuff a Google map and the required JavaScript magic into the template myself.
UPDATE
I've tried to overwrite queryset in the ModelAdmin like so:
def queryset(self, request):
try:
lat = float(request.REQUEST['lat'])
lng = float(request.REQUEST['lng'])
dist = int(request.REQUEST['dist'])
matches = Address.objects.nearby(lat=lat, lng=lng, dist=dist)
return matches
except:
return super(ReportAdmin, self).queryset(request)
However, the admin does not like it and returns with ?e=1, without filtering the results.
I've added this to the ModelAdmin of the object class that has the address as a FK:
def lookup_allowed(self, lookup, *args, **kwargs):
if lookup == 'address__dst':
return True
return super(ReportAdmin, self).lookup_allowed(lookup, args, **kwargs)
I've added this to the model
def _filter_or_exclude(self, negate, *args, **kwargs):
try:
value = kwargs.pop('address__dst')
matches = self.nearby(*map(float, value.split(',')))
pks = [m.pk for m in matches]
kwargs.update({ 'pk__in': pks })
except:
pass
return super(ReportQuerySet, self)._filter_or_exclude(negate, *args, **kwargs)
allowing me to filter like ?address_dst=lat,lng,dst.
Is there a nicer solution?
If you need this functionality on admin list display page you can write a custom filter.
See the solution in: Custom Filter in Django Admin on Django 1.3 or below.

Categories