I have a model ProductFilter in the project. An object of this model is a set of lookups defining a queryset. It's like a group of products - but just as a filter, not connected with a ForeignKey (ManyToManyField).
The product can be filtered in multiple ProductFilters and the first is taken to account.
It works like this:
User define their filters. Then assign to each filter some pricing (and other) rules.
For example filter with the name "Products under 10 USD" - rule - increase the price by 10%.
The problem is that when I render products or export them. I have to check for every product if they belong to any filter which means a lot of DB queries. There can be 100k products.
Do you have any ideas how to make it faster?
class Product(...):
...
#property
def price(self):
for filter in self.user.filters():
if filter.get_queryset().filter(pk=self.pk).exists():
return filter.rule.apply_rule(self.price)
class ProductFilter(BaseTimeStampedModel):
project = models.ForeignKey('projects.UserEshop', related_name='product_filters', on_delete=models.CASCADE)
order = models.IntegerField(default=0)
name = models.CharField(max_length=64)
categories = models.ManyToManyField('FeedCategory', blank=True)
brands = models.ManyToManyField('Brand', blank=True)
manufacturers = models.ManyToManyField('Manufacturer', blank=True)
import_price_no_vat_max = models.DecimalField(decimal_places=2, max_digits=64, null=True, blank=True)
import_price_no_vat_min = models.DecimalField(decimal_places=2, max_digits=64, null=True, blank=True)
name_icontains = models.CharField(max_length=128, null=True, blank=True)
def get_queryset(self):
# products for this project
qs = Product.objects.filter(source__user_eshop=self.project)
if self.categories.exists():
qs = qs.filter(category__in=self.categories.all().get_descendants(include_self=True))
if self.brands.exists():
qs = qs.filter(brand__in=self.brands.all())
if self.manufacturers.exists():
qs = qs.filter(manufacturer__in=self.manufacturers.all())
if self.import_price_no_vat_max is not None:
qs = qs.filter(import_price_no_vat__lte=self.import_price_no_vat_max)
if self.import_price_no_vat_min is not None:
qs = qs.filter(import_price_no_vat__gte=self.import_price_no_vat_min)
if self.name_icontains:
qs = qs.filter(name__icontains=self.name_icontains)
return qs
EDIT
My best idea is to create a ManyToManyField on ProductFilter to Product and everytime somethings changes I'll assign products to every filter. I'm just not sure if ManyToManyField is ok to hold thousands or hundreds of thousands products.
Related
So, I am learning Django and trying to make a site similar to AirBNB.
I have models called lisitngs that has latitude and longitude stored in CharField. My model is as follows:
class Listing(models.Model):
class BathRoomType(models.TextChoices):
ATTACHED = 'Attached Bathroom'
COMMON = 'Shared Bathroom'
class RoomVentType(models.TextChoices):
AC = 'Air Conditioner'
NO_AC = 'No Air Conditioner'
class LisitngType(models.TextChoices):
ROOM = 'Room'
APARTEMENT = 'Apartement'
HOUSE = 'Full House'
user = models.ForeignKey(User, on_delete=models.CASCADE)
title = models.CharField(max_length=255)
city = models.ForeignKey(RoomLocation, on_delete=models.CASCADE)
exact_address = models.CharField(max_length=255)
lat = models.CharField(max_length=300, blank=False, null=False, default="0")
lng = models.CharField(max_length=300, blank=False, null=False, default="0")
description = models.TextField()
price = models.IntegerField()
listing_type = models.CharField(max_length=20, choices=LisitngType.choices, default=LisitngType.ROOM)
kitchen_available = models.BooleanField(default=False)
kitchen_description = models.TextField(null=True, blank=True)
bedrooms = models.IntegerField()
max_acomodation = models.IntegerField()
bathroom_type = models.CharField(max_length=20, choices=BathRoomType.choices, default=BathRoomType.ATTACHED)
no_bathrooms = models.IntegerField()
room_type = models.CharField(max_length=30, choices=RoomVentType.choices, default=RoomVentType.AC)
main_photo = models.ImageField(upload_to='room_images', default='default_room.jpg')
photo_1 = models.ImageField(upload_to='room_images', default='default_room.jpg')
photo_2 = models.ImageField(upload_to='room_images', default='default_room.jpg')
photo_3 = models.ImageField(upload_to='room_images', default='default_room.jpg')
is_published = models.BooleanField(default=False)
date_created = models.DateTimeField(default=timezone.now, editable=False)
slug = AutoSlugField(populate_from=['title', 'listing_type', 'bathroom_type', 'room_type'])
rating = models.IntegerField(default=5)
approved = models.BooleanField(default=False)
total_bookings = models.IntegerField(default=0)
def __str__(self):
return self.title
In my homepage what I want to do is show the listings which are nearby me.
For that I have a function named as near_places. This near_place function takes latitude and longitude after querying through the model Listing and returns the distance between the listing and current user accessing the homepage:
import geocoder
from haversine import haversine
def near_places(dest_lat, dest_lng):
g = geocoder.ip('me')
origin = tuple(g.latlng)
destination = (dest_lat, dest_lng)
distance = haversine(origin, destination)
return distance
My homepage function in views.py is as follows:
def home(request):
objects = Listing.objects.filter(is_published=True, approved=True)
for object in objects:
lat, lng = float(object.lat), float(object.lng)
object.distance = near_places(lat, lng)
return render(request, 'listings/home.html')
As you can see I have looped through the query set and for each data I have calculated the distance and appended in the queryset as distance. Now, I would like to only get 10 items that has lowest distance. How, can I do so.
I have tried to user object = objects.order_by('-distance')[:10] but it gives me error as
FieldError at /
Cannot resolve keyword 'distance' into field. Choices are: The_room_booked, approved, bathroom_type, bedrooms, city, city_id, date_created, description, exact_address, id, is_published, kitchen_available, kitchen_description, lat, listing_type, lng, main_photo, max_acomodation, no_bathrooms, photo_1, photo_2, photo_3, price, rating, reviewsandrating, room_type, slug, title, total_bookings, user, user_id
Any way that I can solve it?
Also it takes quite a time to calculate current distance using near_places() function as above. Any suggestions will be helpful.
Thank You
You can't do that, because your model doesn't have a distance field and there is no such DB column as well.
What you can do is either
add such field to your model - I don't recommend with your current logic as you will iterate over every instance and send sql request to update every row.
get your queryest, convert it to a list of dicts and then iterate over your list of dicts with your function adding the distance key to it. Then you can sort the list by the python sort function and pass it to the template.
Like:
objects = Listing.objects.filter(is_published=True, approved=True).values('lat', 'lng') # add fields that you need
for obj in objects:
obj['distance'] = near_places(obj['lat'], obj['lng'])
my_sorted_list = sorted(objects, key=lambda k: k['distance'])
Pass my_sorted_list to your template. You can add reverse=True arg to sorted function if you want another direction sorting.
I’m a python/django begginer. I decided to build a e-commerce website using django for an academic project. I’ve been able to develop enough of the project and build understanding of the subject, but right now I’m having issues finding a way to subtracts the number of items listed in the inventory whenever a order is made.
That’s the code for the models, evey product has it's own stock quantity call inventory:
class Product(models.Model):
name = models.CharField(max_length=200, null=True)
price = models.FloatField()
description = models.TextField(default='', null=True, blank=True)
digital = models.BooleanField(default=False,null=True, blank=True)
image = models.ImageField(null=True, blank=True)
inventory = models.IntegerField(default=0)
def __str__(self):
return self.name
def has_inventory(self):
return self.inventory > 0
This is the code I made to subtract base on quantity of the item ordered, but I can’t make it work, it won’t subtract the number of items from the inventory on the product stock.
class OrderItem(models.Model):
product = models.ForeignKey(Product, on_delete=models.SET_NULL, null=True)
order = models.ForeignKey(Order, on_delete=models.SET_NULL, null=True)
quantity = models.IntegerField(default=0, null=True, blank=True)
date_added = models.DateTimeField(auto_now_add=True)
def __str__(self):
return str(self.product) + " x " + str(self.quantity)
def inventory(self):
product.inventory = self.inventory
product.inventory -= int(self.quantity)
return inventory
What could I do to make it work?
All logic/action should be written under views.py file. You could create a function where it takes in a request, and when it does, it takes in all the value inputted through a form, and you could use filter to filter out the products you want to subtract its inventory and update query by Django to update the inventory.
It should look something like this inside your views function:
Product.objects.filter(name = name, description = description, digital = digital).update(Inventory = F('Inventory')-inventory)
Here is Django's documentation on queries: Django's Making Queries
I think there are a few problems with the snippet above.
First, the OrderItem.inventory is not referring the right value, it should be like the snippet below.
def inventory(self):
// remember the current stock is stored on Product.inventory
return self.product.inventory - self.quantity
Second, The method name should be remaining_stock not inventory to prevent misunderstanding.
def remaining_stock(self):
return self.product.inventory - self.quantity
Also, don't forget if you want to store inventory of the product please call the save method after successfully inserting the OrderItem.
It's an eCom site that I am working on. I need to filter results based on email, as in obtain all those orders placed by this email account. On executing the query OrderItem.objects.filter(order__email='john#john.com').select_related('order') I get queryset object for each product. So if an order had multiple products then each product is a set on it's own. What I need is all items combined in the one set for an order ID so that I can iterate through them in the template.
OrderItem.objects.filter(order__email='john#john.com').select_related('order').values('order_id').annotate(ct=Count('pk')) - This query gets all those IDs and shows how many sets are there per ID. Likewise I just need the results but with other fields like created_date, order_id, transaction_id, product.
I have explored the django documentation but don't seem to find an appropriate solution. Another option is to manually loop through each item in the queryset and segregate based on the order_id having multiple items but then again that doesn't seem efficient. So any thoughts on how to proceed would be great.
Models are below:
class Order(models.Model):
payment_date = models.DateTimeField(blank=True, null=True)
created_date = models.DateTimeField(auto_now_add=True)
transaction_id = models.CharField(max_length=256, blank=True, null=True)
email = models.EmailField(max_length=128, blank=True, null=True)
def __str__(self):
return "Order #{},Date:{}".format(self.id,self.created_date)
class OrderItem(models.Model):
product = models.ForeignKey(PRODUCT_VARIANT_MODEL, on_delete=models.DO_NOTHING)
quantity = models.IntegerField(default=1)
order = models.ForeignKey(Order, related_name='items', on_delete=models.CASCADE)
def __str__(self):
return "Quantity:{},Item:{},{}".format(self.quantity, self.product.get_product_title(),self.order)
Result of query 1:(There are 2 orders placed by this email ID) OrderItem.objects.filter(order__email='john#john.com').select_related('order')
<QuerySet [<OrderItem: Quantity:3,Item:sampleitem 1,Order #21,Date:2019-06-07 06:14:22.929052+00:00>, <OrderItem: Quantity:1,Item:sampleitem 91,Order #22,Date:2019-06-24 05:33:16.509479+00:00>, <OrderItem: Quantity:2,Item:sampleitem 44,Order
#22,Date:2019-06-24 05:33:16.509479+00:00>, <OrderItem: Quantity:1,Item:sampleitem 8,Order #22,Date:2019-06-24 05:33:16.509479+00:00>]>
Result of query 2:
OrderItem.objects.filter(order__email='john#john.com').select_related('order').values('order_id').annotate(ct=Count('pk'))
<QuerySet [{'order_id': 21, 'ct': 1}, {'order_id': 22, 'ct': 3}]>
So i have two models:
class Product(models.Model):
name = models.CharField(_('Name'), max_length=200)
class Like(models.Model):
product = models.ForeignKey(Product, related_name='likes', on_delete=models.DO_NOTHING)
user = models.ForeignKey(settings.AUTH_USER_MODEL, blank=True, null=True, related_name='likes', on_delete=models.DO_NOTHING)
What I want is to get the total number of base products (<5 likes), standard products (<10 likes) and premium products.
I don't know how I should count FK of each product and then group them; and preferably in 1 query.
python 3.6, django 2.1, posgtresql
I don't understand how that should be acomplished in a single query, but you could try this:
from django.db.models import Count
qs = Product.objects.annotate(num_likes=Count('like'))
base_products = qs.filter(num_likes__lt=5)
standard_products = qs.filter(num_likes__gte=5, num_likes__lt=10)
premium_products = qs.filter(num_likes__gte=10)
This code gives you 3 QuerySet objects with the data you asked for.
If you want a single database query (it still includes both tables in the query) but don't mind loading the data in python lists instead of querysets (needs more memory and maybe even more time), then you could try:
from django.db.models import Count
qs = Product.objects.annotate(num_likes=Count('like'))
base_products = []
standard_products = []
premium_products = []
for p in qs:
if p.num_likes < 5:
base_products.append(p)
elif p.num_likes < 5:
standard_products.append(p)
else:
premium_products.append(p)
The SQL I want to accomplish is this -
SELECT jobmst_id, jobmst_name, jobdtl_cmd, jobdtl_params FROM jobmst
INNER JOIN jobdtl ON jobmst.jobdtl_id = jobdtl.jobdtl_id
WHERE jobmst_id = 3296
I've only had success once with an inner join in django off of a annote and order_by but I can't seem to get it to work doing either prefetch_related() or select_related()
My models are as so -
class Jobdtl(models.Model):
jobdtl_id = models.IntegerField(primary_key=True)
jobdtl_cmd = models.TextField(blank=True)
jobdtl_fromdt = models.DateTimeField(blank=True, null=True)
jobdtl_untildt = models.DateTimeField(blank=True, null=True)
jobdtl_fromtm = models.DateTimeField(blank=True, null=True)
jobdtl_untiltm = models.DateTimeField(blank=True, null=True)
jobdtl_priority = models.SmallIntegerField(blank=True, null=True)
jobdtl_params = models.TextField(blank=True) # This field type is a guess.
class Meta:
managed = False
db_table = 'jobdtl'
class Jobmst(MPTTModel):
jobmst_id = models.IntegerField(primary_key=True)
jobmst_type = models.SmallIntegerField()
jobmst_prntid = TreeForeignKey('self', null=True, blank=True, related_name='children', db_column='jobmst_prntid')
jobmst_name = models.TextField(db_column='jobmst_name', blank=True)
# jobmst_owner = models.IntegerField(blank=True, null=True)
jobmst_owner = models.ForeignKey('Owner', db_column='jobmst_owner', related_name = 'Jobmst_Jobmst_owner', blank=True, null=True)
jobmst_crttm = models.DateTimeField()
jobdtl_id = models.ForeignKey('Jobdtl', db_column='jobdtl_id', blank=True, null=True)
jobmst_prntname = models.TextField(blank=True)
class MPTTMeta:
order_insertion_by = ['jobmst_id']
class Meta:
managed = True
db_table = 'jobmst'
I have a really simple view like so -
# Test Query with Join
def test_queryjoin(request):
queryset = Jobmst.objects.filter(jobmst_id=3296).order_by('jobdtl_id')
queryresults = serializers.serialize("python", queryset, fields=('jobmst_prntid', 'jobmst_id', 'jobmst_prntname', 'jobmst_name', 'jobmst_owner', 'jobdtl_cmd', 'jobdtl_params'))
t = get_template('test_queryjoin.html')
html = t.render(Context({'query_output': queryresults}))
return HttpResponse(html)
I've tried doing a bunch of things -
queryset = Jobmst.objects.all().prefetch_related()
queryset = Jobmst.objects.all().select_related()
queryset = jobmst.objects.filter(jobmst_id=3296).order_by('jobdtl_id')
a few others as well I forget.
Each time the json I'm getting is only from the jobmst table with no mention of the jobdtl results which I want. If I go the other way and do Jobdtl.objects.xxxxxxxxx same thing it's not giving me the results from the other model.
To recap I want to display fields from both tables where a certain clause is met.
What gives?
Seems that I was constantly looking in the wrong place. Coming from SQL I kept thinking in terms of inner joining tables which is not how this works. I'm joining the results from models.
Hence, rethinking my search I came across itertools and the chain function.
I now have 2 queries under a def in my views.py
from itertools import chain
jobmstquery = Jobmst.objects.filter(jobmst_id=3296)
jobdtlquery = Jobdtl.objects.filter(jobdtl_id=3296)
queryset = chain(jobmstquery, jobdtlquery)
queryresults = serializers.serialize("python", queryset)
That shows me the results from each table "joined" like I would want in SQL. Now I can focus on filtering down the results to give me what I want.
Remember folks, the information you need is almost always there, it's just a matter of knowing how to look for it :)
What you are looking for might be this
queryset = Jobmst.objects.filter(id=3296).values_list(
'id', 'name', 'jobmst_owner__cmd', 'jobmst_owner__params')
You would get your results with only one query and you should be able to use sort with this.
P.S. Coming from SQL you might find some great insights playing with queryset.query (the SQL generated by django) in a django shell.