We Keep Coding

How to filter out duplicate rows from child model before being displayed in Django ListView

I am using two related models in my Django application. The objects so created in the models are being displayed using the listview class. In the child model I can create multiple rows based on some key date. When I try to display values from both the models (linked with an FK field), all the child objects for the respective FK fields are displayed (wherever more than one records are there).
Is there a way that I may use select distinct to filter out duplicate rows. I have tried:
myModel.objects.distinct().order_by('id')
but still getting all the child rows for the parent id.
In my template I am using:
{% for obj in object_list %}
{{ obj.<field1> }} <!-- field1: Parent model field -->
{% for item in obj.<child_model>_set.all %}
{{ item.<field2> }} <!-- field2: Child model field -->
{% endfor %}
{% endfor %}
My question is:
How do I filter out duplicate rows before the data is displayed?
The backend is sqlite3 on Django 2.0.6 / Python 3.6
Edit
This is how the current list is being generated:
The first column is pk of parent model record and the next column is for child records' pk.
What I am trying to get is:
Option 1: Get only the last (date wise) record for the combination of parent/child rec numbers (i.e. for parent record number 32, only the combination 32|156 should be displayed, and those with values for child records 149 and 148 should not be displayed).
OR
Option 2: Get all combination of records grouped by ParentModel pk field and ChildModel pk field to be shown separately in successive rows (as you can see, the multiple values for a parent record (wherever existing), is being shown in the same row, successive columns).
PS. I am sorry things are getting quite dense here.
Edit 2
This is the class view I am using for displaying data:
class myRateListView(ListView):
template_name = "rate_list.html"
context_object_name = 'ratelists'
model = ParentModel
def get_context_data(self, **kwargs):
context = super(myRateListView, self).get_context_data(**kwargs)
context.update({
'rate_item_list': ChildModel.objects.order_by('field3'),
})
return context
def get_queryset(self):
return ParentModel.objects.values('field1', 'childmodel__field2').distinct()
Here I am getting error :
Cannot resolve keyword 'childmodel' into field. Choices are....
I think my class view is wrong??
Edit 3
Models and view details:
models.py
class TptRateDoc(models.Model):
tpt_rate_doc_number = models.IntegerField(null=True, blank=True.....)
loc_from = models.ForeignKey(Location, related_name='locn_from', on_delete=.......)
loc_to = models.ForeignKey(Location, related_name='locn_to', on_delete=.......)
create_date = models.DateField(default=timezone.now,...)
class TptRateItems(models.Model):
tpt_doc_header = models.ForeignKey(TptRateDoc, on_delete=...)
tpt_rate_item_num = models.CharField(max_length=3, default=10,...)
tpt_rate_valid_from_date = models.DateField(null=True, verbose_name='From date')
tpt_rate_valid_to_date = models.DateField(null=True, verbose_name='To date')
tpt_rate = models.DecimalField(max_digits=7, decimal_places=2, null=True, blank=True,...)
views.py
class TptRateListView(ListView):
template_name = "tpt_rate_list.html"
context_object_name = 'tptrates'
model = TptRateDoc
def get_context_data(self, **kwargs):
context = super(TptRateListView, self).get_context_data(**kwargs)
context.update({
'tpt_rate_item_list': TptRateItems.objects.order_by('tpt_rate_valid_to_date'), # .distinct()
})
return context
def get_queryset(self):
# return TptRateDoc.objects.order_by('loc_from')
return TptRateDoc.objects.values('tpt_rate_doc_number', 'tptrateitems__tpt_rate_item_num').distinct()
Note: The commented out parts are what I had tried earlier.

did you try to catch the returned queryset of distinct filter to a new queryset? I mean like this:(* according docs, distinct will come after order_by())
queryset = myModel.objects.order_by('id').distinct()
and then pass this updated queryset to template.
return render(request, "html_file.html", { "objects": queryset})
On PostgreSQL only, you can pass positional arguments (*fields) in order to specify the names of fields to which the DISTINCT should apply. This translates to a SELECT DISTINCT ON SQL query. Here’s the difference. For a normal distinct() call, the database compares each field in each row when determining which rows are distinct. For a distinct() call with specified field names, the database will only compare the specified field names.

You have not stated when you consider objects to be "distinct". If you do somthing like
queryset = MyModel.objects.distinct()
your queryset will contain all the MyModel instances. Why? Because each instance will have a different id (or pk) even if all other fields are identical. So for distinct() to work you need to specifify the fields to consider, i.e. you need to use values(). So in order to get a queryset containing the distinct values of field2 in your ChildModel you would need to do something like
queryset = ChildModel.objects.values('field2').distinct()
So if I understand you correctly you want to display all "field1" values and associated "field2" values where "field2" values should be unique. What I would recomend is a 2 step approach.
First in your view create a queryset containing distinct combinations of field1 and field2, e.g.:
queryset = ParentModel.objects.values('field1', 'childmodel__field2').distinct()
and then pass this queryset to your template
return render(request, 'parent_child.html', {'objects': queryset})
Then, in order to do the hierachical rendering, you can use the {% regroup %} template tag as described in the docs:
{% regroup objects by field1 as obj_list %}
<ul>
{% for obj in obj_list %}
<li>
{{ obj.grouper }}
<ul>
{% for child in obj.list %}
<li>
{{ child.childmodel__field2 }}
</li>
{% endfor %}
</ul>
</li>
{% endfor %}
</ul>
As OP has not posted his models these are the models used for this answer
class ParentModel(models.Model):
field1 = models.CharField(max_length=100)
class ChildModel(models.Model):
parentmodel = models.ForeignKey(ParentModel, on_delete=models.CASCADE)
field2 = models.CharField(max_length=100)
field3 = models.IntegerField()

Django queryset optimisation: Reverse lookup on _set with filter

I have these models and I need to do some calculations and present them to the user. I render approx 2-3k of rows which results in 4k queries done to the DB (shown from debug toolbar). Is there any way to optimize this? I've tried with prefetch_related but it just adds another query on top of 4k already that are being done..
class Cart(models.Model):
name = models.CharField(max_length=15)
user = models.OneToOneField(User)
def sum_for_this(self, taxtype, tax):
return self.carttax_set.filter(tax__type__name=taxtype,
tax__tax=tax).aggregate(
sum=Coalesce(Sum('tax_amount'), Value('0'))
).get('sum')
class TaxType(models.Model):
name = models.CharField(max_length=10)
class Tax(models.Model):
name = models.CharField(max_length=100)
type = models.ForeignKey(TaxType)
tax = models.DecimalField()
class CartTax(models.Model):
cart = models.ForeignKey(Cart)
tax = models.ForeignKey(Tax)
base = models.IntegerField()
tax_amount = models.IntegerField()
What I do in template is:
{% for cart in cartlist %}
{{ cart.sum_for_this }}
{% endfor %}
I've tried with this but no effect:
Cart.objects.prefetch_related('carttax_set').all()
That method def sum_for_this is doing all the queries..

Try to use annotate with conditional expressions. Your query will look like this:
from django.db.models import Q, F, Sum, Case, When, IntegerField
from django.db.models.functions import Coalesce
cartlist = Cart.objects.annotate(
sum=Coalesce(Sum(Case(
When(Q(carttax__tax__type__name=taxtype) & Q(carttax__tax__tax=tax), then=F('carttax__tax_amount')),
output_field=IntegerField()
)), 0)
)
And in template:
{% for cart in cartlist %}
{{ cart.sum }}
{% endfor %}

Iterating 2 querysets in Django html?

I have two querysets. My first Queryset (services) has all of my service objects. My second Queryset (rating_values) has all of my rating values, where each index corresponds to the index in the services Queryset. For example services[1] corresponds with rating_values[1].
How do I loop through these two queries in an html page in django so so that I can display the service name along with the corresponding rating value?
To be clear, services contains service objects and rating_values contains decimal values. I pass the variables like this:
return render_to_response('services/display_services.html', {'user': request.user, 'services':services, 'rating_values': rating_values })
I would like to do something like this:
{% for service in services %}
<p> {{service.service_name}} : {{ corresponding rating_value}} </p>
Edit
Here is my Service model:
class Service(models.Model):
user_id= models.IntegerField(default=1)
service_name = models.CharField(max_length=100)
address = models.CharField(max_length=200)
def __str__(self): # __unicode__ on Python 2
return self.service_name
Here is my Rating Model:
class Rating(models.Model):
service_id= models.IntegerField(default=1)
service_name= models.CharField(max_length=200)
number_of_ratings = models.IntegerField()
total_rating_value = models.IntegerField()
rating_average = models.FloatField()
def __str__(self): # __unicode__ on Python 2
return self.service_name

Just to answer the original question as well.
If you for some reason don't want a database relation between the models, you could convert the querysets to lists and zip them in the view, turning the querysets to a list of tuples.
view:
return render_to_response(
'services/display_services.html',
context_instance={
'user': request.user,
'services_and_ratings': zip(
services.all(),
rating_values.all()
)
}
)
template:
{% for service, rating in services_and_ratings %}
<p> {{ service.service_name }} : {{ rating.rating_average }} </p>

You can establish the relationship right in the model.
class Rating(models.Model):
service= models.OneToOneField(Service, primary_key=True)
service_name= models.CharField(max_length=200)
number_of_ratings = models.IntegerField()
total_rating_value = models.IntegerField()
rating_average = models.FloatField()
....
You can then pass services and get the corresponding ratings..
{% for service in services %}
<p> {{service.service_name}} : {{ service.rating.rating_average }} </p>

Django model ordering issue

This is my model:
class Feature(models.Model):
name = models.CharField(max_length=75, blank=True)
order = models.SmallIntegerField()
group = models.ForeignKey(FeatureGroup)
def __unicode__(self):
return self.name
class Meta:
ordering = ['order']
The "Features" are being correctly display in the admin control panel based on the value specified in "order".
I have this in my view:
p = get_object_or_404(Phone.objects.prefetch_related('brand', 'features__feature', 'photo_set'), id=id)
I templates I have {% for feature in phone.features.all %}... {{ feature.feature }} ...
The values are being displayed correctly but in random order.
What's wrong and how can I overcome this problem?
Thanks.

How about the template filter dictsort
https://docs.djangoproject.com/en/dev/ref/templates/builtins/#dictsort
{% for feature in phone.features.all|dictsort:"order" %}

last record in django models database

i am trying to do this:
i have a table of rating records of locations. in this table, one location can also have multiple rating scores. but i want to restrict the search only to the last record of same locations. for example:
one location with id=1 has 3 rating scores: 1, 2, 4. now when user searchs for rating score 2, this location should NOT appear, because its last record is 4.
EDIT
there are two tables(django models): location and rating.
can i do this:
all_locations = Location.objects.all()
then in template. locations_rating is a related_name for foreignkey locationID in rating table.
{% for location in all_locations %}
{{ location.locations_rating }}
{% endfor %}

models.py
class Location(models.Model):
locationname = models.CharField(max_length=100)
def __unicode__(self):
return self.locationname
def latest(self):
return Rating.objects.values('rating').filter(von_location=self).order_by('-id')[0]
class Rating(models.Model):
von_location = models.ForeignKey(Location,related_name="locations_rate")
rating = models.IntegerField()
def __unicode__(self):
return "{0}".format(self.rating)
views.py
all_locs = Location.objects.all()
template
{% for location in all_locs %}
{{ location.locationname }} - {{ location.latest.rating }}<br/>
{% endfor %}

This is pure guessing, but can you do something like this?
Rating.objects.filter(location_id=1).order_by(id).reverse()[0]

Ahh, I misinterpreted the question. Here's a not very efficient way to do what you're asking:
locations = Location.objects.all();
filtered = []
for location in locations:
try:
r = Rating.objects.filter(location=location).order_by(-id).[0]
if r.rating = 2:
filtered.append(location)
except Exception as ex:
pass