I have the following query:
sales_for_date_for_provider.exclude(sales_or_return='R').values_list('royalty_price', 'conversion_to_usd'))
Is it possible to multiple the royalty_price * conversion_to_usd in the query? Or do I need to do a list comprehension or dive into raw SQL?
The Django documentation warns us that the extra() method will soon be deprecated. Since I came upon this answer when searching for a similar question, I thought I'd share an alternative approach using Django's built-in expressions, specifically the F() object.
In response to the original question, you could achieve the desired outcome with the following:
sales_for_date_for_provider.annotate(
result=F('royalty_price') * F('conversion_to_usd'))
From the documentation:
F() can be used to create dynamic fields on your models by combining different fields with arithmetic:
And here's another example taken from the documentation that further illustrates its usage:
company = Company.objects.annotate(
chairs_needed=F('num_employees') - F('num_chairs'))
You can use extra() and specify select argument:
sales_for_date_for_provider.extra(select={'result': 'royalty_price * conversion_to_usd'})
The result would contain a QuerySet where each object would have an attribute result containing the multiplication of royalty_price and conversion_to_usd fields.
Related
I have a problem with Django and would like to ask for some advice:
One of my models contains specific indicators with variable values. This is why I am using an Arrayfield for those.
For example one Indicator has 3 values, another only 2 and some only have 1 value.
Now I want to compare values from within the Arrayfield with each other.
So I know there are those fancy F Expressions and I tried to use them, but here I am stuck, because it seems like F doesn't allow an positional-lookup from inside ArrayField.
Does anyone know if i can use F for positional lookup of arrayfield? Or do I have to make a direct SQL-Call from Django or change my db somehow to make one value per field?
Here is the Model:
class Indicators(models.Model):
datetime = models.DateTimeField(default=datetime.now)
name = models.TextField(default="none")
values = ArrayField(models.FloatField(default=0))
This is what I want to achieve:
indicator_compareable = Indicators.objects.filter(name='compareable',
values__0=F('values__1')).values_list('values')
And it raises an Exception..
operator does not exist: double precision = double precision[]
LINE 1: ...areable' AND "indicators"."values"[1] = ("...
As Mikhail suggested (in first comment to my post), here is the translated SQL from my Django filter:
SELECT "indicators"."values"[1] FROM "indicators"
WHERE ("indicators"."name" = 'compareable'
AND "indicators"."values"[1] = ("indicators"."values"))
From the Exception it seems like I am not allowed to give ArrayField position to F Expression (or that Django ignores the position..). And the translated sql-query shows, that it seems like F doesn't handle ArrayField-position...
In postgres a correct query for my needs would be:
SELECT values FROM indicators WHERE name='compareable' and
values[1]=values[2];
During my search I found those links here, but they didn't help..:
Django F field iteration
https://code.djangoproject.com/ticket/24709
So actually after investing the translated postgres query it's clear that django's F Expression is (still) not offering positional lookup (and might not be implemented).
I'll leave this as the answer, maybe it helps someone out there who is stuck at the same point.
I decided to use Django's SQL-RAW for my specific needs. But in general the rest of my code uses Django ORM whenever possible.
Here there has been a Django-issue to this topic, but by now it's 1 and a half years old: https://code.djangoproject.com/ticket/24709
by the way, my django version: Django 2.1 (the mentioned django-issue was with v1.8)
Thanks again to Mikhail for his suggestions.
How can I add accent-insensitive search to following snippet from the django docs:
>>> from django.contrib.postgres.search import TrigramSimilarity
>>> Author.objects.create(name='Katy Stevens')
>>> Author.objects.create(name='Stephen Keats')
>>> test = 'Katie Stephens'
>>> Author.objects.annotate(
... similarity=TrigramSimilarity('name', test),
... ).filter(similarity__gt=0.3).order_by('-similarity')
[<Author: Katy Stevens>, <Author: Stephen Keats>]
How could this match test = 'Kâtié Stéphèns'?
There exist the unaccent lookup:
The unaccent lookup allows you to perform accent-insensitive lookups using a dedicated PostgreSQL extension.
Also if you take a look at the aggregation part of django docs, you can read the following:
When specifying the field to be aggregated in an aggregate function,
Django will allow you to use the same double underscore notation that
is used when referring to related fields in filters. Django will then
handle any table joins that are required to retrieve and aggregate the
related value.
Derived from the above:
You can use the trigram_similar lookup, combined with unaccent, then annotate on the result:
Author.objects.filter(
name__unaccent__trigram_similar=test
).annotate(
similarity=TrigramSimilarity('name__unaccent', test),
).filter(similarity__gt=0.3).order_by('-similarity')
OR
if you want to keep it as close as possible to the original sample (and omit one potentially slow filtering followed by another):
Author.objects.annotate(
similarity=TrigramSimilarity('name__unaccent', test),
).filter(similarity__gt=0.3).order_by('-similarity')
Those will only work in Django version >= 1.10
EDIT:
Although the above should work, #Private reports this error occurred:
Cannot resolve keyword 'unaccent' into a field. Join on 'unaccented' not permitted.
This may be a bug, or unaccent is not intended to work that way. The following code works without the error:
Author.objects.filter(
name__unaccent__trigram_similar=test
).annotate(
similarity=TrigramSimilarity('name', test),
).filter(similarity__gt=0.3).order_by('-similarity')
My application is heavily reliant on APIs that unpredictably make changes to the way they return data. For this reason, I've chosen to use PSQL and JSONFields with Django.
I've seen plenty of examples/docs on how to filter by values in a JSONField, but I haven't seen any that allow me to SELECT on these values.
What I know works;
queryset.filter(jsonfield__key_name = 'value')
What I want to know how to do;
queryset.values('jsonfield__key_name')
Thanks in advance!
The answer is a RawSQL expression;
queryset.annotate(value = RawSQL("(jsonfield->%s)", ('key_name',)))
queryset.values('value')
The first argument to RawSQL is like a template string, the second argument will fill in the first's %s
UPDATE: apparently Django 2.1+ now supports my original expected behavior;
queryset.values('jsonfield__key_name')
Since Django 2.1, transforms are supported in order_by(), values() and values_list(), so you can now use this:
queryset.values('jsonfield__key_name')
Here's the relevant ticket and pull request.
Consider this query:
query = Novel.objects.< ...some filtering... >.annotate(
latest_chapter_id=Max("volume__chapter__id")
)
Actually what I need is to annotate each Novel with its latest Chapter object, so after this query, I have to execute another query to select actual objects by annotated IDs. IMO this is ugly. Is there a way to combine them into a single query?
Yes, it's possible.
To get a queryset containing all Chapters which are the last in their Novels, simply do:
from django.db.models.expressions import F
from django.db.models.aggregates import Max
Chapters.objects.annotate(last_chapter_pk=Max('novel__chapter__pk')
).filter(pk=F('last_chapter_pk'))
Tested on Django 1.7.
Possible with Django 3.2+
Make use of django.db.models.functions.JSONObject (added in Django 3.2) to combine multiple fields (in this example, I'm fetching the latest object, however it is possible to fetch any arbitrary object provided that you can get LIMIT 1) to yield your object):
MainModel.objects.annotate(
last_object=RelatedModel.objects.filter(mainmodel=OuterRef("pk"))
.order_by("-date_created")
.values(
data=JSONObject(
id="id", body="body", date_created="date_created"
)
)[:1]
)
Yes, using Subqueries, docs: https://docs.djangoproject.com/en/3.0/ref/models/expressions/#subquery-expressions
latest_chapters = Chapter.objects.filter(novel = OuterRef("pk"))\
.order_by("chapter_order")
novels_with_chapter = Novel.objects.annotate(
latest_chapter = Subquery(latest_chapters.values("chapter")[:1]))
Tested on Django 3.0
The subquery creates a select statement inside the select statement for the novels, then adds this as an annotation. This means you only hit the database once.
I also prefer this to Rune's answer as it actually annotates a Novel object.
Hope this helps, anyone who came looking like much later like I did.
No, it's not possible to combine them into a single query.
You can read the following blog post to find two workarounds.
I want to run a django update through the ORM that looks something like this:
MyModel.objects.filter(**kwargs).update(my_field=F('my_other_field')+'a string')
This causes MySQL to throw an exception. Is there anyway to do this without writing raw SQL?
I had a similar issue; basically I wanted to concatenate two fields to the get the full name of a user. I got it solved this way(but must say that I was using Postgres):
from django.db.models.functions import Concat
from django.db.models import F, Value, CharField
AnyModel.objects.filter(**kwargs).annotate(full_name=Concat(F('model__user_first_name'), Value(' '), F('model__user_last_name'), output_field=CharField()))
where, F('...') evaluates its argument as a query, so you can query a field of the model itself, or span across models as you would do in filter/get, while Value('...') evaluates its argument literally(in my case I needed a space to be placed in between first_name and last_name), and output_field=... specifies the Type of the annotated field(I wanted to be a CharField).
For more info, you can read Django docs about Concat
What's happening is that Django is passing the '+' through to SQL - but SQL doesn't allow the use of '+' for concatenation, so it tries to add numerically. If you use an integer in place of 'a string', it does work in the sense that it adds the integer value of my_other_field to your variable.
It's debatable whether this is a bug. The documentation for F() objects in lookup queries states:
Django supports the use of addition, subtraction, multiplication, division and modulo arithmetic with F() objects
so it could be argued that you shouldn't be trying to use it to update with strings. But that's certainly not documented, and the error message 'Incorrect DOUBLE value' is not very helpful. I'll open a ticket.
You can use Concat function https://docs.djangoproject.com/en/1.9/ref/models/database-functions/#concat
from django.db.models.functions import Concat
from django.db.models import Value
MyModel.objects.filter(**kwargs).update(my_field=Concat('my_other_field', Value('a string'))