I want to run a django update through the ORM that looks something like this:
MyModel.objects.filter(**kwargs).update(my_field=F('my_other_field')+'a string')
This causes MySQL to throw an exception. Is there anyway to do this without writing raw SQL?
I had a similar issue; basically I wanted to concatenate two fields to the get the full name of a user. I got it solved this way(but must say that I was using Postgres):
from django.db.models.functions import Concat
from django.db.models import F, Value, CharField
AnyModel.objects.filter(**kwargs).annotate(full_name=Concat(F('model__user_first_name'), Value(' '), F('model__user_last_name'), output_field=CharField()))
where, F('...') evaluates its argument as a query, so you can query a field of the model itself, or span across models as you would do in filter/get, while Value('...') evaluates its argument literally(in my case I needed a space to be placed in between first_name and last_name), and output_field=... specifies the Type of the annotated field(I wanted to be a CharField).
For more info, you can read Django docs about Concat
What's happening is that Django is passing the '+' through to SQL - but SQL doesn't allow the use of '+' for concatenation, so it tries to add numerically. If you use an integer in place of 'a string', it does work in the sense that it adds the integer value of my_other_field to your variable.
It's debatable whether this is a bug. The documentation for F() objects in lookup queries states:
Django supports the use of addition, subtraction, multiplication, division and modulo arithmetic with F() objects
so it could be argued that you shouldn't be trying to use it to update with strings. But that's certainly not documented, and the error message 'Incorrect DOUBLE value' is not very helpful. I'll open a ticket.
You can use Concat function https://docs.djangoproject.com/en/1.9/ref/models/database-functions/#concat
from django.db.models.functions import Concat
from django.db.models import Value
MyModel.objects.filter(**kwargs).update(my_field=Concat('my_other_field', Value('a string'))
Related
You can add to an existing Postgresql tsvector value using ||, for example:
UPDATE acme_table
SET my_tsvector = my_tsvector ||
to_tsvector('english', 'some new words to add to existing ones')
WHERE id = 1234;
Is there any way to access this functionality via the Django ORM? I.e. incrementally add to an existing SearchVectorField value rather than reconstruct from scratch?
The issue I'm having is the SearchVectorField property returns the tsvector as a string. So when I use the || operator as +, eg:
from django.contrib.postgres.search import SearchVector
instance.my_tsvector_prop += SearchVector(
["new", "fields"],
weight="A",
config='english'
)
I get the error:
TypeError: SearchVector can only be combined with other SearchVector instances, got str.
Because:
type(instance.my_tsvector_prop) == str
A fix to this open Django bug whereby a SearchVectorField property returns a SearchVector instance would probably enable this, if possible. (Although less efficient than combining in the database. In our case the update will run asynchronously so performance is not too important.)
MyModel.objects
.filter(pk=1234)
.update(my_tsvector_prop=
F("my_tsvector_prop") +
SearchVector(
["new_field_name"],
weight="A",
config='english')
)
)
Returns:
FieldError: Cannot resolve expression type, unknown output_field
Another solution would be to run a raw SQL UPDATE, although I'd rather do it through the Django ORM if possible as our tsvector fields often reference values many joins away, so it'd be nice to find a sustainable solution.
I have a problem with Django and would like to ask for some advice:
One of my models contains specific indicators with variable values. This is why I am using an Arrayfield for those.
For example one Indicator has 3 values, another only 2 and some only have 1 value.
Now I want to compare values from within the Arrayfield with each other.
So I know there are those fancy F Expressions and I tried to use them, but here I am stuck, because it seems like F doesn't allow an positional-lookup from inside ArrayField.
Does anyone know if i can use F for positional lookup of arrayfield? Or do I have to make a direct SQL-Call from Django or change my db somehow to make one value per field?
Here is the Model:
class Indicators(models.Model):
datetime = models.DateTimeField(default=datetime.now)
name = models.TextField(default="none")
values = ArrayField(models.FloatField(default=0))
This is what I want to achieve:
indicator_compareable = Indicators.objects.filter(name='compareable',
values__0=F('values__1')).values_list('values')
And it raises an Exception..
operator does not exist: double precision = double precision[]
LINE 1: ...areable' AND "indicators"."values"[1] = ("...
As Mikhail suggested (in first comment to my post), here is the translated SQL from my Django filter:
SELECT "indicators"."values"[1] FROM "indicators"
WHERE ("indicators"."name" = 'compareable'
AND "indicators"."values"[1] = ("indicators"."values"))
From the Exception it seems like I am not allowed to give ArrayField position to F Expression (or that Django ignores the position..). And the translated sql-query shows, that it seems like F doesn't handle ArrayField-position...
In postgres a correct query for my needs would be:
SELECT values FROM indicators WHERE name='compareable' and
values[1]=values[2];
During my search I found those links here, but they didn't help..:
Django F field iteration
https://code.djangoproject.com/ticket/24709
So actually after investing the translated postgres query it's clear that django's F Expression is (still) not offering positional lookup (and might not be implemented).
I'll leave this as the answer, maybe it helps someone out there who is stuck at the same point.
I decided to use Django's SQL-RAW for my specific needs. But in general the rest of my code uses Django ORM whenever possible.
Here there has been a Django-issue to this topic, but by now it's 1 and a half years old: https://code.djangoproject.com/ticket/24709
by the way, my django version: Django 2.1 (the mentioned django-issue was with v1.8)
Thanks again to Mikhail for his suggestions.
How can I add accent-insensitive search to following snippet from the django docs:
>>> from django.contrib.postgres.search import TrigramSimilarity
>>> Author.objects.create(name='Katy Stevens')
>>> Author.objects.create(name='Stephen Keats')
>>> test = 'Katie Stephens'
>>> Author.objects.annotate(
... similarity=TrigramSimilarity('name', test),
... ).filter(similarity__gt=0.3).order_by('-similarity')
[<Author: Katy Stevens>, <Author: Stephen Keats>]
How could this match test = 'Kâtié Stéphèns'?
There exist the unaccent lookup:
The unaccent lookup allows you to perform accent-insensitive lookups using a dedicated PostgreSQL extension.
Also if you take a look at the aggregation part of django docs, you can read the following:
When specifying the field to be aggregated in an aggregate function,
Django will allow you to use the same double underscore notation that
is used when referring to related fields in filters. Django will then
handle any table joins that are required to retrieve and aggregate the
related value.
Derived from the above:
You can use the trigram_similar lookup, combined with unaccent, then annotate on the result:
Author.objects.filter(
name__unaccent__trigram_similar=test
).annotate(
similarity=TrigramSimilarity('name__unaccent', test),
).filter(similarity__gt=0.3).order_by('-similarity')
OR
if you want to keep it as close as possible to the original sample (and omit one potentially slow filtering followed by another):
Author.objects.annotate(
similarity=TrigramSimilarity('name__unaccent', test),
).filter(similarity__gt=0.3).order_by('-similarity')
Those will only work in Django version >= 1.10
EDIT:
Although the above should work, #Private reports this error occurred:
Cannot resolve keyword 'unaccent' into a field. Join on 'unaccented' not permitted.
This may be a bug, or unaccent is not intended to work that way. The following code works without the error:
Author.objects.filter(
name__unaccent__trigram_similar=test
).annotate(
similarity=TrigramSimilarity('name', test),
).filter(similarity__gt=0.3).order_by('-similarity')
My application is heavily reliant on APIs that unpredictably make changes to the way they return data. For this reason, I've chosen to use PSQL and JSONFields with Django.
I've seen plenty of examples/docs on how to filter by values in a JSONField, but I haven't seen any that allow me to SELECT on these values.
What I know works;
queryset.filter(jsonfield__key_name = 'value')
What I want to know how to do;
queryset.values('jsonfield__key_name')
Thanks in advance!
The answer is a RawSQL expression;
queryset.annotate(value = RawSQL("(jsonfield->%s)", ('key_name',)))
queryset.values('value')
The first argument to RawSQL is like a template string, the second argument will fill in the first's %s
UPDATE: apparently Django 2.1+ now supports my original expected behavior;
queryset.values('jsonfield__key_name')
Since Django 2.1, transforms are supported in order_by(), values() and values_list(), so you can now use this:
queryset.values('jsonfield__key_name')
Here's the relevant ticket and pull request.
I have the following query:
sales_for_date_for_provider.exclude(sales_or_return='R').values_list('royalty_price', 'conversion_to_usd'))
Is it possible to multiple the royalty_price * conversion_to_usd in the query? Or do I need to do a list comprehension or dive into raw SQL?
The Django documentation warns us that the extra() method will soon be deprecated. Since I came upon this answer when searching for a similar question, I thought I'd share an alternative approach using Django's built-in expressions, specifically the F() object.
In response to the original question, you could achieve the desired outcome with the following:
sales_for_date_for_provider.annotate(
result=F('royalty_price') * F('conversion_to_usd'))
From the documentation:
F() can be used to create dynamic fields on your models by combining different fields with arithmetic:
And here's another example taken from the documentation that further illustrates its usage:
company = Company.objects.annotate(
chairs_needed=F('num_employees') - F('num_chairs'))
You can use extra() and specify select argument:
sales_for_date_for_provider.extra(select={'result': 'royalty_price * conversion_to_usd'})
The result would contain a QuerySet where each object would have an attribute result containing the multiplication of royalty_price and conversion_to_usd fields.