I'm using Django 1.9 and Postgres 9.5. I have a model (MyModel) with a JSONField (django.contrib.postgres.fields). The JSON value has the same structure in all the objects in the model.
{
"key1": int_val_1,
"key2": int_val_2
}
I want to order my queryset by a value of a specified key in my JSONField. Something like
MyModel.objects.annotate(val=F('jsonfield__key1')).order_by('val')
This does not work - i get the following error
Cannot resolve keyword 'key1' into field. Join on 'jsonfield' not permitted.
Is there anyway i can achieve this?
for Django >= 1.11, you can use:
from django.contrib.postgres.fields.jsonb import KeyTextTransform
MyModel.objects.annotate(val=KeyTextTransform('key1', 'jsonfield')).order_by('val')
An updated answer for this question:
At current time of writing (Django 3.1), you can now order exactly as you had hoped without needing any helper functions:
MyModel.objects.order_by('jsonfield__key1')
JSONExtract is use with mysql
Syntax ('json-field-name', 'json-field-key')
~Q is used for "not equal"
from django_mysql.models.functions import JSONExtract
Listing.objects.annotate(weight=JSONExtract('package_dimensions','$.p>ackage_weight')).filter(~Q(weight=0))
Related
I need to do a values/values_list query on nested key on a postgres backed jsonfield in django 1.10
eg.
class AbcModel(models.model):
context = fields.JSONField()
If it has values like:
{
'lev1': {
'lev': 2
}
}
I want to run a queries like
AbcModel.objects.values('context__lev1__lev2').distinct()
AbcModel.objects.values_list('context__lev1__lev2', flat=True).distinct()
EDIT: The JSON fields are the official django JSONField from django.contrib.postgres.fields
So I found a solution, this works with django 1.10 and above.
I used the KeyTransform to annotate and extract the nexted key and did a values_list on that.
from django.contrib.postgres.fields.jsonb import KeyTransform
extracted_query = AbcModel.objects.annotate(lev1=KeyTransform('lev1', 'context')).annotate(lev2=KeyTransform('lev', 'lev1'))
This query allows me to use lev1 and lev2 as normal fields in the model, so I can do a values, values_list or any other valid query on the fields.
Django 1.11 allows to nest the the two Transforms in one annotate, not sure about 1.10 about the nesting as I have upgraded to 1.11
It's not ideal, but I was able to get this working by adding the json field as an extra field and then calling values on that extra field:
AbcModel.objects.extra(select={
"extra_field": "context->'lev1'->'lev2'"
}).values('extra_field').distinct()
AbcModel.objects.extra(select={
"extra_field": "context->'lev1'->'lev2'"
}).values_list('extra_field', flat=True).distinct()
My application is heavily reliant on APIs that unpredictably make changes to the way they return data. For this reason, I've chosen to use PSQL and JSONFields with Django.
I've seen plenty of examples/docs on how to filter by values in a JSONField, but I haven't seen any that allow me to SELECT on these values.
What I know works;
queryset.filter(jsonfield__key_name = 'value')
What I want to know how to do;
queryset.values('jsonfield__key_name')
Thanks in advance!
The answer is a RawSQL expression;
queryset.annotate(value = RawSQL("(jsonfield->%s)", ('key_name',)))
queryset.values('value')
The first argument to RawSQL is like a template string, the second argument will fill in the first's %s
UPDATE: apparently Django 2.1+ now supports my original expected behavior;
queryset.values('jsonfield__key_name')
Since Django 2.1, transforms are supported in order_by(), values() and values_list(), so you can now use this:
queryset.values('jsonfield__key_name')
Here's the relevant ticket and pull request.
Edit: Added solution below. No-one was able to suggest a native Postgres, one-hit search-and-replace so I had to parse the log in Python and then do an update.
Versions: Django==1.10.3 and Postgres 9.6
Is there a way to search within a nested Django JSONField (backed by Postgres jsonb) for a specific key/value state? Ideally this would be a native Django but I can break out into raw sql if necessary.
e.g. search for one or more occurrences of {"status":"running} within the following data:
{"subtask1": {"status":"running"},
"subtask2": {"status":"complete"}}
Background:
I'm using a JSONField to log the current status of long-running subtasks. Each subtask updates it's element of the json field selectively via a nativePostgres jsonb_set() operation on the server.
After each subtask, I want to query the log field to see whether this subtask was the last to complete. If all are complete (ie no occurences for {"status":"running"} within the nested json tree) then I'll update the main .complete field for the Django RunningTask instance.
Sample & simplified model:
class LongRunningTask(models.model):
id = models.AutoField(primary_key=True)
complete = models.BooleanField(default=False)
log = JSONField(null=True, blank=True, default=dict)
Example data for log field:
{"subtask1": {"status":"running"},
"subtask2": {"status":"complete"}}
Thanks in advance for any pointers.
Note:
I've tried Django's built-in contains operator but this won't find the values because they are nested one level down. I got excited by the prospect of the values operator but from my testing it's not implemented on the JSONField (only HStoreField).
Solution:
Couldn't find a native Postgres one-hit search and replace call, so I ended up parsing the log in Python after each sub-task completed to find out if it's the last one. I hope to find a better solution in time.
_current_log = LongRunningTask.objects.get(pk=current_task.id).log
_statuses = [True if _current_log[i]["status"] == "complete"
else False
for i in _current_log]
if all(_statuses):
LongRunningTask.objects.filter(pk=adhoc_task.id).update(complete=True)
Try this way:
import json
all_completed=True
json_dict=json.loads(json_log)
for key,value in json_dict.items():
if json_dict[key]['status']!='complete':
all_completed=False
break
This is just a hint. You can change it according to your requirements.
Try this:-
LongRunningTask.objects.filter(log__values__contains={"status":"running"})
I'm using json field on my django model:
class JsonTable(models.Model):
data = JSONField()
type = models.IntegerField()
I tried next query, which works for normal sql fields:
JsonTable.objects.filter(type=1).values('type').annotate(Avg('data__superkey'))
But this throws next error:
FieldError: Cannot resolve keyword 'superkey' into field. Join on 'data' not permitted.
Is there way to make group by on json key, using Django ORM or some python lib, without use of raw sql?
Versions: Django 1.9b, PostgreSQL 9.4
UPDATE
Example 2:
JsonTable.objects.filter(type=1).values('data__happykey').annotate(Avg('data_superkey'))
throws same error on happykey
After some researching I found next solution:
from django.db.models import Count
from django.contrib.postgres.fields.jsonb import KeyTextTransform
superkey = KeyTextTransform('superkey', 'data')
table_items = JsonTable.objects.annotate(superkey = superkey).values('superkey').annotate(Count('id')).order_by()
I did not sure about order_by(), but documentation says that is needed.
For another aggregation function type casting needed:
from django.db.models import IntegerField
from django.db.models.functions import Cast
superkey = Cast(KeyTextTransform('superkey', 'data'), IntegerField())
I test with another model, hope that write this code without misprints. PostgreSQL 9.6, Django 2.07
If you are using this package https://github.com/bradjasper/django-jsonfield,
there is nothing in the code for managing such simulated related queries (data__some_json_key)
As Json data is text, you will have to go to raw sql or better : use queryset extra() method, but parsing Json in sql seems to be difficult.
In my Django application, I need to write JSon coming from front end into tables in models.py. The first step to do this, would be convert that Json to dictionary. My question is how to write that dictionary directly to models.py tables? What if some of the fields should be written to one table and some fields should be written to any other table? Is it possible or there is another solution for this? Kindly suggest solutions.
example
There are two tables in models.py called as chart and columnChart , chart_name and chart_description should be written to chart table whereas x_axis_mapping and y_axis_mapping should be written to columnChart table.
All the fields are received in one object called json_data.
json_data can be converted to dictionary. So what is the next step?
Thanks in advance.
A simple way to write directly a dictionary into a Model's table is using python's expansion with django ORM
Ex:
myitems = {'key1': 'value1', 'key2': 'value2', 'key3': 'value3'}
mymodel = MyModel(**myitems)
mymodel.save()
For multiple models/tables you can easily filter dictionary and then use it in specific model/table.
Ex:
myitems1 = dict((k,myitems[k]) for k in ('key1',) if k in myitems)
mymodel1 = MyModel1(**myitems1)
mymodel1.save()
myitems2 = dict((k,myitems[k]) for k in ('key2', 'key3') if k in myitems)
mymodel2 = MyModel2(**myitems2)
mymodel2.save()
I would suggest looking into http://www.django-rest-framework.org/
The framework handles serializing, deserializing, and dealing with various input types as well as building RESTful API's. The serializers also follow Django's DRY philosophy.