I am fairly new to python and transitioning from Java, so I'll use the Java terminologies as I need to know the equivalent python version of it.
So I have a Django model as described below:
class Order(models.Model) :
order_id = models.TextField(null=False, blank=False)
is_completed = models.BooleanField(null=False, blank=False, default=False)
Also I have a kafka broker to process these orders. To push these into a kafka queue, I am transforming them into JSON objects as depicted below:
from django.core import serializers
serialized_obj = serializers.serialize('json', [order])
print("Pushing to kafka topic ", "some_topic")
print(serialized_obj)
send_message("some_topic", serialized_obj)
Now I need this JSON object to be converted back to Django model object, in Java we have something called as Jackson which could have done the same for me, but I am not sure how to do this in Python3.
I tried the below code snippet, it returned me an object of type <generator object Deserializer at 0x7fe000323bf8>
# consumer.py
try:
print(json.loads(msg.value()))
print("---------------------------------")
obj = serializers.deserialize("json", msg.value())
print(obj)
except Exception as e:
import traceback
print(traceback.format_exc())
How can I achieve this in Python3?
First to serialize the object do this:
from django.core import serializers
d = serializers.serialize('json', Order.objects.all()) # serialize all the objects in Order model
To deserialize the object from it
for obj in serializers.deserialize('json', d):
print(obj.object) ## return the django model class object
serializers.deserialize() gives an Deserializedobject
To get more details refer Django object deserialization
Related
The basic problem is that in a Flask application trying to return a MongoDB document I get the error
TypeError: Object of type 'BaseQuerySet' is not JSON serializable
the get method is the following:
def get(self, projectId):
response = MyObject.objects(project=projectId)
return response , 200, None
There may be multiple objects with the same projectId so a BaseQuerySet is returned.
I tried using BSON json_util (as suggested here: JSON serializing Mongodb) but the code below:
response=bson.json_util.dumps(response)
returns just the list of the document's fields without any value.
The only workaround I figure is to return a naive string concatenation of the fields I need.
The same code was working fine few time ago, have anyone got a similar problem?
EDIT
The Class MyObject is like to the one below:
from flask_mongoengine import MongoEngine
from mongoengine.fields import *
db = MongoEngine()
class User(db.Document):
email = db.StringField(max_length=120)
project = db.StringField(db.StringField(max_length=64))
creation_date = db.DateTimeField(default=datetime.datetime.now)
modified_date = db.DateTimeField(default=datetime.datetime.now)
I am using Django 2.1.1.
I have a model Analysis that, among other fields, contains a ForeignKey to a MyFile model (a model I wrote to handle files):
from polymorphic.models import PolymorphicModel
from django.db.models import Model, DateTimeField, FileField, SET_NULL
from django.db.models.signals import pre_delete
class MyFile(Model):
file = FileField(upload_to='./', null=False, blank=False)
description = CharField(max_length=255, null=True, blank=True)
date_added = DateTimeField(auto_now_add=True)
#receiver(pre_delete, sender=MyFile)
def mymodel_delete(sender, instance, **kwargs):
"""
To delete the file connected to the `sender` class: receive the pre_delete signal
and delete the file associated with the model instance.
"""
instance.file.delete(False)
class Analysis(PolymorphicModel):
# ... other fields ...
file_results = ForeignKey(MyFile, on_delete=SET_NULL,
related_name='file_results',
null=True, blank=True)
Analysis is a PolymorphicModel for reasons related to the bigger project.
In Analysis.file_results I set on_delete=SET_NULL because I want to allow an Analysis instance to exist even without a file_result, which can be populated later.
Let's suppose I have added a few files (the MyFile table has a few rows) and a few Analysis instances. Now, if I want to delete the file related to one of the instances of Analysis I do:
a = Analysis.objects.get(pk=0)
a.file_results.delete()
a.save()
but I get the following error:
File "/Users/mtazzari/djangos/views.py" in update_job_refs
377. a.save()
File "/Users/mtazzari/anaconda/envs/djangos/lib/python3.6/site-packages/polymorphic/models.py" in save
83. return super(PolymorphicModel, self).save(*args, **kwargs)
File "/Users/mtazzari/anaconda/envs/djangos/lib/python3.6/site-packages/django/db/models/base.py" in save
670. "unsaved related object '%s'." % field.name
ValueError: save() prohibited to prevent data loss due to unsaved
related object 'file_results'.
The mymodel_delete function that is called on pre_delete signal works correctly as the file gets actually deleted from the file system.
However, I really don't understand how to solve the ValueError.
Interestingly, I notice that the following lines work fine, i.e. do not raise any ValueError, get the file deleted from the file system, and the FK in a.file_results set to Null:
a = Analysis.objects.get(pk=0)
tmp = a.file_results
a.file_results = None
tmp.file_results.delete()
a.save()
But, is this a proper way of doing this? What is the best practice for deleting a related object?
Thanks!
First, note that you don't need to save() just because of the delete(). The delete() will update the database as required.
That said, it's reasonable to want to continue using the instance to do other operations, leading to a save(). The reason you're getting the error is that the a.file_results Python object still exists, and references a database row that is now missing. The documentation for delete() mentions this:
This only deletes the object in the database; the Python instance will still exist and will still have data in its fields.
So if you want to continue to work with the instance object, just set the attribute to None yourself. Similar to your code above, except you don't need the temp object.
a = Analysis.objects.get(pk=0)
a.file_results.delete()
a.file_results = None
# ... more operations on a
a.save() # no error
I`m trying to pass database objects from one view to another view. But when I try to achieve this using SESSION, I am getting this "is not JSON serializiable" error.
My Views.py:
def index(request):
listset = TheaterBase.objects.all()
request.session['s_listset'] = listset
def otherview(request):
result = request.session.get('s_listset')
How to pass the Database objects in between the views?
Thanks in advance
Server sessions can store JSON objects only. You are trying to store a complex Django QuerySet object, which naturally is not JSON serializable.
And trust me, even if it was, you wouldn't want to do this. It's not healthy to abuse your server's session with a high amount of data.
Let's just assume that your TheaterBase class is something like below(pseudo code)
class TheaterBase:
field1 ...
fielld2 ...
-------
# a method to produce json serializable representation
def as_dict(self):
return {'field1': self.field1, 'fileld2': self.fielld2}
Then on you view do
listset = [x.as_dict() for x in TheaterBase.objects.all()]
The issue here is the object coming out of your db query are not json serializable. The as_dict method above is constructing a json serializable representation of that object.
You can try using django serializers
from django.core import serializers
listset = serializers.serialize("json", TheaterBase.objects.all())
I'm trying to figure out a way to serialize some Django model object to JSON format, something like:
j = Job.objects.get(pk=1)
##############################################
#a way to get the JSON for that j variable???
##############################################
I don't want:
from django.core import serializers
serializers.serialize('json', Job.objects.get(pk=1),ensure_ascii=False)
Because it returns JSON array, not a single object representation.
Any ideas?
One way I'm thinking of: is to find a way to get a hash(attribute,value) of the object and then use simplejson to get the JSON representation of it, however I don't know how to get that hash.
How about just massaging what you get back from serializers.serialize? It is not that hard to trim off the square brackets from the front and back of the result.
job = Job.objects.get(pk=1)
array_result = serializers.serialize('json', [job], ensure_ascii=False)
just_object_result = array_result[1:-1]
Not a fancy answer but it will give you just the object in json notation.
Method-1
Use Django Serializer with python format
from django.core import serializers
j = Job.objects.get(pk=1)
response = serializers.serialize('python', [j], ensure_ascii=False)
Method-2
use json format while serializing and loads the string response
import json
from django.core import serializers
j = Job.objects.get(pk=1)
json_str_response = serializers.serialize('json', [j], ensure_ascii=False)
response = json.loads(json_str_response)[0]
Method-3
Use Django REST Framework's Serializer class
define a serializer class and serialize the instance as
from rest_framework import serializers
class JobSerializer(serializers.ModelSerializer):
class Meta:
model = Job
fields = '__all__'
j = Job.objects.get(pk=1)
response = JobSerializer(instance=j).data
Reference
1. Serializer Django model object
I would suggest using Django's model_to_dict. If I'm not mistaken, serializers.serialize() relies on it, too, but it only works for list, not single model instance. That's how you get a dict instance with your model fields out of a single model:
from django.forms.models import model_to_dict
# assuming obj is your model instance
dict_obj = model_to_dict( obj )
You now just need one straight json.dumps call:
import json
json.dumps(dict_obj)
Is it possible to use Django serializer without a Model?
How it is done?
Will it work with google-app-engine?
I don't use Django framework, but since it is available, I would want to use its resources here and there.
Here is the code I tried:
from django.core import serializers
obj = {'a':42,'q':'meaning of life'}
serialised = serializers.serialize('json', obj)
this generates an error
ERROR ... __init__.py:385] 'str' object has no attribute '_meta'
Serializers are only for models. Instead you can use simplejson bundled with Django.
from django.utils import simplejson
json_str = simplejson.dumps(my_object)
Simplejson 2.0.9 docs are here.
The GQLEncoder class in this library can take a db.Model entity and serialize it. I'm not sure if this is what you're looking for, but it's been useful to me.