I'm trying to create a CRUD application with Google Cloud ndb and REST architecture.
Therefore I have different API calls for creating and retrieving.
Now to update the entities I need to display them in the front end but also give an identifier, so ndb knows which entity to update later.
I try to get the entities with model.query and then encode them to JSON with an extended encoder to serialize datetime and ndb.Key:
# JSONEncoder extension to handle datetime & ndb.Key
class CustomJsonEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, datetime):
return obj.strftime('%Y-%m-%d-%H-%M-%S')
if isinstance(obj, ndb.Key):
return obj.urlsafe()
return json.JSONEncoder.default(self, obj)
# Get JSON data from ndb
query = Card.query(Card.category==category, ancestor=ancestor_key()).fetch()
cards = json.dumps([c.to_dict() for c in query], cls=CustomJsonEncoder)
self.response.headers['Content-Type'] = 'application/json'
self.response.out.write(cards)
My problem now is that the entity key just disappears and isn't shown in the json.dump anymore. I don't get any errors, it's just not encoded and passed along.
The datetime object is shown correctly when printed. Any ideas on how I can send the URL safe ndb key along with the json.dump?
You're dumping Card entity objects, which do not include the respective object keys. To confirm this just temporarily add a logging statement like this to your CustomJsonEncoder:
if isinstance(obj, ndb.Key):
logging.error('key found: %s' % obj.urlsafe())
return obj.urlsafe()
You need to explicitly add the entity keys to the data to be json-encoded if you want them.
Or you could use a ComputedProperty in that entity's model (but then you'd be wasting some storage):
key_str = ndb.ComputedProperty(lambda self: self.key.urlsafe())
Related
I have a flask-restful project that interfaces with some custom classes, containing uuid (uuid.UUID) types used as ids. There are a couple of api endpoints which return the object associated with the given id, parsed by flask as an UUID. The issue is that, when I return them as a json payload, I get the following exception:
UUID('…') is not JSON serializable
I want to have those uuids represented as strings to the final user, making the process seamless (the user can take the returned uuid and use it for his next api request).
In order to fix this problem, I had to put together suggestions from two different places:
first, I need to create a custom json encoder, which when dealing with uuids, returns their string representation. StackOverflow answer here
import json
from uuid import UUID
class UUIDEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, UUID):
# if the obj is uuid, we simply return the value of uuid
return str(obj) # <- notice I'm not returning obj.hex as the original answer
return json.JSONEncoder.default(self, obj)
second, I need to take this new encoder and set it as the flask-restful encoder used for the responses. GitHub answer here
class MyConfig(object):
RESTFUL_JSON = {'cls': MyCustomEncoder}
app = Flask(__name__)
app.config.from_object(MyConfig)
api = Api(app)
putting it together:
# ?: custom json encoder to be able to fix the UUID('…') is not JSON serializable
class UUIDEncoder(json.JSONEncoder):
def default(self, obj: Any) -> Any: # pylint:disable=arguments-differ
if isinstance(obj, UUID):
return str(obj) # <- notice I'm not returning obj.hex as the original answer
return json.JSONEncoder.default(self, obj)
# ?: api configuration to switch the json encoder
class MyConfig(object):
RESTFUL_JSON = {"cls": UUIDEncoder}
app = Flask(__name__)
app.config.from_object(MyConfig)
api = Api(app)
on a side note, if you are using vanilla flask, the process is simpler, just set your app json encoder directly (app.json_encoder = UUIDEncoder)
I hope it's useful to someone!
I want to be able to query my mongo db and get the results for all entries in my stocksCollection collection. I am using allStocks = list(stocksCollection.find({})) which gives me a list of all the entries in that collection. However, when I try to return this as the response to a get request, I get the error:
TypeError: The view function did not return a valid response.
The return type must be a string, dict, tuple, Response instance, or WSGI callable, but it was a list.
Here is the simple code I used to get this:
#app.route("/stocks", methods=['GET'])
def getAllStocks():
return list(stocksCollection.find({}))
I have also tried: return stocksCollection.find({}) (errors because it is a type: cursor) and
allStocks = list(stocksCollection.find({}))
return {'data': allStocks}
But that just gives me the error TypeError: Object of type ObjectId is not JSON serializable. Any ideas on what formatt I can change this cursor to so that I am able to return it to my api call (this will not be serving up directly to a webpage, just being called by the frontend to do some calculations)
Create a JSONEncoder class to make ObjectId JSON serializable
import json
from bson import ObjectId
class JSONEncoder(json.JSONEncoder):
def default(self, o):
if isinstance(o, ObjectId):
return str(o)
return json.JSONEncoder.default(self, o)
And then use JSONEncoder class as below (in json.dumps) and make the _id value JSON serializable and append the data into a new list return it.
#app.route("/stocks", methods=['GET'])
def getAllStocks():
output = []
all_data = stocksCollection.find({})
for data in all_data:
data['_id'] = json.dumps(data['_id'], cls=JSONEncoder)
output.appned(data)
return {"data": output}
I think this will solve your problem.
I have a problem with returning a JSON response with translated strings in my application's endpoints.
The default pyramid renderer is my custom JSON renderer. Some of the objects in response are TranslationStrings. I would like to have them automatically translated.
Now I am using this: Pyramid TranslationString not working on json renderer, but it's not an ideal solution for me. I do not want to translate all of the responses manually.
For translations I am using the TransationStringFactory:
_ = i18n.TranslationStringFactory('coma')
I already have some renderer's adapters. So I added a new one - for TransationString class:
def includeme(config):
json_renderer = JSON()
def date_adapter(obj, request):
return obj.isoformat()
def set_adapter(obj, request):
return list(obj)
def uuid_adapter(obj, request):
return str(obj)
def enum_adapter(obj, request):
return obj.value
def trans_string_adapter(obj, request):
return request.localizer.translate(obj)
json_renderer.add_adapter(TranslationString, trans_string_adapter)
json_renderer.add_adapter(datetime.date, date_adapter)
json_renderer.add_adapter(set, set_adapter)
json_renderer.add_adapter(uuid.UUID, uuid_adapter)
json_renderer.add_adapter(enum.Enum, enum_adapter)
config.add_renderer('json', json_renderer)
Here is the example of the JSON object I want to return:
return {
'label': _('Estimated net income'),
'value': round(income_net, self.decimal_places),
...
}
Why my custom JSON renderer cannot call adapter for TranslationString object?
The reason it's not being called is because json.dumps only invokes the default adapters if a type is not json-serializable. A TranslationString subclasses str so it is json-serializable and your adapters are not used.
I think, in general, this is an issue with TranslationString and how it works. It expects you to always pass the string through the localizer, and so you should do that as soon as possible instead of waiting for egress. Unfortunately that basically means passing the localizer all over the place, or making it available as a threadlocal.
I'm trying to save a date in my sessions. I always receive the error Object of type 'datetime' is not JSON serializable. I found this here in the Django documentation: stored as seconds since epoch since datetimes are not serializable in JSON.
How can I save my expiry_date as seconds instead of datetime?
code = social_ticketing_form.cleaned_data['a']
expiry_date = timezone.now() + timezone.timedelta(days=settings.SOCIAL_TICKETING_ATTRIBUTION_WINDOW)
request.session[social_ticketing_cookie_name(request.event)] = {'code': code, 'expiry_date': expiry_date}
Either write your own session serialiser to allow you to serialise datetime objects directly, or store the datetime value in some other form.
If you want to save it as seconds, then use the datetime.timestamp() method:
request.session[social_ticketing_cookie_name(request.event)] = {
'code': code,
'expiry_date': expiry_date.timestamp()
}
Your own SESSION_SERIALIZER class only needs to provide loads and dumps methods, directly analogous to json.loads() and json.dumps() (which is how the standard JSON serializer is implemented).
If you want to encode datetime objects and be able to transparently turn those back into datetime objects again, I'd use a nested object format to flag such values as special:
from datetime import datetime
class JSONDateTimeSerializer:
#staticmethod
def _default(ob):
if isinstance(ob, datetime):
return {'__datetime__': ob.isoformat()}
raise TypeError(type(ob))
#staticmethod
def _object_hook(d):
if '__datetime__' in d:
return datetime.fromisoformat(d['__datetime__'])
return d
def dumps(self, obj):
return json.dumps(
obj, separators=(',', ':'), default=self._default
).encode('latin-1')
def loads(self, data):
return json.loads(
data.decode('latin-1'), object_hook=self._object_hook
)
and set SESSION_SERIALIZER to the full qualified name of the above module (path.to.module.JSONDateTimeSerializer).
The above uses the datetime.fromisoformat() method, new in Python 3.7.
I`m trying to pass database objects from one view to another view. But when I try to achieve this using SESSION, I am getting this "is not JSON serializiable" error.
My Views.py:
def index(request):
listset = TheaterBase.objects.all()
request.session['s_listset'] = listset
def otherview(request):
result = request.session.get('s_listset')
How to pass the Database objects in between the views?
Thanks in advance
Server sessions can store JSON objects only. You are trying to store a complex Django QuerySet object, which naturally is not JSON serializable.
And trust me, even if it was, you wouldn't want to do this. It's not healthy to abuse your server's session with a high amount of data.
Let's just assume that your TheaterBase class is something like below(pseudo code)
class TheaterBase:
field1 ...
fielld2 ...
-------
# a method to produce json serializable representation
def as_dict(self):
return {'field1': self.field1, 'fileld2': self.fielld2}
Then on you view do
listset = [x.as_dict() for x in TheaterBase.objects.all()]
The issue here is the object coming out of your db query are not json serializable. The as_dict method above is constructing a json serializable representation of that object.
You can try using django serializers
from django.core import serializers
listset = serializers.serialize("json", TheaterBase.objects.all())