How can I get a JSON Object in python from getting data via Google App Engine Datastore?
I've got model in datastore with following field:
id
key_name
object
userid
created
Now I want to get all objects for one user:
query = Model.all().filter('userid', user.user_id())
How can I create a JSON object from the query so that I can write it?
I want to get the data via AJAX call.
Not sure if you got the answer you were looking for, but did you mean how to parse the model (entry) data in the Query object directly into a JSON object? (At least that's what I've been searching for).
I wrote this to parse the entries from Query object into a list of JSON objects:
def gql_json_parser(query_obj):
result = []
for entry in query_obj:
result.append(dict([(p, unicode(getattr(entry, p))) for p in entry.properties()]))
return result
You can have your app respond to AJAX requests by encoding it with simplejson e.g.:
query_data = MyModel.all()
json_query_data = gql_json_parser(query_data)
self.response.headers['Content-Type'] = 'application/json'
self.response.out.write(simplejson.dumps(json_query_data))
Your app will return something like this:
[{'property1': 'value1', 'property2': 'value2'}, ...]
Let me know if this helps!
If I understood you correctly I have implemented a system that works something like this. It sounds like you want to store an arbitrary JSON object in a GAE datastore model. To do this you need to encode the JSON into a string of some sort on the way into the database and decode it from a string into a python datastructure on the way out. You will need to use a JSON coder/decoder to do this. I think the GAE infrastructure includes one. For example you could use a "wrapper class" to handle the encoding/decoding. Something along these lines...
class InnerClass(db.Model):
jsonText = db.TextProperty()
def parse(self):
return OuterClass(self)
class Wrapper:
def __init__(self, storage=None):
self.storage = storage
self.json = None
if storage is not None:
self.json = fromJsonString(storage.jsonText)
def put(self):
jsonText = ToJsonString(self.json)
if self.storage is None:
self.storage = InnerClass()
self.storage.jsonText = jsonText
self.storage.put()
Then always operate on parsed wrapper objects instead of the inner class
def getall():
all = db.GqlQuery("SELECT * FROM InnerClass")
for x in all:
yield x.parse()
(untested). See datastoreview.py for some model implementations that work like this.
I did the following to convert the google query object to json. I used the logic in jql_json_parser above as well except for the part where everything is converted to unicode. I want to preserve the data-types like integer, floats and null.
import json
class JSONEncoder(json.JSONEncoder):
def default(self, obj):
if hasattr(obj, 'isoformat'): #handles both date and datetime objects
return obj.isoformat()
else:
return json.JSONEncoder.default(self, obj)
class BaseResource(webapp2.RequestHandler):
def to_json(self, gql_object):
result = []
for item in gql_object:
result.append(dict([(p, getattr(item, p)) for p in item.properties()]))
return json.dumps(result, cls=JSONEncoder)
Now you can subclass BaseResource and call self.to_json on the gql_object
Related
I'm trying to create an app that will let me store files on a database and retrieve them (I know in most cases it's better not to store files on the database itself, but in this instance, that's not what I want to do). I can get the file (a jpg image) stored in the database with:
class File(db.Model):
id = db.Column(db.Integer, primary_key=True)
data = db.Column(db.LargeBinary, nullable=False)
def __init__(self, data):
self.data = data
#app.route("/file/add", methods=["POST"])
def add_file():
data = request.files.get("data")
record = File(data.read())
db.session.add(record)
db.session.commit()
Now how do I return the file?
#app.route("/file/get", methods=["GET"])
def get_file():
returned_file = db.session.query(File.data).first()
return # What goes here?
Some things I've tried (Many of which I didn't expect to work, but I think the error messages are helpful):
return returned_files
Gets me: TypeError: The view function did not return a valid response. The return type must be a string, dict, tuple, Response instance, or WSGI callable, but it was a File.
return jsonify(returned_file)
Gets me: TypeError: Object of type File is not JSON serializable
return send_file(returned_file, attachment_filename="Test.jpg")
Gets me: AttributeError: 'File' object has no attribute 'read'
Aha, I got it! I needed to send it as a buffered stream.
import io
return send_file(io.BytesIO(returned_file.data))
I have the following problem.
I have a class User simplified example:
class User:
def __init__(self, name, lastname, status, id=None):
self.id = id
self.name = name
self.lastname = lastname
self.status = status
def set_status(self,status)
# call to the api to change status
def get_data_from_db_by_id(self)
# select data from db where id = self.id
def __eq__(self, other):
if not isinstance(other, User):
return NotImplemented
return (self.id, self.name, self.lastname, self.status) == \
(other.id, other.name, other.lastname, other.status)
And I have a database structure like:
id, name, lastname, status
1, Alex, Brown, free
And json response from an API:
{
"id": 1,
"name": "Alex",
"lastname": "Brown",
"status": "Sleeping"
}
My question is:
What the best way to compare json vs sql responses?
What for? - it's only for testing purposes - I have to check that API has changed the DB correctly.
How can I deserialize Json and DB resul to the same class? Is there any common /best practices ?
For now, I'm trying to use marshmallow for json and sqlalchemy for DB, but have no luck with it.
Convert the database row to a dictionary:
def row2dict(row):
d = {}
for column in row.__table__.columns:
d[column.name] = str(getattr(row, column.name))
return d
Then convert json string to a dictionary:
d2 = json.loads(json_response)
And finally compare:
d2 == d
If you are using SQLAlchemy for the database, then I would recommend using SQLAthanor (full disclosure: I am the library’s author).
SQLAthanor is a serialization and de-serialization library for SQLAlchemy that lets you configure robust rules for how to serialize / de-serialize your model instances to JSON. One way of checking your instance and JSON for equivalence is to execute the following logic in your Python code:
First, serialize your DB instance to JSON. Using SQLAthanor you can do that as simply as:
instance_as_json = my_instance.dump_to_json()
This will take your instance and dump all of its attributes to a JSON string. If you want more fine-grained control over which model attributes end up on your JSON, you can also use my_instance.to_json() which respects the configuration rules applied to your model.
Once you have your serialized JSON string, you can use the Validator-Collection to convert your JSON strings to dicts, and then check if your instance dict (from your instance JSON string) is equivalent to the JSON from the API (full disclosure: I’m also the author of the Validator-Collection library):
from validator_collection import checkers, validators
api_json_as_dict = validators.dict(api_json_as_string)
instance_json_as_dict = validators.dict(instance_as_json)
are_equivalent = checkers.are_dicts_equivalent(instance_json_as_dict, api_json_as_dict)
Depending on your specific situation and objectives, you can construct even more elaborate checks and validations as well, using SQLAthanor’s rich serialization and deserialization options.
Here are some links that you might find helpful:
SQLAthanor Documentation on ReadTheDocs
SQLAthanor on Github
.dump_to_json() documentation
.to_json() documentation
Validator-Collection Documentation
validators.dict() documentation
checkers.are_dicts_equivalent() documentation
Hope this helps!
So, lets say I have a JSON object stored in the variable names jsonobject and want to read a specific property from it, lets say address.state. If I say jsonobject.address.state I am getting the expected output, but what if the property that I am trying to fins i.e. address.state is stored in a variable lets say key.
So key = "address.state" and when I try to get jsonobject.key I get an error saying jsonobject has no property names key.
How can I implement this .
def main():
#messagebody='{"name":"vivke", "age":"26", "comname":"Infracloud", "address":{ "street":44, "state":"NY" } }'
#i am HTTP POSTing above message format to the function
messagebody = request.get_data().decode("utf-8")
key = "address.state"
#convert messagebody to JSON
jsondata = jsonparser.json2obj(messagebody)
return jsondata.address.state # this works file
return jsondata.key #isnt working
here is the code of jsonparser
import json
from collections import namedtuple
def _json_object_hook(d): return namedtuple('X', d.keys())(*d.values())
def json2obj(data): return json.loads(data, object_hook=_json_object_hook)
The key in jsondata.key is the name of the attribute. Under the hood, something like jsondata.__getattr__("key"). It has nothing to do with your key = "address.state".
If you insist on it, you may need to override the __getattr__ function to split the "address.state" and call the superclass's __getattr__ recursively.
i'm developing a little app for a University project and i need to json encode the result of a query to pass it to a js library, i've read elsewhere that i can use model_to_dict to accomplish that, but i'm getting this error
AttributeError: 'SelectQuery' object has no attribute '_meta'
and i don't know why or what to do, does anyone know how to solve that?
I'm using python 2.7 and the last version of peewee
#app.route('/ormt')
def orm():
doitch = Player.select().join(Nationality).where(Nationality.nation % 'Germany')
return model_to_dict(doitch)
This is because doitch is a SelectQuery instance it is not model, you have to call get()
from flask import jsonify
#app.route('/ormt')
def orm():
doitch = Player.select().join(Nationality).where(Nationality.nation % 'Germany')
return jsonify(model_to_dict(doitch.get()))
Also you could use dicts method to get data as dict. This omits creation a whole model stuff.
from flask import jsonify
#app.route('/ormt')
def orm():
doitch = Player.select().join(Nationality).where(Nationality.nation % 'Germany')
return jsonify(doitch.dicts().get())
edit
As #lord63 pointed out, you cannot simply return dict, it must be a Flask response so convert it to jsonify.
edit 2
#app.route('/ormt')
def orm():
doitch = Player.select().join(Nationality).where(Nationality.nation % 'Germany')
# another query
sth = Something.select()
return jsonify({
'doitch': doitch.dicts().get(),
'something': sth_query.dicts().get()
})
I have Pyramid/cornice resource, that requires a ?query=keyword in end of the url. But I don't know how to add this in a pyramid's dummyRequest object. Code works perfectly on browser and I will get correct response when using this url to get stuff: *url*/foo?query=keyword.
My class/resource is defined like this:
#resource(path='/bar/search/foo')
class SearchFooResource(object):
def __init__(self, request):
self.request = request
#view(renderer='json')
def get(self):
#get query string, it's a tuple
req = self.request.GET.items()
#do stuff with req
Now req should contain the all the query string 'stuffs' in a list that contains them as a tuple's, for example: [('query', 'bar'),('query', 'asd')]. But how do I make unittest to this resource? I can't seem to add anything to self.request.GET.items() method. When running unittest req is empty, and I will get this error: AttributeError: 'list' object has no attribute 'items'.
My current unittest:
def test_passing_GetFooBaarResource(self):
request = testing.DummyRequest()
request.GET = [('query', 'keyword')]
info = SearchFooResource.get(SearchFooResource(request))
self.assertEqual(info['foo'], 'baar')
In addition to what #matino has suggested, you can just use a plain dictionary (instead of a list of tuples you tried).
def test_passing_GetFooBaarResource(self):
request = testing.DummyRequest()
request.GET = {'query': 'keyword'}
info = SearchShowResource.get(SearchShowResource(request))
self.assertEqual(info['foo'], 'baar')
This will work in uncomplicated cases where you don't have multiple parameters with the same name (/someurl?name=foo&name=baz&name=bar).
If you need to test those more complicated queries you can replace your DummyRequest's GET attribute with a WebOb MultiDict
from webob.multidict import MultiDict
def test_passing_GetFooBaarResource(self):
request = testing.DummyRequest()
request.GET = MultiDict([('query', 'foo'), ('query', 'bar'), ('query', 'baz')])
info = SearchShowResource.get(SearchShowResource(request))
self.assertEqual(info['foo'], 'baar')
Then, normally, in your actual view method, if you need to handle multiple parameters with the same name you use request.GET.getall('query') which should return ['foo', 'bar', 'baz'].
In simpler cases you can just use request.GET['query'] or request.GET.get('query', 'default'). I mean, your use of request.GET.items() is a bit unusual...
According to the docs I think you need to pass it as params argument (not tested):
request = testing.DummyRequest(params={'query': 'keyword'})