I have a fairly basic CRUDMixin
class CRUDMixin(object):
""" create, read, update and delete methods for SQLAlchemy """
id = db.Column(db.Integer, primary_key=True)
#property
def columns(self):
return [ c.name for c in self.__table__.columns ]
def read(self):
""" return json of this current model """
return dict([ (c, getattr(self, c)) for c in self.columns ])
# ...
For something like an Article class which will subclass this, it might have a relationship with another class, like so:
author_id = db.Column(db.Integer, db.ForeignKey('users.id'))
The only real problem is that it will not return any user details in the json. Ideally, the json should look like this:
{
'id': 1234,
'title': 'this is an article',
'body': 'Many words go here. Many shall be unread. Roman Proverb.',
'author': {
'id': 14
'name': 'Thor',
'joined': October 1st, 1994
}
}
As it is right now, it will just give author_id: 14.
Can I detect if a column is a relationship and load it as json as well in this way?
You have to setup the entire relation by adding something like
author = db.relationship("Author") # I assume that you have an Author model
Then to json your result you have differents way to handle relations.
Take a look at this 2 responses :
jsonify a SQLAlchemy result set in Flask
How to serialize SqlAlchemy result to JSON?
You can also take a look at flask-restful which provide a method/decorator (marshal_with) to marshal your results in a good way with nested object (relations).
http://flask-restful.readthedocs.org/en/latest/fields.html#advanced-nested-field
Related
Im fairly new to working with flask and want to know how to insert a dictionary into a column.
I am making a shopping website and the dictionary I want to insert into the order table is shown below:
product_features = {
product_id: {'name': product.product_name,
'price': product.product_price,
'quantity': product_quantity,
'discount': product.product_discount,
'information': product.product_information,
'size': product_size,
'color': product_color,
'image': product.product_image_1,
'all_colors': product.product_color,
'all_sizes': product.product_size,
'max_quantity': product.product_quantity}
}
session['cart'] = product_features
I then store this in a session called cart.
This represents the goods that the user is ordering. Once the user makes an order I then want to add this session data into a column called orders in a table called CustomerOrder.
In a previous thread I saw the code below being used:
class Person(db.Model):
__tablename__ = 'persons'
id = db.Column(db.Integer, primary_key=True)
fancy_name = db.Column(JsonEncodedDict)
import json
from sqlalchemy.ext import mutable
db = SQLAlchemy()
class JsonEncodedDict(db.TypeDecorator):
"""Enables JSON storage by encoding and decoding on the fly."""
impl = db.Text
def process_bind_param(self, value, dialect):
if value is None:
return '{}'
else:
return json.dumps(value)
def process_result_value(self, value, dialect):
if value is None:
return {}
else:
return json.loads(value)
mutable.MutableDict.associate_with(JsonEncodedDict)
I am unsure of how this exactly works and do not want to just copy code that I do not understand. Any advice on this would be very much appreciated as I have looked on YouTube and online and their is not much information on this.
I am studying the library graphene, (https://github.com/graphql-python/graphene) and I was trying to understand how I can serialize / return a nested json into the graphene and perform the query in the correct way.
The code that I will insert below follows the example of the link available in the repository (it is at the end of the question).
import graphene
from graphene.types.resolver import dict_resolver
class User(graphene.ObjectType):
id = graphene.ID()
class Meta:
default_resolver = dict_resolver
class Patron(graphene.ObjectType):
id = graphene.ID()
name = graphene.String()
age = graphene.Int()
user = User
class Meta:
default_resolver = dict_resolver
class Query(graphene.ObjectType):
patron = graphene.Field(Patron)
#staticmethod
def resolve_patron(root, info):
return Patron(**{"id":1, "name": "Syrus", "age": 27, "user": {"id": 2}})
schema = graphene.Schema(query=Query)
query = """
query something{
patron {
id
}
}
"""
if __name__ == "__main__":
result = schema.execute(query)
print(result.data)
The idea is basically to be able to use a multi-level json to "resolve" with graphql. This example is very simple, in the actual use case I plan, there will be several levels in json.
I think that if you use the setattr at the lowest level of json and go up, it works, but I would like to know if someone has already implemented or found a more practical way of doing it.
original example:
https://github.com/graphql-python/graphene/blob/master/examples/simple_example.py
I have a specify use case but my question pertains to the best way of doing this in general.
I have three tables
Order - primary key order_id
OrderLine - Linking table with order_id, product_id and quantity. An order has 1 or more order lines
Product - primary key product_id, each order line has one product
In sqlachemy / python how do I generate nested JSON along the lines of:
{
"orders": [
{
"order_id": 1
"some_order_level_detail": "Kansas"
"order_lines": [
{
"product_id": 1,
"product_name": "Clawhammer",
"quantity": 5
},
...
]
},
...
]
}
Potential Ideas
Hack away doing successive queries
First idea which I want to get away from if possible is using list comprehesion and a brute force approach.
def get_json():
answer = {
"orders": [
{
"order_id": o.order_id,
"some_order_level_detail": o.some_order_level_detail,
"order_lines": [
{
"product_id": 1,
"product_name": Product.query.get(o_line.product_id).product_name,
"quantity": 5
}
for o_line in OrderLine.query.filter(order_id=o.order_id).all()
]
}
for o in Order.query.all()
]
}
This gets hard to maintain mixing the queries with json. Ideally I'd like to do a query first...
Get joined results first, somehow manipulate later
The second idea is to do a join query to join the three tables showing per row in OrderLine the order and product details.
My question to pythonista out there is is there a nice way to convert this to nested json.
Another way?
This really seems like such a common requirement I'm really wondering whether there is a book method for this sort of thing?
Is there an SQLAchemy version of this
Look into marshmallow-sqlalchemy, as it does exactly what you're looking for.
I strongly advise against baking your serialization directly into your model, as you will eventually have two services requesting the same data, but serialized in a different way (including fewer or more nested relationships for performance, for instance), and you will either end up with either (1) a lot of bugs that your test suite will miss unless you're checking for literally every field or (2) more data serialized than you need and you'll run into performance issues as the complexity of your application scales.
With marshmallow-sqlalchemy, you'll need to define a schema for each model you'd like to serialize. Yes, it's a bit of extra boilerplate, but believe me - you will be much happier in the end.
We build applications using flask-sqlalchemy and marshmallow-sqlalchemy like this (also highly recommend factory_boy so that you can mock your service and write unit tests in place of of integration tests that need to touch the database):
# models
class Parent(Base):
__tablename__ = 'parent'
id = Column(Integer, primary_key=True)
children = relationship("Child", back_populates="parent")
class Child(Base):
__tablename__ = 'child'
id = Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey('parent.id'))
parent = relationship('Parent', back_populates='children',
foreign_keys=[parent_id])
# schemas. Don't put these in your models. Avoid tight coupling here
from marshmallow_sqlalchemy import ModelSchema
import marshmallow as ma
class ParentSchema(ModelSchema):
children = ma.fields.Nested(
'myapp.schemas.child.Child', exclude=('parent',), many=True)
class Meta(ModelSchema.Meta):
model = Parent
strict = True
dump_only = ('id',)
class ChildSchema(ModelSchema):
parent = ma.fields.Nested(
'myapp.schemas.parent.Parent', exclude=('children',))
class Meta(ModelSchema.Meta):
model = Child
strict = True
dump_only = ('id',)
# services
class ParentService:
'''
This service intended for use exclusively by /api/parent
'''
def __init__(self, params, _session=None):
# your unit tests can pass in _session=MagicMock()
self.session = _session or db.session
self.params = params
def _parents(self) -> typing.List[Parent]:
return self.session.query(Parent).options(
joinedload(Parent.children)
).all()
def get(self):
schema = ParentSchema(only=(
# highly recommend specifying every field explicitly
# rather than implicit
'id',
'children.id',
))
return schema.dump(self._parents()).data
# views
#app.route('/api/parent')
def get_parents():
service = ParentService(params=request.get_json())
return jsonify(data=service.get())
# test factories
class ModelFactory(SQLAlchemyModelFactory):
class Meta:
abstract = True
sqlalchemy_session = db.session
class ParentFactory(ModelFactory):
id = factory.Sequence(lambda n: n + 1)
children = factory.SubFactory('tests.factory.children.ChildFactory')
class ChildFactory(ModelFactory):
id = factory.Sequence(lambda n: n + 1)
parent = factory.SubFactory('tests.factory.parent.ParentFactory')
# tests
from unittest.mock import MagicMock, patch
def test_can_serialize_parents():
parents = ParentFactory.build_batch(4)
session = MagicMock()
service = ParentService(params={}, _session=session)
assert service.session is session
with patch.object(service, '_parents') as _parents:
_parents.return_value = parents
assert service.get()[0]['id'] == parents[0].id
assert service.get()[1]['id'] == parents[1].id
assert service.get()[2]['id'] == parents[2].id
assert service.get()[3]['id'] == parents[3].id
I would add a .json() method to each model, so that they call each other. It's essentially your "hacked" solution but a bit more readable/maintainable. Your Order model could have:
def json(self):
return {
"id": self.id,
"order_lines": [line.json() for line in self.order_lines]
}
Your OrderLine model could have:
def json(self):
return {
"product_id": self.product_id,
"product_name": self.product.name,
"quantity": self.quantity
}
Your resource at the top level (where you're making the request for orders) could then do:
...
orders = Order.query.all()
return {"orders": [order.json() for order in orders]}
...
This is how I normally structure this JSON requirement.
Check my answer in this thread Flask Sqlalchmey - Marshmallow Nested Schema fails for joins with filter ( where ) conditions and using the Marshmallow package you include in your schema something like this:
name = fields.Nested(Schema, many=True)
I'm wondering how to serialize a dict of nested Schema.
Naively, I would expect syntaxes like this to work:
fields.List(Schema)
fields.Dict(Schema)
or maybe
fields.List(fields.Nested(Schema))
fields.Dict(fields.Nested(Schema))
Serializing a list of Schema can be achieved through Nested(Schema, many=True), but I don't know about a dict of Schema.
Assume, for example's sake, that my object is defined like this:
from marshmallow import Schema, fields, pprint
class AlbumSchema(Schema):
year = fields.Int()
class ArtistSchema(Schema):
name = fields.Str()
# What should I write, here?
# This won't work
albums = fields.Nested(AlbumSchema(), many=True)
# If I write this, AlbumSchema is ignored, so this is equivalent to
albums = fields.Dict(AlbumSchema(), many=True)
# this, which is not satisfying (AlbumSchema unused)
albums = fields.Dict()
# This is not the way either
albums = fields.Dict(fields.Nested(AlbumSchema))
album_1 = dict(year=1971)
album_2 = dict(year=1970)
bowie = dict(name='David Bowie',
albums={
'Hunky Dory': album_1,
'The Man Who Sold the World': album_2
}
)
schema = ArtistSchema()
result = schema.dump(bowie)
pprint(result.data, indent=2)
I expect my object to be serialized as
{ 'albums': { 'Hunky Dory': {'year': 1971},
'The Man Who Sold the World': {'year': 1970}},
'name': 'David Bowie'}
(Question also discussed on GitHub.)
This is not possible right now, but it is a feature request:
https://github.com/marshmallow-code/marshmallow/issues/483
https://github.com/marshmallow-code/marshmallow/issues/496
and it has been worked on already:
https://github.com/marshmallow-code/marshmallow/compare/dev...deckar01:483-structured-dict
2017-12-31: This feature was added to Marshmallow 3.0.0b5 (https://github.com/marshmallow-code/marshmallow/pull/700).
I am using the following code to serialize my appengine datastore to JSON
class DictModel(db.Model):
def to_dict(self):
return dict([(p, unicode(getattr(self, p))) for p in self.properties()])
class commonWordTweets(DictModel):
commonWords = db.StringListProperty(required=True)
venue = db.ReferenceProperty(Venue, required=True, collection_name='commonWords')
class Venue(db.Model):
id = db.StringProperty(required=True)
fourSqid = db.StringProperty(required=False)
name = db.StringProperty(required=True)
twitter_ID = db.StringProperty(required=True)
This returns the following JSON response
[
{
"commonWords": "[u'storehouse', u'guinness', u'badge', u'2011"', u'"new', u'mayor', u'dublin)']",
"venue": "<__main__.Venue object at 0x1028ad190>"
}
]
How can I return the actual venue name to appear?
Firstly, although it's not exactly your question, it's strongly recommended to use simplejson to produce json, rather than trying to turn structures into json strings yourself.
To answer your question, the ReferenceProperty just acts as a reference to your Venue object. So you just use its attributes as per normal.
Try something like:
cwt = commonWordTweets() # Replace with code to get the item from your datastore
d = {"commonWords":cwt.commonWords, "venue": cwt.venue.name}
jsonout = simplejson.dumps(d)