I'm trying to get Marshmallow-SQLAlchemy to deserialize an object with a nested object without specifying the foreign key for the nested object (which should be the primary key of the parent object). Here's a standalone example:
# Python version == 3.8.2
from datetime import datetime
import re
# SQLAlchemy == 1.3.23
from sqlalchemy import func, create_engine, Column, ForeignKey, Text, DateTime
from sqlalchemy.ext.declarative import as_declarative, declared_attr
from sqlalchemy.orm import relationship, sessionmaker
# marshmallow==3.10.0
# marshmallow-sqlalchemy==0.24.2
from marshmallow import fields
from marshmallow.fields import Nested
from marshmallow_sqlalchemy import SQLAlchemyAutoSchema
################################################################################
# Set up
################################################################################
engine = create_engine("sqlite:///test.db")
Session = sessionmaker()
Session.configure(bind=engine)
session = Session()
################################################################################
# Models
################################################################################
#as_declarative()
class Base(object):
#declared_attr
def __tablename__(cls):
# From https://stackoverflow.com/questions/1175208/elegant-python-function-to-convert-camelcase-to-snake-case
name = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', cls.__name__)
return re.sub('([a-z0-9])([A-Z])', r'\1_\2', name).lower()
#declared_attr
def updated(cls):
return Column(DateTime, default=func.now(), onupdate=func.now(), nullable=False)
class Account(Base):
id = Column(Text, primary_key=True)
name = Column(Text, nullable=False)
tags = relationship("AccountTag", backref="account")
class AccountTag(Base):
account_id = Column(Text, ForeignKey('account.id'), primary_key=True)
Key = Column(Text, primary_key=True)
Value = Column(Text, nullable=False)
################################################################################
# Schemas
################################################################################
class AutoSchemaWithUpdate(SQLAlchemyAutoSchema):
class Meta:
load_instance = True
sqla_session = session
updated = fields.DateTime(default=lambda: datetime.now())
class AccountSchema(AutoSchemaWithUpdate):
class Meta:
model = Account
include_relationships = True
tags = Nested("AccountTagSchema", many=True)
class AccountTagSchema(AutoSchemaWithUpdate):
class Meta:
model = AccountTag
include_fk = True
################################################################################
# Test
################################################################################
Base.metadata.create_all(engine)
account_object = AccountSchema().load({
"id": "ABC1234567",
"name": "Account Name",
"tags": [
{
"Value": "Color",
"Key": "Blue"
}
]
})
session.merge(account_object)
session.commit()
And here's the error I'm getting:
Traceback (most recent call last):
File "example.py", line 88, in <module>
account_object = AccountSchema().load({
File "C:\python\site-packages\marshmallow_sqlalchemy\schema\load_instance_mixin.py", line 92, in load
return super().load(data, **kwargs)
File "C:\python\site-packages\marshmallow\schema.py", line 727, in load
return self._do_load(
File "C:\python\site-packages\marshmallow\schema.py", line 909, in _do_load
raise exc
marshmallow.exceptions.ValidationError: {'tags': {0: {'account_id': ['Missing data for required field.']}}}
I feel like I'm trying to do something intuitive but I'm not sure anymore. I'm sure that I'm close here but am having no luck getting this to work. Help is much appreciated.
You are getting the error because you've specified include_fk in the Meta class for AccountTagSchema.
You can inspect the fields that have been generated for the schema:
print(AccountTagSchema._declared_fields["account_id"])
# <fields.String(default=<marshmallow.missing>, attribute=None, validate=[], required=True, load_only=False, dump_only=False, missing=<marshmallow.missing>, allow_none=False, error_messages={'required': 'Missing data for required field.', 'null': 'Field may not be null.', 'validator_failed': 'Invalid value.', 'invalid': 'Not a valid string.', 'invalid_utf8': 'Not a valid utf-8 string.'})>
Notice that it generates account_id with required=True, this is due to the sqlalchemy column that it represents being NOT NULL as it is part of the primary key.
So the simplest thing is to remove include_fk from the schema meta:
class AccountTagSchema(AutoSchemaWithUpdate):
class Meta(AutoSchemaWithUpdate.Meta):
model = AccountTag
# include_fk = True <--- remove
...however, run the script and you'll run into another problem:
sqlalchemy.orm.exc.UnmappedInstanceError: Class 'builtins.dict' is not mapped
This means that we end up passing a dict to SQLAlchemy session where it is expecting a mapped Base subclass.
The reason for this is that when a child class inherits from a base schema, e.g., AutoSchemaWithUpdate in this case, the child class does not automatically inherit the parent's meta configuration. The docs provide a couple of strategies for this, the simplest being that the child's Meta class should also inherit from the parent's Meta class:
class AccountSchema(AutoSchemaWithUpdate):
class Meta(AutoSchemaWithUpdate.Meta): # <--- this here
model = Account
include_relationships = True
tags = Nested("AccountTagSchema", many=True)
Once we do that for both AccountSchema and AccountTagSchema we are ready to run the script again and it works...the first time. Immediately run the script again, and another error occurs:
AssertionError: Dependency rule tried to blank-out primary key column 'account_tag.account_id' on instance '<AccountTag at 0x7f14b0f9b670>'
This is a consequence of the design decision to have the loaded AccountTag instances unidentifiable (i.e., excluding the primary key from the payload) and the decision to include the foreign key field as part of the primary key for AccountTag.
SQLAlchemy cannot identify that the newly created AccountTag instances are the same as the ones that already exist, so it first tries to disassociate the original account tags from the account by setting the value of the foreign key field to None. However, this isn't allowed as the foreign key is also the primary key and cannot be set NULL.
The solution for this is described here and involves setting an explicit cascade on the relationship:
class Account(Base):
id = Column(Text, primary_key=True)
name = Column(Text, nullable=False)
tags = relationship("AccountTag", backref="account", cascade="all,delete-orphan")
Now run the script again, and it will work every time.
Related
I have two models Parent and Child in my database, there's a one to many relationship between these two models i.e. one parent can have multiple children.
from flask import Flask, request
from flask_restful import Resource
from flask_sqlalchemy import SQLAlchemy
from flask_marshmallow import Marshmallow
from flask_restful import Api
from marshmallow import fields
app = Flask(__name__)
app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///relationships.db"
db = SQLAlchemy(app)
ma = Marshmallow(app)
api = Api(app, prefix="/api")
class Parent(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(20), nullable=False, unique=True)
children = db.relationship("Child", backref="parent")
class Child(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(20), nullable=False, unique=True)
parent_id = db.Column(db.Integer, db.ForeignKey("parent.id"))
class ChildSchema(ma.SQLAlchemyAutoSchema):
class Meta:
model = Child
ordered = True
include_fk = True
dump_only = ("id",)
load_instance = True
class ParentSchema(ma.SQLAlchemyAutoSchema):
children = fields.Pluck(ChildSchema, "name", many=True)
class Meta:
model = Parent
ordered = True
include_relationships = True
dump_only = ("id",)
load_instance = True
class ParentResource(Resource):
#classmethod
def get(cls, _id: int):
parent_schema = ParentSchema()
return parent_schema.dump(parent.query.filter_by(id=_id).first()), 200
#classmethod
def put(cls, _id: int):
parent_json = request.get_json()
parent_schema = ParentSchema()
parent_input_data = parent_schema.load(parent_json)
parent = Parent.query.filter_by(id=_id).first()
parent.name = parent_input_data.name
child_names = [child.name.lower() for child in parent.children]
# Check if child is not already in parent children list
for child_input in parent_input_data.children:
if child_input.name.lower() not in child_names:
parent.children.append(child_input)
db.session.add(parent)
db.session.commit()
return {"message": "Updated"}, 200
api.add_resource(ParentResource, "/parent/<int:_id>")
if __name__ == "__main__":
db.create_all()
app.logger.info("Starting app...")
app.run("127.0.0.1", 3003)
When I try to update the parent by adding some new children, I get a unique key constraint error as it seems that SQLAlchemy is trying to run an insert query in the parent table rather than trying to update the parent table record. There is already a record with the same name 'ABCD' in the parent table i.e. I just tried to leave the parent name as is and just update the children.
This is the input I have given to the PUT request.
{
"name": "ABCD",
"children": [
"Tom",
"Spot"
]
}
Can some one please help me understand where I am going wrong ? When I try to update the parent without trying to update the children, the update seems to work as expected. The issue happens only when I try to update the child relationship.
Try changing your query from using INSERT INTO to using UPDATE. INSERT INTO attempts to add a new record while UPDATE will modify existing records.
I tried to run Scrapy using itemLoader to collect all the data and put them into SQLite 3. I am success in gathering all the info I wanted but I cannot get the foreign keys to be generated in my ThreadInfo and PostInfo tables using back_populates with foreign key. I did try with back_ref but it also did not work.
All the other info was inserted to SQLite database after my Scrapy finished.
My goal is to have four tables, boardInfo, threadInfo, postInfo, and authorInfo linked to each others.
boardInfo will have one-to-many relationship with threadInfo
threadInfo will have one-to-many relationship with postInfo
authorInfo will have one-to-many relationship with threadInfo and
postInfo.
I used DB Browser for SQLite and found that the values of my foreign keys are Null.
I tried query for the value (threadInfo.boardInfos_id), and it displayed None. I try to fix this for many days and read through the document but cannot solve the issue.
How can I have the foriegn keys generated in my threadInfo and postInfo tables?
Thank you for all guidances and comments.
Here is my models.py
from sqlalchemy import create_engine, Column, Table, ForeignKey, MetaData
from sqlalchemy import Integer, String, Date, DateTime, Float, Boolean, Text
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
from scrapy.utils.project import get_project_settings
Base = declarative_base()
def db_connect():
'''
Performs database connection using database settings from settings.py.
Returns sqlalchemy engine instance
'''
return create_engine(get_project_settings().get('CONNECTION_STRING'))
def create_table(engine):
Base.metadata.create_all(engine)
class BoardInfo(Base):
__tablename__ = 'boardInfos'
id = Column(Integer, primary_key=True)
boardName = Column('boardName', String(100))
threadInfosLink = relationship('ThreadInfo', back_populates='boardInfosLink') # One-to-Many with threadInfo
class ThreadInfo(Base):
__tablename__ = 'threadInfos'
id = Column(Integer, primary_key=True)
threadTitle = Column('threadTitle', String())
threadLink = Column('threadLink', String())
threadAuthor = Column('threadAuthor', String())
threadPost = Column('threadPost', Text())
replyCount = Column('replyCount', Integer)
readCount = Column('readCount', Integer)
boardInfos_id = Column(Integer, ForeignKey('boardInfos.id')) # Many-to-One with boardInfo
boardInfosLink = relationship('BoardInfo', back_populates='threadInfosLink') # Many-to-One with boardInfo
postInfosLink = relationship('PostInfo', back_populates='threadInfosLink') # One-to-Many with postInfo
authorInfos_id = Column(Integer, ForeignKey('authorInfos.id')) # Many-to-One with authorInfo
authorInfosLink = relationship('AuthorInfo', back_populates='threadInfosLink') # Many-to-One with authorInfo
class PostInfo(Base):
__tablename__ = 'postInfos'
id = Column(Integer, primary_key=True)
postOrder = Column('postOrder', Integer, nullable=True)
postAuthor = Column('postAuthor', Text(), nullable=True)
postContent = Column('postContent', Text(), nullable=True)
postTimestamp = Column('postTimestamp', Text(), nullable=True)
threadInfos_id = Column(Integer, ForeignKey('threadInfos.id')) # Many-to-One with threadInfo
threadInfosLink = relationship('ThreadInfo', back_populates='postInfosLink') # Many-to-One with threadInfo
authorInfos_id = Column(Integer, ForeignKey('authorInfos.id')) # Many-to-One with authorInfo
authorInfosLink = relationship('AuthorInfo', back_populates='postInfosLink') # Many-to-One with authorInfo
class AuthorInfo(Base):
__tablename__ = 'authorInfos'
id = Column(Integer, primary_key=True)
threadAuthor = Column('threadAuthor', String())
postInfosLink = relationship('PostInfo', back_populates='authorInfosLink') # One-to-Many with postInfo
threadInfosLink = relationship('ThreadInfo', back_populates='authorInfosLink') # One-to-Many with threadInfo
Here is my pipelines.py
from sqlalchemy import exists, event
from sqlalchemy.orm import sessionmaker
from scrapy.exceptions import DropItem
from .models import db_connect, create_table, BoardInfo, ThreadInfo, PostInfo, AuthorInfo
from sqlalchemy.engine import Engine
from sqlite3 import Connection as SQLite3Connection
import logging
#event.listens_for(Engine, "connect")
def _set_sqlite_pragma(dbapi_connection, connection_record):
if isinstance(dbapi_connection, SQLite3Connection):
cursor = dbapi_connection.cursor()
cursor.execute("PRAGMA foreign_keys=ON;")
# print("####### PRAGMA prog is running!! ######")
cursor.close()
class DuplicatesPipeline(object):
def __init__(self):
'''
Initializes database connection and sessionmaker.
Creates tables.
'''
engine = db_connect()
create_table(engine)
self.Session = sessionmaker(bind=engine)
logging.info('****DuplicatesPipeline: database connected****')
def process_item(self, item, spider):
session = self.Session()
exist_threadLink = session.query(exists().where(ThreadInfo.threadLink == item['threadLink'])).scalar()
exist_thread_replyCount = session.query(ThreadInfo.replyCount).filter_by(threadLink = item['threadLink']).scalar()
if exist_threadLink is True: # threadLink is in DB
if exist_thread_replyCount < item['replyCount']: # check if replyCount is more?
return item
session.close()
else:
raise DropItem('Duplicated item found and replyCount is not changed')
session.close()
else: # New threadLink to be added to BoardPipeline
return item
session.close()
class BoardPipeline(object):
def __init__(self):
'''
Initializes database connection and sessionmaker
Creates tables
'''
engine = db_connect()
create_table(engine)
self.Session = sessionmaker(bind=engine)
def process_item(self, item, spider):
'''
Save scraped info in the database
This method is called for every item pipeline component
'''
session = self.Session()
# Input info to boardInfos
boardInfo = BoardInfo()
boardInfo.boardName = item['boardName']
# Input info to threadInfos
threadInfo = ThreadInfo()
threadInfo.threadTitle = item['threadTitle']
threadInfo.threadLink = item['threadLink']
threadInfo.threadAuthor = item['threadAuthor']
threadInfo.threadPost = item['threadPost']
threadInfo.replyCount = item['replyCount']
threadInfo.readCount = item['readCount']
# Input info to postInfos
# Due to info is in list, so we have to loop and add it.
for num in range(len(item['postOrder'])):
postInfoNum = 'postInfo' + str(num)
postInfoNum = PostInfo()
postInfoNum.postOrder = item['postOrder'][num]
postInfoNum.postAuthor = item['postAuthor'][num]
postInfoNum.postContent = item['postContent'][num]
postInfoNum.postTimestamp = item['postTimestamp'][num]
session.add(postInfoNum)
# Input info to authorInfo
authorInfo = AuthorInfo()
authorInfo.threadAuthor = item['threadAuthor']
# check whether the boardName exists
exist_boardName = session.query(exists().where(BoardInfo.boardName == item['boardName'])).scalar()
if exist_boardName is False: # the current boardName does not exists
session.add(boardInfo)
# check whether the threadAuthor exists
exist_threadAuthor = session.query(exists().where(AuthorInfo.threadAuthor == item['threadAuthor'])).scalar()
if exist_threadAuthor is False: # the current threadAuthor does not exists
session.add(authorInfo)
try:
session.add(threadInfo)
session.commit()
except:
session.rollback()
raise
finally:
session.close()
return item
From the code I can see, it doesn't look to me like you are setting ThreadInfo.authorInfosLink or ThreadInfo.authorInfos_id anywhere (the same goes for all of your FK/relationships).
For the related objects to be attached to a ThreadInfo instance, you need to create them and then attach them something like:
# Input info to authorInfo
authorInfo = AuthorInfo()
authorInfo.threadAuthor = item['threadAuthor']
threadInfo.authorInfosLink = authorInfo
You probably don't want to session.add() each object if it's related via FK. You'll want to:
instantiate a BoardInfo object bi
then instantiate attach your related ThreadInfo object ti
attach your the related object eg bi.threadInfosLink = ti
At the end of all of your chained relationships, you can simply add bi to the session using session.add(bi) -- all of the related objects will be added through their relationships and the FKs will be correct.
Per the discussion in the comments of my other answer, below is how I would rationalize your models to make them make more sense to me.
Notice:
I have removed the unnecessary "Info" everywhere
I have removed explicit column names from your model definitions and will rely instead on SQLAlchemy's ability to infer those for me based on my attribute names
In a "Post" object I do not name the attribute PostContent, it's implied that the content relates to the Post because that's how we're accessing it -- instead simply call the attribute "Post"
I've removed all "Link" terminology -- in places where I think you want a reference to a collection of related objects I've provided a plural attribute of that object as the relationship.
I've left a line in the Post model for you to remove. As you can see, you don't need "author" twice -- once as a related object and once on the Post, that defeats the purpose of the FKs.
With these changes, when you attempt to use these models from your other code it becomes obvious where you need to use .append() and where you simply assign the related object. For a given Board object you know that 'threads' is a collection just based on the attribute name, so you're going to do something like b.threads.append(thread)
from sqlalchemy import create_engine, Column, Table, ForeignKey, MetaData
from sqlalchemy import Integer, String, Date, DateTime, Float, Boolean, Text
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
class Board(Base):
__tablename__ = 'board'
id = Column(Integer, primary_key=True)
name = Column(String(100))
threads = relationship(back_populates='board')
class Thread(Base):
__tablename__ = 'thread'
id = Column(Integer, primary_key=True)
title = Column(String())
link = Column(String())
author = Column(String())
post = Column(Text())
reply_count = Column(Integer)
read_count = Column(Integer)
board_id = Column(Integer, ForeignKey('Board.id'))
board = relationship('Board', back_populates='threads')
posts = relationship('Post', back_populates='threads')
author_id = Column(Integer, ForeignKey('Author.id'))
author = relationship('Author', back_populates='threads')
class Post(Base):
__tablename__ = 'post'
id = Column(Integer, primary_key=True)
order = Column(Integer, nullable=True)
author = Column(Text(), nullable=True) # remove this line and instead use the relationship below
content = Column(Text(), nullable=True)
timestamp = Column(Text(), nullable=True)
thread_id = Column(Integer, ForeignKey('Thread.id'))
thread = relationship('Thread', back_populates='posts')
author_id = Column(Integer, ForeignKey('Author.id'))
author = relationship('Author', back_populates='posts')
class AuthorInfo(Base):
__tablename__ = 'author'
id = Column(Integer, primary_key=True)
name = Column(String())
posts = relationship('Post', back_populates='author')
threads = relationship('Thread', back_populates='author')
Problem: Simply put, I am trying to redefine a SQLAlchemy ORM table's primary key after it has already been defined.
Example:
class Base:
#declared_attr
def __tablename__(cls):
return f"{cls.__name__}"
#declared_attr
def id(cls):
return Column(Integer, cls.seq, unique=True,
autoincrement=True, primary_key=True)
Base = declarative_base(cls=Base)
class A_Table(Base):
newPrimaryKeyColumnsDerivedFromAnotherFunction = []
# Please Note: as the variable name tries to say,
# these columns are auto-generated and not known until after all
# ORM classes (models) are defined
# OTHER CLASSES
def changePriKeyFunc(model):
pass # DO STUFF
# Then do
Base.metadata.create_all(bind=arbitraryEngine)
# After everything has been altered and tied into a little bow
*Please note, this is a simplification of the true problem I am trying to solve.
Possible Solution: Your first thought might have been to do something like this:
def possibleSolution(model):
for pricol in model.__table__.primary_key:
pricol.primary_key = False
model.__table__.primary_key = PrimaryKeyConstraint(
*model.newPrimaryKeyColumnsDerivedFromAnotherFunction,
# TODO: ADD all the columns that are in the model that are also a primary key
# *[col for col in model.__table__.c if col.primary_key]
)
But, this doesn't work, because when trying to add, flush, and commit, an error gets thrown:
InvalidRequestError: Instance <B_Table at 0x104aa1d68> cannot be refreshed -
it's not persistent and does not contain a full primary key.
Even though this:
In [2]: B_Table.__table__.primary_key
Out[2]: PrimaryKeyConstraint(Column('a_TableId', Integer(),
ForeignKey('A_Table.id'), table=<B_Table>,
primary_key=True, nullable=False))
as well as this:
In [3]: B_Table.__table__
Out[3]: Table('B_Table', MetaData(bind=None),
Column('id', Integer(), table=<B_Table>, nullable=False,
default=Sequence('test_1', start=1, increment=1,
metadata=MetaData(bind=None))),
Column('a_TableId', Integer(),
ForeignKey('A_Table.id'), table=<B_Table>,
primary_key=True, nullable=False),
schema=None)
and finally:
In [5]: b.a_TableId
Out[5]: 1
Also note that the database actually reflects the changed (and true) primary key, so I know that there's something going on with the ORM/SQLAlchemy.
Question: In summary, how can I change the model's primary key after the model has already been defined?
edit: See below for full code (same type of error, just in SQLite)
from sqlalchemy import Column, Integer, ForeignKey
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declared_attr, declarative_base
from sqlalchemy.schema import PrimaryKeyConstraint
from sqlalchemy import Sequence, create_engine
class Base:
#declared_attr
def __tablename__(cls):
return f"{cls.__name__}"
#declared_attr
def seq(cls):
return Sequence("test_1", start=1, increment=1)
#declared_attr
def id(cls):
return Column(Integer, cls.seq, unique=True, autoincrement=True, primary_key=True)
Base = declarative_base(cls=Base)
def relate(model, x):
"""Model is the original class, x is what class needs to be as
an attribute for model"""
attributeName = x.__tablename__
idAttributeName = "{}Id".format(attributeName)
setattr(model, idAttributeName,
Column(ForeignKey(x.id)))
setattr(model, attributeName,
relationship(x,
foreign_keys=getattr(model, idAttributeName),
primaryjoin=getattr(
model, idAttributeName) == x.id,
remote_side=x.id
)
)
return model.__table__.c[idAttributeName]
def possibleSolution(model):
if len(model.defined):
newPriCols = []
for x in model.defined:
newPriCols.append(relate(model, x))
for priCol in model.__table__.primary_key:
priCol.primary_key = False
priCol.nullable = True
model.__table__.primary_key = PrimaryKeyConstraint(
*newPriCols
# TODO: ADD all the columns that are in the model that are also a primary key
# *[col for col in model.__table__.c if col.primary_key]
)
class A_Table(Base):
pass
class B_Table(Base):
defined = [A_Table]
possibleSolution(B_Table)
engine = create_engine('sqlite://')
Base.metadata.create_all(bind=engine)
Session = sessionmaker(bind=engine)
session = Session()
a = A_Table()
b = B_Table(A_TableId=a.id)
print(B_Table.__table__.primary_key)
session.add(a)
session.commit()
session.add(b)
session.commit()
Originally, the error you say the PK reassignment is causing is:
InvalidRequestError: Instance <B_Table at 0x104aa1d68> cannot be refreshed -
it's not persistent and does not contain a full primary key.
I don't get that running you MCVE, instead I get a pretty helpful warning first:
SAWarning: Column 'B_Table.A_TableId' is marked as a member of the
primary key for table 'B_Table', but has no Python-side or server-side
default generator indicated, nor does it indicate 'autoincrement=True'
or 'nullable=True', and no explicit value is passed. Primary key
columns typically may not store NULL.
And a very detailed exception message when the script fails:
sqlalchemy.orm.exc.FlushError: Instance has
a NULL identity key. If this is an auto-generated value, check that
the database table allows generation of new primary key values, and
that the mapped Column object is configured to expect these generated
values. Ensure also that this flush() is not occurring at an
inappropriate time, such as within a load() event.
So assuming that the example accurately describes your problem, the answer is straightforward. A primary key cannot be null.
A_Table inherits off Base:
class A_Table(Base):
pass
Base gives A_Table an autoincrement PK through declared_attr id():
#declared_attr
def id(cls):
return Column(Integer, cls.seq, unique=True, autoincrement=True, primary_key=True)
Similarly, B_Table is defined off Base but the PK is overwritten in possibleSolution() such that it becomes a ForeignKey to A_Table:
PrimaryKeyConstraint(Column('A_TableId', Integer(), ForeignKey('A_Table.id'), table=<B_Table>, primary_key=True, nullable=False))
Then, we instantiate an instance of A_Table without any kwargs and immediately allocate the id attribute of instance a to field A_TableId when constructing b:
a = A_Table()
b = B_Table(A_TableId=a.id)
At this point we can stop and inspect the attribute values of each:
print(a.id, b.A_TableId)
# None None
a.id is None because it's an autoincrement which needs to be populated by the database, not the ORM. So SQLAlchemy doesn't know it's value until after the instance is flushed to the database.
So what happens if we include a flush() operation after adding instance a to the session:
a = A_Table()
session.add(a)
session.flush()
b = B_Table(A_TableId=a.id)
print(a.id, b.A_TableId)
# 1 1
So by issuing the flush first, we've got a value for a.id, meaning that we also have a value for b.A_TableId.
session.add(b)
session.commit()
# no error
In my Rest application I want to return json like JSONAPI format, but I need to create Schema class for it and create every field again that are already there in my model. So instead of creating every field in schema class can I not take it from DB Model..
below is my model class
class Author(db.Model):
id = db.Column(db.Integer)
name = db.Column(db.String(255))
I am defining Schema like below.
class AuthorSchema(Schema):
id = fields.Str(dump_only=True)
name = fields.Str()
metadata = fields.Meta()
class Meta:
type_ = 'people'
strict = True
So here, id and name I have defined it twice. so is there any option in marshmallow-jsonapi to assign model name in schema class so it can take all fields from model
Note: I am using marshmallow-jsonapifor it, I have tried marshmallow-sqlalchemy , it has that option but it not return json in JSONAPI format
You can use flask-marshmallow's ModelSchema and marshmallow-sqlalchemy in combination with marshmallow-jsonapi with the caveat that you have to subclass not only the Schema classes but also the SchemaOpts classes, like this:
# ...
from flask_marshmallow import Marshmallow
from marshmallow_jsonapi import Schema, SchemaOpts
from marshmallow_sqlalchemy import ModelSchemaOpts
# ...
ma = Marshmallow(app)
# ...
class JSONAPIModelSchemaOpts(ModelSchemaOpts, SchemaOpts):
pass
class AuthorSchema(ma.ModelSchema, Schema):
OPTIONS_CLASS = JSONAPIModelSchemaOpts
class Meta:
type_ = 'people'
strict = True
model = Author
# ...
foo = AuthorSchema()
bar = foo.dump(query_results).data # This will be in JSONAPI format including every field in the model
I am using the concrete table inheritance with SQLAlchemy. In declartive style model class, I have configured it successfully.
My code just like:
class Entry(AbstractConcreteBase, db.Model):
"""Base Class of Entry."""
id = db.Column(db.Integer, primary_key=True, nullable=False)
created = db.Column(db.DateTime, nullable=False)
post_id = declared_attr(lambda c: db.Column(db.ForeignKey("post.id")))
post = declared_attr(lambda c: db.relationship("Post", lazy="joined"))
#declared_attr
def __tablename__(cls):
return cls.__name__.lower()
#declared_attr
def __mapper_args__(cls):
# configurate subclasses about concrete table inheritance
return {'polymorphic_identity': cls.__name__,
'concrete': True} if cls.__name__ != "Entry" else {}
class TextEntry(Entry):
"""Text and Article Entry."""
text = db.deferred(db.Column(db.Text, nullable=False))
class PhotoEntry(Entry):
"""Photo Entry."""
path = db.deferred(db.Column(db.String(256), nullable=False))
It works fine while testing it in the shell:
>>> from models.entry import Entry
>>>
>>> Entry.query.all()
[<PhotoEntry 'Title' created by tonyseek>,
<PhotoEntry 'TITLE 2' created by tonyseek>,
<PhotoEntry 'Title 3' created by tonyseek>,
<PhotoEntry 'Title 4' created by tonyseek>,
<TextEntry 'Title' created by tonyseek>]
Then I fall into trouble while setting the relationship in other models. Each entry has a foreign key post_id to join Post model, but I could not define the back reference in Post. That can't work:
class Post(db.Model):
"""An Post."""
id = db.Column(db.Integer, primary_key=True, nullable=False)
description = db.Column(db.Unicode(140), nullable=False)
entries = db.relationship(Entry, lazy="dynamic")
It raised a Exception and said:
InvalidRequestError: One or more mappers failed to initialize - can't proceed with initialization of other mappers. Original exception was: Class 'models.entry.Entry' is not mapped.
Obvious the Entry is a abstract class, which couldn't be mapped to a real exist table. The document in official website has a example but its base class is not abstract. Now how should I do to set the polymorphic relationship with a abstract model?
I have found the reason of the problem and its solution.
According to the document of sqlalchemy offical website, the abstract class could be a mapped class, because the polymorphic_union function could create a virtual table.
I am using the declartive style model, not build mapper by hand, so the virtual table pjoin should not be created by hand. The base class AbstractConcreteBase has a method __delcare_last__ would create the pjoin with polymorphic_union function, but it would be called while the event after_configured triggering.
The relationship with Entry in Post would be created after the Post class be generated, in this time the event after_configured have not been triggered, so __delcare_last__ function have not created the virtual table pjoin and mapped it into Entry. So the exception "Class 'models.entry.Entry' is not mapped." will be raised.
Now, I refactor the Post model, let it create the relationship with Entry in __delcare_last__ function, then it will be success because of the triggered event and the mapped Entry.
My new implemented class like this:
class Post(db.Model):
"""An Post."""
id = db.Column(db.Integer, primary_key=True, nullable=False)
description = db.Column(db.Unicode(140), nullable=False)
#classmethod
def __declare_last__(cls):
cls.entries = db.relationship(Entry, viewonly=True)
def attach_entries(self, entries):
"""Attach Entries To This Post.
Example:
>>> post = Post("An Interesting News", "Wow !!!")
>>> text_entry = TextEntry(*t_args)
>>> photo_entry = PhotoEntry(*p_args)
>>> post.attach_entries([text_entry, photo_entry])
>>> len(post.entries)
2
>>> db.session.commit()
>>>
"""
for entry in entries:
self.entries.append(entry)
entry.post = self
db.session.add(entry)