We Keep Coding

Loading a SQL Alchemy Model with relationships without triggering an insert

The problem I am having is When I create my SQLAlchemy model without loading the relationships and immediately commit after, no action is taken as expected (Great). But when I create it with the relationships loaded (based on some ids passed in) and then immediately commit, it tries to insert the data into the database.
Example:
Hero(id=event.stream_id,
name=event.event_payload["name"])
db.commit()
^ In the above nothing happens, which makes sense because I haven't added the model to the session or tried to insert into a table.
Hero(id=event.stream_id,
name=event.get("name"),
teams=[teams.get(p) for p in event.get("teams")])
db.commit()
^ This fails with a duplicate key exception as for some reason it tries to now insert the model into the DB. Is this expected behaviour? Can it be turned off?
Note:teams.get returns a list of Team()
Models for reference:
class HeroTeamLink(SQLModel, table=True):
__tablename__ = "hero_cr_person"
hero_id: Optional[str] = Field(
default=None, foreign_key="hero.id", primary_key=True
)
team_id: Optional[str] = Field(
default=None, foreign_key="team.id", primary_key=True
)
class Hero(SQLModel, table=True):
__tablename__ = "hero"
id: str
name: str
teams: List["Team"] = Relationship(
back_populates="heroes", link_model=HeroTeamLink
)
class Team(SQLModel, table=True):
__tablename__ = "team"
team_name: str
heroes: List[Hero] = Relationship(
back_populates="teams", link_model=HeroTeamLink
)

Pydantic/SQLAlchemy: How to work with enums?

What is the best way to convert a sqlalchemy model to a pydantic schema (model) if it includes an enum field?
Sqlalchemy
import enum
from sqlalchemy import Enum, Column, String
from sqlalchemy.orm import declarative_base
Base = declarative_base()
class StateEnum(enum.Enum):
CREATED = 'CREATED'
UPDATED = 'UPDATED'
class Adapter(Base):
__tablename__ = 'adapters'
id = Column(String, primary_key=True)
friendly_name = Column(String(256), nullable=False)
state: StateEnum = Column(Enum(StateEnum))
Pydantic
from pydantic import BaseModel
from enum import Enum
class StateEnumDTO(str, Enum):
CREATED = 'CREATED'
UPDATED = 'UPDATED'
class AdapterDTO(BaseModel):
friendly_name: str
state: StateEnumDTO # This currently cannot be converted?
class Config:
allow_population_by_field_name = True
orm_mode = True
use_enum_values = True
Conversion
AdapterDTO.from_orm(Adapter(friendly_name='test', state=StateEnum.CREATED))
This leads to the error
value is not a valid enumeration member; permitted: 'CREATED', 'UPDATED' (type=type_error.enum; enum_values=[<StateEnumDTO.CREATED: 'CREATED'>, <StateEnumDTO.UPDATED: 'UPDATED'>])
How can I configure either
a.) the serialization with the from_orm method?
or
b.) the creation of the state field?
c.) How to convert it the other way around?
Is there a native way to do this with pydantic or how is this typically done?
Update:
Test case
def test_enum_conversion_to_dto():
adapter = Adapter(id='1', friendly_name='test', state=StateEnum.CREATED)
adapter_dto = AdapterDTO.from_orm(adapter)
assert adapter_dto.state == StateEnumDTO.CREATED
assert adapter_dto.state.value == StateEnum.CREATED.value

Pydantic requires that both enum classes have the same type definition.
In your case, StateEnum inherits from enum.Enum, but StateEnumDTO inherits from both str and enum.Enum.
You can fix this issue by changing your SQLAlchemy enum definition:
class StateEnum(str, enum.Enum):
CREATED = 'CREATED'
UPDATED = 'UPDATED'

You must add arbitrary_types_allowed = True
To the model Config class.
from pydantic import BaseModel
from enum import Enum
class StateEnumDTO(str, Enum):
CREATED = 'CREATED'
UPDATED = 'UPDATED'
class AdapterDTO(BaseModel):
friendly_name: str
state: StateEnumDTO # This currently cannot be converted?
class Config:
allow_population_by_field_name = True
orm_mode = True
use_enum_values = True
arbitrary_types_allowed = True

ManyToMany with SQLalchemy in Fastapi

I have a following models with many-to-many relations:
dashboard_customer_association = Table(
"entry_customer",
Base.metadata,
Column("entry_id", ForeignKey("entry.id"), primary_key=True),
Column("customer_id", ForeignKey("customer.id"), primary_key=True),
)
class Customer(Base):
__tablename__ = "customer"
id = Column(Integer, primary_key=True, index=True, autoincrement=True)
name = Column(String(64), unique=True, index=True)
class Entry(Base):
__tablename__ = "entry"
id = Column(String(16), primary_key=True, index=True)
customer = relationship("Customer", secondary=dashboard_customer_association)
Here's my pydantic schema.
class Entry(BaseModel):
id: str
customer: List[str] = []
class Config:
orm_mode = True
I've managed to insert the data and create the customers alongside,
but the problem is when I'm trying to retrieve data:
pydantic.error_wrappers.ValidationError: 2 validation errors for Entry
response -> customer -> 0
str type expected (type=type_error.str)
response -> customer -> 1
str type expected (type=type_error.str)
I understand that the Customer object is not a string, so customer
field cannot be directly serialized as List[str], but I fail to see
how am I supposed to do the conversion.
I return the data with the following function:
def get_data(item_id):
instance = db.query(models.Entry).filter(models.Entry.id == item_id).first()
return instance
I was trying to set instance.customer = [customer.name for customer in instance.customer],
but SQLalchemy prevents that. What is the right way to do that?

The best way would be to simply match the schema to the returned data and have a Customer object as well.
However, if that is not an option, you can use a validator to change the content when it's being populated - i.e. just return a single value from your Customer object.
#validator('customer')
def customer_as_string(cls, v):
return v.name

sqlalchemy [ORM] - add/inset to multiple tables

I've created the schema and 2 tables models with the relation between them.
class SubscriberBase(BaseModel):
subscriber_no: int
is_active: bool = False
class SubscriberCreate(BaseModel):
pass
class Subscriber(SubscriberBase):
owner: int
class Config:
orm_mode = True
class CustomerCreate(BaseModel):
customer_no: int
subscriber: Optional[List[SubscriberBase]] = None
class Customer(CustomerCreate):
id: int
class Config:
orm_mode = True
Models:
class CustomerModel(Base):
__tablename__ = 'customer'
id = Column(Integer, primary_key=True, index=True)
customer_no= Column(Integer,index=True)
subscriber= relationship("SubscriberModel", back_populates="owner")
class SubscriberModel(Base):
__tablename__ = 'subscriber'
id = Column(Integer, ForeignKey("customer.id"))
subscriber_no= Column(Integer, primary_key=True, index=True)
owner = relationship("CustomerModel", back_populates="subscriber")
for the following dictionary input:
test = {'customer_no': 2, 'subscriber': [{'subscriber_no': 2, 'is_active': False}, {'subscriber_no': 1, 'is_active': False}]}
I expect , it will insert one row to customer table ,
and 2 rows in subscriber table.
tried:
db_customer = models.CustomerModel(**test)
db.add(db_customer)
db.commit()
db.refresh(db_customer)
Getting error :
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 1675, in emit_backref_from_collection_append_event
child_state, child_dict = instance_state(child), instance_dict(child)
AttributeError: 'dict' object has no attribute '_sa_instance_state'
How does sqlalchemy orm insert to multiple tables?
Is it possible to insert both customer and subscriber tables in one insert ?
#van
Updating the post with more Info:
Im Working with fastapi where the "customer" (the "test" dict i mention above)
is is actually the pydantic , and i did tried the following:
#customer_router.post("/customer/")
def overloaded_create_customer(customer: CustomerCreate, db: Session = Depends(get_db)):
db_customer = CustomerModel(**dict(customer))
db.add(db_customer)
db.commit()
db.refresh(db_customer)
It return similar error:
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 1675, in emit_backref_from_collection_append_event
child_state, child_dict = instance_state(child), instance_dict(child)
AttributeError: 'SubscriberBase' object has no attribute '_sa_instance_state'

Creating a ORM model instance (like you try with the CustomerModel) is not the way to do it. Instead, with the assumption that those are pydantic schemas, you should try something along these lines:
# create pydantic model from the dictionary
cust_schema = schemas.Customer.parse_obj(test)
# convert to ORM model
db_customer = models.CustomerModel(**dict(cust_schema))
# ... (rest of your code)

Take JSONAPI schema from model

In my Rest application I want to return json like JSONAPI format, but I need to create Schema class for it and create every field again that are already there in my model. So instead of creating every field in schema class can I not take it from DB Model..
below is my model class
class Author(db.Model):
id = db.Column(db.Integer)
name = db.Column(db.String(255))
I am defining Schema like below.
class AuthorSchema(Schema):
id = fields.Str(dump_only=True)
name = fields.Str()
metadata = fields.Meta()
class Meta:
type_ = 'people'
strict = True
So here, id and name I have defined it twice. so is there any option in marshmallow-jsonapi to assign model name in schema class so it can take all fields from model
Note: I am using marshmallow-jsonapifor it, I have tried marshmallow-sqlalchemy , it has that option but it not return json in JSONAPI format

You can use flask-marshmallow's ModelSchema and marshmallow-sqlalchemy in combination with marshmallow-jsonapi with the caveat that you have to subclass not only the Schema classes but also the SchemaOpts classes, like this:
# ...
from flask_marshmallow import Marshmallow
from marshmallow_jsonapi import Schema, SchemaOpts
from marshmallow_sqlalchemy import ModelSchemaOpts
# ...
ma = Marshmallow(app)
# ...
class JSONAPIModelSchemaOpts(ModelSchemaOpts, SchemaOpts):
pass
class AuthorSchema(ma.ModelSchema, Schema):
OPTIONS_CLASS = JSONAPIModelSchemaOpts
class Meta:
type_ = 'people'
strict = True
model = Author
# ...
foo = AuthorSchema()
bar = foo.dump(query_results).data # This will be in JSONAPI format including every field in the model