I'm trying to have my Pydantic/ORM models "output" labels,but when using SqlAlchemy ORM I feel a bit locked/ stuck.
I want the fieldname "test1" ( below in the code) to return ( "Test left side") instead of test1 in the JSONResponse
As an explanation, In sql i would for example use “as”
select test1 as ‘Test left side’ from pretest
I know I can do the same in an ORM statement/query, but I want it to be reachable as a field/ attribute from the orm model class, or perhaps as some validation methods from my Pydantic model.
To explain I have added an example of two short models below
my model in Sqlalchemy:
class Pretest(Base):
__tablename__ = "pretest"
user_id = Column(Integer)
pretest_id = Column(Integer, primary_key=True)
timestamp_pretest = Column(DateTime(timezone=True), default=func.now())
test1 = Column(Integer)
my model in Pydantic: ( I use Optional because im testing at the moment)
class Pretest(BaseModel):
user_id: Optional[int] = None
pretest_id: Optional[int] = None
timestamp_pretest: Optional[datetime] = None
test1: Optional[int] = None
class Config:
orm_mode= True
So I’m wondering if Pydantic have a possibility to validate against a labelslist/array/ object that could contain a type of test1= “Test left side”
Or if the ORM models have some additional metadata that could be used like
test1 = Column(Integer, alias=“Test left side”)
I hope I make this question understandable?
My endpoint look a bit simplified something like this:
#router.post("/pretest", tags=["Medicaldata"], status_code=status.HTTP_200_OK)
def pretest(pretest: Pretest, token: str = Depends(oauth2_scheme)):
try:
query = db.query(models.Pretest).first()
except:
query = "query failed"
return JSONResponse(content=query)
Where I use the pretest-object which is type defined by the Pydantic model as query parameters( not shown here)
This response will create a json-object of the fields and values in the database.
The field/variable test1 will return as test1 instead of "Test left side", since I do not have a place to add labels or some sort of aliases.
I can add and map the json object manually in Python before I return it, but it’s a lot of complex queries spanning several tables, so it feels a bit “wrong” to do it that way.
The reason for all this is so that I can have model and label consistency and use the map function with spread operators in components in React as shown below.
get_backend(“/pretest”,data)
.then setPretestlist(response)
{pretestlist.map((item) => {
return <ShowPretest {...item} key={item.name} />;
})}
This will now show as test1 in the webpage instead of a more explanatory text like this "Test 1 left side"
#snakecharmerb, Thx, you put me on the right track.
The solution, if someone else wonder:
the Pydantic model needs to be changed from this:
class Pretest(BaseModel):
user_id: Optional[int] = None
pretest_id: Optional[int] = None
timestamp_pretest: Optional[datetime] = None
test1: Optional[int] = None
class Config:
orm_mode = True
To this:
class Pretest(BaseModel):
user_id: Optional[int] = None
pretest_id: Optional[int] = None
timestamp_pretest: Optional[datetime] = None
test1: Optional[int] = None
class Config:
fields = {
"test1": "Test left side",
"timestamp_pretest": "Time tested",
}
orm_mode = True
The endpoint needed to change in the way it does its response to this:
#router.post("/pretest", tags=["Medicaldata"], status_code=status.HTTP_200_OK)
def pretest(pretest: Pretest, token: str = Depends(oauth2_scheme)):
try:
query = db.query(models.Pretest).first()
query = Pretest.from_orm(query)
except:
query = "query failed"
return query.dict(by_alias=True)
Related
Say I have a domain model with an id field plus eq and hash. Then there is a simple SqlAlchemy ORM mapping.
#dataclass
class Foo:
_id: int
value: 50
def __eq__(self, other):
if not isinstance(other, Foo):
return False
return other._id == self._id
def __hash__(self):
return hash(self._id)
foo = Table(
"foo",
mapper_registry.metadata,
Column("_id", Integer, primary_key=True, autoincrement=True),
Column("value", Float, nullable=False)
mapper_registry.map_imperatively(Foo, foo)
This seems simple enough and follows the documentation for SqlAlchemy. My problem is the _id column/property. If I create a test to load Foo from the database:
def test_foo_mapper_can_load_foos(session):
with session:
session.execute(
'INSERT INTO foo (value)'
'VALUES (50)'
)
session.commit()
expected = [
Foo(_id=1, value=50),
]
assert session.query(Foo).all() == expected
this works, fine. The model is initialised with ids from the database.
But, what about initialising a model to commit to the database. If the client creates a new foo to try to write to the database, how should I approach the id for the model, before gets committed?
def test_foo_mapper_can_save_foos(session):
#option1 - manually set it (collides with auto_increment)
new_foo = Foo(_id=1, value: 50)
#option2 - set to None (collides with __eq__/hashing)
new foo = Foo(_id=None, value: 50)
#option3 - get rid of id from domain model and only have it in db
new_foo = Foo(value: 50)
session.add(new_foo)
session.commit()
rows = list(session.execute('SELECT * FROM "foo"'))
assert rows == [(1, 50)]
Each of the test options can work, but none of the implementations seem like good code.
In option 1, when the client creates a new foo an id must be set, the dataclass requires it in the constructor... but it seems to not really be in line with the auto_increment primary key idea on the table - the client can pass any id, whether it is the next in sequence or not and the mapper will try to use it. I feel the database should be responsible for setting primary key.
So, on to option 2. Set the id to None, and the database will take care of it on commit. However, the eq and hash functions rely on id for equality and the object becomes unhashable. This could also be done by setting _id: int = None as a default value on the domain model itself. But again, seems like a bad solution.
Finally option 3... remove the _id field from the domain model... which has popped up in a couple of articles, but also seem less than ideal as Foo now has no unique id for use in select statements, and use in other business logic etc...
I'm sure I'm thinking about this all wrong, I just can't figure out where.
I have the following code for the Beanie ODM (for Python):
class PlanetDocument(Document):
created_at: datetime
name: str = "Planet"
class UserDocument(Document):
id: Indexed(str)
username: Optional[str] = None
planets: List[PlanetDocument] = []
class Settings:
name = "user"
When I store a PlanetDocument, I can see it has it's id, but when I store the same planet on the UserDocument planets list the object doesnt has id, I store it in user the following way:
user = await UserDocument.get(planet_data.user)
new_planet = PlanetDocument("time", "name")
new_planet = await new_planet.create()
user.planets.append(new_planet)
await user.save()
Then I lookup user[0].planets[0] and it has not it's id set
This solved my issue:
https://roman-right.github.io/beanie/tutorial/relations/
Instead of using normal typing, I missed to setup relation by using Link.
I'm just getting started with ORM SQLAlchemy / SQLModel and I wonder if there is a shortcut function to get an entity from a session given all columns except the primary key.
For example, right now i have this (which works):
from sqlmodel import SQLModel, Field, create_engine, Session, select
class Hero(SQLModel, table=True):
id: int | None = Field(default=None, primary_key=True)
name: str
secret_name: str
engine = create_engine("sqlite:///hero_db.db")
def get_hero(hero: Hero) -> Hero:
with Session(engine) as session:
return session.exec(
select(Hero).where(
Hero.name == hero.name,
Hero.secret_name == hero.secret_name,
)
).one()
And i wonder if there is a shortcut without explicitly listing every column. For example like this:
...
return session.exec(
select(Hero).where(
Hero == hero
)
).one()
Or even better like this:
...
return session.get_reverse(Hero, hero)
Assume the following setup:
from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class MyClass(Base):
id = Column(Integer, primary_key=True)
name = Column(String)
The normal paradigm to query the DB with SQLAlchemy is to do the following:
Session = sessionmaker()
engine = 'some_db_location_string'
session = Session(bind=engine)
session.query(MyClass).filter(MyClass.id == 1).first()
Suppose, I want to simplify the query to the following:
MyClass(s).filter(MyClass.id == 1).first()
OR
MyClass(s).filter(id == 1).first()
How would I do that? My first attempt at that to use a model Mixin class failed. This is what I tried:
class ModelMixins(object)
def __init__(self, session):
self.session = session
def filter(self, *args):
self.session.query(self).filter(*args)
# Redefine MyClass to use the above class
class MyClass(ModelMixins, Base):
id = Column(Integer, primary_key=True)
name = Column(String)
The main failure seems to be that I can't quite transfer the expression 'MyClass.id == 1' to the actual filter function that is part of the session object.
Folks may ask why would I want to do:
MyClass(s).filter(id == 1).first()
I have seen something similar like this used before and thought that the syntax becomes so much cleaner I can achieve this. I wanted to replicate this but have not been able to. Being able to do something like this:
def get_stuff(some_id):
with session_scope() as s:
rec = MyClass(s).filter(MyClass.id== some_id').first()
if rec:
return rec.name
else:
return None
...seems to be the cleanest way of doing things. For one, session management is kept separate. Secondly, the query itself is simplified. Having a Mixin class like this would allow me to add the filter functionality to any number of classes...So can someone help in this regard?
session.query takes a class; you're giving it self, which is an instance. Replace your filter method with:
def filter(self, *args):
return session.query(self.__class__).filter(*args)
and at least this much works:
In [45]: MyClass(session).filter(MyClass.id==1)
Out[45]: <sqlalchemy.orm.query.Query at 0x10e0bbe80>
The generated SQL looks right, too (newlines added for clarity):
In [57]: str(MyClass(session).filter(MyClass.id==1))
Out[57]: 'SELECT "MyClass".id AS "MyClass_id", "MyClass".name AS "MyClass_name"
FROM "MyClass"
WHERE "MyClass".id = ?'
No guarantees there won't be oddities; I've never tried anything like this before.
Ive been using this mixin to good success. Most likely not the most efficient thing in the world and I am no expert. I define a date_created column for every table
class QueryBuilder:
"""
This class describes a query builer.
"""
q_debug = False
def query_from_dict(self, db_session: Session, **q_params: dict):
"""
Creates a query.
:param db_session: The database session
:type db_session: Session
:param q_params: The quarter parameters
:type q_params: dictionary
"""
q_base = db_session.query(type(self))
for param, value in q_params.items():
if param == 'start_date':
q_base = q_base.filter(
type(self).__dict__.get('date_created') >= value
)
elif param == 'end_date':
q_base = q_base.filter(
type(self).__dict__.get('date_created') <= value
)
elif 'like' in param:
param = param.replace('_like', '')
member = type(self).__dict__.get(param)
if member:
q_base = q_base.filter(member.ilike(f'%{value}%'))
else:
q_base = q_base.filter(
type(self).__dict__.get(param) == value
)
if self.q_debug:
print(q_base)
return q_base
I made this statement using flask-sqlalchemy and I've chosen to keep it in its original form. Post.query is equivalent to session.query(Post)
I attempted to make a subquery that would filter out all posts in a database which are in the draft state and not made or modified by the current user. I made this query,
Post.query\
.filter(sqlalchemy.and_(
Post.post_status != Consts.PostStatuses["Draft"],
sqlalchemy.or_(
Post.modified_by_id == current_user.get_id(),
Post.created_by_id == current_user.get_id()))
which created:
Where true AND ("Post".modified_by_id = :modified_by_id_1 OR
"Post".created_by_id = :created_by_id_1)
Expected outcome:
Where "Post".post_status != "Draft" AND (
"Post".modified_by_id = :modified_by_id_1 OR
"Post".created_by_id = :created_by_id_1)
I'm wondering, why this is happening? How can I increase the error level in SQLAlchemy? I think my project is silently failing and I would like to confirm my guess.
Update:
I used the wrong constants dictionary. One dictionary contains ints, the other contains strings (one for data base queries, one for printing).
_post_status = db.Column(
db.SmallInteger,
default=Consts.post_status["Draft"])
post_status contains integers, Consts.PostStatuses contains strings. In hind sight, really bad idea. I'm going to make a single dictionary that returns a tuple instead of two dictionaries.
#property
def post_status(self):
return Consts.post_status.get(getattr(self, "_post_status", None))
the problem is that your post_status property isn't acceptable for usage in an ORM level query, as this is a python descriptor which at the class level by default returns itself:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class A(Base):
__tablename__ = 'a'
id = Column(Integer, primary_key=True)
_post_status = Column(String)
#property
def post_status(self):
return self._post_status
print (A.post_status)
print (A.post_status != 5678)
output:
$ python test.py
<property object at 0x10165bd08>
True
the type of usage you're looking for seems like that of a hybrid attribute, which is a SQLAlchemy-included extension to a "regular" python descriptor which produces class-level behavior that's compatible with core SQL expressions:
from sqlalchemy.ext.hybrid import hybrid_property
class A(Base):
__tablename__ = 'a'
id = Column(Integer, primary_key=True)
_post_status = Column(String)
#hybrid_property
def post_status(self):
return self._post_status
print (A.post_status)
print (A.post_status != 5678)
output:
$ python test.py
A._post_status
a._post_status != :_post_status_1
be sure to read the hybrid doc carefully including how to establish the correct SQL expression behavior, descriptors that work both at the instance and class level is a somewhat advanced Python technique.