I would like to query a user based on their code and mobile number.
Of note, I am using an async session when connecting to the DB.
This is my models and query:
class model_user(Base):
__tablename__ = "users"
id = Column(UUID)
code = Column(Integer)
user_name = Column(String(11), unique=True)
first_name = Column(String(255), index=True)
last_name = Column(String(255), index=True)
contact = relationship("model_contact", back_populates="user")
class model_contact(Base):
__tablename__ = "contacts"
id = Column(Integer)
mobile = Column(String(320))
user_id = Column(
UUID(as_uuid=True), ForeignKey("users.id"), nullable=False, index=True
)
user = relationship("model_user", back_populates="contact")
# Here is my query:
query = await db.execute(
select(user_model)\
.filter_by(code==5)\
.options(
joinedload(model_user.contact)
)\
.filter(model_contact.mobile == mobile)
This really has nothing to do with whether or not you are using the async extension of SQLAlchemy. Queries are constructed the same way. Only the session setup and interaction is obviously different.
Side notes:
You should use PascalCase to name your classes and including the term "model" in the name is typically not good style, i.e. User and Contact.
Since you have a one-to-many relationship between your user model and your contact model (i.e. one user can have multiple sets of contact info), you should name the relationship attribute on the user model with plural, i.e. contacts.
The simplest way to do what you want that I can think of is using the Select.where method. You can then construct an SQL EXISTS subquery using the any method of the relationship. The statement would look like this:
statement = select(User).where(
User.code == 123,
User.contacts.any(Contact.mobile == "555")
)
Here is a full working example using aiosqlite just to demonstrate that this works with async tools:
from asyncio import run
from sqlalchemy import Column, ForeignKey, Integer, String, select
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship, sessionmaker
Base = declarative_base()
class User(Base):
__tablename__ = "users"
id = Column(Integer, primary_key=True)
code = Column(Integer)
contacts = relationship("Contact", back_populates="user")
class Contact(Base):
__tablename__ = "contacts"
id = Column(Integer, primary_key=True)
mobile = Column(String(320))
user_id = Column(
Integer(), ForeignKey("users.id"), nullable=False, index=True
)
user = relationship(User, back_populates="contacts")
async def main():
engine = create_async_engine("sqlite+aiosqlite://", echo=True)
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
async_session = sessionmaker(
engine, expire_on_commit=False, class_=AsyncSession
)
async with async_session() as session:
statement = select(User).where(
User.code == 123,
User.contacts.any(Contact.mobile == "555")
)
await session.execute(statement)
if __name__ == "__main__":
run(main())
Running this script produces the following SQL output for the select query:
SELECT users.id, users.code
FROM users
WHERE users.code = ? AND (EXISTS (SELECT 1
FROM contacts
WHERE users.id = contacts.user_id AND contacts.mobile = ?))
...
(123, '555')
This approach should give you the results you want.
Related
I am new in Sqlalchemy and trying to achieve the following goal with relationship():
There is an User table which stores user data.
Every user is able to invite other user with an invite_code.
Every user keeps a list of invitation, every invitation includes the invite_code and the invitee User
I think the relationship between User and Invitation is one-to-many. Since Invitation contains User, then I think it is probably better to use self-referential relationship to represent the inviter-to-invitaions(invitees) relationship and use an association object to store the invite_code.
I checked the sqlalchemy documentation and the question, tried to implement the classed like this:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, String
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Invitation(Base):
__tablename__ = 'invitation'
invite_code = Column(Integer)
inviter_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
invitee_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
invitee = relationship('User') #Need HELP here
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
inviters = relationship('User',
secondary='invitation',
primaryjoin=id==Invitation.invitee_id,
secondaryjoin=id==Invitation.inviter_id,
backref='invitees')
invitations = relationship('Invitation')# Need HELP here
def __repr__(self):
return f'User: {self.name}'
if __name__ == '__main__':
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(engine)
db = Session()
inviter1 = User(name='inviter1')
inviter2 = User(name='inviter2')
invitee1= User(name='invitee1')
invitee2 = User(name='invitee2')
inviter1.invitees = [invitee1, invitee2]
inviter2.invitees = [invitee1]
db.add(inviter1)
db.add(inviter2)
db.add(invitee1)
db.add(invitee2)
db.commit()
users = db.query(User).all()
for user in users:
print(user)
print(' Inviter: ', user.inviters)
print(' Invitee: ', user.invitees)
print()
If the lines with comment #Need HELP here are deleted, I can get the corresponding inviters and invitees, but cannot get the invite_code. If the #Need HELP here code are added, the error is:
Exception has occurred: AmbiguousForeignKeysError
Could not determine join condition between parent/child tables on relationship Invitation.invitee - there are multiple foreign key paths linking the tables. Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as containing a foreign key reference to the parent table.
Is there a way to add extra data column in association object like association object for many-to-many relationship for self referential table?
Sorry for the too much text, I didn't find any reference document on the web.
Finally, I figured it out with the help of foreign_keys:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, String
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
sent_invitations = relationship('Invitation', foreign_keys='Invitation.inviter_id', back_populates='inviter', cascade='all, delete')
received_invitations=relationship('Invitation', foreign_keys='Invitation.invitee_id', back_populates='invitee', cascade='all, delete')
def __repr__(self):
return f'User: {self.name}'
class Invitation(Base):
__tablename__ = 'invitation'
id = Column(Integer, primary_key=True)
invite_code = Column(Integer)
inviter_id = Column(Integer, ForeignKey('user.id'))
invitee_id = Column(Integer, ForeignKey('user.id'))
inviter=relationship('User', foreign_keys=[inviter_id], back_populates='sent_invitations')
invitee=relationship('User', foreign_keys=[invitee_id], back_populates='received_invitations')
def __repr__(self):
return f'Invitation: {self.inviter} invited {self.invitee} with {self.invite_code}'
if __name__ == '__main__':
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(engine)
db = Session()
inviter1 = User(name='inviter1')
inviter2 = User(name='inviter2')
invitee1= User(name='invitee1')
invitee2 = User(name='invitee2')
invitation1 = Invitation(invite_code=50, inviter=inviter1, invitee=invitee1)
invitation2 = Invitation(invite_code=20, inviter=inviter2, invitee=invitee2)
invitation3 = Invitation(invite_code=22, inviter=inviter1, invitee=inviter2)
invitation4 = Invitation(invite_code=44, inviter=invitee1, invitee=inviter2)
db.add(inviter1)
db.add(inviter2)
db.add(invitee1)
db.add(invitee2)
db.commit()
users = db.query(User).all()
for user in users:
print(user)
print(' sent_invitation: ', user.sent_invitations)
print(' received_invitation: ', user.received_invitations)
print()
invitations = db.query(Invitation).all()
for invitation in invitations:
print(invitation)
db.delete(inviter1)
db.delete(invitee2)
db.commit()
I'm trying to implement a basic resource access using SQL Alchemy 1.4 and a PostgreSQL database.
Python code
from sqlalchemy.ext.declarative import DeclarativeMeta, declarative_base
Base: DeclarativeMeta = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
email = Column(String(length=255), index=True, nullable=False)
class Resource(Base):
__tablename__ = "resource"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name = Column(String(length=255), index=True, nullable=False)
class UserResourceRole(Base):
__tablename__ = "user_resource_role"
user_id = Column(
UUID(as_uuid=True),
ForeignKey("user.id", ondelete="CASCADE"),
primary_key=True
)
resource_id = Column(
UUID(as_uuid=True),
ForeignKey("resource.id", ondelete="CASCADE"),
primary_key=True,
)
can_edit = Column(Boolean, default=False, nullable=False)
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
from sqlalchemy.ext.asyncio.engine import AsyncEngine
from sqlalchemy.orm import sessionmaker
from the_other_file import User, Resource, UserResourceRole
async def select_all(user: User, db_session: AsyncSession):
results = await db_session.execute(
select(Resource, UserResourceRole)
.join(
UserResourceRole,
Resource.id == UserResourceRole.resource_id
)
.where(UserResourceRole.user_id == user.id)
)
return results.scalars().all()
engine: AsyncEngine = create_async_engine(POSTGRES_URL, future=True)
async_session = sessionmaker(
bind=engine, class_=AsyncSession, expire_on_commit=False, future=True
)
# ignore the fact that it's not executed in asyncio loop
a_db_session = await async_session()
resources = await select_all(user=a_real_user_is_here, db_session=a_db_session)
print(resources)
I can't retrieve anything from UserResourceRole in my results. It only contains data from Resource. If i swap the objects in the select call, then I can only retrieve data from UserResourceRole.
What I'm expecting
I'm expecting to have the same result of that SQL query :
SELECT *
FROM resource
INNER JOIN user_resource_role
ON resource.id = user_resource_role.resource_id
WHERE user_resource_role.user_id = :a_user_id
The query generated by SQL Alchemy is exactly the same (except the verbosity) :
SELECT resource.id, resource.name, user_resource_role.user_id, user_resource_role.resource_id, user_resource_role.can_edit
FROM resource
JOIN user_resource_role
ON resource.id = user_resource_role.resource_id
WHERE user_resource_role.user_id = :user_id_1
If you try
for entry in results:
print(entry)
it will show you a list of tuple of (Resource, UserResourceRole). Apparently the call to .scalars().all() only leave the first value.
My current solution is to turn results into a list and manually manipulate it.
Remove scalars(). So, it should be
return results.all()
This will return a list of tuples
[(resource_1, userresourcerole_1),(resource_2, userresourcerole_2),...]
I am trying to port the following query to SQLAlchemy:
SELECT u.username, GROUP_CONCAT(DISTINCT userS.name)
FROM Skills AS filterS
INNER JOIN UserSkills AS ufs ON filterS.id = ufs.skill_id
INNER JOIN Users AS u ON ufs.user_id = u.id
INNER JOIN UserSkills AS us ON u.id = us.user_id
INNER JOIN Skills AS userS ON us.skill_id = userS.id
WHERE filterS.name IN ('C#', 'SQL')
GROUP BY u.id;
I don't understand how to achieve AS statement in SQLAlchemy. Here is what I currently have:
# User class has attribute skills, that points to class UserSkill
# UserSkill class has attribute skill, that points to class Skill
db.session.query(User.id, User.username, func.group_concat(Skill.name).label('skills')).\
join(User.skills).\
join(UserSkill.skill).filter(Skill.id.in_(skillIds)).\
order_by(desc(func.count(Skill.id))).\
group_by(User.id).all()
Please help.
I figured this out. Here are the classes that are used in my Flask app:
class User(Model):
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
username = Column(db.String(80), unique=True, nullable=False)
skills = db.relationship('UserSkill')
class Skill(Model):
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
name = Column(db.String(80))
class UserSkill(Model):
status = db.Column(db.Enum(SkillStatus))
user_id = db.Column(db.Integer, db.ForeignKey('users.id'), primary_key=True)
skill_id = db.Column(db.Integer, db.ForeignKey('skills.id'), primary_key=True)
skill = db.relationship("Skill")
So, the actual code would look like this:
from sqlalchemy.orm import aliased
userSkillF = aliased(UserSkill)
userSkillI = aliased(UserSkill)
skillF = aliased(Skill)
skillI = aliased(Skill)
db.session.query(User.id, User.username,\
func.group_concat(func.distinct(skillF.name)).label('skills'),\
func.group_concat(func.distinct(skillI.name)).label('other_skills')).\
join(userSkillF, User.skills).\
join(userSkillI, User.skills).\
join(skillF, userSkillF.skill).filter(skillF.id.in_(skillIds)).\
join(skillI, userSkillI.skill).\
group_by(User.id).all()
Many thanks Ilja Everilä, fresh look on SqlAlchemy docs made me understand aliased now.
We can do the join without relationships as well. Explicitly mention the condition on join.
Example
from sqlalchemy.orm import aliased
user1 = aliased(UserSkill)
user2 = aliased(UserSkill)
query_result = db.session.query(
func.distinct(User.id).label('user_id'),
User.username,
).join(
user1,
User.id == user1.user_id,
).join(
user2,
user2.id == User.id,
).filter(
user1.user_id == id,
).all()
I'm trying to figure out why I need to use a no_autoflush block when inserting data into an association proxy if the association proxy data has been accessed first. An example of this is bellow (using MySQL):
from sqlalchemy import create_engine, Integer, Column, String, ForeignKey, UniqueConstraint
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker, relationship, scoped_session
Base = declarative_base()
engine = create_engine('{}://{}:{}#{}/{}'.format(...))
session_factory = sessionmaker(bind=engine)
Session = scoped_session(session_factory)
class DomainModel(Base):
__tablename__ = 'domains'
id = Column(Integer, primary_key=True)
name = Column(String(255), nullable=False, unique=True)
domains_to_servers = relationship("DomainServerModel", back_populates="domain")
servers = association_proxy('domains_to_servers', 'server',
creator=lambda s: DomainServerModel(server=s))
class ServerModel(Base):
__tablename__ = 'servers'
id = Column(Integer, primary_key=True)
name = Column(String(128), nullable=False, unique=True, index=True)
domains_to_servers = relationship("DomainServerModel", back_populates="server")
domains = association_proxy('domains_to_servers', 'domain',
creator=lambda d: DomainServerModel(domain=d))
class DomainServerModel(Base):
__tablename__ = 'domains_to_servers'
id = Column(Integer, primary_key=True)
domain_id = Column(Integer, ForeignKey('domains.id'), nullable=False)
server_id = Column(Integer, ForeignKey('servers.id'), nullable=False)
server = relationship('ServerModel', back_populates="domains_to_servers")
domain = relationship('DomainModel', back_populates="domains_to_servers")
def test():
session = Session()
with session.no_autoflush:
s = session.query(ServerModel).filter_by(name='test.com').one()
print(s.domains)
d = DomainModel(name='test1.com')
session.add(d)
session.commit()
s.domains.append(d)
session.commit()
if __name__ == '__main__':
Base.metadata.drop_all(engine)
Base.metadata.create_all(engine)
session = Session()
session.add(ServerModel(name='test.com'))
session.commit()
test()
I'm trying to add a new domain_to_server mapping via the server/domain association proxy. If I don't access the association proxy first, ie remove the print statement in test(), then I can add the domain without needing the session.no_autoflush block. But with the print statement in there, it will fail without the session.no_autoflush block with an IntegrityError, saying that server_id cannot be null in the domains to servers table.
I'm trying to figure out why the no_autoflush block is needed here. I don't see any mention of it in the association_proxy docs. Is this simply the way it is, and all inserts into an association_proxy should to happen in a no_autoflush bock in case it has been accessed prior to the insert?
In a Pyramid application I'm working on, I have the following scenario:
class Widget(Base):
__tablename__ = 'widgets'
id = Column(Integer, primary_key=True)
name = Column(String(50))
sidebar = Column(mysql.TINYINT(2))
def __init__(self, name, sidebar):
self.name = name
self.sidebar = sidebar
class Dashboard(Base):
__tablename__ = 'dashboard'
user_id = Column(Integer, ForeignKey('users.id'), primary_key=True)
widget_id = Column(Integer, ForeignKey('widgets.id'), primary_key=True)
delta = Column(mysql.TINYINT)
widget = relationship('Widget')
def __init__(self, user_id, widget_id, delta):
self.user_id = user_id
self.widget_id = widget_id
self.delta = delta
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
login = Column(Unicode(255), unique=True)
password = Column(Unicode(60))
fullname = Column(Unicode(100))
dashboard = relationship('Dashboard', order_by='Dashboard.widget.sidebar, Dashboard.delta')
def __init__(self, login, password, fullname):
self.login = login
self.password = crypt.encode(password)
self.fullname = fullname
So, I want the User 'dashboard' relationship to have the dashboard records for the user but ordered by 'sidebar' (which is a relationship property of Dashboard). Currently I am getting this error:
sqlalchemy.exc.InvalidRequestError: Property 'widget' is not an instance of ColumnProperty (i.e. does not correspond directly to a Column).
Is this ordering possible in a relationship declaration?
Thanks!
With this, try to think what SQL SQLAlchemy should emit when it tries to load User.dashboard. Like SELECT * FROM dashboard JOIN widget ... ORDER BY widget.sidebar ? Or SELECT * FROM dashboard ORDER BY (SELECT sidebar FROM widget... ? ordering the results by a different table is too open-ended of a job for relationship() to decide on it's own. The way this can be done is by providing a column expression in terms of Dashboard that can provide this ordering, when the ORM emits a simple SELECT against dashboard's table, as well as when it refers to it in a not-so-simple SELECT where it might be joining across User, Dashboard tables at once (e.g. eager loading).
We provide custom SQL expressions, particularly those that involve other tables, using column_property(), or alternatively with deferred() when we don't want that expression to be loaded by default (as is likely the case here). Example:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Widget(Base):
__tablename__ = 'widgets'
id = Column(Integer, primary_key=True)
name = Column(String(50))
sidebar = Column(Integer)
class Dashboard(Base):
__tablename__ = 'dashboard'
user_id = Column(Integer, ForeignKey('users.id'), primary_key=True)
widget_id = Column(Integer, ForeignKey('widgets.id'), primary_key=True)
delta = Column(Integer)
widget = relationship('Widget')
widget_sidebar = deferred(select([Widget.sidebar]).where(Widget.id == widget_id))
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
login = Column(Unicode(255), unique=True)
dashboard = relationship('Dashboard', order_by='Dashboard.widget_sidebar, Dashboard.delta')
e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)
s = Session(e)
w1, w2 = Widget(name='w1', sidebar=1), Widget(name='w2', sidebar=2)
s.add_all([
User(login='u1', dashboard=[
Dashboard(
delta=1, widget=w1
),
Dashboard(
delta=2, widget=w2
)
]),
])
s.commit()
print s.query(User).first().dashboard
the final SQL emitted by the load of ".dashboard" is:
SELECT dashboard.user_id AS dashboard_user_id, dashboard.widget_id AS dashboard_widget_id, dashboard.delta AS dashboard_delta
FROM dashboard
WHERE ? = dashboard.user_id ORDER BY (SELECT widgets.sidebar
FROM widgets
WHERE widgets.id = dashboard.widget_id), dashboard.delta
Keep in mind that MySQL does a terrible job optimizing for subqueries like the one above. If you need high performance here, you might consider copying the value of "sidebar" into "dashboard", even though that makes consistency more difficult to maintain.