I am trying to port the following query to SQLAlchemy:
SELECT u.username, GROUP_CONCAT(DISTINCT userS.name)
FROM Skills AS filterS
INNER JOIN UserSkills AS ufs ON filterS.id = ufs.skill_id
INNER JOIN Users AS u ON ufs.user_id = u.id
INNER JOIN UserSkills AS us ON u.id = us.user_id
INNER JOIN Skills AS userS ON us.skill_id = userS.id
WHERE filterS.name IN ('C#', 'SQL')
GROUP BY u.id;
I don't understand how to achieve AS statement in SQLAlchemy. Here is what I currently have:
# User class has attribute skills, that points to class UserSkill
# UserSkill class has attribute skill, that points to class Skill
db.session.query(User.id, User.username, func.group_concat(Skill.name).label('skills')).\
join(User.skills).\
join(UserSkill.skill).filter(Skill.id.in_(skillIds)).\
order_by(desc(func.count(Skill.id))).\
group_by(User.id).all()
Please help.
I figured this out. Here are the classes that are used in my Flask app:
class User(Model):
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
username = Column(db.String(80), unique=True, nullable=False)
skills = db.relationship('UserSkill')
class Skill(Model):
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
name = Column(db.String(80))
class UserSkill(Model):
status = db.Column(db.Enum(SkillStatus))
user_id = db.Column(db.Integer, db.ForeignKey('users.id'), primary_key=True)
skill_id = db.Column(db.Integer, db.ForeignKey('skills.id'), primary_key=True)
skill = db.relationship("Skill")
So, the actual code would look like this:
from sqlalchemy.orm import aliased
userSkillF = aliased(UserSkill)
userSkillI = aliased(UserSkill)
skillF = aliased(Skill)
skillI = aliased(Skill)
db.session.query(User.id, User.username,\
func.group_concat(func.distinct(skillF.name)).label('skills'),\
func.group_concat(func.distinct(skillI.name)).label('other_skills')).\
join(userSkillF, User.skills).\
join(userSkillI, User.skills).\
join(skillF, userSkillF.skill).filter(skillF.id.in_(skillIds)).\
join(skillI, userSkillI.skill).\
group_by(User.id).all()
Many thanks Ilja Everilä, fresh look on SqlAlchemy docs made me understand aliased now.
We can do the join without relationships as well. Explicitly mention the condition on join.
Example
from sqlalchemy.orm import aliased
user1 = aliased(UserSkill)
user2 = aliased(UserSkill)
query_result = db.session.query(
func.distinct(User.id).label('user_id'),
User.username,
).join(
user1,
User.id == user1.user_id,
).join(
user2,
user2.id == User.id,
).filter(
user1.user_id == id,
).all()
Related
I have the following SQLAlchemy mapped classes:
class User(Base):
__tablename__ = 'users'
email = Column(String, primary_key=True)
name = Column(String)
class Document(Base):
__tablename__ = "documents"
name = Column(String, primary_key=True)
author = Column(String, ForeignKey("users.email"))
class DocumentsPermissions(Base):
__tablename__ = "documents_permissions"
readAllowed = Column(Boolean)
writeAllowed = Column(Boolean)
document = Column(String, ForeignKey("documents.name"))
I need to get a table like this for user.email = "user#email.com":
email | name | document_name | document_readAllowed | document_writeAllowed
How can it be made using one query request for SQLAlchemy? The code below does not work for me:
result = session.query(User, Document, DocumentPermission).filter_by(email = "user#email.com").all()
Thanks,
Try this
q = Session.query(
User, Document, DocumentPermissions,
).filter(
User.email == Document.author,
).filter(
Document.name == DocumentPermissions.document,
).filter(
User.email == 'someemail',
).all()
As #letitbee said, its best practice to assign primary keys to tables and properly define the relationships to allow for proper ORM querying. That being said...
If you're interested in writing a query along the lines of:
SELECT
user.email,
user.name,
document.name,
documents_permissions.readAllowed,
documents_permissions.writeAllowed
FROM
user, document, documents_permissions
WHERE
user.email = "user#email.com";
Then you should go for something like:
session.query(
User,
Document,
DocumentsPermissions
).filter(
User.email == Document.author
).filter(
Document.name == DocumentsPermissions.document
).filter(
User.email == "user#email.com"
).all()
If instead, you want to do something like:
SELECT 'all the columns'
FROM user
JOIN document ON document.author_id = user.id AND document.author == User.email
JOIN document_permissions ON document_permissions.document_id = document.id AND document_permissions.document = document.name
Then you should do something along the lines of:
session.query(
User
).join(
Document
).join(
DocumentsPermissions
).filter(
User.email == "user#email.com"
).all()
One note about that...
query.join(Address, User.id==Address.user_id) # explicit condition
query.join(User.addresses) # specify relationship from left to right
query.join(Address, User.addresses) # same, with explicit target
query.join('addresses') # same, using a string
For more information, visit the docs.
A good style would be to setup some relations and a primary key for permissions (actually, usually it is good style to setup integer primary keys for everything, but whatever):
class User(Base):
__tablename__ = 'users'
email = Column(String, primary_key=True)
name = Column(String)
class Document(Base):
__tablename__ = "documents"
name = Column(String, primary_key=True)
author_email = Column(String, ForeignKey("users.email"))
author = relation(User, backref='documents')
class DocumentsPermissions(Base):
__tablename__ = "documents_permissions"
id = Column(Integer, primary_key=True)
readAllowed = Column(Boolean)
writeAllowed = Column(Boolean)
document_name = Column(String, ForeignKey("documents.name"))
document = relation(Document, backref = 'permissions')
Then do a simple query with joins:
query = session.query(User, Document, DocumentsPermissions).join(Document).join(DocumentsPermissions)
Expanding on Abdul's answer, you can obtain a KeyedTuple instead of a discrete collection of rows by joining the columns:
q = Session.query(*User.__table__.columns + Document.__table__.columns).\
select_from(User).\
join(Document, User.email == Document.author).\
filter(User.email == 'someemail').all()
This function will produce required table as list of tuples.
def get_documents_by_user_email(email):
query = session.query(
User.email,
User.name,
Document.name,
DocumentsPermissions.readAllowed,
DocumentsPermissions.writeAllowed,
)
join_query = query.join(Document).join(DocumentsPermissions)
return join_query.filter(User.email == email).all()
user_docs = get_documents_by_user_email(email)
I would like to query a user based on their code and mobile number.
Of note, I am using an async session when connecting to the DB.
This is my models and query:
class model_user(Base):
__tablename__ = "users"
id = Column(UUID)
code = Column(Integer)
user_name = Column(String(11), unique=True)
first_name = Column(String(255), index=True)
last_name = Column(String(255), index=True)
contact = relationship("model_contact", back_populates="user")
class model_contact(Base):
__tablename__ = "contacts"
id = Column(Integer)
mobile = Column(String(320))
user_id = Column(
UUID(as_uuid=True), ForeignKey("users.id"), nullable=False, index=True
)
user = relationship("model_user", back_populates="contact")
# Here is my query:
query = await db.execute(
select(user_model)\
.filter_by(code==5)\
.options(
joinedload(model_user.contact)
)\
.filter(model_contact.mobile == mobile)
This really has nothing to do with whether or not you are using the async extension of SQLAlchemy. Queries are constructed the same way. Only the session setup and interaction is obviously different.
Side notes:
You should use PascalCase to name your classes and including the term "model" in the name is typically not good style, i.e. User and Contact.
Since you have a one-to-many relationship between your user model and your contact model (i.e. one user can have multiple sets of contact info), you should name the relationship attribute on the user model with plural, i.e. contacts.
The simplest way to do what you want that I can think of is using the Select.where method. You can then construct an SQL EXISTS subquery using the any method of the relationship. The statement would look like this:
statement = select(User).where(
User.code == 123,
User.contacts.any(Contact.mobile == "555")
)
Here is a full working example using aiosqlite just to demonstrate that this works with async tools:
from asyncio import run
from sqlalchemy import Column, ForeignKey, Integer, String, select
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship, sessionmaker
Base = declarative_base()
class User(Base):
__tablename__ = "users"
id = Column(Integer, primary_key=True)
code = Column(Integer)
contacts = relationship("Contact", back_populates="user")
class Contact(Base):
__tablename__ = "contacts"
id = Column(Integer, primary_key=True)
mobile = Column(String(320))
user_id = Column(
Integer(), ForeignKey("users.id"), nullable=False, index=True
)
user = relationship(User, back_populates="contacts")
async def main():
engine = create_async_engine("sqlite+aiosqlite://", echo=True)
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
async_session = sessionmaker(
engine, expire_on_commit=False, class_=AsyncSession
)
async with async_session() as session:
statement = select(User).where(
User.code == 123,
User.contacts.any(Contact.mobile == "555")
)
await session.execute(statement)
if __name__ == "__main__":
run(main())
Running this script produces the following SQL output for the select query:
SELECT users.id, users.code
FROM users
WHERE users.code = ? AND (EXISTS (SELECT 1
FROM contacts
WHERE users.id = contacts.user_id AND contacts.mobile = ?))
...
(123, '555')
This approach should give you the results you want.
There are tables for my question.
class TemplateExtra(ExtraBase, InsertMixin, TimestampMixin):
__tablename__ = 'template_extra'
id = Column(Integer, primary_key=True, autoincrement=False)
name = Column(Text, nullable=False)
roles = relationship(
'RecipientRoleExtra',
secondary='template_to_role',
)
class RecipientRoleExtra(
ExtraBase, InsertMixin, TimestampMixin,
SelectMixin, UpdateMixin,
):
__tablename__ = 'recipient_role'
id = Column(Integer, primary_key=True, autoincrement=True)
name = Column(Text, nullable=False)
description = Column(Text, nullable=False)
class TemplateToRecipientRoleExtra(ExtraBase, InsertMixin, TimestampMixin):
__tablename__ = 'template_to_role'
id = Column(Integer, primary_key=True, autoincrement=True)
template_id = Column(Integer, ForeignKey('template_extra.id'))
role_id = Column(Integer, ForeignKey('recipient_role.id'))
I want to select all templates with prefetched roles in two sql-queries like Django ORM does with prefetch_related. Can I do it?
This is my current attempt.
def test_custom():
# creating engine with echo=True
s = DBSession()
for t in s.query(TemplateExtra).join(RecipientRoleExtra, TemplateExtra.roles).all():
print(f'id = {t.id}')
for r in t.roles:
print(f'-- {r.name}')
But..
it generates select query for every template to select its roles. Can I make sqlalchemy to do only one query?
generated queries for roles are without join, just FROM recipient_role, template_to_role with WHERE %(param_1)s = template_to_role.template_id AND recipient_role.id = template_to_role.role_id. Is it correct?
Can u help me?
Based on this answer:
flask many to many join as done by prefetch_related from django
Maybe somthing like this:
roles = TemplateExtra.query.options(db.joinedload(TemplateExtra.roles)).all
Let me know if it worked.
Still wrapping my head around SqlAlchemy and have run into a few issues. Not sure if it is because I am creating the relationships incorrectly, querying incorrect, or both.
The general idea is...
one-to-many from location to user (a location can have many users but users can only have one location).
many-to-many between group and user (a user can be a member of many groups and a group can have many members).
Same as #2 above for desc and user.
My tables are created as follows:
Base = declarative_base()
class Location(Base):
__tablename__ = 'location'
id = Column(Integer, primary_key=True)
name = Column(String)
group_user_association_table = Table('group_user_association_table', Base.metadata,
Column('group_id', Integer, ForeignKey('group.id')),
Column('user_id', Integer, ForeignKey('user.id')))
class Group(Base):
__tablename__ = 'group'
id = Column(Integer, primary_key=True)
name = Column(String)
users = relationship('User', secondary=group_user_association_table, backref='group')
desc_user_association_table = Table('desc_user_association', Base.metadata,
Column('desc_id', Integer, ForeignKey('desc.id')),
Column('user_id', Integer, ForeignKey('user.id')))
class Desc(Base):
__tablename__ = 'desc'
id = Column(Integer, primary_key=True)
name = Column(String)
users = relationship('User', secondary=desc_user_association_table, backref='desc')
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
user_name = Column(String)
location_id = Column(Integer, ForeignKey('location.id'))
groups = Column(String, ForeignKey('group.id'))
descs = Column(String, ForeignKey('desc.id'))
location = relationship('Location', backref='user')
Here are some examples as to how I am creating the data (all being scraped from the web):
location = Location(id=city[1], name=city[0]) #city = ('name', id)
profile = User()
profile.id = int(str(span2class[0].a['href'][7:]))
profile.user_name = str(span2class[0].a.img['alt'])
profile.location_id = location.id
g = Group(id=gid, name=str(group.contents[0])) # add the group to the Group table
self.db_session.add(g)
# Now add the gid to a list that will be added to the profile that eventually gets added to the user table
profile.groups.append(str(gid)) # stick the gid into the list
profile.groups = ','.join(profile.groups) # convert list to csv string
# Repeat basically same thing above for desc
self.db_session.add(profile)
self.db_session.commit()
As far as queries go, I've got some of the basic ones working such as:
for instance in db_session.query(User).all():
print instance.id, instance.user_name
But when it comes to performing a join to get (for example) group.id and group.name for a specific user.id... nothing I've tried has worked. I am guessing that the form would be something like the following:
db_session.query(User, Group).join('users').filter(User.id==42)
but that didn't work.
Joins works from left to right, so you should join on the relationship from User to Group:
db_session.query(User, Group).join(User.group).filter(User.id == 42)
But this return you a list of tuples (<User>, <Group>), so if the user belongs to 2 or more groups, you will receive 2 or more rows.
If you really want to load both the user and its groups in one (SQL) query, a better way would be to load a user, but configure query to preload groups:
u = (session.query(User)
.options(joinedload(User.group))
.get(42)
)
print("User = {}".format(u))
for g in u.group:
print(" Group = {}".format(g))
I have this kind of model for User. Users are able to have multiple friends and also one best friend.
friendship = Table('friendship', Base.metadata,
Column('m1_id', Integer, ForeignKey('user.id')),
Column('m2_id', Integer, ForeignKey('user.id'))
)
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(80), unique=True)
email = Column(String(120), unique=True, info={'validators': Email()})
age = Column(Integer())
best_friend_id = Column(Integer, ForeignKey('user.id'))
best_friend = relationship(lambda: User, remote_side=[id])
friends = relationship('User',
secondary=friendship,
primaryjoin=id==friendship.c.m1_id,
secondaryjoin=id==friendship.c.m2_id,
backref=backref('ofriends', lazy='dynamic'),
lazy='dynamic'
)
I m trying to get a query witch contains count of friends and is ordered by the name of persons best friend, but i cannot figure out how.
query = db_session.query(User, func.count(friendship)-1).join(friendship, User.id==friendship.c.m1_id).group_by(User)
Then this fails:
query.order_by(User.best_friend.name).all()
Thank you
friendship and best_friend are, in your model, unrelated. If you want to use the best_friend relationship in your query, you'll have to join it in somehow; since this is a self join, you also need to come up with an alias for the left and right instances of User, so they can be distinguished.
The following query gives users and the total number of friends they have, ordered by the name of whoever their best friend is (data about that best friend is not included in the result set)
>>> best_friend_alias = sa.orm.aliased(User)
>>> query = (Query([User, sa.func.count(friendship)-1])
... .join(friendship, User.id==friendship.c.m1_id)
... .join((best_friend_alias, User.best_friend))
... .group_by(User)
... .order_by(best_friend_alias.name))
>>> print query
SELECT "user".id AS user_id, "user".name AS user_name, "user".email AS user_email, "user".age AS user_age, "user".best_friend_id AS user_best_friend_id, count() - :count_1 AS anon_1
FROM "user" JOIN friendship ON "user".id = friendship.m1_id JOIN "user" AS user_1 ON user_1.id = "user".best_friend_id GROUP BY "user".id, "user".name, "user".email, "user".age, "user".best_friend_id ORDER BY user_1.name