I want to be able to query through multiple many to many relationships in SQLAlchemy.
I have Users, which are associated to Groups, which are associated to Roles. All associations are many-to-many. I want to get the list of Roles that are associated with the User through the Groups.
Here are my striped down models:
class User(Model):
id = Column(Integer)
groups = relationship('Group', secondary=user_group)
class Group(Model):
id = Column(Integer)
roles = relationship('Role', secondary=role_group)
class Role(Model):
id = Column(Integer)
I have a rough idea of what SQL would be used:
select distinct role.* from role, role_group
where role.id = role_group.role_id
and role_group.group_id in
(select "group".id from "group", user_group
WHERE user_group.user_id = 1
and "group".id = user_group."group.id")
But I am having a hard time figuring out how to translate that into SQLAlchemy...and I am not sure even if the SQL here is the best way to do this or not.
Try this
s = Session()
s.query(Role).join(User.groups).join(Group.roles).filter(User.id==1).all()
it will generate following sql query
SELECT role.id AS role_id
FROM user
JOIN user_group AS user_group_1 ON user.id = user_group_1.user_id
JOIN "group" ON "group".id = user_group_1.group_id
JOIN role_group AS role_group_1 ON "group".id = role_group_1.group_id
JOIN role ON role.id = role_group_1.role_id
WHERE user.id = 1;
however sqlalchemy will only return distinct roles (if you will query Role.id then sqlalchemy will return what sql query actually returns).
I did find a solution to my own question that would generate close to the SQL I was looking for. However, I believe zaquest's answer is much better and will be using that instead. This is just for posterity's sake.
Here is the query I came up with:
s = Session()
s.query(Role).join(Role.groups).filter(
Group.id.in_(s.query(Group.id).join(User.groups)
.filter(User.id == self.id)))
This would generate the following SQL
SELECT role.id AS role_id
FROM role JOIN role_group AS role_group_1 ON role.id = role_group_1.role_id
JOIN "group" ON "group".id = role_group_1.group_id
WHERE "group".id IN
(SELECT "group".id AS group_id
FROM "user" JOIN user_group AS user_group_1 ON "user".id = user_group_1.user_id
JOIN "group" ON "group".id = user_group_1."group.id"
WHERE "user".id = :id_1)
This is similar to the SQL I was looking for. The key that I forgot about was that I can use another query as part of the in operator in SQLAlchemy.
Related
I have 3 tables, users, projects and project_users is_admin table and I am trying to write ORM to get data from them.
My Models are here: https://pastebin.com/ZrmhKyNL
In simple SQL, we could join and select particular columns and get the desired output. But in ORM when I write query like this:
sql = """ select * from
projects p, users u, projects_users pu
where
p.name = '%s' and
p.id = pu.project_id and
pu.user_id = u.id and
p.is_active = true and
u.is_active = true
""" % project_name
and it works well and returns response in this format:
[
{
All columns of above 3 tables.
}
]
But when I try to convert this to sqlalchamey ORM, it doesn't work well:
return (
db.query(User)
.filter(Project.id == ProjectsUser.project_id)
.filter(ProjectsUser.user_id == User.id)
.filter(Project.is_active == True)
.filter(User.is_active == True)
.filter(Project.name == project_name)
.all()
)
I want is_admin value to be returned along with user object. This seems very common use case, but I couldn't find any solution related to SQLalchemy ORM.
If this is a one-to-one relationship, meaning that for every user you can have one and only one entry in the admin table, you could add a column to serve as back-reference:
class User(Base):
id = ...
name = ...
last_login = ...
admin = relationship("Admin", uselist=False, back_populates="user")
class Admin(Base):
id = ...
is_admin = ...
user_id = Column(Integer, ForeignKey('user.id'))
user = relationship("User", back_populates="admin")
Then you can query only the table user, and access the values from the relationship via user.admin.is_admin.
Read more:
One To One - SqlAlchemy docs
relationship.back_populates - SqlAlchemy docs
I'm trying to use SQLAlchemy in a situation where I have a one to many table construct and but I essentially want to create a one to one mapping between tables using a subquery.
For example
class User:
__tablename__='user'
userid = Column(Integer)
username = Column(String)
class Address:
__tablename__='address'
userid = Column(Integer)
address= Column(String)
type= Column(String)
In this case the type column of Address includes strings like "Home", "Work" etc. I would like the output to look something like this
I tried using a subquery where I tried
session.query(Address).filter(Address.type =="Home").subquery("HomeAddress")
and then joining against that but then I lose ORM "entity" mapping.
How can I subquery but retain the ORM attributes in the results object?
You don't need to use a subquery. The join condition is not limited to foreign key against primary key:
home_address = aliased(Address, "home_address")
work_address = aliased(Address, "work_address")
session.query(User) \
.join(home_address, and_(User.userid == home_address.userid,
home_address.type == "Home")) \
.join(work_address, and_(User.userid == work_address.userid,
work_address.type == "Work")) \
.with_entities(User, home_address, work_address)
I want use Django ORM. I build SQL query:
select itinerary.id, count(users.home_location_id) from itinerary_itinerary as itinerary left join (select to_applicationuser_id as id, users.home_location_id from custom_auth_applicationuser_friends as friends join custom_auth_applicationuser as users on friends.to_applicationuser_id = users.id where from_applicationuser_id = 28)
as users on itinerary.location_id = users.home_location_id
WHERE user_id = 28
GROUP BY itinerary.id, users.home_location_id
Could anybody tell me how make left join with table from subquery?
28 is current user_id.
I use something like:
Itinerary.object.filter(user_id=28).extra(
tables=['(select to_applicationuser_id as id, users.home_location_id from custom_auth_applicationuser_friends as friends join custom_auth_applicationuser as users on friends.to_applicationuser_id = users.id where from_applicationuser_id = 28) as users'],
where=['itinerary.location_id = users.home_location_id']
)
But I got error
ProgrammingError relation "(select to_applicationuser_id as id,
users.home_location_id fro" does not exist
UPD
Models (it is just simple scheme):
class ApplicationUser(models.Model):
name = models.CharField(max_length=255)
home_location = models.ForeignKey(Location)
friends = models.ManyToManyFieled('self')
class Location(models.Model):
loc_name = models.CharFiled(max_length=255)
class Itinerary(models.Model):
user = models.ForeignKey(ApplicationUser)
location = models.ForeignKey(Location)
When you add tables with extra, they get added to the from list, which does not accept an sql statement.
I don't think you need to use extra at all here, you can get a similar query with the ORM without the need to join on a select statement. The following code, using filtering and annotations, will give the same results as much as I was able to understand your query:
ApplicationUser.objects.filter(
Q(itinerary__location_id = F('friends__home_location_id')) |
Q(friends__home_location__isnull=True),
id=28,
).values_list(
'itinerary__id', 'friends__home_location_id'
).annotate(location_count = Count('friends__home_location_id')
).values_list('itinerary__id', 'location_count')
I want to delete some elements in tables that have a polymorphic relationship in sqlalchemy. Here's the model:
class Employee(Base):
__tablename__ = 'employee'
id = Column(Integer, primary_key=True)
name = Column(String(50))
type = Column(String(50))
__mapper_args__ = {
'polymorphic_identity':'employee',
'polymorphic_on':type
}
class Engineer(Employee):
__tablename__ = 'engineer'
id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
engineer_name = Column(String(30))
__mapper_args__ = {
'polymorphic_identity':'engineer',
}
And here's how I delete it:
e = Engineer();
e.name = "John";
e.engineer_name = "Doe";
DBSession.add(e);
q = session.query(Engineer).filter(Employee.name == "John")
q.delete(False)
I get the following error, is that a bug or am I doing it the wrong way ?
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such
column: employee.name [SQL: u'DELETE FROM engineer WHERE employee.name
= ?'] [parameters: ('John',)]
I'm expecting sqlalchemy to delete the entres in the engineer and employee tables.
First you should define the on delete behaviour of this relationship:
id = Column(Integer, ForeignKey('employee.id', ondelete='CASCADE'), primary_key=True)
Then, using the ORM, you can delete all engineers with name "John" through a loop:
eng_list = session.query(Engineer).filter(Employee.name == "John").all()
for eng in eng_list:
session.delete(eng)
session.commit()
This will delete the records from both the Employee and Engineer tables.
update: comment on error message:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such
column: employee.name [SQL: u'DELETE FROM engineer WHERE employee.name
= ?'] [parameters: ('John',)]
Your attempt tries to Delete from Engineer with a join to Employee (to access the field Employee.name). But this join is missing from the query sqlalchemy is emitting to the backend.
I don't think SQLite supports deleting with joins. Perhaps you can try to run session.query(Engineer).filter(Employee.name == "John").delete() against a different backend, and sqlalchemy may be able to emit the proper SQL statement. I haven't tried it though.
update 2: On backends that respect foreign key constraints (and the onupdate constraint has been set to cascade), it should be sufficient to delete the row in the parent row, and the linked rows in the child will automatically be deleted.
I tried this example with both MySQL & Postgresql backends, and the following query deleted the row from both tables (employee & engineer):
session.query(Employee).filter(Employee.name=='John').delete()
For some reason, on Sqlite, this only deletes the record from employee.
Because doing the joined DELETE is not supported directly, I found an easy workaround is to use your normal joined query to select the ids to delete, then pass those ids to a separate DELETE query.
One minor annoyance is that since your returned ids are integers you would likely run into this error like I did if you try to pass those ids (technically an array of tuples) directly to the DELETE query. A simple intermediate conversion to strings fixes that.
So all together:
ids_to_delete = session.query(Engineer.id). \
filter(Employee.name == "John"). \
all()
# Convert the resulting int tuples to simple strings:
id_strings = [str(id_[0]) for id_ in ids_to_delete]
session.query(Engineer). \
filter(Engineer.id.in_(id_strings)). \
delete(synchronize_session=False)
I have a table of data given timestamps and a couple of values where I want to get the last value of each day.
In raw SQL I would do it like this:
SELECT strftime('%Y-%m-%d', a1.created_at) AS created_at,
a1.my_value
FROM my_table a1
JOIN
(SELECT max(id) AS MAX
FROM mytable
GROUP BY strftime('%Y-%m-%d', created_at)) a2 ON a1.id = a2.MAX;
I am working on a Flask Application where I want to use the Flask-SQLAlchemy extension and not sqlalchemy.orm.session.Session or raw SQL. The defined Model looks like this:
class MyModel(SurrogatePK, Model):
__tablename__ = 'my_table'
id = Column(db.Integer(), nullable=False, primary_key=True)
created_at = Column(db.DateTime(), nullable=False, default=dt.datetime.utcnow)
my_value = Column(db.Integer(), nullable=True)
What I have so far is:
Outer query:
MyModel.query.with_entities(MyModel.created_at, MyModel.my_value)...
Inner query:
MyModel.query.with_entities(func.max(MyModel.id).label('max')).group_by(func.strftime('%Y-%m-%d', MyModel.created_at))
But I cannot find the way to join both together to get the desired result.
This is how i would do it in sqlalchemy,
from sqlalchemy import literal_column
a2 = session.query(func.max(mytable.id).label("MAX"))
.group_by(func.convert(literal_column('DATE'), mytable.created_at)).subquery()
session.query(func.convert(literal_column('DATE'),
mytable.created_at).label("created_at"),
my_table.my_value).join(a2, my_table.id==a2.c.id)