I'm trying to use SQLAlchemy in a situation where I have a one to many table construct and but I essentially want to create a one to one mapping between tables using a subquery.
For example
class User:
__tablename__='user'
userid = Column(Integer)
username = Column(String)
class Address:
__tablename__='address'
userid = Column(Integer)
address= Column(String)
type= Column(String)
In this case the type column of Address includes strings like "Home", "Work" etc. I would like the output to look something like this
I tried using a subquery where I tried
session.query(Address).filter(Address.type =="Home").subquery("HomeAddress")
and then joining against that but then I lose ORM "entity" mapping.
How can I subquery but retain the ORM attributes in the results object?
You don't need to use a subquery. The join condition is not limited to foreign key against primary key:
home_address = aliased(Address, "home_address")
work_address = aliased(Address, "work_address")
session.query(User) \
.join(home_address, and_(User.userid == home_address.userid,
home_address.type == "Home")) \
.join(work_address, and_(User.userid == work_address.userid,
work_address.type == "Work")) \
.with_entities(User, home_address, work_address)
Related
I build default equal filters for all columns of the table and then add/override only specific filters I want on a few columns, like this:
# Table definition
class OrderTable(Base):
__tablename__ = "orders"
id = Column(Integer, unique=True, nullable=False, primary_key=True)
name = Column(String, nullable=False)
# Build all default filters for this table
table_filters = {column.name: lambda value: column == value for column in OrderTable.__table__.columns}
# Add/override specific filters
all_filters = {**table_filters, "name": lambda value: OrderTable.name.like(value)}
# Execute the query
query = ...
query = query.filter(all_filters["id"](123))
query.delete()
But I get this warning when using default filters:
SAWarning: Evaluating non-mapped column expression 'orders.id' onto ORM instances; this is a deprecated use case. Please make use of the actual mapped columns in ORM-evaluated UPDATE / DELETE expressions.
Is there a better way to get all columns to be able to filter on them without getting this warning?
I tried different ways of gettings all columns for a table with OrderTable.__mapper__.attrs and inspect(OrderTable).attrs but then the filters do not work.
I am not used to post so please tell me if I can improve my question and I will edit it.
It appears to be related to fetching columns from the table vs using attributes directly.
When you use the attibute directly, you get an InstrumentedAttribute, when you get the column from __table__.columns you get a Column.
With the Column you get that warning:
id_filter_col = OrderTable.__table__.c["id"] == 1
query = session.query(OrderTable)
query = query.filter(id_filter_col)
query.delete() # SAWarning: Evaluating non-mapped column expression 'orders.id' onto ORM instances...
But now when you use the InstrumentedAttribute:
id_filter_attr = OrderTable.id == 2
query = session.query(OrderTable)
query = query.filter(id_filter_attr)
query.delete() # OK
You can access the attributes from the mapper via __mapper__.all_orm_descriptors, which should solve your problem.
class OrderTable(Base):
__tablename__ = "orders"
id = Column(Integer, unique=True, nullable=False, primary_key=True)
name = Column(String, nullable=False)
table_filters = {column.key: lambda value: column == value for column in OrderTable.__mapper__.all_orm_descriptors}
all_filters = {**table_filters, "name": lambda value: OrderTable.name.like(value)}
query = ...
query = query.filter(all_filters["id"](123))
query.delete() # OK
Question
How do I get a list of all primary key columns of a model class and all its parents in the polymorphic hierarchy, so I can use it in a session.query().group_by?
Details
In SQLAlchemy ORM, if I'm querying a class that is part of a polymorphic hierarchy and I want to GROUP BY it's primary key, I must also GROUP BY all primary keys of its parents in the polymorphic hierarchy.
Imagine the following setup inspired by the Mapping Class Inheritance Hierarchies section of the sqlalchemy documentation:
class Employee(Base):
__tablename__ = 'employee'
id = Column(Integer, primary_key=True)
name = Column(String(50))
type = Column(String(50))
__mapper_args__ = {
'polymorphic_identity':'employee',
'polymorphic_on':type
}
class Engineer(Employee):
__tablename__ = 'engineer'
id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
engineer_name = Column(String(30))
__mapper_args__ = {
'polymorphic_identity':'engineer',
}
class EngineeringTask(Base):
__tablename__ = 'engineering_task'
id = Column(Integer, primary_key=True)
name = Column(String(50))
assigned_engineer_id = Column(ForeignKey(Engineer.id))
assigned_engineer = relationship(Engineer, backref=backref("assigned_tasks", lazy="dynamic"))
If I want to do a query like
session.query(
Engineer,
).join(
Engineer.assigned_tasks,
).add_columns(
func.count(EngineeringTask.id).label('assigned_task_count'),
).group_by(Engineer.id, Employee.id)
Such a query, which is selecting all columns from Engineer and Employee but not aggregating them, is possible in PostgreSQL because (emphasis mine):
When GROUP BY is present (...) it is not valid for the SELECT list expressions to refer to ungrouped columns except within aggregate functions or when the ungrouped column is functionally dependent on the grouped columns, since there would otherwise be more than one possible value to return for an ungrouped column. A functional dependency exists if the grouped columns (or a subset thereof) are the primary key of the table containing the ungrouped column.
But it requires that I know / care / remember all the primary key columns of the mapped class I'm selecting and all it's parents in the polymorphic hierarchy (Engineer.id and Employee.id in this case)
How can I obtain the list of all the primary key columns of Engineer and all parents in the polymorphic hierarchy, dynamically?
The best I could come up with so far is this function:
from sqlalchemy import inspect
def polymorphic_primary_keys(cls):
mapper = inspect(cls)
yield from (column for column in mapper.columns if column.primary_key)
if mapper.inherits is not None:
yield from polymorphic_primary_keys(mapper.inherits.entity)
Use it like this:
query = session.query(
Engineer,
).join(
Engineer.assigned_tasks,
).add_columns(
func.count(EngineeringTask.id).label('assigned_task_count'),
).group_by(*polymorphic_primary_keys(Engineer))
Where inspect(Engineer) returns Engineer.__mapper__ which is a Mapper containing:
Mapper.columns, an iterator/dictionary/attribute accessor of the columns of the mapped model
Mapper.inherits, which is the mapper of the parent entity, if any.
It's unsatisfying for a few reasons:
There is Mapper.primary_key which is supposed to be an iterator over the primary keys, but I can't use it since it returns the primary keys of the topmost parent, not the currently mapped entity, so I have to iterate over Mapper.columns and check primary_key attribute.
There is a Mapper.polymorphic_iterator() method and a Mapper.self_and_descendants attribute that both return an iterator containing the current mapper and the mappers of all "downstream" subclasses of the current entity. Why isn't there an equivalent iterator for the "upstream" superclasses?
But it gets the job done...
Supposing we have two tables, linked by a many-to-many relationship.
class Student(db.Model):
id = db.Column(UUIDType, primary_key=True)
name = db.Column(db.String(255))
courses = db.relationship('Course',
secondary=student_courses,
backref=db.backref('students'))
class Course(db.Model):
id = db.Column(UUIDType, primary_key=True)
name = db.Column(db.String(255))
I am trying to query the name of the students with the names of the courses s/he is subscribed to using a subquery, but it only shows the name of the first matching course (not all of them). In other words, I would like to retrieve (student_id, student_name, [list of course_names]).
sq = db.session.query(Student.id.label('student_id'),
Course.id.label('course_id'),
Course.name.label('course_name')) \
.join(Student.courses) \
.group_by(Student.id, Course.id).subquery('pattern_links_sq')
db.session.query(Student.id, Student.name, sq.c.course_name) \
.join(Student.courses)
.filter(Student.id == sq.c.student_id).all()
You can use array_agg function in PostgreSQL
from sqlalchemy import func
db.session.query(Student.id, Student.name, func.array_agg(Course.name))\
.join(Student.courses)\
.group_by(Student.id)\
.all()
I want to delete some elements in tables that have a polymorphic relationship in sqlalchemy. Here's the model:
class Employee(Base):
__tablename__ = 'employee'
id = Column(Integer, primary_key=True)
name = Column(String(50))
type = Column(String(50))
__mapper_args__ = {
'polymorphic_identity':'employee',
'polymorphic_on':type
}
class Engineer(Employee):
__tablename__ = 'engineer'
id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
engineer_name = Column(String(30))
__mapper_args__ = {
'polymorphic_identity':'engineer',
}
And here's how I delete it:
e = Engineer();
e.name = "John";
e.engineer_name = "Doe";
DBSession.add(e);
q = session.query(Engineer).filter(Employee.name == "John")
q.delete(False)
I get the following error, is that a bug or am I doing it the wrong way ?
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such
column: employee.name [SQL: u'DELETE FROM engineer WHERE employee.name
= ?'] [parameters: ('John',)]
I'm expecting sqlalchemy to delete the entres in the engineer and employee tables.
First you should define the on delete behaviour of this relationship:
id = Column(Integer, ForeignKey('employee.id', ondelete='CASCADE'), primary_key=True)
Then, using the ORM, you can delete all engineers with name "John" through a loop:
eng_list = session.query(Engineer).filter(Employee.name == "John").all()
for eng in eng_list:
session.delete(eng)
session.commit()
This will delete the records from both the Employee and Engineer tables.
update: comment on error message:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such
column: employee.name [SQL: u'DELETE FROM engineer WHERE employee.name
= ?'] [parameters: ('John',)]
Your attempt tries to Delete from Engineer with a join to Employee (to access the field Employee.name). But this join is missing from the query sqlalchemy is emitting to the backend.
I don't think SQLite supports deleting with joins. Perhaps you can try to run session.query(Engineer).filter(Employee.name == "John").delete() against a different backend, and sqlalchemy may be able to emit the proper SQL statement. I haven't tried it though.
update 2: On backends that respect foreign key constraints (and the onupdate constraint has been set to cascade), it should be sufficient to delete the row in the parent row, and the linked rows in the child will automatically be deleted.
I tried this example with both MySQL & Postgresql backends, and the following query deleted the row from both tables (employee & engineer):
session.query(Employee).filter(Employee.name=='John').delete()
For some reason, on Sqlite, this only deletes the record from employee.
Because doing the joined DELETE is not supported directly, I found an easy workaround is to use your normal joined query to select the ids to delete, then pass those ids to a separate DELETE query.
One minor annoyance is that since your returned ids are integers you would likely run into this error like I did if you try to pass those ids (technically an array of tuples) directly to the DELETE query. A simple intermediate conversion to strings fixes that.
So all together:
ids_to_delete = session.query(Engineer.id). \
filter(Employee.name == "John"). \
all()
# Convert the resulting int tuples to simple strings:
id_strings = [str(id_[0]) for id_ in ids_to_delete]
session.query(Engineer). \
filter(Engineer.id.in_(id_strings)). \
delete(synchronize_session=False)
Now I sort the data in the database by its attribute 1.
If there is a tie of different items with same value of attribute 1, the data seems to be sorted by its id.
However, I would like to break the tie by sorting by desc(id). How could I change the default sorting criteria of the database if there is a tie?
Thanks!
Update
Since version 1.1 the order_by parameter in the mapper configuration has been deprecated. Instead Query.order_by must be used.
db.query(User).order_by(User.fullname)
# or in desc order
db.query(User).order_by(User.fullname.desc())
I left here the original answer for historial purposes:
This is possible by means of the mapper configuration.
If you have a user table and want to retrieve the records always ordered by fullname something like this should works:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String)
fullname = Column(String)
password = Column(String)
__mapper_args__ = {
"order_by": fullname,
}
def __repr__(self):
return f"<User(id='{self.id}', name='{self.name}', fullname='{self.fullname}')>"
Default order_by is ascending, if you want to reverse the order this can be used:
__mapper_args__ = {
"order_by": fullname.desc(),
}
The order is entirely determined by the database, not SQLAlchemy. With plain SQL you just add additional ORDER BY clauses, in SQLAlchemy ORM you do the same by chaining order_by methods. For example:
for eq in session.query(Equipment).order_by(Equipment.model_id).order_by(Equipment.name).all():
print (eq)
Whichever is left-most is the primary sort.