I have following Group and Contact model in flask with Sql Alchemy ORM
group_contact = db.Table(
'group_contact',
db.Column('group_id', db.Integer, db.ForeignKey(
'group.id')),
db.Column('contact_id', db.Integer, db.ForeignKey(
'contact.id')),
db.PrimaryKeyConstraint('group_id', 'contact_id')
)
class Group(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100))
class Contact(db.Model):
id = db.Column(db.Integer, primary_key=True)
phone = db.Column(db.String(15), nullable=False, unique=True)
groups = db.relationship(
"Group", secondary=group_contact, backref='contacts')
Now I need to query Contact with groups:
contacts = Contact.query.join(Group, Contact.groups).all()
for contact in contacts:
print(contact.groups)
Here the problem is number of SQL query increases as number of contact increases when I execute above code.
Django ORM has prefetch_related() with queryset which does the following according to django docs.
prefetch_related, on the other hand, does a separate lookup for each relationship, and does the ‘joining’ in Python. This allows it to prefetch many-to-many and many-to-one objects, which cannot be done using select_related, in addition to the foreign key and one-to-one relationships that are supported by select_related.
Now I am trying to do the same thing with Sql Alchemy by the following code:
contacts = Contact.query.all()
contact_groups = group_contact.query.join(
Group
).filter(group_contact.contact_id.in_([item.id for item in contacts]))
But this gives me this error:
AttributeError: 'Table' object has no attribute 'query'
How can I get prefetch_related like feature from django with SqlAlchemy?
You want to tell SQLAlchemy to eagerly load related objects by using a relationship loading technique. SQLAlchemy can be told to load the groups together with the contacts in a single query.
For just this one query, you can add joinedload() option (it is available via the Flask-SQLAlchemy db object):
contacts = Contact.query.options(db.joinedload(Contact.groups)).all()
This pre-loads the Contact.groups attribute on each matched contact:
for contact in contacts:
# no new query issued to fetch groups, the data for the groups
# is already available
print(contact.groups)
The query executed looks like this:
SELECT
contact.id AS contact_id,
contact.phone AS contact_phone,
group_1.id AS group_1_id,
group_1.name AS group_1_name
FROM contact
LEFT OUTER JOIN (
group_contact AS group_contact_1
JOIN "group" AS group_1 ON group_1.id = group_contact_1.group_id
) ON contact.id = group_contact_1.contact_id
You can also set a default loading strategy for the relationship on the model; to always eagerly load groups, use lazy='joined' on the relationship:
class Contact(db.Model):
# ...
groups = db.relationship(
"Group", secondary=group_contact, backref='contacts',
lazy='joined')
Related
My app Model is structured as so:
user_jobs = db.Table('user_jobs',
db.Column('user_id', db.Integer, db.ForeignKey('user.id')),
db.Column('job_id', db.Integer, db.ForeignKey('market.id'))
)
class User(UserMixin, db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(64), index=True, unique=True)
# Other user model fields....
jobs = db.relationship('Job', secondary=user_jobs, backref='users')
class Job(db.Model):
id = db.Column(db.Integer, primary_key=True)
# Other related fields and relationships
quotes = db.relationship('Quote', backref='job', lazy='dynamic')
class Quote(db.Model):
id = db.Column(db.Integer, primary_key=True)
timestamp = db.Column(db.DateTime, index=True, default=datetime.utcnow)
price = db.Column(db.Integer())
# Other related fields
job_id = db.Column(db.Integer, db.ForeignKey('job.id'))
This model allows users to follow multiple jobs while jobs can have multiple followed users (Many to Many). A job can have multiple Quotes (One to Many).
In my flask app, I am creating a dashboard that displays the users followed jobs. For the followed jobs on the dashboard, I want to display the most recent Quote price and timestamp.
My current thinking is to create a function on the user model to return a joined table of User - Job - Quote, ordering by desc and limit(1). I however am stuck on how to do this.
class User(UserMixin, db.Model):
.....
def get followed_jobs(self):
return ...
Any help would be greatly appreciated.
EDIT:
Given there is a list of users and I'm trying to find the latest quotes that user 1 is following, the raw SQL appears to be:
Select
*
FROM
(
SELECT
job.id, job.job_name, latest_quote.timestamp,
latest_quote.price, user_job.user_id
FROM
(SELECT
job_id, max(timestamp) AS timestamp,
price FROM quote
GROUP BY job_id) AS latest_quote
JOIN
job
ON
job.id = latest_quote.job_id
JOIN
user_job
ON
user_job.job_id = latest_quote.job_id
) as aquery
WHERE user_id = 1;
Can this be made more efficient in SQL?
The below answer might be helpful to get the required data for many-to-many relationship.
SqlAlchemy and Flask, how to query many-to-many relationship
If you require data in serialisable format in many-to-many relationship which is your use-case, I would suggest you use nested schemas in marshmallow.
Flask Marshmallow/SqlAlchemy: Serializing many-to-many relationships
I'm trying to build tests for flask-sqlalchemy models to confirm they match the tables in the database. Using reflection, I can get the db tables, columns, and foreign keys. With the models I know how to check tablename and column names, but I can't figure out how to see the defined foreign key relationships programatically.
Model example:
class Example(db.Model):
__table_args__ = {'schema': 'defined_schema'}
__tablename__ = 'example'
id = db.Column(db.BigInteger, primary_key=True)
name = db.Column(db.String)
info = db.Column(db.BigInteger, db.ForeignKey(Info.id))
descr = db.Column(db.String)
Given a Model m, I've tried:
dir(m)
m.__dict__
Using sqlalchemy.inspect(model).columns.foreign_keys and a bit of string manipulation, I was able to compare model foreign key relationships to those in the DB
I am trying to store a list of models within the field of another model. Here is a trivial example below, where I have an existing model, Actor, and I want to create a new model, Movie, with the field Movie.list_of_actors:
import uuid
from sqlalchemy import Boolean, Column, Integer, String, DateTime
from sqlalchemy.schema import ForeignKey
rom sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
class Actor(Base):
__tablename__ = 'actors'
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name = Column(String)
nickname = Column(String)
academy_awards = Column(Integer)
# This is my new model:
class Movie(Base):
__tablename__ = 'movies'
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
title = Column(String)
# How do I make this a list of foreign keys???
list_of_actors = Column(UUID(as_uuid=True), ForeignKey('actors.id'))
I understand that this can be done with a many-to-many relationship, but is there a more simple solution? Note that I don't need to look up which Movie's an Actor is in - I just want to create a new Movie model and access the list of my Actor's. And ideally, I would prefer not to add any new fields to my Actor model.
I've gone through the tutorials using the relationships API, which outlines the various one-to-many/many-to-many combinations using back_propagates and backref here: http://docs.sqlalchemy.org/en/latest/orm/basic_relationships.html But I can't seem to implement my list of foreign keys without creating a full-blown many-to-many implementation.
But if a many-to-many implementation is the only way to proceed, is there a way to implement it without having to create an "association table"? The "association table" is described here: http://docs.sqlalchemy.org/en/latest/orm/basic_relationships.html#many-to-many ? Either way, an example would be very helpful!
Also, if it matters, I am using Postgres 9.5. I see from this post there might be support for arrays in Postgres, so any thoughts on that could be helpful.
Update
It looks like the only reasonable approach here is to create an association table, as shown in the selected answer below. I tried using ARRAY from SQLAlchemy's Postgres Dialect but it doesn't seem to support Foreign Keys. In my example above, I used the following column:
list_of_actors = Column('actors', postgresql.ARRAY(ForeignKey('actors.id')))
but it gives me an error. It seems like support for Postgres ARRAY with Foreign Keys is in progress, but still isn't quite there. Here is the most up to date source of information that I found: http://blog.2ndquadrant.com/postgresql-9-3-development-array-element-foreign-keys/
If you want many actors to be associated to a movie, and many movies be associated to an actor, you want a many-to-many. This means you need an association table. Otherwise, you could chuck away normalisation and use a NoSQL database.
An association table solution might resemble:
class Actor(Base):
__tablename__ = 'actors'
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
name = Column(String)
nickname = Column(String)
academy_awards = Column(Integer)
class Movie(Base):
__tablename__ = 'movies'
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
title = Column(String)
actors = relationship('ActorMovie', uselist=True, backref='movies')
class ActorMovie(Base):
__tablename__ = 'actor_movies'
actor_id = Column(UUID(as_uuid=True), ForeignKey('actors.id'))
movie_id = Column(UUID(as_uuid=True), ForeignKey('movies.id'))
If you don't want ActorMovie to be an object inheriting from Base, you could use sqlachlemy.schema.Table.
I have these models:
class User(UserMixin, db.Model):
__tablename__ = 'users_user'
...
country = db.Column(db.Integer, db.ForeignKey('countries.id'))
class Country(db.Model):
__tablename__ = 'countries'
id = db.Column(db.Integer, primary_key=True)
...
user_country = db.relationship('User', backref='user_country', lazy='joined')
I am trying this query:
User.query.options(joinedload(Country.user_country)).filter_by(id=current_user.get_id()).first()
That will throw this error:
ArgumentError: Can't find property 'user_country' on any entity specified in this Query.
Note the full path from root (Mapper|User|users_user) to target entity must be specified.
What is wrong here?
The joinedload here is unnecessary.
By default relationships are lazily-loaded. This causes additional SELECT queries to be issued to retrieve the data. joinedload is one of the ways to force the relationship to be eagerly loaded by using a JOIN instead.
In this case, however, you've defaulted the relationship between User and Country to use eager loading by specifying lazy='joined'. This would reduce your query to
User.query.filter(id=current_user.get_id()).first()
While this will help you with the ArgumentError, we can go a little further. The query itself is unnecessary as well. current_user already has the data for its related Country because of the eager join. Accessing current_user.user_country will not send any additional queries to the database.
I have the following scenario:
class Author(Base):
__tablename__ = 'author'
id = Column(Integer, primary_key = True)
name = Column(String)
books = relationship('Books', backref='author')
class Book(Base):
__tablename__ = 'book'
id = Column(Integer, primary_key = True)
title = Column(String)
What I would like to do is load all authors who have a book containing SQL in
the title. i.e.
authors = session.query(Author)\
.join(Author.books)\
.filter(Book.title.like('%SQL%')\
.all()
Seems simple.
What I would then like to do is iterate over the authors and display their
books. I would expect that when accessing authors[0].books, it will return ONLY
books that have 'SQL' in their title. However, I am getting ALL books assigned
to that author. The filter is applied to the list of authors but not their
books when I access the relationship.
How can I structure my query such that if I filter on a relationship (i.e.
books), when I go to access that relationship, the filtering is still applied?
Please read Routing Explicit Joins/Statements into Eagerly Loaded Collections. Then using contains_eager you can structure your query and get exactly what you want:
authors = (
session.query(Author)
.join(Author.books)
.options(contains_eager(Author.books)) # tell SA that we load "all" books for Authors
.filter(Book.title.like('%SQL%'))
).all()
Please note that you are actually tricking sqlalchemy into thinking that it has loaded all the collection of Author.books, and as such your session will know false information about the real state of the world.
In short, this is not possible. (If it were, an Author instance would have different books attribute depending on how it was queried, which doesn't make sense.)
What you could do instead is query the reverse relationship:
books = session.query(Book) \
.filter(Book.title.like('%SQL%')) \
.all()
Then you can access .author on each book to collect books written by the same author together.