My app Model is structured as so:
user_jobs = db.Table('user_jobs',
db.Column('user_id', db.Integer, db.ForeignKey('user.id')),
db.Column('job_id', db.Integer, db.ForeignKey('market.id'))
)
class User(UserMixin, db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(64), index=True, unique=True)
# Other user model fields....
jobs = db.relationship('Job', secondary=user_jobs, backref='users')
class Job(db.Model):
id = db.Column(db.Integer, primary_key=True)
# Other related fields and relationships
quotes = db.relationship('Quote', backref='job', lazy='dynamic')
class Quote(db.Model):
id = db.Column(db.Integer, primary_key=True)
timestamp = db.Column(db.DateTime, index=True, default=datetime.utcnow)
price = db.Column(db.Integer())
# Other related fields
job_id = db.Column(db.Integer, db.ForeignKey('job.id'))
This model allows users to follow multiple jobs while jobs can have multiple followed users (Many to Many). A job can have multiple Quotes (One to Many).
In my flask app, I am creating a dashboard that displays the users followed jobs. For the followed jobs on the dashboard, I want to display the most recent Quote price and timestamp.
My current thinking is to create a function on the user model to return a joined table of User - Job - Quote, ordering by desc and limit(1). I however am stuck on how to do this.
class User(UserMixin, db.Model):
.....
def get followed_jobs(self):
return ...
Any help would be greatly appreciated.
EDIT:
Given there is a list of users and I'm trying to find the latest quotes that user 1 is following, the raw SQL appears to be:
Select
*
FROM
(
SELECT
job.id, job.job_name, latest_quote.timestamp,
latest_quote.price, user_job.user_id
FROM
(SELECT
job_id, max(timestamp) AS timestamp,
price FROM quote
GROUP BY job_id) AS latest_quote
JOIN
job
ON
job.id = latest_quote.job_id
JOIN
user_job
ON
user_job.job_id = latest_quote.job_id
) as aquery
WHERE user_id = 1;
Can this be made more efficient in SQL?
The below answer might be helpful to get the required data for many-to-many relationship.
SqlAlchemy and Flask, how to query many-to-many relationship
If you require data in serialisable format in many-to-many relationship which is your use-case, I would suggest you use nested schemas in marshmallow.
Flask Marshmallow/SqlAlchemy: Serializing many-to-many relationships
Related
I need help improving my SQLAlchemy query. I'm using Python 3.7, SQLAlchemy 1.3.15 and PosgresSQL 9.4.3 as database. I'm trying to return the count of appointments for a specific date and timeslot. However, my appointments and appointment slot tables are separate and I'm having to query both models/tables to get the desired results. Here's what I have;
Appointments Model
The appointment table have a few columns, which includes a foreign key to the appointment slots table.
class Appointment(ResourceMixin, db.Model):
__tablename__ = 'appointments'
id = db.Column(db.Integer, primary_key=True)
user_id = db.Column(db.Integer, db.ForeignKey('users.id', onupdate='CASCADE', ondelete='CASCADE'), index=True, nullable=True)
slot_id = db.Column(db.Integer, db.ForeignKey('appointment_slots.id', onupdate='CASCADE', ondelete='CASCADE'), index=True, nullable=False)
appointment_date = db.Column(db.DateTime(), nullable=False)
appointment_type = db.Column(db.String(128), nullable=False, default='general')
Appointment Slots Table
The appointment slots table contains the time slots for the appointments. The Model consist of a relationship back to the appointments table.
class AppointmentSlot(ResourceMixin, db.Model):
__tablename__ = 'appointment_slots'
id = db.Column(db.Integer, primary_key=True)
# Relationships.
appointments = db.relationship('Appointment', uselist=False,
backref='appointments', lazy='joined', passive_deletes=True)
start_time = db.Column(db.String(5), nullable=False, server_default='08:00')
end_time = db.Column(db.String(5), nullable=False, server_default='17:00')
SQLAlchemy Query
Currently I'm running the following SQLAlchemy query to get the appointment count for a specific date and time slot;
appointment_count = db.session.query(func.count(Appointment.id)).join(AppointmentSlot)\
.filter(and_(Appointment.appointment_date == date, AppointmentSlot.id == Appointment.id,
AppointmentSlot.start_time == time)).scalar()
The query above return the correct results, which is a single digit value, but I'm worried that the query is not optimized. Currently the query returns in 380ms , but there's only 8 records in the appointments and appointment_slots tables. These tables will eventually have in the 100s of thousands of records. I'm worried that even though the query is working now that it will eventually struggle with an increase of records.
How can I improved or optimized this query to improve performance? I was looking at SQLAlchemy subqueries using the appointment relationship on the appointment_slots table, but was unable to get it to work and confirm the performance. I'm thinking there must be a better way to run this query especially using the appointments relationship on the appointment_slots table, but I'm currently stumped. Any suggestions?
I was incorrect about the query load time. I was actually looking at the page load that was 380ms. I also change the some fields on the models by removing the slot_id from the appointments model and adding a appointment_id foreign key to the appointment_slots model. The page load for the following query;
appointment_count = db.session.query(func.count(Appointment.id)).join(AppointmentSlot)\
.filter(and_(Appointment.appointment_date == date,
AppointmentSlot.appointment_id == Appointment.id, AppointmentSlot.start_time == time)).scalar()
ended up being; 0.4637ms.
However, I still tried to improve the query and was able to do so by using a SQLAlchemy subquery. The following subquery;
subquery = db.session.query(Appointment.id).filter(Appointment.appointment_date == date).subquery()
query = db.session.query(func.count(AppointmentSlot.id))\
.filter(and_(AppointmentSlot.appointment_id.in_(subquery),
AppointmentSlot.start_time == time)).scalar()
Return a load time of 0.3700ms which shows a much better performance than using the join query.
I'm a bit of a database noob. I'm using flask sqlalchemy and postgresql to design a user system on a website, where each user has a unique, not null referral code. Other users can each use a single referral code, and when they do, both the owner and the user of the code receive some benefits. I'm wondering what the best way to design this is. A many to many relationship?
What I'm currently doing on the user table (other columns like email/password omitted) is:
class User(db.Model):
__tablename__ = 'user'
id = db.Column(db.Integer, primary_key=True)
referral_code = db.relationship('Refcodes', backref='owner', uselist=False)
referrals = db.relationship('Referrals', backref='user', uselist=False)
And on the 'refcodes' and 'referrals' table:
class Refcodes(db.Model):
__tablename__ = 'refcodes'
id = db.Column(db.Integer, primary_key=True)
code = db.Column(db.String(10), nullable=False, unique=True)
owner_id = db.Column(db.ForeignKey('user.id'))
referrals = db.Relationship('Referrals', backref='code')
class Referrals(db.Model):
__tablename__ = 'referrals'
id = db.Column(db.Integer, primary_key=True)
code_id = db.Column(db.ForeignKey('refcodes.id'))
user_id = db.Column(db.ForeignKey('users.id'))
I haven't tested it yet since I'm still designing the login/registration page, but my question is if there is some sort of best practice/exact method for similar problems. Apologies for the bad English and the possibly vague/confusing initial questions and thanks in advance for any replies.
I'm writing the backend of a simple e-commerce site with Flask and Flask-SQLAlchemy but i'm having trouble with SQLAlchemy db creation.
The DB diagram is as follows
I wrote the models for my database but when i try to test them i'm getting this error message on the One to Many relationship between Parent and OrderItem:
sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Product.order_item - there are no foreign keys linking these tables. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression.
The models are the following:
class OrderItem(db.Model):
__tablename__ = 'OrderItem'
order_item_number = db.Column(db.Integer(), primary_key=True)
order_quantity = db.Column(db.Integer())
order_item_product_number = db.Column(db.Integer(), db.ForeignKey('Product.product_number'))
order_item_order_number = db.Column(db.Integer(), db.ForeignKey('Order.order_number'))
def __repr__(self):
return '<OrderItem {}{}>'.format(self.order_item_number, self.order_order_item)
class Product(db.Model):
_tablename__ = 'Product'
product_number = db.Column(db.Integer(), primary_key=True)
product_availability = db.Column(db.Boolean())
product_price = db.Column(db.Integer())
product_unit = db.Column(db.Integer())
product_discount = db.Column(db.Integer(), default=0)
order_item = db.relationship('OrderItem', backref='Product')
def __repr__(self):
return '<Product {}'.format(self.product_number)
Everything seems correct, i have the very same relationship pattern with Customer and Order but i don't get why i'm getting prompted with this error here.
I've already checked for any mistyping in attribute names but everything seems correct, from the table names to the attribute names.
Any help would be appreciated, thanks in advance.
I have following Group and Contact model in flask with Sql Alchemy ORM
group_contact = db.Table(
'group_contact',
db.Column('group_id', db.Integer, db.ForeignKey(
'group.id')),
db.Column('contact_id', db.Integer, db.ForeignKey(
'contact.id')),
db.PrimaryKeyConstraint('group_id', 'contact_id')
)
class Group(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100))
class Contact(db.Model):
id = db.Column(db.Integer, primary_key=True)
phone = db.Column(db.String(15), nullable=False, unique=True)
groups = db.relationship(
"Group", secondary=group_contact, backref='contacts')
Now I need to query Contact with groups:
contacts = Contact.query.join(Group, Contact.groups).all()
for contact in contacts:
print(contact.groups)
Here the problem is number of SQL query increases as number of contact increases when I execute above code.
Django ORM has prefetch_related() with queryset which does the following according to django docs.
prefetch_related, on the other hand, does a separate lookup for each relationship, and does the ‘joining’ in Python. This allows it to prefetch many-to-many and many-to-one objects, which cannot be done using select_related, in addition to the foreign key and one-to-one relationships that are supported by select_related.
Now I am trying to do the same thing with Sql Alchemy by the following code:
contacts = Contact.query.all()
contact_groups = group_contact.query.join(
Group
).filter(group_contact.contact_id.in_([item.id for item in contacts]))
But this gives me this error:
AttributeError: 'Table' object has no attribute 'query'
How can I get prefetch_related like feature from django with SqlAlchemy?
You want to tell SQLAlchemy to eagerly load related objects by using a relationship loading technique. SQLAlchemy can be told to load the groups together with the contacts in a single query.
For just this one query, you can add joinedload() option (it is available via the Flask-SQLAlchemy db object):
contacts = Contact.query.options(db.joinedload(Contact.groups)).all()
This pre-loads the Contact.groups attribute on each matched contact:
for contact in contacts:
# no new query issued to fetch groups, the data for the groups
# is already available
print(contact.groups)
The query executed looks like this:
SELECT
contact.id AS contact_id,
contact.phone AS contact_phone,
group_1.id AS group_1_id,
group_1.name AS group_1_name
FROM contact
LEFT OUTER JOIN (
group_contact AS group_contact_1
JOIN "group" AS group_1 ON group_1.id = group_contact_1.group_id
) ON contact.id = group_contact_1.contact_id
You can also set a default loading strategy for the relationship on the model; to always eagerly load groups, use lazy='joined' on the relationship:
class Contact(db.Model):
# ...
groups = db.relationship(
"Group", secondary=group_contact, backref='contacts',
lazy='joined')
I have a User table and a Friend table. The Friend table holds two foreign keys both to my User table as well as a status field. I am trying to be able to call attributes from my User table on a Friend object. For example, I would love to be able to do something like, friend.name, or friend.email.
class User(Base):
""" Holds user info """
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(25), unique=True)
email = Column(String(50), unique=True)
password = Column(String(25))
admin = Column(Boolean)
# relationships
friends = relationship('Friend', backref='Friend.friend_id',primaryjoin='User.id==Friend.user_id', lazy='dynamic')
class Friend(Base):
__tablename__ = 'friend'
user_id = Column(Integer, ForeignKey(User.id), primary_key=True)
friend_id = Column(Integer, ForeignKey(User.id), primary_key=True)
request_status = Column(Boolean)
When I get friend objects all I have is the 2 user_ids and i want to display all properties of each user so I can use that information in forms, etc. I am new to sqlalchemy - still trying to learn more advanced features. This is just a snippet from a larger Flask project and this feature is going to be for friend requests, etc. I've tried to look up association objects, etc, but I am having a hard with it.
Any help would be greatly appreciated.
First, if you're using flask-sqlalchemy, why are you using directly sqlalchemy instead of the Flask's db.Model?
I strongly reccomend to use flask-sqlalchemy extension since it leverages the sessions and some other neat things.
Creating a proxy convenience object is straightforward. Just add the relationship with it in the Friend class.
class Friend(Base):
__tablename__ = 'friend'
user_id = Column(Integer, ForeignKey(User.id), primary_key=True)
friend_id = Column(Integer, ForeignKey(User.id), primary_key=True)
request_status = Column(Boolean)
user = relationship('User', foreign_keys='Friend.user_id')
friend = relationship('User', foreign_keys='Friend.friend_id')
SQLAlchemy will take care of the rest and you can access the user object simply by:
name = friend.user.name
If you plan to use the user object every time you use the friend object specify lazy='joined' in the relationship. This way it loads both object in a single query.