I'm getting a bit stuck with relationships within relationships in sqlalchemy. I have a model like this:
engine = create_engine('sqlite:///tvdb.db', echo=True)
Base = declarative_base(bind=engine)
episodes_writers = Table('episodes_writers_assoc', Base.metadata,
Column('episode_id', Integer, ForeignKey('episodes.id')),
Column('writer_id', Integer, ForeignKey('writers.id')),
Column('series_id', Integer, ForeignKey('episodes.series_id'))
class Show(Base):
__tablename__ = 'shows'
id = Column(Integer, primary_key = True)
episodes = relationship('Episode', backref = 'shows')
class Episode(Base):
__tablename__ = 'episodes'
id = Column(Integer, primary_key = True)
series_id = Column(Integer, ForeignKey('shows.id'))
writers = relationship('Writer', secondary = 'episodes_writers',
backref = 'episodes')
class Writer(Base):
__tablename__ = 'writers'
id = Column(Integer, primary_key = True)
name = Column(Unicode)
So there is one-to-many between shows and episodes, and many-to-many between episodes and writers, but I now want to create a many-to-many relationship between shows and writers, based on the fact that a writer is associated with an episode of a show. I tried to add another column to the assoc table but that results in the following exception:
sqlalchemy.exc.ArgumentError: Could not determine join condition between parent/child tables on relationship Episode.writers. Specify a 'primaryjoin' expression. If 'secondary' is present, 'secondaryjoin' is needed as well.
So I'm obviously doing something wrong with my relationship declarations. How do I create the right primaryjoins to achieve what I'm trying to do here?
The fix is suggested right in the error message. Anyway, you have no need for a column that denormalises your schema; writer.shows can be an associationproxy.
Related
I am still a beginner in Python and I am stuck with the following relation.
Three tables:
tx_bdproductsdb_domain_model_product
sys_category
sys_category_record_mm
sys_category class looks like this:
class Category(Base):
__tablename__ = "sys_category"
uid = Column(
Integer,
ForeignKey("sys_category_record_mm.uid_local"),
primary_key=True,
autoincrement=True,
)
title = Column(String)
products = relationship(
"Product",
uselist=False,
secondary="sys_category_record_mm",
back_populates="categories",
foreign_keys=[uid],
)
Products looks like this:
class Product(Base):
__tablename__ = "tx_bdproductsdb_domain_model_product"
uid = Column(
Integer,
ForeignKey(SysCategoryMMProduct.uid_foreign),
primary_key=True,
autoincrement=True,
)
category = Column(Integer)
categories = relationship(
Category,
secondary=SysCategoryMMProduct,
back_populates="products",
foreign_keys=[uid],
)
And here is the mm table class that should link the two.
class SysCategoryMMProduct(Base):
__tablename__ = "sys_category_record_mm"
uid_local = Column(Integer, ForeignKey(Category.uid), primary_key=True)
uid_foreign = Column(
Integer, ForeignKey("tx_bdproductsdb_domain_model_product.uid")
)
fieldname = Column(String)
I'm currently stuck, does anyone have any ideas? I get the following messages in the console:
sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Category.products - there are no foreign keys linking these tables via secondary table 'sys_category_record_mm'. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify 'primaryjoin' and 'secondaryjoin' expressions.
root#booba:/var/pythonWorks/crawler/develop/releases/current# python3 Scraper2.py
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/relationships.py", line 2739, in _determine_joins
self.secondaryjoin = join_condition(
File "<string>", line 2, in join_condition
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/selectable.py", line 1229, in _join_condition
raise exc.NoForeignKeysError(
sqlalchemy.exc.NoForeignKeysError: Can't find any foreign key relationships between 'tx_bdproductsdb_domain_model_product' and 'sys_category_record_mm'.
sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Category.products - there are no foreign keys linking these tables via secondary table 'sys_category_record_mm'. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify 'primaryjoin' and 'secondaryjoin' expressions.
Thank you :)
When using an association class you should reference the association directly. You need this instead of secondary because you have data associated with the link (ie. fieldname). I changed some of your naming schema to make it more clear.
There is a pretty good explanation of the association pattern in the sqlalchemy docs. There is a big red warning at the end of that section about mixing the use of the secondary and the Association pattern.
I use backref="related_categories" to automatically create the property related_categories on Product. This is a list of association objects, and not actual categories.
from sqlalchemy import (
create_engine,
Integer,
String,
ForeignKey,
)
from sqlalchemy.schema import (
Column,
)
from sqlalchemy.orm import declarative_base, relationship
from sqlalchemy.orm import Session
Base = declarative_base()
# This connection string is made up
engine = create_engine(
'postgresql+psycopg2://user:pw#/db',
echo=False)
class Category(Base):
__tablename__ = "categories"
uid = Column(
Integer,
primary_key=True,
autoincrement=True,
)
title = Column(String)
class Product(Base):
__tablename__ = "products"
uid = Column(
Integer,
primary_key=True,
autoincrement=True,
)
title = Column(String)
class SysCategoryMMProduct(Base):
__tablename__ = "categories_products"
uid = Column(Integer, primary_key=True)
category_uid = Column(Integer, ForeignKey("categories.uid"))
product_uid = Column(Integer, ForeignKey("products.uid"))
fieldname = Column(String)
product = relationship(
"Product",
backref="related_categories",
)
category = relationship(
"Category",
backref="related_products",
)
Base.metadata.create_all(engine)
with Session(engine) as session:
category = Category(title="kitchen")
session.add(category)
product = Product(title="spoon")
session.add(product)
association = SysCategoryMMProduct(
product=product,
category=category,
fieldname="Extra metadata")
session.add(association)
session.commit()
category = session.query(Category).first()
assert len(category.related_products) == 1
assert category.related_products[0].product.related_categories[0].category == category
q = session.query(Category).join(Category.related_products).join(SysCategoryMMProduct.product).filter(Product.title == "spoon")
print (q)
assert q.first() == category
The last query looks like:
SELECT categories.uid AS categories_uid, categories.title AS categories_title
FROM categories JOIN categories_products ON categories.uid = categories_products.category_uid JOIN products ON products.uid = categories_products.product_uid
WHERE products.title = 'spoon'
I can return the name of the table of which a foreign key references, but I want to do the opposite.
Suppose I have two tables, Users and Address, and Address has a foreign key to Users (one-to-many). I also made this bidirectional so that I can get the User from the address table.
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
address = relationship("Address", back_populates="user")
class Address(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True, autoincrement=True)
company_name = Column(String)
user_id = Column(Integer, ForeignKey('user.id'))
user = relationship("User", back_populates="address")
If I wanted to figure out the name of the parent table, in a way that works for any other table in a similar relationship, I can to this:
for i in Address.__table__.foreign_key_constraints:
print(i.referred_table)
tab = Table(Address.__tablename__, Base.metadata, autoload_with=engine, extend_existing=True)
for i in tab.foreign_key_constraints:
print(i.referred_table)
#both output "user" which is the __tablename__ of User
This was me accessing the User table using the foreign key constraint attribute. But how do I do the opposite, using an attribute from User to access Address? I mainly want to know so I can handle relationships in a bulk core insert.
tab = Table(User.__tablename__, Base.metadata, autoload_with=engine, extend_existing=True)
records = df.to_dict(orient="records")
insert_stmt = sqlalchemy.dialects.sqlite.insert(tab).values(records)
#list of primary keys
pks = [pk.name for pk in tab.primary_key]
update_columns = {col.name: col for col in insert_stmt.excluded if col.name not in pks}
update_statement = insert_stmt.on_conflict_do_update(index_elements=pks, set_=update_columns)
The above works once I execute the on_conflict_do_update statement, but it does not generate any Address rows.
I found one way of doing it, but it's not as clean as I wanted.
tab = list(inspect(User).relationships)[0].entity.local_table
For any declarative table (classes), you can inspect them to get a Mapper Object. The Mapper Object has relationships as an attribute, which returns a collection of relationships.
inspect(User).relationships
#Get the relationship, note this will return the relationship attribute from the class
list(inspect(User).relationships)[0]
So I then get the mapper/class this relationship belongs to with .entity. Then from the declarative table, a table object is returned with the local_table attribute
entity = list(inspect(User).relationships)[0].entity
tab = entity.local_table
This statement should evaluate to True
tab == Address.__table__
In my Flask application I am using SQLAlchemy, all tables are defined in one single file models.py:
training_ids_association_table = db.Table(
"training_ids_association",
db.Model.metadata,
Column("training_id", Integer, ForeignKey("training_sessions.id")),
Column("ids_id", Integer, ForeignKey("image_data_sets.id")),
)
class ImageDataSet(db.Model):
__tablename__ = "image_data_sets"
id = Column(Integer, primary_key=True)
trainings = relationship("Training", secondary=training_ids_association_table, back_populates="image_data_sets")
class TrainingSession(db.Model):
__tablename__ = "training_sessions"
id = Column(Integer, primary_key=True)
image_data_sets = relationship("DataSet", secondary=training_ids_association_table, back_populates="trainings")
So what I want to achieve here is a many-to-many relationship:
One ImageDataSet can belong to multiple TrainingSession's
One TrainingSession can include multiple ImageDataSet's
However, as soon as I call TrainingSession.query() in my code, the following error is raised:
Exception has occurred: InvalidRequestError
When initializing mapper mapped class ImageDataSet->image_data_sets, expression 'Training' failed to locate a name ('Training'). If this is a class name, consider adding this relationship() to the <class 'app.base.models.ImageDataSet'> class after both dependent classes have been defined.
I found some related threads here, but they are either asking for one-to-many relationships, or they define their tables in different files. Both is not the case here.
Any ideas what I am doing wrong?
You mispelled the names of the models, try this:
training_ids_association_table = db.Table(
"training_ids_association",
db.Model.metadata,
Column("training_id", Integer, ForeignKey("training_sessions.id")),
Column("ids_id", Integer, ForeignKey("image_data_sets.id")),
)
class ImageDataSet(db.Model):
__tablename__ = "image_data_sets"
id = Column(Integer, primary_key=True)
trainings = relationship("TrainingSession", secondary=training_ids_association_table, back_populates="image_data_sets")
class TrainingSession(db.Model):
__tablename__ = "training_sessions"
id = Column(Integer, primary_key=True)
image_data_sets = relationship("ImageDataSet", secondary=training_ids_association_table, back_populates="trainings")
I am playing with a toy example to see the back populates in action but hitting an error that I can't understand. Below I have two 'models' that back populate each other. When I try to create a User object it throws an error. What am I missing?
"sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Child.user - there are no foreign keys linking these tables. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression."
engine = create_engine('sqlite:///:memory:', echo=False)
Session = sessionmaker(bind=engine)
session = Session()
Base = declarative_base(bind=engine)
class User(Base):
__tablename__ = 'user'
user_id = Column("UserID", Integer, primary_key=True)
name = Column("Name", String(50))
age = Column("Age", SmallInteger)
child = relationship("Child", back_populates="user")
class Child(Base):
__tablename__ = 'child'
child_id = Column("ChildID", Integer, primary_key=True)
school = Column("School", String)
grade = Column("Grade", String)
user_id = Column("UserID", Integer, ForeignKey('User.UserID'), index=True, nullable=True)
user = relationship("User", back_populates="child")
ForeignKey requires the table and column name, not model and attribute name, so it should be:
user_id = Column("UserID", Integer, ForeignKey('user.UserID'), ...)
because your User model has a table name of user.
I'm a little confused over the use of the two modules from SQLAlchemy. This is the code I have:
Base = declarative_base()
class Restaurant(Base):
__tablename__ = 'restaurant'
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False)
class MenuItem(Base):
__tablename__ = 'menu_item'
name =Column(String(80), nullable = False)
id = Column(Integer, primary_key = True)
description = Column(String(250))
price = Column(String(8))
course = Column(String(250))
restaurant_id = Column(Integer,ForeignKey('restaurant.id'))
restaurant = relationship(Restaurant)
I understand that ForeignKey is used to define the foreign key relationship between the restaurant_id column of menu_item table and the id column of restaurant table. But why then is restaurant = relationship(Restaurant) used?
restaurant_id will refer to an id (the column value). restaurant will refer to a Restaurant instance that will be lazy loaded from the db on access (or eager loaded if you set up the right stuff earlier). If you set backref on the relationship you can also access a list of MenuItem objects from a Restaurant.