I have a simple data model of customer and addresses with SQL Alchemy annotations to make the objects persistable in a database. Unfortunately, when I try to create a customer object with c = Customer() I receive an error:
sqlalchemy.exc.AmbiguousForeignKeysError: Could not determine join condition between parent/child
tables on relationship Customer.addresses - there are multiple foreign key paths linking the tables.
Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as
containing a foreign key reference to the parent table.
This is pretty clear -- I need to further annotate the line:
addresses = relationship("Address", back_populates="customer")
with something disambiguate the foreign key relationships. However, I can't understand what I need to (or could) specify in this case. Can anyone point me in the right direction?
Update: Looking further, it seems to me that SQLAlchemy is attempting to infer the direction of the addresses relationship and is unable to do so because there are PK/FK relationships in each direction between these classes. This cannot be resolved by adding foreign_keys= on the addresses relationship because the foreign key for this relationship is in the other table.
I can get this to work by removing the addresses relation entirely from Customer and instead doing customer = relationship("Customer", backref="addresses", foreign_keys=[customer_id]) in the Address class. I don't really like this solution, however, as I want to "express" addresses in the Customer class rather than as the side-effect of creating an otherwise unwanted customer relationship in the Address class.
So, still looking for a way of modifying the addresses relationship to make it work.
Here is my entire model:
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Customer(Base):
__tablename__ = "customer"
id = Column("customer_id", Integer, primary_key=True)
name = Column("name", String, nullable=False)
bill_address_id = Column("bill_id", Integer, ForeignKey("addresses.address_id"))
ship_address_id = Column("ship_id", Integer, ForeignKey("addresses.address_id"))
bill_address = relationship("Address", foreign_keys=[bill_address_id])
ship_address = relationship("Address", foreign_keys=[ship_address_id])
addresses = relationship("Address", back_populates="customer")
class Address(Base):
__tablename__ = "addresses"
id = Column("address_id", Integer, primary_key=True)
address = Column(String, nullable=False)
Related
I am trying to set up many-to-many relationship in SQLAlchemy but I am getting the error:
from shopapp import db
db.create_all()
sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'shoppinglists_products.shoppinglist_id_v2' could not find table 'shoppinglist' with which to generate a foreign key to target column 'id'
My code:
from sqlalchemy import ForeignKey
from shopapp import db
shoppinglists_products = db.Table("shoppinglists_products",
db.Column("shoppinglist_id", db.Integer, ForeignKey("shoppinglist.id")),
db.Column("product_id", db.Integer, ForeignKey("product.id")))
class ShoppingList(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(20), unique=True, nullable=False)
products = db.relationship('Product', back_populates="shoppinglists", secondary="shoppinglists_products")
class Product(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(20), unique=True, nullable=False)
Where is the problem?
It seems like Flask-SQLAlchemy has problem finding the table for foreign key reference. Based on your code, here are the two ways you can fix this:
1) Fix shoppinglists_products table:
Flask-SQLAlchemy often converts the CamelCased model names into a syntax similar to this: camel_cased. In your case, ShoppingList will be referred to as shopping_list. Therefore, changing the ForeignKey("shoppinglist.id") to ForeignKey("shopping_list.id") will do the trick.
shoppinglists_products = db.Table("shoppinglists_products",
db.Column("shoppinglist_id", db.Integer, ForeignKey("shopping_list.id")), # <-- fixed
2) Change the model names:
If you'd like, you could go ahead and change the model name from ShoppingList to Shopping and later refer to this as shopping. This would prevent any confusion from rendering further. Usually, developers don't quite often go for a class name which is combined of two words, especially for the ORM cases. This is because various frameworks has different ways of interpreting the class names to create tables.
Expanding on #P0intMaN's answer - explicitly providing the SQL Alchemy table name with __tablename__ = "ShoppingList" (for example) lets you use your preferred case style and prevents SQLAlchemy from 'helping' you by changing the name of something kind of important without telling you.
class ShoppingList(db.Model):
__tablename__ = "ShoppingList"
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(20), unique=True, nullable=False)
products = db.relationship('Product', back_populates="shoppinglists", secondary="shoppinglists_products")
In many/most Flask tutorials and books, simplistic table names (e.g. posts, comments, users) are used, which elide this issue. Thus a trap awaits for those of us who insist on meaningful CamelCased class names. This is mentioned somewhat casually in the documentation here: https://flask-sqlalchemy.palletsprojects.com/en/2.x/models/
Some parts that are required in SQLAlchemy are optional in
Flask-SQLAlchemy. For instance the table name is automatically set for
you unless overridden. It’s derived from the class name converted to
lowercase and with “CamelCase” converted to “camel_case”. To override
the table name, set the tablename class attribute.
I am using python 2.7 with SQLAlchemy, and trying to model a friendship relation with a many to many relationship.
I need the table to be completely symmetric; if A is B's friend, then it must be the other way around also.
I've tried to model the relationship with secondary friendship table, and connect it to the model using primary- and secondaryjoin, but I'm starting to get the feeling that I'm going in the wrong direction.
I found this post where someone tried to model the same thing using a one to many relationship, but this does not work for me, because my friendship relationship is not a one to many.
I have managed to achieve a working model using many to many table, if I am kipping a "duplicate": when I want to add B as A's friend, I add A as B's friend too. But I feel that the proposed solution should be more neat.
The end game here is similar to Facebook's friendship modeling. A can only be B's friend if B is A's friend.
The first attempt of using custom primary- and secondaryjoin conditions could be augmented with a composite "secondary", which in this case would be a union of both possible ways of selecting from the association table. Given a toy user model such as
class User(Base):
__tablename__ = "user"
id = Column(Integer, primary_key=True)
email = Column(Unicode(255), unique=True)
the association table could look like
friendship = Table(
"friendship", Base.metadata,
Column("left_id", ForeignKey("user.id"), primary_key=True),
Column("right_id", ForeignKey("user.id"), primary_key=True))
and the composite "secondary"
friends = select([friendship.c.left_id.label("left_id"),
friendship.c.right_id.label("right_id")]).\
union_all(select([friendship.c.right_id,
friendship.c.left_id])).\
alias("friends")
Using the above, User model would define the relationship as
User.friends = relationship(
"User", secondary=friends,
primaryjoin=User.id == friends.c.left_id,
secondaryjoin=User.id == friends.c.right_id,
viewonly=True)
The unfortunate side effect is that the relationship is readonly and you'd have to manually insert rows to friendship to make users friends. Also there's the issue of duplicates, because friendship could still contain both (1, 2) and (2, 1), for example. Adding a check constraint that enforces an ordering on left and right id tackles the duplicate issue:
# NOTE: This has to be done *before* creating your tables. You could also
# pass the CheckConstraint as an argument to Table directly.
chk = CheckConstraint(friendship.c.left_id < friendship.c.right_id)
friendship.append_constraint(chk)
The application would have to order the ids upon insertion, though. To remedy this the union used as "secondary" could be hidden in a writable view. SQLAlchemy does not have a construct for handling views out of the box, but there's a usage recipe for just that. Using the recipe friends becomes:
friends = view(
"friends",
Base.metadata,
select([friendship.c.left_id.label("left_id"),
friendship.c.right_id.label("right_id")]).\
union_all(select([friendship.c.right_id,
friendship.c.left_id])))
and to make the view writable some triggers are required:
# For SQLite only. Other databases have their own syntax for triggers.
DDL("""
CREATE TRIGGER friends_insert_trg1 INSTEAD OF INSERT ON friends
WHEN new.left_id < new.right_id
BEGIN
INSERT INTO friendship (left_id, right_id)
VALUES (new.left_id, new.right_id);
END;
""").execute_at("after-create", Base.metadata)
DDL("""
CREATE TRIGGER friends_insert_trg2 INSTEAD OF INSERT ON friends
WHEN new.left_id > new.right_id
BEGIN
INSERT INTO friendship (left_id, right_id)
VALUES (new.right_id, new.left_id);
END;
""").execute_at("after-create", Base.metadata)
It'd be nice to bind these to the creation of the view more closely, but this will do as well, as long as you register them after defining the view. With the triggers in place you can remove the viewonly=True argument from the User.friends relationship.
Putting it all together:
from sqlalchemy import \
Table, Column, Integer, Unicode, ForeignKey, CheckConstraint, DDL, \
select
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
from view import view
Base = declarative_base()
class User(Base):
__tablename__ = "user"
id = Column(Integer, primary_key=True)
email = Column(Unicode(255), unique=True)
friendship = Table(
"friendship",
Base.metadata,
Column("left_id", ForeignKey("user.id"), primary_key=True),
Column("right_id", ForeignKey("user.id"), primary_key=True),
CheckConstraint("left_id < right_id"))
friends = view(
"friends",
Base.metadata,
select([friendship.c.left_id.label("left_id"),
friendship.c.right_id.label("right_id")]).\
union_all(select([friendship.c.right_id,
friendship.c.left_id])))
User.friends = relationship(
"User", secondary=friends,
primaryjoin=User.id == friends.c.left_id,
secondaryjoin=User.id == friends.c.right_id)
DDL("""
CREATE TRIGGER friends_insert_trg1 INSTEAD OF INSERT ON friends
WHEN new.left_id < new.right_id
BEGIN
INSERT INTO friendship (left_id, right_id)
VALUES (new.left_id, new.right_id);
END;
""").execute_at("after-create", Base.metadata)
DDL("""
CREATE TRIGGER friends_insert_trg2 INSTEAD OF INSERT ON friends
WHEN new.left_id > new.right_id
BEGIN
INSERT INTO friendship (left_id, right_id)
VALUES (new.right_id, new.left_id);
END;
""").execute_at("after-create", Base.metadata)
I was reading through the SQLAlchemy documentation on basic relationships and I feel like I'm missing some basic understandings as to how to create the relationship declarations. When I run my code, I'm running into errors such as:
sqlalchemy.exc.NoForeignKeysError: Can't find any foreign key relationships between 'entity' and 'category'.
sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Entity.categories - there are no foreign keys linking these tables. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression.
I thought that the purpose of the relationship() directive was to minimize the creation of manual keys and ids.
I'm also a little confused on the syntax with regards to one-to-many and many-to-many, and many-to-one and how the syntax would differentiate between the different types of relationships.
Heres my example where I create an Entity and various classes around it to try out the various relationships:
class Entity(Base):
__tablename__ = 'entity'
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False )
# many-to-one - many entities will belong to one manufacturer
# do i need to define the mfg_id manually?
manufacturer_id = Column(Integer, ForeignKey('manufacturer.id'))
manufacturer = relationship("Manufacturer")
# one-to-many relationship where an entity will have lots of
# properties that belong to it. Each property will only belong to one entity
properties = relationship("EntityProperty", backref="entity")
# this is a many-to-many relationship mapping where entity can belong
# to multiple categories and you can look up entities by category
categories = relationship("Category", backref="entities")
class EntityProperty(Base):
__tablename__ = 'entity_property'
id = Column(Integer, primary_key=True)
key = Column(String(250), nullable=False )
value = Column(String(250), nullable=False )
# do we need to define this? or can this be implied by relationship?
entity_id = Column(Integer, ForeignKey('entity.id'))
class Category(Base):
__tablename__ = 'category'
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False )
# Does this need to know anything about entities? Many entities
# can belong to a category and entities can also belong to multiple
# categories. Usage is to look up entities that belong to a category.
class Manufacturer(Base):
__tablename__ = 'manufacturer'
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False )
# Similar to category except not all manufactures would have entities.
# How to decouple code at this level from entity
Can someone point me in the right direction to learn more about the proper usage of relationship()? Thank you
First, there has to be a foreign key in the entity table referencing a column in the category table (or vice versa) to establish a relationship. You currently have none.
However, if you intend to have a many-to-many relationship between entity and category then kindly see
SQLAlchemy many-to-many Relationships
Sometimes you end up solving your own questions as soon as they are posed. So heres what I learned.
I was getting a little confused with the back_populdate vs backref. I was thinking that if I added a backref I wouldn't need to add the foreignkey in the opposite class, but this was incorrect.
So for the one to many:
This is declared in the Entity
properties = relationship("EntityProperty", backref="entity")
And this is declared in the EntityProperty to facilitate the necessary back-linking and is required:
entity_id = Column(Integer, ForeignKey("entity.id"))
In the many-to-many case, I was missing an association table:
cat_entity_association_table = Table("cat_entity_assocaition", Base.metadata,
Column("category_id", Integer, ForeignKey("category.id")),
Column("entity_id", Integer, ForeignKey("entity.id")),
)
This association is used to construct the bi-directional linking between entities:
categories = relationship("Category", secondary=cat_entity_association_table, back_populates="entities")
and categories:
entities = relationship("Entity", secondary=cat_entity_association_table, back_populates="categories")
There was some ambiguity on when to use the external table, but hopefully this will help someone else as well.
I'm trying to create a table of profiles where profiles, tests, and folders can be children of profiles, but I'm getting an AmbiguousForeignKeysError when I call create_all(db). My idea for a single table, which then references itself via a mapper or association table seems like it should work, but I haven't been able to get sqlalchemy to cooperate. I used examples for a many-to-many relationship on the sqlalchemy website to create my code. Here's my snippet:
from sqlalchemy import Column, Integer, String, Enum, Table, ForeignKey
from sqlalchemy.orm import relationship,validates
from core.coyote.models import Base #declarative base
association_table = Table('association', Base.metadata,
Column('left_id', Integer, ForeignKey('profile_member.id')),
Column('right_id', Integer, ForeignKey('profile_member.id'))
)
class ProfileMember(Base):
__tablename__ = "profile_member"
id = Column(Integer, primary_key=True)
name = Column(String(250),nullable=True)
type = Column(Enum("Test","Profile","Folder"))
children = relationship("ProfileMember",secondary=association_table)
I also plan to put in some validation code so that I can enforce some parentage rules. Here is the complete error I'm getting:
sqlalchemy.exc.AmbiguousForeignKeysError: Could not determine join condition between parent/child tables on relationship ProfileMember.children - there are multiple foreign key paths linking the tables via secondary table 'association'. Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as containing a foreign key reference from the secondary table to each of the parent and child tables
Any clues as to things I can try would be greatly appreciated.
You have to specify how to join to the secondary table:
children = relationship("ProfileMember", secondary=association_table,
primaryjoin=id == association_table.c.left_id,
secondaryjoin=association_table.c.right_id == id)
I'm working in Flask with Flask-SQLAlchemy, and trying to setup a many-to-many-to-many relationship: clients can have multiple orders and orders can also have multiple clients (many to many); each order in turn contains a list of unique items (one to many).
I followed the SQLAlchemy documents to setup an association table for the many-to-many relationship, and used the normal relationship/foreign key for the one-to-many relationship; all references are set to lazy='dynamic'.
association_table = Table('association', Base.metadata,
Column('left_id', Integer, ForeignKey('left.id')),
Column('right_id', Integer, ForeignKey('right.id'))
)
what is an efficient way to retrieve all items associated with a client? I'm assuming [item for item in order.items.all() for order in client.orders] will work (less the problem above), but is there more efficient way to do it? What if the results needs to be ordered?
Update
I now know two ways to retrieve the items for the orders of a client by the following two methods (the first one from Audrius' answer):
db.session.query(Item).\
join(Order).join(Order.clients).\
filter(Client.id == 42).\
order_by(Item.timestamp)
and
Item.query.filter(Item.order_id._in(client.order_ids)).order_by(Item.timestamp)
I believe they both provide the same result, but which one should be preferred for efficiency?
When writing query directly in SQL, you would be using joins to retrieve the data you want efficiently (as Rachcha demonstrated in his answer). The same applies to SQLAlchemy. Refer to the SA docs on join() for more examples.
If your model is defined like the following (using Flask-SQLAlchemy, since you tagged your question with its tag):
clients_orders = db.Table('clients_orders',
db.Column('client_id', db.Integer, db.ForeignKey('client.id'),
primary_key=True),
db.Column('order_id', db.Integer, db.ForeignKey('order.id'),
primary_key=True)
)
class Client(db.Model):
id = db.Column(db.Integer, primary_key=True)
orders = db.relationship('Order', secondary=clients_orders,
backref='clients')
# Define other columns...
class Order(db.Model):
id = db.Column(db.Integer, primary_key=True)
# Define other columns...
class Item(db.Model):
id = db.Column(db.Integer, primary_key=True)
order_id = db.Column(db.Integer, db.ForeignKey('order.id'), nullable=False)
order = db.relationship('Order', backref='items')
timestamp = db.Column(db.DateTime)
# Define other columns...
Then your query could look like this:
db.session.query(Item).\
join(Order).join(Order.clients).\
filter(Client.id == 42).\
order_by(Item.timestamp)