We Keep Coding

SQLAlchemy: How to disambiguate Foreign Key relationships?

I have a simple data model of customer and addresses with SQL Alchemy annotations to make the objects persistable in a database. Unfortunately, when I try to create a customer object with c = Customer() I receive an error:
sqlalchemy.exc.AmbiguousForeignKeysError: Could not determine join condition between parent/child
tables on relationship Customer.addresses - there are multiple foreign key paths linking the tables.
Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as
containing a foreign key reference to the parent table.
This is pretty clear -- I need to further annotate the line:
addresses = relationship("Address", back_populates="customer")
with something disambiguate the foreign key relationships. However, I can't understand what I need to (or could) specify in this case. Can anyone point me in the right direction?
Update: Looking further, it seems to me that SQLAlchemy is attempting to infer the direction of the addresses relationship and is unable to do so because there are PK/FK relationships in each direction between these classes. This cannot be resolved by adding foreign_keys= on the addresses relationship because the foreign key for this relationship is in the other table.
I can get this to work by removing the addresses relation entirely from Customer and instead doing customer = relationship("Customer", backref="addresses", foreign_keys=[customer_id]) in the Address class. I don't really like this solution, however, as I want to "express" addresses in the Customer class rather than as the side-effect of creating an otherwise unwanted customer relationship in the Address class.
So, still looking for a way of modifying the addresses relationship to make it work.
Here is my entire model:
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Customer(Base):
__tablename__ = "customer"
id = Column("customer_id", Integer, primary_key=True)
name = Column("name", String, nullable=False)
bill_address_id = Column("bill_id", Integer, ForeignKey("addresses.address_id"))
ship_address_id = Column("ship_id", Integer, ForeignKey("addresses.address_id"))
bill_address = relationship("Address", foreign_keys=[bill_address_id])
ship_address = relationship("Address", foreign_keys=[ship_address_id])
addresses = relationship("Address", back_populates="customer")
class Address(Base):
__tablename__ = "addresses"
id = Column("address_id", Integer, primary_key=True)
address = Column(String, nullable=False)

SQLAlchemy Self-Referential Many to Many Symmetric Relationship

I am using python 2.7 with SQLAlchemy, and trying to model a friendship relation with a many to many relationship.
I need the table to be completely symmetric; if A is B's friend, then it must be the other way around also.
I've tried to model the relationship with secondary friendship table, and connect it to the model using primary- and secondaryjoin, but I'm starting to get the feeling that I'm going in the wrong direction.
I found this post where someone tried to model the same thing using a one to many relationship, but this does not work for me, because my friendship relationship is not a one to many.
I have managed to achieve a working model using many to many table, if I am kipping a "duplicate": when I want to add B as A's friend, I add A as B's friend too. But I feel that the proposed solution should be more neat.
The end game here is similar to Facebook's friendship modeling. A can only be B's friend if B is A's friend.

The first attempt of using custom primary- and secondaryjoin conditions could be augmented with a composite "secondary", which in this case would be a union of both possible ways of selecting from the association table. Given a toy user model such as
class User(Base):
__tablename__ = "user"
id = Column(Integer, primary_key=True)
email = Column(Unicode(255), unique=True)
the association table could look like
friendship = Table(
"friendship", Base.metadata,
Column("left_id", ForeignKey("user.id"), primary_key=True),
Column("right_id", ForeignKey("user.id"), primary_key=True))
and the composite "secondary"
friends = select([friendship.c.left_id.label("left_id"),
friendship.c.right_id.label("right_id")]).\
union_all(select([friendship.c.right_id,
friendship.c.left_id])).\
alias("friends")
Using the above, User model would define the relationship as
User.friends = relationship(
"User", secondary=friends,
primaryjoin=User.id == friends.c.left_id,
secondaryjoin=User.id == friends.c.right_id,
viewonly=True)
The unfortunate side effect is that the relationship is readonly and you'd have to manually insert rows to friendship to make users friends. Also there's the issue of duplicates, because friendship could still contain both (1, 2) and (2, 1), for example. Adding a check constraint that enforces an ordering on left and right id tackles the duplicate issue:
# NOTE: This has to be done *before* creating your tables. You could also
# pass the CheckConstraint as an argument to Table directly.
chk = CheckConstraint(friendship.c.left_id < friendship.c.right_id)
friendship.append_constraint(chk)
The application would have to order the ids upon insertion, though. To remedy this the union used as "secondary" could be hidden in a writable view. SQLAlchemy does not have a construct for handling views out of the box, but there's a usage recipe for just that. Using the recipe friends becomes:
friends = view(
"friends",
Base.metadata,
select([friendship.c.left_id.label("left_id"),
friendship.c.right_id.label("right_id")]).\
union_all(select([friendship.c.right_id,
friendship.c.left_id])))
and to make the view writable some triggers are required:
# For SQLite only. Other databases have their own syntax for triggers.
DDL("""
CREATE TRIGGER friends_insert_trg1 INSTEAD OF INSERT ON friends
WHEN new.left_id < new.right_id
BEGIN
INSERT INTO friendship (left_id, right_id)
VALUES (new.left_id, new.right_id);
END;
""").execute_at("after-create", Base.metadata)
DDL("""
CREATE TRIGGER friends_insert_trg2 INSTEAD OF INSERT ON friends
WHEN new.left_id > new.right_id
BEGIN
INSERT INTO friendship (left_id, right_id)
VALUES (new.right_id, new.left_id);
END;
""").execute_at("after-create", Base.metadata)
It'd be nice to bind these to the creation of the view more closely, but this will do as well, as long as you register them after defining the view. With the triggers in place you can remove the viewonly=True argument from the User.friends relationship.
Putting it all together:
from sqlalchemy import \
Table, Column, Integer, Unicode, ForeignKey, CheckConstraint, DDL, \
select
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
from view import view
Base = declarative_base()
class User(Base):
__tablename__ = "user"
id = Column(Integer, primary_key=True)
email = Column(Unicode(255), unique=True)
friendship = Table(
"friendship",
Base.metadata,
Column("left_id", ForeignKey("user.id"), primary_key=True),
Column("right_id", ForeignKey("user.id"), primary_key=True),
CheckConstraint("left_id < right_id"))
friends = view(
"friends",
Base.metadata,
select([friendship.c.left_id.label("left_id"),
friendship.c.right_id.label("right_id")]).\
union_all(select([friendship.c.right_id,
friendship.c.left_id])))
User.friends = relationship(
"User", secondary=friends,
primaryjoin=User.id == friends.c.left_id,
secondaryjoin=User.id == friends.c.right_id)
DDL("""
CREATE TRIGGER friends_insert_trg1 INSTEAD OF INSERT ON friends
WHEN new.left_id < new.right_id
BEGIN
INSERT INTO friendship (left_id, right_id)
VALUES (new.left_id, new.right_id);
END;
""").execute_at("after-create", Base.metadata)
DDL("""
CREATE TRIGGER friends_insert_trg2 INSTEAD OF INSERT ON friends
WHEN new.left_id > new.right_id
BEGIN
INSERT INTO friendship (left_id, right_id)
VALUES (new.right_id, new.left_id);
END;
""").execute_at("after-create", Base.metadata)

SQLAlchemy how to create various relationship with backrefs?

I was reading through the SQLAlchemy documentation on basic relationships and I feel like I'm missing some basic understandings as to how to create the relationship declarations. When I run my code, I'm running into errors such as:
sqlalchemy.exc.NoForeignKeysError: Can't find any foreign key relationships between 'entity' and 'category'.
sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Entity.categories - there are no foreign keys linking these tables. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression.
I thought that the purpose of the relationship() directive was to minimize the creation of manual keys and ids.
I'm also a little confused on the syntax with regards to one-to-many and many-to-many, and many-to-one and how the syntax would differentiate between the different types of relationships.
Heres my example where I create an Entity and various classes around it to try out the various relationships:
class Entity(Base):
__tablename__ = 'entity'
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False )
# many-to-one - many entities will belong to one manufacturer
# do i need to define the mfg_id manually?
manufacturer_id = Column(Integer, ForeignKey('manufacturer.id'))
manufacturer = relationship("Manufacturer")
# one-to-many relationship where an entity will have lots of
# properties that belong to it. Each property will only belong to one entity
properties = relationship("EntityProperty", backref="entity")
# this is a many-to-many relationship mapping where entity can belong
# to multiple categories and you can look up entities by category
categories = relationship("Category", backref="entities")
class EntityProperty(Base):
__tablename__ = 'entity_property'
id = Column(Integer, primary_key=True)
key = Column(String(250), nullable=False )
value = Column(String(250), nullable=False )
# do we need to define this? or can this be implied by relationship?
entity_id = Column(Integer, ForeignKey('entity.id'))
class Category(Base):
__tablename__ = 'category'
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False )
# Does this need to know anything about entities? Many entities
# can belong to a category and entities can also belong to multiple
# categories. Usage is to look up entities that belong to a category.
class Manufacturer(Base):
__tablename__ = 'manufacturer'
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False )
# Similar to category except not all manufactures would have entities.
# How to decouple code at this level from entity
Can someone point me in the right direction to learn more about the proper usage of relationship()? Thank you

First, there has to be a foreign key in the entity table referencing a column in the category table (or vice versa) to establish a relationship. You currently have none.
However, if you intend to have a many-to-many relationship between entity and category then kindly see
SQLAlchemy many-to-many Relationships

Sometimes you end up solving your own questions as soon as they are posed. So heres what I learned.
I was getting a little confused with the back_populdate vs backref. I was thinking that if I added a backref I wouldn't need to add the foreignkey in the opposite class, but this was incorrect.
So for the one to many:
This is declared in the Entity
properties = relationship("EntityProperty", backref="entity")
And this is declared in the EntityProperty to facilitate the necessary back-linking and is required:
entity_id = Column(Integer, ForeignKey("entity.id"))
In the many-to-many case, I was missing an association table:
cat_entity_association_table = Table("cat_entity_assocaition", Base.metadata,
Column("category_id", Integer, ForeignKey("category.id")),
Column("entity_id", Integer, ForeignKey("entity.id")),
)
This association is used to construct the bi-directional linking between entities:
categories = relationship("Category", secondary=cat_entity_association_table, back_populates="entities")
and categories:
entities = relationship("Entity", secondary=cat_entity_association_table, back_populates="categories")
There was some ambiguity on when to use the external table, but hopefully this will help someone else as well.

Many-to-many recursive table definition in SQLAlchemy results in AmbiguousForeignKeysError

I'm trying to create a table of profiles where profiles, tests, and folders can be children of profiles, but I'm getting an AmbiguousForeignKeysError when I call create_all(db). My idea for a single table, which then references itself via a mapper or association table seems like it should work, but I haven't been able to get sqlalchemy to cooperate. I used examples for a many-to-many relationship on the sqlalchemy website to create my code. Here's my snippet:
from sqlalchemy import Column, Integer, String, Enum, Table, ForeignKey
from sqlalchemy.orm import relationship,validates
from core.coyote.models import Base #declarative base
association_table = Table('association', Base.metadata,
Column('left_id', Integer, ForeignKey('profile_member.id')),
Column('right_id', Integer, ForeignKey('profile_member.id'))
)
class ProfileMember(Base):
__tablename__ = "profile_member"
id = Column(Integer, primary_key=True)
name = Column(String(250),nullable=True)
type = Column(Enum("Test","Profile","Folder"))
children = relationship("ProfileMember",secondary=association_table)
I also plan to put in some validation code so that I can enforce some parentage rules. Here is the complete error I'm getting:
sqlalchemy.exc.AmbiguousForeignKeysError: Could not determine join condition between parent/child tables on relationship ProfileMember.children - there are multiple foreign key paths linking the tables via secondary table 'association'. Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as containing a foreign key reference from the secondary table to each of the parent and child tables
Any clues as to things I can try would be greatly appreciated.

You have to specify how to join to the secondary table:
children = relationship("ProfileMember", secondary=association_table,
primaryjoin=id == association_table.c.left_id,
secondaryjoin=association_table.c.right_id == id)

How to create relationship many to many in SQLAlchemy (python, flask) for model User to itself

I need to create a table called friends, it should looks like:
friends:
user_id
friend_id
I was trying to do this with tutorials from SQLALchemy, but I have not found how to make relation many-to-many for same table.
Here's what I have tried:
# friends table
# many to many - user - user
_friends = db.Table('friends',
db.Column('user_id', db.Integer, db.ForeignKey('users.id')),
db.Column('friend_id', db.Integer, db.ForeignKey('users.id'))
)
class User(db.Model, UserMixin):
# table name in database
__tablename__ = 'users'
# primary key for table in db
id = db.Column(db.Integer, primary_key=True)
# email is unique!
email = db.Column(db.String(255), unique=True)
# password, max = 255
password = db.Column(db.String(255))
# category relation
categories = relationship("Category")
# cards relation
cards = relationship("BusinessCard")
# friends
friends = db.relationship(
'User',
backref="users",
secondary=_friends
)
it says:
AmbiguousForeignKeysError: Could not determine join condition between
parent/child tables on relationship User.friends - there are multiple
foreign key paths linking the tables via secondary table 'friends'.
Specify the 'foreign_keys' argument, providing a list of those columns
which should be counted as containing a foreign key reference from the
secondary table to each of the parent and child tables.
does anyone know how to do that properly?

The pattern you're trying to implement is a special case of many-to-many relationship. SQLAlchemy calls this an Adjacency List Relationship, and I recommend trying to follow through the code there:
http://docs.sqlalchemy.org/en/rel_0_9/orm/relationships.html#adjacency-list-relationships
The key is the 'remote_side' kwarg there.
Here's why: the error that you're getting is because you association table ('friends') has two foreign keys pointing to table 'users': one on column 'user_id', and one on column 'friend_id'. SQLAlchemy tries to auto-detect relationships based on foreign keys, but it fails because it can't tell which direction the relationship goes. So if you have an entry in table 'friends' like so
user_id : 1
friend_id : 2
SQLAlchemy can't tell whether user_1 has user_2 as a friend, or vice-versa.
If that seems confusing, it is. Friendship in the sense of social networks can be unijective, in which case user_1 having friend user_2 does not mean that user_2 has user_1 as a friend; or it can be bijective, in which case the two are equivalent. I'm showing my age here, but the former is represented by Livejournal, whereas the latter is represented by Facebook.
I don't know off the top of my head how to implement a unijective relationship in SQLAlchemy. It's an ugly UNION ALL or something like that in MySQL.

Miguel Grinberg's tutorial explain this type of application.
http://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-viii-followers-contacts-and-friends

You can't directly make Many-To-Many relation between 2 tables, instead you have to use 3rd table.