We Keep Coding

Self referencing many-to-many relationship with extra column in association object

I am new in Sqlalchemy and trying to achieve the following goal with relationship():
There is an User table which stores user data.
Every user is able to invite other user with an invite_code.
Every user keeps a list of invitation, every invitation includes the invite_code and the invitee User
I think the relationship between User and Invitation is one-to-many. Since Invitation contains User, then I think it is probably better to use self-referential relationship to represent the inviter-to-invitaions(invitees) relationship and use an association object to store the invite_code.
I checked the sqlalchemy documentation and the question, tried to implement the classed like this:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, String
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Invitation(Base):
__tablename__ = 'invitation'
invite_code = Column(Integer)
inviter_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
invitee_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
invitee = relationship('User') #Need HELP here
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
inviters = relationship('User',
secondary='invitation',
primaryjoin=id==Invitation.invitee_id,
secondaryjoin=id==Invitation.inviter_id,
backref='invitees')
invitations = relationship('Invitation')# Need HELP here
def __repr__(self):
return f'User: {self.name}'
if __name__ == '__main__':
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(engine)
db = Session()
inviter1 = User(name='inviter1')
inviter2 = User(name='inviter2')
invitee1= User(name='invitee1')
invitee2 = User(name='invitee2')
inviter1.invitees = [invitee1, invitee2]
inviter2.invitees = [invitee1]
db.add(inviter1)
db.add(inviter2)
db.add(invitee1)
db.add(invitee2)
db.commit()
users = db.query(User).all()
for user in users:
print(user)
print(' Inviter: ', user.inviters)
print(' Invitee: ', user.invitees)
print()
If the lines with comment #Need HELP here are deleted, I can get the corresponding inviters and invitees, but cannot get the invite_code. If the #Need HELP here code are added, the error is:
Exception has occurred: AmbiguousForeignKeysError
Could not determine join condition between parent/child tables on relationship Invitation.invitee - there are multiple foreign key paths linking the tables. Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as containing a foreign key reference to the parent table.
Is there a way to add extra data column in association object like association object for many-to-many relationship for self referential table?
Sorry for the too much text, I didn't find any reference document on the web.

Finally, I figured it out with the help of foreign_keys:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, String
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
sent_invitations = relationship('Invitation', foreign_keys='Invitation.inviter_id', back_populates='inviter', cascade='all, delete')
received_invitations=relationship('Invitation', foreign_keys='Invitation.invitee_id', back_populates='invitee', cascade='all, delete')
def __repr__(self):
return f'User: {self.name}'
class Invitation(Base):
__tablename__ = 'invitation'
id = Column(Integer, primary_key=True)
invite_code = Column(Integer)
inviter_id = Column(Integer, ForeignKey('user.id'))
invitee_id = Column(Integer, ForeignKey('user.id'))
inviter=relationship('User', foreign_keys=[inviter_id], back_populates='sent_invitations')
invitee=relationship('User', foreign_keys=[invitee_id], back_populates='received_invitations')
def __repr__(self):
return f'Invitation: {self.inviter} invited {self.invitee} with {self.invite_code}'
if __name__ == '__main__':
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(engine)
db = Session()
inviter1 = User(name='inviter1')
inviter2 = User(name='inviter2')
invitee1= User(name='invitee1')
invitee2 = User(name='invitee2')
invitation1 = Invitation(invite_code=50, inviter=inviter1, invitee=invitee1)
invitation2 = Invitation(invite_code=20, inviter=inviter2, invitee=invitee2)
invitation3 = Invitation(invite_code=22, inviter=inviter1, invitee=inviter2)
invitation4 = Invitation(invite_code=44, inviter=invitee1, invitee=inviter2)
db.add(inviter1)
db.add(inviter2)
db.add(invitee1)
db.add(invitee2)
db.commit()
users = db.query(User).all()
for user in users:
print(user)
print(' sent_invitation: ', user.sent_invitations)
print(' received_invitation: ', user.received_invitations)
print()
invitations = db.query(Invitation).all()
for invitation in invitations:
print(invitation)
db.delete(inviter1)
db.delete(invitee2)
db.commit()

Hybrid attribute and relationship error: Neither 'InstrumentedAttribute' object nor 'Comparator' object associated with ... has an attribute

I've been following the Working with Relationships section of the Hybrid Attributes SQLAlchemy guide to develop the following:
from sqlalchemy import Column, Integer, select, create_engine, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.ext.hybrid import hybrid_property
from sqlalchemy.orm import sessionmaker
from sqlalchemy.orm import relationship
Base = declarative_base()
class Parent(Base):
__tablename__ = "parent"
id_ = Column(Integer, primary_key=True)
example_value = Column(Integer)
child = relationship("Child", back_populates="parent")
class Child(Base):
__tablename__ = "child"
id_ = Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey("parent.id_"))
parent = relationship("Parent", foreign_keys=[parent_id])
#hybrid_property
def example_value_check(self):
if self.parent.example_value == 0:
return "No"
elif self.parent.example_value > 0:
return "Yes"
engine = create_engine("sqlite:///:memory:")
Base.metadata.drop_all(engine)
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
session.add(Parent(id_=1, example_value=0))
session.add(Parent(id_=2, example_value=1))
session.add(Child(parent_id=1))
session.add(Child(parent_id=2))
session.commit()
example_value_checks = session.execute(select(Child.example_value_check)).scalars()
print(example_value_checks)
However, I'm getting the following error:
AttributeError: Neither 'InstrumentedAttribute' object nor 'Comparator' object associated with Child.parent has an attribute 'example_value'
From what I can see my case is slightly different because the hybrid_property is in my Child class which also has the foreign key. However, as I'd added the equivalent relationship to the Parent class with back_populates then I figured things would work - which they haven't :(
Where should I go from here?

Sqlalchemy self referencing many-to-many with association proxy and object mapped table

I'm trying to build a self referencing many-to-many model in Flask. I've already checked a lot of example on the internet, but in my case I would like to use an object mapped table for handling the relationship. Also because I'm using Flask with marshmallow I would like to include association proxies in my table to make the serialization of the model easier. Every example what I found is using backref but I wanted to make it with back_populates for the sake of readability. Currently I'm not sure if it's possible.
Please find below my minimal example to demonstrate the issue.
from sqlalchemy import Table, Column, Integer, String, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
engine = create_engine('sqlite:///:memory:', echo=True)
Session = sessionmaker(bind=engine)
session = Session()
Base = declarative_base()
class NodeRelation(Base):
__tablename__ = "node_relation"
parent_id = Column(Integer, ForeignKey('node.id'), primary_key=True)
parent = relationship('Node',
primaryjoin="NodeRelation.parent_id == node.c.id",
back_populates='parent_childs',
#foreign_keys=parent_id
)
child_id = Column(Integer, ForeignKey('node.id'), primary_key=True)
child = relationship('Node',
primaryjoin="NodeRelation.child_id == node.c.id",
back_populates='child_parents',
#foreign_keys=child_id
)
def __init__(self, parent=None, child=None, **kwargs):
super().__init__(**kwargs)
if child:
self.child = child
if parent:
self.parent = parent
def __repr__(self):
return "(parent_id: %s, child_id: %s)" % (self.parent_id, self.child_id)
class Node(Base):
__tablename__ = "node"
id = Column(Integer, primary_key=True, autoincrement=True)
title = Column(String())
parent_childs = relationship('NodeRelation',
primaryjoin="Node.id==node_relation.c.parent_id",
back_populates='parent',
cascade='all, delete',
#foreign_keys=NodeRelation.parent_id
)
parents = association_proxy('parent_childs', 'parent',
creator=lambda parent: NodeRelation(parent=parent))
child_parents = relationship('NodeRelation',
primaryjoin="Node.id==node_relation.c.child_id",
back_populates='child',
cascade='all, delete',
#foreign_keys=NodeRelation.child_id
)
childs = association_proxy('child_parents', 'child',
creator=lambda child: NodeRelation(child=child))
def __init__(self, title, **kwargs):
super().__init__(**kwargs)
self.title = title
def __repr__(self):
return "(id: %s, title: %s, childs: %s)" % (self.id, self.title, self.childs)
Base.metadata.create_all(engine)
n1 = Node("First")
n2 = Node("Second")
"""
# This is failing with: NOT NULL constraint failed: node_relation.parent_id
n1.childs.append(n2)
session.add(n1)
session.add(n2)
session.commit()
"""
# This one is working
c = NodeRelation(n1, n2)
session.add(n1)
session.add(n2)
session.add(c)
# Node 1 and Node 2 exists
q = session.query(NodeRelation).all()
print(q)
# This is failing with infinite recursion when childs property is displayed.
q2 = session.query(Node).all()
print(q2)
If I use the n1.childs.append() I've got a null contsraint error. If I directly construct the mapper object with n1 and n2 it's working fine, but as soon as I access the childs property I've got infinite recursion.
EDIT:
I've figured out that the custom creator lambda causing the appending error. So the updated code look like this:
from sqlalchemy import Table, Column, Integer, String, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
engine = create_engine('sqlite:///:memory:', echo=True)
Session = sessionmaker(bind=engine)
session = Session()
Base = declarative_base()
class NodeRelation(Base):
__tablename__ = "node_relation"
parent_id = Column(Integer, ForeignKey('node.id'), primary_key=True)
parent = relationship('Node',
primaryjoin="NodeRelation.parent_id == node.c.id",
back_populates='parent_childs',
#foreign_keys=parent_id
)
child_id = Column(Integer, ForeignKey('node.id'), primary_key=True)
child = relationship('Node',
primaryjoin="NodeRelation.child_id == node.c.id",
back_populates='child_parents',
#foreign_keys=child_id
)
def __init__(self, parent=None, child=None, **kwargs):
super().__init__(**kwargs)
if child:
self.child = child
if parent:
self.parent = parent
def __repr__(self):
return "(parent_id: %s, child_id: %s)" % (self.parent_id, self.child_id)
class Node(Base):
__tablename__ = "node"
id = Column(Integer, primary_key=True, autoincrement=True)
title = Column(String())
parent_childs = relationship('NodeRelation',
primaryjoin="Node.id==node_relation.c.parent_id",
back_populates='parent',
cascade='all, delete',
#foreign_keys=NodeRelation.parent_id
)
parents = association_proxy('parent_childs', 'parent')
child_parents = relationship('NodeRelation',
primaryjoin="Node.id==node_relation.c.child_id",
back_populates='child',
cascade='all, delete',
#foreign_keys=NodeRelation.child_id
)
childs = association_proxy('child_parents', 'child')
def __init__(self, title, **kwargs):
super().__init__(**kwargs)
self.title = title
def __repr__(self):
return "(id: %s, title: %s, childs: %s)" % (self.id, self.title, self.childs)
Base.metadata.create_all(engine)
n1 = Node("First")
n2 = Node("Second")
# This is failing with: NOT NULL constraint failed: node_relation.parent_id
n1.childs.append(n2)
session.add(n1)
session.add(n2)
session.commit()
# Node 1 and Node 2 exists
q = session.query(NodeRelation).all()
print(q)
# This is failing with infinite recursion when childs property is displayed.
q2 = session.query(Node).all()
print(q2)
So the only problem is that, when I try to access the childs property I've got a infinite recursion. I guess I messed up something with the relations.

Parent/child one-to-many distinguish Null vs. zero children

I have a parent child relationship as follows. I'd like to be able to distinguish between null children (e.g. information not known yet) vs. zero children. This is the approach I'm currently taking. It works, but seems a bit cumbersome. Is there a better way to go about this?
from sqlalchemy import Column, Integer, ForeignKey, create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
engine = create_engine('sqlite:///')
class Parent(Base):
__tablename__ = 'parent'
id = Column(Integer, primary_key=True)
children = relationship('Child', uselist=False)
class Child(Base):
__tablename__ = 'child'
id = Column(Integer, ForeignKey('parent.id'), primary_key=True)
child_items = relationship('ChildItem')
class ChildItem(Base):
__tablename__ = 'childitems'
id = Column(Integer, ForeignKey('child.id'), primary_key=True)
Base.metadata.create_all(engine)
p = Parent()
assert(p.children is None) # Would like to be able to do something like this.
c = Child()
c.child_items.append(ChildItem())
p.children = c
assert(p.children is not None)

SQLAlchemy many-to-many without foreign key

Could some one help me figure out how should i write primaryjoin/secondaryjoin
on secondary table that lacking one ForeignKey definition. I can't modify database
itself since it's used by different application.
from sqlalchemy import schema, types, func, orm
from sqlalchemy.dialects.postgresql import ARRAY
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class A(Base):
__tablename__ = 'atab'
id = schema.Column(types.SmallInteger, primary_key=True)
class B(Base):
__tablename__ = 'btab'
id = schema.Column(types.SmallInteger, primary_key=True)
a = orm.relationship(
'A', secondary='abtab', backref=orm.backref('b')
)
class AB(Base):
__tablename__ = 'abtab'
id = schema.Column(types.SmallInteger, primary_key=True)
a_id = schema.Column(types.SmallInteger, schema.ForeignKey('atab.id'))
b_id = schema.Column(types.SmallInteger)
I've tried specifing foreign on join condition:
a = orm.relationship(
'A', secondary='abtab', backref=orm.backref('b'),
primaryjoin=(id==orm.foreign(AB.b_id))
)
But received following error:
ArgumentError: Could not locate any simple equality expressions involving locally mapped foreign key columns for primary join condition '"atab".id = "abtab"."a_id"' on relationship Category.projects. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or are annotated in the join condition with the foreign() annotation. To allow comparison operators other than '==', the relationship can be marked as viewonly=True.

You can add foreign_keys to your relationship configuration. They mention this in a mailing list post:
from sqlalchemy import create_engine
from sqlalchemy import Column, Integer, String
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
logon = Column(String(10), primary_key=True)
group_id = Column(Integer)
class Group(Base):
__tablename__ = 'groups'
group_id = Column(Integer, primary_key=True)
users = relationship('User', backref='group',
primaryjoin='User.group_id==Group.group_id',
foreign_keys='User.group_id')
engine = create_engine('sqlite:///:memory:', echo=True)
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
u1 = User(logon='foo')
u2 = User(logon='bar')
g = Group()
g.users = [u1, u2]
session.add(g)
session.commit()
g = session.query(Group).first()
print([user.logon for user in g.users])
output:
['foo', 'bar']