We Keep Coding

Self referencing many-to-many relationship with extra column in association object

I am new in Sqlalchemy and trying to achieve the following goal with relationship():
There is an User table which stores user data.
Every user is able to invite other user with an invite_code.
Every user keeps a list of invitation, every invitation includes the invite_code and the invitee User
I think the relationship between User and Invitation is one-to-many. Since Invitation contains User, then I think it is probably better to use self-referential relationship to represent the inviter-to-invitaions(invitees) relationship and use an association object to store the invite_code.
I checked the sqlalchemy documentation and the question, tried to implement the classed like this:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, String
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Invitation(Base):
__tablename__ = 'invitation'
invite_code = Column(Integer)
inviter_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
invitee_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
invitee = relationship('User') #Need HELP here
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
inviters = relationship('User',
secondary='invitation',
primaryjoin=id==Invitation.invitee_id,
secondaryjoin=id==Invitation.inviter_id,
backref='invitees')
invitations = relationship('Invitation')# Need HELP here
def __repr__(self):
return f'User: {self.name}'
if __name__ == '__main__':
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(engine)
db = Session()
inviter1 = User(name='inviter1')
inviter2 = User(name='inviter2')
invitee1= User(name='invitee1')
invitee2 = User(name='invitee2')
inviter1.invitees = [invitee1, invitee2]
inviter2.invitees = [invitee1]
db.add(inviter1)
db.add(inviter2)
db.add(invitee1)
db.add(invitee2)
db.commit()
users = db.query(User).all()
for user in users:
print(user)
print(' Inviter: ', user.inviters)
print(' Invitee: ', user.invitees)
print()
If the lines with comment #Need HELP here are deleted, I can get the corresponding inviters and invitees, but cannot get the invite_code. If the #Need HELP here code are added, the error is:
Exception has occurred: AmbiguousForeignKeysError
Could not determine join condition between parent/child tables on relationship Invitation.invitee - there are multiple foreign key paths linking the tables. Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as containing a foreign key reference to the parent table.
Is there a way to add extra data column in association object like association object for many-to-many relationship for self referential table?
Sorry for the too much text, I didn't find any reference document on the web.

Finally, I figured it out with the help of foreign_keys:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, String
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
sent_invitations = relationship('Invitation', foreign_keys='Invitation.inviter_id', back_populates='inviter', cascade='all, delete')
received_invitations=relationship('Invitation', foreign_keys='Invitation.invitee_id', back_populates='invitee', cascade='all, delete')
def __repr__(self):
return f'User: {self.name}'
class Invitation(Base):
__tablename__ = 'invitation'
id = Column(Integer, primary_key=True)
invite_code = Column(Integer)
inviter_id = Column(Integer, ForeignKey('user.id'))
invitee_id = Column(Integer, ForeignKey('user.id'))
inviter=relationship('User', foreign_keys=[inviter_id], back_populates='sent_invitations')
invitee=relationship('User', foreign_keys=[invitee_id], back_populates='received_invitations')
def __repr__(self):
return f'Invitation: {self.inviter} invited {self.invitee} with {self.invite_code}'
if __name__ == '__main__':
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(engine)
db = Session()
inviter1 = User(name='inviter1')
inviter2 = User(name='inviter2')
invitee1= User(name='invitee1')
invitee2 = User(name='invitee2')
invitation1 = Invitation(invite_code=50, inviter=inviter1, invitee=invitee1)
invitation2 = Invitation(invite_code=20, inviter=inviter2, invitee=invitee2)
invitation3 = Invitation(invite_code=22, inviter=inviter1, invitee=inviter2)
invitation4 = Invitation(invite_code=44, inviter=invitee1, invitee=inviter2)
db.add(inviter1)
db.add(inviter2)
db.add(invitee1)
db.add(invitee2)
db.commit()
users = db.query(User).all()
for user in users:
print(user)
print(' sent_invitation: ', user.sent_invitations)
print(' received_invitation: ', user.received_invitations)
print()
invitations = db.query(Invitation).all()
for invitation in invitations:
print(invitation)
db.delete(inviter1)
db.delete(invitee2)
db.commit()

SQLAlchemy association proxy and lists

I'm trying to work out how to get an association proxy on a many to many relationship to return an empty list by default.
So based on the example at:
https://raw.githubusercontent.com/zzzeek/sqlalchemy/master/examples/generic_associations/discriminator_on_association.py
We have:
from sqlalchemy.ext.declarative import as_declarative, declared_attr
from sqlalchemy import create_engine, Integer, Column, \
String, ForeignKey
from sqlalchemy.orm import Session, relationship, backref
from sqlalchemy.ext.associationproxy import association_proxy
#as_declarative()
class Base(object):
"""Base class which provides automated table name
and surrogate primary key column.
"""
#declared_attr
def __tablename__(cls):
return cls.__name__.lower()
id = Column(Integer, primary_key=True)
class AddressAssociation(Base):
"""Associates a collection of Address objects
with a particular parent.
"""
__tablename__ = "address_association"
discriminator = Column(String)
"""Refers to the type of parent."""
__mapper_args__ = {"polymorphic_on": discriminator}
class Address(Base):
"""The Address class.
This represents all address records in a
single table.
"""
association_id = Column(Integer, ForeignKey("address_association.id"))
street = Column(String)
city = Column(String)
zip = Column(String)
association = relationship("AddressAssociation", backref="addresses")
parent = association_proxy("association", "parent")
def __repr__(self):
return "%s(street=%r, city=%r, zip=%r)" % \
(self.__class__.__name__, self.street,
self.city, self.zip)
class HasAddresses(object):
"""HasAddresses mixin, creates a relationship to
the address_association table for each parent.
"""
#declared_attr
def address_association_id(cls):
return Column(Integer, ForeignKey("address_association.id"))
#declared_attr
def address_association(cls):
name = cls.__name__
discriminator = name.lower()
assoc_cls = type(
"%sAddressAssociation" % name,
(AddressAssociation, ),
dict(
__tablename__=None,
__mapper_args__={
"polymorphic_identity": discriminator
}
)
)
cls.addresses = association_proxy(
"address_association", "addresses",
creator=lambda addresses: assoc_cls(addresses=addresses)
)
return relationship(assoc_cls,
backref=backref("parent", uselist=False))
class Customer(HasAddresses, Base):
name = Column(String)
class Supplier(HasAddresses, Base):
company_name = Column(String)
engine = create_engine('sqlite://', echo=True)
Base.metadata.create_all(engine)
session = Session(engine)
But if I try to look at the address of a Customer, I get None rather than [] that I'd expect:
(Pdb) foo = Customer()
(Pdb) foo.addresses
(Pdb)
(Pdb) type(foo.addresses)
<type 'NoneType'>
Which means I can't simply iterate over foo.addresses without first checking it is not None.
I tried setting uselist = True on the association proxy:
return relationship(assoc_cls,
uselist=True,
backref=backref("parent", uselist=False))
Which sort of works, but then the constructor fails:
(Pdb) foo.addresses
[]
(Pdb) foo.addresses.append(Address())
*** TypeError: Incompatible collection type: Address is not list-like
(Pdb)
I tried messing with the creator to make it construct a list, but then cause all manner of side effects, included nested lists:
cls.addresses = association_proxy(
"address_association", "addresses",
creator=lambda addresses: assoc_cls(addresses=[addresses,])
Any idea the best way to sort this out properly?
-Matt

Automating sqlalchemy declarative model creation

Let’s say we have several sqlalchemy models for catalogues:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer
from sqlalchemy.orm import relationship
Base = declarative_base()
class Plane(Base):
__tablename__ = 'Plane'
plane_id = Column(Integer, primary_key=True)
class Car(Base):
__tablename__ = 'Car'
car_id = Column(Integer, primary_key=True)
Now for import/export purposes we want to relate these to external ids. So for Plane we would write:
class PlaneID(Base):
issuer = Column(String(32), primary_key=True)
external_id = Column(String(16), primary_key=True)
plane_id = Column(Integer, ForeignKey(Plane.plane_id))
plane = relationship(Plane, backref='external_ids')
A CarID model would be defined in exactly the same way.
What are possibilities to automate this process?
Maybe we could use a mixin, factory, decorator or meta class. How would we generate the dynamically named Columns then? It would be good to be able to add more Columns to the generated models as needed. For example:
class CarID(ExternalID):
valid_from = Column(Date)

You can subclass DeclarativeMeta - the metaclass used in declarative_base function:
from sqlalchemy.ext.declarative import DeclarativeMeta, declarative_base
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.orm import relationship
class ExternalObject(DeclarativeMeta):
def __new__(mcs, name, bases, attributes):
if 'issuer' not in attributes:
attributes['issuer'] = Column(String(32), primary_key=True)
if 'external_id' not in attributes:
attributes['external_id'] = Column(String(16), primary_key=True)
if name[-2:] == 'ID':
ext_cls_name = name[:-2]
attr_rel = ext_cls_name.lower()
attr_id = '%s_id' % attr_rel
if attr_rel in attributes or attr_id in attributes:
# Some code here in case 'car' or 'car_id' attribute is defined in new class
pass
attributes[attr_id] = Column(Integer, ForeignKey('%s.%s' % (ext_cls_name, attr_id)))
attributes[attr_rel] = relationship(ext_cls_name, backref='external_ids')
new_cls = super().__new__(mcs, name, bases, attributes)
return new_cls
ExternalID = declarative_base(metaclass=ExternalObject)
After that you can create subclass from ExternalID and add another attributes like you did for CarID.

Creating multiple objects with foreign key

I need to create ten sample users (User) and each of them must have fifty documents (Doc). How to do this in tests.py using factoryboy?
#factories.py
from app_name.models import *
import factory
from datetime import datetime, timedelta, time
from django.contrib.auth.models import User
class UserFactory(factory.Factory):
FACTORY_FOR = User
username = factory.Sequence(lambda n: 'User ' + n)
email = 'demo#mail.com'
password = '1234567'
class DocFactory(factory.Factory):
FACTORY_FOR = Doc
user = factory.SubFactory(UserFactory)
kategories = '1'
doc_number = '12345678'
date_join = factory.Sequence(lambda n:(datetime.now() + timedelta(days=n)).date(), int)
in my tests.py:
from django.test import TestCase
from django_dynamic_fixture import G
from factories import *

users = UserFactory.create_batch(10)
for user in users:
doc = DocFactory.create(user=user)

You can use a post_generation decorator:
class UserFactory(factory.Factory):
...
#factory.post_generation
def create_docs(self, create, extracted, **kwargs):
if not create:
return
for i in range(50):
doc = DocFactory.create(user=self)

For those of you working with SQLAlchemy, this can be done with the following recipe (notice that I'm using the Person/Address models instead of the User/Docs model example above).
from sqlalchemy import create_engine, Integer, Text, ForeignKey, Column
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship, scoped_session, sessionmaker
import factory
from factory.alchemy import SQLAlchemyModelFactory as sqla_factory
import random
engine = create_engine("sqlite:////tmp/factory_boy.sql")
session = scoped_session(sessionmaker(bind=engine))
Base = declarative_base()
class Person(Base):
id = Column(Integer, primary_key=True)
name = Column(Text)
addresses = relationship("Address", backref="person")
class Address(Base):
id = Column(Integer, primary_key=True)
city = Column(Text)
street = Column(Text)
street_number = Column(Integer)
person_id = Column(Integer, ForeignKey('person.id'))
class AddressFactory(sqla_factory):
class Meta:
model = Address
sqlalchemy_session = session
street_number = random.randint(0, 10000)
street = "Commonwealth Ave"
city = "Boston"
class PersonFactory(sqla_factory):
class Meta:
model = Person
sqlalchemy_session = session
name = "John Doe"
Base.metadata.create_all(engine)
for i in range(100):
person = PersonFactory(addresses=AddressFactory.create_batch(3))
This creates 3 workouts for each person created, where each workout references the person via the person_id FK.

You can simply do batch_create so that for each DocFactory object new UserFactory object will create
`DocFactory.create_batch(10)`

sqlalchemy relation through another (declarative)

Is anyone familiar with ActiveRecord's "has_many :through" relations for models? I'm not really a Rails guy, but that's basically what I'm trying to do.
As a contrived example consider Projects, Programmers, and Assignments:
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy import Column, ForeignKey
from sqlalchemy.types import Integer, String, Text
from sqlalchemy.orm import relation
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Assignment(Base):
__tablename__ = 'assignment'
id = Column(Integer, primary_key=True)
description = Column(Text)
programmer_id = Column(Integer, ForeignKey('programmer.id'))
project_id = Column(Integer, ForeignKey('project.id'))
def __init__(self, description=description):
self.description = description
def __repr__(self):
return '<Assignment("%s")>' % self.description
class Programmer(Base):
__tablename__ = 'programmer'
id = Column(Integer, primary_key=True)
name = Column(String(64))
assignments = relation("Assignment", backref='programmer')
def __init__(self, name=name):
self.name = name
def __repr__(self):
return '<Programmer("%s")>' % self.name
class Project(Base):
__tablename__ = 'project'
id = Column(Integer, primary_key=True)
name = Column(String(64))
description = Column(Text)
assignments = relation("Assignment", backref='project')
def __init__(self, name=name, description=description):
self.name = name
self.description = description
def __repr__(self):
return '<Project("%s", "%s...")>' % (self.name, self.description[:10])
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
Projects have many Assignments.
Programmers have many Assignments. (understatement?)
But in my office at least, Programmers also have many Projects - I'd like this relationship to be inferred through the Assignments assigned to the Programmer.
I'd like the Programmer model to have a attribute "projects" which will return a list of Projects associated to the Programmer through the Assignment model.
me = session.query(Programmer).filter_by(name='clay').one()
projects = session.query(Project).\
join(Project.assignments).\
join(Assignment.programmer).\
filter(Programmer.id==me.id).all()
How can I describe this relationship clearly and simply using the sqlalchemy declarative syntax?
Thanks!

There are two ways I see:
Define a relation Programmer.projects with secondary='assignment'.
I define Assignment.project as relation and Programmer.projects as association_proxy('assignments', 'project') (probably you'd also like to define a creator). See Simplifying Association Object Relationships chapter for more information.