I am playing with a toy example to see the back populates in action but hitting an error that I can't understand. Below I have two 'models' that back populate each other. When I try to create a User object it throws an error. What am I missing?
"sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Child.user - there are no foreign keys linking these tables. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression."
engine = create_engine('sqlite:///:memory:', echo=False)
Session = sessionmaker(bind=engine)
session = Session()
Base = declarative_base(bind=engine)
class User(Base):
__tablename__ = 'user'
user_id = Column("UserID", Integer, primary_key=True)
name = Column("Name", String(50))
age = Column("Age", SmallInteger)
child = relationship("Child", back_populates="user")
class Child(Base):
__tablename__ = 'child'
child_id = Column("ChildID", Integer, primary_key=True)
school = Column("School", String)
grade = Column("Grade", String)
user_id = Column("UserID", Integer, ForeignKey('User.UserID'), index=True, nullable=True)
user = relationship("User", back_populates="child")
ForeignKey requires the table and column name, not model and attribute name, so it should be:
user_id = Column("UserID", Integer, ForeignKey('user.UserID'), ...)
because your User model has a table name of user.
Related
I am new in Sqlalchemy and trying to achieve the following goal with relationship():
There is an User table which stores user data.
Every user is able to invite other user with an invite_code.
Every user keeps a list of invitation, every invitation includes the invite_code and the invitee User
I think the relationship between User and Invitation is one-to-many. Since Invitation contains User, then I think it is probably better to use self-referential relationship to represent the inviter-to-invitaions(invitees) relationship and use an association object to store the invite_code.
I checked the sqlalchemy documentation and the question, tried to implement the classed like this:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, String
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Invitation(Base):
__tablename__ = 'invitation'
invite_code = Column(Integer)
inviter_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
invitee_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
invitee = relationship('User') #Need HELP here
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
inviters = relationship('User',
secondary='invitation',
primaryjoin=id==Invitation.invitee_id,
secondaryjoin=id==Invitation.inviter_id,
backref='invitees')
invitations = relationship('Invitation')# Need HELP here
def __repr__(self):
return f'User: {self.name}'
if __name__ == '__main__':
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(engine)
db = Session()
inviter1 = User(name='inviter1')
inviter2 = User(name='inviter2')
invitee1= User(name='invitee1')
invitee2 = User(name='invitee2')
inviter1.invitees = [invitee1, invitee2]
inviter2.invitees = [invitee1]
db.add(inviter1)
db.add(inviter2)
db.add(invitee1)
db.add(invitee2)
db.commit()
users = db.query(User).all()
for user in users:
print(user)
print(' Inviter: ', user.inviters)
print(' Invitee: ', user.invitees)
print()
If the lines with comment #Need HELP here are deleted, I can get the corresponding inviters and invitees, but cannot get the invite_code. If the #Need HELP here code are added, the error is:
Exception has occurred: AmbiguousForeignKeysError
Could not determine join condition between parent/child tables on relationship Invitation.invitee - there are multiple foreign key paths linking the tables. Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as containing a foreign key reference to the parent table.
Is there a way to add extra data column in association object like association object for many-to-many relationship for self referential table?
Sorry for the too much text, I didn't find any reference document on the web.
Finally, I figured it out with the help of foreign_keys:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, String
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
sent_invitations = relationship('Invitation', foreign_keys='Invitation.inviter_id', back_populates='inviter', cascade='all, delete')
received_invitations=relationship('Invitation', foreign_keys='Invitation.invitee_id', back_populates='invitee', cascade='all, delete')
def __repr__(self):
return f'User: {self.name}'
class Invitation(Base):
__tablename__ = 'invitation'
id = Column(Integer, primary_key=True)
invite_code = Column(Integer)
inviter_id = Column(Integer, ForeignKey('user.id'))
invitee_id = Column(Integer, ForeignKey('user.id'))
inviter=relationship('User', foreign_keys=[inviter_id], back_populates='sent_invitations')
invitee=relationship('User', foreign_keys=[invitee_id], back_populates='received_invitations')
def __repr__(self):
return f'Invitation: {self.inviter} invited {self.invitee} with {self.invite_code}'
if __name__ == '__main__':
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(engine)
db = Session()
inviter1 = User(name='inviter1')
inviter2 = User(name='inviter2')
invitee1= User(name='invitee1')
invitee2 = User(name='invitee2')
invitation1 = Invitation(invite_code=50, inviter=inviter1, invitee=invitee1)
invitation2 = Invitation(invite_code=20, inviter=inviter2, invitee=invitee2)
invitation3 = Invitation(invite_code=22, inviter=inviter1, invitee=inviter2)
invitation4 = Invitation(invite_code=44, inviter=invitee1, invitee=inviter2)
db.add(inviter1)
db.add(inviter2)
db.add(invitee1)
db.add(invitee2)
db.commit()
users = db.query(User).all()
for user in users:
print(user)
print(' sent_invitation: ', user.sent_invitations)
print(' received_invitation: ', user.received_invitations)
print()
invitations = db.query(Invitation).all()
for invitation in invitations:
print(invitation)
db.delete(inviter1)
db.delete(invitee2)
db.commit()
I am still a beginner in Python and I am stuck with the following relation.
Three tables:
tx_bdproductsdb_domain_model_product
sys_category
sys_category_record_mm
sys_category class looks like this:
class Category(Base):
__tablename__ = "sys_category"
uid = Column(
Integer,
ForeignKey("sys_category_record_mm.uid_local"),
primary_key=True,
autoincrement=True,
)
title = Column(String)
products = relationship(
"Product",
uselist=False,
secondary="sys_category_record_mm",
back_populates="categories",
foreign_keys=[uid],
)
Products looks like this:
class Product(Base):
__tablename__ = "tx_bdproductsdb_domain_model_product"
uid = Column(
Integer,
ForeignKey(SysCategoryMMProduct.uid_foreign),
primary_key=True,
autoincrement=True,
)
category = Column(Integer)
categories = relationship(
Category,
secondary=SysCategoryMMProduct,
back_populates="products",
foreign_keys=[uid],
)
And here is the mm table class that should link the two.
class SysCategoryMMProduct(Base):
__tablename__ = "sys_category_record_mm"
uid_local = Column(Integer, ForeignKey(Category.uid), primary_key=True)
uid_foreign = Column(
Integer, ForeignKey("tx_bdproductsdb_domain_model_product.uid")
)
fieldname = Column(String)
I'm currently stuck, does anyone have any ideas? I get the following messages in the console:
sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Category.products - there are no foreign keys linking these tables via secondary table 'sys_category_record_mm'. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify 'primaryjoin' and 'secondaryjoin' expressions.
root#booba:/var/pythonWorks/crawler/develop/releases/current# python3 Scraper2.py
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/relationships.py", line 2739, in _determine_joins
self.secondaryjoin = join_condition(
File "<string>", line 2, in join_condition
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/selectable.py", line 1229, in _join_condition
raise exc.NoForeignKeysError(
sqlalchemy.exc.NoForeignKeysError: Can't find any foreign key relationships between 'tx_bdproductsdb_domain_model_product' and 'sys_category_record_mm'.
sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Category.products - there are no foreign keys linking these tables via secondary table 'sys_category_record_mm'. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify 'primaryjoin' and 'secondaryjoin' expressions.
Thank you :)
When using an association class you should reference the association directly. You need this instead of secondary because you have data associated with the link (ie. fieldname). I changed some of your naming schema to make it more clear.
There is a pretty good explanation of the association pattern in the sqlalchemy docs. There is a big red warning at the end of that section about mixing the use of the secondary and the Association pattern.
I use backref="related_categories" to automatically create the property related_categories on Product. This is a list of association objects, and not actual categories.
from sqlalchemy import (
create_engine,
Integer,
String,
ForeignKey,
)
from sqlalchemy.schema import (
Column,
)
from sqlalchemy.orm import declarative_base, relationship
from sqlalchemy.orm import Session
Base = declarative_base()
# This connection string is made up
engine = create_engine(
'postgresql+psycopg2://user:pw#/db',
echo=False)
class Category(Base):
__tablename__ = "categories"
uid = Column(
Integer,
primary_key=True,
autoincrement=True,
)
title = Column(String)
class Product(Base):
__tablename__ = "products"
uid = Column(
Integer,
primary_key=True,
autoincrement=True,
)
title = Column(String)
class SysCategoryMMProduct(Base):
__tablename__ = "categories_products"
uid = Column(Integer, primary_key=True)
category_uid = Column(Integer, ForeignKey("categories.uid"))
product_uid = Column(Integer, ForeignKey("products.uid"))
fieldname = Column(String)
product = relationship(
"Product",
backref="related_categories",
)
category = relationship(
"Category",
backref="related_products",
)
Base.metadata.create_all(engine)
with Session(engine) as session:
category = Category(title="kitchen")
session.add(category)
product = Product(title="spoon")
session.add(product)
association = SysCategoryMMProduct(
product=product,
category=category,
fieldname="Extra metadata")
session.add(association)
session.commit()
category = session.query(Category).first()
assert len(category.related_products) == 1
assert category.related_products[0].product.related_categories[0].category == category
q = session.query(Category).join(Category.related_products).join(SysCategoryMMProduct.product).filter(Product.title == "spoon")
print (q)
assert q.first() == category
The last query looks like:
SELECT categories.uid AS categories_uid, categories.title AS categories_title
FROM categories JOIN categories_products ON categories.uid = categories_products.category_uid JOIN products ON products.uid = categories_products.product_uid
WHERE products.title = 'spoon'
I am new to sqlalchemy. I can create database tables by declarative mapping like this:
engine = create_engine("--engine works---")
Base = declarative_base()
class Customer(Base):
__tablename__ = 'customer'
customer_id = Column(Integer, primary_key=True)
name = Column(String(30))
email = Column(String(30))
invoices = relationship(
'Invoice',
order_by="Invoice.inv_id",
back_populates='customer',
cascade="all, delete, delete-orphan"
)
class Invoice(Base):
__tablename__ = 'invoice'
inv_id = Column(Integer, primary_key=True)
name = Column(String(30))
created = Column(Date)
customer_id = Column(ForeignKey('customer.customer_id'))
customer = relationship('Customer', back_populates='invoices')
Base.metadata.create_all(engine)
This is fine. I added some data into both customer and invoice tables.
So far so good. Next, I would try out automap_base on this existing database like this:
from sqlalchemy import select, text
from sqlalchemy.orm import declarative_base, sessionmaker
from sqlalchemy.ext.automap import automap_base
engine = create_engine('--engine works---')
Base = automap_base()
# reflect
Base.prepare(engine, reflect=True)
Customer = Base.classes.customer
Invoice = Base.classes.invoice
Session = sessionmaker(bind=engine, future=True)
session = Session()
# query invoice
stmt = select(Customer, Invoice).join(Customer.invoices).order_by(Customer.customer_id, Invoice.inv_id)
res = session.execute(stmt)
for c in res:
print(c.customer_id)
When I ran the code, I got:
AttributeError: type object 'customer' has no attribute 'invoices'
What did I miss for the relationship on the Customer (one side) or Invoice (many side) in this case so that when I query for customers with its invoices attibute and for invoices with customer attribute? Thanks for any help.
By default, automap will create the relation in the parent by appending "_collection" the lower-cased classname, so the name will be Customer.invoice_collection.
While answering this, I found that the join would raise an AttributeError on Customer.invoice_collection unless I performed a query on Customer beforehand, for example
session.execute(sa.select(Customer).where(False))
I'm not sure why that happens, however you don't necessarily need the join as you can iterate over Customer.invoice_collection directly, or join against the invoice table:
stmt = sa.select(Customer, Invoice).join(Invoice)
res = session.execute(stmt)
for c, i in res:
print(c.customer_id, i.inv_id)
I am trying to implement very simple example table from an old course now in SQLAlchemy...
I have got this far but when I run the code...
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, Date, MetaData
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy import create_engine
Base = declarative_base()
engine = create_engine('mysql://x # amazonaws.com:3306/db', echo=True)
class Guest(Base):
__tablename__ = "guests"
guest_no = Column(String(4), primary_key=True)
g_name = Column(String(20))
g_address = Column(String(30))
booking = relationship("Booking", back_populates="guests")
class Hotel(Base):
__tablename__ = "hotels"
hotel_no = Column(String(4), primary_key=True)
h_name = Column(String(20))
h_address = Column(String(30))
room = relationship("Room", back_populates="hotels")
booking = relationship("Booking", back_populates="hotels")
class Room(Base):
__tablename__ = "rooms"
hotel_no = Column(String(4), ForeignKey('hotels.hotel_no'), primary_key=True)
room_no = Column(String(4), primary_key=True)
r_type = Column(String(1))
r_price = Column(Integer)
hotel = relationship("Hotel", back_populates="rooms")
booking = relationship("Booking", back_populates="rooms")
class Booking(Base):
__tablename__ = "bookings"
hotel_no = Column(String(4), ForeignKey('hotels.hotel_no'), primary_key=True)
guest_no = Column(String(4), ForeignKey('guests.guest_no'), primary_key=True)
date_form = Column(Date, primary_key=True)
date_to = Column(Date)
room_no = Column(String(4), ForeignKey('rooms.room_no'), primary_key=True)
hotel = relationship("Hotel", back_populates="bookings")
guest = relationship("Guest", back_populates="bookings")
room = relationship("Room", back_populates="bookings")
Base.metadata.create_all(engine)
it gives me an error about the room_no foreign key...
2017-09-11 16:16:03 2b8010c29700 Error in foreign key constraint of table db/bookings:
FOREIGN KEY(room_no) REFERENCES rooms (room_no)
):
Cannot find an index in the referenced table where the
referenced columns appear as the first columns, or column types
in the table and the referenced table do not match for constraint.
Note that the internal storage type of ENUM and SET changed in
tables created with >= InnoDB-4.1.12, and such columns in old tables
cannot be referenced by such columns in new tables.
See http://dev.mysql.com/doc/refman/5.6/en/innodb-foreign-key-constraints.html
for correct foreign key definition.
I looked around a bit and I made sure they were both the same type (they were) and were both primary keys (they previously were not) but the error persists.
Does anyone have insight into what is causing this?
Because rooms has a composite primary key: (hotel_no, room_no) you'll need to specify both columns in your foreign key relationship on the booking table:
__table_args__ = (
ForeignKeyConstraint(
['hotel_no', 'room_no'],
['rooms.hotel_no', 'rooms.room_no']
),
)
I have two tables, users and contacts. I query the contacts table and get a list of a user's contacts. I would then like to be able to write Contact.first_name (where first_name is a row from the users table) and print out that contact's first name.
Currently, my Contact object does not recognize any attributes of the user table.
Here is some code:
class User(Base):
""" Basic User definition """
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
first_name = Column(Unicode(255))
last_name = Column(Unicode(255))
contacts = relationship('Contact', backref='users')
class Contact(Base):
__tablename__ = 'contacts'
id = Column(Integer, primary_key=True)
user_id = Column(Integer)
contact_id = Column(Integer)
__table_args__ = (ForeignKeyConstraint([id], [User.id]), {})
Here is my query:
Contact.query.filter(Contact.user_id == self.user_id).filter(Contact.state == True).all()
To be honest, I'm unsure of how to properly map my two foreign keys Contact.user_id and Contact.contact_id to the User.id row. Maybe this is the source of my problem?
I'm very new to using SQLAlchemy, so this is a learning experience here. Thanks for your help.
What you have here is class User which essentially refers to itself. In other words, it's a self-referential many-to-many relationship. Your model definitions should look like this:
# This is so called association table, which links two tables in many-to-many
# relationship. In this case it links same table's ('users') different rows.
user_contacts = Table(
'user_contacts', Base.metadata,
Column('user_id', Integer, ForeignKey('users.id'), primary_key=True),
Column('contact_id', Integer, ForeignKey('users.id'), primary_key=True),
)
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
first_name = Column(String)
last_name = Column(String)
contacts = relationship(
'User',
secondary=user_contacts,
primaryjoin=id==user_contacts.c.user_id,
secondaryjoin=id==user_contacts.c.contact_id
)
Then you can do things like the following:
u1 = User(first_name='Foo', last_name='Foo')
u2 = User(first_name='Bar', last_name='Bar')
u3 = User(first_name='Baz', last_name='Baz')
u1.contacts = [u2, u3]
session.add(u1)
session.commit()
# ... and in some other place in your code ...
u = User.query.get(1)
print u.contacts[0].first_name