I am trying to implement very simple example table from an old course now in SQLAlchemy...
I have got this far but when I run the code...
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, Date, MetaData
from sqlalchemy import ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy import create_engine
Base = declarative_base()
engine = create_engine('mysql://x # amazonaws.com:3306/db', echo=True)
class Guest(Base):
__tablename__ = "guests"
guest_no = Column(String(4), primary_key=True)
g_name = Column(String(20))
g_address = Column(String(30))
booking = relationship("Booking", back_populates="guests")
class Hotel(Base):
__tablename__ = "hotels"
hotel_no = Column(String(4), primary_key=True)
h_name = Column(String(20))
h_address = Column(String(30))
room = relationship("Room", back_populates="hotels")
booking = relationship("Booking", back_populates="hotels")
class Room(Base):
__tablename__ = "rooms"
hotel_no = Column(String(4), ForeignKey('hotels.hotel_no'), primary_key=True)
room_no = Column(String(4), primary_key=True)
r_type = Column(String(1))
r_price = Column(Integer)
hotel = relationship("Hotel", back_populates="rooms")
booking = relationship("Booking", back_populates="rooms")
class Booking(Base):
__tablename__ = "bookings"
hotel_no = Column(String(4), ForeignKey('hotels.hotel_no'), primary_key=True)
guest_no = Column(String(4), ForeignKey('guests.guest_no'), primary_key=True)
date_form = Column(Date, primary_key=True)
date_to = Column(Date)
room_no = Column(String(4), ForeignKey('rooms.room_no'), primary_key=True)
hotel = relationship("Hotel", back_populates="bookings")
guest = relationship("Guest", back_populates="bookings")
room = relationship("Room", back_populates="bookings")
Base.metadata.create_all(engine)
it gives me an error about the room_no foreign key...
2017-09-11 16:16:03 2b8010c29700 Error in foreign key constraint of table db/bookings:
FOREIGN KEY(room_no) REFERENCES rooms (room_no)
):
Cannot find an index in the referenced table where the
referenced columns appear as the first columns, or column types
in the table and the referenced table do not match for constraint.
Note that the internal storage type of ENUM and SET changed in
tables created with >= InnoDB-4.1.12, and such columns in old tables
cannot be referenced by such columns in new tables.
See http://dev.mysql.com/doc/refman/5.6/en/innodb-foreign-key-constraints.html
for correct foreign key definition.
I looked around a bit and I made sure they were both the same type (they were) and were both primary keys (they previously were not) but the error persists.
Does anyone have insight into what is causing this?
Because rooms has a composite primary key: (hotel_no, room_no) you'll need to specify both columns in your foreign key relationship on the booking table:
__table_args__ = (
ForeignKeyConstraint(
['hotel_no', 'room_no'],
['rooms.hotel_no', 'rooms.room_no']
),
)
Related
I am new in Sqlalchemy and trying to achieve the following goal with relationship():
There is an User table which stores user data.
Every user is able to invite other user with an invite_code.
Every user keeps a list of invitation, every invitation includes the invite_code and the invitee User
I think the relationship between User and Invitation is one-to-many. Since Invitation contains User, then I think it is probably better to use self-referential relationship to represent the inviter-to-invitaions(invitees) relationship and use an association object to store the invite_code.
I checked the sqlalchemy documentation and the question, tried to implement the classed like this:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, String
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Invitation(Base):
__tablename__ = 'invitation'
invite_code = Column(Integer)
inviter_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
invitee_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
invitee = relationship('User') #Need HELP here
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
inviters = relationship('User',
secondary='invitation',
primaryjoin=id==Invitation.invitee_id,
secondaryjoin=id==Invitation.inviter_id,
backref='invitees')
invitations = relationship('Invitation')# Need HELP here
def __repr__(self):
return f'User: {self.name}'
if __name__ == '__main__':
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(engine)
db = Session()
inviter1 = User(name='inviter1')
inviter2 = User(name='inviter2')
invitee1= User(name='invitee1')
invitee2 = User(name='invitee2')
inviter1.invitees = [invitee1, invitee2]
inviter2.invitees = [invitee1]
db.add(inviter1)
db.add(inviter2)
db.add(invitee1)
db.add(invitee2)
db.commit()
users = db.query(User).all()
for user in users:
print(user)
print(' Inviter: ', user.inviters)
print(' Invitee: ', user.invitees)
print()
If the lines with comment #Need HELP here are deleted, I can get the corresponding inviters and invitees, but cannot get the invite_code. If the #Need HELP here code are added, the error is:
Exception has occurred: AmbiguousForeignKeysError
Could not determine join condition between parent/child tables on relationship Invitation.invitee - there are multiple foreign key paths linking the tables. Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as containing a foreign key reference to the parent table.
Is there a way to add extra data column in association object like association object for many-to-many relationship for self referential table?
Sorry for the too much text, I didn't find any reference document on the web.
Finally, I figured it out with the help of foreign_keys:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, String
from sqlalchemy.orm import relationship, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
sent_invitations = relationship('Invitation', foreign_keys='Invitation.inviter_id', back_populates='inviter', cascade='all, delete')
received_invitations=relationship('Invitation', foreign_keys='Invitation.invitee_id', back_populates='invitee', cascade='all, delete')
def __repr__(self):
return f'User: {self.name}'
class Invitation(Base):
__tablename__ = 'invitation'
id = Column(Integer, primary_key=True)
invite_code = Column(Integer)
inviter_id = Column(Integer, ForeignKey('user.id'))
invitee_id = Column(Integer, ForeignKey('user.id'))
inviter=relationship('User', foreign_keys=[inviter_id], back_populates='sent_invitations')
invitee=relationship('User', foreign_keys=[invitee_id], back_populates='received_invitations')
def __repr__(self):
return f'Invitation: {self.inviter} invited {self.invitee} with {self.invite_code}'
if __name__ == '__main__':
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
Session = sessionmaker(engine)
db = Session()
inviter1 = User(name='inviter1')
inviter2 = User(name='inviter2')
invitee1= User(name='invitee1')
invitee2 = User(name='invitee2')
invitation1 = Invitation(invite_code=50, inviter=inviter1, invitee=invitee1)
invitation2 = Invitation(invite_code=20, inviter=inviter2, invitee=invitee2)
invitation3 = Invitation(invite_code=22, inviter=inviter1, invitee=inviter2)
invitation4 = Invitation(invite_code=44, inviter=invitee1, invitee=inviter2)
db.add(inviter1)
db.add(inviter2)
db.add(invitee1)
db.add(invitee2)
db.commit()
users = db.query(User).all()
for user in users:
print(user)
print(' sent_invitation: ', user.sent_invitations)
print(' received_invitation: ', user.received_invitations)
print()
invitations = db.query(Invitation).all()
for invitation in invitations:
print(invitation)
db.delete(inviter1)
db.delete(invitee2)
db.commit()
I am still a beginner in Python and I am stuck with the following relation.
Three tables:
tx_bdproductsdb_domain_model_product
sys_category
sys_category_record_mm
sys_category class looks like this:
class Category(Base):
__tablename__ = "sys_category"
uid = Column(
Integer,
ForeignKey("sys_category_record_mm.uid_local"),
primary_key=True,
autoincrement=True,
)
title = Column(String)
products = relationship(
"Product",
uselist=False,
secondary="sys_category_record_mm",
back_populates="categories",
foreign_keys=[uid],
)
Products looks like this:
class Product(Base):
__tablename__ = "tx_bdproductsdb_domain_model_product"
uid = Column(
Integer,
ForeignKey(SysCategoryMMProduct.uid_foreign),
primary_key=True,
autoincrement=True,
)
category = Column(Integer)
categories = relationship(
Category,
secondary=SysCategoryMMProduct,
back_populates="products",
foreign_keys=[uid],
)
And here is the mm table class that should link the two.
class SysCategoryMMProduct(Base):
__tablename__ = "sys_category_record_mm"
uid_local = Column(Integer, ForeignKey(Category.uid), primary_key=True)
uid_foreign = Column(
Integer, ForeignKey("tx_bdproductsdb_domain_model_product.uid")
)
fieldname = Column(String)
I'm currently stuck, does anyone have any ideas? I get the following messages in the console:
sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Category.products - there are no foreign keys linking these tables via secondary table 'sys_category_record_mm'. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify 'primaryjoin' and 'secondaryjoin' expressions.
root#booba:/var/pythonWorks/crawler/develop/releases/current# python3 Scraper2.py
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/relationships.py", line 2739, in _determine_joins
self.secondaryjoin = join_condition(
File "<string>", line 2, in join_condition
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/selectable.py", line 1229, in _join_condition
raise exc.NoForeignKeysError(
sqlalchemy.exc.NoForeignKeysError: Can't find any foreign key relationships between 'tx_bdproductsdb_domain_model_product' and 'sys_category_record_mm'.
sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Category.products - there are no foreign keys linking these tables via secondary table 'sys_category_record_mm'. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify 'primaryjoin' and 'secondaryjoin' expressions.
Thank you :)
When using an association class you should reference the association directly. You need this instead of secondary because you have data associated with the link (ie. fieldname). I changed some of your naming schema to make it more clear.
There is a pretty good explanation of the association pattern in the sqlalchemy docs. There is a big red warning at the end of that section about mixing the use of the secondary and the Association pattern.
I use backref="related_categories" to automatically create the property related_categories on Product. This is a list of association objects, and not actual categories.
from sqlalchemy import (
create_engine,
Integer,
String,
ForeignKey,
)
from sqlalchemy.schema import (
Column,
)
from sqlalchemy.orm import declarative_base, relationship
from sqlalchemy.orm import Session
Base = declarative_base()
# This connection string is made up
engine = create_engine(
'postgresql+psycopg2://user:pw#/db',
echo=False)
class Category(Base):
__tablename__ = "categories"
uid = Column(
Integer,
primary_key=True,
autoincrement=True,
)
title = Column(String)
class Product(Base):
__tablename__ = "products"
uid = Column(
Integer,
primary_key=True,
autoincrement=True,
)
title = Column(String)
class SysCategoryMMProduct(Base):
__tablename__ = "categories_products"
uid = Column(Integer, primary_key=True)
category_uid = Column(Integer, ForeignKey("categories.uid"))
product_uid = Column(Integer, ForeignKey("products.uid"))
fieldname = Column(String)
product = relationship(
"Product",
backref="related_categories",
)
category = relationship(
"Category",
backref="related_products",
)
Base.metadata.create_all(engine)
with Session(engine) as session:
category = Category(title="kitchen")
session.add(category)
product = Product(title="spoon")
session.add(product)
association = SysCategoryMMProduct(
product=product,
category=category,
fieldname="Extra metadata")
session.add(association)
session.commit()
category = session.query(Category).first()
assert len(category.related_products) == 1
assert category.related_products[0].product.related_categories[0].category == category
q = session.query(Category).join(Category.related_products).join(SysCategoryMMProduct.product).filter(Product.title == "spoon")
print (q)
assert q.first() == category
The last query looks like:
SELECT categories.uid AS categories_uid, categories.title AS categories_title
FROM categories JOIN categories_products ON categories.uid = categories_products.category_uid JOIN products ON products.uid = categories_products.product_uid
WHERE products.title = 'spoon'
I have a model in SQLAlchemy which defines a many-to-many relationship using an association table (automap is being used here because I'm using an existing database):
from sqlalchemy import (Column, Table, MetaData, Integer, Text, LargeBinary,
ForeignKey, Float, Boolean, Index)
from sqlalchemy.ext.automap import automap_base, AutomapBase
from sqlalchemy.orm import Session, deferred, relationship
Base: AutomapBase = automap_base()
class VariantAssociation(Base):
__tablename__ = "sample_variant_association"
vid = Column(Integer, ForeignKey("variants.variant_id"),
primary_key=True)
sid = Column(Integer, ForeignKey("samples.sample_id"),
primary_key=True)
vdepth = Column(Integer)
valt_depth = Column(Integer)
gt = Column(Text)
gt_type = Column(Integer)
fraction = Column(Float)
variant = relationship("Variant", back_populates="samples")
sample = relationship("Sample", back_populates="variants")
__table_args__ = (Index('ix_sample_variant_association_valt_depth',
"valt_depth"),
Index('ix_sample_variant_association_vdepth',
"vdepth"),
Index('ix_sample_variant_association_vid', 'vid'),
Index('ix_sample_variant_association_sid', 'sid'),
Index('ix_sample_variant_association_fraction',
'fraction')
)
class Variant(Base):
__tablename__ = "variants"
variant_id = Column(Integer, primary_key=True)
info = deferred(Column(LargeBinary))
samples = relationship("VariantAssociation",
back_populates="variant")
class Sample(Base):
__tablename__ = "samples"
sample_id = Column(Integer, primary_key=True, index=True)
name = Column(Text, index=True)
variants = relationship("VariantAssociation",
back_populates="sample")
class SampleGenotypeCount(Base):
__tablename__ = 'sample_genotype_counts'
sample_id = Column(Integer, primary_key=True)
num_hom_ref = Column(Integer)
num_het = Column(Integer)
num_hom_alt = Column(Integer)
num_unknown = Column(Integer)
class DataMigration(Base):
__tablename__ = "datamigration"
done = Column(Boolean, primary_key=True)
On querying, this eventually generates this warning:
Query:
query = session.query(Variant).join(
Variant.samples).join(Sample)
Warning:
/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/relationships.py:3441: SAWarning:
relationship 'Variant.variantassociation_collection' will copy column variants.variant_id to
column sample_variant_association.vid, which conflicts with relationship(s): 'Variant.samples'
(copies variants.variant_id to sample_variant_association.vid). If this is not the intention,
consider if these relationships should be linked with back_populates, or if viewonly=True
should be applied to one or more if they are read-only. For the less common case that foreign
key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate
the columns that should be written towards. The 'overlaps' parameter may be used to remove
this warning. (Background on this error at: http://sqlalche.me/e/14/qzyx)
I've been looking through SO and the SQLAlchemy documentation but I was unable to find what could cause this issue since (in my view) the back_populates parameters are in the right places.
Where would the error in the model be? SQLAlchemy 1.3.23 did not generate one, FTR.
In order to set your own relationship names, you need to prevent Automap from generating relationships by iteself. You can achieve this by setting 'generate_relationship' to a function that returns None.
def generate_relationships(base, direction, return_fn, attrname, local_cls, referred_cls, **kw):
return None
Base.prepare(generate_relationship=generate_relationships)
I'm new to the SQLAlchemy and I wrote a simple CRUD database layout like this: I have three tables, Customer, Product, and Template. The idea is this: Each customer can have a template of the products he usually orders. When I fetch a particular customer from the database, his template along with all the products should be fetched as well. So I have a one-to-one relationship (customer-template) and one many-to-many relationship (template-product). Now, a template should contain fields such as quantity of a particular product, along with its net, gross and tax values. I'd like to have a listener on the quantity column, such that when the quantity of a particular product is changed, other attributes I mentioned will be changed too. So the code I wrote is as follows (please, if you can, also verify whether all the relationships are written appropriately)
from sqlalchemy import event
from sqlalchemy.orm import relationship, exc, column_property, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm.query import Query as _Query
from sqlalchemy import Table, Column, Integer, String, Boolean, ForeignKey, UniqueConstraint, create_engine, Numeric
from decimal import *
# decimal operations settings
getcontext().prec = 6
getcontext().rounding = ROUND_HALF_UP
engine = create_engine('sqlite:///test.db')
Base = declarative_base()
# Initalize the database if it is not already.
Base.metadata.create_all(engine)
# Create a session to handle updates.
Session = sessionmaker(bind=engine)
# customer has a one-to-one relationship with template, where customer is the parent
class Customer(Base):
__tablename__ = "customers"
id = Column(Integer, primary_key=True)
alias = Column(String)
firm_name = Column(String)
last_name = Column(String)
first_name = Column(String)
tax_id = Column(String, nullable=False)
address = Column(String)
postal_code = Column(String)
city = Column(String)
payment = Column(Boolean)
template = relationship("Template", uselist=False, back_populates="customer")
# product has many-to-many relationship with template, where template is the parent
class Product(Base):
__tablename__ = "products"
id = Column(Integer, primary_key=True)
product_name = Column(String, nullable=False)
symbol = Column(String)
unit = Column(String, nullable=False)
unit_net_price = Column(Numeric, nullable=False)
vat_rate = Column(Numeric, nullable=False)
per_month = Column(Boolean, nullable=False)
# association table for the products-template many-to-many relationship
association_table = Table('association', Base.metadata,
Column('product_id', Integer, ForeignKey('product.id')),
Column('template_id', Integer, ForeignKey('template.id'))
)
# template implements one-to-one relationship with customer and many-to-many relationship with product
class Template(Base):
id = Column(Integer, primary_key=True)
customer_id = Column(Integer, ForeignKey("customer.id"))
customer = relationship("Customer", back_populates="template")
products = relationship("Product", secondary=association_table)
quantity = Column(Numeric)
net_val = Column(Numeric)
tax_val = Column(Numeric)
gross_val = Column(Numeric)
# TODO: implement constructor
def __init__(self, **kwargs):
self.quantity = Decimal(0.0)
self.net_val = Decimal(0.0)
self.tax_val = Decimal(0.0)
self.gross_val = Decimal(0.0)
#event.listens_for(Template.quantity, "set")
def quantity_listener(target, value, oldvalue, initiator):
print(target)
print(initiator)
# target.net_val =
# target.tax_val =
# target.gross_val =
Now, I'm unsure how should I get a particular initiator and set its values, since products in Template table is a list (?)
I'd probably do it like this. It's hard to use Sqlalchemy orm in such cases as you can't access the Session object we all got used to use.
#event.listens_for(Template, "after_update")
def quantity_listener(mapper, connection, target):
field = 'quantity'
added, _, deleted = get_history(target, field)
# added is a new values of the specified field
# deleted is the old values
# so we basically check if 'quantity' has changed
# its a tuple btw
if added and deleted and added[0] != deleted[0]:
stmt = Template.update(). \
values(Template.something=Template.something * Template.other_something). \
where(Template.id == target.id)
conn.execute(stmt)
# here goes another updates for products or you can have another
# listener for them
There might be a better way to accomplish this. I can't debug this right now and I can't manage to get your examples working. Hope it helps.
I am playing with a toy example to see the back populates in action but hitting an error that I can't understand. Below I have two 'models' that back populate each other. When I try to create a User object it throws an error. What am I missing?
"sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Child.user - there are no foreign keys linking these tables. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression."
engine = create_engine('sqlite:///:memory:', echo=False)
Session = sessionmaker(bind=engine)
session = Session()
Base = declarative_base(bind=engine)
class User(Base):
__tablename__ = 'user'
user_id = Column("UserID", Integer, primary_key=True)
name = Column("Name", String(50))
age = Column("Age", SmallInteger)
child = relationship("Child", back_populates="user")
class Child(Base):
__tablename__ = 'child'
child_id = Column("ChildID", Integer, primary_key=True)
school = Column("School", String)
grade = Column("Grade", String)
user_id = Column("UserID", Integer, ForeignKey('User.UserID'), index=True, nullable=True)
user = relationship("User", back_populates="child")
ForeignKey requires the table and column name, not model and attribute name, so it should be:
user_id = Column("UserID", Integer, ForeignKey('user.UserID'), ...)
because your User model has a table name of user.