I have found many explanations for how to create a self-referential many-to-many relationship (for user followers or friends) using a separate table or class:
Below are three examples, one from Mike Bayer himself:
Many-to-many self-referential relationship in sqlalchemy
How can I achieve a self-referencing many-to-many relationship on the SQLAlchemy ORM back referencing to the same attribute?
Miguel Grinberg's Flask Megatutorial on followers
But in every example I've found, the syntax for defining the primaryjoin and secondaryjoin in the relationship is an early-binding one:
# this relationship is used for persistence
friends = relationship("User", secondary=friendship,
primaryjoin=id==friendship.c.friend_a_id,
secondaryjoin=id==friendship.c.friend_b_id,
)
This works great, except for one circumstance: when one uses a Base class to define the id column for all of your objects as shown in Mixins: Augmenting the base from the docs
My Base class and followers table are defined thusly:
from flask_sqlchalchemy import SQLAlchemy
db = SQLAlchemy()
class Base(db.Model):
__abstract__ = True
id = db.Column(db.Integer, primary_key=True)
user_flrs = db.Table(
'user_flrs',
db.Column('follower_id', db.Integer, db.ForeignKey('user.id')),
db.Column('followed_id', db.Integer, db.ForeignKey('user.id')))
But now I have trouble with the followers relationship that has served me loyally for a while before I moved the id's to the mixin:
class User(Base):
__table_name__ = 'user'
followed_users = db.relationship(
'User', secondary=user_flrs, primaryjoin=(user_flrs.c.follower_id==id),
secondaryjoin=(user_flrs.c.followed_id==id),
backref=db.backref('followers', lazy='dynamic'), lazy='dynamic')
db.class_mapper(User) # trigger class mapper configuration
Presumably because the id is not present in the local scope, though it seems to throw a strange error for that:
ArgumentError: Could not locate any simple equality expressions involving locally mapped foreign key columns for primary join condition 'user_flrs.follower_id = :follower_id_1' on relationship User.followed_users. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or are annotated in the join condition with the foreign() annotation. To allow comparison operators other than '==', the relationship can be marked as viewonly=True.
And it throws the same error if I change the parentheses to quotes to take advantage of late-binding. I have no idea how to annotate this thing with foreign() and remote() because I simply don't know what sqlalchemy would like me to describe as foreign and remote on a self-referential relationship that crosses a secondary table! I've tried many combinations of this, but it hasn't worked thus far.
I had a very similar (though not identical) problem with a self-referential relationship that did not span a separate table and the key was simply to convert the remote_side argument to a late-binding one. This makes sense to me, as the id column isn't present during an early-binding process.
If it is not late-binding that I am having trouble with, please advise. In the current scope, though, my understanding is that id is mapped to the Python builtin id() and thus will not work as an early-binding relationship.
Converting id to Base.id in the joins results in the following error:
ArgumentError: Could not locate any simple equality expressions involving locally mapped foreign key columns for primary join condition 'user_flrs.follower_id = "<name unknown>"' on relationship User.followed_users. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or are annotated in the join condition with the foreign() annotation. To allow comparison operators other than '==', the relationship can be marked as viewonly=True.
You can't use id in your join filters, no, because that's the built-in id() function, not the User.id column.
You have three options:
Define the relationship after creating your User model, assigning it to a new User attribute; you can then reference User.id as it has been pulled in from the base:
class User(Base):
# ...
User.followed_users = db.relationship(
User,
secondary=user_flrs,
primaryjoin=user_flrs.c.follower_id == User.id,
secondaryjoin=user_flrs.c.followed_id == User.id,
backref=db.backref('followers', lazy='dynamic'),
lazy='dynamic'
)
Use strings for the join expressions. Any argument to relationship() that is a string is evaluated as a Python expression when configuring the mapper, not just the first argument:
class User(Base):
# ...
followed_users = db.relationship(
'User',
secondary=user_flrs,
primaryjoin="user_flrs.c.follower_id == User.id",
secondaryjoin="user_flrs.c.followed_id == User.id",
backref=db.backref('followers', lazy='dynamic'),
lazy='dynamic'
)
Define the relationships as callables; these are called at mapper configuration time to produce the final object:
class User(Base):
# ...
followed_users = db.relationship(
'User',
secondary=user_flrs,
primaryjoin=lambda: user_flrs.c.follower_id == User.id,
secondaryjoin=lambda: user_flrs.c.followed_id == User.id,
backref=db.backref('followers', lazy='dynamic'),
lazy='dynamic'
)
For the latter two options, see the sqlalchemy.orgm.relationship() documentation:
Some arguments accepted by relationship() optionally accept a callable function, which when called produces the desired value. The callable is invoked by the parent Mapper at “mapper initialization” time, which happens only when mappers are first used, and is assumed to be after all mappings have been constructed. This can be used to resolve order-of-declaration and other dependency issues, such as if Child is declared below Parent in the same file*[.]*
[...]
When using the Declarative extension, the Declarative initializer allows string arguments to be passed to relationship(). These string arguments are converted into callables that evaluate the string as Python code, using the Declarative class-registry as a namespace. This allows the lookup of related classes to be automatic via their string name, and removes the need to import related classes at all into the local module space*[.]*
[...]
primaryjoin –
[...]
primaryjoin may also be passed as a callable function which is evaluated at mapper initialization time, and may be passed as a Python-evaluable string when using Declarative.
[...]
secondaryjoin –
[...]
secondaryjoin may also be passed as a callable function which is evaluated at mapper initialization time, and may be passed as a Python-evaluable string when using Declarative.
Both the string and the lambda define the same user_flrs.c.followed_id == User.id / user_flrs.c.follower_id == User.id expressions as used in the first option, but because they are given as a string and callable function, respectively, you postpone evaluation until SQLAlchemy needs to have those declarations finalised.
Related
I need to create a many to many relationship between two tables. I need to specify the foreign_keys option because I have other references in both tables to each other.
Tried multiple approaches using Class declaration of the assiciation table and Table object directly.
When I remove the foreign_keys option on both User andFeature` class, it works but when I add other fields with mappings between these two classes, I get the Multiple paths exception.
feature_user = Table(
'feature_user',
Base.metadata,
Column('feature_id', Integer, ForeignKey('features.id')),
Column('user_id', Integer, ForeignKey('users.id')),
)
class Feature(Base):
__tablename__ = 'features'
id = getIdColumn()
# other fields...
cpm_engineers = relationship(
'User',
secondary=feature_user,
foreign_keys=[feature_user.columns.user_id],
back_populates='cpm_engineer_of',
)
class User(Base):
__tablename__ = 'users'
id = getIdColumn()
# other fields...
cpm_engineer_of = relationship(
'Feature',
secondary=feature_user,
foreign_keys=[feature_user.columns.feature_id],
back_populates='cpm_engineers',
)
When creating a new User of a Feature, I get the following error:
sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Feature.cpm_engineers - there are no foreign keys linking these tables via secondary table 'feature_user'. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify 'primaryjoin' and 'secondaryjoin' expressions.
I believe the issue was I was trying to use the same table to store many-to-many associations for multiple bindings, e.g. feature-tester-user, feature-developer-user, etc. This is, of course, wrong. I needed to add a custom JOIN using a third column (e.g. add a new role column) or create a separate table for each association (which is what i did in the end).
I am during creating my first database project in SQLAlchemy and SQLite. I want to connect two entity as relational database's relational model. Here is the source:
class Models(Base):
__tablename__ = "models"
id_model = Column(Integer, primary_key=True)
name_of_model = Column(String, nullable = False)
price = Column(Integer, nullable = False)
def __init__(self, name_of_model):
self.name_of_model = name_of_model
class Cars(Base):
__tablename__ = "cars"
id_car = Column(Integer, primary_key=True)
id_equipment = Column(Integer, nullable = False)
id_package = Column(Integer, nullable = False)
id_model = Column(Integer, ForeignKey('Models'))
model = relationship("Models", backref=backref('cars', order_by = id_model))
I want to achieve a relationship like this:
https://imgur.com/af62zli
The error which occurs is:
The foreign key associated with column 'cars.id_model' could not find table 'Models' with which to generate a foreign key to target column 'None'.
Any ideas how to solve this problem?
From the docs:
The argument to ForeignKey is most commonly a string of the form
<tablename>.<columnname>, or for a table in a remote schema or “owner”
of the form <schemaname>.<tablename>.<columnname>. It may also be an
actual Column object...
In defining your ForeignKey on Cars.id_model you pass the string form of a class name ('Models') which is not an accepted form.
However, you can successfully define your foreign key using one of the below options:
ForeignKey(Models.id_model)
This uses the actual Column object to specify the foreign key. The disadvantage of this method is that you need to have the column in your namespace adding extra complexity in needing to import the model into a module if it is not defined there, and also may cause you to care about the order of instantiation of your models. This is why it's more common to use one of the string-based options, such as:
ForeignKey('models.id_model')
Notice that this example doesn't include the string version of the class name (not Models.id_model) but rather the string version of the table name. The string version means that table objects required are only resolved when needed and as such avoid the complexities of dealing with Column objects themselves.
Another interesting example that works in this case:
ForeignKey('models')
If the two columns are named the same on both tables, SQLAlchemy seems to infer the column from the table. If you alter the name of either of the id_model columns definitions in your example so that they are named differently, this will cease to work. Also I haven't found this to be well documented and it is less explicit, so not sure if really worth using and am really just mentioning for completeness and because I found it interesting. A comment in the source code of ForeignKey._column_tokens() seemed to be more explicit than the docs with respect to acceptable formatting of the column arg:
# A FK between column 'bar' and table 'foo' can be
# specified as 'foo', 'foo.bar', 'dbo.foo.bar',
# 'otherdb.dbo.foo.bar'. Once we have the column name and
# the table name, treat everything else as the schema
# name.
I have a table, MenuOptions which represents any option found in a dropdown in my app. Each option can be identified by the menu it is part of (e.g. MenuOptions.menu_name) and the specific value of that option (MenuOptions.option_value).
This table has relationships all across my db and doesn't use foreign keys, so I'm having trouble getting it to mesh with SQLAlchemy.
In SQL it would be as easy as:
SELECT
*
FROM
document
JOIN
menu_options ON menu_options.option_menu_name = 'document_type'
AND menu_options.option_value = document.document_type_id
to define this relationship. However I'm running into trouble when doing this in SQLAlchemy because I can't map this relationship cleanly without foreign keys. In SQLAlchemy the best I've done so far is:
the_doc = db.session.query(Document, MenuOptions).filter(
Document.id == document_id
).join(
MenuOptions,
and_(
MenuOptions.menu_name == text('"document_type"'),
MenuOptions.value == Document.type_id
)
).first()
Which does work, and does return the correct values, but returns them as a list of two separate model objects such that I have to reference the mapped Document properties via the_doc[0] and the mapped MenuOptions properties via the_doc[1]
Is there a way I can get this relationship returned as a single query object with all the properties on it without using foreign keys or any ForeignKeyConstraint in my model? I've tried add_columns and add_entity but I get essentially the same result.
you can use with_entities
entities = [getattr(Document, c) for c in Document.__table__.columns.keys()] + \
[getattr(MenuOptions, c) for c in MenuOptions.__table__.columns.keys()]
session.query(Document, MenuOptions).filter(
Document.id == document_id
).join(
MenuOptions,
and_(
MenuOptions.menu_name == text('"document_type"'),
MenuOptions.value == Document.type_id
)
).with_entities(*entities)
I ended up taking a slightly different approach using association_proxy, but if you ended up here from google then this should help you. In the following example, I store a document_type_id in the document table and hold the corresponding values for that id in a table called menu_options. Normally you would use foreign keys for this, but our menu_options is essentially an inhouse lookup table, and it contains relationships to several other tables so foreign keys are not a clean solution.
By first establishing a relationship via the primaryjoin property, then using associationproxy, I can immediately load the document_type based on the document_type_id with the following code:
from sqlalchemy import and_
from sqlalchemy.ext.associationproxy import association_proxy
class Document(db.Model):
__tablename__ = "document"
document_type_id = db.Column(db.Integer)
document_type_proxy = db.relationship(
"MenuOptions",
primaryjoin=(
and_(
MenuOptions.menu_name=='document_type',
foreign('Document.document_type_id')==MenuOptions.value
)
),
lazy="immediate"
viewonly=True
)
If all you need is a mapped relationship without the use of foreign keys within your database, then this will do just fine. If, however, you want to be able to access the remote attribute (in this case the document_type) directly as an attribute on the initial class (in this case Document) then you can use association_proxy to do so by simply passing the name of the mapped relationship and the name of the remote property:
document_type = association_proxy("document_type_proxy", "document_type")
I have no experience with sqlalchemy, and I have the following code:
class ForecastBedSetting(Base):
__tablename__ = 'forecast_bed_settings'
__table_args__ = (
Index('idx_forecast_bed_settings_forecast_id_property_id_beds',
'forecast_id', 'property_id',
'beds'),
)
forecast_id = Column(ForeignKey(u'forecasts.id'), nullable=False)
property_id = Column(ForeignKey(u'properties.id'), nullable=False, index=True)
# more definition of columns
Although I have checked this, I cannot understand what is the purpose of __table_args__, so I have no clue what this line is doing:
__table_args__ = (
Index('idx_forecast_bed_settings_forecast_id_property_id_beds',
'forecast_id', 'property_id',
'beds'),
)
Could somebody please explain me what is the purpose of __table_args__, and what the previous piece of code is doing.
This attribute accommodates both positional as well as keyword arguments that are normally sent to the Table constructor.
During construction of a declarative model class – ForecastBedSetting in your case – the metaclass that comes with Base creates an instance of Table. The __table_args__ attribute allows passing extra arguments to that Table. The created table is accessible through
ForecastBedSetting.__table__
The code defines an index inline, passing the Index instance to the created Table. The index uses string names to identify columns, so without being passed to a table SQLAlchemy could not know what table it belongs to.
I have a class:
class ExampleClass(Base):
__tablename__ = 'chart'
id = Column(Integer, primary_key=True)
element_id = Column(Integer, ForeignKey('anotherTable.id'))
element = relationship(AnotherClass)
element2_id = Column(Integer, ForeignKey('anotherTable2.id'))
element2 = relationship(AnotherClass2)
I want to do a lookup based on the element_id and element2_id :
class ExampleClass(Base):
...
def get_with_element2(self, element2):
return session.query(ExampleClass).\
filter_by(element_id = self.element_id,
element2_id = element2.id).first()
The problem I find is that if I instantiate a new ExampleClass object and assign it an element, the element_id field is not being set:
a = ExampleClass(element=element_obj)
a.element_id => None
How can I solve this? What's the best way to deal with this kind of situation?
First off, all the examples below assume that your ExampleClass instance is at least in the pending state if not the "persistent" state (that is, session.add(a)). In other words, if you aren't yet interacting with a Session and have not added the ExampleClass object to one, then you won't get any of the database-level behavior of relationship(), of which maintaining foreign key column values is the primary feature. You are of course free to make this assignment directly:
a = ExampleClass(element_id=element_obj.id)
but this is obviously not making use of the automation provided by the relationship() construct.
The assignment of foreign key attributes by relationship() occurs during a flush, which is a process that only occurs when interaction with the database is necessary, such as before you emit a SQL statement using session.query() or before you complete your transaction using session.commit().
Generally, the philosophy of relationship() is that you'd deal only with the "element" and "element2" attributes here, and let the foreign key attributes be handled behind the scenes. You can write your query like this:
session.query(ExampleClass).\
filter_by(element=self.element).\
filter_by(element2=element2)
The ORM will take a comparison such as SomeClass.somerelationship=someobject and convert that into the foreign-key expression SomeClass.some_fk=some_id, but the difference is, the evaluation of the ultimate value of "some_id" is deferred until the right before the query is executed. Before the query is executed, the Query() object tells the Session to "autoflush", which will have the effect of your ExampleClass row being inserted along with the primary key identifier of element_obj being assigned to the element_id attribute on the ExampleClass object.
you could get a similar effect while still using the FK attributes like this, this is mostly just to understand how it works though:
session.query(ExampleClass).\
filter_by(element_id=bindparam(callable_=lambda: self.element_id)).\
filter_by(element2_id=element2.id)
or even more explicit, do the flush first:
session.flush()
session.query(ExampleClass).\
filter_by(element_id=self.element_id).\
filter_by(element2_id=element2.id)
So to the degree you'd want to refer to foreign-key attributes like element_id explicitly, you'd also need to do the things relationship() does for you explicitly, as well. If you deal strictly with object instances and the relationship()-bound attribute, and leave typical defaults like autoflush enabled, it will generally do the "right thing" and make sure attributes are ready when needed.