Existing PostgreSQL database have tables organized in different "schemas" to split a large database (both for scaling and implementing fine-tuned security at server level). Similarly, the declarative_base table descriptions are organized in different files in a package - one file per schema:
package
__init__.py
tables_in_schema1.py
tables_in_schema2.py
Metadata and engine object goes into each file from the top of the package as db.Model. For example, tables_in_schema1.py would have (ignoring the necessary ORM imports and then need for back references) table alpha:
from package import db
class TableAlpha(db.Model, object):
__tablename__ = "alpha"
__table_args__ = ({"schema": "schema1"})
id_alpha = Column(INTEGER, Sequence("pk_alpha", 1, 1), primary_key=True)
class MixinAlphaRelation(object):
#declared_attr
def id_alpha(cls):
return Column(INTEGER, ForeignKey("schema1.alpha.id_alpha"))
Now in tables_in_schema2.py, two tables are defined. One is a stand-alone table, called beta, and the other is a linking table for the one-to-many relationship between alpha and beta to called table rho:
from package import db
from package.tables_in_schema1 import MixinAlphaRelation
class TableBeta(db.Model, object):
__tablename__ = "beta"
__table_args__ = ({"schema": "schema2"})
id_beta = Column(INTEGER, Sequence("pk_beta", 1, 1), primary_key=True)
class MixinBetaRelation(object):
#declared_attr
def id_beta(cls):
return Column(INTEGER, ForeignKey("schema2.beta.id_beta"))
class TableRho(db.Model, MixinAlphaRelation, MixinBetaRelation):
__tablename__ = "rho"
__table_args__ = (
UniqueConstraint("id_alpha", "id_beta", name="uq_rho_alpha-beta"),
{"schema": "schema2"})
id_row = Column(INTEGER, Sequence("pk_rho", 1, 1), primary_key=True)
The intended goal for the inheritance of table rho of both mixins is to produce a table consisting of three rows (and to reuse the mixin for other tables that are also referencing to either alpha or beta):
CREATE TABLE schema2.rho (
id_row INTEGER PRIMARY KEY,
id_alpha INTEGER REFERENCES schema1.alpha(id_alpha),
id_beta INTEGER REFERENCES schema2.beta(id_beta)
);
CREATE INDEX uq_rho_alpha-beta ON schema2.rho(id_alpha, id_beta);
However, when trying to recreate all these tables by calling db.create_all() SQLAlchemy will spew out an error:
sqlalchemy.exc.NoReferencedTableError: Foreign key associated with
column 'rho.id_alpha' could not find table 'schema2.alpha' with
which to generate a foreign key to target column 'id_alpha'
It appears that instead of locating table alpha being in schema1 as specified in the imported mixin, SQLAlchemy seems to be looking for it in schema2.
How can this be solved? Is there a way to pass/force the correct schema for the mixin? Thanks.
I finally found the error - a typo or extra underscore in table beta declaration: instead of the correct __tablename__ I had __table_name__. Without the __tablename__ class member specified, SQLAlchemy will create the table on the server using the class name as the table name (default behavior). In my case, it created table TableBeta instead of intended beta. That caused the foreign key mixin to not find the table.
The approach I took (detailed in my question) is the correct way of using mixins and specifying schema in SQLAlchemy for the use case (tables in different schemas and table class declarations in different model files). Schema name is specified in table declarations using the __table_args__ class member passed with a keyword dictionary {"schema": "schema_name"}. Schema name is qualified in mixin/column declarations in the form "schema_name.table_name.column_name".
Related
I am trying to set up many-to-many relationship in SQLAlchemy but I am getting the error:
from shopapp import db
db.create_all()
sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'shoppinglists_products.shoppinglist_id_v2' could not find table 'shoppinglist' with which to generate a foreign key to target column 'id'
My code:
from sqlalchemy import ForeignKey
from shopapp import db
shoppinglists_products = db.Table("shoppinglists_products",
db.Column("shoppinglist_id", db.Integer, ForeignKey("shoppinglist.id")),
db.Column("product_id", db.Integer, ForeignKey("product.id")))
class ShoppingList(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(20), unique=True, nullable=False)
products = db.relationship('Product', back_populates="shoppinglists", secondary="shoppinglists_products")
class Product(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(20), unique=True, nullable=False)
Where is the problem?
It seems like Flask-SQLAlchemy has problem finding the table for foreign key reference. Based on your code, here are the two ways you can fix this:
1) Fix shoppinglists_products table:
Flask-SQLAlchemy often converts the CamelCased model names into a syntax similar to this: camel_cased. In your case, ShoppingList will be referred to as shopping_list. Therefore, changing the ForeignKey("shoppinglist.id") to ForeignKey("shopping_list.id") will do the trick.
shoppinglists_products = db.Table("shoppinglists_products",
db.Column("shoppinglist_id", db.Integer, ForeignKey("shopping_list.id")), # <-- fixed
2) Change the model names:
If you'd like, you could go ahead and change the model name from ShoppingList to Shopping and later refer to this as shopping. This would prevent any confusion from rendering further. Usually, developers don't quite often go for a class name which is combined of two words, especially for the ORM cases. This is because various frameworks has different ways of interpreting the class names to create tables.
Expanding on #P0intMaN's answer - explicitly providing the SQL Alchemy table name with __tablename__ = "ShoppingList" (for example) lets you use your preferred case style and prevents SQLAlchemy from 'helping' you by changing the name of something kind of important without telling you.
class ShoppingList(db.Model):
__tablename__ = "ShoppingList"
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(20), unique=True, nullable=False)
products = db.relationship('Product', back_populates="shoppinglists", secondary="shoppinglists_products")
In many/most Flask tutorials and books, simplistic table names (e.g. posts, comments, users) are used, which elide this issue. Thus a trap awaits for those of us who insist on meaningful CamelCased class names. This is mentioned somewhat casually in the documentation here: https://flask-sqlalchemy.palletsprojects.com/en/2.x/models/
Some parts that are required in SQLAlchemy are optional in
Flask-SQLAlchemy. For instance the table name is automatically set for
you unless overridden. It’s derived from the class name converted to
lowercase and with “CamelCase” converted to “camel_case”. To override
the table name, set the tablename class attribute.
I have a simple data model of customer and addresses with SQL Alchemy annotations to make the objects persistable in a database. Unfortunately, when I try to create a customer object with c = Customer() I receive an error:
sqlalchemy.exc.AmbiguousForeignKeysError: Could not determine join condition between parent/child
tables on relationship Customer.addresses - there are multiple foreign key paths linking the tables.
Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as
containing a foreign key reference to the parent table.
This is pretty clear -- I need to further annotate the line:
addresses = relationship("Address", back_populates="customer")
with something disambiguate the foreign key relationships. However, I can't understand what I need to (or could) specify in this case. Can anyone point me in the right direction?
Update: Looking further, it seems to me that SQLAlchemy is attempting to infer the direction of the addresses relationship and is unable to do so because there are PK/FK relationships in each direction between these classes. This cannot be resolved by adding foreign_keys= on the addresses relationship because the foreign key for this relationship is in the other table.
I can get this to work by removing the addresses relation entirely from Customer and instead doing customer = relationship("Customer", backref="addresses", foreign_keys=[customer_id]) in the Address class. I don't really like this solution, however, as I want to "express" addresses in the Customer class rather than as the side-effect of creating an otherwise unwanted customer relationship in the Address class.
So, still looking for a way of modifying the addresses relationship to make it work.
Here is my entire model:
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Customer(Base):
__tablename__ = "customer"
id = Column("customer_id", Integer, primary_key=True)
name = Column("name", String, nullable=False)
bill_address_id = Column("bill_id", Integer, ForeignKey("addresses.address_id"))
ship_address_id = Column("ship_id", Integer, ForeignKey("addresses.address_id"))
bill_address = relationship("Address", foreign_keys=[bill_address_id])
ship_address = relationship("Address", foreign_keys=[ship_address_id])
addresses = relationship("Address", back_populates="customer")
class Address(Base):
__tablename__ = "addresses"
id = Column("address_id", Integer, primary_key=True)
address = Column(String, nullable=False)
This question already has answers here:
How to define a table without primary key with SQLAlchemy?
(7 answers)
Closed 3 years ago.
I have a flask application that relies on an existing Teradata Database to serve up information to and accept input from its users. I am able to successfully make the connection between the application and the Teradata Database, however, I am not able to then define classes that will represent tables already existing in my database.
Currently, I am defining a 'Base' class using sqlalchemy that represents the connection to my database. There is no problem here and I am even able to execute queries using the connection used to build the 'Base' class. However, my problem is in using this 'Base' class to create a subclass 'Users' for my teradata table 'users'. My understanding is that sqlalchemy should allow for me to define a subclass of the superclass 'Base' which will inherit the metadata from the underlying teradata table that the subclass represents - in this case, my 'users' table. Here is the code I have so far:
import getpass
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.schema import MetaData
user = 'user_id_string'
pasw=getpass.getpass()
host = 'host_string'
db_name = 'db_name'
engine = create_engine(f'{host}?user={user}&password={pasw}&logmech=LDAP')
connection = engine.connect()
connection.execute(f'DATABASE {db_name}')
md = MetaData(bind=connection, reflect=False, schema='db_name')
md.reflect(only=['users'])
Base = declarative_base(bind=connection, metadata=md)
class Users(Base):
__table__ = md.tables['db_name.users']
This is the error that I receive when constructing the subclass 'Users':
sqlalchemy.exc.ArgumentError: Mapper mapped class Users->users could not assemble any primary key columns for mapped table 'users'
Is there some reason that my subclass 'Users' is not automatically being mapped to the table metadata from the existing teradata table 'users' that I have assigned it to in defining the class? The underlying table already has a primary key set so I don't understand why sqlalchemy is not assuming the existing primary key. Thanks for your help in advance.
EDIT: The underlying table DOES NOT have a primary KEY, only a primary INDEX.
From SQLAlchmey documentation: (https://docs.sqlalchemy.org/en/13/faq/ormconfiguration.html#how-do-i-map-a-table-that-has-no-primary-key)
The SQLAlchemy ORM, in order to map to a particular table, needs there to be at least one column denoted as a primary key column; multiple-column, i.e. composite, primary keys are of course entirely feasible as well. These columns do not need to be actually known to the database as primary key columns, though it’s a good idea that they are. It’s only necessary that the columns behave as a primary key does, e.g. as a unique and not nullable identifier for a row.
Most ORMs require that objects have some kind of primary key defined because the object in memory must correspond to a uniquely identifiable row in the database table; at the very least, this allows the object can be targeted for UPDATE and DELETE statements which will affect only that object’s row and no other. However, the importance of the primary key goes far beyond that. In SQLAlchemy, all ORM-mapped objects are at all times linked uniquely within a Session to their specific database row using a pattern called the identity map, a pattern that’s central to the unit of work system employed by SQLAlchemy, and is also key to the most common (and not-so-common) patterns of ORM usage.
In almost all cases, a table does have a so-called candidate key, which is a column or series of columns that uniquely identify a row. If a table truly doesn’t have this, and has actual fully duplicate rows, the table is not corresponding to first normal form and cannot be mapped. Otherwise, whatever columns comprise the best candidate key can be applied directly to the mapper:
class SomeClass(Base):
__table__ = some_table_with_no_pk
__mapper_args__ = {
'primary_key':[some_table_with_no_pk.c.uid, some_table_with_no_pk.c.bar]
}
Better yet is when using fully declared table metadata, use the primary_key=True flag on those columns:
class SomeClass(Base):
__tablename__ = "some_table_with_no_pk"
uid = Column(Integer, primary_key=True)
bar = Column(String, primary_key=True)
I am during creating my first database project in SQLAlchemy and SQLite. I want to connect two entity as relational database's relational model. Here is the source:
class Models(Base):
__tablename__ = "models"
id_model = Column(Integer, primary_key=True)
name_of_model = Column(String, nullable = False)
price = Column(Integer, nullable = False)
def __init__(self, name_of_model):
self.name_of_model = name_of_model
class Cars(Base):
__tablename__ = "cars"
id_car = Column(Integer, primary_key=True)
id_equipment = Column(Integer, nullable = False)
id_package = Column(Integer, nullable = False)
id_model = Column(Integer, ForeignKey('Models'))
model = relationship("Models", backref=backref('cars', order_by = id_model))
I want to achieve a relationship like this:
https://imgur.com/af62zli
The error which occurs is:
The foreign key associated with column 'cars.id_model' could not find table 'Models' with which to generate a foreign key to target column 'None'.
Any ideas how to solve this problem?
From the docs:
The argument to ForeignKey is most commonly a string of the form
<tablename>.<columnname>, or for a table in a remote schema or “owner”
of the form <schemaname>.<tablename>.<columnname>. It may also be an
actual Column object...
In defining your ForeignKey on Cars.id_model you pass the string form of a class name ('Models') which is not an accepted form.
However, you can successfully define your foreign key using one of the below options:
ForeignKey(Models.id_model)
This uses the actual Column object to specify the foreign key. The disadvantage of this method is that you need to have the column in your namespace adding extra complexity in needing to import the model into a module if it is not defined there, and also may cause you to care about the order of instantiation of your models. This is why it's more common to use one of the string-based options, such as:
ForeignKey('models.id_model')
Notice that this example doesn't include the string version of the class name (not Models.id_model) but rather the string version of the table name. The string version means that table objects required are only resolved when needed and as such avoid the complexities of dealing with Column objects themselves.
Another interesting example that works in this case:
ForeignKey('models')
If the two columns are named the same on both tables, SQLAlchemy seems to infer the column from the table. If you alter the name of either of the id_model columns definitions in your example so that they are named differently, this will cease to work. Also I haven't found this to be well documented and it is less explicit, so not sure if really worth using and am really just mentioning for completeness and because I found it interesting. A comment in the source code of ForeignKey._column_tokens() seemed to be more explicit than the docs with respect to acceptable formatting of the column arg:
# A FK between column 'bar' and table 'foo' can be
# specified as 'foo', 'foo.bar', 'dbo.foo.bar',
# 'otherdb.dbo.foo.bar'. Once we have the column name and
# the table name, treat everything else as the schema
# name.
I'm trying to create a table of profiles where profiles, tests, and folders can be children of profiles, but I'm getting an AmbiguousForeignKeysError when I call create_all(db). My idea for a single table, which then references itself via a mapper or association table seems like it should work, but I haven't been able to get sqlalchemy to cooperate. I used examples for a many-to-many relationship on the sqlalchemy website to create my code. Here's my snippet:
from sqlalchemy import Column, Integer, String, Enum, Table, ForeignKey
from sqlalchemy.orm import relationship,validates
from core.coyote.models import Base #declarative base
association_table = Table('association', Base.metadata,
Column('left_id', Integer, ForeignKey('profile_member.id')),
Column('right_id', Integer, ForeignKey('profile_member.id'))
)
class ProfileMember(Base):
__tablename__ = "profile_member"
id = Column(Integer, primary_key=True)
name = Column(String(250),nullable=True)
type = Column(Enum("Test","Profile","Folder"))
children = relationship("ProfileMember",secondary=association_table)
I also plan to put in some validation code so that I can enforce some parentage rules. Here is the complete error I'm getting:
sqlalchemy.exc.AmbiguousForeignKeysError: Could not determine join condition between parent/child tables on relationship ProfileMember.children - there are multiple foreign key paths linking the tables via secondary table 'association'. Specify the 'foreign_keys' argument, providing a list of those columns which should be counted as containing a foreign key reference from the secondary table to each of the parent and child tables
Any clues as to things I can try would be greatly appreciated.
You have to specify how to join to the secondary table:
children = relationship("ProfileMember", secondary=association_table,
primaryjoin=id == association_table.c.left_id,
secondaryjoin=association_table.c.right_id == id)