I have a simple data structure, where a film table has a foreign key to country table.
In order to retrieve all the films that are from the same country, I have this property 'same_country_films', a self-referential relationship.
It almost does the job correctly, however, it also includes the film itself in the list. How can I exclude it and just have other films?
Many thanks!
from sqlalchemy import Table, Column, Integer, String, MetaData, ForeignKey
from sqlalchemy.orm import mapper, relationship
metadata = MetaData()
country_table = Table('country', metadata,
Column('id', String, primary_key=True),
Column('name', String),
)
film_table = Table('film', metadata,
Column('id', Integer, primary_key=True),
Column('title', String),
Column('year', Integer),
Column('country_id', Integer, ForeignKey('country.id'))
)
class Country(object):
pass
class Film(object):
pass
mapper(Country, country_table)
mapper(Film, film_table,
properties={
'country':relationship(
Country,
backref='films'),
'same_country_films':relationship(
Film,
primaryjoin=film_table.c.country_id==\
film_table.c.country_id,
foreign_keys=[
film_table.c.country_id,
]
)
}
)
The simplest solution is to code this property yourself instead of relation:
class Film(object):
#property
def same_country_films(self):
return [f for f in self.country.films if f!=self]
This solution won't do separate query for this property when both film.same_country_films and country.films are accessed during session life. The property can't be updated as you can usualy do with relation, but I doubt it's realy needed.
The bad thing is that it's evaluated for each access (not so much work). You can change property decorator to chaching one (like cached_property in werkzeug), but then the property won't reflect changes in country.films after first access to it.
This should do it I think, (though I haven't actually tested it):
primaryjoin=sqlalchemy.and_(
film_table.c.country_id==film_table.c.country_id,
film_table.c.id!=film_table.c.id)
Related
In my postgres server we have a database database with 2 schemas: public and api.
public has several tables, and I need to create a table in api with a foreign key to a table in public called model.
So it's:
-Schemas
--public
---tables
----models
--api
---tables
Using SQLAlchemy I have the following class:
from sqlalchemy import create_engine, MetaData, Table, Column
class __PostgresService:
def __init__(self):
self.__client = create_engine("postgresql://postgres#localhost:5432/database")
metadata = MetaData(self.__client, schema="public")
self.__table = Table("training", metadata,
Column("id", String, primary_key=True, nullable=False),
Column("model_id", ForeignKey("model.id"), nullable=False),
schema="api")
metadata.create_all()
postgres_service = __PostgresService()
However upon launch I receive the following error:
sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'training.model_id' could not find table 'public.model' with which to generate a foreign key to target column 'id'
It seems it does look for the correct thing but can't find it? I'm very confused as to why this is happening, especially because the error refers to not finding "public", which is created by default by postgres, rather than "api" which I created myself in pgAdmin.
Am I missing some cruicial config?
The error you are getting means that you are trying to create a foreign key referencing a table that SQLAlchemy does not know about. You can tell sqlalchemy about it by creating a Table associated with the same MetaData describing the referenced table. You can also do this using sqlalchemy's reflection capabilities. For example:
from sqlalchemy import create_engine, MetaData, Table, Column
class __PostgresService:
def __init__(self):
self.__client = create_engine("postgresql://postgres#localhost:5432/database")
metadata = MetaData(self.__client, schema="public")
metadata.reflect(schema="public", only=["model"])
self.__table = Table("training", metadata,
Column("id", String, primary_key=True, nullable=False),
Column("model_id", ForeignKey("model.id"), nullable=False),
schema="api")
metadata.create_all()
postgres_service = __PostgresService()
By default, MetaData.create_all() will check for the existence of tables first, before creating them, but you can also specify the exact tables to create: metadata.create_all(tables=[self.__table])
I have a table that already exists:
USERS_TABLE = Table("users", META_DATA,
Column("id", Integer, Sequence("user_id_seq"), primary_key=True),
Column("first_name", String(255)),
Column("last_name", String(255))
)
I created this table by running this:
CONN = create_engine(DB_URL, client_encoding="UTF-8")
META_DATA = MetaData(bind=CONN, reflect=True)
# ... table code
META_DATA.create_all(CONN, checkfirst=True)
the first time it worked and I was able to create the table. However, the 2nd time around I got this error:
sqlalchemy.exc.InvalidRequestError: Table 'users' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object.
which makes sense since the table users already exists. I'm able to see if the table exists like so:
TABLE_EXISTS = CONN.dialect.has_table(CONN, "users")
However, how do I actually get the existing table object? I can't find this anywhere in the documentation. Please help.
We have 3 different approaches here:
assume that required tables have been created already, reflecting them and getting with MetaData.tables dictionary field like
from sqlalchemy import MetaData, create_engine
CONN = create_engine(DB_URL, client_encoding="UTF-8")
META_DATA = MetaData(bind=CONN, reflect=True)
USERS_TABLE = META_DATA.tables['users']
removing reflect flag from MetaData object initialization, because we don't use it and moreover – trying to create tables that've been already reflected:
from sqlalchemy import (MetaData, Table, Column, Integer, String, Sequence,
create_engine)
CONN = create_engine('sqlite:///db.sql')
META_DATA = MetaData(bind=CONN)
USERS_TABLE = Table("users", META_DATA,
Column("id", Integer, Sequence("user_id_seq"),
primary_key=True),
Column("first_name", String(255)),
Column("last_name", String(255)))
META_DATA.create_all(CONN, checkfirst=True)
assuming that we are keeping reflected table if it was previously created by setting in Table object initializer keep_existing flag to True:
from sqlalchemy import (MetaData, Table, Column, Integer, String, Sequence,
create_engine)
CONN = create_engine('sqlite:///db.sql')
META_DATA = MetaData(bind=CONN, reflect=True)
USERS_TABLE = Table("users", META_DATA,
Column("id", Integer, Sequence("user_id_seq"),
primary_key=True),
Column("first_name", String(255)),
Column("last_name", String(255)),
keep_existing=True)
META_DATA.create_all(CONN, checkfirst=True)
Which one to choose? Depends on your use case, but I prefer second one since it looks like you aren't using reflection, also it is simplest modification: just removing flag from MetaData initializer.
P. S.
we can always make reflection after initialization of MetaData object with MetaData.reflect method:
META_DATA.reflect()
also we can specify which tables to reflect with only parameter (may be any iterable of str objects):
META_DATA.reflect(only=['users'])
and many more.
This works for me pretty well -
import sqlalchemy as db
engine = db.create_engine("your_connection_string")
meta_data = db.MetaData(bind=engine)
db.MetaData.reflect(meta_data)
USERS = meta_data.tables['users']
# View the columns present in the users table
print(USERS.columns)
# You can run sqlalchemy queries
query = db.select([
USERS.c.id,
USERS.c.first_name,
USERS.c.last_name,
])
result = engine.execute(query).fetchall()
Note that using reflect parameter in Metadata(bind=engine, reflect=True) is deprecated and will be removed in a future release. Above code takes care of it.
__table_args__ = {'extend_existing': True}
right below __tablename__
If you're using async Sqlalchemy, you can use
metadata = MetaData()
async with engine.connect() as conn:
await conn.run_sync(metadata.reflect, only=["harshit_table"])
harshit_table = Table("harshit_table", metadata, autoload_with=engine)
print("tables: ", harshit_table, type(harshit_table))
I'm quite new to this, but what worked for me was this (variables are declared in the original question)
USERS_TABLE_NEW = Table("users", META_DATA, autoload_with=CONN)
I have a class Contract to represent my contracts:
.../mypackage/Contract.py
class Contract:
# setter and getters.
def isValid( self, contract_number=None ):
#code
def cancelTheContract( self, contract_number=None ):
# code
And my SQLAlchemy Contract class:
.../mypackage/orm.py
from sqlalchemy import create_engine
from sqlalchemy import MetaData
from sqlalchemy import Column, ForeignKey, Integer, String, Table
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
db = create_engine( 'mysql://myuser:mypasswd#localhost/mydatabase' )
contracts = Table( 'contracts', MetaData( bind = None ) )
class Connection:
def connect( self ):
Session = sessionmaker( bind = db )
session = Session()
return session
class Contract( Base ):
__tablename__ = 'contracts'
id = Column( Integer, primary_key = True )
type = Column( String )
price = Column( Float )
So...
Would be ok to merge both Contract classes in a single one?
If not, I have to instantiate a class specific for the database table and another class specific for the business logic, so when I have to deal with database data, manipulate it and put it back, I have to deal with two objects that are basically the thing.
Well... I guess I'm missing some important concept here.
What should I read to understand better about my question implications?
Thanks!
Gio
Yes, the philosophy of ORMs is to map physical tables to business entity objects, so it is best practice to combine your two classes. SQLA attributes manage the persistent fields of your entity and you can encapsulate all the business logic in that class per standard object-oriented modeling techniques.
Considering Storm, a python ORM, I would like to automatically generate the schema for a (mysql) database. The home page states
"Storm works well with existing database schemas." ( https://storm.canonical.com/FrontPage ),
hence I was hoping to not having to create model classes. However, the 'getting started' tutorial ( https://storm.canonical.com/Tutorial ) suggests that a class, like the one below, needs to be created manually for each table, and each field needs to be manually specified:
class Person(object):
__storm_table__ = "person"
id = Int(primary=True)
name = Unicode()
Alternatively, SQLAlchemy doesn't seem to support a reverse engineering feature either, but does need a schema like this one:
user = Table('user', metadata,
Column('user_id', Integer, primary_key = True),
Column('user_name', String(16), nullable = False),
Column('email_address', String(60)),
Column('password', String(20), nullable = False)
)
Of course, these classes/schemas make sense, since each table will likely represent some 'object of interest' and one can extend them with all kinds of functionality. However, they are tedious to create, and their (initial) content is straight forward when a database already exists.
One ORM that does allow for reverse engineering is:
http://docs.doctrine-project.org/en/2.0.x/reference/tools.html
Are there similar reverse engineering tools for Storm or SQLAlchemy or any python ORM or python database fancyfier?
I am not aware of how Storm manages this process, but you can certainly reflect tables in a database with sqlalchemy. For example, below is a basic example using a SQL Server instance that I have access to at the moment.
AN ENTIRE DATABASE
>>> from sqlalchemy import create_engine, MetaData
>>> engine = create_engine('mssql+pyodbc://<username>:<password>#<host>/<database>') # replace <username> with user name etc.
>>> meta = MetaData()
>>> meta.reflect(bind=engine)
>>> funds_table = meta.tables['funds'] # tables are stored in meta.tables dict
>>> funds_table # now stores database schema object
Table(u'funds', MetaData(bind=None), Column(u'fund_token', INTEGER(), table=<funds>, primary_key=True, nullable=False), Column(u'award_year_token', INTEGER(), ForeignKey(u'award_year_defn.award_year_token'), table=<funds>, nullable=False), ... Column(u'fin_aid_disclosure_category', VARCHAR(length=3, collation=u'SQL_Latin1_General_CP1_CI_AS'), table=<funds>), Column(u'report_as_additional_unsub', BIT(), table=<funds>, server_default=DefaultClause(<sqlalchemy.sql.expression.TextClause object at 0x000000000545B6D8>, for_update=False)), schema=None)
If you merely want to reflect one table at a time, you can use the following code instead.
ONE TABLE AT A TIME (much faster)
>>> from sqlalchemy import Table, create_engine, MetaData
>>> engine = create_engine('mssql+pyodbc://<username>:<password>#<host>/<database>')
>>> meta = MetaData()
>>> funds_table = Table('funds', meta, autoload=True, autoload_with=engine) # indicate table name (here 'funds') with a string passed to Table as the first argument
>>> funds_table # now stores database schema object
Table(u'funds', MetaData(bind=None), Column(u'fund_token', INTEGER(), table=<funds>, primary_key=True, nullable=False), Column(u'award_year_token', INTEGER(), ForeignKey(u'award_year_defn.award_year_token'), table=<funds>, nullable=False), ... Column(u'fin_aid_disclosure_category', VARCHAR(length=3, collation=u'SQL_Latin1_General_CP1_CI_AS'), table=<funds>), Column(u'report_as_additional_unsub', BIT(), table=<funds>, server_default=DefaultClause(<sqlalchemy.sql.expression.TextClause object at 0x000000000545B6D8>, for_update=False)), schema=None)
As you can probably imagine, you can then save the relevant tables' data for accessing the tables more quickly again in the future.
I've got a sqlalchemy model that is set up like this:
class Entry(Base):
__tablename__ = 'entries'
__table__ = Table('entries', Base.metadata,
Column('id', Integer, primary_key=True, unique=True),
Column('user_id', Integer, ForeignKey('users.id', onupdate="CASCADE", ondelete="RESTRICT")),
Column('title', String(128)),
Column('slug', String(128), index=True),
Column('url', String(256), index=True),
Column('entry', Text),
Column('cached_entry', Text),
Column('created', DateTime, server_default=text('current_timestamp')),
Column('modified', DateTime, server_onupdate=text('current_timestamp')),
Column('pubdate', DateTime),
)
What I would like is that when I update entry that cached_entry gets re-generated, cached_entry is the markdown parsed version of entry. Basically I am caching the output of the markdown parsing so that I don't have to do it on each showing of the entry. I've ttried using #hybrid_method however that didn't seem to work as that is not stored in the database at all. I've got it working on Google AppEngine, but I can't seem to figure out how to do the same thing using SQLAlchemy.
I really would prefer not to have to add a function to the class that is used instead of the names in the model because it is harder to enforce it from an application standpoint, I don't want to accidentally miss something.
#hybrid_descriptor certainly does it using the form described at http://www.sqlalchemy.org/docs/orm/mapper_config.html#using-descriptors . You assign to the database-mapped attribute, which you can map under a different name - since you're using the __table__, you can use a form like:
class Entry(Base):
__table__ = ...
_entry = __table__.c.entry
#hybrid_property
def entry(self):
return self._entry
#entry.setter
def entry(self, value):
self._entry = value
self.cached_entry = markdown(value)
Another is to use the before_insert and before_update events to populate the column at flush time - this is a simple approach but has the disadvantage that you have to wait for a flush() for it to happen.
I think the quickest way for an "on-set" is to use #validates:
from sqlalchemy.orm import validates
class Entry(Base):
__table__ = ...
#validates('entry')
def _set_entry(self, key, value):
self.cached_entry = markdown(value)
return value