I'm creating a REST-like API to get data in and out of a legacy database. At the moment I don't have the option to alter the db structure. Nearly every table in the database has the exact same structure. Here's how I'm currently handling it using Flask-SQLAlchemy:
class MyDatasetBase(object):
timestamp = db.Column('colname', db.Float, primary_key=True)
val1 = db.Column(db.Float)
val2 = db.Column(db.Float)
val3 = db.Column(db.Float)
fkeyid = db.Column(db.Integer)
#declared_attr
def foreigntable(self):
return db.relationship('ForeignTableClass', uselist=False)
class Table1(MyDatasetBase, db.Model):
__tablename__ = 'table1'
class Table2(MyDatasetBase, db.Model):
__tablename__ = 'table2'
class ForeignTableClass(db.Model):
__tablename__ = 'ForeignTable'
id = db.Column(db.Integer, db.ForeignKey('table1.fkeyid'), db.ForeignKey('table2.fkeyid'), primary_key=True)
name = db.Column(db.String)
This all works, but some of these datasets contain a lot of tables and I feel like there has to be a more efficient way to do a few of these things.
Is there a way to get around explicitly defining a class for each of the tables derived from MyDatasetBase? If there is, will that cause problems with the foreign key stuff, where in ForeignTableClass I have to define id as being a foreign key for every table listed above?
one way to do that is to use type method :
names = ['table1', 'table2', 'table3']
for name in names:
type(name.title(), (MyDatasetBase, db.Model), { '__tablename__' : name })
# generate <class 'flask_sqlalchemy.Table1'> ...
There is maybe a more pythonic/elegant way to do that with the Metedata and Table.tometadata() method.
Related
I have a problem with SQL Alchemy, while trying to think about an SQL schema I encountered the following problem.
My schema is based on 2 classes, Flight and Trip.
A Trip includes 2 fields: flights_to and flights_from.
Any of the fields is basically a list of flights, it could be made of one flight, or many flights (Connection flights).
class Trip(Base):
__tablename__ = "Trip"
__table_args__ = {'sqlite_autoincrement': True}
id = Column(Integer, primary_key = True)
flights_to = relationship("Flight", backref="Trip")
flights_from = relationship("Flight", backref="Trip")
class Flight(Base):
__tablename__ = "Flight"
__table_args__ = {'sqlite_autoincrement': True}
id = Column(Integer, primary_key = True)
arrival_airport = Column(String(20))
departure_airport = Column(String(20))
flight_number = Column(Integer)
trip_id = Column(Integer, ForeignKey('Trip.id'))
The problem happens when I create 2 fields in the same type:
sqlalchemy.exc.ArgumentError: Error creating backref 'Trip' on relationship 'Trip.flights_from': property of that name exists on mapper 'Mapper|Flight|Flight'
I have thought about using 2 inheriting classes of types FlightTo and FlightFrom and saving them at two different tables, but what if I want to use a FlightFrom as a FlightTo? will the flight be duplicated in 2 tables?
I would appreciate your help.
backref is used to define a new property on the other class you are using relationship with. So you can't have two property which have the same name
You should rename your backref for the flights_from to any other name than Trip.
It will work then.
For Example:
class Person(Model):
id = Column(Integer, primary_key=True)
name = Column(String)
address = relationship("Address",backref="address")
class Address(Model):
id = Column(Integer, primary_key=True)
house_no = Column(Integer)
person_id = Column(Integer, ForeignKey('person.id'))
So you can access the person name with house_no 100 by:
query_address = Address.query.filter_by(house_no=100).first()
person = query_address.address
This returns you the person object.
Thus if you have multiple such names , it will give you an error
I'm working on a SQLAlchemy defining a bunch of mixin classes that applications should be able to import and extend their model.
When looking at the documentation, mixin classes are create knowing the final table name however, in the case of a generic library, the final table name that will be used by the application is not known.
Take the following mixin classes:
import sqlalchemy as sa
class UserMixin(object):
id = sa.Column(sa.Integer(), primary_key=True)
first_name = sa.Column(sa.Unicode(255))
last_name = sa.Column(sa.Unicode(255))
class ItemMixin(object):
id = sa.Column(sa.Integer(), primary_key=True)
name = sa.Column(sa.Unicode(255))
short_description = sa.Column(sa.Unicode(255))
class OrdersMixin(object):
id = sa.Column(sa.Integer(), primary_key=True)
user_id = sa.Column(sa.Integer(), sa.ForeignKey('???'))
item_id = sa.Column(sa.Integer(), sa.ForeignKey('???'))
Then an application defining its models:
import sqlalchemy as sa
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class MyUser(UserMixin, Base):
__tablename__ = 'myuser'
class MyItem(ItemMixin, Base):
__tablename__ = 'myitem'
total = sa.Column(sa.Integer())
class MyOrders(OrdersMixin, Base):
__tablename__ = 'myorders'
I have two issues with this model:
Except from redefining the relationship columns in the extending models, how can the mixin class build the relationship on its own.
Type of the foreign key is assumed by the mixin class, but the id of the table may come from the application itself or from another mixin class.
Is the model I'm trying to implement correct? What would be the right way to tackle this problem?
A Mixin is a class that copies its data into a table, but for SQL it matters if you're the owner of that data (the table) vs being a reference (the foreign key).
It looks like you're attempting to create Mixins that are both sources of truth and references. Which isn't possible in SQL.
Taking your example one step further and defining OrdersMixin like this will make the issues more obvious I think.
class OrdersMixin(UserMixin, ItemMixin):
id = sa.Column(sa.Integer(), primary_key=True)
For example MyOrders would end up like this once things are resolved.
class MyOrders(Base):
__tablename__ = 'myorders'
# This is from UserMixin
id = sa.Column(sa.Integer(), primary_key=True)
first_name = sa.Column(sa.Unicode(255))
last_name = sa.Column(sa.Unicode(255))
# This is from ItemMixin
id = sa.Column(sa.Integer(), primary_key=True)
name = sa.Column(sa.Unicode(255))
short_description = sa.Column(sa.Unicode(255))
# From OrdersMixin
id = sa.Column(sa.Integer(), primary_key=True) # This is defined last so it overrides all others with the same name.
user_id = sa.Column(sa.Integer(), sa.ForeignKey('???'))
item_id = sa.Column(sa.Integer(), sa.ForeignKey('???'))
With how you have the Mixin defined any table that used that Mixin would have primary_keys for the id column, which would conflict. Additionally you are duplicating every column in the Mixin, which in general you want to avoid in SQL (see Database normal form).
The final result would be something like this. Which is a whole bunch of columns meaning you wouldn't need to refer to any other tables and all of the references id you had were overwritten, meaning you wouldn't be able to join them anyway.
class MyOrders(Base):
__tablename__ = 'myorders'
first_name = sa.Column(sa.Unicode(255))
last_name = sa.Column(sa.Unicode(255))
name = sa.Column(sa.Unicode(255))
short_description = sa.Column(sa.Unicode(255))
id = sa.Column(sa.Integer(), primary_key=True) # This is defined last so it overrides all others with the same name.
user_id = sa.Column(sa.Integer(), sa.ForeignKey('???'))
item_id = sa.Column(sa.Integer(), sa.ForeignKey('???'))
To avoid that I keep my Mixins separate from initial table definition. I.e. I use a Mixin for when I want another table to refer to that table.
The following is close to what I think you were hoping to achieve.
import sqlalchemy as sa
from sqlalchemy import orm
class UserMixin(object):
user_id = sa.Column(sa.Integer(), ForeignKey("myuser.id"), index=True)
user = orm.relationship("MyUser")
class ItemMixin(object):
item_id = sa.Column(sa.Integer(), ForeignKey("myitem.id"), index=True)
item = orm.relationship("MyItem")
class OrdersMixin(UserMixin, ItemMixin):
order_id = sa.Column(sa.Integer(), sa.ForeignKey('myorders.id'))
user_id = sa.Column(sa.Integer(), sa.ForeignKey('myorders.user_id'))
item_id = sa.Column(sa.Integer(), sa.ForeignKey('myorders.item_id'))
Note in the Mixins I gave every column a unique name so that there aren't conflicts and in OrdersMixin even though I'm using UserMixin and ItemMixin I'm overriding the user_id and item_id columns because otherwise anything using the OrdersMixin would have foreign keys pointing to three different tables which would confuse the automatic query builder. But it will still add the user and item relations (and since they are defined as foreign keys to the original tables in MyOrders table I think the relationship will just work).
Then I would change your tables to look like this.
import sqlalchemy as sa
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class MyUser(Base):
__tablename__ = "myuser"
id = sa.Column(sa.Integer(),primary_key=True)
first_name = sa.Column(sa.Unicode(255))
last_name = sa.Column(sa.Unicode(255))
class MyItem(Base):
__tablename__ = "myitem"
id = sa.Column(sa.Integer(),primary_key=True)
name = sa.Column(sa.Unicode(255))
short_description = sa.Column(sa.Unicode(255))
class MyOrders(Base, UserMixin, OrdersMixin):
__tablename__ = "myorders"
id = sa.Column(sa.Integer(),primary_key=True)
The original table definition owns the columns (source of truth) defining them individually and Mixins (of this kind) are good to define references so subsequent references don't need define each of them individually. A Mixin can't be defined to be both a reference and a source of truth. In light of that instead of overriding the column each time like OrdersMixin it's better to just define it once canonically (the table) and once as a reference (the Mixin).
I have following models:
class Details(db.Model):
details_id = db.Column(db.Integer, primary_key=True)
details_main = db.Column(db.String(50))
details_desc = db.Column(db.String(50))
class Data(db.Model):
data_id = db.Column(db.Integer, primary_key=True)
data_date = db.Column(db.Date)
details_main = db.Column(db.String(50))
#property
def details_desc(self):
result = object_session(self).\
scalar(
select([Details.details_desc]).
where(Details.details_main == self.details_main)
)
return result
Now, I would like to run query using filter which depends on defined property. I get empty results (of course proper data is in DB). It doesn't work because, probably, I have to map this property. The question is how to do this? (One limitation: FK are not allowed).
Data.query\
.filter(Data.details_desc == unicode('test'))\
.all()
You can implement this with a regular relationship and an association proxy:
class Data(db.Model):
data_id = db.Column(db.Integer, primary_key=True)
data_date = db.Column(db.Date)
details_main = db.Column(db.String(50))
details = relationship(
Details,
primaryjoin=remote(Details.details_main) == foreign(details_main))
details_desc = association_proxy('details', 'details_desc')
Since there are no foreign keys in the schema, you need to tell SQLAlchemy yourself what the join condition for the relationship should be. This is what the remote() and foreign() annotations do.
With that in place, you can use an association_proxy "across" the relationship to create a property on Data which will work the way you want.
I am creating a Flask application and accessing the MySQL database using Flask-Alchemy.
I have following Class to access a table:
class price_table(db.Model):
id = db.Column(db.Integer, primary_key = True)
trans_id = db.Column(db.Integer)
timestamp = db.Column(db.Integer)
order_type = db.Column(db.String(25))
price = db.Column(db.Numeric(15,8))
quantity = db.Column(db.Numeric(25,8))
def __repr__(self):
return 'id'
For the table 'price_table' this works brilliantly, but problem is I have a few tables with the same columns as 'price_table' from which I only know the name at runtime.
I want to reuse the class above so I thought I could change tablename to the name of the table I need to read, but that does not work, the program keeps reading the 'price-table'
How do I override the tablename at runtime?
You should use: __tablename__ :
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50), unique=True)
email = Column(String(120), unique=True)
http://flask.pocoo.org/docs/0.12/patterns/sqlalchemy/
Based on the comment left by jbub I found the following solution that does the trick just as needed.
from app import db
def ClassFactory(name):
tabledict={'id':db.Column(db.Integer, primary_key = True),
'trans_id':db.Column(db.Integer),
'timestamp':db.Column(db.Integer),
'order_type':db.Column(db.String(25)),
'price':db.Column(db.Numeric(25,8)),
'quantity':db.Column(db.Numeric(25,8)),}
newclass = type(name, (db.Model,), tabledict)
return newclass
You can overwrite price_table.table.name attribute, yet keep in mind that it will affect your price_table model so, unless you want to use it to create a new specialized version of this table in the db and you are not interacting with price_table model - I wouldn't recommend that.
There are two tables that one column of table A is pointing another table B's primary key.
But they are placed in different database, so I cannot configure them with foreign key.
Configuring via relationship() is unavailable, so I implemented property attribute manually.
class User(Base):
__tablename__ = 'users'
id = Column(BigInteger, id_seq, primary=True)
name = Column(Unicode(256))
class Article(Base):
__tablename__ = 'articles'
__bind_key__ = 'another_engine'
# I am using custom session configures bind
# each mappers to multiple database engines via this attribute.
id = Column(BigInteger, id_seq, primary=True)
author_id = Column(BigInteger, nullable=False, index=True)
body = Column(UnicodeText, nullable=False)
#property
def author(self):
_session = object_session(self)
return _session.query(User).get(self.author_id)
#author.setter
def author(self, user):
if not isinstance(user, User):
raise TypeError('user must be a instance of User')
self.author_id = user.id
This code works well for simple operations. But it causes dirty queries making SQLAlchemy's features meaningless.
Code would be simple if it was configured via relationship() (e.g. query.filter(author=me)) got messed up(e.g. query.filter(author_id=me.id)).
Relationship(e.g. join) related features are never able to be used in query building.
Can I use property attribute, at least, in building query criterion(filter()/filter_by())?
you can still use relationship here. If you stick to "lazy loading", it will query for the related item in database B after loading the lead item in database A. You can place a ForeignKey() directive in the Column, even if there isn't a real one in the database. Or you can use primaryjoin directly:
class User(Base):
__tablename__ = 'users'
id = Column(BigInteger, id_seq, primary=True)
name = Column(Unicode(256))
class Article(Base):
__tablename__ = 'articles'
__bind_key__ = 'another_engine'
id = Column(BigInteger, id_seq, primary=True)
author_id = Column(BigInteger, nullable=False, index=True)
body = Column(UnicodeText, nullable=False)
author = relationship("User",
primaryjoin="foreign(Article.author_id) == User.id")