I'm new to SQLAlchemy and am using Flask-SQLAlchemy for my current project. I'm getting an error that has me stumped:
sqlalchemy.orm.exc.UnmappedInstanceError: Class 'flask_sqlalchemy._BoundDeclarativeMeta' is not mapped; was a class (app.thing.Thing) supplied where an instance was required?
I've tried changing the notation in the instatiation, and moving some things around. Could the problem be related to this question, or something else?
Here is my module (thing.py) with the class that inherits from Flask-SQLAlchemy's db.Model (like SQLAlchemy's Base class):
from app import db
class Thing(db.Model):
indiv_id = db.Column(db.INTEGER, primary_key = True)
stuff_1 = db.Column(db.INTEGER)
stuff_2 = db.Column(db.INTEGER)
some_stuff = db.Column(db.BLOB)
def __init__():
pass
And here is my call to populate the database in the thing_wrangler.py module:
import thing
from app import db
def populate_database(number_to_add):
for each_thing in range(number_to_add):
a_thing = thing.Thing
a_thing.stuff_1 = 1
a_thing.stuff_2 = 2
a_thing.some_stuff = [1,2,3,4,5]
db.session.add(a_thing)
db.session.commit()
You need to create an instance of your model. Instead of a_thing = thing.Thing, it should be a_thing = thing.Thing(). Notice the parentheses. Since you overrode __init__, you also need to fix it so it takes self as the first argument.
Related
The app has such logic: list of people stored in the database, each man has a rating calculated in realtime and this value is never stored in database. And I want to use one class to work with dababase fields: name, age etc. and non database field: rating.
Is it possible in sqlalchemy? Now I'm using inheritance Man -> ManMapping:
class Man:
rating = None
def get_rating(self):
return self.rating
...
class ManMapping(Base, Man):
__tablename__ = 'man'
id = Column('man_id', Integer, primary_key=True)
name = Column(Unicode)
...
It works but it looks terrible for me. Is it right approach or I have to do something else?
This is the correct solution: https://docs.sqlalchemy.org/en/13/orm/constructors.html
Hybrid properties are somewhat less flexible that this. The accepted answer is not an actual answer to the problem.
The SQLAlchemy ORM does not call init when recreating objects from database rows. The ORM’s process is somewhat akin to the Python standard library’s pickle module, invoking the low level new method and then quietly restoring attributes directly on the instance rather than calling init.
If you need to do some setup on database-loaded instances before they’re ready to use, there is an event hook known as InstanceEvents.load() which can achieve this; it is also available via a class-specific decorator called reconstructor(). When using reconstructor(), the mapper will invoke the decorated method with no arguments every time it loads or reconstructs an instance of the class. This is useful for recreating transient properties that are normally assigned in init:
from sqlalchemy import orm
class MyMappedClass(object):
def __init__(self, data):
self.data = data
# we need stuff on all instances, but not in the database.
self.stuff = []
#orm.reconstructor
def init_on_load(self):
self.stuff = []
If you are using any data from the DB to calculate rating I would recommend looking at hybrid property. Otherwise I would add self.rating to init and have your function inside the ManMapping class. Something like:
class ManMapping(Base):
__tablename__ = 'man'
id = Column('man_id', Integer, primary_key=True)
name = Column(Unicode)
def __init__(self)
self.rating = None
def get_rating(self):
return self.rating
In my point of view, you should have two distincts classes.
One for the logic in your code and one to communicate with your DB.
class Man(object):
"""This class is for your application"""
def __init__(self, name, rating):
# If the identifier is only used by the DB it should not be in this class
self.name = name
self.rating = rating
class ManModel(Base):
"""This model is only to communicate with the DB"""
__tablename__ = 'man'
id = Column('man_id', Integer, primary_key=True)
name = Column(Unicode)
You should have a provider that does queries to DB with ManModel objects, then maps results to Man objects and return your mapped data to the caller.
Your application will only use Man objects and your provider will do the mapping.
Something like below :
class DbProvider(object):
def get_man(self, id):
man_model = session.query(ManModel).filter(ManModel.id == id).one_or_none()
return self.man_mapper(man_model) if man_model else None
def get_men(self):
men_model = session.query(ManModel).all()
return [self.man_mapper(man_model) for man_model in men_model]
def man_mapper(self, man_model):
return Man(man_model.name, self.calculate_rating(man_model))
class Test(object):
def display_man(self):
man = db_provider.get_man(15)
if man:
print man.name, man.rating
I have an existing, working Flask app that uses SQLAlchemy. Several of the models/tables in this app have columns that store raw HTML, and I'd like to inject a function on a column's setter so that the incoming raw html gets 'cleansed'. I want to do this in the model so I don't have to sprinkle "clean this data" all through the form or route code.
I can currently already do this like so:
from application import db, clean_the_data
from sqlalchemy.ext.hybrid import hybrid_property
class Example(db.Model):
__tablename__ = 'example'
normal_column = db.Column(db.Integer,
primary_key=True,
autoincrement=True)
_html_column = db.Column('html_column', db.Text,
nullable=False)
#hybrid_property
def html_column(self):
return self._html_column
#html_column.setter
def html_column(self, value):
self._html_column = clean_the_data(value)
This works like a charm - except for the model definition the _html_column name is never seen, the cleaner function is called, and the cleaned data is used. Hooray.
I could of course stop there and just eat the ugly handling of the columns, but why do that when you can mess with metaclasses?
Note: the following all assumes that 'application' is the main Flask module, and that it contains two children: 'db' - the SQLAlchemy handle and 'clean_the_data', the function to clean up the incoming HTML.
So, I went about trying to make a new base Model class that spotted a column that needs cleaning when the class is being created, and juggled things around automatically, so that instead of the above code, you could do something like this:
from application import db
class Example(db.Model):
__tablename__ = 'example'
__html_columns__ = ['html_column'] # Our oh-so-subtle hint
normal_column = db.Column(db.Integer,
primary_key=True,
autoincrement=True)
html_column = db.Column(db.Text,
nullable=False)
Of course, the combination of trickery with metaclasses going on behind the scenes with SQLAlchemy and Flask made this less than straight-forward (and is also why the nearly matching question "Custom metaclass to create hybrid properties in SQLAlchemy" doesn't quite help - Flask gets in the way too). I've almost gotten there with the following in application/models/__init__.py:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.ext.hybrid import hybrid_property
# Yes, I'm importing _X stuff...I tried other ways to avoid this
# but to no avail
from flask_sqlalchemy import (Model as BaseModel,
_BoundDeclarativeMeta,
_QueryProperty)
from application import db, clean_the_data
class _HTMLBoundDeclarativeMeta(_BoundDeclarativeMeta):
def __new__(cls, name, bases, d):
# Move any fields named in __html_columns__ to a
# _field/field pair with a hybrid_property
if '__html_columns__' in d:
for field in d['__html_columns__']:
if field not in d:
continue
hidden = '_' + field
fget = lambda self: getattr(self, hidden)
fset = lambda self, value: setattr(self, hidden,
clean_the_data(value))
d[hidden] = d[field] # clobber...
d[hidden].name = field # So we don't have to explicitly
# name the column. Should probably
# force a quote on the name too
d[field] = hybrid_property(fget, fset)
del d['__html_columns__'] # Not needed any more
return _BoundDeclarativeMeta.__new__(cls, name, bases, d)
# The following copied from how flask_sqlalchemy creates it's Model
Model = declarative_base(cls=BaseModel, name='Model',
metaclass=_HTMLBoundDeclarativeMeta)
Model.query = _QueryProperty(db)
# Need to replace the original Model in flask_sqlalchemy, otherwise it
# uses the old one, while you use the new one, and tables aren't
# shared between them
db.Model = Model
Once that's set, your model class can look like:
from application import db
from application.models import Model
class Example(Model): # Or db.Model really, since it's been replaced
__tablename__ = 'example'
__html_columns__ = ['html_column'] # Our oh-so-subtle hint
normal_column = db.Column(db.Integer,
primary_key=True,
autoincrement=True)
html_column = db.Column(db.Text,
nullable=False)
This almost works, in that there's no errors, data is read and saved correctly, etc. Except the setter for the hybrid_property is never called. The getter is (I've confirmed with print statements in both), but the setter is ignored totally and the cleaner function is thus never called. The data is set though - changes are made quite happily with the un-cleaned data.
Obviously I've not quite completely emulated the static version of the code in my dynamic version, but I honestly have no idea where the issue is. As far as I can see, the hybrid_property should be registering the setter just like it has the getter, but it's just not. In the static version, the setter is registered and used just fine.
Any ideas on how to get that final step working?
Maybe use a custom type ?
from sqlalchemy import TypeDecorator, Text
class CleanedHtml(TypeDecorator):
impl = Text
def process_bind_param(self, value, dialect):
return clean_the_data(value)
Then you can just write your models this way:
class Example(db.Model):
__tablename__ = 'example'
normal_column = db.Column(db.Integer, primary_key=True, autoincrement=True)
html_column = db.Column(CleanedHtml)
More explanations are available in the documentation here: http://docs.sqlalchemy.org/en/latest/core/custom_types.html#augmenting-existing-types
We use a central class model for a wide variety of python modules. This model is defined using SQLAlchemy. The classes all inherit from declarative_base.
For example, our model definitions look something like this:
Base = declarative_base()
class Post(Base):
__tablename__ = 'Posts'
id = Column(INT, primary_key=True, autoincrement=True)
body = Column(TEXT)
timestamp = Column(TIMESTAMP)
user_id = Column(INT, ForeignKey('Users.uid'))
We have been building a flask web-application in which we employ this same model. We have discovered a tricky problem in that flask-sqlalchemy appears to be designed in such a way that it expects all classes used in its model to have been defined by passing in an active instance of the session. Here is an example of a "proper" flask-sqalchemy class model definition:
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
email = db.Column(db.String(120), unique=True)
Note that the above example for flask-sqlalchemy requires an already-instantiated sql session. This has horrified us, because we are completely at a loss as to how to integrate our SqlAlchemy model into flask. We really want to use the flask-security suite in particular.
This problem has been brought up before on SO. Here, for example:
How to use flask-sqlalchemy with existing sqlalchemy model?
Our requirements are different from those of whoever accepted the response there. The response points out that one loses the ability to use User.query, but this is precisely one of the things we must retain.
It is not feasible to abandon our nice, elegant, central class model definition in favor of what flask-sqlalchemy appears to require. Is there any way for us to associate our model with the SQLAlchemy() object? Bonus points for getting us the .query() method on our classes which appears to be required by flask-security.
Solution:
As of today, the best way to do this is as follows:
Implement or import sqlalchemy base
from sqlalchemy.ext.declarative import declarative_base
base = declarative_base()
class Base(base):
__abstract__ = True
uid = Column(Integer, primary_key=True, autoincrement=True)
Register the external base:
from flask_sqlalchemy import SQLAlchemy
from model.base import Base
app = Flask(__name__)
db = SQLAlchemy(app, model_class=Base)
Archived for posterity:
I spent a lot of time looking for an answer to this. This is a lot easier to do today than it was when I originally asked the question, but it still isn't exactly simple.
For anyone who decides to do security themselves, I recommend the following excellent exposition of common design patterns which use flask, but which avoid employing unnecessary dependencies like flask-security:
https://exploreflask.com/users.html
UPDATE:
For anyone interested, a patch has been in the works for some time related to this. As of now it still isn't released, but you can check its progress here:
https://github.com/mitsuhiko/flask-sqlalchemy/pull/250#issuecomment-77504337
UPDATE:
I have taken the code from the above mentioned patch and created a local override for the SQLAlchemy object which allows one to register an external base. I think this is the best option available until such time as FSA gets around to adding this officially. Here is the code from that class for anyone interested. Tested working with Flask-SqlAlchemy 2.2
Patching in register_external_base:
import flask_sqlalchemy
'''Created by Isaac Martin 2017. Licensed insofar as it can be according to the standard terms of the MIT license: https://en.wikipedia.org/wiki/MIT_License. The author accepts no liability for consequences resulting from the use of this software. '''
class SQLAlchemy(flask_sqlalchemy.SQLAlchemy):
def __init__(self, app=None, use_native_unicode=True, session_options=None,
metadata=None, query_class=flask_sqlalchemy.BaseQuery, model_class=flask_sqlalchemy.Model):
self.use_native_unicode = use_native_unicode
self.Query = query_class
self.session = self.create_scoped_session(session_options)
self.Model = self.make_declarative_base(model_class, metadata)
self._engine_lock = flask_sqlalchemy.Lock()
self.app = app
flask_sqlalchemy._include_sqlalchemy(self, query_class)
self.external_bases = []
if app is not None:
self.init_app(app)
def get_tables_for_bind(self, bind=None):
"""Returns a list of all tables relevant for a bind."""
result = []
for Base in self.bases:
for table in flask_sqlalchemy.itervalues(Base.metadata.tables):
if table.info.get('bind_key') == bind:
result.append(table)
return result
def get_binds(self, app=None):
"""Returns a dictionary with a table->engine mapping.
This is suitable for use of sessionmaker(binds=db.get_binds(app)).
"""
app = self.get_app(app)
binds = [None] + list(app.config.get('SQLALCHEMY_BINDS') or ())
retval = {}
for bind in binds:
engine = self.get_engine(app, bind)
tables = self.get_tables_for_bind(bind)
retval.update(dict((table, engine) for table in tables))
return retval
#property
def bases(self):
return [self.Model] + self.external_bases
def register_base(self, Base):
"""Register an external raw SQLAlchemy declarative base.
Allows usage of the base with our session management and
adds convenience query property using self.Query by default."""
self.external_bases.append(Base)
for c in Base._decl_class_registry.values():
if isinstance(c, type):
if not hasattr(c, 'query') and not hasattr(c, 'query_class'):
c.query_class = self.Query
if not hasattr(c, 'query'):
c.query = flask_sqlalchemy._QueryProperty(self)
# for name in dir(c):
# attr = getattr(c, name)
# if type(attr) == orm.attributes.InstrumentedAttribute:
# if hasattr(attr.prop, 'query_class'):
# attr.prop.query_class = self.Query
# if hasattr(c , 'rel_dynamic'):
# c.rel_dynamic.prop.query_class = self.Query
To be used like so:
app = Flask(__name__)
db = SQLAlchemy(app)
db.register_base(base)
I have model declared as:
class SAProduct(Base):
sku = Column(PRODUCT_SKU_TYPE, primary_key=True)
i_want_to_hide = Column(String(20), nullable=False)
name = Column(Unicode(255), nullable=True)
#property
def my_property(self):
return i_calculate_property_here(self)
and Spyne model declared as:
db = create_engine('sqlite:///:memory:')
Session = sessionmaker(bind=db)
class TableModel(ComplexModelBase):
__metaclass__ = ComplexModelMeta
__metadata__ = MetaData(bind=db)
class SProduct(TableModel):
__table__ = SAProduct.__table__
How can I make attribute i_want_to_hide to be excluded from Spyne model, and property my_property to be included as Spyne model attribute?
P.S.
Now I use monkey patching Spyne to support this syntax:
class SProduct(GComplexModel):
__model__ = Product
class Attributes:
exclude_attrs = ('i_want_to_hide',)
add_attrs = {'my_property': Boolean}
But I want get rid of it.
This doesn't directly answer your question, but please consider the following code:
from spyne import *
TableModel = TTableModel()
class SomeClass(TableModel):
__tablename__ = 'some_table'
id = Integer(pk=True)
s = Unicode
i = Integer(exc_table=True)
Here, pk stands for primary key (you can use the long form primary_key if you wish) and the i attribute will just be ignored by SqlAlchemy. e.g. it won't be created in the table, it won't be instrumented by SqlAlchemy's metaclass, etc.
As for an attribute that will be hidden from the RPC Parts of Spyne, but not from SqlAlchemy, that's a new feature coming in 2.12.
You will be able to say e.g.:
i = Integer(exc_table=True, pa={JsonObject: dict(exc=true)})
where pa stands for protocol attributes. (you can use the long form prot_attrs if you wish) Here i is ignored by every protocol that inherits JsonObject.
If you don't want it on the wsdl either, you'll have to do:
i = Integer(exc_table=True, exc_interface=True)
https://github.com/arskom/spyne/blob/fa4b1eef5815d3584287d1fef66b61846f82d2f8/spyne/interface/xml_schema/model.py#L197
Spyne offers a richer object model interface compared to SqlAlchemy. Trying to replicate this functionality without adding Spyne as a dependency means you'll have to duplicate all the work done in Spyne in your project. It's your choice!
Is there a way to create custom methods to the query object so you can do something like this?
User.query.all_active()
Where all_active() is essentially .filter(User.is_active == True)
And be able to filter off of it?
User.query.all_active().filter(User.age == 30)
You can subclass the base Query class to add your own methods:
from sqlalchemy.orm import Query
class MyQuery(Query):
def all_active(self):
return self.filter(User.is_active == True)
You then tell SQLAlchemy to use this new query class when you create the session (docs here). From your code it looks like you might be using Flask-SQLAlchemy, so you would do it as follows:
db = SQLAlchemy(session_options={'query_cls': MyQuery})
Otherwise you would pass the argument directly to the sessionmaker:
sessionmaker(bind=engine, query_cls=MyQuery)
As of right now, this new query object isn't that interesting because we hardcoded the User class in the method, so it won't work for anything else. A better implementation would use the query's underlying class to determine which filter to apply. This is slightly tricky but can be done as well:
class MyOtherQuery(Query):
def _get_models(self):
"""Returns the query's underlying model classes."""
if hasattr(query, 'attr'):
# we are dealing with a subquery
return [query.attr.target_mapper]
else:
return [
d['expr'].class_
for d in query.column_descriptions
if isinstance(d['expr'], Mapper)
]
def all_active(self):
model_class = self._get_models()[0]
return self.filter(model_class.is_active == True)
Finally, this new query class won't be used by dynamic relationships (if you have any). To let those also use it, you can pass it as argument when you create the relationship:
users = relationship(..., query_class=MyOtherQuery)
this work for me finely
from sqlalchemy.orm import query
from flask_sqlalchemy import BaseQuery
class ParentQuery(BaseQuery):
def _get_models(self):
if hasattr(query, 'attr'):
return [query.attr.target_mapper]
else:
return self._mapper_zero().class_
def FilterByCustomer(self):
model_class = self._get_models()
return self.filter(model_class.customerId == int(g.customer.get('customerId')))
#using like this
class AccountWorkflowModel(db.Model):
query_class = ParentQuery
.................
To provide a custom method that will be used by all your models that inherit from a particular parent, first as mentioned before inherit from the Query class:
from flask_sqlalchemy import SQLAlchemy, BaseQuery
from sqlalchemy.inspection import inspect
class MyCustomQuery(BaseQuery):
def all_active(self):
# get the class
modelClass = self._mapper_zero().class_
# get the primary key column
ins = inspect(modelClass)
# get a list of passing objects
passingObjs = []
for modelObj in self:
if modelObj.is_active == True:
# add to passing object list
passingObjs.append(modelObj.__dict__[ins.primary_key[0].name])
# change to tuple
passingObjs = tuple(passingObjs)
# run a filter on the query object
return self.filter(ins.primary_key[0].in_(passingObjs))
# add this to the constructor for your DB object
myDB = SQLAlchemy(query_class=MyCustomQuery)
This is for flask-sqlalchemy, for which people will still get here when looking for this answer.