This is a weird bug I've stumbled upon, and I am not sure why is it happening, whether it's a bug in SQLAlchemy, in Flask-SQLAlchemy, or any feature of Python I'm not yet aware of.
We are using Flask 0.11.1, with Flask-SQLAlchemy 2.1 using a PostgreSQL as DBMS.
Examples use the following code to update data from the database:
entry = Entry.query.get(1)
entry.name = 'New name'
db.session.commit()
This works totally fine when executing from the Flask shell, so the database is correctly configured. Now, our controller for updating entries, slightly simplified (without validation and other boilerplate), looks like this:
def details(id):
entry = Entry.query.get(id)
if entry:
if request.method == 'POST':
form = request.form
entry.name = form['name']
db.session.commit()
flash('Updated successfully.')
return render_template('/entry/details.html', entry=entry)
else:
flash('Entry not found.')
return redirect(url_for('entry_list'))
# In the application the URLs are built dynamically, hence why this instead of #app.route
app.add_url_rule('/entry/details/<int:id>', 'entry_details', details, methods=['GET', 'POST'])
When I submit the form in details.html, I can see perfectly fine the changes, meaning the form has been submitted properly, is valid and that the model object has been updated. However, when I reload the page, the changes are gone, as if it had been rolled back by the DBMS.
I have enabled app.config['SQLALCHEMY_ECHO'] = True and I can see a "ROLLBACK" before my own manual commit.
If I change the line:
entry = Entry.query.get(id)
To:
entry = db.session.query(Entry).get(id)
As explained in https://stackoverflow.com/a/21806294/4454028, it does work as expected, so my guess what there was some kind of error in Flask-SQLAlchemy's Model.query implementation.
However, as I prefer the first construction, I did a quick modification to Flask-SQLAlchemy, and redefined the query #property from the original:
class _QueryProperty(object):
def __init__(self, sa):
self.sa = sa
def __get__(self, obj, type):
try:
mapper = orm.class_mapper(type)
if mapper:
return type.query_class(mapper, session=self.sa.session())
except UnmappedClassError:
return None
To:
class _QueryProperty(object):
def __init__(self, sa):
self.sa = sa
def __get__(self, obj, type):
return self.sa.session.query(type)
Where sa is the Flask-SQLAlchemy object (ie db in the controller).
Now, this is where things got weird: it still doesn't save the changes. Code is exactly the same, yet the DBMS is still rolling back my changes.
I read that Flask-SQLAlchemy can execute a commit on teardown, and tried adding this:
app.config['SQLALCHEMY_COMMIT_ON_TEARDOWN'] = True
Suddenly, everything works. Question is: why?
Isn't teardown supposed to happen only when the view has finished rendering? Why is the modified Entry.query behaving different to db.session.query(Entry), even if the code is the same?
Below is the correct way to make changes to a model instance and commit them to the database:
# get an instance of the 'Entry' model
entry = Entry.query.get(1)
# change the attribute of the instance; here the 'name' attribute is changed
entry.name = 'New name'
# now, commit your changes to the database; this will flush all changes
# in the current session to the database
db.session.commit()
Note: Don't use SQLALCHEMY_COMMIT_ON_TEARDOWN, as it's considered harmful and also removed from docs. See the changelog for version 2.0.
Edit: If you have two objects of normal session (created using sessionmaker()) instead of scoped session , then on calling db.session.add(entry) above code will raise error sqlalchemy.exc.InvalidRequestError: Object '' is already attached to session '2' (this is '3'). For more understanding about sqlalchemy session, read below section
Major Difference between Scoped Session vs. Normal Session
The session object we mostly constructed from the sessionmaker() call and used to communicate with our database is a normal session. If you call sessionmaker() a second time, you will get a new session object whose states are independent of the previous session. For example, suppose we have two session objects constructed in the following way:
from sqlalchemy import Column, String, Integer, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String)
from sqlalchemy import create_engine
engine = create_engine('sqlite:///')
from sqlalchemy.orm import sessionmaker
session = sessionmaker()
session.configure(bind=engine)
Base.metadata.create_all(engine)
# Construct the first session object
s1 = session()
# Construct the second session object
s2 = session()
Then, we won't be able to add the same User object to both s1 and s2 at the same time. In other words, an object can only be attached at most one unique session object.
>>> jessica = User(name='Jessica')
>>> s1.add(jessica)
>>> s2.add(jessica)
Traceback (most recent call last):
......
sqlalchemy.exc.InvalidRequestError: Object '' is already attached to session '2' (this is '3')
If the session objects are retrieved from a scoped_session object, however, then we don't have such an issue since the scoped_session object maintains a registry for the same session object.
>>> session_factory = sessionmaker(bind=engine)
>>> session = scoped_session(session_factory)
>>> s1 = session()
>>> s2 = session()
>>> jessica = User(name='Jessica')
>>> s1.add(jessica)
>>> s2.add(jessica)
>>> s1 is s2
True
>>> s1.commit()
>>> s2.query(User).filter(User.name == 'Jessica').one()
Notice thats1 and s2 are the same session object since they are both retrieved from a scoped_session object who maintains a reference to the same session object.
Tips
So, try to avoid creating more than one normal session object. Create one object of the session and use it everywhere from declaring models to querying.
Our project is separated in several files to ease mantenaince. One is routes.py with the controllers, and another one is models.py, which contains the SQLAlchemy instance and models.
So, while I was removing boilerplate to get a minimal working Flask project to upload it to a git repository to link it here, I found the cause.
Apparently, the reason is that my workmate, while attempting to insert data using queries instead of the model objects (no, I have no idea why on earth he wanted to do that, but he spent a whole day coding it), had defined another SQLAlchemy instance in the routes.py.
So, when I was trying to insert data from the Flask shell using:
from .models import *
entry = Entry.query.get(1)
entry.name = 'modified'
db.session.commit()
I was using the correct db object, as defined in models.py, and it was working completely fine.
However, as in routes.py there was another db defined after the model import, this one was overwriting the reference to the correct SQLAlchemy instance, so I was commiting with a different session.
Related
Original question
I have set up a soft deletion on SQLAlchemy 1.4 based on this example in the official doc here. The
_add_filtering_criteria filters out the soft-deleted objects whenever the execute_state.execution_options "include_deleted" is False.
At some point, I would like that some queries are able to search in the soft-deleted objects. I am able to do it per queries, i.e. specifying query.execution_options(include_deleted=True) but I would like to make all queries of a particular session include soft-deleted objects without having to specify it for each query.
I have tried to declare the execution_options(include_deleted=True) on the engine's creation but it does not work.
from sqlalchemy import orm
class SoftDeleted:
deletion_date = Column(DateTime, nullable=True)
#event.listens_for(orm.Session, "do_orm_execute")
def _add_filtering_criteria(execute_state: orm.ORMExecuteState) -> None:
"""Intercepts all ORM queries. Adds a with_loader_criteria option to all
of them.
This option applies to SELECT queries and adds a global WHERE criteria
(or as appropriate ON CLAUSE criteria for join targets)
to all objects of a certain class or superclass.
"""
if (not execute_state.is_column_load
and not execute_state.is_relationship_load
and not execute_state.execution_options.get("include_deleted", False)):
execute_state.statement = execute_state.statement.options(
orm.with_loader_criteria(
SoftDeleted,
lambda cls: cls.deletion_date.is_(None),
include_aliases=True,
)
)
engine = sa.create_engine(url, echo=False).execution_options(include_deleted=True)
session_factory = orm.sessionmaker(bind=engine)
session = session_factory()
# let SomeClass be a class that inherits from the SoftDeleted mixin
myquery = session.query(SomeClass).get(1)
# will not retrieve SomeClass with uid=1 if it is soft-deleted
myquery2 = session.query(SomeClass).execution_options(include_deleted=True).get(1)
# will retrieve SomeClass with uid=1 even if it is soft-deleted
As I said, I would like that all queries of the session are able to include soft-deleted objects. Would someone know how I can do?
Solution, thanks to snakecharmerb's answer
After snakecharmerb answers, I modified the following and I got the wanted behaviour
#event.listens_for(orm.Session, "do_orm_execute")
def _add_filtering_criteria(execute_state):
if (not execute_state.is_column_load
and not execute_state.is_relationship_load
and not execute_state.execution_options.get("include_deleted", False)
and not execute_state.session.info.get("include_deleted", False)):
[...]
engine = sa.create_engine(url, echo=False) \
.execution_options(include_deleted=include_deleted)
session_factory = orm.sessionmaker(
bind=engine,
info={'include_deleted':
engine.get_execution_options().get('include_deleted', False)})
session = session_factory()
[...]
You have set your flag in the engine's execution options, so you must retrieve it from there. The engine can be accessed through the session's bind attribute:
(Pdb) execute_state.session.bind.get_execution_options()
immutabledict({'include_deleted': True})
Your code does not error because execute_state has an execution_options attribute, but it contains other things:
(Pdb) execute_state.execution_options
immutabledict({'_sa_orm_load_options': <class 'sqlalchemy.orm.context.QueryContext.default_load_options'>, '_result_disable_adapt_to_context': True, 'future_result': True})
Session objects (and sessionsmakers) have an info attribute, a dictionary that you can populate as you wish. This could used to pass the flag if you wanted to set the flag per session rather than per engine.
I'm attempting to modularize my flask app by splitting models, the db, routers, etc into separate files. This has been giving me some trouble, because it seems that my tables are not being created as I expect.
I have this file called flask_postgres_server.py
from . import flask_server
from flask_sqlalchemy import SQLAlchemy
from flask import jsonify, request
from . import models
Tree = models.tree.Tree_postgres.Tree
app = flask_server.app # this simply exports an instance of a flask app, so I can use it in another server with mongo
#app.route("/trees", methods=['POST'])
def create_tree_postgres():
label = request.get_json['label']
tree = Tree(label=label)
tree.save()
return jsonify(tree.get_dict())
I am importing a Tree model:
from ... import postgres_db
db = postgres_db.db
class Tree(db.Model):
__tablename__ = 'trees'
id = db.Column(db.Integer, primary_key=True)
label = db.Column(db.String(), nullable=False)
def save(self):
db.session.add(self)
db.session.commit()
db.session.expunge_all()
db.session.close()
def get_dict(self):
return {"label": self.label, "_id": self.id}
which in turn imports the db:
from . import flask_server
app = flask_server.app
import os
from dotenv import load_dotenv, find_dotenv
from flask_sqlalchemy import SQLAlchemy
"""
Imported by a server in order to set up a postgres connection for use by a Flask server
"""
load_dotenv(find_dotenv())
DB_NAME = os.environ.get("DB_NAME")
POSTGRES_HOST = os.environ.get("POSTGRES_HOST")
POSTGRES_PORT = os.environ.get("POSTGRES_PORT")
POSTGRES_USER = os.environ.get("POSTGRES_USER")
POSTGRES_PASSWORD = os.environ.get("POSTGRES_PASSWORD")
DB_URI = f'postgresql://{POSTGRES_USER}:{POSTGRES_PASSWORD}#{POSTGRES_HOST}:{POSTGRES_PORT}/{DB_NAME}'
app.config['SQLALCHEMY_DATABASE_URI'] = DB_URI
db = SQLAlchemy(app)
db.create_all()
When I run my server and get post data at /trees, I get this error message:
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "trees" does not exist
LINE 1: INSERT INTO trees (label) VALUES ('countries') RETURNING tre...
^
[SQL: INSERT INTO trees (label) VALUES (%(label)s) RETURNING trees.id]
[parameters: {'label': 'countries'}]
Shouldn't this relation have been created automatically?
I can go into psql and create this table manually. Then I get another error saying:
DetachedInstanceError: Instance <Tree at 0x10d2facd0> is not bound to a Session
Is there something wrong with the way I've structured my code?
The trees table does not get created because db.create_all() is called before the Tree model is imported.
In the code in the question, you can fix this by moving db.create_all() to after the definition of the Tree model. In your overall structure, db.create_all() will need to be called after all the model definitions have been executed (usually by importing the files that contain them).
Note that it's not a good idea to close the session in the save method - you won't be able to access the Tree instance afterwards when calling its get_dict method. I'd suggest leaving session life-cycle management to flask-sqlalchemy, Tree.save should just add and commit.
Finally label = request.get_json['label'] in the route handler should be label = request.get_json()['label'].
DetachedInstanceError: Instance <Tree at 0x10d2facd0> is not bound to a Session implicitly give users clue that the object 0x10d2facd0 that you've created is already "disconnected" with the session.
You've already closed the session with db.session.close() but then tried to access the "closed" object afterward.
It is better to save the get_dict result before you close it. It can be rewritten like this:
#app.route("/trees", methods=['POST'])
def create_tree_postgres():
label = request.get_json['label']
tree = Tree(label=label)
response = tree.get_dict()
tree.save()
return jsonify(response)
I'm using Flask, Flask-SQLAlchemy, Flask-Marshmallow + marshmallow-sqlalchemy, trying to implement REST api PUT method. I haven't found any tutorial using SQLA and Marshmallow implementing update.
Here is the code:
class NodeSchema(ma.Schema):
# ...
class NodeAPI(MethodView):
decorators = [login_required, ]
model = Node
def get_queryset(self):
if g.user.is_admin:
return self.model.query
return self.model.query.filter(self.model.owner == g.user)
def put(self, node_id):
json_data = request.get_json()
if not json_data:
return jsonify({'message': 'Invalid request'}), 400
# Here is part which I can't make it work for me
data, errors = node_schema.load(json_data)
if errors:
return jsonify(errors), 422
queryset = self.get_queryset()
node = queryset.filter(Node.id == node_id).first_or_404()
# Here I need some way to update this object
node.update(data) #=> raises AttributeError: 'Node' object has no attribute 'update'
# Also tried:
# node = queryset.filter(Node.id == node_id)
# node.update(data) <-- It doesn't if know there is any object
# Wrote testcase, when user1 tries to modify node of user2. Node doesn't change (OK), but user1 gets status code 200 (NOT OK).
db.session.commit()
return jsonify(), 200
UPDATED, 2022-12-08
Extending the ModelSchema from marshmallow-sqlalchemy instead of Flask-Marshmallow you can use the load method, which is defined like this:
load(data, *, session=None, instance=None, transient=False, **kwargs)
Putting that to use, it should look like that (or similar query):
node_schema.load(json_data, session= current_app.session, instance=Node().query.get(node_id))
And if you want to load without all required fields of Model, you can add the partial=True argument, like this:
node_schema.load(json_data, instance=Node().query.get(node_id), partial=True)
See the docs for more info (does not include definition of ModelSchema.load).
See the code for the load definition.
I wrestled with this issue for some time, and in consequence came back again and again to this post. In the end what made my situation difficult was that there was a confounding issue involving SQLAlchemy sessions. I figure this is common enough to Flask, Flask-SQLAlchemy, SQLAlchemy, and Marshmallow, to put down a discussion. I certainly, do not claim to be an expert on this, and yet I believe what I state below is essentially correct.
The db.session is, in fact, closely tied to the process of updating the DB with Marshmallow, and because of that decided to to give the details, but first the short of it.
Short Answer
Here is the answer I arrived at for updating the database using Marshmallow. It is a different approach from the very helpful post of Jair Perrut. I did look at the Marshmallow API and yet was unable to get his solution working in the code presented, because at the time I was experimenting with his solution I was not managing my SQLAlchemy sessions properly. To go a bit further, one might say that I wasn't managing them at all. The model can be updated in the following way:
user_model = user_schema.load(user)
db.session.add(user_model.data)
db.session.commit()
Give the session.add() a model with primary key and it will assume an update, leave the primary key out and a new record is created instead. This isn't all that surprising since MySQL has an ON DUPLICATE KEY UPDATE clause which performs an update if the key is present and creates if not.
Details
SQLAlchemy sessions are handled by Flask-SQLAlchemy during a request to the application. At the beginning of the request the session is opened, and when the request is closed that session is also closed. Flask provides hooks for setting up and tearing down the application where code for managing sessions and connections may be found. In the end, though, the SQLAlchemy session is managed by the developer, and Flask-SQLAlchemy just helps. Here is a particular case that illustrates the management of sessions.
Consider a function that gets a user dictionary as an argument and uses that with Marshmallow() to load the dictionary into a model. In this case, what is required is not the creation of a new object, but the update of an existing object. There are 2 things to keep in mind at the start:
The model classes are defined in a python module separate from any code, and these models require the session. Often the developer (Flask documentation) will put a line db = SQLAlchemy() at the head of this file to meet this requirement. This in fact, creates a session for the model.
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
In some other separate file there may be a need for a SQLAlchemy session as well. For example, the code may need to update the model, or create a new entry, by calling a function there. Here is where one might find db.session.add(user_model) and db.session.commit(). This session is created in the same way as in the bullet point above.
There are 2 SQLAlchemy sessions created. The model sits in one (SignallingSession) and the module uses its own (scoped_session). In fact, there are 3. The Marshmallow UserSchema has sqla_session = db.session: a session is attached to it. This then is the third, and the details are found in the code below:
from marshmallow_sqlalchemy import ModelSchema
from donate_api.models.donation import UserModel
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class UserSchema(ModelSchema):
class Meta(object):
model = UserModel
strict = True
sqla_session = db.session
def some_function(user):
user_schema = UserSchema()
user['customer_id'] = '654321'
user_model = user_schema.load(user)
# Debug code:
user_model_query = UserModel.query.filter_by(id=3255161).first()
print db.session.object_session(user_model_query)
print db.session.object_session(user_model.data)
print db.session
db.session.add(user_model.data)
db.session.commit()
return
At the head of this module the model is imported, which creates its session, and then the module will create its own. Of course, as pointed out there is also the Marshmallow session. This is entirely acceptable to some degree because SQLAlchemy allows the developer to manage the sessions. Consider what happens when some_function(user) is called where user['id'] is assigned some value that exists in the database.
Since the user includes a valid primary key then db.session.add(user_model.data) knows that it is not creating a new row, but updating an existing one. This behavior should not be surprising, and is to be at least somewhat expected since from the MySQL documentation:
13.2.5.2 INSERT ... ON DUPLICATE KEY UPDATE Syntax
If you specify an ON DUPLICATE KEY UPDATE clause and a row to be inserted would cause a duplicate value in a UNIQUE index or PRIMARY KEY, an UPDATE of the old row occurs.
The snippet of code is then seen to be updating the customer_id on the dictionary for the user with primary key 32155161. The new customer_id is '654321'. The dictionary is loaded with Marshmallow and a commit done to the database. Examining the database it can be found that it was indeed updated. You might try two ways of verifying this:
In the code: db.session.query(UserModel).filter_by(id=325516).first()
In MySQL: select * from user
If you were to consider the following:
In the code: UserModel.query.filter_by(id=3255161).customer_id
You would find that the query brings back None. The model is not synchronized with the database. I have failed to manage our SQLAlchemy sessions correctly. In an attempt to bring clarity to this consider the output of the print statements when separate imports are made:
<sqlalchemy.orm.session.SignallingSession object at 0x7f81b9107b90>
<sqlalchemy.orm.session.SignallingSession object at 0x7f81b90a6150>
<sqlalchemy.orm.scoping.scoped_session object at 0x7f81b95eac50>
In this case the UserModel.query session is different from the Marshmallow session. The Marshmallow session is what gets loaded and added. This means that querying the model will not show our changes. In fact, if we do:
db.session.object_session(user_model.data).commit()
The model query will now bring back the updated customer_id! Consider the second alternative where the imports are done through flask_essentials:
from flask_sqlalchemy import SQLAlchemy
from flask_marshmallow import Marshmallow
db = SQLAlchemy()
ma = Marshmallow()
<sqlalchemy.orm.session.SignallingSession object at 0x7f00fe227910>
<sqlalchemy.orm.session.SignallingSession object at 0x7f00fe227910>
<sqlalchemy.orm.scoping.scoped_session object at 0x7f00fed38710>
And the UserModel.query session is now the same as the user_model.data (Marshmallow) session. Now the UserModel.query does reflect the change in the database: the Marshmallow and UserModel.query sessions are the same.
A note: the signalling session is the default session that Flask-SQLAlchemy uses. It extends the default session system with bind selection and modification tracking.
I have rolled out own solution. Hope it helps someone else. Solution implements update method on Node model.
Solution:
class Node(db.Model):
# ...
def update(self, **kwargs):
# py2 & py3 compatibility do:
# from six import iteritems
# for key, value in six.iteritems(kwargs):
for key, value in kwargs.items():
setattr(self, key, value)
class NodeAPI(MethodView):
decorators = [login_required, ]
model = Node
def get_queryset(self):
if g.user.is_admin:
return self.model.query
return self.model.query.filter(self.model.owner == g.user)
def put(self, node_id):
json_data = request.get_json()
if not json_data:
abort(400)
data, errors = node_schema.load(json_data) # validate with marshmallow
if errors:
return jsonify(errors), 422
queryset = self.get_queryset()
node = queryset.filter(self.model.id == node_id).first_or_404()
node.update(**data)
db.session.commit()
return jsonify(message='Successfuly updated'), 200
Latest Update [2020]:
You might facing the issue of mapping keys to the database models. Your request body have only updated fields so, you want to change only those without affecting others. There is an option to write multiple if conditions but that's not a good approach.
Solution
You can implement patch or put methods using sqlalchemy library only.
For example:
YourModelName.query.filter_by(
your_model_column_id = 12 #change 12: where condition to find particular row
).update(request_data)
request_data should be dict object. For ex.
{
"your_model_column_name_1": "Hello",
"your_model_column_name_2": "World",
}
In above case, only two columns will be updated that is: your_model_column_name_1 and your_model_column_name_2
Update function maps request_data to the database models and creates update query for you. Checkout this: https://docs.sqlalchemy.org/en/13/core/dml.html#sqlalchemy.sql.expression.update
Previous answer seems to be outdated as ModelSchema is now deprecated.
You should instead SQLAlchemyAutoSchema with the proper options.
class NodeSchema(SQLAlchemyAutoSchema):
class Meta:
model = Node
load_instance = True
sqla_session = db.session
node_schema = NodeSchema()
# then when you need to update a Node orm instance :
node_schema.load(node_data, instance=node, partial=True)
db.session.update()
Below is my solution with Flask-Marshmallow + marshmallow-sqlalchemy bundle as the author requested initially.
schemas.py
from flask import current_app
from flask_marshmallow import Marshmallow
from app.models import Node
ma = Marshmallow(current_app)
class NodeSchema(ma.SQLAlchemyAutoSchema):
class Meta:
model = Node
load_instance = True
load_instance is a key point here to make an update further.
routes.py
from flask import jsonify, request
from marshmallow import ValidationError
from app import db
#bp.route("/node/<node_uuid>/edit", methods=["POST"])
def edit_node(node_uuid):
json_data = request.get_json(force=True, silent=True)
node = Node.query.filter_by(
node_uuid=node_uuid
).first()
if node:
try:
schema = NodeSchema()
json_data["node_uuid"] = node_uuid
node = schema.load(json_data, instance=node)
db.session.commit()
return schema.jsonify(node)
except ValidationError as err:
return jsonify(err.messages), 422
else:
return jsonify("Not found"), 404
You have to check for existence of Node first, otherwise the new instance will be created.
First time on the site, so hi to all and thanks in advance. Longtime lurker and newb.
I'm working on a web app in flask, using Flask-SqlAlchemy and SqlAlchemy-Searchable (docs-> https://sqlalchemy-searchable.readthedocs.org/en/latest/index.html). For a reason I can't figure out, when I try a similar example to the code shown on the docs page:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy, BaseQuery
from sqlalchemy_searchable import SearchQueryMixin
from sqlalchemy_utils.types import TSVectorType
from sqlalchemy_searchable import make_searchable
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql+psycopg2://usr:admin#localhost/dev'
app.config['SECRET_KEY'] = 'notreallyasecret'
db = SQLAlchemy(app)
make_searchable()
class ArticleQuery(BaseQuery, SearchQueryMixin):
pass
class Article(db.Model):
query_class = ArticleQuery
__tablename__ = 'article'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.Unicode(255))
content = db.Column(db.UnicodeText)
search_vector = db.Column(TSVectorType('name', 'content'))
My search queries don't work properly. I opened a python shell and created the db, and inserted five identical articles
a= Article(name='finland',content='finland')
db.session.add(a)
db.session.commit() #a-e
with 'finland' both as name and content. According to the example:
Article.query.search(u'finland').limit(5).all()
There should be articles returned that have finland somewhere in them. In my case, I get an empty list. I get an object back if I modify the example query to:
Article.query.search(' ').first()
But it's rather useless searching for empty spaces. Any ideas?
Adding a bit more to it: I noticed in the article table, the 'search_vector tsvector' column is completely empty despite data being in the content and name columns; I'm not sure if that has anything to do with it.
I ran into this exact issue once, too, when using Flask-Script to add a manage.py management tool to my application.
The fact that the search_vector column is empty despite you having added the appropriate TSVectorType parameters means that the SQLAlchemy-Searchable trigger isn't present in the postgres DB. You can verify its absence by doing a \df+ in psql command line tool -- you will not see a trigger named article_search_vector_update. SQLAlchemy-Searchable sets up this trigger to update the content of the search_vector column when the columns named in TSVectorType(...) change.
In the case of manage.py, I had to first call:
db.configure_mappers()
Essentially, you have to configure SQLAlchemy's mappers before calling create_all(). Without doing this, SQLAlchemy-Searchable will not be given the opportunity to add its search_vector trigger to populate the TSVectorType column in the model.The SQLAlchemy-Searchable docs have more on this.
In total, a minimal manage.py that properly configures SQLAlchemy-Searchable as you require might look like:
#!/usr/bin/env python
from flask.ext.script import Manager
from app import app, db
manager = Manager(app)
#manager.command
def init_db():
"""
Drops and re-creates the SQL schema
"""
db.drop_all()
db.configure_mappers()
db.create_all()
db.session.commit()
On Collin Allen's answer: actually, the flask-sqlalchemy ''db'' exposes the configure_mappers function.
Replace:
from sqlalchemy.orm.mapper import configure_mappers
...
configure_mappers()
with:
...
db.configure_mappers()
I have a python module UserManager that takes care for all things user management related - users, groups, rights, authentication. Access to these assets is provided via master class that is passed SQLAlchemy engine parameter at constructor. The engine is needed to make the table-class mappings (using mapper objects), and to emit sessions.
This is how the gobal variables are established in the app module:
class UserManager:
def __init__(self, db):
self.db = db
self._db_session = None
meta = MetaData(db)
user_table = Table(
'USR_User', meta,
Column('field1'),
Column('field3')
)
mapper(User, user_table)
#property
def db_session(self):
if self._db_session is None:
self._db_session = scoped_session(sessionmaker())
self._db_session.configure(bind=self.db)
return self._db_session
class User(object):
def init(self, um):
self.um = um
from flask.ext.sqlalchemy import SQLAlchemy
db = SQLAlchemy(app)
um = UserManager(db.engine)
This module as such is designed to be context-agnostic by purpose, so that it can be used both for locally run and web application.
But here the problems arise: time to time I get the dreaded "Can't reconnect until invalid transaction is rolled back" error, presumably caused by some failed transaction in the UserManager code.
I am now trying to identify the problem source. Maybe it is not right way how to handle the database in the dynamic context of web server? Perhaps I have to pass the db.session to the um object so that I can be sure that the db connections are not mixed up?
In web context you should consider the request for every user isolated. For this you must use the flask.g
To share data that is valid for one request only from one function to
another, a global variable is not good enough because it would break
in threaded environments.Flask provides you with a special object
that ensures it is only valid for the active request and that will
return different values for each request. In a nutshell: it does the
right thing, like it does for request and session.
You can see more about here.