We Keep Coding

Flask_SQLAlchemy, db.create_all() is unable to "see" my tables when imported though a service class

The intent: Refactor my code into MVC (this is just the model/database part), and have the server create the database with tables on first run if the database or tables does not exist.
This works when using a "flat" file with all the classes and functions defined in that file, but after moving out the functions into a service class and the models into their own folder with model classes, the db.create_all() function does not seem to be able to detect the table class correctly any more.
Example structure, (minimum viable problem):
server.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///database.sqlite'
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
db = SQLAlchemy(app)
def main():
# Intentionally moved into the main() function to prevent import loops
from services.users import UserService
users = UserService(db)
db.create_all()
app.run(debug=True)
if __name__ == '__main__':
main()
services\users.py
# Class used to access users data using a session
from models.users import Users
class UserService:
def __init__(self, db):
self.db = db
def get_all(self):
return self.db.session.query(Users).all()
def get(self, uid):
return self.db.session.query(Users).get(uid)
def add(self, json):
user = Users(email=json['email'], password=json['password'])
self.db.session.add(user)
self.db.session.commit()
return user
models\users.py
# The actual model
from server import db
class Users(db.Model):
_id = db.Column("id", db.Integer, primary_key=True)
email = db.Column(db.Text)
password = db.Column(db.Text)
Result: The database is created, but it is just an empty file with no tables inside of it.
I have also tried placing the db.create_all() inside the service class def __init__(self, db) (grasping at straws here), both as a self reference and as an argument reference. Neither have worked.
I am sure it is something obvious I am missing, but I have boiled down my project to just the bare minimum and still fail to see why it is not working - so I have to ask. How can I get the db.create_all() to detect my table classes correctly and actually create the required tables, while using this code structure (or something similar, in case I have misunderstood MVC)?

The problem is that server.py is executed twice
when it's imported in models/users.py
when server.py is called to run the app
Each execution generates a new db instance. The db imported by the model file adds the models to its metadata, the db created when the app is run has empty metadata.
You can confirm this by printing id(db) and db.metadata.tables at the end of models/users.py and just before the call to db.create_all() in the main function.
You need to structure your code so that only one db gets created. For example, you could move the app configuration and creation code into its own module, mkapp.py (feel free to come up with a better name):
mkapp.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///database.sqlite'
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
app.config['SQLALCHEMY_ECHO'] = True
db = SQLAlchemy(app)
And in server.py do
from mkapp import app, db
and in models/users.py do
from mkapp import db
As a bonus, this should also remove the import cycle.
I don't use flask much, so this solution can probably be improved on. For example, having a function create app and db and memoise the results might be better than creating them in top-level module code.

How to do Flask-SQLAlchemy pooling?

This is how I basically create my app and db:
from flask import Flask
from flask_restful import Api
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
api = Api(app)
app.config['SQLALCHEMY_DATABASE_URI'] = mysql+pymysql://user:pass#heroku-hostname
db = SQLAlchemy(app)
#app.before_first_request
def create_tables():
db.create_all()
I do not explicitly handle any connection object myself, only using this db, like:
def save_to_db(self):
db.session.add(self)
db.session.commit()
def find_by_email(email):
return User.query.filter_by(email=email).first()
Pure SQLAlchemy creates some kind of Engine object with pooling parameters while Flask-SQLAlchemy's configuration documentation says there are keys for pooling options, but they are getting deprecated.
This clearly seems as an abstraction built upon this Engine object. Also, it shows these settings are actually getting default values if not specified.
The last config field is the SQLALCHEMY_ENGINE_OPTIONS, and it states indeed: A dictionary of keyword args to send to create_engine().
So how is this happening from here? Are we just basically using this single config field from now to run the original SQLAlchemy Engine functions?
What kind of dictionary values should I provide; do I still get any default values?
Coming from Flask and learning things in a top-down approach I'm a bit confused how things are working at lower layers.

Correct way to register flask admin views with application factory

I am using an application factory to add views to my flask application like so :
(this is not my actual application factory, and has been shortened for the sake of brevity)
def create_app(config_name='default'):
app = Flask(__name__, template_folder="templates", static_folder='static')
admin_instance = Admin(app, name='Admin')
admin_instance.add_view(EntityAdmin(Entity, db.session))
My EntityAdmin class looks like this :
class EntityAdmin(ModelView):
column_filters = [
MyCustomFilter(column=None, name='Custom')
]
My custom filter looks like this :
class MyCustomFilter(BaseSQLAFilter):
def get_options(self, view):
entities = Entity.query.filter(Entity.active == True).all()
return [(entity.id, entity.name) for entity in entities]
The problem is that it seems that the get_options function is called when the app is instantiated, running a select query every time the create_app function gets called.
So if I update my database schema and run the flask db migrate command, I get an error because the new column I added does not exist when the select query is run. The query raises an error because my database schema is not in sync with the actual database.
Can I register my views only when an actual HTTP request is made ? How can I differentiate between a request and a command ?

You have one more problem with this filter: its options are created on the application instantiation so if your list of entities was changed during the application running it would still return the same list of options.
To fix both problems you don't need to postpone views registrations. You need the filter to get the list of options every time it is used.
This SO answer to the question "Resetting generator object in Python" describes a way to reuse a generator (in your case — a database query):
from flask import has_app_context
def get_entities():
# has_app_context is used to prevent database access
# when application is not ready yet
if has_app_context():
for entity in Entity.query.filter(Entity.active.is_(True)):
yield entity.id, entity.name
class ReloadingIterator:
def __init__(self, iterator_factory):
self.iterator_factory = iterator_factory
def __iter__(self):
return self.iterator_factory()
class MyCustomFilter(BaseSQLAFilter):
def get_options(self, view):
# This will return a generator which is
# reloaded every time it is used
return ReloadingIterator(get_entities)
The problem is that the query to the Entity table can be called multiple times during request. So I usually cache the result for a single request using Flask globals:
def get_entities():
if has_app_context():
if not hasattr(g, 'entities'):
query = Entity.query.filter(Entity.active.is_(True))
g.entities = [(entity.id, entity.name) for entity in query]
for entity_id, entity_name in g.entities:
yield entity_id, entity_name

working outside of application context - Flask

def get_db(self,dbfile):
if hasattr(g, 'sqlite_db'): self.close_db(g.sqlite_db)
try:
g.sqlite_db = self.connect_db('{}/{}'.format(app.root_path, dbfile))
except sqlite3.OperationalError as e:
raise e
return g.sqlite_db
Hi this code is located inside DB class, The error I get is
RuntimeError: working outside of application context
the error occurs on this line
g.sqlite_db = self.connect_db('{}/{}'.format(app.root_path, dbfile))
I think the problem is with g, it is imported like that from flask import g
How this error can be fixed?
Thanks.

Maybe you need to call your function inside an application context:
with app.app_context():
# call your method here

When creating your app, use:
app.app_context().push()
for example like this:
from yourapp import create_app
app = create_app()
app.app_context().push()
for further information

From the Flask source code in flask/globals.py:
_app_ctx_err_msg = '''\
Working outside of application context.
This typically means that you attempted to use functionality that needed
to interface with the current application object in a way. To solve
this set up an application context with app.app_context(). See the
documentation for more information.\
'''
Following the documentation, you can see that you need to make flask.current_app point to your application and it currently doesn't.
You're probably calling your DB function before Flask has initialized. My guess is that your app object has not been created yet with the Flask constructor.

Two possible solution
First method
Instead of calling create_all() in your code, call manually in the flask shell which is CLI
Go to your terminal
type flask shell, then
db.create_all()
Second method
As it says in the runtime error message
This typically means that you attempted to use functionality that needed
the current application. To solve this, set up an application context
with app.app_context().
Open the python terminal in your project directory and manually add a context
from project_name import app, db
app.app_context().push()
db.create_all()
Check out this video for better understanding.
YouTube Video

Simple Example To Avoid This Error
Please check out the Purpose of context
#filename = run.py (inside root directory)
from flaskblog import create_app
app = create_app()
if __name__ == "__main__":
app.run(debug=True)
Inside flaskblog folder
filename = __init __.py (inside flaskblog folder)
app = Flask(__name__)
db = SQLAlchemy()
login_manager = LoginManager()
login_manager.login_view = "users.login"
def create_app(config_class=Config):
app = Flask(__name__)
app.config.from_object(Config)
db.init_app(app)
from flaskblog.user.routes import users
app.register_blueprint(users)
return app
filename = config.py (inside flaskblog folder)
class Config:
SECRET_KEY = 'your secret key'
SQLALCHEMY_DATABASE_URI = 'your db uri'
filename = models.py
#login_manager.user_loader
def load_user(user_id):
return User.query.get(int(user_id))
class User(db.Model, UserMixin):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(20), unique=True, nullable=False)
users folder (inside flaskblog)
users folder contain one __init__.py file
Filename = form.py (inside users folder)
class LoginForm(FlaskForm):
# define your field
pass
Filename = routes.py (inside users folder)
users = Blueprint('users',__name__)
#users.route('/login', methods=['GET', 'POST'])
def login():
# do your stuff
pass

Other users have pointed out how to solve the immediate problem, however you might consider modifying how the database connection is created to solve this issue.
Instead of having a method within you DB class instantiate the database connection you could have the connection created in the controller before every request. Then use the teardown_request decorator to close the connection.
Then when within a route you could pass the connection to the DB class as part of instantiating a new DB object.
This would ensure that you never create a database connection unless you need one. And it prevent you from accessing Flask globals out of the app context.
#app.before_request
def before_request():
try:
g.sqlite_db = self.connect_db('{}/{}'.format(app.root_path, dbfile))
except sqlite3.OperationalError as e:
raise e
#app.teardown_request
def teardown_request(e):
if hasattr(g, 'sqlite_db'): self.close_db(g.sqlite_db)
#app.route('/someroute', methods=["GET"]:
def someroute():
db_obj = DB(g.sqlite_db)
.
.
.

Use
pip install flask-sqlalchemy==2.5.1
This might solve the error

To expand on #VadimK's answer. If you want to prevent your code from executing outside of an app_context you can use flask.has_app_context() to see if the code is currently inside an app context:
See also: flask.has_request_context()

This is what fixed it for me. I hope it helps someone else.
if __name__ == "__main__":
with app.app_context():
db.create_all()
app.run(debug=True)

I had the same issue while doing some unit testing.
Adding the following function to my test class solved my issue:
#classmethod
def setUpClass(self):
self.app = create_app("testing")
self.client = self.app.test_client()

When initializing the app I use this block instead :
def create_app():
app = Flask(__name__)
with app.app_context():
init_db()
return app

when creating an app just add the below code.
app = Flask(__name__)
app.app_context().push()

Install this version of flask using
pip install flask-sqlalchemy==2.5.1
then run db.create_all() and it will run.

ERROR:This typically means that you attempted to use functionality that needed
to interface with the current application object in a way. To solve
this set up an application context with app.app_context(). See the
documentation for more information.

Flask-Sqlalchemy + Sqlalchemy-searchable returning empty list

First time on the site, so hi to all and thanks in advance. Longtime lurker and newb.
I'm working on a web app in flask, using Flask-SqlAlchemy and SqlAlchemy-Searchable (docs-> https://sqlalchemy-searchable.readthedocs.org/en/latest/index.html). For a reason I can't figure out, when I try a similar example to the code shown on the docs page:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy, BaseQuery
from sqlalchemy_searchable import SearchQueryMixin
from sqlalchemy_utils.types import TSVectorType
from sqlalchemy_searchable import make_searchable
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql+psycopg2://usr:admin#localhost/dev'
app.config['SECRET_KEY'] = 'notreallyasecret'
db = SQLAlchemy(app)
make_searchable()
class ArticleQuery(BaseQuery, SearchQueryMixin):
pass
class Article(db.Model):
query_class = ArticleQuery
__tablename__ = 'article'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.Unicode(255))
content = db.Column(db.UnicodeText)
search_vector = db.Column(TSVectorType('name', 'content'))
My search queries don't work properly. I opened a python shell and created the db, and inserted five identical articles
a= Article(name='finland',content='finland')
db.session.add(a)
db.session.commit() #a-e
with 'finland' both as name and content. According to the example:
Article.query.search(u'finland').limit(5).all()
There should be articles returned that have finland somewhere in them. In my case, I get an empty list. I get an object back if I modify the example query to:
Article.query.search(' ').first()
But it's rather useless searching for empty spaces. Any ideas?
Adding a bit more to it: I noticed in the article table, the 'search_vector tsvector' column is completely empty despite data being in the content and name columns; I'm not sure if that has anything to do with it.

I ran into this exact issue once, too, when using Flask-Script to add a manage.py management tool to my application.
The fact that the search_vector column is empty despite you having added the appropriate TSVectorType parameters means that the SQLAlchemy-Searchable trigger isn't present in the postgres DB. You can verify its absence by doing a \df+ in psql command line tool -- you will not see a trigger named article_search_vector_update. SQLAlchemy-Searchable sets up this trigger to update the content of the search_vector column when the columns named in TSVectorType(...) change.
In the case of manage.py, I had to first call:
db.configure_mappers()
Essentially, you have to configure SQLAlchemy's mappers before calling create_all(). Without doing this, SQLAlchemy-Searchable will not be given the opportunity to add its search_vector trigger to populate the TSVectorType column in the model.The SQLAlchemy-Searchable docs have more on this.
In total, a minimal manage.py that properly configures SQLAlchemy-Searchable as you require might look like:
#!/usr/bin/env python
from flask.ext.script import Manager
from app import app, db
manager = Manager(app)
#manager.command
def init_db():
"""
Drops and re-creates the SQL schema
"""
db.drop_all()
db.configure_mappers()
db.create_all()
db.session.commit()

On Collin Allen's answer: actually, the flask-sqlalchemy ''db'' exposes the configure_mappers function.
Replace:
from sqlalchemy.orm.mapper import configure_mappers
...
configure_mappers()
with:
...
db.configure_mappers()