We Keep Coding

Flask_SQLAlchemy, db.create_all() is unable to "see" my tables when imported though a service class

The intent: Refactor my code into MVC (this is just the model/database part), and have the server create the database with tables on first run if the database or tables does not exist.
This works when using a "flat" file with all the classes and functions defined in that file, but after moving out the functions into a service class and the models into their own folder with model classes, the db.create_all() function does not seem to be able to detect the table class correctly any more.
Example structure, (minimum viable problem):
server.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///database.sqlite'
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
db = SQLAlchemy(app)
def main():
# Intentionally moved into the main() function to prevent import loops
from services.users import UserService
users = UserService(db)
db.create_all()
app.run(debug=True)
if __name__ == '__main__':
main()
services\users.py
# Class used to access users data using a session
from models.users import Users
class UserService:
def __init__(self, db):
self.db = db
def get_all(self):
return self.db.session.query(Users).all()
def get(self, uid):
return self.db.session.query(Users).get(uid)
def add(self, json):
user = Users(email=json['email'], password=json['password'])
self.db.session.add(user)
self.db.session.commit()
return user
models\users.py
# The actual model
from server import db
class Users(db.Model):
_id = db.Column("id", db.Integer, primary_key=True)
email = db.Column(db.Text)
password = db.Column(db.Text)
Result: The database is created, but it is just an empty file with no tables inside of it.
I have also tried placing the db.create_all() inside the service class def __init__(self, db) (grasping at straws here), both as a self reference and as an argument reference. Neither have worked.
I am sure it is something obvious I am missing, but I have boiled down my project to just the bare minimum and still fail to see why it is not working - so I have to ask. How can I get the db.create_all() to detect my table classes correctly and actually create the required tables, while using this code structure (or something similar, in case I have misunderstood MVC)?

The problem is that server.py is executed twice
when it's imported in models/users.py
when server.py is called to run the app
Each execution generates a new db instance. The db imported by the model file adds the models to its metadata, the db created when the app is run has empty metadata.
You can confirm this by printing id(db) and db.metadata.tables at the end of models/users.py and just before the call to db.create_all() in the main function.
You need to structure your code so that only one db gets created. For example, you could move the app configuration and creation code into its own module, mkapp.py (feel free to come up with a better name):
mkapp.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///database.sqlite'
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
app.config['SQLALCHEMY_ECHO'] = True
db = SQLAlchemy(app)
And in server.py do
from mkapp import app, db
and in models/users.py do
from mkapp import db
As a bonus, this should also remove the import cycle.
I don't use flask much, so this solution can probably be improved on. For example, having a function create app and db and memoise the results might be better than creating them in top-level module code.

relation does not exist in flask sqlalchemy, although create function called

I'm attempting to modularize my flask app by splitting models, the db, routers, etc into separate files. This has been giving me some trouble, because it seems that my tables are not being created as I expect.
I have this file called flask_postgres_server.py
from . import flask_server
from flask_sqlalchemy import SQLAlchemy
from flask import jsonify, request
from . import models
Tree = models.tree.Tree_postgres.Tree
app = flask_server.app # this simply exports an instance of a flask app, so I can use it in another server with mongo
#app.route("/trees", methods=['POST'])
def create_tree_postgres():
label = request.get_json['label']
tree = Tree(label=label)
tree.save()
return jsonify(tree.get_dict())
I am importing a Tree model:
from ... import postgres_db
db = postgres_db.db
class Tree(db.Model):
__tablename__ = 'trees'
id = db.Column(db.Integer, primary_key=True)
label = db.Column(db.String(), nullable=False)
def save(self):
db.session.add(self)
db.session.commit()
db.session.expunge_all()
db.session.close()
def get_dict(self):
return {"label": self.label, "_id": self.id}
which in turn imports the db:
from . import flask_server
app = flask_server.app
import os
from dotenv import load_dotenv, find_dotenv
from flask_sqlalchemy import SQLAlchemy
"""
Imported by a server in order to set up a postgres connection for use by a Flask server
"""
load_dotenv(find_dotenv())
DB_NAME = os.environ.get("DB_NAME")
POSTGRES_HOST = os.environ.get("POSTGRES_HOST")
POSTGRES_PORT = os.environ.get("POSTGRES_PORT")
POSTGRES_USER = os.environ.get("POSTGRES_USER")
POSTGRES_PASSWORD = os.environ.get("POSTGRES_PASSWORD")
DB_URI = f'postgresql://{POSTGRES_USER}:{POSTGRES_PASSWORD}#{POSTGRES_HOST}:{POSTGRES_PORT}/{DB_NAME}'
app.config['SQLALCHEMY_DATABASE_URI'] = DB_URI
db = SQLAlchemy(app)
db.create_all()
When I run my server and get post data at /trees, I get this error message:
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "trees" does not exist
LINE 1: INSERT INTO trees (label) VALUES ('countries') RETURNING tre...
^
[SQL: INSERT INTO trees (label) VALUES (%(label)s) RETURNING trees.id]
[parameters: {'label': 'countries'}]
Shouldn't this relation have been created automatically?
I can go into psql and create this table manually. Then I get another error saying:
DetachedInstanceError: Instance <Tree at 0x10d2facd0> is not bound to a Session
Is there something wrong with the way I've structured my code?

The trees table does not get created because db.create_all() is called before the Tree model is imported.
In the code in the question, you can fix this by moving db.create_all() to after the definition of the Tree model. In your overall structure, db.create_all() will need to be called after all the model definitions have been executed (usually by importing the files that contain them).
Note that it's not a good idea to close the session in the save method - you won't be able to access the Tree instance afterwards when calling its get_dict method. I'd suggest leaving session life-cycle management to flask-sqlalchemy, Tree.save should just add and commit.
Finally label = request.get_json['label'] in the route handler should be label = request.get_json()['label'].

DetachedInstanceError: Instance <Tree at 0x10d2facd0> is not bound to a Session implicitly give users clue that the object 0x10d2facd0 that you've created is already "disconnected" with the session.
You've already closed the session with db.session.close() but then tried to access the "closed" object afterward.
It is better to save the get_dict result before you close it. It can be rewritten like this:
#app.route("/trees", methods=['POST'])
def create_tree_postgres():
label = request.get_json['label']
tree = Tree(label=label)
response = tree.get_dict()
tree.save()
return jsonify(response)

In a flask project, what's an idiomatic way to define sql models after the app has been created?

I'm rather confused on where to define sql (flask-sqlalchemy) models in my flask app. I have an app.py which creates the flask app (app = Flask(__name__)), and I have a models.py which instantiates SQLAlchemy and defines my models:
from flask import current_app
db = SQLAlchemy(current_app)
class Habit(db.Model):
rowid = db.Column(db.Integer, primary_key=True)
However, the only way for this to work is if my app.py imports it after the app has been created, which goes against the python idiom of putting all imports at the top of the file. However, this is what many example apps that I've seen do (Ex. 1) (Ex. 2).
There's got to be a better way to define your models, but it feels like a catch 22. The models subclass db.Model, db is instantiated (using flask-sqlalchemy) using the flask app, and the flask app must import the models (or else that code won't run). I could create a new file that first imports the app and then the db, but idiomatically, it seems like the file that instantiates the Flask app tends to be the entry point for the application.
Any advice? Thanks!

You can use init_app method of Flask extensions to bound them to your Flask application after instantiating. For example, your models.py module:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Habit(db.Model):
rowid = db.Column(db.Integer, primary_key=True)
And your launch file (app.py):
from flask import Flask
from models import db
app = Flask()
db.init_app(app)
app.run()
It helps you to build modular applications with logic separated in different files.

Do I need to create a sessions table to use Flask-Session SqlAlchemySessionInterface

I am attempting to implement Flask-Session in my python application. I read in the docs that its recommended to use another interface like the SqlAlchemySessionInterface instead of the default NullSessionInterface which is used when nothing is provided to the SESSION_TYPE configuration key.
From the flask_session/init.py file under class Session it reads
By default Flask-Session will use :class:NullSessionInterface, you
really should configurate your app to use a different SessionInterface.
After setting the SESSION_TYPE configuration key to "sqlalchemy" I get an error
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) relation "sessions" does not exist
This indicates that Flask-Session is looking to use a table with the name "sessions" in my database model but I cannot find anywhere in the Flask-Session documentation where it points out that a table should be created and what fields it should have.
Can anyone suggest a solution to this please?

I wanted to use Flask-session, but I was also using Flask-migrate and didn't want to call db.create_all() manually and break the migration path. Fortunately, #Flashspeedlife's suggestion of just importing the Interface and instantiating it worked.
app/__init__.py:
from flask_session import SqlAlchemySessionInterface
from app.extensions import db, sess, migrate # My extensions file
def create_app():
app = Flask(__name__)
with app.app_context():
db.init_app(app)
migrate.init_app(app, db)
sess.init_app(app)
SqlAlchemySessionInterface(app, db, "sessions", "sess_")
Now, flask db migrate generates an alembic script with the new sessions table.

After studying the Flask-Session/init.py code I found that class SqlAlchemySessionInterface under its __init__ contains a Flask-SQLAlchemy model
class Session(self.db.Model).
To cause this table model to be created, in the file where I create my models I imported SqlAlchemySessionInterface from flask_sessionstore and put the line
SqlAlchemySessionInterface(myApp, sqlAlchemyDbObject, "table_name", "prefix_")
and then ran db.create_all().
class SqlAlchemySessionInterface(SessionInterface):
"""Uses the Flask-SQLAlchemy from a flask app as a session backend.
.. versionadded:: 0.2
:param app: A Flask app instance.
:param db: A Flask-SQLAlchemy instance.
:param table: The table name you want to use.
:param key_prefix: A prefix that is added to all store keys.
:param use_signer: Whether to sign the session id cookie or not.
:param permanent: Whether to use permanent session or not.
"""
serializer = pickle
session_class = SqlAlchemySession
def __init__(self, app, db, table, key_prefix, use_signer=False,
permanent=True):
if db is None:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy(app)
self.db = db
self.key_prefix = key_prefix
self.use_signer = use_signer
self.permanent = permanent
class Session(self.db.Model):
__tablename__ = table
id = self.db.Column(self.db.Integer, primary_key=True)
session_id = self.db.Column(self.db.String(255), unique=True)
data = self.db.Column(self.db.LargeBinary)
expiry = self.db.Column(self.db.DateTime)
def __init__(self, session_id, data, expiry):
self.session_id = session_id
self.data = data
self.expiry = expiry
def __repr__(self):
return '<Session data %s>' % self.data
# self.db.create_all()
self.sql_session_model = Session
I'am definitely using Django for my next project. Documentation for many Flask Extensions aren't great at all.
EDIT
Changed (imported SqlAlchemySessionInterface from flask_session) to (imported SqlAlchemySessionInterface from flask_sessionstore)

Creating a database in flask sqlalchemy

I'm building a Flask app with Flask-SQLAlchemy and I'm trying to write a script that will create a Sqlite3 database without running the main application. In order to avoid circular references, I've initialized the main Flask app object and the SQLAlchemy database object in separate modules. I then import and combine them in a third file when running the app. This works fine when I'm running the app, as the database is built and operates properly when create rows and query them. However, when I try to import them in another module, I get the following error:
RuntimeError: application not registered on db instance and no applicationbound to current context
My code looks like the following:
root/create_database.py
from application.database import db
from application.server import app
db.init_app(app)
db.create_all()
root/run.sh
export FLASK_APP=application/server.py
flask run
root/application/init.py
from database import db
from server import app
db.init_app(app)
from routes import apply_routes
apply_routes(app)
root/application/database.py
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
root/application/server.py
from flask import Flask
import os
app = Flask(__name__)
path = os.path.dirname( os.path.realpath(__file__) )
database_path = os.path.join(path, '../mydb.sqlite')
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + database_path
root/application/models/init.py
from user import User
root/application/models/user.py
from application.database import db
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
password = db.Column(db.String(120))
def __init__(self, username, password):
self.username = username
self.password = password
In my create_database.py script I'm trying to make sure that the SQLAlchemy db instance is configured with the config details from the app object, but it doesn't seem to be connecting for some reason. Am I missing something important here?

You either have to create a request or you have to create the models with sqlalchemy directly. We do something similar at work and chose the former.
Flask lets you create a test request to initialize an app. Try something like
from application.database import db
from application.server import app
with app.test_request_context():
db.init_app(app)
db.create_all()