I'm attempting to modularize my flask app by splitting models, the db, routers, etc into separate files. This has been giving me some trouble, because it seems that my tables are not being created as I expect.
I have this file called flask_postgres_server.py
from . import flask_server
from flask_sqlalchemy import SQLAlchemy
from flask import jsonify, request
from . import models
Tree = models.tree.Tree_postgres.Tree
app = flask_server.app # this simply exports an instance of a flask app, so I can use it in another server with mongo
#app.route("/trees", methods=['POST'])
def create_tree_postgres():
label = request.get_json['label']
tree = Tree(label=label)
tree.save()
return jsonify(tree.get_dict())
I am importing a Tree model:
from ... import postgres_db
db = postgres_db.db
class Tree(db.Model):
__tablename__ = 'trees'
id = db.Column(db.Integer, primary_key=True)
label = db.Column(db.String(), nullable=False)
def save(self):
db.session.add(self)
db.session.commit()
db.session.expunge_all()
db.session.close()
def get_dict(self):
return {"label": self.label, "_id": self.id}
which in turn imports the db:
from . import flask_server
app = flask_server.app
import os
from dotenv import load_dotenv, find_dotenv
from flask_sqlalchemy import SQLAlchemy
"""
Imported by a server in order to set up a postgres connection for use by a Flask server
"""
load_dotenv(find_dotenv())
DB_NAME = os.environ.get("DB_NAME")
POSTGRES_HOST = os.environ.get("POSTGRES_HOST")
POSTGRES_PORT = os.environ.get("POSTGRES_PORT")
POSTGRES_USER = os.environ.get("POSTGRES_USER")
POSTGRES_PASSWORD = os.environ.get("POSTGRES_PASSWORD")
DB_URI = f'postgresql://{POSTGRES_USER}:{POSTGRES_PASSWORD}#{POSTGRES_HOST}:{POSTGRES_PORT}/{DB_NAME}'
app.config['SQLALCHEMY_DATABASE_URI'] = DB_URI
db = SQLAlchemy(app)
db.create_all()
When I run my server and get post data at /trees, I get this error message:
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "trees" does not exist
LINE 1: INSERT INTO trees (label) VALUES ('countries') RETURNING tre...
^
[SQL: INSERT INTO trees (label) VALUES (%(label)s) RETURNING trees.id]
[parameters: {'label': 'countries'}]
Shouldn't this relation have been created automatically?
I can go into psql and create this table manually. Then I get another error saying:
DetachedInstanceError: Instance <Tree at 0x10d2facd0> is not bound to a Session
Is there something wrong with the way I've structured my code?
The trees table does not get created because db.create_all() is called before the Tree model is imported.
In the code in the question, you can fix this by moving db.create_all() to after the definition of the Tree model. In your overall structure, db.create_all() will need to be called after all the model definitions have been executed (usually by importing the files that contain them).
Note that it's not a good idea to close the session in the save method - you won't be able to access the Tree instance afterwards when calling its get_dict method. I'd suggest leaving session life-cycle management to flask-sqlalchemy, Tree.save should just add and commit.
Finally label = request.get_json['label'] in the route handler should be label = request.get_json()['label'].
DetachedInstanceError: Instance <Tree at 0x10d2facd0> is not bound to a Session implicitly give users clue that the object 0x10d2facd0 that you've created is already "disconnected" with the session.
You've already closed the session with db.session.close() but then tried to access the "closed" object afterward.
It is better to save the get_dict result before you close it. It can be rewritten like this:
#app.route("/trees", methods=['POST'])
def create_tree_postgres():
label = request.get_json['label']
tree = Tree(label=label)
response = tree.get_dict()
tree.save()
return jsonify(response)
Related
I am trying to build an API using Flask. For database actions I use flask_sqlalchemy.
In my main file, the flask app is initalized. I pass the resulting instance to another file where the configuration is set and to my database module that handles database operations.
main.py:
app = flask.Flask(__name__) # initialize flask app
#initialize modules with app
config.init(app)
database.init(app)
The problem is, the relations I use in the database are in a seperate file and it needs the db object to declare the classes for ORM.
My idea was to declare db and initialize it later in an init function, but that doesn't work in this case, because the db object is undefined when the pythonfile is loaded by an import.
relations.py
db: SQLAlchemy
def init(db):
Relations.db = db
class Series(db.Model):
"""Representation of a series
"""
id = db.Column(db.String(255), primary_key=True)
title = db.Column(db.String(255))
class User(db.Model):
"""Representation of a user
"""
id = db.Column(db.INT, primary_key=True)
name = db.Column(db.String(255))
class Subscription(db.Model):
"""Representation of a subscription
"""
series_id = db.Column(db.INT, primary_key=True)
user_id = db.Column(db.String(255), primary_key=True)
My database module uses the way and it works fine(init.py file):
db: SQLAlchemy
def init(app):
database.db = SQLAlchemy(app)
# handle database operations...
One approach to solve the issue is just using another instance in the relations.py like that:
app = flask.Flask(__name__)
db = SQLAlchemy(app)
# declare classes...
I tried it out and it workes, but that is not a nice way to solve this and leads to other problems.
Importing it from main does also not work because of circular import.
I have no idea how to smoothly solve this without removing modularization. I would be thankful for any inputs. If I should add any further information, just let me know.
I would create the app variable in your main.py file but leave out the initializing part. From there you call a function from init.py to basically set up the database. That is what I did for my last flask project.
Main.py:
from init import create_app
app = create_app()
if __name__ == '__main__':
app.run(debug=True)
Init.py:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
DB_NAME = "database.db"
def create_app():
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = f'sqlite:///{DB_NAME}'
db.init_app(app)
create_database(app)
#Other operations ...
return app
Relations.py
from init import db
#all your classes ...
db.create_all()
So now you can import the db object to your relations.py file from the init.py.
The intent: Refactor my code into MVC (this is just the model/database part), and have the server create the database with tables on first run if the database or tables does not exist.
This works when using a "flat" file with all the classes and functions defined in that file, but after moving out the functions into a service class and the models into their own folder with model classes, the db.create_all() function does not seem to be able to detect the table class correctly any more.
Example structure, (minimum viable problem):
server.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///database.sqlite'
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
db = SQLAlchemy(app)
def main():
# Intentionally moved into the main() function to prevent import loops
from services.users import UserService
users = UserService(db)
db.create_all()
app.run(debug=True)
if __name__ == '__main__':
main()
services\users.py
# Class used to access users data using a session
from models.users import Users
class UserService:
def __init__(self, db):
self.db = db
def get_all(self):
return self.db.session.query(Users).all()
def get(self, uid):
return self.db.session.query(Users).get(uid)
def add(self, json):
user = Users(email=json['email'], password=json['password'])
self.db.session.add(user)
self.db.session.commit()
return user
models\users.py
# The actual model
from server import db
class Users(db.Model):
_id = db.Column("id", db.Integer, primary_key=True)
email = db.Column(db.Text)
password = db.Column(db.Text)
Result: The database is created, but it is just an empty file with no tables inside of it.
I have also tried placing the db.create_all() inside the service class def __init__(self, db) (grasping at straws here), both as a self reference and as an argument reference. Neither have worked.
I am sure it is something obvious I am missing, but I have boiled down my project to just the bare minimum and still fail to see why it is not working - so I have to ask. How can I get the db.create_all() to detect my table classes correctly and actually create the required tables, while using this code structure (or something similar, in case I have misunderstood MVC)?
The problem is that server.py is executed twice
when it's imported in models/users.py
when server.py is called to run the app
Each execution generates a new db instance. The db imported by the model file adds the models to its metadata, the db created when the app is run has empty metadata.
You can confirm this by printing id(db) and db.metadata.tables at the end of models/users.py and just before the call to db.create_all() in the main function.
You need to structure your code so that only one db gets created. For example, you could move the app configuration and creation code into its own module, mkapp.py (feel free to come up with a better name):
mkapp.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///database.sqlite'
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
app.config['SQLALCHEMY_ECHO'] = True
db = SQLAlchemy(app)
And in server.py do
from mkapp import app, db
and in models/users.py do
from mkapp import db
As a bonus, this should also remove the import cycle.
I don't use flask much, so this solution can probably be improved on. For example, having a function create app and db and memoise the results might be better than creating them in top-level module code.
I am attempting to implement Flask-Session in my python application. I read in the docs that its recommended to use another interface like the SqlAlchemySessionInterface instead of the default NullSessionInterface which is used when nothing is provided to the SESSION_TYPE configuration key.
From the flask_session/init.py file under class Session it reads
By default Flask-Session will use :class:NullSessionInterface, you
really should configurate your app to use a different SessionInterface.
After setting the SESSION_TYPE configuration key to "sqlalchemy" I get an error
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) relation "sessions" does not exist
This indicates that Flask-Session is looking to use a table with the name "sessions" in my database model but I cannot find anywhere in the Flask-Session documentation where it points out that a table should be created and what fields it should have.
Can anyone suggest a solution to this please?
I wanted to use Flask-session, but I was also using Flask-migrate and didn't want to call db.create_all() manually and break the migration path. Fortunately, #Flashspeedlife's suggestion of just importing the Interface and instantiating it worked.
app/__init__.py:
from flask_session import SqlAlchemySessionInterface
from app.extensions import db, sess, migrate # My extensions file
def create_app():
app = Flask(__name__)
with app.app_context():
db.init_app(app)
migrate.init_app(app, db)
sess.init_app(app)
SqlAlchemySessionInterface(app, db, "sessions", "sess_")
Now, flask db migrate generates an alembic script with the new sessions table.
After studying the Flask-Session/init.py code I found that class SqlAlchemySessionInterface under its __init__ contains a Flask-SQLAlchemy model
class Session(self.db.Model).
To cause this table model to be created, in the file where I create my models I imported SqlAlchemySessionInterface from flask_sessionstore and put the line
SqlAlchemySessionInterface(myApp, sqlAlchemyDbObject, "table_name", "prefix_")
and then ran db.create_all().
class SqlAlchemySessionInterface(SessionInterface):
"""Uses the Flask-SQLAlchemy from a flask app as a session backend.
.. versionadded:: 0.2
:param app: A Flask app instance.
:param db: A Flask-SQLAlchemy instance.
:param table: The table name you want to use.
:param key_prefix: A prefix that is added to all store keys.
:param use_signer: Whether to sign the session id cookie or not.
:param permanent: Whether to use permanent session or not.
"""
serializer = pickle
session_class = SqlAlchemySession
def __init__(self, app, db, table, key_prefix, use_signer=False,
permanent=True):
if db is None:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy(app)
self.db = db
self.key_prefix = key_prefix
self.use_signer = use_signer
self.permanent = permanent
class Session(self.db.Model):
__tablename__ = table
id = self.db.Column(self.db.Integer, primary_key=True)
session_id = self.db.Column(self.db.String(255), unique=True)
data = self.db.Column(self.db.LargeBinary)
expiry = self.db.Column(self.db.DateTime)
def __init__(self, session_id, data, expiry):
self.session_id = session_id
self.data = data
self.expiry = expiry
def __repr__(self):
return '<Session data %s>' % self.data
# self.db.create_all()
self.sql_session_model = Session
I'am definitely using Django for my next project. Documentation for many Flask Extensions aren't great at all.
EDIT
Changed (imported SqlAlchemySessionInterface from flask_session) to (imported SqlAlchemySessionInterface from flask_sessionstore)
I'm building a Flask app with Flask-SQLAlchemy and I'm trying to write a script that will create a Sqlite3 database without running the main application. In order to avoid circular references, I've initialized the main Flask app object and the SQLAlchemy database object in separate modules. I then import and combine them in a third file when running the app. This works fine when I'm running the app, as the database is built and operates properly when create rows and query them. However, when I try to import them in another module, I get the following error:
RuntimeError: application not registered on db instance and no applicationbound to current context
My code looks like the following:
root/create_database.py
from application.database import db
from application.server import app
db.init_app(app)
db.create_all()
root/run.sh
export FLASK_APP=application/server.py
flask run
root/application/init.py
from database import db
from server import app
db.init_app(app)
from routes import apply_routes
apply_routes(app)
root/application/database.py
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
root/application/server.py
from flask import Flask
import os
app = Flask(__name__)
path = os.path.dirname( os.path.realpath(__file__) )
database_path = os.path.join(path, '../mydb.sqlite')
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + database_path
root/application/models/init.py
from user import User
root/application/models/user.py
from application.database import db
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
password = db.Column(db.String(120))
def __init__(self, username, password):
self.username = username
self.password = password
In my create_database.py script I'm trying to make sure that the SQLAlchemy db instance is configured with the config details from the app object, but it doesn't seem to be connecting for some reason. Am I missing something important here?
You either have to create a request or you have to create the models with sqlalchemy directly. We do something similar at work and chose the former.
Flask lets you create a test request to initialize an app. Try something like
from application.database import db
from application.server import app
with app.test_request_context():
db.init_app(app)
db.create_all()
First time on the site, so hi to all and thanks in advance. Longtime lurker and newb.
I'm working on a web app in flask, using Flask-SqlAlchemy and SqlAlchemy-Searchable (docs-> https://sqlalchemy-searchable.readthedocs.org/en/latest/index.html). For a reason I can't figure out, when I try a similar example to the code shown on the docs page:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy, BaseQuery
from sqlalchemy_searchable import SearchQueryMixin
from sqlalchemy_utils.types import TSVectorType
from sqlalchemy_searchable import make_searchable
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql+psycopg2://usr:admin#localhost/dev'
app.config['SECRET_KEY'] = 'notreallyasecret'
db = SQLAlchemy(app)
make_searchable()
class ArticleQuery(BaseQuery, SearchQueryMixin):
pass
class Article(db.Model):
query_class = ArticleQuery
__tablename__ = 'article'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.Unicode(255))
content = db.Column(db.UnicodeText)
search_vector = db.Column(TSVectorType('name', 'content'))
My search queries don't work properly. I opened a python shell and created the db, and inserted five identical articles
a= Article(name='finland',content='finland')
db.session.add(a)
db.session.commit() #a-e
with 'finland' both as name and content. According to the example:
Article.query.search(u'finland').limit(5).all()
There should be articles returned that have finland somewhere in them. In my case, I get an empty list. I get an object back if I modify the example query to:
Article.query.search(' ').first()
But it's rather useless searching for empty spaces. Any ideas?
Adding a bit more to it: I noticed in the article table, the 'search_vector tsvector' column is completely empty despite data being in the content and name columns; I'm not sure if that has anything to do with it.
I ran into this exact issue once, too, when using Flask-Script to add a manage.py management tool to my application.
The fact that the search_vector column is empty despite you having added the appropriate TSVectorType parameters means that the SQLAlchemy-Searchable trigger isn't present in the postgres DB. You can verify its absence by doing a \df+ in psql command line tool -- you will not see a trigger named article_search_vector_update. SQLAlchemy-Searchable sets up this trigger to update the content of the search_vector column when the columns named in TSVectorType(...) change.
In the case of manage.py, I had to first call:
db.configure_mappers()
Essentially, you have to configure SQLAlchemy's mappers before calling create_all(). Without doing this, SQLAlchemy-Searchable will not be given the opportunity to add its search_vector trigger to populate the TSVectorType column in the model.The SQLAlchemy-Searchable docs have more on this.
In total, a minimal manage.py that properly configures SQLAlchemy-Searchable as you require might look like:
#!/usr/bin/env python
from flask.ext.script import Manager
from app import app, db
manager = Manager(app)
#manager.command
def init_db():
"""
Drops and re-creates the SQL schema
"""
db.drop_all()
db.configure_mappers()
db.create_all()
db.session.commit()
On Collin Allen's answer: actually, the flask-sqlalchemy ''db'' exposes the configure_mappers function.
Replace:
from sqlalchemy.orm.mapper import configure_mappers
...
configure_mappers()
with:
...
db.configure_mappers()