We Keep Coding

How to use PostgreSQL with SQLAlchemy in a Flask application with Redis queued job

When executing SQL queries using SQLAlchemy directly on API calls in Flask app they are executed successfully but when I try to perform one inside my function that gets executed using Redis job queue I get the following error: RuntimeError: No application found. Either work inside a view function or push an application context. The function is stored in a separate file, not app.py.
My app.py:
app = Flask(__name__)
app.secret_key = os.environ.get('FLASK_SECRET_KEY')
worker_queue = Queue('default', connection=conn)
worker_registry = StartedJobRegistry(queue=worker_queue)
# database configuration
setup_db(app)
in my models.py the setup_db look like this:
db = SQLAlchemy()
def setup_db(app):
database_name = 'dev'
default_database_path = "postgresql://{}:{}#{}/{}".format('postgres', 'password', 'localhost:5432', database_name)
database_path = os.getenv('DATABASE_URL', default_database_path)
app.config['SQLALCHEMY_DATABASE_URI'] = database_path
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
db.app = app
db.init_app(app)
And then in a function that gets triggered when I enqueue job I call:
users = User.query.all()
where User is a db model I have defined in my models.py.

Flask DB migrate to support multiple databases using binds

I am using Flask Application with flask-migrate. I have initialized the app with single database but now the requirement is to add multiple databases.
Here are the steps that I followed with the below code :-
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
import os
from flask_migrate import Migrate
basedir = os.path.abspath(os.path.dirname(__file__))
app = Flask(__name__)
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + os.path.join(basedir, 'database1.db')
"""
app.config['SQLALCHEMY_BINDS'] = {
'database2' : 'sqlite:///' + os.path.join(basedir, 'database2.db')
}
"""
db = SQLAlchemy(app)
migrate = Migrate(app, db)
class Table1(db.Model):
id = db.Column(db.Integer, primary_key=True)
"""
class Table2(db.Model):
__bind_key__ = 'database2'
id = db.Column(db.Integer, primary_key=True)
"""
#app.route('/add1')
def index():
item = Table1(id=2)
db.session.add(item)
db.session.commit()
return 'Added value'
"""
#app.route('/add2')
def index2():
item = Table2(id=2)
db.session.add(item)
db.session.commit()
return 'Added table2 value'
"""
if __name__ == '__main__':
app.run(debug=True)
Initialized the code as below
flask db init
flask db migrate
flask db upgrade
Visit http://127.0.0.1:5000/add1 to add to database1
Now uncomment the code so that the database2 can be added. On running flask db init again with --multidb switch, an error is reported.
(venv) bash-4.1$ flask db init --multidb
Error: Directory migrations already exists and is not empty
How can I convert an existing flask project with migrations to support multiple databases? Also, these multiple databases will evolve over the time i.e. I might have to add more databases. So, what is the best way to keep on adding more databases?
EDIT
One solution, I find is to remove the alembic_version table from existing database, remove the migrations folder and then do db init --multidb. I am okay resetting the migrations but want to know if this is the only solution?

Why SQLAlchemy won't commit changes?

I've done this so many times before successfully but all of the sudden I can't seem
to make it work. I'm creating tables in my MySQL database using Flask SQLAlchemy as a middleman.
Also using pymysql adapter.
Can you please check my "plumbing" is correct ?
my init.py :
import werkzeug
werkzeug.cached_property = werkzeug.utils.cached_property
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_bcrypt import Bcrypt
from .config import DevConfig
db = SQLAlchemy()
bcrypt = Bcrypt()
def create_app(script_info=None):
app = Flask(__name__)
app.config.from_object(DevConfig)
bcrypt.init_app(app)
db.init_app(app)
from .flask_app import users_blueprint
app.register_blueprint(users_blueprint)
#app.shell_context_processor
def ctx():
return {'app': app, 'db': db}
return app
models.py:
from project import db, bcrypt
class User(db.Model):
__tablename__ = 'users'
...
...
config.py:
import os
class BaseConfig:
SQLALCHEMY_TRACK_MODIFICATIONS = False
SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://mark:supersecret#localhost/database1?charset=utf8mb4' # the "?charset" thingy is there to avoid encoding warnings from SQLAlchemy
SECRET_KEY = 'pythonrocks'
class DevConfig(BaseConfig):
DEBUG = True
class ProdConfig(BaseConfig):
DEBUG = False
interpreter:
Instance: /home/mark/project/instance
In [1]: db
Out[1]: <SQLAlchemy engine=mysql+pymysql://mark:***#localhost/database1?charset=utf8mb4>
In [2]: db.create_all()
In [3]: db.session.commit()
I get no error output. The tables are just not being created on commit.
mysql> show tables;
Empty set (0.00 sec)
How can I check what's causing this ? The dev server is running.

You're not importing your models before calling db.create_all() - SQLAlchemy simply doesn't know about your models - and thus can't create the tables.
Import it first, then call db.create_all() and db.session.commit().

How to define and access the database postgresql like flask sample code

I'm new use python flask, I want connected to postgresql use code like flask sample in this link, but in code sample use sqlite3. I try to search code sample and make me confused because every sample use different approach. This my code run but when use CLI to initialize database error.
Error: No such command "init-db".
My structure file
This my code:
run.py
#run.py
import os
from smart_app import create_app
app = create_app()
if __name__ == '__main__':
app.run()
init.py
#__init__.py
from flask import Flask
def create_app(config_filename=None):
app = Flask(__name__, instance_relative_config=True)
# load default configuration
app.config.from_object('config.default')
# load the configuration from the instance folder
app.config.from_pyfile('config.py')
# Load the file specified by the APP_CONFIG_FILE environment variable
# Variables defined here will override those in the default configuration
app.config.from_envvar('APP_CONFIG_FILE')
# Connect to database
from . import db
db.init_app(app)
return app
db.py
db.py
import click
from flask import current_app, g
from flask.cli import with_appcontext
from flask_sqlalchemy import SQLAlchemy
# db = SQLAlchemy()
def get_db():
if 'db' not in g:
g.db = SQLAlchemy()
return g.db
def close_db(e=None):
db = g.pop('db', None)
if db is not None:
db.close()
def init_db():
db = get_db()
with current_app.open_resource('schema.sql') as f:
db.executescript(f.read().decode('utf8'))
#click.command('init-db')
#with_appcontext
def init_db_command():
"""Clear the existing data and create new tables."""
init_db()
click.echo('Initialized the database')
def init_app(app):
app.teardown_appcontext(close_db)
app.cli.add_command(init_db_command)

If you familiar with SQL, use the psycopg2 to connect to your postgresql database.
Or if you want use ORM to operate the database in flask, you can use flask_sqlalchemy.

How to fix 'flask db.create_all() error'?

I'm have flask init with create app function. I create test file for my unitests. In unitest class add setUp function were i create new flask app and add context, push context to it. Next i create test db with create_all() function and, where i start test file, i have next error:
in _execute_for_all_tables op(bind=self.get_engine(app, bind), **extra)
TypeError: create_all() got an unexpected keyword argument 'forms'
I haven't 'forms' files or variable on models or anywere.
Before this error place in flask_sqlalchemy/init.py MetaData(bind=None), may be it some help.
from flask import Flask
from config import Config
from blocker_application.database import db
from flask_migrate import Migrate
from flask_login import LoginManager
from flask_moment import Moment
migrate = Migrate()
login = LoginManager()
login.login_view = 'user.login'
moment = Moment()
def create_app(config_class=Config):
app = Flask(__name__)
app.config.from_object(config_class)
db.init_app(app)
migrate.init_app(app, db)
login.init_app(app)
moment.init_app(app)
from blocker_application.main.routes import bp_main
from blocker_application.errors.handlers import bp_error
from blocker_application.reports.routes import bp_reports
from blocker_application.user.routes import bp_user
from blocker_application.applications.routes import bp_applications
app.register_blueprint(bp_main)
app.register_blueprint(bp_error)
app.register_blueprint(bp_reports, url_prefix='/reports')
app.register_blueprint(bp_user, url_prefix='/user')
app.register_blueprint(bp_applications, url_prefix='/applications')
return app
from blocker_application import models
________________________________________________________________________
/config.py
import os
basedir = os.path.abspath(os.path.dirname(__file__))
class Config:
SECRET_KEY = os.environ.get('SECRET_KEY') or 'not realy finish secret key configuration'
SQLALCHEMY_DATABASE_URI = 'mysql://some_database'
SQLALCHEMY_TRACK_MODIFICATIONS = False
class TestConfig(Config):
TESTING = True
SQLALCHEMY_BINDS = {'test': 'mysql://some_database_test'}
______________________________________________________________________
/tests.py
import unittest
from blocker_application import create_app, db
from blocker_application import models
from config import TestConfig
class UserModelCase(unittest.TestCase):
def setUp(self):
self.app = create_app(TestConfig)
self.app_context = self.app.app_context()
self.app_context.push()
db.create_all(bind='test')
def tearDown(self):
db.session.remove()
db.drop_all(bind='test')
self.app_context.pop()
def test_password(self):
u = models.User(username='Mark')
u.set_password('Mark')
self.assertTrue(u.check_password('Mark'))
self.assertFalse(u.check_password('Tony'))
if __name__ == '__main__':
unittest.main(verbosity=2)

I found decision. Unittest work ok, after recreate the virtual enveroment.