We Keep Coding

Is there a way to add mySQL database/schema name in flask sqlalchemy connection

I have a flask restful app connected to mySQL database and I am using SQLAlchemy. We can connect to the mySQL server using the following -
app.config['SQLALCHEMY_DATABASE_URI'] = f"mysql+pymysql://root:password#127.0.0.1:3306"
I am working on a use case where the database name will be provided on real-time basis through a GET request. Based on the database name provided, the app will connect to the respective database and perform the operations. For this purpose, I would like to have a way where I can tell the flask app to talk to the provided database (Flask app is already connected to the mySQL server). Currently, I am creating the connection again in the API class.
API: Calculate.py
from flask_restful import Resource, reqparse
from app import app
class Calculate(Resource):
def get(self):
parser = reqparse.RequestParser()
parser.add_argument('schema', type=str, required=True, help='Provide schema name.')
args = parser.parse_args()
session['schema_name'] = args.get('schema')
app.config['SQLALCHEMY_DATABASE_URI'] = f"mysql+pymysql://root:password#127.0.0.1:3306/{session['schema_name']}"
from db_models.User import User
...
DB Model: User.py
from flask import session
from flask_sqlalchemy import SQLAlchemy
from app import app
db = SQLAlchemy(app)
class User(db.Model):
__tablename__ = 'user'
__table_args__ = {"schema": session['schema_name']}
User_ID = db.Column(db.Integer, primary_key=True)
Name = db.Column(db.String(50))
db.create_all()
The above thing works for me. But I would want to understand if there is an alternative to this or a better way of doing this.
Edit: The above code does not work. It references the first schema name that was provided even if I provide a new schema name in the same running instance of the app.

you can write the SQLALCHEMY path like this:
SQLALCHEMY_DATABASE_URI='mysql+pymysql://root:password#localhost:3306/database name'

According to the docs not all values can be updated (first parragraph), in your use case you should use SQLALCHEMY_BINDS variable in your use case this is a dict and create a Model for each schema. Example:
Db Model
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
SQLALCHEMY_DATABASE_URI = f"mysql+pymysql://root:password#127.0.0.1:3306/schema_name1"
SQLALCHEMY_BINDS = {
'db1': SQLALCHEMY_DATABASE_URI, # default
'db2': f"mysql+pymysql://root:password#127.0.0.1:3306/schema_name2"
}
app = Flask(__name__)
db = SQLALchemy(app)
then create a model for each schema
class UserModeldb1(db.Model):
__tablename__ = 'user'
__bind_key__ = 'db1' #this parameter is set according to the database
id = db.Column(db.Integer, primary_key=True)
...
class UserModeldb2(db.Model):
__tablename__ = 'user'
__bind_key__ = 'db2'
id = db.Column(db.Integer, primary_key=True)
...
finally in your get method add some logic to capture the schema and execute your model accorddingly. you should look this question is really helpful Configuring Flask-SQLAlchemy to use multiple databases with Flask-Restless

Pycharm SqlAlchemy autocomplete not working

I'm using SQLAlchemy and Pycharm, but PyCharm can't see methods of SQLAlchemy for autocomplete function.
Code:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///data.sqlite3'
db = SQLAlchemy(app)
class User(db.Model):
__tablename__ = 'users'
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(16), index=True, unique=True)
if __name__ == '__main__':
db.create_all()
app.run(debug=True)
For example, if I want call SQLAlchemy method of User class i must type full method name manually User.query.filter_by(username='peter').first()
Autocomplete example
How to make autocomplete work for SQLAlchemy?
1) Yes, project was reloaded several times
2) Yes, right interpreter in File | settings | Project
3) Yes, PyCharm is not in Power Save Mode
4) Yes, I have Professional edition.

PyCharm (or any other IDE) can't find most methods (query(), add(), commit() etc) as they are not defined in flask_sqlalchemy.SQLAlchemy class. SQLAlchemy methods are added to flask_sqlalchemy.SQLAlchemy objects dynamically during initialization. You can find this initialization function in flask_sqlalchemy module:
def _include_sqlalchemy(obj):
for module in sqlalchemy, sqlalchemy.orm:
for key in module.__all__:
if not hasattr(obj, key):
setattr(obj, key, getattr(module, key))
Just for the testing: you can type from sqlalchemy.orm import Query. I think PyCharm will find it's objects just fine.
I can't suggest any solution, maybe someone else here know about possibility of dynamically added attributes autocompletion.

class User(db.Model):
query: db.Query # Type hint here
__tablename__ = 'users'
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(16), index=True, unique=True)
adding type hint query: db.Query

How is using SQLAlchemy different in Flask framework

I've written things in SQLAlchemy that work fine but the same workflow/commands seem to be askew when doing them in Flask. For example, these types of lines are getting flagged as "undefined":
metadata = MetaData(db)
abc = Table('abc', metadata, autoload=True)
s = select([abc.name])
I do have from flask.ext.sqlalchemy import SQLAlchemy at the top.
Is there something else I need to be doing additionally or instead?
(For reference, I'm running it at pythonanywhere)

Importing SQLAlchemy will not give you direct access to the names inside that module. You should also be aware that using the flask_sqlalchemy (formerly flask.ext.sqlalchemy) module uses a somewhat different mechanism to access SQLAlchemy features. This means that any attempt to transfer your current knowledge of SQLAlchemy shouold be informed by a study of the flask_sqlalchemy documentation.
Typically you will create a Flask application and then pass that to a call to SQLAlchemy as in this example. The relevant code is shown below.
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
The db object now has the Model, Column and the various datatypes as attributes, so you can define a table/model in the following way.
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
email = db.Column(db.String(120), unique=True)
def __init__(self, username, email):
self.username = username
self.email = email
def __repr__(self):
return '<User %r>' % self.username
I personally am not fond of this particular access model, since it forces me to qualify the various names inside the db namespace rather than simply importing them from a module and using them unqualified, but it seems to work (at least for relatively uncomplicated databases).
If you are an experienced SQLAlchemy user you might want to consider using the standard access mechanisms, though this may render you vulnerable to subtle bugs due to unanticipated thread/web session interactions. I have also heard that it can be tricky to deploy multiple databases. I have no direct evidence of this, so please regard it as anecdotal.

Flask-Sqlalchemy setup engine configuration

SqlAlchemy extension:
https://pythonhosted.org/Flask-SQLAlchemy/index.html
and i want to setup the engine with customer configuration using the parameters here:
http://docs.sqlalchemy.org/en/rel_0_9/core/engines.html
I'm using the way described on Flask-SqlAlchemy documentation:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
email = db.Column(db.String(120), unique=True)
def __init__(self, username, email):
self.username = username
self.email = email
def __repr__(self):
return '<User %r>' % self.username
EDIT:
How can I pass the following engine parameters in this kind of configuration:
isolation_level = 'AUTOCOMMIT', encoding='latin1', echo=True
method: SQLAlchemy() doesn't take these as arguments.

it's an open issue: https://github.com/mitsuhiko/flask-sqlalchemy/issues/166
you can try this
class SQLiteAlchemy(SQLAlchemy):
def apply_driver_hacks(self, app, info, options):
options.update({
'isolation_level': 'AUTOCOMMIT',
'encoding': 'latin1',
'echo': True
})
super(SQLiteAlchemy, self).apply_driver_hacks(app, info, options)
db = SQLiteAlchemy(app)

it’s just a config option. Here’s ours:
SQLALCHEMY_ENGINE_OPTIONS = {
"pool_pre_ping": True,
"pool_recycle": 300,
}

I set {'pool_pre_ping':True} like above!TypeError: Invalid argument(s) 'pool_pre_ping' sent to create_engine(), using configuration MySQLDialect_pymysql/QueuePool/Engine.
Please check that the keyword arguments are appropriate for this combination of components.

you can define different engine options for different binds overwriting the apply_driver_hacks and define the options for each of your databases. Forexample, if you want to define different pool classes for different databases:
app.config['SQLALCHEMY_DATABASE_URI'] = "monetdb://..//.."
app.config['SQLALCHEMY_BINDS '] = {
'pg': 'postgres+psycopg2://..//..'
}
app.config['POOL_CLASS'] = {'monetdb' : StaticPool , "postgres+psycopg2" : QueuePool}
class MySQLAlchemy(SQLAlchemy):
def apply_driver_hacks(self, app, info, options):
super().apply_driver_hacks(app, info, options)
try:
options['poolclass'] = app.config['POOL_CLASS'][info.drivername]
except KeyError: #if the pool class is not defined it will be ignored, means that it will use the default option
pass
db = MySQLAlchemy(app)

Associate "external' class model with flask sqlalchemy

We use a central class model for a wide variety of python modules. This model is defined using SQLAlchemy. The classes all inherit from declarative_base.
For example, our model definitions look something like this:
Base = declarative_base()
class Post(Base):
__tablename__ = 'Posts'
id = Column(INT, primary_key=True, autoincrement=True)
body = Column(TEXT)
timestamp = Column(TIMESTAMP)
user_id = Column(INT, ForeignKey('Users.uid'))
We have been building a flask web-application in which we employ this same model. We have discovered a tricky problem in that flask-sqlalchemy appears to be designed in such a way that it expects all classes used in its model to have been defined by passing in an active instance of the session. Here is an example of a "proper" flask-sqalchemy class model definition:
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
email = db.Column(db.String(120), unique=True)
Note that the above example for flask-sqlalchemy requires an already-instantiated sql session. This has horrified us, because we are completely at a loss as to how to integrate our SqlAlchemy model into flask. We really want to use the flask-security suite in particular.
This problem has been brought up before on SO. Here, for example:
How to use flask-sqlalchemy with existing sqlalchemy model?
Our requirements are different from those of whoever accepted the response there. The response points out that one loses the ability to use User.query, but this is precisely one of the things we must retain.
It is not feasible to abandon our nice, elegant, central class model definition in favor of what flask-sqlalchemy appears to require. Is there any way for us to associate our model with the SQLAlchemy() object? Bonus points for getting us the .query() method on our classes which appears to be required by flask-security.

Solution:
As of today, the best way to do this is as follows:
Implement or import sqlalchemy base
from sqlalchemy.ext.declarative import declarative_base
base = declarative_base()
class Base(base):
__abstract__ = True
uid = Column(Integer, primary_key=True, autoincrement=True)
Register the external base:
from flask_sqlalchemy import SQLAlchemy
from model.base import Base
app = Flask(__name__)
db = SQLAlchemy(app, model_class=Base)
Archived for posterity:
I spent a lot of time looking for an answer to this. This is a lot easier to do today than it was when I originally asked the question, but it still isn't exactly simple.
For anyone who decides to do security themselves, I recommend the following excellent exposition of common design patterns which use flask, but which avoid employing unnecessary dependencies like flask-security:
https://exploreflask.com/users.html
UPDATE:
For anyone interested, a patch has been in the works for some time related to this. As of now it still isn't released, but you can check its progress here:
https://github.com/mitsuhiko/flask-sqlalchemy/pull/250#issuecomment-77504337
UPDATE:
I have taken the code from the above mentioned patch and created a local override for the SQLAlchemy object which allows one to register an external base. I think this is the best option available until such time as FSA gets around to adding this officially. Here is the code from that class for anyone interested. Tested working with Flask-SqlAlchemy 2.2
Patching in register_external_base:
import flask_sqlalchemy
'''Created by Isaac Martin 2017. Licensed insofar as it can be according to the standard terms of the MIT license: https://en.wikipedia.org/wiki/MIT_License. The author accepts no liability for consequences resulting from the use of this software. '''
class SQLAlchemy(flask_sqlalchemy.SQLAlchemy):
def __init__(self, app=None, use_native_unicode=True, session_options=None,
metadata=None, query_class=flask_sqlalchemy.BaseQuery, model_class=flask_sqlalchemy.Model):
self.use_native_unicode = use_native_unicode
self.Query = query_class
self.session = self.create_scoped_session(session_options)
self.Model = self.make_declarative_base(model_class, metadata)
self._engine_lock = flask_sqlalchemy.Lock()
self.app = app
flask_sqlalchemy._include_sqlalchemy(self, query_class)
self.external_bases = []
if app is not None:
self.init_app(app)
def get_tables_for_bind(self, bind=None):
"""Returns a list of all tables relevant for a bind."""
result = []
for Base in self.bases:
for table in flask_sqlalchemy.itervalues(Base.metadata.tables):
if table.info.get('bind_key') == bind:
result.append(table)
return result
def get_binds(self, app=None):
"""Returns a dictionary with a table->engine mapping.
This is suitable for use of sessionmaker(binds=db.get_binds(app)).
"""
app = self.get_app(app)
binds = [None] + list(app.config.get('SQLALCHEMY_BINDS') or ())
retval = {}
for bind in binds:
engine = self.get_engine(app, bind)
tables = self.get_tables_for_bind(bind)
retval.update(dict((table, engine) for table in tables))
return retval
#property
def bases(self):
return [self.Model] + self.external_bases
def register_base(self, Base):
"""Register an external raw SQLAlchemy declarative base.
Allows usage of the base with our session management and
adds convenience query property using self.Query by default."""
self.external_bases.append(Base)
for c in Base._decl_class_registry.values():
if isinstance(c, type):
if not hasattr(c, 'query') and not hasattr(c, 'query_class'):
c.query_class = self.Query
if not hasattr(c, 'query'):
c.query = flask_sqlalchemy._QueryProperty(self)
# for name in dir(c):
# attr = getattr(c, name)
# if type(attr) == orm.attributes.InstrumentedAttribute:
# if hasattr(attr.prop, 'query_class'):
# attr.prop.query_class = self.Query
# if hasattr(c , 'rel_dynamic'):
# c.rel_dynamic.prop.query_class = self.Query
To be used like so:
app = Flask(__name__)
db = SQLAlchemy(app)
db.register_base(base)