We Keep Coding

Configuration handling in Flask for MySQL database connection

I am new to Flask and currently exploring the official tutorial available in https://flask.palletsprojects.com/en/2.0.x/tutorial/factory/.
The example creates a blog based on a SQLite database. The database is defined in the following statement:
app.config.from_mapping(
SECRET_KEY='dev',
DATABASE=os.path.join(app.instance_path, 'flaskr.sqlite'),
)
Program Code
flaskr/__init__.py
import os
from flask import Flask
def create_app(test_config=None):
# create and configure the app
app = Flask(__name__, instance_relative_config=True)
app.config.from_mapping(
SECRET_KEY='dev',
DATABASE=os.path.join(app.instance_path, 'flaskr.sqlite'),
)
if test_config is None:
# load the instance config, if it exists, when not testing
app.config.from_pyfile('config.py', silent=True)
else:
# load the test config if passed in
app.config.from_mapping(test_config)
# ensure the instance folder exists
try:
os.makedirs(app.instance_path)
except OSError:
pass
# a simple page that says hello
#app.route('/hello')
def hello():
return 'Hello, World!'
return app
However, instead of am SQLite database, I want to use a MySQL database for storing information. How can I define the MySQL configuration at this stage?
I have modified the following code in the db.py file of the tutorial:
https://flask.palletsprojects.com/en/2.0.x/tutorial/database/
import mysql.connector
import click
from flask import current_app, g
from flask.cli import with_appcontext
# Function to create a database connection.
def get_db():
if 'db' not in g:
g.db=mysql.connector.connect(host="localhost",user="dbuser",password="database123",database="testdb")
return g.db
# Function to close an existing database connection.
def close_db(e=None):
db=g.pop('db',None)
if db is not None:
db.close()
# Function to initialize the database from a script.
def init_db():
db = get_db()
with open('schema.sql', 'r') as f:
with db.cursor() as cursor:
cursor.execute(f.read(), multi=True)
db.commit()
#click.command('init-db')
#with_appcontext
def init_db_command():
# Delete existing data and re-create tables.
init_db()
click.echo('Initialized the database.')
# Register with the application.
def init_app(app):
app.teardown_appcontext(close_db)
app.cli.add_command(init_db_command)
Can someone please provide some guidance on how to setup a MySQL database connection?

I did some digging around and figured that it is not mandatory to define the MySQL database connection inside the factory function for the application to work. I removed the database parameter values from app.config.from_mapping and only kept the database connection string inside the actual db.py file. So far, the application has been working fine.
app=Flask(__name__, instance_relative_config=True)
app.config.from_mapping(SECRET_KEY='development',)

Here you can use pymysql to manage your connections. Below is an example showing that how you can manage the DB connection.
import pymysql
def db_connect():
conn = pymysql.connect(user='Your user',
password="Your password",
host="Your host",
database="Your database",
cursorclass=pymysql.cursors.DictCursor)
return conn
def db_close(conn):
if conn and conn.open:
conn.close()
class DatabaseHandler(object):
def __init__(self):
self.conn = None
def __enter__(self):
self.conn = db_connect()
return self.conn
def __exit__(self, exc_type, exc_val, exc_tb):
db_close(self.conn)
You can use it as,
with DatabaseHandler() as db_conn:
Any database operation with db_conn

cx_oracle persistent connection on flask+apache+mod_wsgi

I have deployed my flask application on apache+mod_wsgi
I'm using WSGI Daemon mode and have this config in apache httpd.conf:
WSGIDaemonProcess flask_test user=apache group=apache threads=20
For simplicity lets say for each request, I need to execute a query to insert data into Oracle DataBase.
So in my flask application, I have done something like this:
# DB.py
import cx_Oracle
class DB:
def __init__(self, connection_string):
self.conn = cx_Oracle.connect(connection_string, threaded=True)
def insert(query):
cur = self.conn.cursor()
cur.execute(query)
cur.close()
self.conn.commit()
# flask_app.py
from flask import Flask, request, jsonify
from DB import DB
app = Flask(__name__)
db = DB(connection_string)
#app.route("/foo", methods=["POST"])
def foo():
post_data = request.get_json()
# parse above data
# create insert query with parsed data values
db.insert(insert_processed_data_QUERY)
# generate response
return jsonify(response)
When I start the apache+mod_wsgi server, the DB object is created and the DB connection is established.
For all incoming requests, the same DB object is used to execute insert query.
So far this works fine for me. However my concern is that if there are no requests for a long period of time, the DB connection might time out, and then my app will not work for a new request when it comes.
I've been monitoring my application and have observed that the DB connection persists for hours and hours. But I'm pretty sure it might timeout if there is no request for 2-3 days(?)
What would be the correct way to ensure that the DB connection will stay open forever? (i.e. as long as the apache server is running)

Use a pool instead of a standalone connection. When you acquire a connection from the pool it will check to see if the connection is no longer valid and automatically dispense a new one. So you need something like this:
pool = cx_Oracle.SessionPool(user=user, password=password, dsn=dsn, min=1,
max=2, increment=1)
Then in your code you need to do the following:
with pool.acquire() as connection:
# do what you need to do with the connection

Setting Up Flask DB2 connection

I am trying to connect my flask application to a DB2 database,but somehow I am unable to do. here is what I am trying to do,
I am using flask extension flask_db2 for this purpose and below is how I am trying to connect
from flask import Flask
from flask_db2 import DB2
app = Flask(__name__)
#app.route('/')
def index()
cur = db.connection.cursor("DATABASE=name;HOSTNAME=host;PORT=60000;PROTOCOL=TCPIP;UID=username;PWD=password)

from flask import Flask
from flask_db2 import DB2
app = Flask(__name__)
app.config['DB2_DATABASE'] = 'sample'
app.config['DB2_HOSTNAME'] = 'localhost'
app.config['DB2_PORT'] = 50000
app.config['DB2_PROTOCOL'] = 'TCPIP'
app.config['DB2_USER'] = 'db2inst1'
app.config['DB2_PASSWORD'] = 'db2inst1'
db = DB2(app) #You forgot that
#app.route('/')
def index():
cur = db.connection.cursor()
cur.execute(...)
First: You forgot to have an object to use the flask db2 module.
The DB2(app) is a constructor for your Flask extension and will bind your application to this module.
where as
db = DB2(app)
Initialises your db object with help of your extension's constructor which you can further use for your Database operations.
Secondly: use app configs to play with DB2 credentials. (This step should be done before you bind the app)
Third: You can execute your query like this
cur.execute("SELECT id FROM users") #something like that
I hope I have clarified :D just drop message. Ill try helping more.

Flask-MySQL gives "closing a closed connection" error the second time a view runs

I am using Flask-MySQL to connect to my database in a view. The view works the first time I go to it, but when I go to it the second time it always crashes with the error:
ProgrammingError: closing a closed connection
Why am I getting this error? How do I connect successfully the second time?
from flask import Flask, render_template, request
from flaskext.mysql import MySQL
app = Flask(__name__)
#app.route('/hello/', methods=['POST'])
def hello():
app.config['MYSQL_DATABASE_USER'] = 'root'
app.config['MYSQL_DATABASE_PASSWORD'] = 'xxx'
app.config['MYSQL_DATABASE_DB'] = 'pies'
app.config['MYSQL_DATABASE_HOST'] = 'localhost'
query = request.form['yourname']
mysql = MySQL(app)
conn = mysql.connect()
with conn as cursor:
try:
cursor.execute(query)
name = str(cursor.fetchone())
except:
name = "SQL is wrong"
conn.close()
return render_template('form_action.html', name=name)
if __name__ == '__main__':
app.run(debug=True)

You should not be initializing the extension during every request. Create it during app setup, then use it during the requests. Setting configuration should also be done during app setup.
The extension adds a teardown function that executes after every request and closes the connection, which is stored as a threadlocal. Since on the second request you've registered the extension multiple times, it is trying to close the connection multiple times.
There's no need to call connect, the extension adds a before request function that does that. Use get_db to get the connection for the request. Since the extension closes this connection for you, don't call close either.
from flask import Flask
from flaskext.mysql import MySQL
MYSQL_DATABASE_USER = 'root'
MYSQL_DATABASE_PASSWORD = 'xxx'
MYSQL_DATABASE_DB = 'pies'
MYSQL_DATABASE_HOST = 'localhost'
app = Flask(__name__)
app.config.from_object(__name__)
mysql = MySQL(app)
#app.route('/hello/')
def hello():
conn = mysql.get_db()
# ...
Note that Flask-MySQL is using a very old extension pattern that is no longer supported by Flask. It also depends on an unsupported mysql package that does not support Python 3. Consider using Flask-MySQLdb instead.

flask sqlalchemy keeps Postgres transaction idle

Im writting a flask api using flaskrestful,sqlalchemy, Postgres, nginx,uwsgi. Im a newbie to python.These are my configuration
database.py
from cuewords import app
from flask.ext.sqlalchemy import SQLAlchemy
from sqlalchemy.pool import NullPool
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import create_engine
from sqlalchemy import Column, Integer, String , Text , Boolean , DateTime, MetaData, Table ,Sequence
from sqlalchemy.dialects.postgresql import JSON
Base = declarative_base()
db_name="postgresql+psycopg2://admin:password#localhost:5434/database_name"
from sqlalchemy.orm import sessionmaker
engine = create_engine(db_name,poolclass=NullPool ,echo=True)
Session = sessionmaker(autocommit=False ,bind=engine)
connection = engine.connect()
metadata = MetaData()
api.py
class Webcontent(Resource):
def post(self):
session=Session()
...assign some params...
try:
insert_data=Websitecontent(site_url,publisher_id)
session.add(insert_data)
session.commit()
Websitecontent.update_url(insert_data.id,session)
except:
session.rollback()
raise
finally:
return "Data created "
session.close()
else:
return "some value"
Here im first saving the just the url then saving all the content of the site using boilerpipe later .Idea is to move to queue later
model.py
class Websitecontent(Base):
#classmethod
def update_url(self,id,session):
existing_record=session.query(Websitecontent).filter_by(id=int(id)).first()
data=Processing.processingContent(str(existing_record.url))
#boilerpipe processing the content here
#assigning some data to existing record in session
session.add(existing_record)
session.commit()
Websitecontent.processingWords(existing_record,session)
#classmethod
def processingWords(self,record,session)
...processing
Websitecontent.saveKeywordMapping(session,keyword_url,record)
#classmethod
def saveKeywordMapping(session,keyword_url,record)
session.commit()
session.close()
So this code works perfectly in locally but its doesnt work in production .So when i check pag_stat_activity it show the state "idle in transaction". The app hangs then i have to restart the servers. i dont get it why session.close() does not close the pool connection why its keeping psql transaction state busy . Guys any help would be really appreciated.

You are returning before closing the session:
return "Data created "
session.close()
I think returning inside finally might swallow the exception, as well.