def get_engine():
engine = create_engine('mysql+mysqlconnector://...my_conn_string...', echo=True)
return engine
def generic_execute(sql):
db = get_engine()
connection = db.connect()
connection.execute(sql)
The code above executes the query properly but appears to hang infinitely.
How does one properly "close" or "kill" this connection? Thank you very much!
As you said the connection need to be needs to be closed as stated by the documentation.
So after you are done executing the sql query you need to call:
connection.close()
Also if you are done with the engine db you can call db.dispose() to clean everything.
Related
Here is my python flask code:
from flask import *
import mysql.connector
app = Flask(__name__)
conn = mysql.connector.connect(
host="<my db host>",
user="<my db user>",
password = "<my db password>",
database = '<my db>'
)
#app.route('/posts/<int:post_id>')
def get_post(post_id):
with conn.cursor(dictionary=True) as cur:
cur.execute('select * from posts where ID=%s',(post_id,))
result = cur.fetchone()
ans=result['post_content']
return ans
app.run(debug=False,threaded=True,host='0.0.0.0',port=80)
Note how I don't create a new connection for each request. Instead, I use the same connection for all requests.
My question is: is there any potential problems in this approach?
You should not use the same connection, because it will stay open forever, only open an SQL connection when you need to use it, and close it afterward. Not doing so can lead to a lot of errors.
To get my feet wet with deploying Lambda functions on AWS, I wrote what I thought was a simple one: calculate the current time and write it to a table in RDS. A separate process triggers the lambda every 5 minutes. For a few hours it will work as expected, but after some time the connection starts to hang. Then, after a few hours, it will magically work again. Error message is
(pymysql.err.OperationalError) (2003, \"Can't connect to MySQL server on '<server info redacted>' (timed out)\")
(Background on this error at: http://sqlalche.me/e/e3q8)
The issue is not with the VPC, or else the lambda wouldn't run at all I don't think. I have tried defining the connection outside the lambda handler (as many suggest), inside the handler, closing the connection after every run; now, I have the main code running in a separate helper function the lambda handler calls. The connection is created, used, closed, and even explicitly deleted within a try/except block. Still, the intermittent connection issue persists. At this point, I am not sure what else to try. Any help would be appreciated; thank you! Code is below:
import pandas as pd
import sqlalchemy
from sqlalchemy import event as eventSA
# This function is something I added to fix an issue with writing floats
def add_own_encoders(conn, cursor, query, *args):
cursor.connection.encoders[np.float64] = lambda value, encoders: float(value)
def writeTestRecord(event):
try:
connStr = "mysql+pymysql://user:pwd#server.host:3306/db"
engine = sqlalchemy.create_engine(connStr)
eventSA.listen(engine, "before_cursor_execute", add_own_encoders)
conn = engine.connect()
timeRecorded = pd.Timestamp.now().tz_localize("GMT").tz_convert("US/Eastern").tz_localize(None)
s = pd.Series([timeRecorded,])
s=s.rename("timeRecorded")
s.to_sql('lambdastest',conn,if_exists='append',index=False,dtype={'timeRecorded':sqlalchemy.types.DateTime()})
conn.close()
del conn
del engine
return {
'success' : 'true',
'dateTimeRecorded' : timeRecorded.strftime("%Y-%m-%d %H:%M:%S")
}
except:
conn.close()
del conn
del engine
return {
'success' : 'false'
}
def lambda_handler(event, context):
toReturn = writeTestRecord(event)
return toReturn
I am in a project which needs to get datum from an oracle database.
The source code reaches the database by cx_Oracle.
Now I wanna find a way to make a fixture for the connection.
Is there an elegant way to make such a fixture?
Or could somebody recommend a pytest-plugin for Oracle connection?
Thanks in advance.
You can use SQLAlchemy for this. Just plug in your connection string into the create_engine string and create a fixture for the connection (and session) like this:
engine = create_engine('your connection string goes here with your login creds')
#pytest.fixture(scope='module')
def connection():
connection = engine.connect()
yield connection
connection.close()
You can read more about the cx_Oracle connection engine from the SQLAlchemy docs here:
Location
Your create_engine might look something like this:
engine = create_engine("oracle+cx_oracle://<username>:<password>#(DESCRIPTION = (LOAD_BALANCE=on) (FAILOVER=ON) (ADDRESS = (PROTOCOL = TCP)(HOST = <host>)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = devdb)))")
Which was pulled from this post
I'm a newbie to Python and Flask, and I use Oracle, when learning Flask tutorial, I code as follow, but it smells really bad, please help me with these questions, thanks a lot!
1) need I release connection to poll explicitly?
2) how can I implement poll acquire and release gracefully?
def get_dbpool():
if not hasattr(g, 'db_pool'):
g.dbPool = connect_db()
return g.dbPool
#app.teardown_appcontext
def close_db(error):
if hasattr(g, 'db_pool'):
g.dbPool.close()
#app.route('/')
def hello_world():
db = get_dbpool().acquire()
cursor=db.cursor()
sql=''
cursor.execute(sql)
rows = cursor.fetchall()
cursor.close()
get_dbpool().release(db)
return json.jsonify(combines=rows)
There is no need to release the connection to the pool explicitly unless you intend to keep processing for some time and don't need the connection any longer. cx_Oracle automatically releases the connection back to the pool when the connection goes out of scope (function ends), provided that you haven't implemented a circular reference to the connection, of course! In that case you would have to wait until garbage collection executes. Hopefully that answers your questions!
I've the code like this below. Is it necessary to close mysql connection because whenever my home page is requested, a new sql connection will be created?
I randomly get Connection Limit error. But I'm not sure if the DB connection is the problem.
#app.route("Home", methods=["GET"])
def get_home_page():
db = mysql.connect(host, user, password, db_name, charset='utf8', use_unicode=True)
...
It is good practice to close the connection. You can put your codes inside a try..finally block.
#app.route("Home", methods=["GET"])
def get_home_page():
db = mysql.connect(host, user, password, db_name, charset='utf8', use_unicode=True)
try:
... do something ...
finally:
db.close()
from my experience, close session after use it take significant difference amount of time to response in api whom i've experienced in flask
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
try:
db.session.query( Any Model ...
except:
finally:
db.close_all_sessions