We Keep Coding

python mysql connection auto connect on error

I have a problem with my python mysql connection which I need help with.
My setup is two Pi's running servers on each one. One Pi (SolartPi) has Mysql database collecting data. The other pi (OfficePi) is connecting to the solarPi database to retrieve and update data over a network connection.
My main script works all ok until I have to reboot the SolarPi for a maintenance or power problem and the connection to the OfficePi is lost. The python script on the officePi then goes into a fault loop "2006, MYSQL Server has gone away" Below is a sample of this script.
import MySQLdb
connSolar = MySQLdb.connect("192.xxx.x.x", "external", "xxxxx", "xxxxx")
#eternal connection to solar pi database
cursSolar = connSolar.cursor()
while 1:
try:
cursSolar.execute("SELECT * FROM dashboard")
connSolar.commit()
for reading in cursSolar.fetchall():
heatingDemand = reading[2] #get heating demand from dB
print heatingDemand
except (MySQLdb.Error, MySQLdb.Warning) as e:
print (e)
connSolar.close()
So I tried rewriting this with the script from stackoverflow and a web site as shown below, but this now terminates the program when SolarPi is rebooted with the following error
_mysql_exceptions.OperationalError: (2003, 'Can\'t connect to MySQL server on \'192.xxx.x.x' (111 "Connection refused")')
import MySQLdb
class DB:
con = None
def connect(self):
self.conn = MySQLdb.connect("192.xxx.x.x", "xxxxx", "xxxxxx", "house") #eternal connection to solar pi database
def query(self, sql):
try:
cursor = self.conn.cursor()
cursor.execute(sql)
except (AttributeError, MySQLdb.OperationalError):
self.connect()
cursor = self.conn.cursor()
cursor.execute(sql)
return cursor
while 1:
db = DB()
sql = "SELECT * FROM dashboard"
cur = db.query(sql)
for reading in cur.fetchall():
heatingDemand = reading[2] #get heating demand from dB
print heatingDemand
Is there a way for the OfficePi to keep trying to connect to SolarPi mysql database when it has shut down.

Change your code to check a valid connection each loop otherwise pass:
import MySQLdb
class DB:
def connect(self):
try:
self.conn = MySQLdb.connect("192.xxx.x.x", "xxxxx", "xxxxxx", "house")
except (MySQLdb.Error, MySQLdb.Warning) as e:
print (e)
self.conn = None
return self.conn
def query(self, sql):
try:
cursor = self.conn.cursor()
cursor.execute(sql)
except (AttributeError, MySQLdb.OperationalError):
self.connect()
cursor = self.conn.cursor()
cursor.execute(sql)
return cursor
while 1:
db = DB()
conn = db.connect()
if conn:
sql = "SELECT * FROM dashboard"
cur = db.query(sql)
for reading in cur.fetchall():
heatingDemand = reading[2] #get heating demand from dB
print heatingDemand

AssertRaises Doesn't Work with PsycoPg2

I am trying to test that an error is thrown in unittest with Python 3.6 when working with psycopg2. In this case, the database named foo does not existm, and it should throw an OperationalError because the database is not a valid one.
Here is a sample test case:
# -*- coding: utf-8 -*-
import unittest
import psycopg2
import logging
def pull_pg_data(conn, cursor, first, last):
try:
cursor.execute('SELECT first_name, last_name, email, street FROM user_data WHERE first_name =%s AND last_name=%s',(first, last))
except psycopg2.OperationalError as msg:
logging.error(msg)
raise
conn.commit()
result = cursor.fetchall()
return result
class SampleTestCases(unittest.TestCase):
def setUp(self):
try:
self.conn = psycopg2.connect(dbname='postgres', user='postgres', host='localhost', port=5432,
connect_timeout=10)
except psycopg2.OperationalError as msg:
logging.critical(msg)
raise
self.conn.autocommit = True
self.cursor = self.conn.cursor()
self.cursor.execute('DROP TABLE IF EXISTS user_data', [])
self.cursor.execute('CREATE TABLE user_data (first_name TEXT, last_name TEXT, email TEXT, street TEXT)', [])
self.cursor.execute("INSERT INTO user_data(first_name, last_name, email, street) VALUES('someFirstName', 'someLastName', 'some#example.com', '123 road')",[])
def test_pull_pg_data_fail(self):
conn = psycopg2.connect(dbname='foo', user='postgres', host='localhost', port=5432, connect_timeout=10)
cursor = conn.cursor()
self.assertRaises(psycopg2.OperationalError, pull_pg_data, conn, cursor, 'someFirstName', 'someLastName')
if __name__ == '__main__':
unittest.main()
When I run the test, it does indeed throw an error, but it fails the test. I don't know what I am doing incorrectly with assertRaises, because I do want to test that the error is thrown when I try to connect to a database that does not exist.

"Error Locating Server/Instance Specified" when connecting to SQL Server

I'm using python 2.7 have the code below to connect sql instances
class SqlConnector:
def __init__(self, driver, sqlserver_ip, database, **kwargs):
if 'port' in kwargs:
conn_string = 'DRIVER='+ driver + ';SERVER='+ sqlserver_ip +';PORT=' + kwargs['port'] + ';DATABASE=' + database + ';trusted_connection=yes;'
else:
conn_string = 'DRIVER='+ driver + ';SERVER='+ sqlserver_ip + ';DATABASE=' + database + ';trusted_connection=yes;'
print conn_string
self.conn = pypyodbc.connect(conn_string)
self.cur = self.conn.cursor()
def __enter__(self):
return self
def query(self, query_string):
self.cur.execute(query_string)
return
def get_all_table_columns(self):
columns = [column[0] for column in self.cur.description]
return columns
def get_all_table_rows(self):
rows = self.cur.fetchall()
return rows
def __repr__(self):
conn_string = 'DRIVER='+ driver + ';SERVER='+ sqlserver_ip + ';DATABASE=' + database + ';trusted_connection=yes;'
return conn_string
def __exit__(self, exc_type, exc_val, exc_tb):
if self.conn or self.cur:
# close cursor
self.cur.close()
# close connection
self.conn.close()
My SQL instance is like "hostname\PEWKA", but error below saying
(u'08001', u'[08001] [Microsoft][SQL Server Native Client 11.0]SQL
Server Network Interfaces: Error Locating Server/Instance Specified
[xFFFFFFFF]. ')
is this the correct way to connect SQL instance using pypyodbc? can't really find too much info out of it.
Anyone can shed some light will be very appreciated.
Thanks

Connecting by instance name
Provide the server name and instance name separated by a backslash, e.g., SERVER=hostname\PEWKA. Do not supply a port number. If the SQL Server instance is on a remote machine then the SQL Server Browser service must be running on that machine.
Connecting by port number
Provide the server name and port number separated by a comma, e.g., SERVER=hostname,49242. Do not supply the instance name. Note also that the SQL Server ODBC drivers do not support a PORT= parameter in the connection string.

single database connection throughout the python application (following singleton pattern)

My Question is what is the best way to maintain the single database connection in the entire application? Using Singleton Pattern? How?
Conditions that are needed to be taken care of:
In case of multiple requests, I should be using the same connection
In case connection is closed, create a new connection
If the connection has timed-out, on new request my code should create a new connection.
The driver to my Database is not supported by the Django ORM. And due to same driver related issues, I am using pyodbc to connect to the database. Right now I am having below class for creating and managing the DB connections:
class DBConnection(object):
def __init__(self, driver, serve,
database, user, password):
self.driver = driver
self.server = server
self.database = database
self.user = user
self.password = password
def __enter__(self):
self.dbconn = pyodbc.connect("DRIVER={};".format(self.driver) +\
"SERVER={};".format(self.server) +\
"DATABASE={};".format(self.database) +\
"UID={};".format(self.user) +\
"PWD={};".format(self.password) + \
"CHARSET=UTF8",
# "",
ansi=True)
return self.dbconn
def __exit__(self, exc_type, exc_val, exc_tb):
self.dbconn.close()
But the issue with this approach is that it will create new database connection for each query. What will be the better way to do it following singleton pattern? The way I can think of will hold the reference to the connection if the connection is closed. Something like:
def get_database_connection():
conn = DBConnection.connection
if not conn:
conn = DBConnection.connection = DBConnection.create_connection()
return conn
What will be the best way to achieve this? Any suggestion/ideas/examples?
PS: I was checking about using weakref which allows to create weak references to objects. I think it will be good idea to use weakref along with singleton pattern for storing the connection variable. This way I won't have to keep the connection alive when DB is not in use. What you guys say about this?

For now, I am going ahead with the singleton class approach. Anyone seeing the potential flaws in this, feel to mention them :)
DBConnector class for creating a connection
class DBConnector(object):
def __init__(self, driver, server, database, user, password):
self.driver = driver
self.server = server
self.database = database
self.user = user
self.password = password
self.dbconn = None
# creats new connection
def create_connection(self):
return pyodbc.connect("DRIVER={};".format(self.driver) + \
"SERVER={};".format(self.server) + \
"DATABASE={};".format(self.database) + \
"UID={};".format(self.user) + \
"PWD={};".format(self.password) + \
"CHARSET=UTF8",
ansi=True)
# For explicitly opening database connection
def __enter__(self):
self.dbconn = self.create_connection()
return self.dbconn
def __exit__(self, exc_type, exc_val, exc_tb):
self.dbconn.close()
DBConnection class for managing the connections
class DBConnection(object):
connection = None
#classmethod
def get_connection(cls, new=False):
"""Creates return new Singleton database connection"""
if new or not cls.connection:
cls.connection = DBConnector().create_connection()
return cls.connection
#classmethod
def execute_query(cls, query):
"""execute query on singleton db connection"""
connection = cls.get_connection()
try:
cursor = connection.cursor()
except pyodbc.ProgrammingError:
connection = cls.get_connection(new=True) # Create new connection
cursor = connection.cursor()
cursor.execute(query)
result = cursor.fetchall()
cursor.close()
return result

class DBConnector(object):
def __new__(cls):
if not hasattr(cls, 'instance'):
cls.instance = super(DBConnector, cls).__new__(cls)
return cls.instance
def __init__(self):
#your db connection code in constructor
con = DBConnector()
con1 = DBConnector()
con is con1 # output is True
Hope, above code will helpful.

Python: MySQL connection is open, but can't create cursor

I'm trying to open a cursor to a MySQL-DB. But I'm getting this error:
'NoneType' object has no attribute 'cursor'
Here is a small sourcecode:
class Sample:
def __init__(self):
self.conn = None
self.value = self.setValue()
def connect(self):
self.conn = MySQLdb.connect(...)
#cursor = self.conn.cursor()
#cursor.execute("SELECT ...")
#value = str(cursor.fetchone()[0])
#raise Exception(value)
#cursor.close() <- here everything is working fine
def setValue(self):
if (self.conn == None):
self.connect()
#raise Exception(self.conn.open)
cursor = self.conn.cursor() # ERROR: 'NoneType' object has no attribute 'cursor'
...
If I use the exception I get a 1 ... connection is open.
And if I do the cursor creation and the SQL statement in the 'connect' function everything is working well.
The strange this is, everything looks correct and for some other connections with the same functions everything is working well, too. I don't know how to solve this error. I hope someone can point me in the right direction.

I would change the statement that checks if the connection is open to both check if conn is none as well as if the connection is open. And because you always execute the setValue function I would recommend that you call the connect inside the__init__ function.
class Sample:
conn = None
def __init__(self):
self.connect()
self.value = self.setValue()
self.close()
def connect(self):
self.conn = MySQLdb.connect(...)
def close(self):
if self.conn:
self.conn.close()
def setValue(self):
if not self.conn and not self.conn.open:
self.connect()
cursor = self.conn.cursor()
Also, remember that with the Python MySQL Connector you need to call commit after you execute a insert or update statement.
cur = self.conn.cursor()
cur.execute("...")
self.conn.commit()