I am using Python 2.7.6 and MySqldb module. I have a MySQL query that crashes sometimes. It is hard to catch the rootcause for the time being. How can I avoid crashing the python script when executing the SQL query? How can I make it fail gracefully?
The code looks like something;
cursor.execute(query)
You should throw an exception:
try:
cursor.execute(query)
except mysql.connector.Error:
"""your handling here"""
Here is a link to the MySQL Python Dev guide:
You can handle run time errors by using try except block.
At last you must use finally for cleanups like close the connection , rollback , free all the used resources etc.
Here is the example ,
import mysql.connector
try:
cnx = mysql.connector.connect(user='scott', database='employees')
cursor = cnx.cursor()
cursor.execute("SELECT * FORM employees") # Syntax error in query
cnx.close()
except mysql.connector.Error as err:
print("Something went wrong: {}".format(err))
finally:
# cleanup (close the connection, etc...)
Related
I am attempting to execute some basic SQL via Python using the teradatasql module. The code appears to run and the SQL is executed: however, the execution of the Python itself ends with an error on the end of the code reproduced below. Currently, I need to run additional data preprocessing steps using pandas on the output of the SQL, but the larger program will not continue past the Operational Error (not even via a try/except block excepting the teradatsql.OperationalError). Therefore even though the SQL executes fine with this issue, I need to resolve it.
Any suggestions? Thank you!
Error:
teradatasql.OperationalError: 1 is not a valid connection pool handle
Code:
import teradatasql
import os
def refresh_table():
usr = ****1
with open(f'C:\\Users\\{usr}\\Documents\\my_td_password.txt', 'r') as my_pwd_f:
pw = my_pwd_f.read()
with teradatasql.connect(host = '*******2'
, user = usr
, password = pw
, ) as con:
with con.cursor() as cur:
with open('C:\\Users\\****1\\Documents\\test.sql', 'r') as my_sql:
sql_script = my_sql.read()
for sql_block in sql_script.split(';'):
try:
cur.execute(sql_block)
print("Block executed")
except ValueError:
print("Failure to execute block: ValueError")
finally:
print(sql_block)
my_sql.close()
print("SQL file closed")
con.close()
print("Connection closed")
refresh_table()
Fixed by removing con.close() from the end - as Fred pointed out, the with block implicitly closes the connection when it finishes execution
https://stackoverflow.com/users/11552426/fred
I have run into a very strange issue where my code below was working (I was able to add a table into the DB). After dropping a test table via the mysql workbench I am able to add a database and the code continues to run with no errors but no table is added. I have slimmed the SQL right down and attempted on different dbs. I have also restarted. Am I doing something wrong here? Any help is greatly appreciated.
import mysql.connector
import os
dirname = os.path.abspath('')
sql_filename = dirname + '\SQL_Creation\Test.sql'
class connectionsetup:
def __init__(self):
self.mydb = mysql.connector.connect(
host="localhost",
user="root",
password="")
try:
self.mycursor = self.mydb.cursor()
print('Conneciton Succesful')
self.mycursor.execute("CREATE DATABASE AP_Application_Db_test")
print('Database Created')
self.mycursor.execute('USE ap_application_db_test; Create TABLE test (SERIAL_NUMBER VARCHAR(255),VIOLATION_STATUS VARCHAR(255))', multi=True)
self.mydb.commit()
self.mydb.close()
except mysql.connector.Error as err:
print(err)
print("Error Code:", err.errno)
print("SQLSTATE", err.sqlstate)
print("Message", err.msg )
connectionsetup()
According to the docs, cursor.execute(..., multi=True)
[...] returns an iterator that enables processing the result of each statement
So the code needs to be like this:
for _ in self.mycursor.execute(multiple_sql_statements, multi=True):
pass
im trying to create a schema in postgres database using psycopg2.
For some reason the schema is not created and later on the code crashes because it tries to refer to the missing schema. The connection is set to auto commit mode, which definetly works because i can create a database with this specific connection.
For debugging purposes i have wrapped every step in it's own try/except statement.
Code is below, as it is right there, it does not raise any exceptions, just the follow up crashes because the schema is missing.
def createDB(dbName, connString):
conn = psycopg2.connect(connString)
conn.set_session(autocommit =True) # autocommit must be True sein, else CREATE DATABASE will fail https://www.psycopg.org/docs/usage.html#transactions-control
cursor = conn.cursor()
createDB = sql.SQL('CREATE DATABASE {};').format(
sql.Identifier(dbName)
)
createSchema = sql.SQL('CREATE SCHEMA IF NOT EXISTS schema2;')
searchpath = sql.SQL('ALTER DATABASE {} SET search_path TO public, schema2;').format(
sql.Identifier(dbName)
)
dropDB = sql.SQL('DROP DATABASE IF EXISTS {};').format(
sql.Identifier(dbName)
)
try:
cursor.execute(dropDB)
except Exception as e:
print('drop DB failed')
logging.error(e)
conn.close()
exit()
try:
cursor.execute(createDB)
except Exception as e:
print('create DB failed')
logging.error(e)
conn.close()
exit()
try:
cursor.execute(createSchema)
print('schema created')
except Exception as e:
print('create schema failed')
logging.error(e)
conn.close()
exit()
try:
cursor.execute(searchpath)
except Exception as e:
print('set searchpath failed')
logging.error(e)
conn.close()
exit()
conn.close()
Adding an explicit commit does not do the trick either.
What am i missing?
EDIT
I have added a small screenshot with the console logs. As you can see, the code below gets executed.
EDIT 2
Out of sheer curiosity, i have tried to execute this very SQL statement in pgadmin:
CREATE SCHEMA IF NOT EXISTS schema2
and it works just fine, which shows, that my SQL is not wrong, so back to square one.
EDIT 3 -- Solution
So i have come up with a solution, thank to you #jjanes for pointing me in the right direction. This function does not connect to a specific database, but the server as a whole, since im using it to create new databases, hence the connection string looks something like this :
user=postgres password=12345 host=localhost port=5432
Which allows me to perform server level operations like create and drop database. But schemas are a Database level operation. Moving the exact same logic to the part of the code which is connected to the newly created database works like a charm.
You create the schema in the original database specified by the connect string. Once you create the new database, you need to connect to it in order to work in it. Otherwise, you are just working in the old database.
I need to execute certain MySQL commands in a python script, which is a straight-forward task. For testing purposes I have boiled down the commands to this:
import mysql.connector
script = """
CREATE DATABASE `new_project`;
CREATE TABLE `new_project`.`category` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) NOT NULL,
UNIQUE KEY `unq_name` (`name`),
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4;
"""
connection = mysql.connector.connect(
host="localhost",
port="3306",
user="root",
passwd="somepassword",
)
cursor = connection.cursor()
try:
print("begin execution")
cursor.execute(script, multi=True)
warnings = cursor.fetchwarnings()
if warnings:
for warning in warnings:
print(warning)
connection.commit()
cursor.close()
connection.close()
print("connection closed")
except mysql.connector.Error as err:
print(err.msg)
The user credentials are replaced with the correct information when running the script.
The output of this script is
begin execution
connection closed
with no errors, warnings or other output. The database new_project is not created. When I run the same MySQL commands in another interface, they work as expected and create the database and the table.
I must be overlooking something very simple.
The documentation of execute mentions that the method returns an iterator with the results for each query when multi=True. It seems the queries are not doing anything until the iterator has been processed, regardless of commit(). However, CREATE statements do not produce any results and trying to iterate over the return value of execute results in an exception: generator raised StopIteration. This is related to a bug in the connector module and has been fixed in version 8.0.13 with support of python 3.7.
The solution is now to always iterate over the return value of execute, even if no return data is expected, and to upgrade the connector module. If the upgrade is not feasible it is possible to catch the failed iteration and continue.
The fixed code (including the part for earlier versions of the connector module) now looks something like this:
try:
results = cursor.execute(script, multi=True)
try:
for result in results:
pass
except Exception as e:
pass
warnings = cursor.fetchwarnings()
if warnings:
for warning in warnings:
# handle warning
connection.commit()
cursor.close()
connection.close()
except mysql.connector.Error as err:
# handle error
Try using password="somepassword" instead of passwd="somepassword". And remove multi=True. It'll create a warning but will still execute both of your statements.
I have a linux server and I would like to run a python script every day to run mysql procedures but I do not know how to run multiple procedures and put a condition if there is an error that it sends me an email with the description of the error. Here is my script with only one procedure:
#!/usr/bin/python
import MySQLdb
# Open database connection
db = MySQLdb.connect("localhost","user","password","bddname" )
# prepare a cursor object using cursor() method
cursor = db.cursor()
# execute SQL query using execute() method.
cursor.execute("CALL proc_commande_clts_detail7();")
# Fetch a single row using fetchone() method.
data = cursor.fetchone()
print "Database version : %s " % data
# disconnect from server
db.close()
Thank you for your help.
You can use callproc method to execute MySQL procedures
for proc_name in proc_name_list:
try:
result_args = cursor.callproc(proc_name, args=())
except Exception as e:
send_mail(str(e))
If you want to call multiple procedures, you can put callproc in some kind of loop and use try...catch for error handling.
wrapping them in try/except block and trigger email in except block?
Scheduling can be done through cron job.
import traceback
try:
cursor.execute("CALL proc_commande_clts_detail7();")
catch Exception as e:
email_msg = traceback.format_exc()
#send email logic