I'm writing a python CGI script that will query a MySQL database. I'm using the MySQLdb module. Since the database will be queryed repeatedly, I wrote this function....
def getDatabaseResult(sqlQuery,connectioninfohere):
# connect to the database
vDatabase = MySQLdb.connect(connectioninfohere)
# create a cursor, execute and SQL statement and get the result as a tuple
cursor = vDatabase.cursor()
try:
cursor.execute(sqlQuery)
except:
cursor.close()
return None
result = cursor.fetchall()
cursor.close()
return result
My question is... Is this the best practice? Of should I reuse my cursor within my functions? For example. Which is better...
def callsANewCursorAndConnectionEachTime():
result1 = getDatabaseResult(someQuery1)
result2 = getDatabaseResult(someQuery2)
result3 = getDatabaseResult(someQuery3)
result4 = getDatabaseResult(someQuery4)
or do away with the getDatabaseeResult function all together and do something like..
def reusesTheSameCursor():
vDatabase = MySQLdb.connect(connectionInfohere)
cursor = vDatabase.cursor()
cursor.execute(someQuery1)
result1 = cursor.fetchall()
cursor.execute(someQuery2)
result2 = cursor.fetchall()
cursor.execute(someQuery3)
result3 = cursor.fetchall()
cursor.execute(someQuery4)
result4 = cursor.fetchall()
The MySQLdb developer recommends building an application specific API that does the DB access stuff for you so that you don't have to worry about the mysql query strings in the application code. It'll make the code a bit more extendable (link).
As for the cursors my understanding is that the best thing is to create a cursor per operation/transaction. So some check value -> update value -> read value type of transaction could use the same cursor, but for the next one you would create a new one. This is again pointing to the direction of building an internal API for the db access instead of having a generic executeSql method.
Also remember to close your cursors, and commit changes to the connection after the queries are done.
Your getDatabaseResult function doesn't need to have a connect for every separate query though. You can share the connection between the queries as long as you act responsible with the cursors.
Related
Im using the mysql-connector-python library for a script I wrote to access a MySql 8 Database:
def get_document_by_id(conn,id):
cursor = conn.cursor(dictionary=True)
try:
cursor.execute("SELECT * FROM documents WHERE id = %s",(id,))
return cursor.fetchone()
except Exception as e:
print(str(e))
return {}
finally:
cursor.close()
Since i need to call this function multiple times during a loop, I was wondering about the following:
Does the act of creating/closing a cursor actually interact with my MySql-Database in any way or is it just used as an abstraction in python for grouping together SQL queries?
Thanks a lot for your help.
The doc says that clearly
For a connection obtained from a connection pool, close() does not
actually close it but returns it to the pool and makes it available
for subsequent connection requests.
You can also refer to Connector/Python Connection Pooling for further information.
In a database program what does these lines of code mean and do?
conn=sqlite3.connect(filename)
c=conn.cursor()
conn.commit()
You could think of conn = sqlite3.connect(filename) as creating a connection, or a reference, to that database specified in the filename. So anytime you carry out an action with conn, it will be an action performed on the database specified by filename.
c = conn.cursor() is a cursor object, which allows you to carry out SQL queries on the database. It is created using a call on the conn variable created earlier, and so is a cursor object for that specific database. This is most commonly useful for its .execute() method, which is used to execute SQL commands on the database.
conn.commit() 'commits' the changes to the database; that is, when this command is called, any changes that had been made by the cursor will be saved to the database.
This is the reproduced sample:
import mysql.connector
conn = mysql.connector.connect(
user='root', password='12347',
host='localhost')
def getCursor():
return conn.cursor()
def execQuery(cursor=getCursor()):
cursor.execute("SELECT 2")
cursor.fetchall()
cursor.close()
for i in range(4):
cursor = execQuery()
This code works without cursor.close(). But what I find weird is that this sample works even with cursor.close() with a simple change:
def execQuery():
cursor=getCursor()
cursor.execute("SELECT 2")
cursor.fetchall()
cursor.close()
By moving the default parameter to the body of the function.
I don't know if it's the best practice to close the cursor, so I can skip closing cursor while keeping the first form. If it's not the best practice to have a default parameter that uses return value of a function, I can go with the second form. But I want to why they act differently
It's like I'm having the same error as the following:
cursor.execute("SELECT 2")
cursor.fetchall()
cursor.close()
cursor.execute("SELECT 2")
It's like every call of execQuery is using the same cursor, so it gets blocked right at the second call.
When you need to connect to Database you need something like cursor. You need a cursor object to fetch results.
In the sample program when you run loop in range (4) it calls execQuery() . Looking into definition you can find def execQuery(cursor=getCursor()): the function takes input as cursor and by default it used getCursor() function which creates cursor everything when the loop is executed.
While in your program you are closing the cursor but not creating it again hence when second execute query comes there is no cursor present and the program throws an error.
I'm trying to get a cursor from stored procedure in txpostgres.
Psycopg2 has a named cursors which are working fine for it. But there is no curs = conn.cursor('name') statement in txpostgres.
Is there another way to get it ?
txpostgres doesn't have a named cursor feature. However, psycopg2's named cursors are just a convenience wrapper for PostgreSQL's cursors. I don't have much experience with stored procedures, but here's an example with a simple query:
#inlineCallbacks
def transaction(cursor):
yield cursor.execute('mycursor CURSOR FOR SELECT bigtable')
yield cursor.execute('FETCH ALL FROM mycursor')
data = yield cursor.fetchall()
conn.runInteraction(transaction)
I was doing a tutorial and came across a way to handle connections with sqlite3,
Then I studied about the WITH keyword and found out that it is an alternative to try,except,finally way of doing things
It was said that in case of file-handling, 'WITH' automatically handles closing of files and I thought similar with the connection as said in zetcode tutorial:-
"With the with keyword, the Python interpreter automatically releases
the resources. It also provides error handling." http://zetcode.com/db/sqlitepythontutorial/
so I thought it would be good to use this way of handling things, but I couldn't figure out why both (inner scope and outer scope) statements work? shouldn't the WITH release the connection?
import sqlite3
con = sqlite3.connect('test.db')
with con:
cur = con.cursor()
cur.execute('SELECT 1,SQLITE_VERSION()')
data = cur.fetchone()
print data
cur.execute('SELECT 2,SQLITE_VERSION()')
data = cur.fetchone()
print data
which outputs
(1, u'3.6.21')
(2, u'3.6.21')
I don't know what exactly the WITH is doing here(or does in general), so, if you will please elaborate on the use of WITH over TRY CATCH in this context.
And should the connections be opened and closed on each query? (I am formulating queries inside a function which I call each time with an argument) Would it be a good practice?
In general, a context manager is free to do whatever its author wants it to do when used. Set/reset a certain system state, cleaning up resources after use, acquiring/releasing a lock, etc.
In particular, as Jon already writes, a database connection object creates a transaction when used as a context manager. If you want to auto-close the connection, you can do
with contextlib.closing(sqlite3.connect('test.db')) as con:
with con as cur:
cur.execute('SELECT 1,SQLITE_VERSION()')
data = cur.fetchone()
print data
with con as cur:
cur.execute('SELECT 2,SQLITE_VERSION()')
data = cur.fetchone()
print data
You could also write your own wrapper around sqlite3 to support with:
class SQLite():
def __init__(self, file='sqlite.db'):
self.file=file
def __enter__(self):
self.conn = sqlite3.connect(self.file)
self.conn.row_factory = sqlite3.Row
return self.conn.cursor()
def __exit__(self, type, value, traceback):
self.conn.commit()
self.conn.close()
with SQLite('test.db') as cur:
print(cur.execute('select sqlite_version();').fetchall()[0][0])
https://docs.python.org/2.5/whatsnew/pep-343.html#SECTION000910000000000000000
From the docs: http://docs.python.org/2/library/sqlite3.html#using-the-connection-as-a-context-manager
Connection objects can be used as context managers that automatically commit or rollback transactions. In the event of an exception, the transaction is rolled back; otherwise, the transaction is committed:
So, the context manager doesn't release the connection, instead, it ensures that any transactions occurring on the connection are rolled back if any exception occurs, or committed otherwise... Useful for DELETE, UPDATE and INSERT queries for instance.