Noobish question to be sure. I have tried to move some common code into a separate module and use the code as an imported function. The code makes a MySQL query through MySQLdb. When the function is part of the main script, it runs just fine. When I import the function from a separate module, the function fails because the cursor object is no longer defined. Is there a way to import functions without defining a separate cursor object for just the imported function?
Here is an coded example. This works:
import MySQLdb
#from mod2 import lookup_value
def get_db_connection(database_name):
db = MySQLdb.connect('localhost', 'user', 'pswrd', database_name)
cur = db.cursor()
return db, cur
def lookup_value(user_name):
query = "SELECT COUNT(*) FROM x_user_%s" % (user_name)
cur.execute("%s" % (query))
return cur.fetchone()
db_name = 'mg_test' # database name
user_name = 'test' # name of a specific table in the database
db, cur = get_db_connection(db_name)
value = lookup_value(user_name)
When I move the code for lookup_value to a second file, and import it ('from mod2 import lookup_value'), the code fails because the cursor object is undefined. The imported version of lookup_value only works if I create a cursor object for its use. This seems very inefficient. What is the best way to handle this problem?
That's because lookup_value searches for the cur within the file you import it in. You could put this all in a class.
class DB():
def __init__(self,database_name):
db = MySQLdb.connect('localhost', 'user', 'pswrd', database_name)
self.cur = db.cursor()
def lookup_value(self,user_name):
query = "SELECT COUNT(*) FROM x_user_%s"
self.cur.execute(query, (user_name,))
self.result = self.cur.fetchone()
return self.result
now you can do
....
db = DB(db_name)
value = db.lookup_value(user_name)
when you import the DB from mod2 import DB the last part should still work.
Also take note of how i execute the query in lookup_value this ensures the data is sanitized
You should pass a cursor variable to your functions if you want them to be independent.
In any case you should use in a function only local variables and variables passed as parameters.
Related
Instead of using:
import sqlite3
conn = sqlite3.connect(':memory:')
c = conn.cursor()
c.execute(...)
c.close()
would it be possible to use the Pythonic idiom:
with conn.cursor() as c:
c.execute(...)
It doesn't seem to work:
AttributeError: __exit__
Note: it's important to close a cursor because of this.
You can use contextlib.closing:
import sqlite3
from contextlib import closing
conn = sqlite3.connect(':memory:')
with closing(conn.cursor()) as cursor:
cursor.execute(...)
This works because closing(object) automatically calls the close() method of the passed in object after the with block.
A simpler alternative would be to use the connection object with the context manager, as specified in the docs.
with con:
con.execute(...)
If you insist on working with the cursor (because reasons), then why not make your own wrapper class?
class SafeCursor:
def __init__(self, connection):
self.con = connection
def __enter__(self):
self.cursor = self.con.cursor()
return self.cursor
def __exit__(self, typ, value, traceback):
self.cursor.close()
You'll then call your class like this:
with SafeCursor(conn) as c:
c.execute(...)
Adding to sudormrfbin's post. I've recently experienced an issue where an INSERT statement wasn't committing to the database. Turns out I was missing the with context manager for just the Connection object.
Also, it is a good practice to always close the Cursor object as well, as mentioned in this post.
Therefore, use two contextlib.closing() methods, each within a with context manager:
import contextlib
import sqlite3
# Auto-closes the Connection object
with contextlib.closing(sqlite3.connect("path_to_db_file")) as conn:
# Auto-commit to the database
with conn:
# Auto-close the Cursor object
with contextlib.closing(conn.cursor()) as cursor:
# Execute method(s)
cursor.execute(""" SQL statements here """)
Using Python 2.7.12 and package cx_Oracle I'm trying to create an extended class of the what the package call OracleCursor. I simply want to inherit the methods from the superclass and extend with some custom methods.
First I get the OracleCursor by
import cx_Oracle
conn = cx_Oracle.connect(username, password, dsn)
cursor = conn.cursor()
and I then have the following
>>> type(cursor)Out[6]:
OracleCursor
>>> isinstance(cursor, cx_Oracle.Cursor)
True
One would think that it is achieved by
class ExtendedCursor(cx_Oracle.Cursor):
def hello_world(self):
print('Hello world')
extended = ExtendedCursor(cursor)
but I get TypeError: argument 1 must be cx_Oracle.Connection, not OracleCursor. To me that error doesn't make sense. Also, I can't use OracleCursor as my superclass since it isn't recognized as a class.
The cursor is returned from the Connection object. You need to create a custom connection that returns your ExtendedCursor.
import cx_Oracle as cxo
class MyCursor(cxo.Cursor):
def helloWorld(self):
print "helloWorld"
class MyConnection(cxo.Connection):
def cursor(self):
return MyCursor(self)
if __name__ == '__main__':
conStr = '<user>/<password>#127.0.0.1:1521/xe'
db = MyConnection(conStr)
c = db.cursor()
print c
c.execute('select 1+1 from dual')
print(c.fetchall())
c.helloWorld()
returns:
<__main__.MyCursor on <__main__.MyConnection to ...#127.0.0.1:1521/xe>>
[(2,)]
helloWorld
How I can pass the data from object oriented programming to mysql in python? Do I need to make connection in every class?
Update:
This is my object orinted
class AttentionDataPoint(DataPoint):
def __init__ (self, _dataValueBytes):
DataPoint._init_(self, _dataValueBytes)
self.attentionValue=self._dataValueBytes[0]
def __str__(self):
if(self.attentionValue):
return "Attention Level: " + str(self.attentionValue)
class MeditationDataPoint(DataPoint):
def __init__ (self, _dataValueBytes):
DataPoint._init_(self, _dataValueBytes)
self.meditationValue=self._dataValueBytes[0]
def __str__(self):
if(self.meditationValue):
return "Meditation Level: " + str(self.meditationValue)
And I try to get the data to mysql using this coding.
import time
import smtplib
import datetime
import MySQLdb
db = MySQLdb.connect("192.168.0.101", "fyp", "123456", "system")
cur = db.cursor()
while True:
Meditation_Level = meditationValue()
Attention_Level = attentionValue()
current_time = datetime.datetime.now()
sql = "INSERT INTO table (id, Meditation_Level, Attention_Level, current_time) VALUES ('test1', %s, %s, %s)"
data = (Meditation_Level, Attention_Level, current_time)
cur.execute(sql, data)
db.commit()
db.close()
Update:
DataPoint
class DataPoint:
def __init__(self, dataValueBytes):
self._dataValueBytes = dataValueBytes
I am thinking of something like a class that has a function for connection and you just pass the instance of that class to every class where you need the connection (here the db instance), or may be not!
It's just an idea that I used in a other languages.
A few options:
Implement a save/load method for each DataPoint child to build the select/insert/update sql commands according with the needs of each class. Here is a basic tutorial: Python MySQL Database Access
Implement a class to build the sql commands above automatically from your class attributes.
The above is called Object Relational Mapping and, so, you don't need to reinvent the wheel: https://www.fullstackpython.com/object-relational-mappers-orms.html
I'd go with the first if it is a simple project with no chance to become big and with an ORM tool for all the other cases.
The most popular for python is SQLAlchemy:
http://docs.sqlalchemy.org/en/latest/orm/tutorial.html
This works okay (some code ommited for brevity):
# main.py
cursor = db.cursor()
cursor.execute('foo')
As does this:
# other.py
def do_something(script, cursor):
cursor.execute(script)
# main.py
from other import do_something
cursor = db.cursor()
do_something('foo', cursor)
But (as I understand it) the "lower" scope of the function should be able to access the "higher" (global?) scoped cursor - why should I need to pass the cursor as an argument on my function? So I tried this:
# other.py
def do_something(script):
cursor.execute(script)
# main.py
from other import do_something
cursor = db.cursor()
do_something('foo')
Which returns:
NameError: global name 'cursor' is not defined
I thought "maybe running a query against the cursor is a write operation, not a read" and tried:
# other.py
def do_something(script):
global cursor
cursor.execute(script)
# main.py
from other import do_something
cursor = db.cursor()
do_something('foo')
Same error. What am I missing?
EDIT: It sounds like "how do I make variable names global across different modules" is the wrong question. The right question - if I have a primary handler, a SQL cursor and a file of common functions, how should I structure my files / imports?
try this code
# other.py
def do_something(script):
global cursor
cursor.execute(script)
# main.py
from other import do_something
import other
other.cursor = db.cursor()
do_something(script)
My English is not good, so you can read these answers
Global Variable from a different file Python
Using global variables between files?
make variable global to multiple files in python
How can I perform post processing on my SQL3 database via python? The following code doesn't work, but what I am trying to do is first create a new database if not exists already, then insert some data, and finally execute the query and close the connection. But I what to do so separately, so as to add additional functionality later on, such as delete / updater / etc... Any ideas?
class TitlesDB:
# initiate global variables
conn = None
c = None
# perform pre - processing
def __init__(self, name):
import os
os.chdir('/../../')
import sqlite3
conn = sqlite3.connect(name)
c = conn.cursor()
c.execute('CREATE TABLE IF NOT EXISTS titles (title VARCHAR(100) UNIQUE)')
# insert a bunch of new titles
def InsertTitles(self, list):
c.executemany('INSERT OR IGNORE INTO titles VALUES (?)', list)
# perform post - processing
def __fina__(self):
conn.commit()
conn.close()
You could create a context manager to do the pre- and postprocessing.
import contextlib
#contextlib.contextmanager
def titles_cursor():
# perform pre - processing
conn = sqlite3.connect(name)
c = conn.cursor()
c.execute('CREATE TABLE IF NOT EXISTS titles (title VARCHAR(100) UNIQUE)')
yield c
# perform post - processing
conn.commit()
conn.close()
Use it in a with statement:
with titles_cursor() as c:
c.executemany('INSERT OR IGNORE INTO titles VALUES (?)', list)
First, wouldn't it be better to avoid having the sql connection inside the __init__?
You will have a problem if you want to use this class in the same instance after using __fina__.
You could have it in another method and call it and call the connection closing method when needed and commit after each method is executed.
Here is what I use : Create a class method that connects to the db and executes a query from an argument,commits and closes connection and you pass any query you want executed as an argument of that method.You can simply call this method anytime you want.
And the best about this is that you can create a method that passes multiple queries as arguments before closing db connection.
This is specially usefull if have to use sql connections to the same db in another class without using a set of methods each time you need to execute a sql query.
Here is a little example I used with MySQLdb module, It's pretty simple but it worked.
import MySQLdb
class DbQuery:
'''Here is the class I talked about'''
def __init__(self):
'''You can define the main queries here but it's not necessary
They can be global variables
If you don't have class dependency from which you get variables
you might not even need to define __init__'''
def Sql_Connect(self):
self.db = MySQLdb.connect("localhost","root","","data_db" )
self.cursor = db.cursor()
def Sql_Commit(self):
self.db.commit()
print "Info : Database updated"
except:
self.db.rollback()
print "Error : Database rollback"
self.db.close()
def Query(self,query):
self.Sql_Connect()
try :
self.cursor.execute(query)
self.Sql_Commit()
The only thing important is to remember the query structure.