We Keep Coding

Insert into mysql database not committing changes (python)

I am trying to run the following code which executes with no issues. The sql that is produced is, "INSERT INTO account_login (groupone_account_id, login_date) VALUES ('100', '10:10:00') which has no syntax errors and executed successfully. But when I check the table, the id has not been inserted. I can complete select queries successfully.
The reason why I created database_connection is because it is an external connection and I wanted to isolate it to be able to test the databse connection easier.
def create_groupone_account_login(groupone_account_id):
groupone_account_login_created = False
cursor = database_connection("cursor")
time = datetime.utcnow().isoformat()[:-3] + 'Z'
sql_create_account_login = "INSERT INTO account_login (groupone_account_id, login_date) VALUES ('%s', '%s')" % (
groupone_account_id, time)
cursor.execute(sql_create_account_login)
connection = database_connection("connection")
connection.commit()
cursor.close()
groupone_account_login_created = True
return groupone_account_login_created
def database_connection(variable):
resp_dict = json.loads(get_secret())
endpoint = resp_dict.get('host')
username = resp_dict.get('username')
password = resp_dict.get('password')
database_name = resp_dict.get('dbname')
port = resp_dict.get('port')
connection = pymysql.connect(host=endpoint, user=username, passwd=password, db=database_name, port=port)
cursor = connection.cursor()
if variable == "connection":
return connection
else:
return cursor

ValueError: not enough values to unpack (expected 2, got 0) when mocking

I have a function for my database connection that returns both cursor and connection.
def database_connection():
resp_dict = json.loads(get_secret())
endpoint = resp_dict.get('host')
username = resp_dict.get('username')
password = resp_dict.get('password')
database_name = resp_dict.get('dbname')
port = resp_dict.get('port')
connection = pymysql.connect(host=endpoint, user=username, passwd=password, db=database_name, port=port)
cursor = connection.cursor()
return cursor, connection
def sql_query_insert(insert, user_id, time):
cursor, connection = database_connection()
sql_query = "INSERT INTO account_login (" + insert + ", login_date) VALUES ('" + str(user_id) + "' ,'" + time + "')"
cursor.execute(sql_query)
connection.commit()
return connection
I then use this to run various sql queries. Right now I am trying to write some test's for these sql queries but do not want to connection to a real database and want to mock the function database_connection(). However, I am having some trouble mocking the function as it return's two values. This is the test I am trying to build out. (p.s I know this will fail but I am trying to build out the test first before getting my expected values, and I will be updating the sql query insert function after figuring this out as as it is open to sql injections)
#mock.patch('lambda_function.database_connection', return_value=['cursor', 'connection'])
def test_sql_query_insert(self):
result = lambda_function.sql_query_insert("select", "table", "condition")
self.assertEqual(result, "test")
This is the error I am getting:
insert = 'select', user_id = 'table', time = 'condition'
def sql_query_insert(insert, user_id, time):
> cursor, connection = database_connection()
E ValueError: not enough values to unpack (expected 2, got 0)
../lambda_function.py:52: ValueError

Python MySQL WHERE Statement

Ok, I am trying to figure out how to make my variable passed to my method, which that is the easy part. My main problem that I am having is that I need that value of that variable in my method equal to the value in the WHERE Statement.
I was told to use %s to equal the value being passed, but MariaDB doesn't like the syntax. Any help would be greatly appreciated.
def ERRORDISPLAY(ErrorTpye):
#value = ErrorType
conn = connection.MySQLConnection(user = 'user', password = '123456',
host = 'localhost', database= 'HomeConnect')
cursor = conn.cursor()
query = ("SELECT errNumber, description FROM Error_List WHERE errNumber = %s value %s")
num = ErrorType
cursor.execut(query,(num))
for (description) in cursor:
return print(num, description)
ERRORDISPLAY(1)

I got it all figured out. I had to cast the integer to a string. for some reason the MariaDB for Pi W does not like certain syntax. So it should look like this:
def ERRORDISPLAY(ErrorTpye):
conn = connection.MySQLConnection(user = 'user', password = '123456',
host = 'localhost', database= 'HomeConnect')
cursor = conn.cursor()
value = ErrorList
query = ("SELECT errNumber, description FROM Error_List WHERE errNumber =" + str(value))
cursor.execute(query, (value))
for (description) in cursor:
return print(num, description)
ERRORDISPLAY(1)

Need Python Programming Tips

I'm learning python since last few weeks. For better learning, I decided to work on some project. So here is my Class for MySQL connection and demo example as well. Can you please tell me. What other improvement can be possible for following code?
Structure?
What else I can do to optimize code?
And Please forgive. If I'm doing some silly mistakes in code. (I'm learning)
#!/usr/bin/python
import pymysql
# select (table, parameter)
# insert (table, data)
# update (table, id, data)
# delete (table, id)
class MySQL:
def __init__(self):
self.sort_by = ""
self.order = ""
# initiate database connection.
self.connection = pymysql.connect(host='localhost',
user='root',
password='',
db='sherlock',
charset='utf8mb4')
self.cursor = self.connection.cursor(pymysql.cursors.DictCursor)
# this function is for selecting any feild on any table.(feilds veriable is optinal)
def select(self, table, *feilds):
flds = "" #differnt name for feilds veriable.
if not feilds:
flds = '*'
else:
for f in feilds:
if not flds:
flds = f
else:
flds += ",`%s`" % f
sql = "SELECT %s FROM `%s` " % (flds, table)
if self.sort_by:
sql = sql +"order by "+ str(self.sort_by) +" "+ str(self.order)
print sql
self.cursor.execute(sql)
result = self.cursor.fetchall()
return result
# This function is for data sorting for Mysql; but optinal.
# example : SELECT * FROM `users` order by id asc
def order_by(self, sort_by="", order="", *args, **kwargs):
self.sort_by = sort_by
self.order = order
# this function is for closing Mysql connection
def close(self):
self.connection.close()
########### END OF MySQL CLASS #############
sql = MySQL()
# sql.order_by function should be called before the sql.select() function.
sql.order_by("email")
# this will select all the feilds from `users` table.
# you can specify whichever feilds you want to return. like : sql.select("users", "id, email")
result = sql.select("users", "password")
for email in result:
print email["password"]
sql.close()

Trying to understand cx_Oracle`s LOB object

I have a database class in python which I use to query the database.
class Database():
def __init__(self, user, password, host, port, service_name, mode, *args):
#mode should be 0 if not cx_Oracle.SYSDBA
self.user = user
self.password = password
self.host = host
self.port = port
self.user = user
self.service_name = service_name
self.logger = logging.getLogger(__name__)
self.mode = 0
self.connection = None
self.connect_string = self.user + '/' + self.password + '#' + dsn
try:
self.connection = cx_Oracle.connect(self.connect_string, mode=self.mode, threaded=True)
self.connection.stmtcachesize = 1000
self.connection.client_identifier = 'my_app_scheduler'
self.cursor = self.connection.cursor()
self.cursor.arraysize = 10000
self.idVar = self.cursor.var(cx_Oracle.NUMBER)
except cx_Oracle.DatabaseError, exc:
error, = exc
self.logger.exception('Exception occured while trying to create database object : %s', error.message)
raise exc
def query(self, q):
try:
self.cursor.execute(q)
return self.cursor.fetchall(), self.cursor.rowcount
except cx_Oracle.DatabaseError, exc:
raise exc
And this is the code to manipulate the fetched data and convert it.
output, rowcount = db_run_query.query(sql_text)
#self.logger.debug('output : %s, type : %s', output, type(output))
end_time=time.time()
time_taken=end_time - start_time
self.logger.debug('Rowcount : %s, time_taken : %s', rowcount, time_taken)
column_name = [d[0] for d in db_run_query.cursor.description]
result = [dict(zip(column_name, row)) for row in output]
#Convert everything to string : Eg: datetime
try:
for each_dict in result:
for key in each_dict:
if isinstance(each_dict[key], cx_Oracle.LOB):
self.logger.debug('%s', each_dict[key].size())
each_dict[key]=each_dict[key].read()
#self.logger.debug('%s %s %s %s %s %s %s', key, each_dict, type(key), type(each_dict[key]), type(each_dict), temp_each_dict, type(temp_each_dict))
else:
each_dict[key]=str(each_dict[key])
except Exception as e:
self.logger.debug(e)
So without self.cursor.arraysize = 10000
and for a query like select clob_value from table it was able to fetch the data and logged Rowcount : 4901, time_taken : 0.196296930313 but was giving me an error like
LOB variable no longer valid after subsequent fetch
but when I mention the arraysize parameter the error goes away. ( Is arraysize only for lob columns coz it works fine for select other_column from table where rownum<20000 <- other_column in varchar)
Why does that happen?

Turns out CLOBs and fetchall don't place nice together:
Internally, Oracle uses LOB locators which are allocated based on the
cursor array size. Thus, it is important that the data in the LOB
object be manipulated before another internal fetch takes place. The
safest way to do this is to use the cursor as an iterator. In
particular, do not use the fetchall() method.
Avoiding cursor.fetchall() and using it like an iterator (e.g. for row in cursor: ...) and i was able to get around this problem.