We Keep Coding

Python: MySQL is not updating record despite of using Commit

I am facing a weird issue. I have the following code. The INSERTS go well but the update query does not work at all. The rowcount is also shown 1 still when I check in Table Plus it does not reflect.
When I directly run the query UPDATE shop_links set product_status = 3 where shop_url = 'https://example.com' in TablePlus it does show record.
The irony is, the update query which set to 1 works just fine and updates instantly
import mysql.connector
def get_connection(host, user, password, db_name):
connection = None
try:
# connection = pymysql.connect(host=host,
# user=user,
# password=password,
# db=db_name,
# charset='utf8',
# max_allowed_packet=1073741824,
# cursorclass=pymysql.cursors.DictCursor)
connection = mysql.connector.connect(
host=host,
user=user,
use_unicode=True,
password=password,
database=db_name
)
connection.set_charset_collation('utf8')
print('Connected')
except Exception as ex:
print(str(ex))
finally:
return connection
with connection.cursor() as cursor:
sql = 'INSERT INTO {} (shop_url,product_url) VALUES (%s, %s)'.format(TABLE_FETCH_PRODUCTS)
cursor.executemany(sql, records)
connection.commit()
with connection.cursor() as cursor:
# Update the shop URL
# sql = "UPDATE {} set product_status = 3 where shop_url = '{}' ".format(TABLE_FETCH, shop_url)
sql = "UPDATE {} set product_status = 3 where shop_url = %s ".format(TABLE_FETCH, shop_url)
print(sql)
print('----------------------------------------------------------------')
cursor.execute(sql, (shop_url,))
connection.commit()

AWS: Unable to insert values in Redshift table

I am trying to insert values into a table within my redshift cluster, it is connected as I can read the table but I can't insert values on it. When I use SELECT statements it works fine but when I try to insert values from lambda function, it is aborted with no error or log info about why was it aborted.
The query part is like this:
conn = psycopg2.connect(dbname = 'dev',
host =
'redshift-cluster-summaries.c0xcgwtgz65l.us-east-2.redshift.amazonaws.com',
port = '5439',
user = '****',
password = '****%')
cur = conn.cursor()
cur.execute("INSERT INTO public.summaries(topic,summary)
values('data', 'data_summary');")
#print(cur.fetchone())
cur.close()
conn.close()
As I said, there is no log information about why was it aborted, neither it is giving me any kind of error. Actually, when I just use a Select statement, it works.
Is there anyone who can guide me through what could be going on?

You forgot to do conn.commit()
conn = psycopg2.connect(dbname = 'dev',
host = 'redshift-cluster-summaries.c0xcgwtgz65l.us-east-2.redshift.amazonaws.com',
port = '5439',
user = '****',
password = '****%')
cur = conn.cursor()
cur.execute("INSERT INTO public.summaries(topic,summary) values('data', 'data_summary');")
cur.close()
conn.commit()
conn.close()
a bit improved way to run this
from contextlib import contextmanager
#contextmanager
def cursor():
with psycopg2.connect(dbname = 'dev',
host = 'redshift-cluster-summaries.c0xcgwtgz65l.us-east-2.redshift.amazonaws.com',
port = '5439',
user = '****',
password = '****%') as conn:
try:
yield conn.cursor()
finally:
try:
conn.commit()
except psycopg2.InterfaceError:
pass
def run_insert(query):
with cursor() as cur:
cur.execute(query)
cur.close()
run_insert("INSERT INTO public.summaries(topic,summary) values('data', 'data_summary');")

ValueError: not enough values to unpack (expected 2, got 0) when mocking

I have a function for my database connection that returns both cursor and connection.
def database_connection():
resp_dict = json.loads(get_secret())
endpoint = resp_dict.get('host')
username = resp_dict.get('username')
password = resp_dict.get('password')
database_name = resp_dict.get('dbname')
port = resp_dict.get('port')
connection = pymysql.connect(host=endpoint, user=username, passwd=password, db=database_name, port=port)
cursor = connection.cursor()
return cursor, connection
def sql_query_insert(insert, user_id, time):
cursor, connection = database_connection()
sql_query = "INSERT INTO account_login (" + insert + ", login_date) VALUES ('" + str(user_id) + "' ,'" + time + "')"
cursor.execute(sql_query)
connection.commit()
return connection
I then use this to run various sql queries. Right now I am trying to write some test's for these sql queries but do not want to connection to a real database and want to mock the function database_connection(). However, I am having some trouble mocking the function as it return's two values. This is the test I am trying to build out. (p.s I know this will fail but I am trying to build out the test first before getting my expected values, and I will be updating the sql query insert function after figuring this out as as it is open to sql injections)
#mock.patch('lambda_function.database_connection', return_value=['cursor', 'connection'])
def test_sql_query_insert(self):
result = lambda_function.sql_query_insert("select", "table", "condition")
self.assertEqual(result, "test")
This is the error I am getting:
insert = 'select', user_id = 'table', time = 'condition'
def sql_query_insert(insert, user_id, time):
> cursor, connection = database_connection()
E ValueError: not enough values to unpack (expected 2, got 0)
../lambda_function.py:52: ValueError

Calling a mysql procedure using pymysql

I have a procedure called'my' in mysql. The query is as below
CREATE DEFINER=`as`#`%` PROCEDURE `my`()
BEGIN
delete from my1 where datediff(curdate(), Date_) <= 20;
insert into my1
select
t.idx auto_increment,
t.date_ as 'Date_',
t.brand as 'Br',
concat(t.so_, '_', t.me_) as 'Md'
from
(select idx, date_, brand, so_, me from md where datediff(curdate(), Date_) <= 20) as t
group by t.date_, t.br;
END
To run this procedure in Python, I wrote the following code using pymysql.
user = 'xxxx'
passw = 'xxxxx'
host = 'xxxx'
port = 3306
database = 'DBxxxx'
conn = pymysql.connect(host=host,
port=port,
user=user,
password=passw,
database=database,
charset='utf8',
local_infile=1)
try:
with conn.cursor() as cursor:
cursor.callproc('my')
conn.commit()
conn.close()
except:
conn.close()
But it doesn't work well. Are there any mistakes in the code I wrote? The procedure I run in mysql works fine

Executing multiple MySQL SELECT queries python

I need to execute multiple MySQL select queries in Python. I am thinking of doing it in this way:
connection = mysql.connector.connect(host = HOSTNAME, user = USERNAME, passwd = PASSWORD, db = DATABASE, port=PORT)
cursor = connection.cursor()
try:
query_table0 = 'SELECT %s FROM %s'%(COLUMN.get(TABLES[0]),TABLES[0])
query_table1 = 'SELECT %s FROM %s'%(COLUMN.get(TABLES[1]),TABLES[1])
query_table2 = 'SELECT %s FROM %s'%(COLUMN.get(TABLES[2]),TABLES[2])
cursor.execute(query_table0)
result_table0 = cursor.fetchall()
cursor.execute(query_table1)
result_table1 = cursor.fetchall()
cursor.execute(query_table2)
result_table2 = cursor.fetchall()
finally:
connection.close()
Is there any more optimised way of executing multiple SQL statements in Python?