We Keep Coding

Python stuck while executing SQL update query

My python script is stuck on cursor execution and I have problem to find out why.
db = cx_Oracle.connect('user/password#user')
cursor = db.cursor()
for i in range(1, 10):
insert_data = """
update products set prd_stock_holder = '~',prd_prod_quality = 'FREE'
where PRD_PRI_ID = (select prd_pri_id from products join PRODUCT_INFOS
on prd_pri_id = pri_id
where PRI_code = '%s')""" %i
cursor.execute(insert_data) # stuck here
print "product %s updated" %i # never printed
db.commit()

I had a simular issue with insert to a MySQL Database.
Moving the db.commit() inside the loop solved my issue.
db = cx_Oracle.connect('user/password#user')
cursor = db.cursor()
for i in range(1, 10):
insert_data = """
update products set prd_stock_holder = '~',prd_prod_quality = 'FREE'
where PRD_PRI_ID = (select prd_pri_id from products join PRODUCT_INFOS
on prd_pri_id = pri_id
where PRI_code = '%s')""" %i
cursor.execute(insert_data) # stuck here
db.commit()
print "product %s updated" %i # never printed

A cause may be a lock on the Oracle RDBMS. You may try checking if your session is locked waiting for a resource (e.g. http://www.dba-oracle.com/t_tracking_oracle_blocking_sessions.htm).
Please keep in mind that locks may be connected to the execution plan the db has decided to apply. By rewriting your update query you might benefit from thinner lock acquisition.

How to change the cursor to the next row using pyodbc in Python

I am trying to fetch records after a regular interval from a database table which growing with records. I am using Python and its pyodbc package to carry out the fetching of records. While fetching, how can I point the cursor to the next row of the row which was read/fetched last so that with every fetch I can only get the new set of records inserted.
To explain more,
my table has 100 records and they are fetched.
after an interval the table has 200 records and I want to fetch rows from 101 to 200. And so on.
Is there a way with pyodbc cursor?
Or any other suggestion would be very helpful.
Below is the code I am trying:
#!/usr/bin/python
import pyodbc
import csv
import time
conn_str = (
"DRIVER={PostgreSQL Unicode};"
"DATABASE=postgres;"
"UID=userid;"
"PWD=database;"
"SERVER=localhost;"
"PORT=5432;"
)
conn = pyodbc.connect(conn_str)
cursor = conn.cursor()
def fetch_table(**kwargs):
qry = kwargs['qrystr']
try:
#cursor = conn.cursor()
cursor.execute(qry)
all_rows = cursor.fetchall()
rowcnt = cursor.rowcount
rownum = cursor.description
#return (rowcnt, rownum)
return all_rows
except pyodbc.ProgrammingError as e:
print ("Exception occured as :", type(e) , e)
def poll_db():
for i in [1, 2]:
stmt = "select * from my_database_table"
rows = fetch_table(qrystr = stmt)
print("***** For i = " , i , "******")
for r in rows:
print("ROW-> ", r)
time.sleep(10)
poll_db()
conn.close()

I don't think you can use pyodbc, or any other odbc package, to find "new" rows. But if there is a 'timestamp' column in your database, or if you can add such a column (some databases allow for it to be automatically populated as the time of insertion so you don't have to change the insert queries) then you can change your query to select only the rows whose timestamp is greater than the previous timestamp. And you can keep changing the prev_timestamp variable on each iteration.
def poll_db():
prev_timestamp = ""
for i in [1, 2]:
if prev_timestamp == "":
stmt = "select * from my_database_table"
else:
# convert your timestamp str to match the database's format
stmt = "select * from my_database_table where timestamp > " + str(prev_timestamp)
rows = fetch_table(qrystr = stmt)
prev_timestamp = datetime.datetime.now()
print("***** For i = " , i , "******")
for r in rows:
print("ROW-> ", r)
time.sleep(10)

Python SQLlite3 loop to find row with fetchone

I have a snippet in my code that needs to find a mac address in the database, then print a statement to show what row it's been found in. I will then use that row number to update the timestamp column in that row.
My sqlite3 DB looks like this:
CREATE TABLE IF NOT EXISTS My_TABLENAME (mac_addr TEXT, timestamp TEXT, location TEXT, serialno TEXT)"
The code looks like this.
import sqlite3 as lite
con = lite.connect("database.db")
con.row_factory = lite.Row
cur = con.cursor()
cur.execute('''SELECT mac_addr, timestamp, (SELECT COUNT(*) FROM MY_TABLENAME AS t2 WHERE t2.mac_addr <= t1.mac_addr) AS i FROM My_TABLENAME AS t1 ORDER BY timestamp, mac_addr''')
_data = cur.fetchone()
if str(_data[0]) != '5c:f5:da:e0:dd:44':
cur.fetchone()
else:
print "Found the device in row", str(_data[2])
when I run the code in the python cli I get the below when I print the _data variable after doing the first fetchone()
>>> _data
(u'34:0a:ff:b2:75:78', u'1433940972.03946', 135)
>>>
So I can see that the row currently in the _data variable is row 135 (this changes obviously).
But when I run the code snippet I posted uptop I get the following output, which makes me think the loop is not working. Am I doing something wrong?
>>> import sqlite3 as lite
>>> con = lite.connect("database.db")
>>> con.row_factory = lite.Row
>>> cur = con.cursor()
>>> cur.execute('''SELECT mac_addr, timestamp, (SELECT COUNT(*) FROM My_TABLENAME AS t2 WHERE t2.mac_addr <= t1.mac_addr) AS i FROM My_TABLENAME AS t1 ORDER BY timestamp, mac_addr''')
<sqlite3.Cursor object at 0x7ff99fc0a8f0>
>>> _data = cur.fetchone()
>>> if str(_data[0]) != '5c:f5:da:e0:dd:44':
... cur.fetchone()
... else:
... print "Found the device in row", str(_data[2])
...
(u'18:83:31:61:83:8c', u'1433940974.39824', 74)
>>>
Any advice would be great please.

Thanks. I managed to change the IF statement to a For loop and got success that way.
Although I do think a WHERE statement would be more efficient in the long run. I'll experiment with it and see.
Thanks

cur.execute('''SELECT mac_addr, timestamp,
(SELECT COUNT(*) FROM My_TABLENAME AS t2
WHERE t2.mac_addr <= t1.mac_addr) AS i
FROM My_TABLENAME AS t1 WHERE mac_addr = ?
ORDER BY timestamp, mac_addr''', '5c:f5:da:e0:dd:44')
It's faster.

In Django cursor.execute("select ..") randomly returns None

I am running Django + MySql on OpenShift server and have this problem.
Code:
from django.db import connection
...
cursor = connection.cursor()
somedate = calculateSomeDate()
query = "SELECT id FROM levelbuffer WHERE START > '%s' ORDER BY START ASC LIMIT 1" % somedate
print "First select: " + query
cursor.execute(query)
sql_result = cursor.fetchone()
if not sql_result == None:
gameId = int(sql_result[0])
cursor.fetchall() #To clear anything left behind (this wasn't here before, but no changes appeared after adding)
query = "SELECT gamescores.skore, users.name FROM gamescores JOIN users ON gamescores.userId = users.id where gamescores.gameId = " + str(gameId) + " ORDER BY skore ASC"
cursor.execute(query);
#cursor.execute("SELECT gamescores.skore, users.name FROM gamescores JOIN users ON gamescores.userId = users.id where gamescores.gameId = %s ORDER BY skore ASC", (gameId))
print "Second select " + query
row = cursor.fetchone()
The weird thing is, that the row is sometimes None and sometimes it contains wanted values. I cant figure out, what i am missing. The first query works fine every time.
When I try to perform the second query printed in log on phpmyadmin - works always fine.

Running python script from automator

I am trying to create an ".app" from Automator to run a simple python script. When I execute the script in Automator an error occurs saying more or less "Check your action properties and execute again".
And in the history it says "Traceback (most recent call last:)". The point is that this script is running well with a Terminal session.
It seems to be at least an error with my loop "While" for renaming databases (see below) since I can execute the script up to this stage. Is it something wrong with managing sqlite databases? But I cannot understand since there is no problem with the Terminal. Is anything missing?
My python script:
#!/usr/bin/python
import sqlite3
import os.path
file_name = "newDB.data"
choice = ""
if os.path.isfile(file_name):
choice = raw_input("Erase DB? press [y] or [n]:\n")
if choice == "y":
print "erase"
while True:
try:
os.remove(file_name)
break
except OSError as e: # name the Exception `e`
print "Failed with:", e.strerror # look what it says
print "Error code:", e.code
if choice == "n":
print "Bye!"
exit()
# start sqlite connection
conn = sqlite3.connect("newDB.data")
c = conn.cursor()
# attach
c.execute("ATTACH database 'store1.data' AS db1")
c.execute("ATTACH database 'store2.data' AS db2")
# rename tables
while True:
try:
c.execute("ALTER TABLE db1.ZPATIENT RENAME TO table1")
print "table 1 renamed"
break
except:
c.execute("ALTER TABLE db1.table1 RENAME TO ZPATIENT")
print "except 1"
while True:
try:
c.execute("ALTER TABLE db2.ZPATIENT RENAME TO table2")
print "table 2 renamed"
break
except:
c.execute("ALTER TABLE db2.table2 RENAME TO ZPATIENT")
print "except 2"
# some information commands (START):
c.execute("SELECT * from table1")
print(c.fetchall())
c.execute("SELECT * from table2")
print(c.fetchall())
# some information commands (END)
#c.execute("create table ZPATIENT as select * from table1 union select * from table2") ---> first union action but some entries duplicated (one column changed?)
# remove some duplicated entries...
c.execute("create table ZPATIENT as select * from (select * from table1 union select * from table2) final group by ZDATECREATED")
c.execute("CREATE TABLE Z_PRIMARYKEY (Z_ENT int, Z_NAME text, Z_SUPER int, Z_MAX int)")
c.execute("CREATE TABLE Z_METADATA (Z_VERSION int, Z_UUID text, Z_PLIST BLOB)")
c.execute("SELECT count(*) FROM ZPATIENT")
result=c.fetchone()
number_of_rows=result[0]
print number_of_rows
start = 0
end = number_of_rows + 1
c.execute('SELECT * FROM ZPATIENT')
newresult=c.fetchall()
for row in newresult:
start += 1
end -= 1
print start
print end
# some information commands (START):
list_of_tuple = list(row)
list_of_tuple[0] = start
list_of_tuple[2] = end
row = tuple(list_of_tuple)
print row
# some information commands (END)
c.execute("UPDATE ZPATIENT SET Z_PK = ? WHERE rowid = ?", (start, start))
c.execute("UPDATE ZPATIENT SET Z_OPT = ? WHERE rowid = ?", (end, start))
c.execute("INSERT INTO Z_PRIMARYKEY (Z_ENT, Z_NAME, Z_SUPER, Z_MAX) VALUES (0, 'Patient', 0, ?)", (start,))
# close
conn.commit()
conn.close()
To be working I have two sqlite databases named store1.data and store2.data in the same folder...
If anyone has a solution... I don't know if maybe there is an easier way to execute this in one click?

One simple solution could be to avoid using automator and just make a bash script to call the python script. You can execute the bash script by double-clicking on it if that's what you wanted.
#! /bin/bash
python scriptname.py
is all you would need.