We Keep Coding

cx_Oracle: select query following an insert produces no result

in my python code I insert a value into a table.
In the table, there is a sequence which automatically assigns an ID.
After the insert, I want to get this it back in to my python application:
import cx_Oracle, sys
with cx_Oracle.connect(user=ORA_USER,password=ORA_PWD,dsn=ORA_DSN) as conn:
with conn.cursor() as cur:
cur.execute("Insert into my_table columns(data) values ('Hello')")
conn.commit()
with cx_Oracle.connect(user=ORA_USER,password=ORA_PWD,dsn=ORA_DSN) as conn:
with conn.cursor() as cur:
r = cur.execute("select id from my_table where data = 'Hello'")
print(r)
if r is None:
print("Cannot retrieve ID")
sys.exit()
Unfortunately, the result set r is always "None" even though the value has been inserted properly (checked via sqldeveloper).
What am I doing wrong?
I even open a new connection to be sure to grab the value...

After calling execute() for a SELECT statement you need to call fetchone(), fetchmany() or fetchall() as shown in the cx_Oracle documentation SQL Queries.
Or you can use an iterator:
with connection.cursor() as cursor:
try:
sql = """select systimestamp from dual"""
for r in cursor.execute(sql):
print(r)
sql = """select 123 from dual"""
(c_id,) = cursor.execute(sql).fetchone()
print(c_id)
except oracledb.Error as e:
error, = e.args
print(sql)
print('*'.rjust(error.offset+1, ' '))
print(error.message)
However to get an automatically generated ID returned without the overhead of an additional SELECT, you can change the INSERT statement to use a RETURNING INTO clause. There is an example in the cx_Oracle documentation DML RETURNING Bind Variables that shows an UPDATE. You can use similar syntax with INSERT.
With the table:
CREATE TABLE mytable
(myid NUMBER(11) GENERATED BY DEFAULT ON NULL AS IDENTITY (START WITH 1),
mydata VARCHAR2(20));
You can insert and get the generated key like:
myidvar = cursor.var(int)
sql = "INSERT INTO mytable (mydata) VALUES ('abc') RETURNING myid INTO :bv"
cursor.execute(sql, bv=myidvar)
i, = myidvar.getvalue()
print(i)
If you just want a unique identifier you get the ROWID of an inserted row without needing a bind variable. Simple access cursor.lastrowid after executing an INSERT.

How to keep continuously write to Oracle table even if table is not accessible?

I am trying to insert multiple records into the one Oracle table continuously. For which I have written below python script.
import cx_Oracle
import config
connection = None
try:
# Make a connection
connection = cx_Oracle.connect(
config.username,
config.password,
config.dsn,
encoding=config.encoding)
# show the version of the Oracle Database
print(connection.version)
# Insert 20000 records
for i in range(1, 20001):
cursor = connection.cursor()
sql = "INSERT into SCHEMA.ABC (EVENT_ID, EVENT_TIME) VALUES( "+ str(i)+" , CURRENT_TIMESTAMP)"
cursor.execute(sql)
connection.commit()
except cx_Oracle.Error as error:
print(error)
finally:
if connection:
connection.close()
So, During the insert, when I change the table name it just create an exception and comes out of script(as the table is not available and cannot write). What I want is, Even if when I do the rename and table is not available, the script needs to keep continuously trying insert. Is there a way this can be possible?

Here's an example of what Ptit Xav was talking about. I added some code to quit after a max number of retries, since that's often desirable.
# Insert 20000 records
for i in range(1, 20001):
retry_count = 0
data_inserted = False
while not data_inserted:
try:
cursor = connection.cursor()
sql = "INSERT into SCHEMA.ABC (EVENT_ID, EVENT_TIME) VALUES( "+ str(i)+" , CURRENT_TIMESTAMP)"
cursor.execute(sql)
connection.commit()
data_inserted = True
except cx_Oracle.Error as error:
print(error)
time.sleep(5) # wait for 5 seconds between retries
retry_count += 1
if retry_count > 100:
print(f"Retry count exceeded on record {i}, quitting")
break
else:
# continue to next record if the data was inserted
continue
# retry count was exceeded; break the for loop.
break
See this answer for more explanation of the while... else logic.

You may want to encapsule the insert logik in a function that catches the possible exception and performs the retry
def safe_insert(con, i):
"""
insert a row with retry after exception
"""
retry_cnt = 0
sql_text = "insert into ABC(EVENT_ID, EVENT_TIME) VALUES(:EVENT_ID,CURRENT_TIMESTAMP) "
while True:
try:
with con.cursor() as cur:
cur.execute(sql_text, [i])
con.commit()
return
except cx_Oracle.Error as error:
print(f'error on inserting row {i}')
print(error)
time.sleep(1)
retry_cnt += 1
if (retry_cnt > 10):
raise error
Similar to #kfinity's answer I also added a limit on retry - if this limit is exceeded the function raise an exception.
Note also that the function uses bind variables in the INSERT statement which is preferable to the concatenation of the values in the statement.
The usage is as simple as
for i in range(1, 20001):
safe_insert(con, i)

sqlalchemy, insert with try except block

I have a query that I intend to insert with a try-except block, so that I can ignore all the queries that fail a certain constraint i.e. IntegrityError.
with sql_engine.begin() as cn:
sql = """INSERT INTO table1 (m_id, pick_on)
SELECT t.m_id, t.pick_on
FROM temp_table t
WHERE NOT EXISTS
(SELECT 1 FROM table1 f
WHERE (t.m_id = f.m_id
OR (t.m_id IS NULL AND f.m_id IS NULL))
AND t.pick_on = f.pick_on)"""
try:
cn.execute(sql)
except:
print("Unexpected error:", sys.exc_info()[0])
print(sql)
pass
but this does not insert all the records to table1. why?

Running python script from automator

I am trying to create an ".app" from Automator to run a simple python script. When I execute the script in Automator an error occurs saying more or less "Check your action properties and execute again".
And in the history it says "Traceback (most recent call last:)". The point is that this script is running well with a Terminal session.
It seems to be at least an error with my loop "While" for renaming databases (see below) since I can execute the script up to this stage. Is it something wrong with managing sqlite databases? But I cannot understand since there is no problem with the Terminal. Is anything missing?
My python script:
#!/usr/bin/python
import sqlite3
import os.path
file_name = "newDB.data"
choice = ""
if os.path.isfile(file_name):
choice = raw_input("Erase DB? press [y] or [n]:\n")
if choice == "y":
print "erase"
while True:
try:
os.remove(file_name)
break
except OSError as e: # name the Exception `e`
print "Failed with:", e.strerror # look what it says
print "Error code:", e.code
if choice == "n":
print "Bye!"
exit()
# start sqlite connection
conn = sqlite3.connect("newDB.data")
c = conn.cursor()
# attach
c.execute("ATTACH database 'store1.data' AS db1")
c.execute("ATTACH database 'store2.data' AS db2")
# rename tables
while True:
try:
c.execute("ALTER TABLE db1.ZPATIENT RENAME TO table1")
print "table 1 renamed"
break
except:
c.execute("ALTER TABLE db1.table1 RENAME TO ZPATIENT")
print "except 1"
while True:
try:
c.execute("ALTER TABLE db2.ZPATIENT RENAME TO table2")
print "table 2 renamed"
break
except:
c.execute("ALTER TABLE db2.table2 RENAME TO ZPATIENT")
print "except 2"
# some information commands (START):
c.execute("SELECT * from table1")
print(c.fetchall())
c.execute("SELECT * from table2")
print(c.fetchall())
# some information commands (END)
#c.execute("create table ZPATIENT as select * from table1 union select * from table2") ---> first union action but some entries duplicated (one column changed?)
# remove some duplicated entries...
c.execute("create table ZPATIENT as select * from (select * from table1 union select * from table2) final group by ZDATECREATED")
c.execute("CREATE TABLE Z_PRIMARYKEY (Z_ENT int, Z_NAME text, Z_SUPER int, Z_MAX int)")
c.execute("CREATE TABLE Z_METADATA (Z_VERSION int, Z_UUID text, Z_PLIST BLOB)")
c.execute("SELECT count(*) FROM ZPATIENT")
result=c.fetchone()
number_of_rows=result[0]
print number_of_rows
start = 0
end = number_of_rows + 1
c.execute('SELECT * FROM ZPATIENT')
newresult=c.fetchall()
for row in newresult:
start += 1
end -= 1
print start
print end
# some information commands (START):
list_of_tuple = list(row)
list_of_tuple[0] = start
list_of_tuple[2] = end
row = tuple(list_of_tuple)
print row
# some information commands (END)
c.execute("UPDATE ZPATIENT SET Z_PK = ? WHERE rowid = ?", (start, start))
c.execute("UPDATE ZPATIENT SET Z_OPT = ? WHERE rowid = ?", (end, start))
c.execute("INSERT INTO Z_PRIMARYKEY (Z_ENT, Z_NAME, Z_SUPER, Z_MAX) VALUES (0, 'Patient', 0, ?)", (start,))
# close
conn.commit()
conn.close()
To be working I have two sqlite databases named store1.data and store2.data in the same folder...
If anyone has a solution... I don't know if maybe there is an easier way to execute this in one click?

One simple solution could be to avoid using automator and just make a bash script to call the python script. You can execute the bash script by double-clicking on it if that's what you wanted.
#! /bin/bash
python scriptname.py
is all you would need.

Number of rows inserted into SQLite db is negative

I'm trying to insert new records into SQLite database from Python code.
con = sqlite.connect(connectionString)
cur = con.cursor()
countOfNewItems = 0
for ...
try:
con.execute("insert or ignore into items ...")
countOfNewItems += cur.rowcount
except:
cur.close()
con.close()
print "Error when inserting item '%s' to database." % item
exit(1)
cur.close()
con.commit()
con.close()
print "%d new items have been inserted." % countOfNewItems
My code reports negative number of inserted records (-5141).
Because my database was empty, I could find out how many records were inserted via command line
select count(*) from items;
4866
Could you advise me what's wrong. Why the two values don't match and why it's negative?

http://docs.python.org/library/sqlite3.html#sqlite3.Cursor.rowcount
Although the Cursor class of the sqlite3 module implements this attribute, the database engine’s own support for the determination of “rows affected”/”rows selected” is quirky.
and
As required by the Python DB API Spec, the rowcount attribute “is -1 in case no executeXX() has been performed on the cursor or the rowcount of the last operation is not determinable by the interface”.

Try cur.execute instead of con.execute. cur.rowcount then returns 1 for me for a simple insert.