We Keep Coding

cx_Oracle: select query following an insert produces no result

in my python code I insert a value into a table.
In the table, there is a sequence which automatically assigns an ID.
After the insert, I want to get this it back in to my python application:
import cx_Oracle, sys
with cx_Oracle.connect(user=ORA_USER,password=ORA_PWD,dsn=ORA_DSN) as conn:
with conn.cursor() as cur:
cur.execute("Insert into my_table columns(data) values ('Hello')")
conn.commit()
with cx_Oracle.connect(user=ORA_USER,password=ORA_PWD,dsn=ORA_DSN) as conn:
with conn.cursor() as cur:
r = cur.execute("select id from my_table where data = 'Hello'")
print(r)
if r is None:
print("Cannot retrieve ID")
sys.exit()
Unfortunately, the result set r is always "None" even though the value has been inserted properly (checked via sqldeveloper).
What am I doing wrong?
I even open a new connection to be sure to grab the value...

After calling execute() for a SELECT statement you need to call fetchone(), fetchmany() or fetchall() as shown in the cx_Oracle documentation SQL Queries.
Or you can use an iterator:
with connection.cursor() as cursor:
try:
sql = """select systimestamp from dual"""
for r in cursor.execute(sql):
print(r)
sql = """select 123 from dual"""
(c_id,) = cursor.execute(sql).fetchone()
print(c_id)
except oracledb.Error as e:
error, = e.args
print(sql)
print('*'.rjust(error.offset+1, ' '))
print(error.message)
However to get an automatically generated ID returned without the overhead of an additional SELECT, you can change the INSERT statement to use a RETURNING INTO clause. There is an example in the cx_Oracle documentation DML RETURNING Bind Variables that shows an UPDATE. You can use similar syntax with INSERT.
With the table:
CREATE TABLE mytable
(myid NUMBER(11) GENERATED BY DEFAULT ON NULL AS IDENTITY (START WITH 1),
mydata VARCHAR2(20));
You can insert and get the generated key like:
myidvar = cursor.var(int)
sql = "INSERT INTO mytable (mydata) VALUES ('abc') RETURNING myid INTO :bv"
cursor.execute(sql, bv=myidvar)
i, = myidvar.getvalue()
print(i)
If you just want a unique identifier you get the ROWID of an inserted row without needing a bind variable. Simple access cursor.lastrowid after executing an INSERT.

Python and SQLite3 - Select Where not working inside script but works in console

As the title says, I'm having problems retrieving data from a SQLite DB when using WHERE statement.
Here is the piece of code that tries to get a row where an ID is given:
def check_attendance(self, cred):
query = """SELECT * FROM clients WHERE dni=?"""
self.conn.cursor().execute(query, (cred,))
record = self.conn.cursor().fetchone()
The var cred is already inside a tuple as specified by SQLite API for Python. Sadly, the query returns None when executed here.
If I do the same but using sqlite.exe, then I do get the right row back. In fact, this is the only query I cannot execute properly from my python script, everything else return rows normally.
Here it is executing from the Python script
And here is in sqlite.exe
Here is the piece that stores values in the DB:
def new_client(self, *args):
success = False
# Check if all inputs are filled
if self.dialog.content_cls.ids.user_name.text and self.dialog.content_cls.ids.user_surname.text and len(self.dialog.content_cls.ids.user_dni.text) == 8 and self.dialog.content_cls.ids.user_date.text:
# Convert str date to a datetime obj in order to use it with timedelta
paid_date = datetime.strptime(self.dialog.content_cls.ids.user_date.text, "%d-%m-%Y")
# paid_date is now YYYY-MM-DD HH-MM-SS format
# Add 30 days to paid_date
exp_date = paid_date + timedelta(days=30)
# Convert YYYY-MM-DD HH-MM-DD to string YYYY-MM-DD as we don't need clock
paid_date = datetime.strptime(str(paid_date), "%Y-%m-%d %H:%M:%S").strftime("%Y-%m-%d")
exp_date = datetime.strptime(str(exp_date), "%Y-%m-%d %H:%M:%S").strftime("%Y-%m-%d")
# Create query blueprint and try executing
query = """INSERT INTO clients (name, surname, dni, membership_date, expiration_date) VALUES (?,?,?,?,?)"""
try:
self.conn.execute(query, (self.dialog.content_cls.ids.user_name.text,
self.dialog.content_cls.ids.user_surname.text,
self.dialog.content_cls.ids.user_dni.text,
paid_date,
exp_date
)
)
success = True
except sqlite3.IntegrityError:
pass
if success:
self.conn.commit()
The try/except was used for other reasons. Adding to the database from the Python script works fine as shown in the second screenshot.
And the table clients is as follows:
c.execute(''' CREATE TABLE clients (id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
surname TEXT NOT NULL,
dni INTEGER NOT NULL UNIQUE,
membership_date date NOT NULL,
expiration_date date NOT NULL); ''')
Using Python v3.7.7 32bit.
Thanks!

In your code, the cursor is initialized two times (cursor()).
You should either get the results from the same cursor you used to execute the SELECT statement:
def check_attendance(self, cred):
query = """SELECT * FROM clients WHERE dni=?"""
cur = self.conn.cursor()
cur.execute(query, (cred,))
record = cur.fetchone()
...or you can avoid the implicit cursor creation by using execute method directly on Connection object:
def check_attendance(self, cred):
query = """SELECT * FROM clients WHERE dni=?"""
record = self.conn.execute(query, (cred,)).fetchone()
You can read more about this approach in the documentation (https://docs.python.org/3/library/sqlite3.html#using-sqlite3-efficiently):
Using the nonstandard execute(), executemany() and executescript() methods of the Connection object, your code can be written more concisely because you don’t have to create the (often superfluous) Cursor objects explicitly. Instead, the Cursor objects are created implicitly and these shortcut methods return the cursor objects.

Python Script to get multi table counts

I'm trying to write a python script to get a count of some tables for monitoring which looks a bit like the code below. I'm trying to get an output such as below and have tried using python multi-dimensional arrays but not having any luck.
Expected Output:
('oltptransactions:', [(12L,)])
('oltpcases:', [(24L,)])
Script:
import psycopg2
# Connection with the DataBase
conn = psycopg2.connect(user = "appuser", database = "onedb", host = "192.168.1.1", port = "5432")
cursor = conn.cursor()
sql = """SELECT COUNT(id) FROM appuser.oltptransactions"""
sql2 = """SELECT count(id) FROM appuser.oltpcases"""
sqls = [sql,sql2]
for i in sqls:
cursor.execute(i)
result = cursor.fetchall()
print('Counts:',result)
conn.close()
Current output:
[root#pgenc python_scripts]# python multi_getrcount.py
('Counts:', [(12L,)])
('Counts:', [(24L,)])
Any help is appreciated.
Thanks!

I am a bit reluctant to show this way, because best practices recommend to never build a dynamic SQL string but always use a constant string and parameters, but this is one use case where computing the string is legit:
a table name cannot be a parameter in SQL
the input only comes from the program itself and is fully mastered
Possible code:
sql = """SELECT count(*) from appuser.{}"""
tables = ['oltptransactions', 'oltpcases']
for t in tables:
cursor.execute(sql.format(t))
result = cursor.fetchall()
print("('", t, "':,", result, ")")

I believe something as below, Unable to test code because of certificate issue.
sql = """SELECT 'oltptransactions', COUNT(id) FROM appuser.oltptransactions"""
sql2 = """SELECT 'oltpcases', COUNT(id) FROM appuser.oltpcases"""
sqls = [sql,sql2]
for i in sqls:
cursor.execute(i)
for name, count in cursor:
print ("")
Or
sql = """SELECT 'oltptransactions :'||COUNT(id) FROM appuser.oltptransactions"""
sql2 = """SELECT 'oltpcases :'||COUNT(id) FROM appuser.oltpcases"""
sqls = [sql,sql2]
for i in sqls:
cursor.execute(i)
result = cursor.fetchall()
print(result)

Use output of one SQL query in another SQL query in Python

I am working with a SQL Database on Python. After making the connection, I want to use the output of one query in another query.
Example: query1 gives me a list of all tables in a schema. I want to use each table name from query1 in my query2.
query2 = "SELECT TOP 200 * FROM db.schema.table ORDER BY ID"
I want to use this query for each of the table in the output of query1.
Can someone help me with the Python code for it?

Here is a working example on how to do what you are looking to do. I didn't look up the schemes for the tablelist, but you can simply substitute the SQL code to do so. I just 'faked it' by unioning a statement of 2 tables. There are plenty of other answer on that SQL code and I don't want to clutter this answer:
How do I get list of all tables in a database using TSQL?
It looks like the key part you may have been missing was the join step to build the second SQL statement. This should be enough of a starting point to craft exactly what you are looking for.
import pypyodbc
def main():
table_list = get_table_list()
for table in table_list:
print_table(table)
def print_table(table):
thesql = " ".join(["SELECT TOP 10 businessentityid FROM", table])
connection = get_connection()
cursor = connection.cursor()
cursor.execute(thesql)
for row in cursor:
print (row["businessentityid"])
cursor.close()
connection.close()
def get_table_list():
table_list = []
thesql = ("""
SELECT 'Sales.SalesPerson' AS thetable
UNION
SELECT 'Person.BusinessEntity' thetable
""")
connection = get_connection()
cursor = connection.cursor()
cursor.execute(thesql)
for row in cursor:
table_list.append(row["thetable"])
cursor.close()
connection.close()
return table_list
def get_connection():
'''setup connection depending on which db we are going to write to in which environment'''
connection = pypyodbc.connect(
"Driver={SQL Server};"
"Server=YOURSERVER;"
"Database=AdventureWorks2014;"
"Trusted_Connection=yes"
)
return connection
main ()

Why does Psycopg2 return list of tuples in with Stored Procedure?

I have been using Psycopg2 to read stored procedures from Postgres successfully and getting a nice tuple returned, which has been easy to deal with. For example...
def authenticate(user, password):
conn = psycopg2.connect("dbname=MyDB host=localhost port=5433 user=postgres password=mypwd")
cur = conn.cursor()
retrieved_pwd = None
retrieved_userid = None
retrieved_user = None
retrieved_teamname = None
cur.execute("""
select "email", "password", "userid", "teamname"
from "RegisteredUsers"
where "email" = '%s'
""" % user)
for row in cur:
print row
The row that prints would give me ('user#gmail.com ', '84894531656894hashedpassword5161651165 ', 36, 'test ')
However, when I run the following code to read a row of fixtures with a Stored Procedure, I get (what looks to me like) an unholy mess.
def get_from_sql(userid):
conn = psycopg2.connect("dbname=MyDB host=localhost port=5433 user=postgres password=pwd")
fixture_cursor = conn.cursor()
callproc_params = [userid]
fixture_cursor.execute("select sppresentedfixtures(%s)", callproc_params)
for row in fixture_cursor:
print row
The resulting output:
('(5,"2015-08-28 21:00:00","2015-08-20 08:00:00","2015-08-25 17:00:00","Team ",,"Team ",,"Final ")',)
I have researched the cursor class and cannot understand why it outputs like this for a stored procedure. When executing within Postgres, the output is in a perfect Tuple. Using Psycopg2 adds onto the tuple and I don't understand why?
How do I change this so I get a tidy tuple? What am I not understanding about the request that I am making that gives me this result?
I have tried the callproc function and get an equally unhelpful output. Any thoughts on this would be great.

This is because you're SELECTing the result of the function directly. Your function returns a set of things, and each "thing" happens to be a tuple, so you're getting a list of stringified tuples back. What you want is this:
SELECT * FROM sppresentedfixtures(...)
But this doesn't work, because you'll get the error:
ERROR: a column definition list is required for functions returning "record"
The solution is to return a table instead:
CREATE OR REPLACE FUNCTION sppresentedfixtures(useridentity integer) RETURNS TABLE(
Fixture_No int,
Fixture_Date timestamp,
...
) AS
$BODY$
select
"Fixtures"."Fixture_No",
"Fixtures"."Fixture_Date",
...
from "Fixtures" ...
$BODY$ LANGUAGE sql