I'm trying to make friends with postgresql (14.0 build 1914 64-bit on windows), psycopg2 (2.9.1 installed using pip) and python 3.8.10 on windows.
I have created a postgresql function in a database that returns a cursor, somthing like below
CREATE get_rows
...
RETURNS refcursor
...
DECLARE
res1 refcursor;
BEGIN
OPEN res1 FOR
SELECT some_field, and_another_field FROM some_table;
RETURN res1;
END
The function can be run from pgAdmin4 Quert tool
SELECT get_rows();
and will then return a cursor like "<unnamed portal 1>"
Still within query tool in pgAdmin4 I can issue:
BEGIN;
SELECT get_rows();
FETCH ALL IN "<unnamed portal 2>"; -- Adjust counter number
And this will get me the rows returned by the cursor.
Now I want to replicate this using psycopg instead of pgAdmin4
I have the below code
conn = psycopg2.connect("dbname='" + db_name + "' "\
"user='" + db_user + "' " +\
"host='" + db_host + "' "+\
"password='" + db_passwd + "'")
cursor = conn.cursor()
cursor.callproc('get_rows')
print("cursor.description: ", end = '')
print(cursor.description)
for record in cursor:
print("record: ", end = '')
print (record)
The above code only gives the cursor string name (as returned by the postgresql function 'get_rows') in the single record of the cursor created by psycopg.
How can I get a cursor-class object from psycopg that provides access the cursor returned by 'get_rows'?
https://www.psycopg.org/docs/cursor.html says cursor.name is read-only and I dont see an obvious way to connect the cursor from 'get_rows' with a psycopg cursor-instance
The cursor link you show refers to the Python DB API cursor not the Postgres one. There is an example of how to do what you want here Server side cursor in section:
Note It is also possible to use a named cursor to consume a cursor created in some other way than using the DECLARE executed by execute(). For example, you may have a PL/pgSQL function returning a cursor:
CREATE FUNCTION reffunc(refcursor) RETURNS refcursor AS $$
BEGIN
OPEN $1 FOR SELECT col FROM test;
RETURN $1;
END;
$$ LANGUAGE plpgsql;
You can read the cursor content by calling the function with a regular, non-named, Psycopg cursor:
cur1 = conn.cursor()
cur1.callproc('reffunc', ['curname'])
and then use a named cursor in the same transaction to “steal the cursor”:
cur2 = conn.cursor('curname')
for record in cur2: # or cur2.fetchone, fetchmany...
# do something with record
pass
UPDATE
Be sure and close the named cursor(cur2) to release the server side cursor. So:
cur2.close()
I'm new in Python and Mysql.
I'm just trying to learn and understand how SQL and Python works together.
I'm trying to build up a database connection via SqlPool. I found the related code here:
class MySQLPool(object):
.
.
.
def execute(self, sql, args=None, commit=False):
""" Execute a sql """
# get connection form connection pool.
conn = self.pool.get_connection()
cursor = conn.cursor()
if args:
cursor.execute(sql, args) #so if there is args in the sql select then it should be run like this: (select * from table , args)
else:
cursor.execute(sql) #if there is no args then it will run the simple select statement
if commit is True:
conn.commit()
self.close(conn, cursor)
return None
else:
res = cursor.fetchall() #else it will fetch all the records
self.close(conn, cursor)
return res
Then in the main.py I'm trying to use this def:
l = 'saxonb'
def testing(request, bid, *args, **kwargs):
mysql_pool = MySQLPool()
query = """select AZON, LOGIN, JELSZO from tmrecords.luzer WHERE LOGIN='?'"""
result = mysql_pool.execute(query)
print('RESULT : \n', result)
query = """select AZON, LOGIN, JELSZO from tmrecords.luzer WHERE LOGIN='?'"""
testing(MySQLPool, query, l)
It should be found 1 record but it is coming back with an empty result.
I have tried several placeholders like ? or %s but it does not work properly and I can't see what the problem is.
It might be a del calling problem? I mean I try to call the def incorrectly?
Or placeholder problem?
Can anyone help me in this? I would really appreciate any helps. Thanks. Hexa
From Python 3.9 i'm trying to do a MySql query like this
select * from brand WHERE text='L\'Orial'.
It works fine from phpMyAdmin but fails from python for all text including quote "'"
brand = "L'Orial"
where = f"text='{brand}'"
brand_pk_id = self.getPrimaryKeyIfExistInTable('brand', where)
def getPrimaryKeyIfExistInTable(self, table, where, key='id'):
try:
sql = f"SELECT {key} FROM {table} WHERE {where}"
self.cursor.execute(sql)
result = self.cursor.fetchone()
return result[key if self.bUseDictCursor else 0] if result else None
except pymysql.MySQLError as e:
logging.error(e)
return None
I can see that python escapes all quotes, which probably causes the problem, but can not figure out how to handle it properly !!
If I turn it around and use query LIKE with underscore( _ ) as wildcard:
brand = "L_Orial"
sql = f"SELECT {key} FROM {table} WHERE text LIKE '{brand}'"
It works fine, but this is not what I want !!
If I am understanding your question correctly, your problem is as follows:
Your query must exactly read:
SELECT * from brand WHERE text='L\'Orial'
But you are currently getting something like this, when you use python to execute the query:
SELECT * from brand WHERE text='L'Orial'
If this is indeed the issue, you should be able to resolve this by simply escaping the backslash that you need to have in the query. The complete python string for your query would be:
# Python String:
"SELECT * from brand WHERE text='L\\'Orial'"
# Resulting Query
SELECT * from brand WHERE text='L\'Orial'
If you wanted to automatically fix this issue for all brands that might include a ', you can simply replace the ' with \\' before making the query. Example:
brand = "L'Orial"
brand = brand.replace("'", "\\'")
# New Python string:
# "L\\'Orial"
# Output in SQL
# "L\'Orial"
Had to fire up my local instance just to make a point.
First, some prep work...
import pymysql
table = 'ps_carrier'
key = 'id_carrier'
mysql = {
"charset": "utf8",
"database": "mystore",
"host": "localhost",
"password": "secret",
"user": "justin"
}
As somebody suggested in the comments, the following
sql = "SELECT %s FROM %s WHERE %s"
where = "name='UPS'"
with pymysql.connect(**mysql) as conn:
with conn.cursor() as cur:
cur.execute(sql, (key, table, where))
Raises an error as expected since all the (string) params are quoted, even the table name!
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
...
File "C:\Python38\site-packages\pymysql\err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''ps_carrier' WHERE 'name=\\'UPS\\''' at line 1")
If you can trust the inputs for the table name, the key, and the column name(s) then perhaps a simple query builder can help.
params = {'name': 'UPS'} # add more key--value pairs here
# use backticks in case we need to escape reserved words (OP uses MySQL)
where = " AND ".join(f"`{k}` = %s" for k in params.keys()) # .keys() just to be explicit
args = tuple([v for v in params.values()])
# backticks again
sql = f"SELECT `{key}` FROM `{table}` WHERE {where}"
print(sql)
print(args)
with pymysql.connect(**mysql) as conn:
with conn.cursor() as cur:
cur.execute(sql, args)
print(cur.fetchall())
If you need something more elaborate, there are a few modules such as Mysql Simple Query Builder and PyPika - Python Query Builder that you may want to look at (I've not used any of these.)
Code is follow. How to get replaced ? by value of variables [table, url]?
Expected SQL command is select * from OTHER_URL where url="http://a.com/a.jpg"
This SQL command occurs no error on the sqlite3 command line interface.
import sqlite3
from contextlib import closing
dbname = "ng.db"
with closing(sqlite3.connect(dbname)) as conn:
c = conn.cursor()
c.execute("CREATE TABLE IF NOT EXISTS OTHER_URL (url TEXT)")
conn.commit()
table = "OTHER_URL"
url = "http://a.com/a.jpg"
with closing(sqlite3.connect(dbname)) as conn:
c = conn.cursor()
c.execute('select * from ? where url="?"', [table, url])
print c.fetchone()
There are two errors here. Firstly, you can't use parameter substitution for table names (or column names), only for values. You need to use string interpolation for anything else.
Secondly, you don't need quotes around the value parameter; the substitution will take care of that.
So:
c.execute('select * from {} where url=?'.format(table), [url])
am trying to insert the data entered into the web form into database table,i am passing the data to the function to insert the data,but it was not successful below is my code
def addnew_to_database(tid,pid,usid,address,status,phno,email,ord_date,del_date):
connection = mysql.connector.connect(user='admin_operations', password='mypassword',host='127.0.0.1',database='tracking_system')
try:
print tid,pid,usid,address,status,phno,email,ord_date,del_date
cursor = connection.cursor()
cursor.execute("insert into track_table (tid,pid,usid,address,status,phno,email,ord_date,del_date) values(tid,pid,usid,address,status,phno,email,ord_date,del_date)")
cursor.execute("insert into user_table (tid,usid) values(tid,usid)")
finally:
connection.commit()
connection.close()
You should pass the variables as an argument to .execute instead of putting them in the actual query. E.g.:
cursor.execute("""insert into track_table
(tid,pid,usid,address,status,phno,email,ord_date,del_date)
values (%s,%s,%s,%s,%s,%s,%s,%s,%s)""",
(tid,pid,usid,address,status,phno,email,ord_date,del_date))
cursor.execute("""insert into user_table
(tid,usid)
values (%s,%s)""",(tid,usid))
You should tell us what API you are using and what the error code is.
You should define the values within the execution, right now within the sql statement as a string they are not referencing anything.
Typically when you use a variable name inside of a sql statement this way, you need to indicate that it is a variable you are binding data to. This might be replacing it with (1,2,3,4..) or (%s,%s,...) that corresponds to an ordered list or using variable names (:tid,:pid,...) that you then define the values of with a dictionary as the second argument of execute().
Like this:
track_table_data = [tid,pid,usid,address,status,phno,email,ord_date,del_date]
user_table_data = [tid,usid]
cursor.execute("insert into track_table (tid,pid,usid,address,status,phno,email,ord_date,del_date) values(1,2,3,4,5,6,7,8,9)", track_table_data)
cursor.execute("insert into user_table (tid,usid) values(1,2)",user_table_data)
or
cursor.execute("insert into track_table (tid,pid,usid,address,status,phno,email,ord_date,del_date) values(:tid,:pid,:usid,:address,:status,:phno,:email,:ord_date,:del_date))", {'tid':tid,'pid':pid,'usid':usid,'address':address,'status':status,'phno':status,'email':email,'ord_date':ord_date,'del_date':del_date})
cursor.execute("insert into user_table (tid,usid) values(:tid,:usid)",{'tid':tid,'usid':usid})