I got some python code (psycopg2) with which should insert data into a database:
def debug(self):
try:
self.connection.execute(
"SELECT test();")
res = self.connection.fetchall()
print(res)
except Exception as e:
print(e)
return
The test() function in pgsql is this:
CREATE OR REPLACE FUNCTION test(
) RETURNS setof varchar
AS $Body$
BEGIN
INSERT INTO Linie(name) VALUES('3');
RETURN QUERY(SELECT * FROM linie);
END;
$Body$ LANGUAGE plpgsql;
When i change the "name" value and execute the query in pgAdmin there is a now entry in the database. However when calling the function from python it always overrides the value.
The table is defined as follows:
CREATE TABLE Linie(
name varchar,
PRIMARY KEY (name)
);
For example with pgAdmin i can insert 1,2,3,4,5.
With python after running 5 equivalent queries it is just 5.
Calling the test function with nodeJS works fine.
When calling the function once from python then changing the insert value and then calling it from python again, the values are not replaced but inserted.
Also it does not throw any errors and returns the table as it should (except the replaced value).
why is this happening and what can i do against it?
Psycopg2 by default will not commit changes made to the database unless you explicitly call connection.commit() after executing your SQL. You could alter you code like so:
def debug(self):
try:
self.connection.execute(
"SELECT test();")
res = self.connection.fetchall()
self.connection.commit()
print(res)
except Exception as e:
print(e)
return
However, please be careful doing this as I have no information on what exactly self.connection is an instance of, therefore I have assumed it to be of type connection :)
Alternatively, when you setup your connection to the DB, set the property autocommit to True, as documented here. Example:
self.connection = psycopg2.connect(user='foo', password='bar', host='localhost', dbname='mydb')
self.connection.autocommit = True
If you are already using autocommit let me know and I'll have another look at your question.
Related
I am unable to understand why there are two queries being executed. First we are executing the prepared statement and we are using the build cypher function. The code can be found here
https://github.com/apache/age/blob/master/drivers/python/age/age.py
def execCypher(conn:ext.connection, graphName:str, cypherStmt:str, cols:list=None, params:tuple=None) -> ext.cursor :
if conn == None or conn.closed:
raise _EXCEPTION_NoConnection
cursor = conn.cursor()
#clean up the string for modification
cypherStmt = cypherStmt.replace("\n", "")
cypherStmt = cypherStmt.replace("\t", "")
cypher = str(cursor.mogrify(cypherStmt, params))
cypher = cypher[2:len(cypher)-1]
preparedStmt = "SELECT * FROM age_prepare_cypher({graphName},{cypherStmt})"
cursor = conn.cursor()
try:
cursor.execute(sql.SQL(preparedStmt).format(graphName=sql.Literal(graphName),cypherStmt=sql.Literal(cypher)))
except SyntaxError as cause:
conn.rollback()
raise cause
except Exception as cause:
conn.rollback()
raise SqlExecutionError("Execution ERR[" + str(cause) +"](" + preparedStmt +")", cause)
stmt = buildCypher(graphName, cypher, cols)
cursor = conn.cursor()
try:
cursor.execute(stmt)
return cursor
except SyntaxError as cause:
conn.rollback()
raise cause
except Exception as cause:
conn.rollback()
raise SqlExecutionError("Execution ERR[" + str(cause) +"](" + stmt +")", cause)
Both statements perform the same operation.
The difference is that preparedStmt and buildCypher function use different form of cypher queries as shown in code. (cypherStmt & cypher) And their code for building the query is a bit different.
I can't tell you why it's done this way but I'll show you why it's different. Also apologies but I'm not used to Python or C.
The preparedStatement is calling a custom postgres function age_prepare_cypher in this file here apache/age/src/backend/utils/adt/age_session_info.c, which calls set_session_info(graph_name_str, cypher_statement_str);.
And the set_session_info in this file here apache/age/src/backend/utils/adt/age_session_info.c just sets it to a global variable session_info_cypher_statement.
So your graph name and query are being set in the session.
There's another function that gets your graph name and query back out of the session, and that is the convert_cypher_to_subquery. It only gets them out if is_session_info_prepared() is true, and only if graph_name and query_str provided to it are NULL.
Seems strange right? But now let's look at this bit of the python buildCypher function code:
stmtArr = []
stmtArr.append("SELECT * from cypher(NULL,NULL) as (")
stmtArr.append(','.join(columnExp))
stmtArr.append(");")
return "".join(stmtArr)
It's taking your query and saying your graph name and query string are NULL.
So we can conclude that the prepare statement is storing those values in session memory, and then when you execute your statement after using buildCypher, it's getting them out of memory and completing the statement again.
I can't explain exactly why or how it does it, but I can see a chunk of test sql in the project that is doing the same sort of thing here:
-- should return true and execute cypher command
SELECT * FROM age_prepare_cypher('analyze', 'MATCH (u) RETURN (u)');
SELECT * FROM cypher(NULL, NULL) AS (result agtype);
So tl;dr, executing the prepareStatement is storing it in session memory, and executing the normal statement after running it through buildCypher is grabbing what was just stored in the session.
using a python script running in a Flask webapp I execute a simple select query on a table of the
connected database (I'm using the mysql python connector library)
import mysql.connector
...
stmt = "SELECT * FROM %s"
try:
self.cur.execute(stmt % table)
print('stmt executed')
result = self.cur.fetchall()
print('result fetched')
except Exception as ex:
self.handle_closure(ex, error=True) # function for handling exceptions
return -1
The issue is that the execution of this code on a particular table raises the exeption mentioned
in the title: "struct.error: unpack requires a buffer of 4 bytes".
The same function works well on a different table
in the stdout "stmt executed" gets printed while "result fetched" doesn't
Thank you
PS. adding the query used to create the table since it may be usefull to know the schema of the table
CREATE TABLE `xxx` (
`id` INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
`aaa` INT NOT NULL,
`bbb` CHAR(15),
`ccc` TIME,
`ddd` TIME
);
ALTER TABLE xxx
ADD CONSTRAINT constraint
FOREIGN KEY (aaa) REFERENCES yyy(id);
I have written the below function to simplify the way i call stored procedures. I need help on how to return the out parameters.
def call_procedure(dbname,procedure_name,parameters_list=[]):
with connections[dbname].cursor() as cursor:
try:
cursor.callproc(procedure_name, parameters_list)
except Exception as e:
raise Exception(e)
Procedure call:
call_procedure('testdb',[1,2,3,''])
In the call I need to print the out parameters.
Django lacks important information in the docs.
For detail explanation,about PL/SQL execution (Oracle DB), check this link in the cx_Oracle docs page
from django.db import connection
with connection.cursor() as cursor:
outVal = cursor.var(int).var
cursor.callproc('myproc', [123, outVal])
print(outVal.getvalue()) # will print 246
Procedure:
create or replace procedure myproc (
a_Value1 number,
a_Value2 out number
) as
begin
a_Value2 := a_Value1 * 2;
end;
I'm trying to execute a stored procedure to query a table, and am having trouble passing through a parameter successfully.
title=cursor.execute("SELECT titlequery(%s)", str(member_id))`
titlequery() is created by this:
CREATE OR REPLACE FUNCTION public.titlequery(mid text)
RETURNS text AS
$BODY$
SELECT title FROM Member WHERE member_id=mid;
$BODY$
LANGUAGE sql
And the error I'm getting:
modules.pg8000.core.ProgrammingError: ('ERROR', '42P18', 'could not
determine data type of parameter $2', 'postgres.c', '1356',
'exec_parse_message', '', '')
Does anyone know what's happening here?
PEP-249 specifies API for database drivers and pg8000 follows this API as well
pg8000 is a DB-API 2.0 compatible pure-Python interface to the PostgreSQL database engine.
From PEP-249 execute method specification:
Parameters may be provided as sequence or mapping and will be bound to variables in the operation.
We can see at pg8000 sources an example of how to pass parameters to query.
So you should pass a tuple/list of values, not value itself.
Also we should execute query first and then fetch its results using fetchone or fetchmany or fetchall because execute itself returns None (more at sources). I guess OP needs one record, so we're going to use fetchone.
Note: fetchone method returns record represented as tuple, so if we need first coordinate, then we should get it using zero index.
In your case you should try:
parameters = (str(member_id),) # WARNING: don't miss the comma
cursor.execute("SELECT titlequery(%s)", parameters)
title = cursor.fetchone()[0]
or
parameters = [str(member_id)]
cursor.execute("SELECT titlequery(%s)", parameters)
title = cursor.fetchone()[0]
Example
This worked for me
import pg8000
table_definition = """
CREATE TABLE Member(
title VARCHAR(40) NOT NULL,
member_id VARCHAR(40) NOT NULL)
"""
procedure_definition = """
CREATE OR REPLACE FUNCTION public.titlequery(mid text)
RETURNS text AS
$BODY$
SELECT title FROM Member WHERE member_id=mid;
$BODY$
LANGUAGE sql
"""
connection = pg8000.connect(database='database',
user='username',
password='password',
host='hostname',
port=5432)
cursor = connection.cursor()
# Preparation
cursor.execute(table_definition)
cursor.execute(procedure_definition)
values = ('Information', 'A000042553')
cursor.execute('INSERT INTO Member (title, member_id) VALUES (%s, %s)',
values)
# Reading stored procedure result
parameters = ('A000042553',)
cursor.execute("SELECT titlequery(%s)", parameters)
title = cursor.fetchone()[0]
print(title)
# Cleanup
cursor.close()
connection.close()
gives us
Information
I am using raw sql queries for inserting the data into DB. The insertion is working right, Now I want to perform some checks on this insert query e.g. If the query has inserted the data or not
suppose I have insert query like
cursor.execute("some insert query" )
Now I want to know whether cursor.execute has inserted the row then show me some text like success and if it fails to insert for some reason then show me text like error and also if the row is already inserted then show, row already exist.
But I don't know how to perform these checks on cursor.execute.
edit
for i in range(numrows):
row = cursor.fetchone()
if row[6]==1:
arr["user_id"]=row[0]
arr["email"]=row[1]
arr["style_quiz_score"]=row[2]
arr["style_quiz_answer"]=row[3]
arr["date_joined"]=row[4]
arr["is_active"]=row[5]
arr['firstname'] = row[7]
arr["username"]=re.sub(r'[^a-zA-Z0-9]', '_', arr["email"])
elif row[6]==2:
arr['lastname'] = row[7]
cursor1.execute("insert into auth_user(id,username,first_name,last_name,email,password,is_staff,is_active,is_superuser,date_joined,last_login) values(%s,%s,%s,%s,%s,'NULL',0,%s,0,%s,0)",[arr["user_id"],arr["username"],arr['firstname'],arr['lastname'],arr["email"],arr["is_active"],arr["date_joined"]])
when i am executing cursor1.execute outside forloop than it insert the last entry , but if i execute it in inside forloop than it gives error and nothing will be inserted
Assuming you're using Django (you're not specific about it in your question, but you're using the django tag), you need to do transaction.commit_unless_managed() (from django.db import transaction) after issuing the insert query with cursor.execute.
You can check for exceptions when calling commit_unless_managed to see if the insert went well or not:
from django.db import connection, transaction, DatabaseError, IntegrityError
cursor = connection.cursor()
cursor.execute("some insert query" )
try:
transaction.commit_unless_managed()
except DatabaseError, IntegrityError:
print 'error'
else:
print 'success'