I have a SQL statement that works in mysql:
SELECT * FROM `ps_message` WHERE `id_order` = 111 ORDER BY id_message asc LIMIT 1
What is wrong with the following statement in Python:
cursor2.execute("SELECT * FROM ps_message WHERE id_order='%s'" % order["id_order"] " ORDER BY id_message asc LIMIT 1")
How should the syntax be in Python to work?
You have a syntax error in string formatting. Should be:
cursor2.execute("SELECT * FROM ps_message WHERE id_order='%s' ORDER BY id_message asc LIMIT 1" % order["id_order"])
Using format() is also preferable over old-style string formatting. Read more about it here.
Pass the order number as a query parameter.
e.g.
cursor2.execute("SELECT * FROM ps_message WHERE id_order=%s ORDER BY id_message asc LIMIT 1", [ order["id_order"] ])
Note that when using query parameters you don't put quotes around the %s.
This approach is recommended to avoid the risk of sql injection attacks.
It should also be more efficient if there are many queries.
https://docs.python.org/2/library/sqlite3.html
http://pymssql.org/en/stable/pymssql_examples.html
I am working with distance between objects, so I use haversine formaula in a raw query:
query = """SELECT *, (%d*acos(cos(radians(%2f))
*cos(radians(client_address.latitude))*cos(radians(client_address.longitude)-radians(%2f))
+sin(radians(%2f))*sin(radians(client_address.latitude))))
AS distance FROM jobs LEFT OUTER JOIN client_address ON ( client_address.id = jobs.location_id) HAVING
distance < %2f AND status = 1 ORDER BY distance LIMIT 0, %d""" % (
earth,
float(self.latitude),
float(self.longitude),
float(self.latitude),
distance,
limit
)
This query is a object method, that's why you see self reference. I find out that I cant do extra filters for the resulting RawQuerySet unless I perform this query in a .extra() or .annotate() method. I wonder if it is possible, Any suggestion? Thanks!
Let's suppose I have the following table :
Id (int, Primary Key) | Value (varchar)
----------------------+----------------
1 | toto
2 | foo
3 | bar
I would like to know if giving two request, the result of the first must be contained in the result of the second without executing them.
Some examples :
# Obvious example
query_1 = "SELECT * FROM example;"
query_2 = "SELECT * FROM example WHERE id = 1;"
is_sub_part_of(query_2, query_1) # True
# An example we can't know before executing the two requests
query_1 = "SELECT * FROM example WHERE id < 2;"
query_2 = "SELECT * FROM example WHERE value = 'toto' or value = 'foo';"
is_sub_part_of(query_2, query_1) # False
# An example we can know before executing the two requests
query_1 = "SELECT * FROM example WHERE id < 2 OR value = 'bar';"
query_2 = "SELECT * FROM example WHERE id < 2 AND value = 'bar';"
is_sub_part_of(query_2, query_1) # True
# An example about columns
query_1 = "SELECT * FROM example;"
query_2 = "SELECT id FROM example;"
is_sub_part_of(query_2, query_1) # True
Do you know if there's a module in Python that is able to do that, or if it's even possible to do ?
Interesting problem. I don't know of any library that will do this for you. My thoughts:
Parse the SQL, see this for example.
Define which filtering operations can be added to a query that can only result in the same or a narrower result set. "AND x" can always be added, I think, without losing the property of being a subset. "OR x" can not. Anything else you can do to the query? For example "SELECT *", vs "SELECT x", vs "SELECT x, y".
Except for that, I can only say it's an interesting idea. You might get some more input on DBA. Is this an idea you're researching or is it related to a real-world problem you are solving, like optimizing a DB query? Maybe your question could be updated with information about this, since this is not a common way to optimize queries (unless you're working on the DB engine itself, I guess).
I have the following code in python. I get this error ->tuple indices must be integers, not str
How can I pass these values into the query? I have other examples where this approach works perfectly, i don't understand why it's failling here.
def request_events_json(uei,interval,conn):
cur = conn.cursor()
events_query ="""select e.nodeid,n.nodelabel,e.ipaddr,count(*) as total,min(e.eventcreatetime),max(e.eventcreatetime),(regexp_matches (e.eventlogmsg,E': %(.*)'))[1] as msglog
from events e, node n where e.eventuei = (%s) and e.eventcreatetime > now() - interval (%s) and n.nodeid=e.nodeid
group by n.nodelabel,e.nodeid,e.ipaddr,msglog
order by e.nodeid, count(*) desc limit 10;"""
try:
print('## Requesting events ##')
cur.execute(events_query,('uei.opennms.org/syslogd/cisco/line','5 min'))
.......
With my version of PostgreSQL the round brackets after interval are forbidden.
Update:
It is the percent-sign in the regexp. Double it.
I've googled around a bit, but maybe I didn't put the correct magik incantation into the search box.
Does anyone know how to get output parameters from a stored procedure in Python? I'm using pymssql to call a stored procedure, and I'm not sure of the correct syntax to get the output parameter back. I don't think I can use any other db modules since I'm running this from a Linux box to connect to a mssql database on a MS Server.
import pymssql
con = pymssql.connect(host='xxxxx',user='xxxx',password='xxxxx',database='xxxxx')
cur = con.cursor()
query = "EXECUTE blah blah blah"
cur.execute(query)
con.commit()
con.close()
I'm not a python expert but after a brief perusing of the DB-API 2.0 I believe you should use the "callproc" method of the cursor like this:
cur.callproc('my_stored_proc', (first_param, second_param, an_out_param))
Then you'll have the result in the returned value (of the out param) in the "an_out_param" variable.
If you cannot or don't want to modify the original procedure and have access to the database you can write a simple wrapper procedure that is callable from python.
For example, if you have a stored procedure like:
CREATE PROC GetNextNumber
#NextNumber int OUTPUT
AS
...
You could write a wrapper like so which is easily callable from python:
CREATE PROC GetNextNumberWrap
AS
DECLARE #RNextNumber int
EXEC GetNextNumber #RNextNumber
SELECT #RNextNumber
GO
Then you could call it from python like so:
import pymssql
con = pymssql.connect(...)
cur = con.cursor()
cur.execute("EXEC GetNextNumberWrap")
next_num = cur.fetchone()[0]
If you make your procedure produce a table, you can use that result as a substitute for out params.
So instead of:
CREATE PROCEDURE Foo (#Bar INT OUT, #Baz INT OUT) AS
BEGIN
/* Stuff happens here */
RETURN 0
END
do
CREATE PROCEDURE Foo (#Bar INT, #Baz INT) AS
BEGIN
/* Stuff happens here */
SELECT #Bar Bar, #Baz Baz
RETURN 0
END
It looks like every python dbapi library implemented on top of freetds (pymssql, pyodbc, etc) will not be able to access output parameters when connecting to Microsoft SQL Server 7 SP3 and higher.
http://www.freetds.org/faq.html#ms.output.parameters
I was able to get an output value from a SQL stored procedure using Python. I could not find good help getting the output values in Python. I figured out the Python syntax myself, so I suspect this is worth posting here:
import sys, string, os, shutil, arcgisscripting
from win32com.client import Dispatch
from adoconstants import *
#skip ahead to the important stuff
conn = Dispatch('ADODB.Connection')
conn.ConnectionString = "Provider=sqloledb.1; Data Source=NT38; Integrated Security = SSPI;database=UtilityTicket"
conn.Open()
#Target Procedure Example: EXEC TicketNumExists #ticketNum = 8386998, #exists output
Cmd = Dispatch('ADODB.Command')
Cmd.ActiveConnection = conn
Cmd.CommandType = adCmdStoredProc
Cmd.CommandText = "TicketNumExists"
Param1 = Cmd.CreateParameter('#ticketNum', adInteger, adParamInput)
Param1.Value = str(TicketNumber)
Param2 = Cmd.CreateParameter('#exists', adInteger, adParamOutput)
Cmd.Parameters.Append(Param1)
Cmd.Parameters.Append(Param2)
Cmd.Execute()
Answer = Cmd.Parameters('#exists').Value
2016 update (callproc support in pymssql 2.x)
pymssql v2.x offers limited support for callproc. It supports OUTPUT parameters using the pymssql.output() parameter syntax. Note, however, that OUTPUT parameters can only be retrieved with callproc if the stored procedure does not also return a result set. That issue is discussed on GitHub here.
For stored procedures that do not return a result set
Given the T-SQL stored procedure
CREATE PROCEDURE [dbo].[myDoubler]
#in int = 0,
#out int OUTPUT
AS
BEGIN
SET NOCOUNT ON;
SELECT #out = #in * 2;
END
the Python code
import pymssql
conn = pymssql.connect(
host=r'localhost:49242',
database='myDb',
autocommit=True
)
crsr = conn.cursor()
sql = "dbo.myDoubler"
params = (3, pymssql.output(int, 0))
foo = crsr.callproc(sql, params)
print(foo)
conn.close()
produces the following output
(3, 6)
Notice that callproc returns the parameter tuple with the OUTPUT parameter value assigned by the stored procedure (foo[1] in this case).
For stored procedures that return a result set
If the stored procedure returns one or more result sets and also returns output parameters, we need to use an anonymous code block to retrieve the output parameter value(s):
Stored Procedure:
ALTER PROCEDURE [dbo].[myDoubler]
#in int = 0,
#out int OUTPUT
AS
BEGIN
SET NOCOUNT ON;
SELECT #out = #in * 2;
-- now let's return a result set, too
SELECT 'foo' AS thing UNION ALL SELECT 'bar' AS thing;
END
Python code:
sql = """\
DECLARE #out_value INT;
EXEC dbo.myDoubler #in = %s, #out = #out_value OUTPUT;
SELECT #out_value AS out_value;
"""
params = (3,)
crsr.execute(sql, params)
rows = crsr.fetchall()
while rows:
print(rows)
if crsr.nextset():
rows = crsr.fetchall()
else:
rows = None
Result:
[('foo',), ('bar',)]
[(6,)]
You might also look at using SELECT rather than EXECUTE. EXECUTE is (iirc) basically a SELECT that doesn't actually fetch anything (, just makes side-effects happen).
You can try to reformat query:
import pypyodc
connstring = "DRIVER=SQL Server;"\
"SERVER=servername;"\
"PORT=1043;"\
"DATABASE=dbname;"\
"UID=user;"\
"PWD=pwd"
conn = pypyodbc.connect(connString)
cursor = conn.cursor()
query="DECLARE #ivar INT \r\n" \
"DECLARE #svar VARCHAR(MAX) \r\n" \
"EXEC [procedure]" \
"#par1=?," \
"#par2=?," \
"#param1=#ivar OUTPUT," \
"#param2=#svar OUTPUT \r\n" \
"SELECT #ivar, #svar \r\n"
par1=0
par2=0
params=[par1, par2]
result = cursor.execute(query, params)
print result.fetchall()
[1]https://amybughunter.wordpress.com/tag/pypyodbc/
Here's how I did it, the key is to declare output parameter first:
import cx_Oracle as Oracle
conn = Oracle.connect('xxxxxxxx')
cur = conn.cursor()
idd = cur.var(Oracle.NUMBER)
cur.execute('begin :idd := seq_inv_turnover_id.nextval; end;', (idd,))
print(idd.getvalue())
I use pyodbc and then convert the pyodbc rows object to a list. Most of the answers show a query declaring variables as part of the query. But I would think you declare your variables as part of the sp, thus eliminating an unnecessary step in python. Then, in python, all you have to do is pass the parameters to fill in those variables.
Here is the function I use to convert the pyodbc rows object to a usable list (of lists) (note that I have noticed pyodbc sometimes adds trailing spaces, so I account for that which works well for me):
def convert_pyodbc(pyodbc_lst):
'''Converts pyodbc rows into usable list of lists (each sql row is a list),
then examines each list for list elements that are strings,
removes trailing spaces, and returns a usable list.'''
usable_lst = []
for row in pyodbc_lst:
e = [elem for elem in row]
usable_lst.append(e)
for i in range(0,len(usable_lst[0])):
for lst_elem in usable_lst:
if isinstance(lst_elem[i],str):
lst_elem[i] = lst_elem[i].rstrip()
return usable_lst
Now if I need to run a stored procedure from python that returns a results set, I simply use:
strtdate = '2022-02-21'
stpdate = '2022-02-22'
conn = mssql_conn('MYDB')
cursor = conn.cursor()
qry = cursor.execute(f"EXEC mystoredprocedure_using_dates
'{strtdate}','{stpdate}' ")
results = convert_pyodbc(qry.fetchall())
cursor.close()
conn.close()
And sample results which I then take and write to a spreadsheet or w/e:
[[datetime.date(2022, 2, 21), '723521', 'A Team Line 1', 40, 9],
[datetime.date(2022, 2, 21), '723522', 'A Team Line 2', 15, 10],
[datetime.date(2022, 2, 21), '723523', 'A Team Line 3', 1, 5],
[datetime.date(2022, 2, 21), '723686', 'B Team Line 1', 39, 27],
[datetime.date(2022, 2, 21), '723687', 'B Team Line 2', 12, 14]]