I'm using pyodbc to query to an SQL Server database
import datetime
import pyodbc
conn = pyodbc.connect("Driver={SQL Server};Server='dbserver',Database='db',
TrustedConnection=Yes")
cursor = conn.cursor()
ratings = ("PG-13", "PG", "G")
st_dt = datetime(2010, 1, 1)
end_dt = datetime(2010, 12, 31)
cursor.execute("""Select title, director, producer From movies
Where rating In ? And release_dt Between ? And ?""",
ratings, str(st_dt), str(end_dt))
but am receiving the error below. Does the tuple parameter need to be handled in a different way? Is there a better way to structure this query?
('42000', "[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]Line 9:
Incorrect syntax near '#P1'. (170) (SQLExecDirectW);
[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]
Statement(s) could not be prepared. (8180)")
UPDATE:
I was able to get this query to work using the string formatting operator, which isn't ideal as it introduces security concerns.
import datetime
import pyodbc
conn = pyodbc.connect("Driver={SQL Server};Server='dbserver',Database='db',
TrustedConnection=Yes")
cursor = conn.cursor()
ratings = ("PG-13", "PG", "G")
st_dt = datetime(2010, 1, 1)
end_dt = datetime(2010, 12, 31)
cursor.execute("""Select title, director, producer From movies
Where rating In %s And release_dt Between '%s' And '%s'""" %
(ratings, st_dt, end_dt))
To expand on Larry's second option - dynamically creating a parameterized string, I used the following successfully:
placeholders = ",".join("?" * len(code_list))
sql = "delete from dbo.Results where RESULT_ID = ? AND CODE IN (%s)" % placeholders
params = [result_id]
params.extend(code_list)
cursor.execute(sql, params)
Gives the following SQL with the appropriate parameters:
delete from dbo.Results where RESULT_ID = ? AND CODE IN (?,?,?)
You cannot parameterize multiple values in an IN () clause using a single string parameter. The only way to accomplish that is:
String substitution (as you did).
Build a parameterized query in the form IN (?, ?, . . ., ?) and then pass in a separate parameter for each place holder. I'm not an expert at Python to ODBC but I imagine that this is particularly easy to do in a language like Python. This is safer because you get the full value of parameterization.
To expand on Larry and geographika's answers:
ratings = ('PG-13', 'PG', 'G')
st_dt = datetime(2010, 1, 1)
end_dt = datetime(2010, 12, 31)
placeholders = ', '.join('?' * len(ratings))
vars = (*ratings, st_dt, end_dt)
query = '''
select title, director, producer
from movies
where rating in (%s)
and release_dt between ? and ?
''' % placeholders
cursor.execute(query, vars)
With the placeholder, this will return a query of:
select title, director, producer
from movies
where rating in (?, ?, ?)
and release_dt between ? and ?
If you pass in ratings, it'll attempt to fit all of its items into one ?. However, if we pass in *ratings, and each item in ratings will take its place in the in() clause. Thus, we pass the tuple (*ratings, st_dt, end_dt) to cursor.execute().
The problem is your tuple. The ODBC connection is expecting a string to construct the query and you are sending a python tuple. And remember that you have to get the string quoting correct. I'm assuming that the number of ratings you will be looking for varies. There is probably a better way, but my pyodbc tends to be simple and straightforward.
Try the following:
import datetime
import pyodbc
conn = pyodbc.connect("Driver={SQL Server};Server='dbserver',Database='db',
TrustedConnection=Yes")
def List2SQLList(items):
sqllist = "%s" % "\",\"".join(items)
return sqllist
cursor = conn.cursor()
ratings = ("PG-13", "PG", "G")
st_dt = datetime(2010, 1, 1)
end_dt = datetime(2010, 12, 31)
cursor.execute("""Select title, director, producer From movies
Where rating In (?) And release_dt Between ? And ?""",
List2SQLList(ratings), str(st_dt), str(end_dt))
Related
I have a database with some records that have a date field of "05221999". I am trying to do a SQL query from the input of the user based on just the month and year. In this case I am interested in all the records with the month of 05 and the year of 1999.
Unfortunately, I can't get the Python/SQL syntax correct. Here is my code so far:
def submitact(self):
date = self.md.get()
month = date[0:2]
year = date[2:7]
db = pymysql.connect("localhost", "username", "password", "database")
cursor = db.cursor()
cursor.execute("SELECT * FROM `table` WHERE `Code` = 'RM' AND `Date` LIKE %s'_'%s", (month, year))
results = cursor.fetchall()
print(results)
cursor.close()
db.close()
I've done several variations on the SELECT statement and they either return errors or nothing.
Thanks!
In the code snippet below, I used f-string style to format the query string
[...]
query = f"SELECT * FROM `table` WHERE `Code` = 'RM' AND LEFT(`Date`, 2) = '{month}' AND RIGHT(`Date`, 4) = '{year}'"
cursor.execute(query)
[...]
try with this:
query = "SELECT * 'table' WHERE 'Code' = 'RM' AND 'Date' LIKE '%{0}_{1}'".format(month, year)
cursor.execute(query)
In this way, 'query' variable value will be:
"SELECT * FROM 'table' WHERE 'Code' = 'RM' AND 'Date' LIKE '%05_1999'"
For more information about string formatting, let's have a look to Python String Formatting Best Practices - Real Python
I'm trying to write a python script to get a count of some tables for monitoring which looks a bit like the code below. I'm trying to get an output such as below and have tried using python multi-dimensional arrays but not having any luck.
Expected Output:
('oltptransactions:', [(12L,)])
('oltpcases:', [(24L,)])
Script:
import psycopg2
# Connection with the DataBase
conn = psycopg2.connect(user = "appuser", database = "onedb", host = "192.168.1.1", port = "5432")
cursor = conn.cursor()
sql = """SELECT COUNT(id) FROM appuser.oltptransactions"""
sql2 = """SELECT count(id) FROM appuser.oltpcases"""
sqls = [sql,sql2]
for i in sqls:
cursor.execute(i)
result = cursor.fetchall()
print('Counts:',result)
conn.close()
Current output:
[root#pgenc python_scripts]# python multi_getrcount.py
('Counts:', [(12L,)])
('Counts:', [(24L,)])
Any help is appreciated.
Thanks!
I am a bit reluctant to show this way, because best practices recommend to never build a dynamic SQL string but always use a constant string and parameters, but this is one use case where computing the string is legit:
a table name cannot be a parameter in SQL
the input only comes from the program itself and is fully mastered
Possible code:
sql = """SELECT count(*) from appuser.{}"""
tables = ['oltptransactions', 'oltpcases']
for t in tables:
cursor.execute(sql.format(t))
result = cursor.fetchall()
print("('", t, "':,", result, ")")
I believe something as below, Unable to test code because of certificate issue.
sql = """SELECT 'oltptransactions', COUNT(id) FROM appuser.oltptransactions"""
sql2 = """SELECT 'oltpcases', COUNT(id) FROM appuser.oltpcases"""
sqls = [sql,sql2]
for i in sqls:
cursor.execute(i)
for name, count in cursor:
print ("")
Or
sql = """SELECT 'oltptransactions :'||COUNT(id) FROM appuser.oltptransactions"""
sql2 = """SELECT 'oltpcases :'||COUNT(id) FROM appuser.oltpcases"""
sqls = [sql,sql2]
for i in sqls:
cursor.execute(i)
result = cursor.fetchall()
print(result)
I want to fetch all rows from MySQL table with
query = "SELECT * FROM %s WHERE last_name=%s"
cursor.execute(query, ("employees","Smith"))
but I'm getting
You have an error in your SQL syntax. When I try
query = "SELECT * FROM employees WHERE last_name=%s"
cursor.execute(query, ("Smith",))
all is fine.
Documentation says
cursor.execute(operation, params=None, multi=False)
The parameters found in the tuple or dictionary params are bound to the variables in the operation.link on docs
The first will generate an SQL like this:
SELECT * FROM 'employees' WHERE last_name='smith'
The parameters are SQL quoted.
If you really need to have a table name as param, you must proceed in 2 steps:
table_name = 'employees'
query_tpl = "SELECT * FROM {} WHERE last_name=%s"
query = query_tpl.format(table_name)
cursor.execute(query, ("Smith",))
you need to add the quote symbol. So the query will be like
SELECT * FROM employees WHERE last_name='Smith'
Change both your query to
query = "SELECT * FROM %s WHERE last_name='%s'"
query = "SELECT * FROM employees WHERE last_name='%s'"
You can't use a parameter for the table name in the execute call.
But you can use Python string interpolation for that:
query = "SELECT * FROM %s WHERE last_name=%s" %("employees","Smith")
cursor.execute(query)
You can't use a table name as a parameter. you are generating invalid sql with your code that is putting quotes around each string. the table name cannot have quotes around it.
sql you are generating
select * from 'employees' where last_name = 'Smith'
What sql you want
select * from employees where last_name = 'Smith'
you would have to format the string first like the example below.
query = "SELECT * from {} wherre last_name ='{}'"
cursor.execute(query.format("employees","Smith"))
using code like this does open up the possibility of SQL injection. so please bear that in mind.
query="SELECT * FROM %s WHERE name=%s",(employees,smith)
cursor.execute(query)
rows = cursor.fetchall()
Try this one. Hopefully it works for you.
I have the following code which is attempting to pull several SQL queries from an Access database
import pyodbc
import datetime
conx = pyodbc.connect("Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=C:\\Users\\Stuart\\PycharmProjects\\untitled\\Databases\\SandP.accdb;")
cursor=conx.cursor()
query=""" SELECT DISTINCT Date_ FROM Closing_prices
WHERE Date_ >= ? AND Date_ < ?"""
params1 = (datetime.date(2011, 8, 10), datetime.date(2014, 4, 30))
cursor.execute(query, params1)
dates = list()
for date in cursor:
dates.append(date[0])
for date in dates:
params2 = date
cursor = conx.cursor()
query= '''SELECT Volatility,Last_price FROM Volatility v,Closing_prices c WHERE c.Date_= ? and v.Date_=c.Date_'''
cursor.execute(query,params2)
for (vol,price) in cursor:
volatility=float(vol)
closing_price=float(price)
cursor.close()
cursor = conx.cursor()
if (date.weekday()==4):
nextDay=(date + datetime.timedelta(days=3))
else:
nextDay=(date + datetime.timedelta(days=1))
query= '''SELECT Date_,Time_, Close_ FROM Intraday_values WHERE (date = ? and time >= ?) or (date = ? and time <= ?)'''
params3 = (date,datetime.time(15, 30),nextDay,datetime.time(15, 14))
cursor.execute(query,params3)
This last bit is throwing up the following error:
Traceback (most recent call last):
File "C:/Users/Stuart/PycharmProjects/untitled/Apache - Copy.py", line 67, in <module>
cursor.execute(query,params3)
pyodbc.Error: ('07002', '[07002] [Microsoft][ODBC Microsoft Access Driver] Too few parameters. Expected 6. (-3010) (SQLExecDirectW)')
The request is attempting to pull out the Date_, Time_ and Close_ items from the table, for the dates passed to the request as it iterates through the list of dates created previously, along with cut off times of "after 15:30 for "date" and "before 15:14" for "date+1".
Firstly, why would this be expecting 6 parameters when there are only 4 question marks (?) in the SQL request - have I not formed this properly?
Also, I took a stab at the parameter creation for a datetime.time. Is this incorrectly formed also?
I'm a bit out of my depth!
Works after changing the query
to
query= '''SELECT Date_,Time_, Close_
FROM Intraday_values
WHERE (Date_ = ? and Time_ >= ?)
OR (Date_ = ? and Time_ <= ?)'''
The following logic works with the mysqldb module (see python mysqldb multiple cursors for one connection), but I am getting the following error with mysql.connector on cursor2.execute(sql)
"Unread result found."
I realize that I can use a join to combine these 2 simple sql statements and avoid the need for a second cursor, but my real world example is more complex and requires a second sql statement.
Assuming I need to execute 2 separate sql statements (1 for the loop and 1 inside the loop), how should this be done with the mysql.connector module?
import datetime
import mysql.connector
db = mysql.connector.connect(user='alan', password='please', host='machine1', database='mydb')
cursor1 = db.cursor()
cursor2 = db.cursor()
sql = """
SELECT userid,
username,
date
FROM user
WHERE date BETWEEN %s AND %s
"""
start_date = datetime.date(1999, 1, 1)
end_date = datetime.date(2014, 12, 31)
cursor1.execute(sql, (start_date, end_date))
for (userid, username, date) in cursor1:
sql = """
select count(*)
from request
where assigned = '%s'
""" % (userid)
cursor2.execute(sql)
requestcount = cursor2.fetchone()[0]
print userid, requestcount
cursor2.close()
cursor1.close()
db.close()
This mysqldb version works just fine:
import datetime
import MySQLdb
db = MySQLdb.connect(user='alan', passwd='please', host='machine1', db='mydb')
cursor1 = db.cursor()
cursor2 = db.cursor()
sql = """
SELECT userid,
username,
date
FROM user
WHERE date BETWEEN %s AND %s
"""
start_date = datetime.date(1999, 1, 1)
end_date = datetime.date(2014, 12, 31)
cursor1.execute(sql, (start_date, end_date))
for (userid, username, date) in cursor1:
sql = """
select count(*)
from request
where assigned = '%s'
""" % (userid)
cursor2.execute(sql)
requestcount = cursor2.fetchone()[0]
print userid, requestcount
cursor2.close()
cursor1.close()
db.close()
MySQL Connector/Python is, by default, non-buffering. This means the data is not fetched automatically and you need to 'consume' all rows. (It works with MySQLdb because that driver is buffering by default.)
Using Connector/Python you have to use the buffered-argument set to True for cursor you use as iterator. In the OP's question, this would be cursor1:
cursor1 = db.cursor(buffered=True)
cursor2 = db.cursor()
You can also use buffered=True as connection argument to make all cursor buffering instantiated by this connection buffering.