cur=connection.cursor()
def fillDoctors(key_bodyloc,proportion):
bodyloc_specialty_query="select distinct Speciality from body_speciality where body_location in (%s) "
#cur.execute(bodyloc_specialty_query)
data1=([key_bodyloc])
#print(bodyloc_specialty_query,data)
cur.execute(bodyloc_specialty_query,data1)
results=cur.fetchall()
specialities=[x[0] for x in results]
condition=""
for speciality in specialities:
print(str(speciality))
condition=condition+"'%"+speciality+"%'"+" or Speciality like "
#print(speciality)
#print(condition)
specialty_doctors_query="select DoctorName,Speciality,ClinicName from Doctors where Speciality like %s limit %s"
data2=([condition,proportion])
print(specialty_doctors_query,data2)
cur.execute(specialty_doctors_query,data2)
final=cur.fetchall()
print(final)
The line final=cur.fetchall() returns an empty tuple in each iteration. I've verified that the table Doctors isn't empty and the code works fine when the 'condition' is hard-coded. The code is supposed to print the doctor details for each speciality. Can anyone tell me why this is happening ?
Seems to me you are not passing your data1 into bodyloc_specialty_query properly when calling cur.execute(bodyloc_specialty_query,data1) and it causes the problem.
The syntax of string injection in Python is the following:
str1 = "some string %s" % "other string"
Instead of this way of adding a string to a string, use Pythons format builtin function:
str1 = "some string {str2}".format(str2="other_string").
But make sure that your str2 is a string or convertable to string.
I can see your data1 is a list, not str. You should convert it to str first. Good luck.
The whole point of passing in parameters to the execute method is that they get escaped. So, your condition is treated as a single string, rather than a series of parameters joined by SQL. Also, you can't use a parameter for the LIMIT value.
You need to build up some SQL and placeholders by interpolation, then just pass the values.
like = "speciality LIKE %%%s%%"
condition = like + (" OR {}".format(like)) * len(specialities)
query = "select DoctorName,Speciality,ClinicName from Doctors where {} LIMIT {}".format(condition, proportion)
cursor.execute(query, specialities)
for speciality in specialities:
condition=condition+"'%"+speciality+"%'"+" or Speciality like "
Related
I am trying to make a dynamic mySQL update statement. My update fails if certain characters are in the string.
import mysql.connector as sql
import MySQLdb
#Values are taken from a wxGrid.
key_id = str("'") + str(self.GetCellValue(event.GetRow(),1)) + str("'") #Cell column with unique ID
target_col = str(self.GetColLabelValue(event.GetCol())) #Column being updated
key_col = str(self.GetColLabelValue(1)) #Unique ID column
nVal = str("'")+self.GetCellValue(event.GetRow(),event.GetCol()) + str("'") #Updated value
sql_update = f"""Update {tbl} set {target_col} = {nVal} where {key_col} = {key_id}"""
self.cursor.execute(sql_update)
My Key column always contains Email addresses or integers. So if key_id = test#email.com, the update is successful, but if key_id = t'est#email.com, it fails. How do I get around this?
You can fix this by using query parameters. Stop concatenating strings into your SQL query. Use placeholders and then pass the values in a separate list argument to execute().
sql_update = f"""Update {tbl} set {target_col} = %s where {key_col} = %s"""
self.cursor.execute(sql_update, (nVal, key_id,))
Query parameters only work where you would use a literal value in your query, like a quoted string literal or a numeric literal.
You can't use query parameters for identifiers like the table name or column names. But I hope your identifiers are less likely to contain ' characters!
Likewise you cannot use query parameters for expressions or SQL keywords or lists of values e.g. for an IN() predicate. One query parameter = one scalar value.
See also:
MySQL parameterized queries
https://dev.mysql.com/doc/connector-python/en/connector-python-example-cursor-transaction.html
Literally any other Python SQL tutorial.
Use execute function instead.
Not recommended solution: A workaround for single quote literal is to replace with an escape character; just before the query key_id.replace("'", "\'"). That you might have to do for each special character like %, , _, and [.
I'm trying to execute a raw sql query and safely pass an order by/asc/desc based on user input. This is the back end for a paginated datagrid. I cannot for the life of me figure out how to do this safely. Parameters get converted to strings so Oracle can't execute the query. I can't find any examples of this anywhere on the internet. What is the best way to safely accomplish this? (I am not using the ORM, must be raw sql).
My workaround is just setting ASC/DESC to a variable that I set. This works fine and is safe. However, how do I bind a column name to the ORDER BY? Is that even possible? I can just whitelist a bunch of columns and do something similar as I do with the ASC/DESC. I was just curious if there's a way to bind it. Thanks.
#default.route('/api/barcodes/<sort_by>/<sort_dir>', methods=['GET'])
#json_enc
def fetch_barcodes(sort_by, sort_dir):
#time.sleep(5)
# Can't use sort_dir as a parameter, so assign to variable to sanitize it
ord_dir = "DESC" if sort_dir.lower() == 'desc' else 'ASC'
records = []
stmt = text("SELECT bb_request_id,bb_barcode,bs_status, "
"TO_CHAR(bb_rec_cre_date, 'MM/DD/YYYY') AS bb_rec_cre_date "
"FROM bars_barcodes,bars_status "
"WHERE bs_status_id = bb_status_id "
"ORDER BY :ord_by :ord_dir ")
stmt = stmt.bindparams(ord_by=sort_by,ord_dir=ord_dir)
rs = db.session.execute(stmt)
records = [dict(zip(rs.keys(), row)) for row in rs]
DatabaseError: (cx_Oracle.DatabaseError) ORA-01036: illegal variable name/number
[SQL: "SELECT bb_request_id,bb_barcode,bs_status, TO_CHAR(bb_rec_cre_date, 'MM/DD/YYYY') AS bb_rec_cre_date FROM bars_barcodes,bars_status WHERE bs_status_id = bb_status_id ORDER BY :ord_by :ord_dir "] [parameters: {'ord_by': u'bb_rec_cre_date', 'ord_dir': 'ASC'}]
UPDATE Solution based on accepted answer:
def fetch_barcodes(sort_by, sort_dir, page, rows_per_page):
ord_dir_func = desc if sort_dir.lower() == 'desc' else asc
query_limit = int(rows_per_page)
query_offset = (int(page) - 1) * query_limit
stmt = select([column('bb_request_id'),
column('bb_barcode'),
column('bs_status'),
func.to_char(column('bb_rec_cre_date'), 'MM/DD/YYYY').label('bb_rec_cre_date')]).\
select_from(table('bars_barcode')).\
select_from(table('bars_status')).\
where(column('bs_status_id') == column('bb_status_id')).\
order_by(ord_dir_func(column(sort_by))).\
limit(query_limit).offset(query_offset)
result = db.session.execute(stmt)
records = [dict(row) for row in result]
response = json_return()
response.addRecords(records)
#response.setTotal(len(records))
response.setTotal(1001)
response.setSuccess(True)
response.addMessage("Records retrieved successfully. Limit: " + str(query_limit) + ", Offset: " + str(query_offset) + " SQL: " + str(stmt))
return response
You could use Core constructs such as table() and column() for this instead of raw SQL strings. That'd make your life easier in this regard:
from sqlalchemy import select, table, column, asc, desc
ord_dir = desc if sort_dir.lower() == 'desc' else asc
stmt = select([column('bb_request_id'),
column('bb_barcode'),
column('bs_status'),
func.to_char(column('bb_rec_cre_date'),
'MM/DD/YYYY').label('bb_rec_cre_date')]).\
select_from(table('bars_barcodes')).\
select_from(table('bars_status')).\
where(column('bs_status_id') == column('bb_status_id')).\
order_by(ord_dir(column(sort_by)))
table() and column() represent the syntactic part of a full blown Table object with Columns and can be used in this fashion for escaping purposes:
The text handled by column() is assumed to be handled like the name of a database column; if the string contains mixed case, special characters, or matches a known reserved word on the target backend, the column expression will render using the quoting behavior determined by the backend.
Still, whitelisting might not be a bad idea.
Note that you don't need to manually zip() the row proxies in order to produce dictionaries. They act as mappings as is, and if you need dict() for serialization reasons or such, just do dict(row).
When running the following code I would like to be able to store the returned values in a variable as a string. However when I run the code it will return the first name from the database in this format
[(u'Kiefer',)]
What would I need to change in order to just return a clean string? I was thinking I could use strip although I have never used it on a tuple (not even sure if you can) but I was hoping there is a more elegant way. Also I am using the sqlite3 module.
def search_db(self,ID,FName,LName):
#try:
FName +='%'
LName += '%'
self.cur.execute(("""SELECT FirstName FROM members WHERE FirstName LIKE '%s'\
AND LastName LIKE '%s'""") % (FName, LName))
value = (self.cur.fetchall())
print(value)
#except:
#print('failed')
cla = Database()
cla.search_db(1,'K','D')
You need to access the first element of the list, which is the tuple, then the first element of the tuple, so:
value[0][0]
Let's say we have a SQL statement that just needs to be completed with the parameters before getting executed against the DB. For instance:
sql = '''
SELECT id, price, date_out
FROM sold_items
WHERE date_out BETWEEN ? AND ?
'''
database_cursor.execute(sql, (start_date, end_date))
How do I get the string that is parsed and executed?, something like this:
SELECT id, price, date_out
FROM sold_items
WHERE date_out BETWEEN 2010-12-05 AND 2011-12-01
In this simple case it's not very important, but I have other SQL Statements much more complicated, and for debugging purposes I would like to execute them myself in my sqlite manager and check the results.
Thanks in advance
UPDATE. I learned from this web page that since Python 3.3 you can trigger printing of executed SQL with
connection.set_trace_callback(print)
Should you want to revert to silent processing, use
connection.set_trace_callback(None)
You can use another function instead of print.
SQLite never actually substitutes parameters into the SQL query string itself; the parameters' values are read directly when it executes the command.
(Formatting those values only to parse them again into the same values would be useless overhead.)
But if you want to find out how the parameters would be written in SQL, you can use the quote function; something like this:
import re
def log_and_execute(cursor, sql, *args):
s = sql
if len(args) > 0:
# generates SELECT quote(?), quote(?), ...
cursor.execute("SELECT " + ", ".join(["quote(?)" for i in args]), args)
quoted_values = cursor.fetchone()
for quoted_value in quoted_values:
s = s.replace('?', quoted_value, 1)
#s = re.sub(r'(values \(|, | = )\?', r'\g<1>' + quoted_value, s, 1)
print "SQL command: " + s
cursor.execute(sql, args)
(This code will fail if there is a ? that is not a parameter, i.e., inside a literal string. Unless you use the re.sub version, which will only match a ? after 'values (' ', ' or ' = '. The '\g<1>' puts back the text before the ? and using '\g<>' avoids clashes with quoted_values that start with a number.)
I've written a function that just fills in the question marks with the arguments.
It's weird that everyone sends you in the direction of using positional arguments, but no one thought about the need to log or preview or check queries in their totality.
Anyways, the code below assumes
That there are no '?' tokens in the query that do not serve as postional argument tokens. I'm not sure whether that is always the case.
That the value of the argument will be wrapped in quotes. This will not be the case when you use an argument for a table name for example. This usecase is unlikely though.
def compile_query(query, *args):
# test for mismatch in number of '?' tokens and given arguments
number_of_question_marks = query.count('?')
number_of_arguments = len(args)
# When no args are given, an empty tuple is passed
if len(args) == 1 and (not args[0]):
number_of_arguments = 0
if number_of_arguments != number_of_question_marks:
return f"Incorrect number of bindings supplied. The current statement uses {number_of_question_marks}, and there are {number_of_arguments} supplied."
# compile query
for a in args:
query = query.replace('?', "'"+str(a)+"'", 1)
return query
Suggested usage
query = "INSERT INTO users (name, password) VALUES (?, ?)"
# sensitive query, we need to log this for security
query_string = compile_query(query, username, password_hash)
fancy_log_function(query_string)
# execute
cursor.execute(query, username, password_hash)
What about string formatting it?
sql = """
SELECT id, price, date_out
FROM sold_items
WHERE date_out BETWEEN {0} AND {1} """.format(start_date, end_date)
"""
database_cursor.execute(sql)
I have a large SQLite database with a mix of text and lots of other columns var1 ... var 50. Most of these are numeric, though some are text based.
I am trying to extract data from the database, process it in python and write it back - I need to do this for all rows in the db.
So far, the below sort of works:
# get row using select and process
fields = (','.join(keys)) # "var1, var2, var3 ... var50"
results = ','.join([results[key] for key in keys]) # "value for var1, ... value for var50"
cur.execute('INSERT OR REPLACE INTO results (id, %s) VALUES (%s, %s);' %(fields, id, results))
This however, nulls the columns that I don't explicitly add back. I can fix this by re-writing the code, but this feels quite messy, as I would have to surround with quotes using string concatenation and rewrite data that was there to begin with (i.e. the columns I didn't change).
Apparently the way to run updates on rows is something like this:
update table set var1 = 4, var2 = 5, var3="some text" where id = 666;
Presumably the way for me would be to run map , and add the = signs somehow (not sure how), but how would I quote all of the results appropriately (Since I would have to quote the text fields, and they might contain quotes within them too .. )?
I'm a bit confused. Any pointers would be very helpful.
Thanks!
As others have stressed, use parametrized arguments. Here is an example of how you might construct the SQL statement when it has a variable number of keys:
sql=('UPDATE results SET '
+ ', '.join(key+' = ?' for key in keys)
+ 'WHERE id = ?')
args = [results[key] for key in keys] + [id]
cur.execute(sql,args)
Use parameter substitution. It's more robust (and safer I think) than string formatting.
So if you did something like
query = 'UPDATE TABLE SET ' + ', '.join(str(f) + '=?,' for f in fields) + ';'
Or alternatively
query = 'UPDATE TABLE SET %s;' % (', '.join(str(f) + '=?,' for f in fields))
Or using new style formatting:
query = 'UPDATE TABLE SET {0};'.format(', '.join(str(f) + '=?,' for f in fields))
So the complete program would look something like this:
vals = {'var1': 'foo', 'var2': 3, 'var24':999}
fields = vals.keys()
results = vals.values()
query = 'UPDATE TABLE SET {0};'.format(', '.join(str(f) + '=?,' for f in fields))
conn.execute(query, results)
And that should work - and I presume do what you want it to.
You don't have to care about things like quotations etc, and in fact you shouldn't. If you do it like this, it's not only more convenient but also takes care of security issues known as sql injections:
sql = "update table set var1=%s, var2=%s, var3=%s where id=666"
cursor.execute(sql, (4, 5, "some text"))
the key point here ist that the sql and the values in the second statement aren't separated by a "%", but by a "," - this is not a string manipulation, but instead you pass two arguments to the execute function, the actual sql and the values. Each %s is replaced by a value from the value tuple. the database driver then knows how to take care of the individual types of the values.
the insert statement can be rewritten the same way, although I'm not sure and currently can't test whether you can also replace field names that way (the first %s in your insert-sql statement)
so to come back to your overall problem, you can loop over your values and dynamically add ", var%d=%%s" % i for your i-th variable while adding the actual value to a list at the same time