I am working in Django 1.8. I am currently running the following unsafe query using django.db.connection:
cursor = connection.cursor()
param = 'cost' # Actually obtained as a GET parameter
query = "SELECT date, pct_id, SUM(%s) as val " % param
query += "FROM mytable "
query += "GROUP BY pct_id, date ORDER BY date"
cursor.execute(query)
This works, but is obviously vulnerable to SQL injection. Instead I want to escape param before passing it into the query string.
So I'm trying this:
query = "SELECT date, pct_id, SUM(%s) as val "
query += "FROM mytable "
query += "GROUP BY pct_id, date ORDER BY date"
cursor.execute(query, param)
However, this gives me the following error:
TypeError: not all arguments converted during string formatting
How can I safely escape this parameter?
Related
I have a string (sql query) in which I want to pass a variable at one point, then pass another variable at another point (list of variables, but just focusing on one for now).
The expected would be something like this:
sql_query = 'SELECT {{field}} FROM {table} WHERE {{field}} IS NULL'.format(table=table_name)
sql_query should now be: 'SELECT {field} FROM table_name WHERE {field} IS NULL'
Then format for field
sql_query = sql_query.format(field_name)
In theory I want sql_query to now be: 'SELECT field_name FROM table_name WHERE field_name IS NULL'
But the above ignores the .format and I get: 'SELECT {field} FROM table_name WHERE {field} IS NULL'
I have tried combining f-strings and .format in multiple ways and the closest to my goal is:
field = field_name
sql_query = f'SELECT {field} FROM {{0}} WHERE {field} IS NULL'.format(table_name)
# Works and I get sql_query : 'SELECT field_name FROM table_name WHERE field_name IS NULL'
The above works but it happens all in the same place and separating where each one happens is the true goal of mine.
sql = "SELECT {{column}} FROM {table}"
sql = sql.format(table="my_table")
print(sql)
sql = sql.format(column="my_column")
print(sql)
Or...
sql = "SELECT {column} FROM {table}"
sql = sql.format(table="my_table", column="{column}")
print(sql)
sql = sql.format(column="my_column")
print(sql)
That said, I'd recommend not actually passing the string around and doing partial replacements on it.
Instead, pass a dictionary around, add the replacements to the dictionary, and call format just once...
Then you don't need to add arbitrary {} around the token you don't want to replace, or be aware that it exists in order to replace it with itself.
sql = "SELECT {column} FROM {table}"
parts = dict()
parts["table"] = "my_table"
# more code here
parts["column"] = "my_column"
print(sql.format(**parts))
NOTE:
As per other warnings here...
NEVER do this with user supplied text.
Literal values should be supplied using parameterisation / prepared statements.
Only ever do this when you are in complete control of the potential values in the dictionary, such as deriving the columns, etc, from ORM meta-data, or some other white list.
Im using python3 and postgres 11.5.
This is the script :
a = cursor.execute("SELECT tablename FROM pg_catalog.pg_tables limit 5")
for table in a:
cursor.execute("SELECT * FROM pg_prewarm(public.%s)", [table[0]])
a query gets some table names , and the loop query should run table name as the %s.
but for some reason i get the arg table[0] with // /n in the query and its messing it up.
if i print a results i get table names as tuple:
[('sa1591354519',), ('sa1591397719',), ('sa1591397719',)]
so [table[0]] is a string.
the error i get:
1574683839 [16177], ERR, execute ({'Error while connecting to PostgreSQL': SyntaxError('syntax error at or near "\'sa1591440919\'"\nLINE 1: SELECT * FROM pg_prewarm(public.\'sa1591440919\')\n ^\n')},)
what can i do ?
The errors don't have anything to do with the newlines you see, which are just an artifact of the error message. If you were to print out the error, would see:
syntax error at or near "'sa1591440919'"
LINE 1: SELECT * FROM pg_prewarm(public.'sa1591440919')
^
In other words, Postgres doesn't like the table name you're passing because it contains quotes. This is happening because you're trying to treat the table names like a normal query parameter, which causes psycopg to quote them...but that's not what you want in this case.
Just replace your use of query templating with normal Python string substitution:
a = cursor.execute("SELECT tablename FROM pg_catalog.pg_tables limit 5")
for table in a:
cursor.execute("SELECT * FROM pg_prewarm(public.%s)" % (table[0]))
But this won't actually work, because cursor.execute doesn't return a value, so a will be None. You would need to do something like:
cursor.execute("SELECT tablename FROM pg_catalog.pg_tables limit 5")
a = cursor.fetchall()
for table in a:
...
This question already has answers here:
imploding a list for use in a python MySQLDB IN clause
(8 answers)
Closed 1 year ago.
I need to pass a batch of parameters to mysql in python. Here is my code:
sql = """ SELECT * from my_table WHERE name IN (%s) AND id=%(Id)s AND puid=%(Puid)s"""
params = {'Id':id,'Puid' : pid}
in_p=', '.join(list(map(lambda x: '%s', names)))
sql = sql %in_p
cursor.execute(sql, names) #todo: add params to sql clause
The problem is I want to pass the name list to sql IN clause, meanwhile I also want to pass the id and puid as parameters to the sql query clause. How do I implement these in python?
Think about the arguments to cursor.execute that you want. You want to ultimately execute
cursor.execute("SELECT * FROM my_table WHERE name IN (%s, %s, %s) AND id = %s AND puid = %s;", ["name1", "name2", "name3", id, pid])
How do you get there? The tricky part is getting the variable number of %ss right in the IN clause. The solution, as you probably saw from this answer is to dynamically build it and %-format it into the string.
in_p = ', '.join(list(map(lambda x: '%s', names)))
sql = "SELECT * FROM my_table WHERE name IN (%s) AND id = %s AND puid = %s;" % in_p
But this doesn't work. You get:
TypeError: not enough arguments for format string
It looks like Python is confused about the second two %ss, which you don't want to replace. The solution is to tell Python to treat those %ss differently by escaping the %:
sql = "SELECT * FROM my_table WHERE name IN (%s) AND id = %%s AND puid = %%s;" % in_p
Finally, to build the arguments and execute the query:
args = names + [id, pid]
cursor.execute(sql, args)
sql = """ SELECT * from my_table WHERE name IN (%s) AND id=%(Id)s AND puid=%(Puid)s""".replace("%s", "%(Clause)s")
print sql%{'Id':"x", 'Puid': "x", 'Clause': "x"}
This can help you.
I'm trying to do a select
query = "select * from snmptt order by id desc limit %s", limit
cursorMYSQL.execute(query)
I get limit from a form
limit = form_limit.cleaned_data['limit']
I already used this syntax (, instead of %) for an insert and it worked, so I don't get why it's not working now.
Thanks!
The query and params should be separate arguments to the execute method.
query = "select * from snmptt order by id desc limit %s"
params = [limit]
cursorMYSQL.execute(query, params)
I'm using PyMYSQL to query data from a MySQL database. An example query that I want to use is:
SELECT count(*) AS count
FROM example
WHERE item = %s
LIMTI 1
which would be run using
query = "SELECT COUNT(*) as count FROM example WHERE item = %s LIMIT 1"
cur.execute(query,(None))
or
cur.execute(query,(10))
The issue is that item can have a value of NULL which results in
WHERE item = NULL
Which doesn't work in MySQL. Rather it should read
WHERE item IS NULL
However the value can be integer also. How can I make PyMySQL adjust the query to handle both situations?
you can make a simple function to catch the None situation:
def null_for_sql(a):
return "is NULL" if not a else "= " + str(a)
and then call the query like that:
query = "SELECT COUNT(*) as count FROM example WHERE item %s LIMIT 1"
cur.execute(query,null_for_sql(None))
Note there is no "=" after "item" in the query
Edit after comments:
def sql_none_safe_generator(query,parameter):
if parameter:
return query.replace(%param, "= " + str(parameter))
else:
return query.replace(%param, is NULL)
query = "SELECT COUNT(*) as count FROM example WHERE item %param LIMIT 1"
cur.execute(sql_none_safe_generator(query,a))