I am working on sql queries in python using PyMySQL. Lets say we have the following function
def f(bid)
con=connection()
cursor=con.cursor
sql = "select b.text from book b where b.id = 'bid'"
cursor.execute(sql)
book_text = cursor.fetchone()
print (book_text)
When I do:
f('123abc')
It prints:
()
But If I replace the above sql query with:
"select b.text from book b where b.id = '123abc'"
It prints the right thing.
Any ideas? Thanks in advance!
You need to 'bind' the parameter to the query:
sql = "select b.text from book b where b.id = %s"
cursor.execute(sql, (bid, ))
Note: The benefit of using placeholders (%s) in the query and passing an additional object to execute() usually handles all different variable types (int, str, dates, ...) for you.
Have a look at the docs which objects you can pass to execute().
you '123abc' argument isn't modifying the string you are using for your sql query.
change to:
sql = "select b.text from book b where b.id = '{}'".format(bid)
Related
I have a string (sql query) in which I want to pass a variable at one point, then pass another variable at another point (list of variables, but just focusing on one for now).
The expected would be something like this:
sql_query = 'SELECT {{field}} FROM {table} WHERE {{field}} IS NULL'.format(table=table_name)
sql_query should now be: 'SELECT {field} FROM table_name WHERE {field} IS NULL'
Then format for field
sql_query = sql_query.format(field_name)
In theory I want sql_query to now be: 'SELECT field_name FROM table_name WHERE field_name IS NULL'
But the above ignores the .format and I get: 'SELECT {field} FROM table_name WHERE {field} IS NULL'
I have tried combining f-strings and .format in multiple ways and the closest to my goal is:
field = field_name
sql_query = f'SELECT {field} FROM {{0}} WHERE {field} IS NULL'.format(table_name)
# Works and I get sql_query : 'SELECT field_name FROM table_name WHERE field_name IS NULL'
The above works but it happens all in the same place and separating where each one happens is the true goal of mine.
sql = "SELECT {{column}} FROM {table}"
sql = sql.format(table="my_table")
print(sql)
sql = sql.format(column="my_column")
print(sql)
Or...
sql = "SELECT {column} FROM {table}"
sql = sql.format(table="my_table", column="{column}")
print(sql)
sql = sql.format(column="my_column")
print(sql)
That said, I'd recommend not actually passing the string around and doing partial replacements on it.
Instead, pass a dictionary around, add the replacements to the dictionary, and call format just once...
Then you don't need to add arbitrary {} around the token you don't want to replace, or be aware that it exists in order to replace it with itself.
sql = "SELECT {column} FROM {table}"
parts = dict()
parts["table"] = "my_table"
# more code here
parts["column"] = "my_column"
print(sql.format(**parts))
NOTE:
As per other warnings here...
NEVER do this with user supplied text.
Literal values should be supplied using parameterisation / prepared statements.
Only ever do this when you are in complete control of the potential values in the dictionary, such as deriving the columns, etc, from ORM meta-data, or some other white list.
My SQL query which runs perfectly in terminal looks like this:
select t.txid, t.from_address, t.to_address, t.value, t.timestamp,
t.conformations, t.spent_flag,t.spent_txid from transaction_details t
where t.to_address =(select distinct a.address from address_master a
inner join panel_user p on a.user = p.user and a.user= "auxesis");
Now I tried using it in Django like this:
sql = """ select t.txid, t.from_address, t.to_address,t.value, t.timestamp, t.conformations, t.spent_flag,t.spent_txid from
transaction_details t where t.to_address =(select distinct a.address from
address_master a inner join panel_user p on a.user = p.user and a.user= "%s" """),%(user)
cursor.execute(sql)
res = cursor.fetchall()
But ya its not working. So any one please help me with it?
You're trying to use string formatting to build an SQL query. Don't do that, use parameterized queries. If you do that, you don't add quotes around the placeholders, the database connector will handle escaping of the parameters for you. Just pass the arguments as a tuple:
sql = """ select t.txid, t.from_address, t.to_address,t.value, t.timestamp, t.conformations, t.spent_flag,t.spent_txid from
transaction_details t where t.to_address =(select distinct a.address from
address_master a inner join panel_user p on a.user = p.user and a.user = %s """)
cursor.execute(sql, (user,))
res = cursor.fetchall()
I want to fetch all rows from MySQL table with
query = "SELECT * FROM %s WHERE last_name=%s"
cursor.execute(query, ("employees","Smith"))
but I'm getting
You have an error in your SQL syntax. When I try
query = "SELECT * FROM employees WHERE last_name=%s"
cursor.execute(query, ("Smith",))
all is fine.
Documentation says
cursor.execute(operation, params=None, multi=False)
The parameters found in the tuple or dictionary params are bound to the variables in the operation.link on docs
The first will generate an SQL like this:
SELECT * FROM 'employees' WHERE last_name='smith'
The parameters are SQL quoted.
If you really need to have a table name as param, you must proceed in 2 steps:
table_name = 'employees'
query_tpl = "SELECT * FROM {} WHERE last_name=%s"
query = query_tpl.format(table_name)
cursor.execute(query, ("Smith",))
you need to add the quote symbol. So the query will be like
SELECT * FROM employees WHERE last_name='Smith'
Change both your query to
query = "SELECT * FROM %s WHERE last_name='%s'"
query = "SELECT * FROM employees WHERE last_name='%s'"
You can't use a parameter for the table name in the execute call.
But you can use Python string interpolation for that:
query = "SELECT * FROM %s WHERE last_name=%s" %("employees","Smith")
cursor.execute(query)
You can't use a table name as a parameter. you are generating invalid sql with your code that is putting quotes around each string. the table name cannot have quotes around it.
sql you are generating
select * from 'employees' where last_name = 'Smith'
What sql you want
select * from employees where last_name = 'Smith'
you would have to format the string first like the example below.
query = "SELECT * from {} wherre last_name ='{}'"
cursor.execute(query.format("employees","Smith"))
using code like this does open up the possibility of SQL injection. so please bear that in mind.
query="SELECT * FROM %s WHERE name=%s",(employees,smith)
cursor.execute(query)
rows = cursor.fetchall()
Try this one. Hopefully it works for you.
I am a Python newbie, I am having troubles in the use of bind variables. If I execute the code below everything works fine.
bind= {"var" : "ciao"}
sql = "select * from sometable where somefield = :bind"
cur.prepare(sql)
cur.execute(sql,bind)
Instead if I add another bind variable I obtain an error.
bind= {"var" : "ciao"}
sql = "select * from sometable where somefield = :bind and otherfield = :bind"
cur.prepare(sql)
cur.execute(sql,(bind,bind))
cur.execute(sql,(bind,bind))
Oracle.NotSupportedError: Variable_TypeByValue(): unhandled data
I have solved it with
cur.execute(sql,(bind["var"],bind["var"]))
but I can't understand why the previous command was not ok.
Which is the correct way to use bind variables? I am using cx_Oracle.
You are misusing the binding.
There are three different ways of binding variables with cx_Oracle as one can see here :
1) by passing a tuple to a SQL statement with numbered variables :
sql = "select * from sometable where somefield = :1 and otherfield = :2"
cur.execute(sql, (aValue, anotherValue))
2) By passing keyword arguments to a SQL statement with named variables :
sql = "select * from sometable where somefield = :myField and otherfield = :anotherOne"
cur.execute(sql, myField=aValue, anotherOne=anotherValue)
3) By passing a dictionary to a SQL statement with named variables :
sql = "select * from sometable where somefield = :myField and otherfield = :anotherOne"
cur.execute(sql, {"myField":aValue, "anotherOne":anotherValue})
Remarks
Why does your code work then ?
Let's try to understand what happens here :
bind= {"var" : "ciao"}
sql = "select * from sometable where somefield = :bind and otherfield = :bind"
cur.execute(sql,(bind["var"], bind["var"]))
Oracle will understand that it expects one variable. This is a named variable, linked by name bind. You should then give a parameter as a named parameter like this :
cur.execute(sql, bind="ciao")
Or using a dictionary, like that :
cur.execute(sql, {bind:"ciao"})
However, as cx_Oracle receives a tuple instead, it fallbacks in a binding by number, as if your SQL statement was :
sql = "select * from sometable where somefield = :1 and otherfield = :2"
And as you are passing bind['var'] twice, which is just the string "ciao". It is mapping the two tuple items to the numbered variables :
cur.execute(sql, ("ciao", "ciao"))
That runs by chance but the code is very misleading.
Tuple with a single value to bind
Also note that the first option requires a tuple. But if you have a single value to bind, you can use this notation to create a tuple of a single value :
sql = "select * from sometable where somefield = :1"
cur.execute(sql, (aValue,))
[EDIT] : Thanks to #tyler-christian for mentioning that passing a dict was supported by cx_Oracle.
#ffarquest says that using a dictionary isn't supported by cx_Oracle but it is in fact, #giovanni-de-ciantis was just using it incorrectly.
named_params = {'dept_id':50, 'sal':1000}
query1 = cursor.execute(
'SELECT * FROM employees WHERE department_id=:dept_id AND salary>:sal',
named_params
)
OR
query2 = cursor.execute(
'SELECT * FROM employees WHERE department_id=:dept_id AND salary>:sal',
dept_id=50,
sal=1000
)
In the given example, I believe the second reference to :bind would need to be replaced with something different because it isn't being done in sequential order. Also, renamed the variable bind to get rid of confusion.
bind_dict = {bind:"var" : diff:"ciao"}
sql = "select * from sometable where somefield=:bind and otherfield=:diff"
cur.prepare(sql)
cur.execute(sql, bind_dict )
This article is from 2007 showing that you can use a dictionary:
http://www.oracle.com/technetwork/articles/dsl/prez-python-queries-101587.html
With Python's DB API spec you can pass an argument of parameters to the execute() method. Part of my statement is a WHERE IN clause and I've been using a tuple to populate the IN. For example:
params = ((3, 2, 1), )
stmt = "SELECT * FROM table WHERE id IN %s"
db.execute(stmt, params)
But when I run into a situation where the parameter tuple is only a tuple of 1 item, the execute fails.
ProgrammingError: ERROR: syntax error at or near ")"
LINE 13: WHERE id IN (3,)
How can I get the tuple to work with clause properly?
Edit: If you think this answer circumvents the built-in protections against SQL-injection attack you're mistaken; look more closely.
Testing with pg8000 (a DB-API 2.0 compatible Pure-Python interface to the PostgreSQL database engine):
This is the recommended way to pass multiple parameters to an "IN" clause.
params = [3,2,1]
stmt = 'SELECT * FROM table WHERE id IN (%s)' % ','.join('%s' for i in params)
cursor.execute(stmt, params)
Full example:
>>> from pg8000 import DBAPI
>>> conn = DBAPI.connect(user="a", database="d", host="localhost", password="p")
>>> c = conn.cursor()
>>> prms = [1,2,3]
>>> stmt = 'SELECT * FROM table WHERE id IN (%s)' % ','.join('%s' for i in prms)
>>> c.execute(stmt,prms)
>>> c.fetchall()
((1, u'myitem1'), (2, u'myitem2'), (3, u'myitem3'))
The error is coming from the comma after the 3. Just leave it off for the single values and you're set.
params = ((3), ... )
stmt = "SELECT * FROM table WHERE id IN %s"
db.execute(stmt, params)
This may not be an answer to exactly the question you asked, but I think it may solve the problem you have.
Python's DB-API doesn't seem to give you a way to pass tuples as safely substituted parameters. The accepted answer from bernie is using the Python % operator for substitution, which is unsafe.
However, you may not have to pass tuples as parameters, particularly when the tuple you want is the result of another SQL query (as you indicated to Daniel). Instead, you can use SQL subqueries.
If the set of IDs you want in your IN clause is the result of SELECT id FROM other_table WHERE use=true, for example:
stmt = "SELECT * FROM table WHERE id IN (SELECT id FROM other_table WHERE use=true)"
db.execute(stmt)
And this can be parameterized (the safe way), too. If the IDs you want to select are the ones with a given parent_id:
stmt = "SELECT * FROM table WHERE id IN (SELECT id FROM other_table WHERE parent_id=%s)"
params = (parent_id,)
db.execute(stmt, params)
A solution with f-string.
params = [...]
stmt = f"SELECT * FROM table WHERE id IN ({','.join(['%s']*len(params ),)})"
db.execute(stmt, params)
If there is another param placeholder it will be like this
age = 18
params = [...]
stmt = f"SELECT * FROM table WHERE age>%s AND id IN ({','.join(['%s']*len(params ),)})"
db.execute(stmt, tuple([age] + params))
As the question said, the following will fail:
params = ((3, 2, 1), )
stmt = "SELECT * FROM table WHERE id IN %s"
db.execute(stmt, params)
Following the pg8000 docs the IN can be replaced with an ANY() to give the same result:
params = ((3, 2, 1), )
stmt = "SELECT * FROM table WHERE id = ANY(%s)"
db.execute(stmt, params)
This sends the query and parameters separately to the server, avoiding SQL injection attacks.
The accepted answer risks SQL injection; you should never ever pass user input directly to the database. Instead, generate a query with the correct number of placeholders, then let pg8000 do the escaping:
params = [3,2,1]
# SELECT * from table where id in (%s,%s,%s)
stmt = 'SELECT * FROM table WHERE id IN ({})'.format(','.join(['%s']*len(params)))
cursor.execute(stmt, tuple(params))