We Keep Coding

How to format a formatted string in Python? Either with .format or f-string

I have a string (sql query) in which I want to pass a variable at one point, then pass another variable at another point (list of variables, but just focusing on one for now).
The expected would be something like this:
sql_query = 'SELECT {{field}} FROM {table} WHERE {{field}} IS NULL'.format(table=table_name)
sql_query should now be: 'SELECT {field} FROM table_name WHERE {field} IS NULL'
Then format for field
sql_query = sql_query.format(field_name)
In theory I want sql_query to now be: 'SELECT field_name FROM table_name WHERE field_name IS NULL'
But the above ignores the .format and I get: 'SELECT {field} FROM table_name WHERE {field} IS NULL'
I have tried combining f-strings and .format in multiple ways and the closest to my goal is:
field = field_name
sql_query = f'SELECT {field} FROM {{0}} WHERE {field} IS NULL'.format(table_name)
# Works and I get sql_query : 'SELECT field_name FROM table_name WHERE field_name IS NULL'
The above works but it happens all in the same place and separating where each one happens is the true goal of mine.

sql = "SELECT {{column}} FROM {table}"
sql = sql.format(table="my_table")
print(sql)
sql = sql.format(column="my_column")
print(sql)
Or...
sql = "SELECT {column} FROM {table}"
sql = sql.format(table="my_table", column="{column}")
print(sql)
sql = sql.format(column="my_column")
print(sql)
That said, I'd recommend not actually passing the string around and doing partial replacements on it.
Instead, pass a dictionary around, add the replacements to the dictionary, and call format just once...
Then you don't need to add arbitrary {} around the token you don't want to replace, or be aware that it exists in order to replace it with itself.
sql = "SELECT {column} FROM {table}"
parts = dict()
parts["table"] = "my_table"
# more code here
parts["column"] = "my_column"
print(sql.format(**parts))
NOTE:
As per other warnings here...
NEVER do this with user supplied text.
Literal values should be supplied using parameterisation / prepared statements.
Only ever do this when you are in complete control of the potential values in the dictionary, such as deriving the columns, etc, from ORM meta-data, or some other white list.

compose mysql query in python

I want to fetch all rows from MySQL table with
query = "SELECT * FROM %s WHERE last_name=%s"
cursor.execute(query, ("employees","Smith"))
but I'm getting
You have an error in your SQL syntax. When I try
query = "SELECT * FROM employees WHERE last_name=%s"
cursor.execute(query, ("Smith",))
all is fine.
Documentation says
cursor.execute(operation, params=None, multi=False)
The parameters found in the tuple or dictionary params are bound to the variables in the operation.link on docs

The first will generate an SQL like this:
SELECT * FROM 'employees' WHERE last_name='smith'
The parameters are SQL quoted.
If you really need to have a table name as param, you must proceed in 2 steps:
table_name = 'employees'
query_tpl = "SELECT * FROM {} WHERE last_name=%s"
query = query_tpl.format(table_name)
cursor.execute(query, ("Smith",))

you need to add the quote symbol. So the query will be like
SELECT * FROM employees WHERE last_name='Smith'
Change both your query to
query = "SELECT * FROM %s WHERE last_name='%s'"
query = "SELECT * FROM employees WHERE last_name='%s'"

You can't use a parameter for the table name in the execute call.
But you can use Python string interpolation for that:
query = "SELECT * FROM %s WHERE last_name=%s" %("employees","Smith")
cursor.execute(query)

You can't use a table name as a parameter. you are generating invalid sql with your code that is putting quotes around each string. the table name cannot have quotes around it.
sql you are generating
select * from 'employees' where last_name = 'Smith'
What sql you want
select * from employees where last_name = 'Smith'
you would have to format the string first like the example below.
query = "SELECT * from {} wherre last_name ='{}'"
cursor.execute(query.format("employees","Smith"))
using code like this does open up the possibility of SQL injection. so please bear that in mind.

query="SELECT * FROM %s WHERE name=%s",(employees,smith)
cursor.execute(query)
rows = cursor.fetchall()
Try this one. Hopefully it works for you.

WHERE IN Clause in python list [duplicate]

This question already has answers here:
imploding a list for use in a python MySQLDB IN clause
(8 answers)
Closed 1 year ago.
I need to pass a batch of parameters to mysql in python. Here is my code:
sql = """ SELECT * from my_table WHERE name IN (%s) AND id=%(Id)s AND puid=%(Puid)s"""
params = {'Id':id,'Puid' : pid}
in_p=', '.join(list(map(lambda x: '%s', names)))
sql = sql %in_p
cursor.execute(sql, names) #todo: add params to sql clause
The problem is I want to pass the name list to sql IN clause, meanwhile I also want to pass the id and puid as parameters to the sql query clause. How do I implement these in python?

Think about the arguments to cursor.execute that you want. You want to ultimately execute
cursor.execute("SELECT * FROM my_table WHERE name IN (%s, %s, %s) AND id = %s AND puid = %s;", ["name1", "name2", "name3", id, pid])
How do you get there? The tricky part is getting the variable number of %ss right in the IN clause. The solution, as you probably saw from this answer is to dynamically build it and %-format it into the string.
in_p = ', '.join(list(map(lambda x: '%s', names)))
sql = "SELECT * FROM my_table WHERE name IN (%s) AND id = %s AND puid = %s;" % in_p
But this doesn't work. You get:
TypeError: not enough arguments for format string
It looks like Python is confused about the second two %ss, which you don't want to replace. The solution is to tell Python to treat those %ss differently by escaping the %:
sql = "SELECT * FROM my_table WHERE name IN (%s) AND id = %%s AND puid = %%s;" % in_p
Finally, to build the arguments and execute the query:
args = names + [id, pid]
cursor.execute(sql, args)

sql = """ SELECT * from my_table WHERE name IN (%s) AND id=%(Id)s AND puid=%(Puid)s""".replace("%s", "%(Clause)s")
print sql%{'Id':"x", 'Puid': "x", 'Clause': "x"}
This can help you.

Python cx_Oracle bind variables

I am a Python newbie, I am having troubles in the use of bind variables. If I execute the code below everything works fine.
bind= {"var" : "ciao"}
sql = "select * from sometable where somefield = :bind"
cur.prepare(sql)
cur.execute(sql,bind)
Instead if I add another bind variable I obtain an error.
bind= {"var" : "ciao"}
sql = "select * from sometable where somefield = :bind and otherfield = :bind"
cur.prepare(sql)
cur.execute(sql,(bind,bind))
cur.execute(sql,(bind,bind))
Oracle.NotSupportedError: Variable_TypeByValue(): unhandled data
I have solved it with
cur.execute(sql,(bind["var"],bind["var"]))
but I can't understand why the previous command was not ok.
Which is the correct way to use bind variables? I am using cx_Oracle.

You are misusing the binding.
There are three different ways of binding variables with cx_Oracle as one can see here :
1) by passing a tuple to a SQL statement with numbered variables :
sql = "select * from sometable where somefield = :1 and otherfield = :2"
cur.execute(sql, (aValue, anotherValue))
2) By passing keyword arguments to a SQL statement with named variables :
sql = "select * from sometable where somefield = :myField and otherfield = :anotherOne"
cur.execute(sql, myField=aValue, anotherOne=anotherValue)
3) By passing a dictionary to a SQL statement with named variables :
sql = "select * from sometable where somefield = :myField and otherfield = :anotherOne"
cur.execute(sql, {"myField":aValue, "anotherOne":anotherValue})
Remarks
Why does your code work then ?
Let's try to understand what happens here :
bind= {"var" : "ciao"}
sql = "select * from sometable where somefield = :bind and otherfield = :bind"
cur.execute(sql,(bind["var"], bind["var"]))
Oracle will understand that it expects one variable. This is a named variable, linked by name bind. You should then give a parameter as a named parameter like this :
cur.execute(sql, bind="ciao")
Or using a dictionary, like that :
cur.execute(sql, {bind:"ciao"})
However, as cx_Oracle receives a tuple instead, it fallbacks in a binding by number, as if your SQL statement was :
sql = "select * from sometable where somefield = :1 and otherfield = :2"
And as you are passing bind['var'] twice, which is just the string "ciao". It is mapping the two tuple items to the numbered variables :
cur.execute(sql, ("ciao", "ciao"))
That runs by chance but the code is very misleading.
Tuple with a single value to bind
Also note that the first option requires a tuple. But if you have a single value to bind, you can use this notation to create a tuple of a single value :
sql = "select * from sometable where somefield = :1"
cur.execute(sql, (aValue,))
[EDIT] : Thanks to #tyler-christian for mentioning that passing a dict was supported by cx_Oracle.

#ffarquest says that using a dictionary isn't supported by cx_Oracle but it is in fact, #giovanni-de-ciantis was just using it incorrectly.
named_params = {'dept_id':50, 'sal':1000}
query1 = cursor.execute(
'SELECT * FROM employees WHERE department_id=:dept_id AND salary>:sal',
named_params
)
OR
query2 = cursor.execute(
'SELECT * FROM employees WHERE department_id=:dept_id AND salary>:sal',
dept_id=50,
sal=1000
)
In the given example, I believe the second reference to :bind would need to be replaced with something different because it isn't being done in sequential order. Also, renamed the variable bind to get rid of confusion.
bind_dict = {bind:"var" : diff:"ciao"}
sql = "select * from sometable where somefield=:bind and otherfield=:diff"
cur.prepare(sql)
cur.execute(sql, bind_dict )
This article is from 2007 showing that you can use a dictionary:
http://www.oracle.com/technetwork/articles/dsl/prez-python-queries-101587.html

How to string format SQL IN clause with Python

I am trying to create a statement as follows:
SELECT * FROM table WHERE provider IN ('provider1', 'provider2', ...)
However, I'm having some trouble with the string formatting of it from the Django API. Here's what I have so far:
profile = request.user.get_profile()
providers = profile.provider.values_list('provider', flat=True) # [u'provider1', u'provider2']
providers = tuple[str(item) for item in providers] # ('provider1', 'provider2')
SQL = "SELECT * FROM table WHERE provider IN %s"
args = (providers,)
cursor.execute(sql,args)
DatabaseError
(1241, 'Operand should contain 1 column(s)')

MySQLdb has a method to help with this:
Doc
string_literal(...)
string_literal(obj) -- converts object obj into a SQL string literal.
This means, any special SQL characters are escaped, and it is enclosed
within single quotes. In other words, it performs:
"'%s'" % escape_string(str(obj))
Use connection.string_literal(obj), if you use it at all.
_mysql.string_literal(obj) cannot handle character sets.
Usage
# connection: <_mysql.connection open to 'localhost' at 1008b2420>
str_value = connection.string_literal(tuple(provider))
# '(\'provider1\', \'provider2\')'
SQL = "SELECT * FROM table WHERE provider IN %s"
args = (str_value,)
cursor.execute(sql,args)

Another answer that I don't like particularly, but will work for your apparent use-case:
providers = tuple[str(item) for item in providers] # ('provider1', 'provider2')
# rest of stuff...
SQL = 'SELECT * FROM table WHERE provider IN {}'.format(repr(providers))
cursor.execute(SQL)

You should probably do the string replacement before passing it to the cursor object to execute:
sql = "SELECT * FROM table WHERE provider IN (%s)" % \
(','.join(str(x) for x in providers))
cursor.execute(sql)

So, you have string input for ID's required:
some_vals = '1 3 5 76 5 4 2 5 7 8'.split() # convert to suitable type if required
SomeModel.objects.filter(provider__in=some_vals)

"SELECT * FROM table WHERE provider IN ({0},{1},{2})".format(*args) #where args is list or tuple of arguments.

try this.... should work.
SQL = "SELECT * FROM table WHERE provider IN %s"%(providers)
exec 'cursor.execute("%s")'%(SQL)