I am working on Databricks and I am trying to pass a python variable into a SQL query:
series_name "LogTest"
query = """SELECT * FROM results_table
WHERE name = $series_name
"""
spark.sql(query).toPandas()
I tried with $ but it does not work.
How do I do this?
Regards
In this case, your variable and queries are just strings. You can do:
query = f"""SELECT * FROM results_table
WHERE name = '{series_name}'
"""
... in python 3. Remember that your query string needs the single quotes around the inserted variable. However, for certain variables, you may need to pass the variable directly to the spark module so it can interpret it. I use pyodbc a lot and would do this:
query = f"""SELECT * FROM results_table
WHERE name = ?"""
cursor.execute(query, series_name)
Following will also work,
series_name = "LogTest"
spark.sql("SELECT * FROM results_table WHERE name = " + series_name).toPandas()
we can also try the following.
series_name = "LogTest"
query = "SELECT * FROM results_table WHERE name = {}".format(series_name)
spark.sql(query).toPandas() #
Related
I have sql file which has table names as formatted string
query.sql
SELECT * FROM {table_name} WHERE LOAD_DT = '{load_date}'
How to read the sql file as f-string to pass it to pd.read_sql() method?
table_name = PRODUCTS
load_date = '15-08-2020'
# n other local variables
with open('query.sql','r') as file:
sql_str = file.read()
Note: I do not prefer .format(table_name,load_date) or .format(**locals()) as I have a custom function to read various sql files and don't want to send a param list of format variables every time, the reason being if the format list is huge it will be laborious while preparing the sql file with positional arguments and chances of mistakes are high
You can use .format method of string:
sql_str = "SELECT * FROM {table_name} WHERE LOAD_DT = '{load_date}'"
sql_str.format(table_name="PRODUCTS", load_date="15-08-2020")
You can also pass all local variables into the .format method:
table_name = "PRODUCTS"
load_date = "15-08-2020"
sql_str.format(**locals())
It is also possible to achieve desired result using eval, which is quite dangerous:
table_name = "PRODUCTS"
load_date = "15-08-2020"
sql_str = "SELECT * FROM {table_name} WHERE LOAD_DT = '{load_date}'"
sql_str_f = f"f\"{sql_str}\""
result = eval(sql_str_f)
Im trying to select from MS Access Database table in python WHERE Itemname is equal to some particular string which is stored in variable , while comparing that string directly in where clause with LIKE operator it is working fine but when i`m trying to pass this string through variable it is showing me syntex error.
query = "SELECT * FROM table1 WHERE table1.Itemname LIKE 'XYZ' "
cursor2.execute(query)
This is working fine
Itemname = "XYZ"
query = "SELECT * FROM table1 WHERE table1.Itemname LIKE %s "
cursor2.execute(query,(Itemname,))
Itemname = "XYZ"
query = "SELECT * FROM table1 WHERE table1.Itemname LIKE {}".format(Itemname)
cursor2.execute(query)
None of the Two above options are working ,Pls point out if there is any syntex problem
This is working finally
Itemname = "XYZ"
query = "SELECT * FROM table1 WHERE table1.Itemname = ? "
cursor2.execute(query,[itemname])
can someone tell me how to make the table a variable?
tableX = "test"
sql = "SELECT * FROM tableX WHERE datum = %s and name = %s"
val = (datumX, nameX,)
I just can't get it right. I've tried {}and also s%.
Thanks
You can use format function or f-string(if python 3.6=<)
tableX = "test"
sql = "SELECT * FROM {} WHERE datum = %s and name = %s".format(tableX)
In Python2.x works first answer:
sql = "SELECT * FROM {} WHERE datum = %s and name = %s" .format(tableX)
In Python3 you can use second one with f-string:
sql = f"SELECT * FROM {tableX} WHERE datum = %s and name = %s"
But, beware of SQL injection. You must be absolutely sure about source of that tableX variable (no user input). By direct format of the string you by-pass all escaping of input.
I want to fetch all rows from MySQL table with
query = "SELECT * FROM %s WHERE last_name=%s"
cursor.execute(query, ("employees","Smith"))
but I'm getting
You have an error in your SQL syntax. When I try
query = "SELECT * FROM employees WHERE last_name=%s"
cursor.execute(query, ("Smith",))
all is fine.
Documentation says
cursor.execute(operation, params=None, multi=False)
The parameters found in the tuple or dictionary params are bound to the variables in the operation.link on docs
The first will generate an SQL like this:
SELECT * FROM 'employees' WHERE last_name='smith'
The parameters are SQL quoted.
If you really need to have a table name as param, you must proceed in 2 steps:
table_name = 'employees'
query_tpl = "SELECT * FROM {} WHERE last_name=%s"
query = query_tpl.format(table_name)
cursor.execute(query, ("Smith",))
you need to add the quote symbol. So the query will be like
SELECT * FROM employees WHERE last_name='Smith'
Change both your query to
query = "SELECT * FROM %s WHERE last_name='%s'"
query = "SELECT * FROM employees WHERE last_name='%s'"
You can't use a parameter for the table name in the execute call.
But you can use Python string interpolation for that:
query = "SELECT * FROM %s WHERE last_name=%s" %("employees","Smith")
cursor.execute(query)
You can't use a table name as a parameter. you are generating invalid sql with your code that is putting quotes around each string. the table name cannot have quotes around it.
sql you are generating
select * from 'employees' where last_name = 'Smith'
What sql you want
select * from employees where last_name = 'Smith'
you would have to format the string first like the example below.
query = "SELECT * from {} wherre last_name ='{}'"
cursor.execute(query.format("employees","Smith"))
using code like this does open up the possibility of SQL injection. so please bear that in mind.
query="SELECT * FROM %s WHERE name=%s",(employees,smith)
cursor.execute(query)
rows = cursor.fetchall()
Try this one. Hopefully it works for you.
I am a Python newbie, I am having troubles in the use of bind variables. If I execute the code below everything works fine.
bind= {"var" : "ciao"}
sql = "select * from sometable where somefield = :bind"
cur.prepare(sql)
cur.execute(sql,bind)
Instead if I add another bind variable I obtain an error.
bind= {"var" : "ciao"}
sql = "select * from sometable where somefield = :bind and otherfield = :bind"
cur.prepare(sql)
cur.execute(sql,(bind,bind))
cur.execute(sql,(bind,bind))
Oracle.NotSupportedError: Variable_TypeByValue(): unhandled data
I have solved it with
cur.execute(sql,(bind["var"],bind["var"]))
but I can't understand why the previous command was not ok.
Which is the correct way to use bind variables? I am using cx_Oracle.
You are misusing the binding.
There are three different ways of binding variables with cx_Oracle as one can see here :
1) by passing a tuple to a SQL statement with numbered variables :
sql = "select * from sometable where somefield = :1 and otherfield = :2"
cur.execute(sql, (aValue, anotherValue))
2) By passing keyword arguments to a SQL statement with named variables :
sql = "select * from sometable where somefield = :myField and otherfield = :anotherOne"
cur.execute(sql, myField=aValue, anotherOne=anotherValue)
3) By passing a dictionary to a SQL statement with named variables :
sql = "select * from sometable where somefield = :myField and otherfield = :anotherOne"
cur.execute(sql, {"myField":aValue, "anotherOne":anotherValue})
Remarks
Why does your code work then ?
Let's try to understand what happens here :
bind= {"var" : "ciao"}
sql = "select * from sometable where somefield = :bind and otherfield = :bind"
cur.execute(sql,(bind["var"], bind["var"]))
Oracle will understand that it expects one variable. This is a named variable, linked by name bind. You should then give a parameter as a named parameter like this :
cur.execute(sql, bind="ciao")
Or using a dictionary, like that :
cur.execute(sql, {bind:"ciao"})
However, as cx_Oracle receives a tuple instead, it fallbacks in a binding by number, as if your SQL statement was :
sql = "select * from sometable where somefield = :1 and otherfield = :2"
And as you are passing bind['var'] twice, which is just the string "ciao". It is mapping the two tuple items to the numbered variables :
cur.execute(sql, ("ciao", "ciao"))
That runs by chance but the code is very misleading.
Tuple with a single value to bind
Also note that the first option requires a tuple. But if you have a single value to bind, you can use this notation to create a tuple of a single value :
sql = "select * from sometable where somefield = :1"
cur.execute(sql, (aValue,))
[EDIT] : Thanks to #tyler-christian for mentioning that passing a dict was supported by cx_Oracle.
#ffarquest says that using a dictionary isn't supported by cx_Oracle but it is in fact, #giovanni-de-ciantis was just using it incorrectly.
named_params = {'dept_id':50, 'sal':1000}
query1 = cursor.execute(
'SELECT * FROM employees WHERE department_id=:dept_id AND salary>:sal',
named_params
)
OR
query2 = cursor.execute(
'SELECT * FROM employees WHERE department_id=:dept_id AND salary>:sal',
dept_id=50,
sal=1000
)
In the given example, I believe the second reference to :bind would need to be replaced with something different because it isn't being done in sequential order. Also, renamed the variable bind to get rid of confusion.
bind_dict = {bind:"var" : diff:"ciao"}
sql = "select * from sometable where somefield=:bind and otherfield=:diff"
cur.prepare(sql)
cur.execute(sql, bind_dict )
This article is from 2007 showing that you can use a dictionary:
http://www.oracle.com/technetwork/articles/dsl/prez-python-queries-101587.html