I am trying to pass three variables in a sql query. These are region, feature, newUser. I am using SQL driver SQL Server Native Client 11.0.
Here is my code that works.
query = "SELECT LicenseNo FROM License_Mgmt_Reporting.dbo.MATLAB_NNU_OPTIONS WHERE Region = ?"
data_df = pd.read_sql_query((query),engine,params={region})
output.
LicenseNo
0 12
1 5
Instead i want to pass in three variables and this code does not work.
query = "SELECT LicenseNo FROM License_Mgmt_Reporting.dbo.MATLAB_NNU_OPTIONS WHERE Region = ? and FeatureName = ? and NewUser =?"
nnu_data_df = pd.read_sql_query((query),engine,params={region, feature, newUser})
Output returns an empty data frame.
Empty DataFrame
Columns: [LicenseNo]
Index: []
try a string in a tuple, also you can take out the () in the query:
so you could do something like
query = "SELECT LicenseNo FROM License_Mgmt_Reporting.dbo.MATLAB_NNU_OPTIONS WHERE Region = ? and FeatureName = ? and NewUser =?"
region = 'US'
feature = 'tall'
newUser = 'john'
data_df = pd.read_sql_query(query, engine, params=(region, feature , newUser))
Operator error by me :( I was using the wrong variable and the database returned no results because it didn't exist!
Related
Unable to pass data from excel as variable in sql query in sql alchemy in Python.
Below is my code where I am accessing list of company ids from excel; But when I try to use that in the sql query in (in condition), it throws me error. Can you tell me how to pass variable in sql query in sql alchemy.
file_loc = path + file
company_id = pd.read_excel(file_loc,sheet_name='Sheet1',index_col=None,usecols="A",header=1)
qry = sqlalchemy.text("SELECT year,name,source,SUM(value) FROM table WHERE id in (:company_id) GROUP BY year,name,source")
qryres = pd.read_sql(qry,conn)
you can use this :
file_loc = path + file
company_id = pd.read_excel(file_loc,sheet_name='Sheet1',index_col=None,usecols="A",header=1)
qry = sqlalchemy.text(f"SELECT year,name,source,SUM(value) FROM table WHERE id in ({company_id}) GROUP BY year,name,source")
qryres = pd.read_sql(qry,conn)
Docs : https://docs.python.org/3.5/library/string.html#format-string-syntax
I'm getting the mentioned error while I'm using a WHERE condition in cx_oracle.
However, there is no error while I fetch all rows.
Please find my code below. The input - cols is a list of all the columns I want to fetch. Similarly, I want to pass the where condition as variable too, wherein I'm passing on the values in a loop.
For testing, lets keep it static.
Thanks
cols = [
'ID',
'CODE',
'LOGINDEX_CODE',
'IS_ACTIVE',
'IS_PORT_GROUP'
]
table_name = 'PORT'
os.chdir(os.path.dirname(__file__))
main_dir = os.getcwd()
import cx_Oracle
try:
cx_Oracle.init_oracle_client(lib_dir= main_dir + "\\instantclient_21_3\\")
except:
pass
dsn_tns = cx_Oracle.makedsn(r'some_db', 1521, service_name=r'some_service')
conn = cx_Oracle.connect(user='abcdef', password=r'ghijkl', dsn=dsn_tns).cursor()
query_cols = ', '.join(cols)
named_params = {
'varx' : query_cols,
'vary' : 'LEG',
'varz' : 242713
}
sql_query = 'SELECT :varx FROM :vary WHERE START_PORT_ID IN :varz'
conn.prepare(sql_query)
conn.execute(sql_query, named_params)
Bind variables cannot be used to replace parts of the SQL statement itself. They can only be used to supply data that is sent to the database. So you would need to do something like this instead:
sql_query = f"select {', '.join(cols)} from LEG where start_port_id = :varz"
with conn.cursor() as cursor:
for row in cursor.execute(sql_query, varz=242713):
print(row)
I am working on Databricks and I am trying to pass a python variable into a SQL query:
series_name "LogTest"
query = """SELECT * FROM results_table
WHERE name = $series_name
"""
spark.sql(query).toPandas()
I tried with $ but it does not work.
How do I do this?
Regards
In this case, your variable and queries are just strings. You can do:
query = f"""SELECT * FROM results_table
WHERE name = '{series_name}'
"""
... in python 3. Remember that your query string needs the single quotes around the inserted variable. However, for certain variables, you may need to pass the variable directly to the spark module so it can interpret it. I use pyodbc a lot and would do this:
query = f"""SELECT * FROM results_table
WHERE name = ?"""
cursor.execute(query, series_name)
Following will also work,
series_name = "LogTest"
spark.sql("SELECT * FROM results_table WHERE name = " + series_name).toPandas()
we can also try the following.
series_name = "LogTest"
query = "SELECT * FROM results_table WHERE name = {}".format(series_name)
spark.sql(query).toPandas() #
I have raw data as:
id = 2345
id_num = 3,6,343,32
I need to pass both the above as parameters in an ORACLE SQL query via a cx_Oracle connection as:
query = “””
select * from mytable where pid = 2345 and id_num in (3,6,343,32)
“””
I am creating a dictionary as:
sparm = {}
sparm['pid'] = id
sparm['idnum'] = id_num
and trying to use it as:
query = “””
select * from mytable where pid = :pid and id_num in :idnum
“””
df = pd.read_sql(query, con=conct, params=sparm)
without success.
The :pid works but the :idnum doesn’t. Any suggestions would be much appreciated.
I have raw data as:
id = 2345
id_num = 3,6,343,32
I need to pass both the above as parameters in an ORACLE SQL query via a cx_Oracle connection as:
query = “””
select * from mytable where pid = 2345 and id_num in (3,6,343,32)
“””
I am creating a dictionary as:
sparm = {}
sparm['pid'] = id
and create a tuple for the where clause as:
where=tuple(list(id_num.split(",")))
and trying to use it as:
query = “””
select * from mytable where pid = :pid and id_num in {}
“””.format(where)
df = pd.read_sql(query, con=conct, params=sparm)
with success.
The :pid works with a dict input and the :idnum works as a tuple input.
Today i am using the sql_debug(True) that helps me to see the queries but without the values.
How could i see how ponyorm translate the query with values ?
Thank you very much.
This is an example of query i'm using.
with db_session:
access = select(p for p in Access if raw_sql('( lower(first_name) = lower($first_name) and lower(last_name) = lower($last_name) ) '
'or ( lower(first_name) = lower($last_name) and lower(last_name) = lower($first_name) ) '
'or (lower(facebook_url) = lower($facebook_url)) '
'or (lower(twitter_url) = lower($twitter_url)) '
'or (lower(linkedin_url) = lower($linkedin_url)) '))
.order_by(desc(Access.twitter_url),desc(Access.facebook_url),desc(Access.linkedin_url),
desc(Access.facebook_url))
print(access.get_sql())
I use
logging.getLogger(__name__).debug('SQL:\n\n\t\t\t%s\n', '\n'.join(unicode(x) for x in request._construct_sql_and_arguments()[:2]).replace('\n', '\n\t\t\t'))
for that.
For example,
19:30:01.902 data.py:231 [DEBUG] SQL:
SELECT "x"."_id", "x"."filename", "x"."_created", "x"."_updated"
FROM "reports" "x"
WHERE "x"."_id" <= ?
AND "x"."_created" >= ?
(50, '2019-04-17 19:30:01.900028')
will be printed out.
You can use set_sql_debug(debug=True, show_values=True).
Reference here.
There is a method called get_sql()
query_obj = select(c for c in Category if c.name.startswith('v'))
sql = query_obj.get_sql()
print(sql)
output:
SELECT "c"."id", "c"."name"
FROM "category" "c"
WHERE "c"."name" LIKE 'v%%'
code continue:
for obj in query_obj:
print('id:', obj.id, 'name:', obj.name)
output:
id: 1 name: viki
here is a link to the docs https://docs.ponyorm.com/api_reference.html#Query.get_sql
You can log the sql or simply print it.
Update:
OP updated the question:
If the sql query has a variable like $name it is passed as a sql parameter.
first_name = 'viki'
query = select(c for c in Category if raw_sql('( lower(name) = lower($first_name))'))
query.get_sql()
so get_sql() will return the value with a placeholder, and the output will look like this:
'SELECT "c"."id", "c"."name", "c"."age"\nFROM "Category" "c"\nWHERE ( lower(name) = lower(?))'
If we want no placeholders should be there in the query then we can avoid passing direct sql to query and instead build it separately in python.
Like this:
query = select(c for C in Category if c.name == 'viki')
query.get_sql()
output:
'SELECT "c"."id", "c"."name", "c"."age"\nFROM "Category" "c"\nWHERE "c"."name" = \'viki\''