I want to use data binding when executing sql.
I just want to binding in the middle of the string, but it doesn't work.
I tried the following, but all of them resulted in execution errors.
Python
param = {
"env": "dev"
"s3_credential": "secret"
}
cursor().execute(sql, param)
sql1
CREATE OR REPLACE STAGE my_s3_stage_demo
URL='s3://my-stage-demo-'%(env)s'/tmp/'
credentials = (aws_role = %(s3_credential)s )
FILE_FORMAT = ( TYPE=JSON);
error message1
snowflake.connector.errors.ProgrammingError: 091006 (22000): Bucket name 'my-stage-demo-'dev'' in the stage location is not supported. Valid bucket names must consist of lowercase letters, digits, hyphens '-', and periods '.'.
SQL2
CREATE OR REPLACE STAGE my_s3_stage_demo
URL='s3://my-stage-demo-%(env)s/tmp/'
credentials = (aws_role = %(s3_credential)s )
FILE_FORMAT = ( TYPE=JSON);
error message2
snowflake.connector.errors.ProgrammingError: 001003 (42000): SQL compilation error:
syntax error line 2 at position 32 unexpected ''/tmp/''.
I want to execute the binding result as follows, but how should I specify it?
CREATE OR REPLACE STAGE my_s3_stage_demo
URL='s3://my-stage-demo-dev/tmp/'
credentials = (aws_role = "secret" )
FILE_FORMAT = ( TYPE=JSON);
You cannot bind substrings, only complete syntactical elements.
In Python, you can do something like:
cursor.execute(
"SELECT t.*, 'P'||:2 p2 FROM IDENTIFIER(:1) t",
[['"my_db"."my_schema"."my_table"', '2. parameter']]
)
You can only use parts of a value where expressions are allowed (like 'P'||:2 above). There is also some provision for identifiers like the table name above using IDENTIFIER().
Unfortunately, for the CREATE OR REPLACE STAGE command there seems to be no support for using expressions for eg an s3 bucket, or bind variables at all.
Which means you have to use replacement for the SQL text, not variable binding.
Related
For purposes of this question, let's say there is a table schema foo.bar.baz
And we have created a cursor object using following boilerplate
import snowflake.connector
ctx = snowflake.connector.connect(...)
cur = ctx.cursor()
With that cursor object, we can put the whole dot deliminated schema into a query like so:
cur.execute('''
select * from foo.bar.baz
'''
)
and have no issues, but we wouldn't be able to do:
cur.execute('''
select * from %(tbl)s
''', {'tbl': 'foo.bar.baz'}
)
Doing that throws this type of error: ProgrammingError: 001011 (42601): SQL compilation error: invalid URL prefix found in: foo.bar.baz
I'm guessing this is because the dots are sql identifiers and not strings, but I don't see any workaround in the snowflake documentation. Does anyone know how this could be done without having to change the connection object.
Using TABLE:
In a FROM clause, the syntax TABLE( { string_literal | session_variable | bind_variable } ) can be used
select * from TABLE(%(tbl)s)
I have a script, in which part of it calls to Snowflake on a query. That query is located in an .sql folder and two params are passed through to it in the script. For example:
#query.sql
select *
from {some_table}
where
date == '{hit_date}'
and
id in ({id_list}) or ({id_list}) is null)
And the piece of script acts like:
def run_query(hit_date, id_list):
conn = snowflake.connector.connect(**snowflake_creds)
cursor = conn.cursor()
with open(query.sql) as f:
query = f.read()
cursor.execute(query.format(hit_date, id_list))
The problem comes when the input for 'id_list' is 'None'. I am thrown this error:
snowflake.connector.errors.ProgrammingError: 000904 (42000): 019b2591-04da-7b13-0000-89bd25e0043a: SQL compilation error: error line 20 at position 42
invalid identifier 'NONE'
From my understanding Python's 'None' will automatically convert to SQL's 'Null'. Is this not the case? What should the input for 'id_list' be if 'None/Null'?
I was unable to find a clear answer for this specific error. However I was able to work around this by replacing the condition with an impossible value when no list is given.
#query.sql
select *
from {some_table}
where
date == '{hit_date}'
and
(id in ({id_list}) or impossible_value in ({id_list}))
I have an oracle procedure that is supposed to return a chunk of json.
The procedure has 4 paramaters
input is an ID value
output json CLOB
output some message in json format CLOB
output if success or failure varchar2
so in my code I have done the following just to test if I can successfully call and return it
ConnectionString = 'someconnection.connection'
con = cx_Oracle.connect(ConnectionString)
cur = con.cursor()
ID = '51858645'
json_out = cur.var(cx_Oracle.CLOB)
message = cur.var(cx_Oracle.CLOB)
status = cur.var(cx_Oracle.STRING)
oracle_return = cur.callproc('project.getsomejson',[ID,json_out,message,status])
However, it fails and returns
PLS-00306: wrong number or types of arguments in call to 'getsomejson'
ORA-06550: line 1, column 7:
PL/SQL: Statement ignored
This is the procedure definition
procedure callOracle(json_in clob,json_out out clob,message out clob,status out varchar2)
This is calling an oracle 12c db
I'm really not familiar at all with calling procedures in python. Typically I callfunc with the stored type and just get a return
The procedure is expecting
CLOB, CLOB, CLOB, VARCHAR2
but you are passing
VARCHAR2, CLOB, CLOB, VARCHAR2
The name you gave (callOracle) also doesn't match what you are calling in Python (project.getsomejson). Perhaps verify that you have the right procedure signature? Assuming it is correct, though, you'll need to change the first one to be a CLOB as well or change the stored procedure to accept VARCHAR2. Something like this should do it:
json_in = conn.createlob(cx_Oracle.DB_TYPE_CLOB)
json_in.write("51858645")
json_out_var = cur.var(cx_Oracle.DB_TYPE_CLOB)
message_var = cur.var(cx_Oracle.DB_TYPE_CLOB)
status_var = cur.var(str)
cur.callproc(json_in, json_out, message, status)
I am trying to add argv[0] as variable to the SQL query below and running into compilation error below,what is the syntax to fix this?
#!/usr/bin/python
import pypyodbc as pyodbc
from sys import argv
component_id=argv[0]
server_name='odsdb.company.com'
database_name='ODS'
cnx = pyodbc.connect("DRIVER={SQL Server};SERVER="+server_name+";DATABASE="+database_name)
db_cursor=cnx.cursor()
SQL = 'SELECT Top 1 cr.ReleaseLabel ' + \
'FROM [ODS].[v000001].[ComponentRevisions] cr ' + \
'WHERE cr.ComponentId=' + component_id + \
'ORDER BY cr.CreatedOn DESC'
resp_rows_obj=db_cursor.execute(SQL)
print '('+', '.join([column_heading_tuple[0] for column_heading_tuple in resp_rows_obj.description])+')'
for row in resp_rows_obj:
print row
Error:-
pypyodbc.ProgrammingError: (u'42000', u"[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]Incorrect syntax near the keyword 'BY'.")
Don't use string interpolation. Use SQL parameters; these are placeholders in the query where your database will insert values:
SQL = '''\
SELECT Top 1 cr.ReleaseLabel
FROM [ODS].[v000001].[ComponentRevisions] cr
WHERE cr.ComponentId = ?
ORDER BY cr.CreatedOn DESC
'''
resp_rows_obj = db_cursor.execute(SQL, (component_id,))
Values for the ? placeholders are sourced from the second argument to the cursor.execute() function, a sequence of values. Here you only have one value, so I used a one-element tuple.
Note that you probably want argv[1], not argv[0]; the latter is the script name, not the first argument.
to retrieve 1st command line argument do component_id=argv[1] instead of 0 which is the script name...
better yet, look at argparse
We had a hyphen in the database name that was being used in a T-SQL query being called from Python code. So we just added square brackets because SQL Server cannot interpolate the hyphen without them.
Before:
SELECT * FROM DBMS-NAME.dbo.TABLE_NAME
After:
SELECT * FROM [DBMS-NAME].dbo.TABLE_NAME
Im trying to pass a sql ( wich works perfectly if i run it on the client ) inside my python script, but i receive the error "not enough arguments for format string"
Following, the code:
sql = """
SELECT
rr.iserver,
foo.*, rr.queue_capacity,
rr.queue_refill_level,
rr.is_concurrent,
rr.max_execution_threads,
rr.retrieval_status,
rr.processing_status
FROM
(
SELECT DISTINCT
ip.package,
it. TRIGGER
FROM
wip.info_package ip,
wip.info_trigger it
WHERE
ip.service = it.service and
ip.iserver = '%(iserver)s' and
it.iserver = %(iserver)s'
AND package = '%(package)s'
UNION
SELECT
'%(package)s' AS package,
TRIGGER
FROM
info_trigger
WHERE
TRIGGER LIKE '%(package)s%'
) AS foo,
info_trigger rr
WHERE
rr. TRIGGER = foo. TRIGGER
""" % {'iserver' : var_iserver,'package' : var_package}
dcon = Database_connection()
getResults = dcon.db_call(sql, dbHost, dbName, dbUser, dbPass)
# more and more code to work the result....
My main problem on this is how i can pass '%(iserver)s' , '%(package)s' correctly. Because usualy, when i select's or insert's on database, i only use two variables , but i dont know how to do it with more than two.
Thanks.
Don't build SQL like this using %:
"SELECT %(foo)s FROM bar WHERE %(baz)s" %
{"foo": "FOO", "baz": "1=1;-- DROP TABLE bar;"}
This opens the door for nasty SQL injection attacks.
Use the proper form of your Python Database API Specification v2.0 adapter. For Psychopg this form is described here.
cur.execute("SELECT %(foo)s FROM bar WHERE %(baz)s",
{"foo": "FOO", "baz": "1=1;-- DROP TABLE bar;"})
WHERE
TRIGGER LIKE '%(package)s%'
you have an EXTRA '%'
if you want the actual character '%', you need to escape with a double '%'.
so it should be
WHERE
TRIGGER LIKE '%(package)s%%'
if you want to display a '%'
and
WHERE
TRIGGER LIKE '%(package)s'
if you dont