I am trying to get the mssql table column names using pyodbc, and getting an error saying
ProgrammingError: No results. Previous SQL was not a query.
Here is my code:
class get_Fields:
def GET(self,r):
web.header('Access-Control-Allow-Origin', '*')
web.header('Access-Control-Allow-Credentials', 'true')
fields = []
datasetname = web.input().datasetName
tablename = web.input().tableName
cnxn = pyodbc.connect(connection_string)
cursor = cnxn.cursor()
query = "USE" + "[" +datasetname+ "]" + "SELECT COLUMN_NAME,* FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = " + "'"+ tablename + "'"
cursor.execute(query)
DF = DataFrame(cursor.fetchall())
columns = [column[0] for column in cursor.description]
return json.dumps(columns)
how to solve this?
You can avoid this by using some of pyodbc's built in methods. For example, instead of:
query = "USE" + "[" +datasetname+ "]" + "SELECT COLUMN_NAME,* FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = " + "'"+ tablename + "'"
cursor.execute(query)
DF = DataFrame(cursor.fetchall())
Try:
column_data = cursor.columns(table=tablename, catalog=datasetname, schema='dbo').fetchall()
print(column_data)
That will return the column names (and other column metadata). I believe the column name is the fourth element per row. This also relieves the very valid concerns about SQL injection. You can then figure out how to build your DataFrame from the resulting data.
Good luck!
Your line
query = "USE" + "[" +datasetname+ "]" + "SELECT COLUMN_NAME,*...
Will produce something like
USE[databasename]SELECT ...
In SSMS this would work, but I'd suggest to look on proper spacing and to separate the USE-statement with a semicolon:
query = "USE " + "[" +datasetname+ "]; " + "SELECT COLUMN_NAME,*...
Set the database context using the Database attribute when building the connection string
Use parameters any time you are passing user input (especially from HTTP requests!) to a WHERE clause.
These changes eliminate the need for dynamic SQL, which can be insecure and difficult to maintain.
Related
I have multiple tables that are updated after a value is changed in a grid. These tables don't always have the same keys or columns so I cannot explicitly name the columns or formats. The only thing that is ever the same, is the column where the keys reside. I know the way I am currently doing this is not correct and leaves me open to injection attacks.
I also ran into an issue where some of the values contain keys that throw an error in the SQL statement. For example, updating WHERE email = t'est#email.com.
I am not really sure of the proper way to write these statements. I did some research and see multiple methods for different purposes but am not sure which is proper. I am looking to do this as dynamically as possible. Can anyone point me in the right direction?
To connect:
import mysql.connector as sql
import MySQLdb
#Connect
self.db_name = 'database'
self.server = 'server'
self.user_id = 'user'
self.pw = 'password'
try:
self.db_con = MySQLdb.connect(user=self.user_id,password=self.pw,database=self.db_name,
host=self.server,charset='utf8',autocommit=True)
self.cursor = self.db_con.cursor()
except:
print("Error connecting")
SQL Statements:
key_id = str("'") + self.GetCellValue(event.GetRow(),1) + str("'")
target_col = self.GetColLabelValue(event.GetCol())
key_col = self.GetColLabelValue(1)
nVal = str("'") + self.GetCellValue(event.GetRow(),event.GetCol()) + str("'")
#SQL STATEMENTS
sql_update = "UPDATE " + tbl + " SET " + target_col + " = " + nVal + " WHERE " + key_col + " = " + key_id + ""
#INSERT
sql_update = ("INSERT INTO " + str(self.tbl) + "(" + self.key_col + ")" + "VALUES (" + str("'") + str(val) + str("'") + ")")
#DELETE
sql_update = "DELETE FROM " + tbl + " WHERE " + self.key_col + " = " + self.key_id + ""
#SELECT
sql_query = "SELECT * FROM " + self.tbl
#Excecute
try:
self.cursor.execute(sql_update)
except:
print('Error')
self.db_con.rollback()
Databases have different notations for "quoting" identifiers (table and column names etc) and values (data).
MySQL uses backticks to quote identifiers. For values, it's best to use the parameter substitution mechanism provided by the connector package: it's more likely to handle tricky cases like embedded quotes correctly, and will reduce the risk of SQL injection.
Here's an example for inserts; the same techniques can be used for the other types of query.
key_id = str("'") + self.GetCellValue(event.GetRow(),1) + str("'")
target_col = self.GetColLabelValue(event.GetCol())
key_col = self.GetColLabelValue(1)
nVal = str("'") + self.GetCellValue(event.GetRow(),event.GetCol()) + str("'")
#INSERT (using f-strings for brevity)
sql_update = (f"INSERT INTO `{self.tbl}` (`{self.key_col}`) VALUES (%s)")
# Pass the statement and values to cursor.execute.
# The values are assumed to be a sequence, so a single value should be
# placed in a tuple or list.
self.cursor.execute(sql_update, (nVal,))
If you have more than one column / value pair you could do something like this:
cols = ['A', 'B', 'C']
vals = ['a', 'b', 'c']
col_names = ','.join([f'`{c}`' for c in cols])
values_placeholder = ','.join(['%s'] * len(cols))
sql_update = (f"INSERT INTO `{self.tbl}` (col_names) VALUES ({values_placeholder})")
self.cursor.execute(sql_update, vals)
Values are not only data for insertion, but also data that we are using for comparison, for example in WHERE clauses. So an update statement with a filter might be created like this:
sql_update = (f"UPDATE `{tbl}` SET (`{target_col}`) = (%s) WHERE (`{key_col}`) = %s")
self.cursor.execute(sql_update, (nVal, key_id))
However sometimes the target of a SET or WHERE clause may be a column, for example we want to do an update based on other values in the row. For example, this statement will set target_col to the value of other_col for all rows where key_col is equal to other_key_col:
sql_update = (f"UPDATE `{tbl}` SET (`{target_col}`) = `{other_col}` WHERE (`{key_col}`) = `{other_key_col}`")
self.cursor.execute(sql_update)
I am newbie to snowflake. I am trying to fetch ddl s of all tables in my db using python script.
import snowflake.connector
import sys
# Gets the version
cnx = snowflake.connector.connect(
user='username',
password='password',
account='account',
database='db',
schema='schema',
warehouse='warehouse',
role='role'
)
cnx.cursor().execute("USE warehouse warehouseName")
cnx.cursor().execute("USE database dbName")
cnx = cnx.cursor()
dbSchema='schema name'
sql_select_objects = "select TABLE_NAME,TABLE_SCHEMA,TABLE_TYPE from INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA='" + dbSchema + "';"
print(sql_select_objects)
try:
cnx.execute(sql_select_objects)
print('Query ID=' + cnx.sfqid)
rows = cnx.fetchall()
print("rows count:", len(rows))
for row in rows:
#print(row[0])
file = open(row[0] + ".sql","w")
rw='"' + row[0] + '"'
print(rw)
sql_ddl_object = "SELECT GET_DDL('TABLE', '" + row[0] + "')"
print(sql_ddl_object)
cnx.execute(sql_ddl_object)
print('SQL2 sfqid=' + cnx.sfqid)
row_ddl_table = cnx.fetchall()
#print(row_ddl_table[0][0])
file.write(str(row_ddl_table[0][0]))
file.write("\n")
finally:
cnx.close()
When I execute above script i get error:
snowflake.connector.errors.ProgrammingError: 002003 (02000): SQL compilation error:
Table 'SNOWFLAKE_TEST' does not exist or not authorized.
in line "SELECT GET_DDL('TABLE', '" + row[0] + "')", i believe python is not parsing value of row[0] correctly. Can you please suggest me where the error is?
Please use both schema and table name as you may have the table SNOWFLAKE_TEST in a different schema:
"SELECT GET_DDL('TABLE', '" + row[1] + "." + row[0] + "')"
I would also surround the table name and schema names in double quotes because maybe your schema or table is created with a case-sensitive name:
"SELECT GET_DDL('TABLE', '\"" + row[1] + "\".\"" + row[0] + "\"')"
Most likely the issue is not with the table name but the role that you are using in the code. The role most likely does not have access to the DB and it's table hence the error message is seen.
I have multiple tables that are updated after a value is changed in a grid. These tables don't always have the same keys or columns so I cannot explicitly name the columns or formats. The only thing that is ever the same, is the column where the keys reside. I know the way I am currently doing this is not correct and leaves me open to injection attacks.
I also ran into an issue where some of the values contain keys that throw an error in the SQL statement. For example, updating WHERE email = t'est#email.com.
I am not really sure of the proper way to write these statements. I did some research and see multiple methods for different purposes but am not sure which is proper. I am looking to do this as dynamically as possible. Can anyone point me in the right direction?
To connect:
import mysql.connector as sql
import MySQLdb
#Connect
self.db_name = 'database'
self.server = 'server'
self.user_id = 'user'
self.pw = 'password'
try:
self.db_con = MySQLdb.connect(user=self.user_id,password=self.pw,database=self.db_name,
host=self.server,charset='utf8',autocommit=True)
self.cursor = self.db_con.cursor()
except:
print("Error connecting")
SQL Statements:
key_id = str("'") + self.GetCellValue(event.GetRow(),1) + str("'")
target_col = self.GetColLabelValue(event.GetCol())
key_col = self.GetColLabelValue(1)
nVal = str("'") + self.GetCellValue(event.GetRow(),event.GetCol()) + str("'")
#SQL STATEMENTS
sql_update = "UPDATE " + tbl + " SET " + target_col + " = " + nVal + " WHERE " + key_col + " = " + key_id + ""
#INSERT
sql_update = ("INSERT INTO " + str(self.tbl) + "(" + self.key_col + ")" + "VALUES (" + str("'") + str(val) + str("'") + ")")
#DELETE
sql_update = "DELETE FROM " + tbl + " WHERE " + self.key_col + " = " + self.key_id + ""
#SELECT
sql_query = "SELECT * FROM " + self.tbl
#Excecute
try:
self.cursor.execute(sql_update)
except:
print('Error')
self.db_con.rollback()
Databases have different notations for "quoting" identifiers (table and column names etc) and values (data).
MySQL uses backticks to quote identifiers. For values, it's best to use the parameter substitution mechanism provided by the connector package: it's more likely to handle tricky cases like embedded quotes correctly, and will reduce the risk of SQL injection.
Here's an example for inserts; the same techniques can be used for the other types of query.
key_id = str("'") + self.GetCellValue(event.GetRow(),1) + str("'")
target_col = self.GetColLabelValue(event.GetCol())
key_col = self.GetColLabelValue(1)
nVal = str("'") + self.GetCellValue(event.GetRow(),event.GetCol()) + str("'")
#INSERT (using f-strings for brevity)
sql_update = (f"INSERT INTO `{self.tbl}` (`{self.key_col}`) VALUES (%s)")
# Pass the statement and values to cursor.execute.
# The values are assumed to be a sequence, so a single value should be
# placed in a tuple or list.
self.cursor.execute(sql_update, (nVal,))
If you have more than one column / value pair you could do something like this:
cols = ['A', 'B', 'C']
vals = ['a', 'b', 'c']
col_names = ','.join([f'`{c}`' for c in cols])
values_placeholder = ','.join(['%s'] * len(cols))
sql_update = (f"INSERT INTO `{self.tbl}` (col_names) VALUES ({values_placeholder})")
self.cursor.execute(sql_update, vals)
Values are not only data for insertion, but also data that we are using for comparison, for example in WHERE clauses. So an update statement with a filter might be created like this:
sql_update = (f"UPDATE `{tbl}` SET (`{target_col}`) = (%s) WHERE (`{key_col}`) = %s")
self.cursor.execute(sql_update, (nVal, key_id))
However sometimes the target of a SET or WHERE clause may be a column, for example we want to do an update based on other values in the row. For example, this statement will set target_col to the value of other_col for all rows where key_col is equal to other_key_col:
sql_update = (f"UPDATE `{tbl}` SET (`{target_col}`) = `{other_col}` WHERE (`{key_col}`) = `{other_key_col}`")
self.cursor.execute(sql_update)
When I run a query from sqlite browser the table get updated but when I use same query from Python the database won't get updated:
def updateDB (number, varCheck=True):
conn = sqlite3.connect(db)
c = conn.cursor()
i = 1
for each_test in number:
c.execute("UPDATE table1 SET val='%s' WHERE amount='%s' AND rank='%s'" % (each_test , str(i), 'rank2'))
i += 1
conn.commit()
conn.close()
return True
How can I fix the issue? I run python code as sudo.
In the past, I had similar issues while creating sql queries. I doubt if your sql query is being correctly formatted. The % string interpolation method can be a problem. Try using the .format() on the sql query string. PEP3101 explains the same about using .format() instead of % operator for string interpolation.
val='"' + each_test + '"'
amount = '"' + str(i) + '"'
rank= '"' + "rank2" + '"'
sql_qeury = "UPDATE table1 SET val={val} WHERE amount={amount} AND rank={rank}".format(val=val,amount=amount,rank=rank)
I have this Python code:
def get_employees(conditions, fields):
cursor.execute("SELECT employeeID FROM employees WHERE name=%s, budget=%s,
%year=%s,(some of conditions))
Is there any way to get employeeIDs if I set in conditions not all parameters, etc. only name and year?
If conditions were a dictionary, you could construct a query string:
def get_employees(conditions):
query = 'select employeeid from employees'
if conditions:
query += ' where ' + ' and '.join(key + ' = %s' for key in conditions.keys())
cursor.execute(query, conditions.values())
(I should note that here I am assuming that conditions does not have user-supplied keys. If there are user-supplied keys, this is definitely vulnerable to SQL injection.)
usually it is done via dynamic sql building, like this:
sql = "SELECT employeeID FROM employees WHERE 1=1";
if condition has name
sql += " and name='" .escape(name) . "'"
if condition has somefield
sql += " and somefield='" .escape(somefield) . "'"
etc
execute final sql