I am new in using python API to send a query to mysql.
My issue is very easy to reproduce. I have a table named "ingredient" and I would like to select the rows from python using parameters
If I do cursor.execute("select * from ?",('ingredient',)) I get the error message : Error while connecting to MySQL Not all parameters were used in the SQL statement MySQL connection is closed
I I do cursor.execute("select * from ?",'ingredient') I get the error message : Error while connecting to MySQL 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '?' at line 1
Same issues using %s instead of ?. Using the other type of single quote on 'ingredient' instead of 'ingredient' does not give results either.
How is this supposed to work here ?
You just can't pass a table name as parameter to a query. The parameterization mechanism is there to pass literal values, not object names. Keep in mind that the database must be able to prepare the query plan from just the parameterized string (without the actual parameter value), which disqualifies using metadata as parameter.
You need string concatenation instead:
cursor.execute("select * from " + yourvar);
Note that, if the variable comes from outside your program, using such contruct exposes your code to SQL injection. You need to manually validate the value of the parameter before execting the query (for example by checking it against a fixed list of allowed values, or by querying the information schema of the database to ensure that the table does exist).
Does your query work if you just write:
cursor.execute("SELECT * FROM ingredient")
?
Related
I would like to write a python script to create new MariaDB databases.
The database name is a user input. I tried to use arguments for creating the database:
#!/usr/bin/python3
import mysql.connector
mariadb_host = '127.0.0.1'
mariadb_port = 3306
mariadb_user = 'root'
mariadb_password = 'password'
mariadb_connection = mysql.connector.connect(
host=mariadb_host,
port=mariadb_port,
user=mariadb_user,
passwd=mariadb_password,
use_pure=True
)
query = 'CREATE DATABASE %(db_name)s;'
args = {'db_name': 'test-db'}
result = None
cursor = mariadb_connection.cursor()
cursor.execute(query, args)
print(cursor.statement)
result = cursor.fetchall()
cursor.close()
The following error appears: mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''test-db'' at line 1
It seems, that the command cursor.execute appends ' around the database name, which results in an invalid sql query.
How could I get around this problem and create safely new database from user input?
Parameter substitution notation - %(name)s or just %s is for interpolating values into an SQL statement.
RDBMSs have different quoting rules for values and identifiers like database, table or column names. For example, a string value will be surrounded by single quotes to tell the RDBMS that is is a character value, but single-quoting an identifier is a syntax error; the RDBMS will require that identifiers are quoted using some other character (for example backticks, double-quotes, square brackets, depending on the RDBMS).
If you want to interpolate identifiers using Python you have to use string formatting techniques. For example, using an f-string
db_name = 'test-db'
query = f'CREATE DATABASE `{db_name}`;'
Note that it is best to quote dynamic identifier names with backticks to handle names which contain special characters.
As always with dynamic SQL generation, you should be aware of the risk of SQL injection when handling data from an untrusted source.
I am trying to run a SQL script in Python where I am passing a variable in the where clause as below:
cursor.execute(f"""select * from table where type = variable_value""")
In the above query, variable_value has the value that I am trying to use in the where clause. I am however getting an error psycopg2.errors.UndefinedColumn: column "variable_value" does not exist in table
As per psycopg2 documentation the execute function takes variables as an extra parameter.
cursor.execute("""select * from table where type = %(value)s """, {"value": variable_value})
More examples in psycopg2 user manual..
Also please read carefully the section about SQL injection - the gist is, you should not quote parameters in your query, the execute function will take care of that to prevent the injection of harmful SQL.
Also to explain the error you are getting - the query you're sending is comparing two identifiers (type and variable_value). The table does not contain variable_value column, hence the error.
I believe, you intended to use string interpolation to construct the query, but you forgot the {}. It would work like this:
cursor.execute(f"""select * from table where type = '{variable_value}'""")
⚠️ but because of previously mentioned SQL injection, it is not a recommended way!.
I am querying an MS Access database from Python using the pyodbc module. I am able to do this if I query all records in a table, but when I add a where clause, I am getting an error.
This is my code:
wpc_ids = ['WPCMOOTEST2', 'WPCMOOTEST1']
conn = pyodbc.connect(r'Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=P:\Conservation Programs\Natural Heritage Program\Data Management\ACCESS databases\POND_entry\POND_be.accdb;')
cursor = conn.cursor()
wpc_list = ','.join(str(x) for x in wpc_ids)
cursor.execute('SELECT * FROM pools WHERE wpc_id IN (%s)'%wpc_list)
I am getting the following error:
Error: ('07002', u'[07002] [Microsoft][ODBC Microsoft Access Driver] Too few parameters. Expected 2. (-3010) (SQLExecDirectW)')
I don't get that error without the where clause, so I'm not sure what the second parameter is that I need. Can anyone help with this?
cursor.execute(
'SELECT * FROM pools WHERE wpc_id IN ({})'.format(
','.join('?'*len(wpc_ids))), wpc_ids
)
Explanation:
There is a PEP about databases, PEP249, you can read it here https://www.python.org/dev/peps/pep-0249/
This PEP defines how API of database modules should be. pyodbc is the database module you're using, and it is compatible with PEP249.
One of the things the PEP defines is that each module should have a paramstyle. pyodbc.paramstyle is qmark so that is why you use '?' with pyodbc. More details https://www.python.org/dev/peps/pep-0249/#paramstyle
Now, instead of building a query as a string and sending it to the database, the idea is to use parameter passing, which is a way to send the query and the parameters separately... It uses the paramstyle to put placeholders in the query, then you pass a sequence of parameters as a second parameter to execute. Example:
sql = 'SELECT * FROM foo WHERE id = ? AND text_col = ?'
params = (12, 'testing')
cursor.execute(sql, params)
Note that this is not mixing the params with the string. The code is passing them as two separate arguments to .execute(). That means it will be the database's job to do the interpolation safely.
Since you want to pass multiple values to the query, you must generate a string containing the number of placeholders separated by comma, same number as the elements in the list:
','.join('?'*len(wpc_ids)))
# will generate ?,?,?,?,? according with length of list
I am using a python to update entries in Sqlite table.
The command I am using is:
handle.execute("UPDATE RECORD set NAME=%s DEVICE=%s PROJECT=%s IP=%s COMMENT=%s where ID = %s"%(arg[2],arg[3],arg[4],arg[5],arg[6],arg[1]))
To this I get am getting an error as:
sqlite3.OperationalError: near "DEVICE": syntax error
I cannot understand what is specifically wrong with Device. Also I have checked the variables are as expected. The data base has a column named device and the database can be opened / accessed and edited using this python file.
There are commas missing between set items.
In addition to that, instead of string formatting, pass parameters to prevent SQL injection:
handle.execute(
"""UPDATE RECORD
SET NAME=%s, DEVICE=%s, PROJECT=%s, IP=%s, COMMENT=%s
WHERE ID = %s""",
(arg[2], arg[3], arg[4], arg[5], arg[6], arg[1]))
UPDATE
If you insist to use string formatting, you should quote %s: '%s'
I'm trying to compose a string SQL query using SQLALchemy 1.1.2. I followed the explanation from the docs about using textual SQL but encountered a syntax error when I ran the following code:
from sqlalchemy.sql import text
# Create a database connection called "connection"...
q = text('USE :name')
connection.execute(q, name='DATABASE_NAME')
Here's the error message:
"You have an error in your SQL syntax; check the manual that
corresponds to your MySQL server version for the right syntax to use
near ''DATABASE_NAME'' at line 1") [SQL: u'USE %s;'] [parameters:
(u'DATABASE_NAME',)]
Since I'm using the named colon format and passing the parameters as arguments to connection.execute I can't figure out why this problem is arising. I'm using a MySQL server, but if I read the docs correctly the text method should be DB-agnostic.
Thanks in advance for the help.
According to the documentation you need to use the bindparams like so:
q = text('USE :name')
q.bindparams(name="DATABASE_NAME")
connection.execute(q)
or like this:
q = text('USE :name')
q = q.bindparams(bindparam("name", String))
connection.execute(q, {"name": "DATABASE_NAME"})
This worked for me with no issues. Edit: I was wrong, it didn't work.
The problem is the bind params is going to auto wrap your value with a single quote. So what's happening is you get the final compiles statement (which is invalid syntax):
use 'DATABASE_NAME'
If you were to create the query: "Select * from mytable where column_a=:name"; this will work. Because it's wrapping the value with single quotes.
I would suggest for your use statement to do:
q = "USE {}".format("DATABASE_NAME")
Or something similar.