I'm using Python + MySQL and want to use parameterized query. I'm stuck. I've encountered an error and can't figure out how to solve it. I've spent a day, checked dozens of articles, used various options (sinle quotes, double quotes, prepared statements) and still no luck.
Requirements: use Parameterized Query
Here is basic demo of the issue:
#!/usr/bin/python3
import mysql.connector as mysql
conn = mysql.connect(host=server, user=username, passwd=password, autocommit=True)
try:
create_database_query = "CREATE DATABASE %s;"
db_name = "BOOKS"
cursor = conn.cursor()
print(f"Creating {db_name} database... ", end='')
cursor.execute(create_database_query, (db_name,))
print("Success")
except mysql.Error as error:
print("Parameterized query failed {}".format(error))
Output:
Creating BOOKS database... Parameterized query failed 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''BOOKS'' at line 1
So it looks like it uses too many quotes (2 single quotes on each side). The code above works fine if I change the following line:
create_database_query = "CREATE DATABASE %s;"
and put backtick around %s
The problem that now it creates a database but with invalid chars - 'BOOKS' (quotes are now part of db name). Duh...
If I use prepared statements then the same issue occurs but slightly different error message:
1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '?' at line 1
Environment:
MacOS Catalina
Python 3.8
PyCharm 2019.3 IDE
MySQL 8.0.19
mysql-connector-python module 8.0.19
What is going on? Any ideas?
Thanks
You can't use query parameters for identifiers (like a database name or table name or column name).
Query parameters can be used only in place of a constant value — a quoted string, quoted date/time, or a numeric value. Not identifiers, expressions, SQL keywords, etc.
To combine a database name with your CREATE DATABASE statement, you have to format it into the string in a way that forms the full statement before it is sent to MySQL.
db_name = "BOOKS"
create_database_query = "CREATE DATABASE %s;" % db_name
cursor.execute(create_database_query)
Because this creates a risk of SQL injection when you format variables into your string, it's up to you to make sure the db_name is safe.
Update: Thanks to #Parfait for the reminder about current best practices of string-formatting.
Prefer:
db_name = "BOOKS"
create_database_query = "CREATE DATABASE {};".format(db_name)
Or F-strings:
db_name = "BOOKS"
create_database_query = f"CREATE DATABASE {db_name};"
(In other words, Python has become Ruby ;-)
Related
I know this has been asked a million times, but noone seems to provide an alternative...
So this doesn't work:
DW = pymysql.connect(...)
table = "myTable"
with DW.cursor() as cur:
cur.execute("SELECT * FROM %s", (table,)) # error here
data = cur.fetchall()
Error:
ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''myTable'' at line 1")
psycopg2 for PostgreSQL has psycopg2.sql.Identifier or AsIs which basically removes the single quotes and acts as an identifier in the query...is there an equivalent for pymysql?
Also, other answers use string formatting and other methods that are vulnerable to SQL injection - this is something I want to avoid.
I would like to write a python script to create new MariaDB databases.
The database name is a user input. I tried to use arguments for creating the database:
#!/usr/bin/python3
import mysql.connector
mariadb_host = '127.0.0.1'
mariadb_port = 3306
mariadb_user = 'root'
mariadb_password = 'password'
mariadb_connection = mysql.connector.connect(
host=mariadb_host,
port=mariadb_port,
user=mariadb_user,
passwd=mariadb_password,
use_pure=True
)
query = 'CREATE DATABASE %(db_name)s;'
args = {'db_name': 'test-db'}
result = None
cursor = mariadb_connection.cursor()
cursor.execute(query, args)
print(cursor.statement)
result = cursor.fetchall()
cursor.close()
The following error appears: mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''test-db'' at line 1
It seems, that the command cursor.execute appends ' around the database name, which results in an invalid sql query.
How could I get around this problem and create safely new database from user input?
Parameter substitution notation - %(name)s or just %s is for interpolating values into an SQL statement.
RDBMSs have different quoting rules for values and identifiers like database, table or column names. For example, a string value will be surrounded by single quotes to tell the RDBMS that is is a character value, but single-quoting an identifier is a syntax error; the RDBMS will require that identifiers are quoted using some other character (for example backticks, double-quotes, square brackets, depending on the RDBMS).
If you want to interpolate identifiers using Python you have to use string formatting techniques. For example, using an f-string
db_name = 'test-db'
query = f'CREATE DATABASE `{db_name}`;'
Note that it is best to quote dynamic identifier names with backticks to handle names which contain special characters.
As always with dynamic SQL generation, you should be aware of the risk of SQL injection when handling data from an untrusted source.
I'm basically using the following code to drop an existing table using Python 3.6 and PyMySQL, with a MySQL 5.7 database:
connection = pymysql.connect(
host=host,
user=username,
password=password,
db=database,
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor
)
table_name = "video_playback_statistics"
sql = "DROP TABLE IF EXISTS %s"
params = (table_name,)
with connection.cursor() as cursor:
cursor.execute(sql, params)
I get the following error:
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''video_playback_statistics'' at line 1"
I've successfully executed other, more complicated SQL statements with more parameters, but this simple one does not work.
When I run the SQL command without parametrization, it works fine:
cursor.execute('DROP TABLE IF EXISTS video_playback_statistics')
I've looked at the following questions:
1064, "You have an error in your SQL syntax;..." Python MySQL
pymysql return error SQL syntax
But these kinds of queries seem to work fine.
Also I've seen this:
Python MYSQL update statement
But here the table name is directly written into the string.
Apparently the parametrization of table names or column names is not possible, according to this comment. The adapter, for some reason, inserts two double quotes for the value, which causes a syntax error.
Two possible solutions would be:
Use backslash escaping for the parameter:
sql = "DROP TABLE IF EXISTS `%s`"
Use Python string formatting:
sql = "DROP TABLE IF EXISTS {}".format(table_name)
For values (e.g. in INSERT or WHERE statements), you should still use parametrization.
I'm now learning to use MySQL with python.
When I'm trying to create a new database like this:
sql = 'CREATE DATABASE IF NOT EXISTS %s'
cursor.execute(sql, (self.DB_NAME,))
DB_NAME is a string, in this case
self.DB_NAME = 'bmagym'
I got this error:
MySQL Error [1064]: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''bmagym'' at line 1
But if I replace the code with:
sql = 'CREATE DATABASE IF NOT EXISTS %s' %self.DB_NAME
cursor.execute(sql)
It works as expected.
My question is how I can pass parameters to execute() instead of using %?
Database names in the SQL syntax are written like
`dbname`
SQL accepts plain dbname as well.
CREATE DATABASE IF NOT EXISTS dbname
# works like
CREATE DATABASE IF NOT EXISTS `dbname`
# BUT: only if dbname is not a SQL keyword.
The cursor.execute() function will automatically format and escape strings (prevent SQL injection). So this query is executed:
CREATE DATABASE IF NOT EXISTS 'dbname'
which is a syntax error. Here is a similar topic on this question. Your second method is fine, just substitute the database name with python % operator.
sql = 'CREATE DATABASE IF NOT EXISTS `%s`' %self.DB_NAME
I'm a postgres newbie and am having some issues querying a text field in postgresql using Python. What is the correct syntax that will allow me to search the content of column "body" from table "jivemessage" out of database "postgres"?
try:
conn = psycopg2.connect("dbname='postgres' user='postgres' host='localhost' password='<password>'")
except:
print "cannot connect"
i = 'test'
cur = conn.cursor()
cur.execute('SELECT * from jivemessage WHERE body LIKE "%'+i+'%"')
Keep getting the following error:
ProgrammingError: column "%test%" does not exist
Thanks for any help.
You are not quoting the query properly. Don't use string concatenation here, use SQL parameters instead:
cur.execute('SELECT * from jivemessage WHERE body LIKE %s', ("%{}%".format(i),))
Here, the %s placeholder signals to the database driver that the first value of the second argument should be placed there when querying.
This leaves the interpolation up to the database driver, giving the database the opportunity to optimize for the query once, even if you were to reuse the same query.
It also prevents SQL injection attacks better than you could yourself, and most of all, guarantees that the correct quoting rules are followed.