I have a simple script that downloads data out of one database (Teradata), does some stuff to it, and uploads it into another (MySQL) database. This has worked well for months now, but yesterday in my logs I noticed that the script failed, and gave me back this error:
An error has occurred:
(<class '_mysql_exceptions.ProgrammingError'>, ProgrammingError(1064,
'You have an error in your SQL syntax; check the manual that
corresponds to your MySQL server version for the right syntax to use
near \'A435", NULL, "2018-01-18", 95,\' at line 1'), <traceback object
at 0x00000000019A1A48>)
It seems like the culprit may be one of the user-editable fields, although each of these fields is processed by mysqli_real_escape_string in PHP prior to writing to the database - so not sure there.
While it would be nice from a programming standpoint to understand exactly what happened here, I'm more concerned with editing the python script to include an error handler that just skips over any line that causes an error instead of exiting the entire script.
Here's the script:
# Upload this to MySQL
db = MySQLdb.connect(host='asdf',user='asdf',passwd='asdf',db='asdf')
cursor = db.cursor()
csv_data = csv.reader(file(csvfilename))
for row in csv_data:
cursor.execute('INSERT INTO `test` (field_1,field_2,field_3,field_4,field_5)'\
'VALUES(%s,%s,%s,%s,%s)',row);
db.commit()
# Close the connection to the database.
cursor.close()
When use MySQLdb package, triple quotes are used for quoting query string:
Document from link: http://mysql-python.sourceforge.net/MySQLdb.html
db.query("""SELECT spam, eggs, sausage FROM breakfast WHERE price < 5""")
In your case, it might coincidence that you have both single quote and double qutoe in the parsed line and then the script crashes the insertion execution.
Related
I wish to create a trigger in MariaDB 5.5.68.
Base on this official example, I built this query:
query = ("""
DELIMITER //
create trigger set_uuid_query
before insert on DLMNT.QUERY for each row
begin
if new.id is null then
set new.id = uuid() ;
end if ;
end//
DELIMITER ;
""")
cursor = mydb.cursor()
cursor.execute(query)
for e in cursor:
print(e)
However, while this worked well with a MariaDB 5.5.64 via MySQL Workbench, this throws:
1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'DELIMITER //
create trigger set_uuid_query
before insert on DLMNT.QUERY for each' at line 1
I am afraid that this is not possible. While it is about MySQL, this answer states that DELIMITER is a client side thing.
Also, based on the last line of this doc, I though "\G" could be used as a delimiter, but this answer states something completely different (and it throws the exact same error anyway when I try it).
So, using this Python library, how can I make such a query ?
PS: the lib I am using is:
mysql-connector-python 8.0.27
You do not DELIMITER to create trigger, when using python (or any other) connector. DELIMITER is a command line client's special trick. Command line client wishes to know when to send a potentially multiline input query to the server. Usually it can rely on semicolons at the end of string, but not in the case of "complex" statements, such as trigger and stored procedure.
I have tried everything and keep getting this error:
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax;
check the manual that corresponds to your MariaDB server version for the right syntax to use near
'INSERT INTO tabSingles (doctype, field, value) VALUES ('Bank Reconciliation', 'a' at line 2")
Expanded query (after python format expansion):
SELECT value INTO #var FROM tabSingles WHERE doctype = 'Bank Reconciliation' AND field = 'bank_account';
INSERT INTO tabSingles (doctype, field, value) VALUES ('Bank Reconciliation', 'account', #var);
DELETE FROM tabSingles WHERE doctype = 'Bank Reconciliation' AND field = 'bank_account';
Can anyone see the problem? Is there some issue with multi-line queries? I have tried the individual lines on the mariadb command line and they appear to work as expected. I've also tried both frappe.db.sql and multisql (thought it meant multiline sql but doesn't). If I comment line 2 out, it also errors on line 3. Sorry to disturb but I've been staring at this for hours and cannot figure it out!
EDIT:
The obvious answer is this, but I'd still like to know why it doesn't like the original query:
UPDATE tabSingles SET field='{new_name}' WHERE doctype='{doctype}' AND field='{old_name}';
For security reasons (mainly SQL injection) MariaDB (and MySQL) servers don't support the execution of multiple SQL statements by default.
For supporting multiple statements execution the client needs to send COM_SET_OPTION command and MYSQL_OPTION_MULTI_STATEMENTS_ON flag to the server, which is not supported by PyMySQL.
Do not try to run more than one statement in a call.
Do use BEGIN and COMMIT.
Do use FOR UPDATE.
You need 5 separate commands:
BEGIN;
SELECT ... FOR UPDATE; -- to keep other connections from messing with the row(s).
UPDATE ...;
DELETE ...
COMMIT; -- do all of the above "atomically"
I had no problem with SELECTing data in python from postgres database using cursor/execute. Just changed the sql to INSERT a row but nothing is inserted to DB. Can anyone let me know what should be modified? A little confused because everything is the same except for the sql statement.
<!-- language: python -->
#app.route("/addcontact")
def addcontact():
# this connection/cursor setting showed no problem so far
conn = pg.connect(conn_str)
cur = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
sql = f"INSERT INTO jna (sid, phone, email) VALUES ('123','123','123')"
cur.execute(sql)
return redirect("/contacts")
first look at your table setup and make sure your variables are named right in the right order, format and all that, if your not logging into the specific database on the sql server it won't know where the table is, you might need to send something like 'USE databasename' before you do your insert statement so your computer is in the right place in the server.
I might not be up to date with the language but is that 'f' supposed to be right before the quotes? if thats in ur code that'd probably throw an error unless it has a use im not aware of or its not relevant to the problem.
You have to commit your transaction by adding the line below after execute(sql)
conn.commit()
Ref: Using INSERT with a PostgreSQL Database using Python
I'm trying to enter data from python 3.4 into a MySQL database, with both of these entities being within pythonAnywhere. In other words, I'm writing a python 3.4 program in pythonAnywhere and need to connect to a MySQL db also within pythonAnywhere. I've checked other answers here and haven't quite figured it out. I've tried the ( -u -h -p) syntax mentioned by a few posts, but I'm not sure if that is just for gaining access from outside of pythonAnywhere.
Any help would be appreciated.
++++++++++++++++++++
Actually I figured it out (with kudos to Tony Darnell at the Scripting MySQL website, from whom I plagiarized most of this:
import MySQLdb
db=MySQLdb.connect(
host='Your_User_Name.mysql.pythonanywhere-services.com',
user='Your_User_Name',
passwd='Your_pythonAnywhere_password',
db='Your_User_Name$Your_Data_Base')
everything above starting with 'Your' refers to you personal account info with pythonanywhere
everything else gets listed exactly as shown. Watch that $ that follows your user name as part of the database name (db = etc.)
cursor = db.cursor ()
execute the SQL query using execute() method.
cursor.execute ("Enter any MySQL query here. use the quotes. no semi-colon")
fetch a single row from query using fetchone() method.
row = cursor.fetchone ()
print(row)
fetch all the rest of the query using fetchall() method
data = cursor.fetchall()
print(data)
close the cursor object
cursor.close ()
close the connection
db.close ()
I'm working on a project that requires me to programmatically create MySQL users from a django app. I can create the users just fine:
from django.db import connection, transaction
cursor = connection.cursor()
cursor.execute("CREATE USER %s#'%'", 'username')
cursor.execute("SET PASSWORD FOR %s#'%' = PASSWORD(%s)", ('username', 'pass'))
That works perfectly. The problem is when I try to grant permissions. The database name is also determined programmatically:
cursor.execute("GRANT SELECT ON %s.* TO %s#'%'", ('dbname', 'username'))
This results in a mysql error because when it does the string substitution, it places single quotes around the database name, which is syntactically incorrect:
DatabaseError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''dbname'.* to 'username'#'%'' at line 1")
How do I prevent the single quotes from being added around the %s for database name? I know that I could simply do the string substitution in Python and fix this, but that could potentially cause a SQL injection vulnerability.
Sometimes placeholders won't work (as you've found out), so you'll have to use string concatenation. Be careful - validate the string, make sure it's only composed of the characters you expect (don't just look for characters you don't expect), and you should be OK. Also get another developer to check your code, and comment it to make sure no-one else thinks you ought to be using placeholders.