Im trying to run a python script on my raspberry pi, in order to store the data that I receive from the gps receiver into a sql table. While Im executing this script I'm getting an error on this part of the code:
sql = "INSERT INTO gps (n_lat, w_long, date_time) VALUES (%s, %s, %s)" % (north, west, t,)
print sql
cur.execute(sql)
print "Rows inserted: %s" % cur.rowcount
con.commit()
time.sleep(0.5)
Error:
Traceback (most recent call last):
File "gps.py", line 48, in <module>
cur.execute(sql)
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ':01:16)' at line 1")
I don't really understand where is the problem, Have you got any ideas why whiz error occurs ?
You did not escape your input values. For numbers this is optional, but datetime might not be a number.
However, you should always escape input values for your database. The keyword here is prepared statements. Instead of parsing your input arguments into the string with pythons % operater, you should use the argument list of cursor.execute.
sql = "INSERT INTO gps (n_lat, w_long, date_time) VALUES (%s, %s, %s)"
print sql
cur.execute(sql, (north, west, t,))
print "Rows inserted: %s" % cur.rowcount
con.commit()
time.sleep(0.5)
Now the function execute will make sure, that all special characters are escaped. E.g. one of your input values might contain a single quote or similar. Using python's string parsing, this would result in something like:
"INSERT INTO gps (n_lat, w_long, date_time) VALUES ('123', '123', '1234'321')"
In best case this would result in a database error, in worst case somebody could manipulate your database with his own SQL statements (so called SQL injection).
The error in the SQL statement you have set values inside a quote:
VALUES ('%s', '%s', '%s')
Related
When I run the below code with psycopg2:
cur.execute(
"""INSERT INTO logmsg (msg_type, file, msg) VALUES %s;""",
["Error", str(file), str(sys.exc_info()[0])])
I get the following error:
TypeError: not all arguments converted during string formatting
Can someone help me with this?
VALUES needs a list of values enclosed in brackets:
cur.execute(
"""INSERT INTO logmsg (msg_type, file, msg) VALUES (%s, %s, %s);""",
["Error", str(file), str(sys.exc_info()[0])])
Do not forget to commit the transaction.
I have an update statement in my database that is not executing. As far as I am aware, it is syntactically correct. I used this app to verify the syntax. The except block is all that is being executed and I do not understand why.
Here is the code:
for res_posts in list_of_response_ids:
temp_str = res_posts[0] # first element of res_posts tuple is string
temp_str += ":" + str(output[0])
try:
sql = "UPDATE POST SET res_post_id = %s WHERE post_id = %d;" % (str(temp_str), int(res_posts[1]))
cursor.execute(sql)
except:
print "uh oh"
I can post more code if this is not enough information.
EDIT: Following Jacob's advice, I used raise and got the following error:
Traceback (most recent call last):
File "post.cgi", line 93, in <module>
cursor.execute(sql)
File "/usr/lib64/python2.6/site-packages/MySQLdb/cursors.py", line 173, in execute
self.errorhandler(self, exc, value)
File "/usr/lib64/python2.6/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ':37 WHERE post_id = 8' at line 1")
Thank you so much!
Based on your traceback, there is something wrong with the type of entry you are using for the res_post_id or post_id. Currently you are not passing a representation of the res_post_id string, but a literal string. If res_post_id is a string in your DB Schema, I would recommend using %r like so:
sql = "UPDATE POST SET res_post_id = %r WHERE post_id = %d;" % (str(temp_str), int(res_posts[1]))
This will properly quote your res_post_id value for insertion.
So your statement should change from this:
UPDATE POST SET res_post_id = :37 WHERE post_id = 8;
...to this:
UPDATE POST SET res_post_id = ':37' WHERE post_id = 8;
With this code I tried to delete a table if it exists. But I need to do it via passing
a variables.
import MySQLdb as mdb
conn = mdb.connect(host='db01.myhost.co.nl,
user='pdbois',
passwd='triplex',
db='myxxx')
cursor = conn.cursor()
# Without passing variables this works OK!
#cursor.execute("""drop table if exists testtable""")
# But this break
table_name = "testtable"
cursor.execute("""drop table if exists %s""",(table_name))
conn.close()
But why the way I do it above breaks by giving this error?
File "test_mysql.py", line 63, in <module>
main()
File "test_mysql.py", line 59, in main
create_table()
File "test_mysql.py", line 25, in create_table
cursor.execute("""drop table if exists %s""",(table_name))
File "build/bdist.linux-x86_64/egg/MySQLdb/cursors.py", line 174, in execute
File "build/bdist.linux-x86_64/egg/MySQLdb/connections.py", line 36, in defaulterrorhandler
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''testtable'' at line 1")
What's the right way to do it?
Update:
Another problem is to create a table via parameter.
sql = "create table %s(
first_name char(20) not null,
last_name char(20))" % mdb.escape_string(table_name)
cursor.execute(sql)
It gives `SyntaxError: EOL while scanning string literal`.
You cannot parameterize the table name, use string formatting and escape the value manually:
cursor.execute("drop table if exists %s" % mdb.escape_string(table_name))
I have a little problem with Connector/python executing insert query.
I have function to insert data into database. Argument data is list of tuples:
[(2652884, 'http://www.example.com/', '5.0.5.239', 1), ..... ]
def insert_url(self, data):
query = "INSERT INTO `sp_urls` (`parent_id`, `url`, `version`, `hits`) VALUES (%d, %s, %s, %d) ON DUPLICATE KEY UPDATE url=url"
try:
cursor = self.__cnx.cursor()
cursor.executemany(query, data)
except (mysql.connector.errors.IntegrityError) as err:
print("Query syntax error:", err, file=sys.stderr)
except (mysql.connector.errors.ProgrammingError) as err:
print("Programming err:{0}".format(err))
finally:
cursor.close()
The query itself works, in self.__cnx is initialized connection.
Here's the traceback:
File "sdi/database/DbValidator.py", line 91, in process_records
self.__driver.insert_url(urldata)
File "/home/david/workspace/stickydi/sdi/database/MySQLDriver.py", line 87, in insert_url
cursor.executemany(query, data)
File "/usr/lib/python3/dist-packages/mysql/connector/cursor.py", line 492, in executemany
return self._batch_insert(operation,seq_params)
File "/usr/lib/python3/dist-packages/mysql/connector/cursor.py", line 428, in _batch_insert
fmt = m.group(1).encode(self._connection.charset)
AttributeError: 'NoneType' object has no attribute 'group'
I have one very similar method, and it works OK, I just can't see, why does executemany( ) goes wrong.
Use only %s as the parameter marker in query. Do not use %d:
query = """
INSERT INTO `sp_urls` (`parent_id`, `url`, `version`, `hits`)
VALUES (%s, %s, %s, %s) ON DUPLICATE KEY UPDATE url=url"""
The %s is the format paramstyle defined in the DB-API. It does not have the same meaning as %s in string formatting.
The correct paramstyle to use depends on the database driver. MySQLdb uses %s. Other database drivers such as oursql and sqlite3 use ?.
You should not use %d for SQL parameters. Stick to %s and let the MySQL connector handle the types:
query = """\
INSERT INTO `sp_urls` (`parent_id`, `url`, `version`, `hits`)
VALUES (%s, %s, %s, %s)
ON DUPLICATE KEY UPDATE url=url
"""
Quoting from the Python-MySQL documentation:
paramstyle
String constant stating the type of parameter marker formatting expected by the interface. Set to 'format' = ANSI C printf format codes, e.g. '...WHERE name=%s'. If a mapping object is used for conn.execute(), then the interface actually uses 'pyformat' = Python extended format codes, e.g. '...WHERE name=%(name)s'. However, the API does not presently allow the specification of more than one style in paramstyle.
Granted, using %s for SQL parameters is confusingly similar to Python string formatting, but it is not the same.
I'm experiencing a problem inserting values into a SQLite database. The data I download from the Norwegian Parliament site data.stortinget.no. The error I get is: sqlite3.OperationalError: unrecognized token: "01T00"
Here is the method in which the error occur: (I know about the indentation error in this excerpt)
def get_perioder(cur):
DOK = "stortingsperioder"
try:
page = urllib2.urlopen(SITE+DOK)
except:
print "Failed to fetch item "+DOK
if page:
tree = ElementTree.parse(page)
root = tree.getroot()
top = list(root)[2]
elements = list(top)
for el in elements:
fra = el.find('{http://data.stortinget.no}fra').text
per_id = el.find('{http://data.stortinget.no}id').text
til = el.find('{http://data.stortinget.no}til').text
print "id: %s fra: %s til: %s" % (per_id, fra, til)
cur.execute("INSERT INTO perioder(fra, id, til) VALUES(%s,%s,%s)" % (fra, per_id, til))
else:
print "Could not load page: "+DOK
The message printed by the print just above cur.execute is:
id: 2009-2013 fra: 2009-10-01T00:00:00 til: 2013-09-30T23:59:59
The whole error trace is:
BigMac:Stortingsdata ola$ python getBasicData.py
id: 2009-2013 fra: 2009-10-01T00:00:00 til: 2013-09-30T23:59:59
Traceback (most recent call last):
File "getBasicData.py", line 169, in <module>
get_perioder(cur)
File "getBasicData.py", line 26, in get_perioder
cur.execute("INSERT INTO perioder(fra, id, til) VALUES(%s,%s,%s)" % (fra, per_id, til))
sqlite3.OperationalError: unrecognized token: "01T00"
I referred with the SQLite manual and it seems that the format is supported, so I'm wondering where the problem come from.
The proper way is to use a parametrized query.
Example:
cur.execute("""INSERT INTO perioder(fra, id, til)
VALUES (?,?,?);""", (fra, per_id, til))
There is a specific parameter "style" for each database driver.
In the case of SQLite that parameter style is ?.
Also note that the parameter values are passed as a second argument to execute().
Using string-interpolation leaves you vulnerable to all kinds of quoting issues (like the one that brought you here) and the possibility of SQL-injection attack.
For more information please read the DB-API and the database programming wiki.
If you want to store the date stamps as strings (TEXT) in SQLite, I recommend you format the text you would like to execute as follows:
cur.execute("""INSERT INTO perioder(fra, id, til)
VALUES (\"%s\",\"%s\",\"%s\")""" % (fra, per_id, til))
SQLite returns errors if the values you insert does not have inverted commas. Formatting your text with \"%s\" instead of %s will insert the string value with inverted commas in your formatted string:
"INSERT INTO perioder(fra, id, til)
VALUES ("2009-2013", "2009-10-01T00:00:00","2013-09-30T23:59:59")"