We Keep Coding

SQL statement failing to execute in Python CGI

I have an update statement in my database that is not executing. As far as I am aware, it is syntactically correct. I used this app to verify the syntax. The except block is all that is being executed and I do not understand why.
Here is the code:
for res_posts in list_of_response_ids:
temp_str = res_posts[0] # first element of res_posts tuple is string
temp_str += ":" + str(output[0])
try:
sql = "UPDATE POST SET res_post_id = %s WHERE post_id = %d;" % (str(temp_str), int(res_posts[1]))
cursor.execute(sql)
except:
print "uh oh"
I can post more code if this is not enough information.
EDIT: Following Jacob's advice, I used raise and got the following error:
Traceback (most recent call last):
File "post.cgi", line 93, in <module>
cursor.execute(sql)
File "/usr/lib64/python2.6/site-packages/MySQLdb/cursors.py", line 173, in execute
self.errorhandler(self, exc, value)
File "/usr/lib64/python2.6/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ':37 WHERE post_id = 8' at line 1")
Thank you so much!

Based on your traceback, there is something wrong with the type of entry you are using for the res_post_id or post_id. Currently you are not passing a representation of the res_post_id string, but a literal string. If res_post_id is a string in your DB Schema, I would recommend using %r like so:
sql = "UPDATE POST SET res_post_id = %r WHERE post_id = %d;" % (str(temp_str), int(res_posts[1]))
This will properly quote your res_post_id value for insertion.
So your statement should change from this:
UPDATE POST SET res_post_id = :37 WHERE post_id = 8;
...to this:
UPDATE POST SET res_post_id = ':37' WHERE post_id = 8;

Python SQL syntax error

Im trying to run a python script on my raspberry pi, in order to store the data that I receive from the gps receiver into a sql table. While Im executing this script I'm getting an error on this part of the code:
sql = "INSERT INTO gps (n_lat, w_long, date_time) VALUES (%s, %s, %s)" % (north, west, t,)
print sql
cur.execute(sql)
print "Rows inserted: %s" % cur.rowcount
con.commit()
time.sleep(0.5)
Error:
Traceback (most recent call last):
File "gps.py", line 48, in <module>
cur.execute(sql)
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ':01:16)' at line 1")
I don't really understand where is the problem, Have you got any ideas why whiz error occurs ?

You did not escape your input values. For numbers this is optional, but datetime might not be a number.
However, you should always escape input values for your database. The keyword here is prepared statements. Instead of parsing your input arguments into the string with pythons % operater, you should use the argument list of cursor.execute.
sql = "INSERT INTO gps (n_lat, w_long, date_time) VALUES (%s, %s, %s)"
print sql
cur.execute(sql, (north, west, t,))
print "Rows inserted: %s" % cur.rowcount
con.commit()
time.sleep(0.5)
Now the function execute will make sure, that all special characters are escaped. E.g. one of your input values might contain a single quote or similar. Using python's string parsing, this would result in something like:
"INSERT INTO gps (n_lat, w_long, date_time) VALUES ('123', '123', '1234'321')"
In best case this would result in a database error, in worst case somebody could manipulate your database with his own SQL statements (so called SQL injection).

The error in the SQL statement you have set values inside a quote:
VALUES ('%s', '%s', '%s')

python sqlite3 cursor.execute() with parameters leads to syntax error near ? (paramstyle qmark)

after searching untill madness, i decided to post a question here.
I try to create a sqlite3 database where i'd like to make use of the secure variable substituation function of the cursor.execute(SQL, param) function. My function goes like this:
#!/usr/bin/env python3
#-*- coding: utf-8 -*-
import sqlite3
def create():
values = ("data")
sql = "CREATE TABLE IF NOT EXISTS ? ( name TEXT, street TEXT, time REAL, age INTEGER )"
con = sqlite3.connect("database.db")
c = con.cursor()
c.execute(sql, values)
con.commit()
c.close()
con.close()
if __name__ = "__main__":
create()
I know that the first argument should be the sql command in form of a string and the second argument must be a tuple of the values which are supposed to be substituted where the ? is in the sql string.
However, when i run the file it returns the following error:
$ ./test.py
Traceback (most recent call last):
File "./test.py", line 21, in <module>
create()
File "./test.py", line 14, in create
c.execute(sql, values)
sqlite3.OperationalError: near "?": syntax error
This also happens when paramstyle is set to named (e.g. the :table form).
I can't spot a syntax error here, so i think that the problem must be caused somewhere in the system. I tested it on an Archlinux and Debian install, both post me the same error.
Now it is up yo you, as I have no idea anymore where to look for the cause.

SQL parameters can only apply to insert data, not table names. That means parameters are not even parsed for DDL statements.
For that you'll have to use string formatting:
sql = "CREATE TABLE IF NOT EXISTS {} ( name TEXT, street TEXT, time REAL, age INTEGER )".format(*values)

As I understand, your parameter is the table name?
so your command would be
tbl = 'my_table'
sql = "CREATE TABLE IF NOT EXISTS '%s' ( name TEXT, street TEXT, time REAL, age INTEGER )" % tbl

Connector/python executemany NoneType

I have a little problem with Connector/python executing insert query.
I have function to insert data into database. Argument data is list of tuples:
[(2652884, 'http://www.example.com/', '5.0.5.239', 1), ..... ]
def insert_url(self, data):
query = "INSERT INTO `sp_urls` (`parent_id`, `url`, `version`, `hits`) VALUES (%d, %s, %s, %d) ON DUPLICATE KEY UPDATE url=url"
try:
cursor = self.__cnx.cursor()
cursor.executemany(query, data)
except (mysql.connector.errors.IntegrityError) as err:
print("Query syntax error:", err, file=sys.stderr)
except (mysql.connector.errors.ProgrammingError) as err:
print("Programming err:{0}".format(err))
finally:
cursor.close()
The query itself works, in self.__cnx is initialized connection.
Here's the traceback:
File "sdi/database/DbValidator.py", line 91, in process_records
self.__driver.insert_url(urldata)
File "/home/david/workspace/stickydi/sdi/database/MySQLDriver.py", line 87, in insert_url
cursor.executemany(query, data)
File "/usr/lib/python3/dist-packages/mysql/connector/cursor.py", line 492, in executemany
return self._batch_insert(operation,seq_params)
File "/usr/lib/python3/dist-packages/mysql/connector/cursor.py", line 428, in _batch_insert
fmt = m.group(1).encode(self._connection.charset)
AttributeError: 'NoneType' object has no attribute 'group'
I have one very similar method, and it works OK, I just can't see, why does executemany( ) goes wrong.

Use only %s as the parameter marker in query. Do not use %d:
query = """
INSERT INTO `sp_urls` (`parent_id`, `url`, `version`, `hits`)
VALUES (%s, %s, %s, %s) ON DUPLICATE KEY UPDATE url=url"""
The %s is the format paramstyle defined in the DB-API. It does not have the same meaning as %s in string formatting.
The correct paramstyle to use depends on the database driver. MySQLdb uses %s. Other database drivers such as oursql and sqlite3 use ?.

You should not use %d for SQL parameters. Stick to %s and let the MySQL connector handle the types:
query = """\
INSERT INTO `sp_urls` (`parent_id`, `url`, `version`, `hits`)
VALUES (%s, %s, %s, %s)
ON DUPLICATE KEY UPDATE url=url
"""
Quoting from the Python-MySQL documentation:
paramstyle
String constant stating the type of parameter marker formatting expected by the interface. Set to 'format' = ANSI C printf format codes, e.g. '...WHERE name=%s'. If a mapping object is used for conn.execute(), then the interface actually uses 'pyformat' = Python extended format codes, e.g. '...WHERE name=%(name)s'. However, the API does not presently allow the specification of more than one style in paramstyle.
Granted, using %s for SQL parameters is confusingly similar to Python string formatting, but it is not the same.

pass table name as parameter in postgres via python

I want to execute postgres query in python.The table name has to be passed as a parameter.Since the table will be created at run time. I have used dict query param style.But i am getting an error.
import psycopg2
CONNECTION_STRING = "dbname='autogist' user='postgres' password=''"
query = "INSERT INTO %(table)s " +\
"(vin_id, vin_details_id, price, mileage, dealer_id, created_on, modified_on) " +\
"VALUES (%(vin_id)s, %(vlookup_id)s, %(price)s, %(mileage)s, %(dealer_id)s,now(),now()) " +\
"RETURNING id"
params = {"table" : "dealer_vehicle_details_2010_01_02",\
"vin_id":"3",\
"vlookup_id":"403",\
"price":"403",\
"mileage":"403",\
"dealer_id":"276092"
}
conn=psycopg2.connect(CONNECTION_STRING)
cursor = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
cursor.execute(query,params)
TRACEBACK:
ERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (262, 0))
---------------------------------------------------------------------------
ProgrammingError Traceback (most recent call last)
/home/gridlex/workspace/<ipython console> in <module>()
/usr/local/lib/python2.6/dist-packages/psycopg2/extras.pyc in execute(self, query, vars)
121 self.index = {}
122 self._query_executed = 1
--> 123 return _cursor.execute(self, query, vars)
124
125 def callproc(self, procname, vars=None):
ProgrammingError: syntax error at or near "E'dealer_vehicle_details_2010_01_02'"
LINE 1: INSERT INTO E'dealer_vehicle_details_2010_01_02' (vin_id, vi...

The statement you send must be syntactically valid when PREPAREd, which a statement with placeholders for table names is not. You can't use placeholders for table names in prepared statements.
Your options are:
Substitute the table name in with regular string substitution, "double quoted". Be very careful with your quoting routine; make sure it doubles any quotes within the table name its self, so the table name double"quote becomes "double""quote". Eg. 'SELECT * FROM "%s"' % quote_ident(tablename). You'd have to roll your own quote_ident as AFAIK psycopg2 doesn't expose a function like that.
Send the table name as a query parameter to a PL/PgSQL function that uses EXECUTE ... USING to create a dynamic SQL statement using the table name. PL/PgSQL can use the quote_ident function to provide safer quoting than a home-rolled implementation.