Though it is a repeated question , but want to know where my code is wrong as I am facing a syntax error .
def update_block():
table_name = input("Enter the name of the table: ")
column_update = input("Enter the column name to be updated: ")
column_name = input("Enter the column where the operation is to be performed: ")
name = input("Enter the name has to get update: ")
column_value = input("Enter the column value: ")
try:
sql_update_query = f"""Update {table_name} set {column_update} = %s where {column_name} = %s"""
inputData = (f"{name},{column_value}" )
my_cursor.execute(sql_update_query,inputData)
mydb.commit()
print("Record Updated successfully ")
except mysql.connector.Error as error:
print("Failed to update record to database: {}".format(error))
finally:
if (mydb.is_connected()):
my_cursor.close()
mydb.close()
print("MySQL connection is closed")
update_block()
error i am getting as :
Failed to update record to database: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds
to your MySQL server version for the right syntax to use near '%s where prj_id = %s' at line 1
MySQL connection is closed
f"""Update {table_name} set {column_update} = %s where {column_name} = %s"""
You used f strings with curly brackets and %s notation here. Do either one and it should work
e.g.:
f"""Update {table_name} set {column_update} = {name} where {column_name} = {column_value}"""
There are two problems with the code.
In this line,
sql_update_query = f"""Update {table_name} set {column_update} = %s where {column_name} = %s"""
the table and column names should be quoted with backticks ("`") to handle names containing spaces or hyphens (or some unicode characters). So the line would look like this.
sql_update_query = f"""Update `{table_name}` set `{column_update}` = %s where `{column_name}` = %s"""
Note that the placeholders for variables should remain as %s.
In this line
inputData = (f"{name},{column_value}" )
the variables' values are being converted to strings within a single string. But the statement expects two variables, not one. Also, it is better* to pass the raw variables to the database connection and let the connection manage formatting them correctly in the final query. So the line should be
inputData = (name, column_value)
And now the statement can be executed with the related variables
my_cursor.execute(sql_update_query, inputData)
* The driver knows how to correctly convert Python data types into those expected by the database, and how to escape and quote these values. This provides at least two benefits:
it helps prevent SQL injection attacks, where a malicious user provides an SQL statement as a variable value (such as "; DELETE FROM mytable;"
it ensures that values are processed as expected; consider this statement:
SELECT '2020-09-01' AS `Quoted Date`, 2020-09-01 AS `Unquoted Date`;
+-------------+---------------+
| Quoted Date | Unquoted Date |
+-------------+---------------+
| 2020-09-01 | 2010 |
+-------------+---------------+
I have the following script:
create_table_WAC = """
create table if not exists %s (
w_geocode text,
C000 text,
CFS04 text,
CFS05 text,
createdate text
)
"""
target_directory = Path(sys.argv[1]).resolve()
for file in target_directory.rglob('*.csv'):
table_name = 'opendata_uscensus_lodes_' + str(file.stem)
print(table_name)
# MAKE SURE THIS IS THE RIGHT TABLE FOR THE FILES
cur.execute(create_table_WAC, (table_name,))
with open(file,'r') as file_in:
# MAKE SURE THIS HAS THE RIGHT TABLE NAME IN THE COPY STATEMENT
cur.copy_expert("copy %s from stdin with csv header delimiter ','", table_name, file_in)
conn.commit()
conn.close()
When I run it, it throws this error related to the CREATE TABLE command. I don't understand why there are '' added -- and how do I remove them?
Here is the error:
psycopg2.ProgrammingError: syntax error at or near "'opendata_uscensus_lodes_ca_wac_SA02_JT03_2003'"
LINE 2: create table if not exists 'opendata_uscensus_lodes_ca_wac_S...
Use SQL string composition:
from psycopg2 import sql
create_table_WAC = """
create table if not exists {} ( -- note changed placeholder
w_geocode text,
C000 text,
CFS04 text,
CFS05 text,
createdate text
)
"""
# ...
cur.execute(sql.SQL(create_table_WAC).format(sql.Identifier(table_name)))
Read the comprehensive explanation in the documentation.
I am trying to pattern match with the LIKE LOWER('% %') command however I think the fact that I am using a python variable with %s is mucking it up. I can't seem to find any escape characters for the percentage symbol and my program gives me no errors. Is this the problem or is there something else I'm missing. It does work if I just run LIKE %s however I need to be able to search like not equals.
# Ask for the database connection, and get the cursor set up
conn = database_connect()
if(conn is None):
return ERROR_CODE
cur = conn.cursor()
print("search_term: ", search_term)
try:
# Select the bays that match (or are similar) to the search term
sql = """SELECT fp.name AS "Name", fp.size AS "Size", COUNT(*) AS "Number of Fish"
FROM FishPond fp JOIN Fish f ON (fp.pondID = f.livesAt)
WHERE LOWER(fp.name) LIKE LOWER('%%s%') OR LOWER(fp.size) LIKE LOWER('%%s%')
GROUP BY fp.name, fp.size"""
cur.execute(sql, (search_term, ))
rows = cur.fetchall()
cur.close() # Close the cursor
conn.close() # Close the connection to the db
return rows
except:
# If there were any errors, return a NULL row printing an error to the debug
print("Error with Database - Unable to search pond")
cur.close() # Close the cursor
conn.close() # Close the connection to the db
return None
Instead of embedding the ampersands in the query string, you could wrap the search term string in ampersands, and then pass that to cursor.execute():
sql = 'SELECT * from FishPond fp WHERE LOWER(fp.name) LIKE LOWER(%s)'
search_term = 'xyz'
like_pattern = '%{}%'.format(search_term)
cur.execute(sql, (like_pattern,))
The query is simplified for the purpose of example.
This is more flexible because the calling code can pass any valid LIKE pattern to the query.
BTW: In Postgresql you can use ILIKE for case insensitive pattern matching, so the example query could be written as this:
sql = 'SELECT * from FishPond fp WHERE fp.name ILIKE %s'
As noted in the documentation ILIKE is a Postgresql extension, not standard SQL.
You can escape % with another %
>>> test = 'test'
>>> a = 'LIKE %%s%'
>>> a % test
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: incomplete format
>>>
>>> a = 'LIKE %%%s%%'
>>> a % test
'LIKE %test%'
P.S. you also have two placeholders, but you are passing only one argument in execute
I have a JSON object in Python. I am Using Python DB-API and SimpleJson. I am trying to insert the json into a MySQL table.
At moment am getting errors and I believe it is due to the single quotes '' in the JSON Objects.
How can I insert my JSON Object into MySQL using Python?
Here is the error message I get:
error: uncaptured python exception, closing channel
<twitstream.twitasync.TwitterStreamPOST connected at
0x7ff68f91d7e8> (<class '_mysql_exceptions.ProgrammingError'>:
(1064, "You have an error in your SQL syntax; check the
manual that corresponds to your MySQL server version for
the right syntax to use near ''favorited': '0',
'in_reply_to_user_id': '52063869', 'contributors':
'NULL', 'tr' at line 1")
[/usr/lib/python2.5/asyncore.py|read|68]
[/usr/lib/python2.5/asyncore.py|handle_read_event|390]
[/usr/lib/python2.5/asynchat.py|handle_read|137]
[/usr/lib/python2.5/site-packages/twitstream-0.1-py2.5.egg/
twitstream/twitasync.py|found_terminator|55] [twitter.py|callback|26]
[build/bdist.linux-x86_64/egg/MySQLdb/cursors.py|execute|166]
[build/bdist.linux-x86_64/egg/MySQLdb/connections.py|defaulterrorhandler|35])
Another error for reference
error: uncaptured python exception, closing channel
<twitstream.twitasync.TwitterStreamPOST connected at
0x7feb9d52b7e8> (<class '_mysql_exceptions.ProgrammingError'>:
(1064, "You have an error in your SQL syntax; check the manual
that corresponds to your MySQL server version for the right
syntax to use near 'RT #tweetmeme The Best BlackBerry Pearl
Cell Phone Covers http://bit.ly/9WtwUO''' at line 1")
[/usr/lib/python2.5/asyncore.py|read|68]
[/usr/lib/python2.5/asyncore.py|handle_read_event|390]
[/usr/lib/python2.5/asynchat.py|handle_read|137]
[/usr/lib/python2.5/site-packages/twitstream-0.1-
py2.5.egg/twitstream/twitasync.py|found_terminator|55]
[twitter.py|callback|28] [build/bdist.linux-
x86_64/egg/MySQLdb/cursors.py|execute|166] [build/bdist.linux-
x86_64/egg/MySQLdb/connections.py|defaulterrorhandler|35])
Here is a link to the code that I am using http://pastebin.com/q5QSfYLa
#!/usr/bin/env python
try:
import json as simplejson
except ImportError:
import simplejson
import twitstream
import MySQLdb
USER = ''
PASS = ''
USAGE = """%prog"""
conn = MySQLdb.connect(host = "",
user = "",
passwd = "",
db = "")
# Define a function/callable to be called on every status:
def callback(status):
twitdb = conn.cursor ()
twitdb.execute ("INSERT INTO tweets_unprocessed (text, created_at, twitter_id, user_id, user_screen_name, json) VALUES (%s,%s,%s,%s,%s,%s)",(status.get('text'), status.get('created_at'), status.get('id'), status.get('user', {}).get('id'), status.get('user', {}).get('screen_name'), status))
# print status
#print "%s:\t%s\n" % (status.get('user', {}).get('screen_name'), status.get('text'))
if __name__ == '__main__':
# Call a specific API method from the twitstream module:
# stream = twitstream.spritzer(USER, PASS, callback)
twitstream.parser.usage = USAGE
(options, args) = twitstream.parser.parse_args()
if len(args) < 1:
args = ['Blackberry']
stream = twitstream.track(USER, PASS, callback, args, options.debug, engine=options.engine)
# Loop forever on the streaming call:
stream.run()
use json.dumps(json_value) to convert your json object(python object) in a json string that you can insert in a text field in mysql
http://docs.python.org/library/json.html
To expand on the other answers:
Basically you need make sure of two things:
That you have room for the full amount of data that you want to insert in the field that you are trying to place it. Different database field types can fit different amounts of data.
See: MySQL String Datatypes. You probably want the "TEXT" or "BLOB" types.
That you are safely passing the data to database. Some ways of passing data can cause the database to "look" at the data and it will get confused if the data looks like SQL. It's also a security risk. See: SQL Injection
The solution for #1 is to check that the database is designed with correct field type.
The solution for #2 is use parameterized (bound) queries. For instance, instead of:
# Simple, but naive, method.
# Notice that you are passing in 1 large argument to db.execute()
db.execute("INSERT INTO json_col VALUES (" + json_value + ")")
Better, use:
# Correct method. Uses parameter/bind variables.
# Notice that you are passing in 2 arguments to db.execute()
db.execute("INSERT INTO json_col VALUES %s", json_value)
Hope this helps. If so, let me know. :-)
If you are still having a problem, then we will need to examine your syntax more closely.
The most straightforward way to insert a python map into a MySQL JSON field...
python_map = { "foo": "bar", [ "baz", "biz" ] }
sql = "INSERT INTO your_table (json_column_name) VALUES (%s)"
cursor.execute( sql, (json.dumps(python_map),) )
You should be able to insert intyo a text or blob column easily
db.execute("INSERT INTO json_col VALUES %s", json_value)
You need to get a look at the actual SQL string, try something like this:
sqlstr = "INSERT INTO tweets_unprocessed (text, created_at, twitter_id, user_id, user_screen_name, json) VALUES (%s,%s,%s,%s,%s,%s)", (status.get('text'), status.get('created_at'), status.get('id'), status.get('user', {}).get('id'), status.get('user', {}).get('screen_name'), status)
print "about to execute(%s)" % sqlstr
twitdb.execute(sqlstr)
I imagine you are going to find some stray quotes, brackets or parenthesis in there.
#route('/shoes', method='POST')
def createorder():
cursor = db.cursor()
data = request.json
p_id = request.json['product_id']
p_desc = request.json['product_desc']
color = request.json['color']
price = request.json['price']
p_name = request.json['product_name']
q = request.json['quantity']
createDate = datetime.now().isoformat()
print (createDate)
response.content_type = 'application/json'
print(data)
if not data:
abort(400, 'No data received')
sql = "insert into productshoes (product_id, product_desc, color, price, product_name, quantity, createDate) values ('%s', '%s','%s','%d','%s','%d', '%s')" %(p_id, p_desc, color, price, p_name, q, createDate)
print (sql)
try:
# Execute dml and commit changes
cursor.execute(sql,data)
db.commit()
cursor.close()
except:
# Rollback changes
db.rollback()
return dumps(("OK"),default=json_util.default)
One example, how add a JSON file into MySQL using Python. This means that it is necessary to convert the JSON file to sql insert, if there are several JSON objects then it is better to have only one call INSERT than multiple calls, ie for each object to call the function INSERT INTO.
# import Python's JSON lib
import json
# use JSON loads to create a list of records
test_json = json.loads('''
[
{
"COL_ID": "id1",
"COL_INT_VAULE": 7,
"COL_BOOL_VALUE": true,
"COL_FLOAT_VALUE": 3.14159,
"COL_STRING_VAULE": "stackoverflow answer"
},
{
"COL_ID": "id2",
"COL_INT_VAULE": 10,
"COL_BOOL_VALUE": false,
"COL_FLOAT_VALUE": 2.71828,
"COL_STRING_VAULE": "http://stackoverflow.com/"
},
{
"COL_ID": "id3",
"COL_INT_VAULE": 2020,
"COL_BOOL_VALUE": true,
"COL_FLOAT_VALUE": 1.41421,
"COL_STRING_VAULE": "GIRL: Do you drink? PROGRAMMER: No. GIRL: Have Girlfriend? PROGRAMMER: No. GIRL: Then how do you enjoy life? PROGRAMMER: I am Programmer"
}
]
''')
# create a nested list of the records' values
values = [list(x.values()) for x in test_json]
# print(values)
# get the column names
columns = [list(x.keys()) for x in test_json][0]
# value string for the SQL string
values_str = ""
# enumerate over the records' values
for i, record in enumerate(values):
# declare empty list for values
val_list = []
# append each value to a new list of values
for v, val in enumerate(record):
if type(val) == str:
val = "'{}'".format(val.replace("'", "''"))
val_list += [ str(val) ]
# put parenthesis around each record string
values_str += "(" + ', '.join( val_list ) + "),\n"
# remove the last comma and end SQL with a semicolon
values_str = values_str[:-2] + ";"
# concatenate the SQL string
table_name = "json_data"
sql_string = "INSERT INTO %s (%s)\nVALUES\n%s" % (
table_name,
', '.join(columns),
values_str
)
print("\nSQL string:\n\n")
print(sql_string)
output:
SQL string:
INSERT INTO json_data (COL_ID, COL_INT_VAULE, COL_BOOL_VALUE, COL_FLOAT_VALUE, COL_STRING_VAULE)
VALUES
('id1', 7, True, 3.14159, 'stackoverflow answer'),
('id2', 10, False, 2.71828, 'http://stackoverflow.com/'),
('id3', 2020, True, 1.41421, 'GIRL: Do you drink? PROGRAMMER: No. GIRL: Have Girlfriend? PROGRAMMER: No. GIRL: Then how do you enjoy life? PROGRAMMER: I am Programmer.');
The error may be due to an overflow of the size of the field in which you try to insert your json. Without any code, it is hard to help you.
Have you considerate a no-sql database system such as couchdb, which is a document oriented database relying on json format?
Here's a quick tip, if you want to write some inline code, say for a small json value, without import json.
You can escape quotes in SQL by a double quoting, i.e. use '' or "", to enter ' or ".
Sample Python code (not tested):
q = 'INSERT INTO `table`(`db_col`) VALUES ("{k:""some data"";}")'
db_connector.execute(q)