Could anyone explain me is it bug or feature?
Debian Stretch
mariadb-server-10.1.26
mariadb-client-10.1.26
MySQLdb-1.2.5
This python code perfectlly works in Debian Jessie, but failed in Stretch with error:
Traceback (most recent call last):
File "bug_check.py", line 17, in <module>
cur.executemany(q, p)
File "/usr/local/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 255, in executemany
self.errorhandler(self, TypeError, msg)
File "/usr/local/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
TypeError: not all arguments converted during string formatting
Python code:
#!/usr/bin/python
# -*- coding: UTF-8 *
import MySQLdb
db = MySQLdb.connect(host='192.168.1.183', user='root', passwd='password', db='test', charset='utf8')
cur = db.cursor()
q = """INSERT INTO test2 (id, value)
VALUES (%s, %s)
ON DUPLICATE KEY
UPDATE value=%s
"""
p = [(1, 7, 7)]
# failed
cur.executemany(q, p)
# working
for i in p:
cur.execute(q, i)
db.commit()
db.close()
Database:
CREATE TABLE `test2` (
`id` bigint(8) NOT NULL,
`value` float NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
ALTER TABLE `test2`
ADD PRIMARY KEY (`id`);
ALTER TABLE `test2`
MODIFY `id` bigint(8) NOT NULL AUTO_INCREMENT, AUTO_INCREMENT=1;
I am trying mariadb-10.2, pymysql but anyway error occurs in Stretch.
This may work: Change UPDATE value=%s to UPDATE value=VALUES(value) and get rid of the last 7 in the array.
If that does not work, then here is more discussion:
I think executemany is trying to build
INSERT ...
VALUES (...),
(...),
(...);
But it does not know how to convert the IODKU syntax into a repeated list like that. Bottom line: you can probably use executemany with INSERT, INSERT IGNORE, REPLACE, but not IODKU.
For IODKU to work, Stretch needs to be smart enough to do this:
INSERT INTO test2 (id, value)
VALUES
(%s, %s),
(%s, %s),
(%s, %s),
(%s, %s),
etc
ON DUPLICATE KEY
UPDATE value=VALUES(value)
Note that the repetition is in the middle, not on the end, as in the other cases. However you have to use the VALUES() pseudo-function to avoid the %s in the UPDATE clause.
Related
I tried to insert each element of the json api into my postgres table.
But I get the follwoing error:
Traceback (most recent call last):
File "c:/Users/myname/Documents/repos/docker-playground/parse_json_to_postgres.py", line 20, in <module>
cursor.execute(f"INSERT into catfacts(data) VALUES ( {cat_fact} )")
psycopg2.errors.SyntaxError: syntax error at or near "{"
LINE 1: INSERT into catfacts(data) VALUES ( {'status': {'verified':...
^
My postgres table:
CREATE TABLE cat_facts (
id serial NOT NULL PRIMARY KEY,
data jsonb NOT NULL
);
My Python code to insert the data into the table:
import requests, json, psycopg2
cat_facts_json = requests.get('https://cat-fact.herokuapp.com/facts').json
conn = psycopg2.connect(user="postgres",
password="password",
host="localhost",
port="5432",
database="postgres")
cursor = conn.cursor()
for cat_fact in cat_facts_json():
cursor.execute(f"INSERT into catfacts(data) VALUES ( \' {cat_fact} \' )")
API = https://cat-fact.herokuapp.com/facts
What I am trying to achieve:
INSERT INTO cat_facts(data) VALUES ('{"status":{"verified":true,"sentCount":1},"type":"cat","deleted":false,"_id":"58e008800aac31001185ed07","user":"58e007480aac31001185ecef","text":"Wikipedia has a recording of a cat meowing, because why not?","__v":0,"source":"user","updatedAt":"2020-08-23T20:20:01.611Z","createdAt":"2018-03-06T21:20:03.505Z","used":false}');
INSERT INTO cat_facts(data) VALUES ('{"status":{"verified":true,"sentCount":1},"type":"cat","deleted":false,"_id":"58e008630aac31001185ed01","user":"58e007480aac31001185ecef","text":"When cats grimace, they are usually \"taste-scenting.\" They have an extra organ that, with some breathing control, allows the cats to taste-sense the air.","__v":0,"source":"user","updatedAt":"2020-08-23T20:20:01.611Z","createdAt":"2018-02-07T21:20:02.903Z","used":false},{"status":{"verified":true,"sentCount":1},"type":"cat","deleted":false,"_id":"58e00a090aac31001185ed16","user":"58e007480aac31001185ecef","text":"Cats make more than 100 different sounds whereas dogs make around 10.","__v":0,"source":"user","updatedAt":"2020-08-23T20:20:01.611Z","createdAt":"2018-02-11T21:20:03.745Z","used":false}');
....
See here JSON Adaption.
So something like:
from psycopg2.extras import Json
cursor.execute("INSERT into catfacts(data) VALUES (%s)", [Json(cat_fact)])
I got it working now:
for cat_fact in cat_facts_json:
data = json.dumps(cat_fact)
insert_query = "insert into cat_facts (data) values (%s) returning data"
cursor.execute(insert_query, (data,))
conn.commit()
conn.close()
I considered your comments #Stefano Frazzetto and #Adrian Klaver.
json.dumps works !
I didn't execute the parameters directly in the execute query
I still think, this is a pretty odd syntax with the comma after data:
cursor.execute(insert_query, (data,))
I've got a problem when trying to insert a json, which was converted from a python object with json.dumps, into a MySQL database. The connection to the database is working with another python file. I've already tried to just insert values, which was working, but with the json file it's not working.
My Python file:
import json
import dbConnection
cur = dbConnection.cursor
cnx = dbConnection.conn
DEVICES = {
"id": "1",
"isPoweredOn": "True",
"os": "Linux"
}
j = json.dumps(DEVICES)
print(j)
sql = "INSERT INTO DEVICES (id, isPoweredOn, os) VALUES (%s, %s, %s)"
val = (json.dumps(DEVICES))
cur.execute(sql, val)
cnx.commit()
print(cur.rowcount, "record inserted.")
Error code I get, when trying to execute:
"id": "1", "isPoweredOn": "True", "os": "Linux"}
Traceback (most recent call last):
File "dbInit.py", line 22, in <module>
cur.execute(sql, val)
File "/home/silvan/.virtualenvs/pyproj1/lib/python3.8/site-packages/mysql/connector/cursor.py", line 551, in execute
self._handle_result(self._connection.cmd_query(stmt))
File "/home/silvan/.virtualenvs/pyproj1/lib/python3.8/site-packages/mysql/connector/connection.py", line 490, in cmd_query
result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
File "/home/silvan/.virtualenvs/pyproj1/lib/python3.8/site-packages/mysql/connector/connection.py", line 395, in _handle_result
raise errors.get_exception(packet)
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%s, %s, %s)' at line 1
My CREATE TABLE code:
CREATE TABLE DEVICES(id INT AUTO_INCREMENT PRIMARY KEY NOT NULL, isPoweredOn BOOLEAN NOT NULL, os VARCHAR(50) NOT NULL);
Thanks for any help in advance!
You need to json.loads(j) and assign it to a variable, then you can access the values properly.
Try :
import json
import dbConnection
cur = dbConnection.cursor
cnx = dbConnection.conn
DEVICES = {
"id": "1",
"isPoweredOn": False ,
"os": "Linux"
}
j = json.dumps(DEVICES)
values = json.loads(j)
'''
# Quick debugging
print(j , type(j))
print(values , type(values))
print(values['isPoweredOn'])
'''
sql = "INSERT INTO DEVICES (id, isPoweredOn, os) VALUES (%s, %s, %s)"
val = ( '' , values['isPoweredOn'] , values['os'])
cur.execute(sql, val)
cnx.commit()
print(cur.rowcount, "record inserted.")
Also since you defined id to be INT AUTO_INCREMENT PRIMARY KEY NOT NULL it , you can't insert device id wich is values['id'] to id column, you can alter DEVICES table and create a new column called device_id for storing the device id if you need really need to store values['id']
Firstly, cast the DEVICES to dict, then Here's the format.
sql = "INSERT INTO DEVICES (`id`, `isPoweredOn`, `os`) VALUES (%(id)s, %(isPoweredOn)s, %(os)s)"
Then Execute it :
try:
cur.execute(sql, DEVICES)
cnx.commit()
except error:
print(error)
Cheers!!
While using MySQL through python, on using execute() function null values are getting updated in the specific table, but on using executemany() the null values are returning errors.
The working commands:
mycursor = mydb.cursor()
mycursor.execute("INSERT INTO users (name, fav) VALUES('John', null)"
mydb.commit()
The code that doesn't work:
mycursor = mydb.cursor()
sql = "INSERT INTO users (name, fav) VALUES(%s, %s)"
val =[
('Samantha', 154),
('Thalia', 155),
('Jacobs', null),
('Jamie', null)
]
mycursor.executemany(sql, val)
mydb.commit()
The error generated is as follows:
Traceback (most recent call last):
File "demo_python.py", line 14, in
('Jacobs', null),
NameError: name 'null' is not defined
I am learning python and i am new bie.
I am trying to use functions with mysql and python and i ma getting errors
This is my script
import MySQLdb
def insert_values(cursor, values):
#cursor = self.connection.cursor()
cursor.executemany("""
insert into pythontest (name1,name2,name3)
values (%s, %s, %s)""", values)
cursor.close()
db = MySQLdb.connect("localhost","root","root","python" )
cursor = db.cursor()
var1 = ['name1','name2','name3']
insert_values(cursor,var1)
db.close()
There may be many errors because i am learning
1)i don't know how can i pass db
object in function or passing cusrsor
is ok. because i have to call that
function many times in for loop
2)is the syntax of values array ok to
go in database
ERRORS
File "mysql.py", line 10, in insert_values
values (%s, %s, %s)""", values)
File "build/bdist.linux-i686/egg/MySQLdb/cursors.py", line 216, in executemany
File "build/bdist.linux-i686/egg/MySQLdb/connections.py", line 36, in defaulterrorhandler
_mysql_exceptions.ProgrammingError: not enough arguments for format string
cursor.executemany("""
insert into pythontest (name1,name2,name3)
values (%s, %s, %s)""", *values)
Here's how I would write that (But untested):
import MySQLdb
def insert_values(db, values):
cursor = db.cursor()
try:
try:
cursor.execute("""
insert into pythontest (name1,name2,name3)
values (%s, %s, %s)""", *values)
except:
db.rollback()
raise
else:
db.commit()
finally:
cursor.close()
db = MySQLdb.connect("localhost","root","root","python" )
vars = ('name1','name2','name3')
insert_values(db, vars)
db.close()
The cursor starts a transaction, so you don't want to re-use the same cursor for multiple updates unless they are part of an atomic transaction.
I am trying to store some parsed feed contents values in Sqlite database table in python.But facing error.Could anybody help me out of this issue.Infact it is so trivial question to ask!I am newbie!..Anyway thanks in advance!
from sqlite3 import *
import feedparser
data = feedparser.parse("some url")
conn = connect('location.db')
curs = conn.cursor()
curs.execute('''create table location_tr
(id integer primary key, title text ,
updated text)''')
for i in range(len(data['entries'])):
curs.execute("insert into location_tr values\
(NULL, data.entries[i].title,data.feed.updated)")
conn.commit()
curs.execute("select * from location_tr")
for row in curs:
print row
And Error is:
Traceback (most recent call last):
File "F:\JavaWorkspace\Test\src\sqlite_example.py", line 16, in <module>
(NULL, data.entries[i].title,data.feed.updated)")
sqlite3.OperationalError: near "[i]": syntax error
Try
curs.execute("insert into location_tr values\
(NULL, '%s', '%s')" % (data.entries[i].title, data.feed.updated))
the error should be this line
curs.execute("insert into location_tr values\
(NULL, data.entries[i].title,data.feed.updated)")
data.entries[i].title comes from Python. So if you enclose it in double quotes, it becomes a literal string, not a value. It should be something like this:
curs.execute("insert into location_tr values (NULL," + data.entries[i].title +","+data.feed.updated+")")