I’m trying to INSERT INTO / ON DUPLICATE KEY UPDATE taking the values from one table and inserting into another. I have the following Python code.
try:
cursor.execute("SELECT LocationId, ProviderId FROM CQCLocationDetailsUpdates")
rows = cursor.fetchall()
for row in rows:
maria_cnxn.execute('INSERT INTO CQCLocationDetailsUpdates2 (LocationId, ProviderId) VALUES (%s,%s) ON DUPLICATE KEY UPDATE ProviderId = VALUES(%s)', row)
mariadb_connection.commit()
except TypeError as error:
print(error)
mariadb_connection.rollback()
If I change this script just to INSERT INTO it work fine, the problem seems to be when I add the ON DUPLICATE KEY UPDATE. What do I have wrong? LocationId is the PRIMARY KEY
I get this error.
Traceback (most recent call last):
File "C:/Users/waynes/PycharmProjects/DRS_Dev/CQC_Locations_Update_MariaDB.py", line 228, in <module>
maria_cnxn.execute('INSERT INTO CQCLocationDetailsUpdates2 (LocationId, ProviderId) VALUES (%s,%s) ON DUPLICATE KEY UPDATE ProviderId = VALUES(%s)', row)
File "C:\Users\waynes\PycharmProjects\DRS_Dev\venv\lib\site-packages\mysql\connector\cursor.py", line 548, in execute
stmt = RE_PY_PARAM.sub(psub, stmt)
File "C:\Users\waynes\PycharmProjects\DRS_Dev\venv\lib\site-packages\mysql\connector\cursor.py", line 79, in __call__
"Not enough parameters for the SQL statement")
mysql.connector.errors.ProgrammingError: Not enough parameters for the SQL statement
Your error is because row is a 2 element tuple and your SQL statement requires three %s vars.
It is however possible to use an INSERT .. SELECT .. ON DUPLICATE KEY like:
maria_cnxn.execute('INSERT INTO CQCLocationDetailsUpdates2 (LocationId,
ProviderId)
SELECT LocationId, ProviderId
FROM CQCLocationDetailsUpdates orig
ON DUPLICATE KEY UPDATE CQCLocationDetailsUpdates2.ProviderID = orig.ProviderID')
Whenever you end up doing a loop around a SQL statement you should look to see if there is a SQL way of doing this.
Related
EDIT: Yup I'm dumb. Missed the typo.
I'm following along with a video in a Udacity course, and getting an error trying to run a SQL command via psycopg2. The code is identical to the instructor's, but mine returns an error and her's doesnt.
import psycopg2
# establish connection to db
connection = psycopg2.connect('dbname=example')
# cursor is essentially an interface that allows you to start
# cuing up work and transactions
cursor = connection.cursor()
# defines SQL transaction
cursor.execute('''
CREATE TABLE table2 (
id INTEGER PRIMARY KEY,
completed BOOLEAN NOT NULL DEFUALT False
);
''')
cursor.execute('INSERT INTO table2 (id, completed) VALUES (1, true);')
# commits the transaction
connection.commit()
# must manually close your session each time one is opened
connection.close()
cursor.close()
Error:
$ python3 demo.py
Traceback (most recent call last):
File "demo.py", line 11, in <module>
cursor.execute("""
psycopg2.errors.SyntaxError: syntax error at or near "DEFUALT"
LINE 4: completed BOOLEAN NOT NULL DEFUALT False
You seem to have made a typo instead of DEFAULT you have written DEFUALT
cursor.execute('''
CREATE TABLE table2 (
id INTEGER PRIMARY KEY,
completed BOOLEAN NOT NULL DEFAULT False
);
''')
I have a trouble with my program. I want input database from file txt. This is my source code
import MySQLdb
import csv
db=MySQLdb.connect(user='root',passwd='toor',
host='127.0.0.1',db='data')
cursor=db.cursor()
csv_data=csv.reader(file('test.txt'))
for row in csv_data:
sql = "insert into `name` (`id`,`Name`,`PoB`,`DoB`) values(%s,%s,%s,%s);"
cursor.execute(sql,row)
db.commit()
cursor.close()
After run that program, here the error
Traceback (most recent call last):
File "zzz.py", line 9, in <module>
cursor.execute(sql,row)
File "/home/tux/.local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 187, in execute
query = query % tuple([db.literal(item) for item in args])
TypeError: not enough arguments for format string
and this is my test.txt
4
zzzz
sby
2017-10-10
Please help, and thanks in advance.
Now that you have posted the CSV file, the error should now be obvious to you - each line contains only one field, not the four that the SQL statement requires.
If that is the real format of your data file, it is not CSV data. Instead you need to read each group of four lines as one record, something like this might work:
LINES_PER_RECORD = 4
SQL = 'insert into `name` (`id`,`Name`,`PoB`,`DoB`) values (%s,%s,%s,%s)'
with open('test.txt') as f:
while True:
try:
record = [next(f).strip() for i in range(LINES_PER_RECORD)]
cursor.execute(SQL, record)
except StopIteration:
# insufficient lines available for record, treat as end of file
break
db.commit()
I am doing this all as a test. I want to take a csv file that has headers and copy the values into a postgresql database table. The tables columns are named the same as the headers in csv file case-sensitive. table has two columns "pkey", "m". the csv just has the "m" for header. pkey is just the primary key setup to auto increment. As a test i just want to copy the "m" column in the csv file the table.
import csv
import psycopg2
database = psycopg2.connect ( database = "testing", user="**",
password="**", host="**", port="**")
ocsvf = open("sample.csv")
def merger(conn, table_name, file_object):
cursor = conn.cursor()
cursor.copy_from(file_object, table_name, sep=',', columns=('mls'))
conn.commit()
cursor.close()
try:
merger(database, 'tests', ocsvf)
finally:
database.close()
when i try to run the code i get this as a error
Traceback (most recent call last):
File "csvtest.py", line 26, in <module>
merger(database, 'tests', ocsvf)
File "csvtest.py", line 21, in merger
cursor.copy_from(file_object, table_name, sep=',', columns=('m'))
psycopg2.ProgrammingError: column "m" of relation "tests" does not exist
I am sure its something simple that i just keep over looking but i have also googled this and the one thing i found was someone said it might be the primary key is setup right but i tested it and the primary keys works fine when i do manual input from pgadmin. any help would be great thanks
In this line:
cursor.copy_from(file_object, table_name, sep=',', columns=('mls'))
The ('mls') evaluated to "mls" which eventually means that iterating over it will result in 3 items ['m','l','s'].
You should write this line as follows:
cursor.copy_from(file_object, table_name, sep=',', columns=('mls',))
The expression ('mls',) evaluated to a tuple with one item: "mls", which is what I guess you meant to do.
I am streaming tweets to a postgres database with a python script (using psycopg2). I would like to be able to schedule this script in a windows task manager. The only issue I have to overcome is to be able to rename the table in postgres. Is it possible?
x = datetime.date.today() - datetime.timedelta(days=1)
con = psycopg2.connect("dbname='test' user='postgres'")
cur = con.cursor()
cur.execute("DROP TABLE IF EXISTS schemaname.%s", (x))
** UPDATE
That answer does get my further, now it just complains about the numbers.
Traceback (most recent call last):
File "Z:/deso-gis/scripts/test123.py", line 26, in <module>
cur.execute("DROP TABLE IF EXISTS tweets_days.%s" % x)
psycopg2.ProgrammingError: syntax error at or near ".2016"
LINE 1: DROP TABLE IF EXISTS tweets_days.2016-02-29
I believe you are getting arror at line
cur.execute("DROP TABLE IF EXISTS schemaname.%s", (x))
because psycopg generates not what you want:
DROP TABLE IF EXISTS schemaname."table_name"
try using
cur.execute("DROP TABLE IF EXISTS schemaname.%s" % x)
This is not as secure as could be but now table name is name not SQL string.
I'm getting the following error:
Traceback (most recent call last):
File "/home/pi/Nike/test_two.py", line 43, in <module>
do_query()
File "/home/pi/Nike/test_two.py", line 33, in do_query
for(Product,Bin,Size,Color) in records:
ValueError: too many values to unpack
Code:
def do_query():
connection = sqlite3.connect('test_db.db')
cursor = connection.cursor()
cursor.execute("SELECT * FROM TESTER ORDER BY CheckNum")
records = cursor.fetchall()
for(Product,Bin,Size,Color) in records:
row_1.append(Product)
row_2.append(Bin)
row_3.append(Size)
row_4.append(Color)
connection.commit()
cursor.close()
connection.close()
do_query()
I'm trying to load each column of a table into seperate python list. I am using Python, and sqlite3. Why am I getting this error?
You are using "SELECT *" which will return every column from the table. My guess is that the table in question contains more columns then the 4 you specified.
A better way would actually be specifying in the SQL which columns you want so that your code will not break if columns are added to the database.
Something like "SELECT col1, col2 FROM table"
You can run the sqlite3 tool on the db file and then view the table schema with ".schema <table_name>"