db = MySQLdb.connect("XXXXXXXX","root", "XXXXXX", database)
cursor = db.cursor()
cursor.execute('INSERT INTO media_files (ID, DATA) VALUES ("test", "test")')
cursor.execute("commit")
This statement errors and I cannot seem to figure out why. Any thoughts? The table media_files btw only has the two columns, ID and DATA each of which are VARCHAR(255)
Thanks
Trent
'INSERT INTO media_files (ID, DATA) VALUES ("test", "test")'
Using the parentheses will fix the problem. You were inserting one piece of data but provided 2 fields.
Related
I was hoping someone could help me, I have a task to do for my College course using Python and SQlite.
I have two excel sheets I would like to make into two tables and a left join on some finance information. I have started by importing the two sheets into pandas and what I thought was creating two tables, but if I'm honest I don't know where I am going wrong. The spreadsheets contain a lot more Columns than I would like to use in the SQL database. Could this be a problem?
I am eventually presented with error code "OperationalError: table Lockedlist has no column named Site Ref"
import pandas as pd
Lockedlist_panda = pd.read_excel (r'Location of Lockedlist_panda')
PO1report_panda = pd.read_excel (r'Location of PO1report_panda')
import sqlite3
conn = sqlite3.connect("data_superstore1.db")
cursor = conn.cursor()
sql='''CREATE TABLE Lockedlist (
NR TEXT,
Programme TEXT NOT NULL,
Sub_Region TEXT NOT NULL,
Site_Type TEXT NOT NULL,
MS13_Actual TEXT NOT NULL,
PRIMARY KEY(NR)
)'''
cursor.execute(sql)
print("Table created successfully........")
#Commit your changes in the database
conn.commit()
#Closing the connection
conn.close()
conn = sqlite3.connect("data_superstore1.db")
cursor = conn.cursor()
sql='''CREATE TABLE PoReportTable (
Tracking_Field TEXT,
VENDOR TEXT NOT NULL,
Short_Text TEXT NOT NULL,
COST INT NOT NULL,
PRIMARY KEY(Tracking_Field)
)'''
cursor.execute(sql)
print("Table created successfully........")
#Commit your changes in the database
conn.commit()
#Closing the connection
conn.close()
conn = sqlite3.connect("data_superstore1.db")
Lockedlist_panda.to_sql('Lockedlist', conn, if_exists='append', index=False)
PO1report_panda.to_sql('PoReportTable', conn, if_exists='append', index=False)
#Commit your changes in the database
conn.commit()
#Closing the connection
conn.close()
Indeed pandas try to insert all columns from the dataframe into the database, hence if it has more columns that what you defined in the database it will complain for the missing columns,
Try to limit your dataframe to the columns you want first:
Lockedlist_panda = Lockedlist_panda[["NR","Programme","Sub_Region","Site_Type","MS13_Actual" ]]
First-time question. I am writing a Python application for personal use, which reads metadata from MP3 files I've collected on CD-ROMs and inserts it into a MySQL database. At least, it should--but when it comes to the actual INSERT statement, the program is throwing a ProgrammingError: "1064: You have an error in your SQL syntax," etc.
In trying to write this program, I've learned how to create parameterized queries, instantiate the cursor object with cursor_class = MySQLCursorPrepared, and instantiate the database connection with use_pure = True. I've searched the Web for similar problems but come up dry.
Here is the offending code (it's the cursor.execute line specifically that throws the exception; for debugging purposes I've temporarily removed the try/except blocks):
table = "mp3_t"
# Parameterized query for SQL INSERT statement
query = '''
INSERT INTO %s
(track_num, title, artist, album, album_year, genre, discname)
VALUES
(%s, '%s', '%s', '%s', %s, '%s', '%s')
'''
conn = self.opendb(self.config)
cursor = conn.cursor(cursor_class = MySQLCursorPrepared)
for track in tracklist:
print("Counter: {}".format(counter))
# Tuple for parameterized query
input = (table, track['track_num'], track['title'],
track['artist'], track['album'], track['album_year'],
track['genre'], track['discname'])
print(query % input) # What is the actual query?
cursor.execute(query, input)
The database table is defined with the following SQL:
CREATE TABLE mp3_t (
id int NOT NULL PRIMARY KEY AUTO_INCREMENT,
track_num int NOT NULL,
title VARCHAR(255) NOT NULL,
artist VARCHAR(255) NOT NULL,
album VARCHAR(255) NOT NULL,
album_year int NOT NULL,
genre VARCHAR(255) NOT NULL,
discname VARCHAR(255) NOT NULL);
For debugging, I've got a print statement that outputs the query that's being sent to MySQL (the error message is particularly unhelpful for trying to pinpoint the cause of the problem), for example:
INSERT INTO mp3_t
(track_num, title, artist, album, album_year, genre, discname)
VALUES
(1, 'Moribund the Burgermeister', 'Peter Gabriel', 'I', 1977, 'Rock', 'Rock 19')
I don't see any error visually, and if I paste directly into the MySQL CLI and add the required semicolon, it inserts a row into the table as expected.
I'm completely stymied where the problem lies.
If it's any help, I'm running Python 3.6.7 and MariaDB 10.1.37 with Connector/Python 8.0.15, on Ubuntu 18.04 64-bit.
Table name should not be replaced by %s. I think your error message should like:
You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near "mp3_t"
So just use table name in your query template.
INSERT INTO mp3_t
(track_num, title, artist, album, album_year, genre, discname)
VALUES
(%s, %s, %s, %s, %s, %s, %s)
If you want to know the query will be executed, you should use these codes:
conn = connect()
cur = conn.cursor()
print(query % cur._get_db().literal(params))
Don't use %s for table name. Use the table name directly.
I created a database in psql and in it, created a table called "tweet".
CREATE TABLE tweet
( tid CHARACTER VARYING NOT NULL, DATA json,
CONSTRAINT tid_pkey PRIMARY KEY (tid) );
Then when I use
SELECT * FROM tweet;
in the psql window it works and shows an empty table.
Now I have a python script that takes JSON data and is loading it into this table.
conn_string = "host='localhost' port=5432 dbname='tweetsql' user='tweetsql' password='tweetsql'"
conn = psycopg2.connect(conn_string)
cur = conn.cursor()
That sets up the connection and I don't think it had any issues.
Now I have some logic to read in the JSON file and then to add it in, I say:
cur.execute("INSERT INTO tweet (tid, data) VALUES (%s, %s)", (cur_tweet['id'], json.dumps(cur_tweet, cls=DecimalEncoder), ))
But this always says that the relation tweet doesn't exist. Am I missing something here? Is there an issue with my connection or can my script somehow not see the table? For reference I'm using psycopg2 for the connection.
EDIT: I updated the DDL to include a transaction I could commit but that didn't fix it either. Is it a schema issue?
This is what I did regarding the table creation to commit:
BEGIN;
CREATE TABLE tweet
( tid CHARACTER VARYING NOT NULL, DATA json,
CONSTRAINT tid_pkey PRIMARY KEY (tid) );
COMMIT;
EDIT 2: I'm posting some code here...
import psycopg2
import json
import decimal
import os
import ctypes
conn_string = "host='localhost' port=5432 dbname='tweetsql' user='tweetsql' password='tweetsql'"
conn = psycopg2.connect(conn_string)
cur = conn.cursor()
cur.execute("CREATE TABLE tweet (tid CHARACTER VARYING NOT NULL, DATA json, CONSTRAINT tid_pkey PRIMARY KEY (tid) );")
cur.commit()
for file in os.listdir(path):
if not is_hidden(file):
with open(path+file, encoding='utf-8') as json_file:
tweets = json.load(json_file, parse_float=decimal.Decimal)
for cur_tweet in tweets:
cur.execute("INSERT INTO tweet (tid, data) VALUES (%s, %s)", (cur_tweet['id'], json.dumps(cur_tweet, cls=DecimalEncoder), ))
cur.commit()
cur.close()
conn.close()
You're probably not committing the table creation, and, (I'm assuming; not seeing your complete code) you're starting a new connection via psycopg2 each time. You need to commit right after the table creation, and not in a new connection, as each connection is its own implicit transaction. So, your code flow should be something like this:
connect to the db
create the table using the cursor
fill the table
commit and disconnect from db.
Or, if you must separate creation from filling, just commit and disconnect after (2) and then reconnect before (3).
I am inserting a couple thousand records into a table via the python code below:
values = ''
for row in cursor:
values = values + "(" + self.quoted_comma_separate(row) + "),"
values = values[:-1]
insert_statement = "INSERT INTO t1 ({0}) VALUES {1};".format(
self.comma_separate(members), values)
db = Database()
conn = db.get_db()
cursor = conn.cursor()
cursor.execute(insert_statement)
conn.commit()
conn.close()
When I check the database after it runs none of the records show up in the database. If I go into an MySQL editor and manually commit the transaction all of the records appear. Why is my conn.commit() not working?
The insert statements were fine. Turns out I had another database connection open and it was getting confused and committing to the wrong connection or something like that. Sorry for the pointless question :)
I'm trying to execute a basic INSERT statement on a MySQL table from a Python script using MySQLdb. My table looks like this:
CREATE TABLE `testtable` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`testfield` varchar(255) NOT NULL,
PRIMARY KEY (`id`)
)
Running this query from the MySQL command line works fine:
INSERT INTO `testtable` (`id`, `testfield`) VALUES (NULL, 'testvalue');
But when I try to execute the query from a Python script, no rows get inserted. Here's my code:
conn = MySQLdb.connect(host=db_host, port=db_port, user=db_user, passwd=db_password, db=db_database)
cursor = conn.cursor ()
cursor.execute ("INSERT INTO `testtable` (`id`, `testfield`) VALUES (NULL, 'testvalue')")
print "Number of rows inserted: %d" % cursor.rowcount
cursor.close()
conn.close()
Oddly, this will print "Number of rows inserted: 1." I can also confirm that this query increments the ID field, because when I add another row via the command line, the value of its ID is the same as if the Python script had successfully inserted its rows. However, running a SELECT query returns none of the rows from the script.
Any idea what's going wrong?
You either need to set conn.autocommit(), or you need to do conn.commit() - see the FAQ
you need to commit:
conn.commit()
http://mysql-python.sourceforge.net/FAQ.html#my-data-disappeared-or-won-t-go-away