I'm trying to use sqlite3 in python to delete a selection of rows from a table. My attempt fails, but I can't work out why.
The sql query works ok, but I can't implement it within the python code.
I have a set of records that are moved from current_table to archive_table after a period of time.
I'd like to clean up the current_table by removing those rows that are in the archive_table (matched on id).
Intended SQL query:
DELETE FROM current_table WHERE id IN ( SELECT id FROM archive_table);
Attempted python code:
import sqlite3
def clean_up(current_table, archive_table):
db = sqlite3.connect(sqlite_db)
cursor = db.cursor()
sql_query_delete = '''DELETE FROM %s WHERE id IN ( SELECT id FROM %s);''' % (current_table, archive_table)
try:
cursor.execute(sql_query_delete)
db.commit()
db.close()
except:
print("error deleting")
Now working. The database file was locked by another process. Removing the pointless try/except led me to the detailed error message.
Related
I am trying to setup a python script to get some data and store it into a SQLite database. However when I am running the script a .fuse_hidden file is created.
On windows no .fuse_hidden file is observed but on ubuntu it generates at each call. The .fuse_hidden file seems to contain some form of sql query with input and tables.
I can delete the files without error during runtime but they are not deleted automatically. I make sure to end my connection to the db when I am finished with the query.
lsof give no information.
I am out of ideas on what to try next to get the files removed automatically. Any suggestions?
Testing
In order to confirm that it is nothing wrong with the code I made a simple script
(Assume there is an empty error.db)
import sqlite3
conn = sqlite3.connect("error.db")
cur = conn.cursor()
create_query = """
CREATE TABLE Errors (
name TEXT
);"""
try:
cur.execute(create_query)
except:
pass
cur.execute("INSERT INTO Errors (name) VALUES(?)", ["Test2"])
conn.commit()
cur.close()
conn.close()
I am working on a PyQT5 application with part of it being a table that displays movie ratings in a QTableWidget. When I add a movie to the database, I clear the old table and run a new select query to get the ten most recent movies logged. This query works correctly the first time, but fails every time afterwards. Each time after the first add, the select query just returns the 10 entries from the first time the query was executed instead of returning movies logged since then. What's confusing me is that if I run the query in MySQL workbench, it runs correctly every time. It's only when the entries are passed to my application that things go wrong. Here is the refresh method that is run after an entry is added:
def refreshLastTen(self):
#Refreshes the table of recently watched movies after a new entry is added or a change
print('Refreshing Last10 Table')
self.LastTenTable.clearContents()
#When adding one new movie, the refresh works fine. Anything more than that, the result from the first add is returned every time
sql = "SELECT * FROM log ORDER BY LOG_MOVIE_DATE desc LIMIT 0, 10" #Selects top 10 results from the table
cursor = dbConnection.cursor()
cursor.execute(sql)
myresult = cursor.fetchall()
cursor.close()
for row_number, row_data in enumerate(myresult): #Adds data from select statement to the table
for column_number, data in enumerate(row_data):
self.LastTenTable.setItem(row_number, column_number,QtWidgets.QTableWidgetItem(str(data)))
For example, I add the movie "Test 1" at the current date (theoretically the most recently watched) to the database and the refresh method returns the following:
After this, I add another movie "Test 2" at the current date and instead of returning "Test 2" and "Test 1" as the most recent 2 movies in the list, the same result from the first query is returned again. This continues no matter how many more movies you add. As stated before, if you were to then run the same query in MySQL workbench, it would correctly return "Test 2" and "Test 1" as the two most recent entries. Does anyone have any ideas of what could be happening?
*Edit 1: Here is the code used in my insert method to add a movie to the database:
if self.validateSubmission() == True:
sql = "INSERT INTO log (LOG_MOVIE_TITLE, LOG_MOVIE_DATE, LOG_MOVIE_RATING, LOG_MOVIE_GENRE, LOG_MOVIE_LOCATION, LOG_MOVIE_COMMENTS) VALUES (%s, %s, %s, %s, %s, %s)"
vals = [self.titleVal.text(), self.dateVal.date().toString('yyyy-MM-dd'), self.ratingVal.currentText(), self.genreVal.currentText(), self.locationVal, self.commentVal.toPlainText()]
cursor = dbConnection.cursor()
cursor.execute(sql, vals)
dbConnection.commit()
cursor.close()
dbConnection.close()
*Edit 2: Here is a screenshot from MySQL Workbench showing the select query working:
*Edit 3: Now I am wondering if it has to do with how I am handling the dbConnection. The refreshLastTen method is in the app's main page class whereas the add happens in an add form class. Each class has dbConnection explicitly defined like this:
dbConnection = mysql.connector.connect(
host="localhost",
user="root",
passwd="pwd",
database="moviesheet"
)
I am not sure if this causes a disconnect or collision of some sort. I added the following to close the connection in the add form class after the add is done just in case:
dbConnection.close()
One other oddity that I noticed is that if I run the query in the add form class after the add, it returns the correct result. It seems like it might just be related to the main window class.
I have a SQLite db with three relational tables. I'm trying to return the max record from a log table along with related columns from the other tables based on the ID relationships.
I created the query in DB Browser and verified it returns the expected record however, when I use the exact same query statement in my python code it never steps into the 'for' loop.
SQL statement in python -
def GetLastLogEntry():
readings = ()
conn = sqlite3.connect(dbName)
conn.row_factory = sqlite3.Row
cursor = conn.cursor()
cursor.execute("SELECT f.FoodCategory, f.FoodName, gs.FoodWeight,
gsl.GrillDateTime, gsl.CurrentGrillTemp, gsl.TargetGrillTemp,
gsl.CurrentFoodTemp, gsl.TargetFoodTemp, gsl.CurrentOutsideTemp,
gsl.CurrentOutsideHumidity FROM Food as f, GrillSession as gs,
GrillSessionLog as gsl WHERE f.FoodId = gs.FoodId AND
gs.GrillSessionID = gsl.GrillSessionID AND gsl.GrillSessionLogID =
(SELECT MAX(GrillSessionLog.GrillSessionLogID) FROM
GrillSessionLog, GrillSession WHERE GrillSessionLog.GrillSessionID
= GrillSession.GrillSessionID AND GrillSession.ActiveSession =
1)")
for row in cursor:
print("In for loop")
readings = readings + (row['FoodCategory'], row['FoodName'])
print("Food Cat = " + row['FoodCategory'])
cursor.close()
return readings
The query in DB Browser returns only one row which is what I'm trying to have happen in the python code.
Just discovered the issue....
Using DB Browser, I updated a record I'm using for testing but failed to "write" the change to the database table. As a result, every time I was executing my python code against the table it was executing the query with the original record values because my change wasn't yet committed via DB Browser.
Huge brain fart on that one.... Hopefully it will be a lesson learned for someone else in the future.
My simple test code is listed below. I created the table already and can query it using the SQLite Manager add-in on Firefox so I know the table and data exist. When I run the query in python (and using the python shell) I get the no such table error
def TroyTest(self, acctno):
conn = sqlite3.connect('TroyData.db')
curs = conn.cursor()
v1 = curs.execute('''
SELECT acctvalue
FROM balancedata
WHERE acctno = ? ''', acctno)
print v1
conn.close()
When you pass SQLite a non-existing path, it'll happily open a new database for you, instead of telling you that the file did not exist before. When you do that, it'll be empty and you'll instead get a "No such table" error.
You are using a relative path to the database, meaning it'll try to open the database in the current directory, and that is probably not where you think it is..
The remedy is to use an absolute path instead:
conn = sqlite3.connect('/full/path/to/TroyData.db')
You need to loop over the cursor to see results:
curs.execute('''
SELECT acctvalue
FROM balancedata
WHERE acctno = ? ''', acctno)
for row in curs:
print row[0]
or call fetchone():
print curs.fetchone() # prints whole row tuple
The problem is the SQL statment. you must specify the db name and after the table name...
'''SELECT * FROM db_name.table_name WHERE acctno = ? '''
I am using raw sql queries for inserting the data into DB. The insertion is working right, Now I want to perform some checks on this insert query e.g. If the query has inserted the data or not
suppose I have insert query like
cursor.execute("some insert query" )
Now I want to know whether cursor.execute has inserted the row then show me some text like success and if it fails to insert for some reason then show me text like error and also if the row is already inserted then show, row already exist.
But I don't know how to perform these checks on cursor.execute.
edit
for i in range(numrows):
row = cursor.fetchone()
if row[6]==1:
arr["user_id"]=row[0]
arr["email"]=row[1]
arr["style_quiz_score"]=row[2]
arr["style_quiz_answer"]=row[3]
arr["date_joined"]=row[4]
arr["is_active"]=row[5]
arr['firstname'] = row[7]
arr["username"]=re.sub(r'[^a-zA-Z0-9]', '_', arr["email"])
elif row[6]==2:
arr['lastname'] = row[7]
cursor1.execute("insert into auth_user(id,username,first_name,last_name,email,password,is_staff,is_active,is_superuser,date_joined,last_login) values(%s,%s,%s,%s,%s,'NULL',0,%s,0,%s,0)",[arr["user_id"],arr["username"],arr['firstname'],arr['lastname'],arr["email"],arr["is_active"],arr["date_joined"]])
when i am executing cursor1.execute outside forloop than it insert the last entry , but if i execute it in inside forloop than it gives error and nothing will be inserted
Assuming you're using Django (you're not specific about it in your question, but you're using the django tag), you need to do transaction.commit_unless_managed() (from django.db import transaction) after issuing the insert query with cursor.execute.
You can check for exceptions when calling commit_unless_managed to see if the insert went well or not:
from django.db import connection, transaction, DatabaseError, IntegrityError
cursor = connection.cursor()
cursor.execute("some insert query" )
try:
transaction.commit_unless_managed()
except DatabaseError, IntegrityError:
print 'error'
else:
print 'success'