sqlite3.OperationalError: database is locked - python

I'm trying to insert all values of a list to my sqlite3 database. When I simulate this query by using the python interactive interpreter, I am able to insert the single value to DB properly. But my code fails while using an iteration:
...
connection=lite.connect(db_name)
cursor=connection.cursor()
for name in match:
cursor.execute("""INSERT INTO video_dizi(name) VALUES (?)""",(name,))
connection.commit()
...
error:cursor.execute("""INSERT INTO video_dizi(name) VALUES (?)""",(name,))
sqlite3.OperationalError: database is locked
Any way to overcome this problem?

Do you have another connection elsewhere in your code that you use to begin a transaction that is still active (not committed) when you try to commit the operation that fails?

As this error can happen because you have opened your site.db or database file in DBbrowser type application to view in interactive database interface. Just close that it will work fine.

Because your database is use by another process or connection. If you need real concurrency, use a real RDBMS.

Related

SQLAlchemy, Postgres: Run SQL without transaction

I am using SQLAlchemy and pg8000 to connect to a Postgres database.
I have checked table pg_stat_activity, which shows me a few select queries in 'idle in transaction' state, many of those. But the application much more reads than writes, that is, inserts are few and far between.
I suspect that a new transaction is created for each query, even for simple select statements.
Is it possible to run a read-only query without the need for a transaction? So that it does not need to be committed/rolled back?
Currently, the app runs its queries with method sqlalchemy.engine.Engine.execute for CRUD operations and cursors for calling stored procedures. How should I update these method calls to indicate I want some of them not to start transactions?

Discrepancy between my SQL python data and the DB Browser

I am using sqlite combined with tkinter to write and delete records within my Python program. The deletion works perfectly fine in my program and also when I restart the program, the record does not exist anymore.
However, I always cross check using the Linux standard software DB Browser for SQLite and look at my SQL Table. Strangely, all records still exist in the DB Browser. Now I am wondering, why's that? Why is it gone within my Python sqlite queries but not in the DB Browser? Somehow the records are still there. How can I completely destroy my records?
For deletion I use:
(The user can chose a specific entry using a listbox. Eventually, I "translate" the selected item into its specific ID and trigger the deletion.)
self.c.execute("DELETE FROM financial_table WHERE ID=?",(entry,))
self.conn.commit()
For my query I use:
(I query the data for a specific year and month.)
self.c.execute("SELECT ID, Date, Item, Price FROM financial_table WHERE strftime('%Y-%m', Date) = '{}' ORDER BY Date ".format(date))
single_dates = self.c.fetchall()
Thank you very much for your help.
The solution to my question is: I am stupid!
I was tired yesterday evening and looked at the wrong sql file in a subfolder which had the same name than the one from my python program. So it is actually working. Please excuse my stupidity.
#Bruceskyaus
Despite my stupidity I learned from your answer, especially the try ... except block. I am going to implement it. Thanks.
You may have an problem with controlling transactions on your database, but it could also be the connection itself. Make sure you don't have any uncommitted DML statements on a different connection (i.e. an INSERT, UPDATE or DELETE in your DB Browser that wasn't committed), this could cause the conn.commit() to fail. With SQLite, an uncommitted transaction could lock the entire database - for a brief period of time.
Try ensuring that there is a new cursor for the delete statement and call conn.close() after the conn.commit(). Before you execute the code, make sure that no other connections are accessing the database - including the DB Browser. Only check in the DB Browser when you have shut down the application (for this test). This eliminates multithreading or locking as a possible cause. See also SQLite - Data Persistence and SQLite - Controlling Transactions
It is also helpful to trap all errors for DML statements using a try...except block. Something like this:
import sqlite3
try:
self.conn = sqlite3.connect('mydb.db')
self.c = conn.cursor()
self.c.execute("DELETE FROM financial_table WHERE ID=?",(entry,))
self.conn.commit()
except sqlite3.Error as e:
print("An error occurred:", e.args[0])
finally:
self.conn.close()

TSQL with SQLAlchemy does not execute

I'm trying to execute TSQL queries in a remote MSSQL database by using SQLAlchemy and pymssql. I've tested my procedural query directly in the database and it works as intended, it also works if I run it directly through pymssql. If I run a regular one liner queries such as:
select table_name from INFORMATION_SCHEMA.tables
Through SQLAlchemy this also works as it should. But when I try to execute the following TSQL query it does not actually create the table:
IF NOT EXISTS (SELECT *
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME = 'SOME_TABLE')
BEGIN
CREATE TABLE SOME_TABLE (SOME_TEXT VARCHAR(255), SOME_TIME DATETIME)
END
it runs it as it was successful and if I try to read the result it from the execution it gives me "Resource already closed error" as expected since it is a CREATE query. However if I try to add data to table 'SOME_TABLE' it pukes at me and says that the table does not exist. Feels like it is only uploading the query as a function but never executes it. Any ideas? Or even better; TSQL queries that actually works when executing with SQLAlchemy and pymssql.
Thanks,
You need to commit your pending changes in the Session. For basic understanding of the process read Using the Session. Quick solution:
session.commit()
TIME and TEXT are reserve words.
I do not know how SQL Alchemy or pymmsql talks to SQL Server. Either native client or ODBC. It eventually all boils down to a tabular data stream (TDS) over a network protocol like TCP/IP.
Check out the reserve word list on TECHNET.
-- Create table ?
IF OBJECT_ID('DBO.SOMETABLE') IS NULL
CREATE TABLE DBO.SOMETABLE (MY_TEXT VARCHAR(255), MY_TIME DATETIME);
I use the OBJECT_ID function since it is less typing.
But NOT EXITS works with both SELECT FROM the sys.objects or information_schema.tables with correct WHERE clauses.

musqldb-python doesnt really update the original database

I used mysqldb to connect to a database in my localhost.
It works, but if I add data to a table in the database when the program is running, it shows that it has been added, but when I check the table from localhost, it hasn't been updated.
if your table uses innodb engine, you should call connection.commit() on every cursor.execute().

PyMysql UPDATE query

I've been trying using PyMysql and so far everything i did worked (Select/insert) but when i try to update it just doesn't work, no errors no nothing, just doesn't do anything.
import pymysql
connection = pymysql.connect(...)
cursor = connection.cursor()
cursor.execute("UPDATE Users SET IsConnected='1' WHERE Username='test'")
cursor.close()
connection.close()
and yes I've double checked that Users, IsConnected and Username are all correct and test does exist (SELECT works on it)
what's my problem here?
When you execute your update, MySQL is implicitly starting a transaction. You need to commit this transaction by calling connection.commit() after you execute your update to keep the transaction from automatically rolling back when you disconnect.
MySQL (at least when using the InnoDB engine for tables) supports transactions, which allow you to run a series of update/insert statements then have them either all commit at once effectively as a single operation, or rollback so that none are applied. If you do not explicitly commit a transaction, it will rollback automatically when you close your connection to the database.
In fact, what #JoeDay has described above has little to do with default MySQL transaction's behaviour. MySQL by default operates in auto-commit mode and normally you don't need any additional twist to persist your changes:
By default, MySQL runs with autocommit mode enabled. This means that as soon as you execute a statement that updates (modifies) a table, MySQL stores the update on disk to make it permanent. The change cannot be rolled back.
PEP-249's (DB API) authors decided to complicate things and break Zen of Python by making a transaction's start implicit, by proposing auto-commit to be disabled by default.
What I suggest to do, is to restore MySQL's default behaviour. And use transactions explicitly, only when you need them.
import pymysql
connection = pymysql.connect(autocommit=True)
I've also written about it here with a few references.

Categories