We Keep Coding

SQLITE with python : Adding the database name in SELECT statement

I am using sqlite3 with python, and after connecting to the database and creating a table, sqlite3 shows an error when I try to execute a SELECT statment on the table with the name of the databse in it :
con = sqlite3.connect("my_databse")
cur = con.cursor()
cur.execute('''CREATE TABLE my_table ... ''')
cur.execute("SELECT * FROM my_database.my_table") # this works fine without the name of the database before the table name
but I get this error from sqlite3 :
no such table : my_database.my_table
Is there a way to do a SELECT statment with the name of the database in it ?

The short answer is no you can't do this with SQLite. This is because you already specify the database name with sqlite3.connect() and SQLite3 doesn't allow multiple databases in the same file.

Make sure of the database is in the same directory with the python script. In order to verify this you can use os library and os.listdir() method. After connecting the database and creating the cursor, you can query with the table name.
cur.execute('SELECT * FROM my_table')

Querying SQLite database file in Google Colab

print ('Files in Drive:')
!ls drive/AI
Files in Drive:
database.sqlite
Reviews.csv
Untitled0.ipynb
fine_food_reviews.ipynb
Titanic.csv
When I run the above code in Google Colab, clearly my sqlite file is present in my drive. But whenever I run some query on this file, it says
# using the SQLite Table to read data.
con = sqlite3.connect('database.sqlite')
#filtering only positive and negative reviews i.e.
# not taking into consideration those reviews with Score=3
filtered_data = pd.read_sql_query("SELECT * FROM Reviews WHERE Score !=3",con)
DatabaseError: Execution failed on sql 'SELECT * FROM Reviews WHERE
Score != 3 ': no such table: Reviews

Below you will find code that addresses the db setup on the Colab VM, table creation, data insertion and data querying. Execute all code snippets in individual notebook cells.
Note however that this example only shows how to execute the code on a non-persistent Colab VM. If you want to save your database to GDrive you will have to mount your Gdrive first (source):
from google.colab import drive
drive.mount('/content/gdrive')
and navigate to the appropriate file directory after.
Step 1: Create DB
import sqlite3
conn = sqlite3.connect('SQLite_Python.db') # You can create a new database by changing the name within the quotes
c = conn.cursor() # The database will be saved in the location where your 'py' file is saved
# Create table - CLIENTS
c.execute('''CREATE TABLE SqliteDb_developers
([id] INTEGER PRIMARY KEY, [name] text, [email] text, [joining_date] date, [salary] integer)''')
conn.commit()
Test whether the DB was created successfully:
!ls
Output:
sample_data SQLite_Python.db
Step 2: Insert Data Into DB
import sqlite3
try:
sqliteConnection = sqlite3.connect('SQLite_Python.db')
cursor = sqliteConnection.cursor()
print("Successfully Connected to SQLite")
sqlite_insert_query = """INSERT INTO SqliteDb_developers
(id, name, email, joining_date, salary)
VALUES (1,'Python','MakesYou#Fly.com','2020-01-01',1000)"""
count = cursor.execute(sqlite_insert_query)
sqliteConnection.commit()
print("Record inserted successfully into SqliteDb_developers table ", cursor.rowcount)
cursor.close()
except sqlite3.Error as error:
print("Failed to insert data into sqlite table", error)
finally:
if (sqliteConnection):
sqliteConnection.close()
print("The SQLite connection is closed")
Output:
Successfully Connected to SQLite
Record inserted successfully into SqliteDb_developers table 1
The SQLite connection is closed
Step 3: Query DB
import sqlite3
conn = sqlite3.connect("SQLite_Python.db")
cur = conn.cursor()
cur.execute("SELECT * FROM SqliteDb_developers")
rows = cur.fetchall()
for row in rows:
print(row)
conn.close()
Output:
(1, 'Python', 'MakesYou#Fly.com', '2020-01-01', 1000)

Try this instead. See what tables are there.
"SELECT name FROM sqlite_master WHERE type='table'"

give similar sharable id to your database file just like you did with Reviews.csv
database_file=drive.CreateFile({'id':'your_sharable_id for sqlite file'})
database_file.GetContentFile('database.sqlite')

If you are trying to access the files from your google drive, you need to mount the drive first:
from google.colab import drive
drive.mount('/content/drive')
After you do this, right click on the file that you intend to read in colab session and select 'Copy Path'and paste it in the connection string.
con = sqlite3.connect('/content/database.sqlite')
You can now read the file.

con = sqlite3.connect('database.sqlite')
filtered_data = pd.read_sql_query("SELECT * FROM Reviews WHERE Score !=3",con)
If you are executing it twice you will definitely end with this type of error.Execute it exactly once without any fail.
If you get any error then remove
database.sqlite
this file and extract it again.This time execute it again without any fail/error .This worked for me .

Using Sqlite with WAL

I've been following Python documentation on the SQLite tutorial and I managed to create an Employee table and write to it.
import sqlite3
conn = sqlite3.connect('employee.db')
c = conn.cursor()
firstname = "Ann Marie"
lastname = "Smith"
email = "ams#cia.com"
employee = (email, firstname, lastname)
c.execute('INSERT INTO Employee Values (?,?,?)', employee)
conn.commit()
# Print the table contents
for row in c.execute("select * from Employee"):
print(row)
conn.close()
I've been reading about the Write-Ahead Logging, but I can't find a tutorial that explains how to implement it. Can someone provide an example?
I notice Firefox, which uses SQLite, locks the file in such a way that if you attempt to delete the sqlite file while using Firefox, it will fail saying "file is open or being used"(or something similar), how do I achieve this? I'm running Python under Windows 10.

conn = sqlite3.connect('app.db', isolation_level=None)
Set journal mode to WAL:
conn.execute('pragma journal_mode=wal')
Or another way (just show how to off wal mode)
cur = conn.cursor()
cur.execute('pragma journal_mode=DELETE')

The PRAGMA journal_mode documentation says:
If the journal mode could not be changed, the original journal mode is returned. […]
Note also that the journal_mode cannot be changed while a transaction is active.
So you have to ensure that the database library does not try to be clever and automatically starts a transaction.

How to display db table names

The end goal is to modify Firefox cookies.sqlite db before starting Firefox.
At present, I want to display what tables are in the db. I copied the cookies.sqlite db to my desktop. I'm working with the db on my desktop.
This is my first time using sqlite. I copied some code, but I'm not able to read what tables are in the db.
List of tables, db schema, dump etc using the Python sqlite3 API
Here is what I get in the terminal. I see the listing of the tables.
me $ ls cookies.sqlite
cookies.sqlite
me $ sqlite3 cookies.sqlite
SQLite version 3.8.10.2 2015-05-20 18:17:19
Enter ".help" for usage hints.
sqlite> .tables
moz_cookies
sqlite> .exit
me $ # I run my python script
me $ ./sqlTutorial.bash
SQLite version: 3.8.10.2
table records...
[]
more table records...
me $ cat dump.sql
BEGIN TRANSACTION;
COMMIT;
me $
python code. All I want to do is display the tables in the cookie.sqlite db.
#! /bin/python
import sqlite3 as lite
import sys
con = lite.connect('cookie.sqlite')
with con:
cur = con.cursor()
cur.execute('SELECT SQLITE_VERSION()')
data = cur.fetchone()
print ("SQLite version: %s" % data)
print ("table records...")
cursor = con.cursor()
cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")
print(cursor.fetchall())
print ("more table records...")
with open('dump.sql', 'w') as f:
for line in con.iterdump():
f.write('%s\n' % line)

As you noticed, the database name is cookies.sqlite not cookie.sqlite.
Why didn't I get an error? I though python would error out on. con = lite.connect('cookie.sqlite')
Connecting to a sqlite database either opens an existing database file, or creates a new one if the database didn't exist.

Search Sqlite Database - All Tables and Columns

Is there a library or open source utility available to search all the tables and columns of an Sqlite database? The only input would be the name of the sqlite DB file.
I am trying to write a forensics tool and want to search sqlite files for a specific string.

Just dump the db and search it.
% sqlite3 file_name .dump | grep 'my_search_string'
You could instead pipe through less, and then use / to search:
% sqlite3 file_name .dump | less

You could use "SELECT name FROM sqlite_master WHERE type='table'"
to find out the names of the tables in the database. From there it is easy to SELECT all rows of each table.
For example:
import sqlite3
import os
filename = ...
with sqlite3.connect(filename) as conn:
conn.row_factory = sqlite3.Row
cursor = conn.cursor()
cursor.execute("SELECT name FROM sqlite_master WHERE type='table'")
for tablerow in cursor.fetchall():
table = tablerow[0]
cursor.execute("SELECT * FROM {t}".format(t = table))
for row in cursor:
for field in row.keys():
print(table, field, row[field])

I know this is late to the party, but I had a similar issue but since it was inside of a docker image I had no access to python, so I solved it like so:
for X in $(sqlite3 database.db .tables) ; do sqlite3 database.db "SELECT * FROM $X;" | grep >/dev/null 'STRING I WANT' && echo $X; done
This will iterate through all tables in a database file and perform a select all operation which I then grep for the string. If it finds the string, it prints the table, and from there I can simply use sqlite3 to find out how it was used.
Figured it might be helpful to other who cannot use python.

#MrWorf's answer didn't work for my sqlite file (an .exb file from Evernote) but this similar method worked:
Open the file with DB Browser for SQLite sqlitebrowser mynotes.exb
File / Export to SQL file (will create mynotes.exb.sql)
grep 'STRING I WANT" mynotes.exb.sql