Is there a library or open source utility available to search all the tables and columns of an Sqlite database? The only input would be the name of the sqlite DB file.
I am trying to write a forensics tool and want to search sqlite files for a specific string.
Just dump the db and search it.
% sqlite3 file_name .dump | grep 'my_search_string'
You could instead pipe through less, and then use / to search:
% sqlite3 file_name .dump | less
You could use "SELECT name FROM sqlite_master WHERE type='table'"
to find out the names of the tables in the database. From there it is easy to SELECT all rows of each table.
For example:
import sqlite3
import os
filename = ...
with sqlite3.connect(filename) as conn:
conn.row_factory = sqlite3.Row
cursor = conn.cursor()
cursor.execute("SELECT name FROM sqlite_master WHERE type='table'")
for tablerow in cursor.fetchall():
table = tablerow[0]
cursor.execute("SELECT * FROM {t}".format(t = table))
for row in cursor:
for field in row.keys():
print(table, field, row[field])
I know this is late to the party, but I had a similar issue but since it was inside of a docker image I had no access to python, so I solved it like so:
for X in $(sqlite3 database.db .tables) ; do sqlite3 database.db "SELECT * FROM $X;" | grep >/dev/null 'STRING I WANT' && echo $X; done
This will iterate through all tables in a database file and perform a select all operation which I then grep for the string. If it finds the string, it prints the table, and from there I can simply use sqlite3 to find out how it was used.
Figured it might be helpful to other who cannot use python.
#MrWorf's answer didn't work for my sqlite file (an .exb file from Evernote) but this similar method worked:
Open the file with DB Browser for SQLite sqlitebrowser mynotes.exb
File / Export to SQL file (will create mynotes.exb.sql)
grep 'STRING I WANT" mynotes.exb.sql
Related
I am using sqlite3 with python, and after connecting to the database and creating a table, sqlite3 shows an error when I try to execute a SELECT statment on the table with the name of the databse in it :
con = sqlite3.connect("my_databse")
cur = con.cursor()
cur.execute('''CREATE TABLE my_table ... ''')
cur.execute("SELECT * FROM my_database.my_table") # this works fine without the name of the database before the table name
but I get this error from sqlite3 :
no such table : my_database.my_table
Is there a way to do a SELECT statment with the name of the database in it ?
The short answer is no you can't do this with SQLite. This is because you already specify the database name with sqlite3.connect() and SQLite3 doesn't allow multiple databases in the same file.
Make sure of the database is in the same directory with the python script. In order to verify this you can use os library and os.listdir() method. After connecting the database and creating the cursor, you can query with the table name.
cur.execute('SELECT * FROM my_table')
I understand how to open a database using a python script:
import sqlite3 as lite
.
.
con = lite.connect(db_name)
However what I have is a lot of database files made with tsk_loaddb, when I am in sqlite3 I can use the following
SQLite version 3.8.7.2 2014-11-18 20:57:56
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> .open /work/jmjohnso1/db_project/db_IN60-1020
sqlite> .tables
blackboard_artifact_types tsk_files_derived_method
blackboard_artifacts tsk_files_path
blackboard_attribute_types tsk_fs_info
blackboard_attributes tsk_image_info
tsk_db_info tsk_image_names
tsk_file_layout tsk_objects
tsk_files tsk_vs_info
tsk_files_derived tsk_vs_parts
However, I am not understanding how to do it in python. My question is how do I translate the open command with the sqlite3 python library.
Turns out I had a newline in the name because I was reading from a file. So the following works.
def open_sql(db_folder, db_name, table):
# databases are located at /work/jmjohnso1/db_project
path_name = os.path.join(db_folder,db_name).strip()
con = lite.connect(path_name)
cur = con.cursor()
cur.execute('SELECT * FROM ' + table)
col_names = [cn[0] for cn in cur.description]
rows = cur.fetchall()
print_header(col_names)
print_output(rows)
con.close()
There is a post that tells me how to write data fom a csv-file into a sqlite database (link). Is there a way to simply copy the whole file into a database table instead of loading the data and then iterate through the rows appending them to the table as suggested in the link.
To do this in sqlite it says here to simply type
sqlite> create table test (id integer, datatype_id integer, level integer, meaning text);
sqlite> .separator ","
sqlite> .import no_yes.csv test
I am new to working with databases and sqlite3, but I thought doing something like this could work
import sqlite3
conn = sqlite3.connect('mydatabase.db')
c = conn.cursor()
def create_table(name):
c.execute("CREATE TABLE IF NOT EXISTS {} (time REAL, event INTEGER, id INTEGER, size INTERGER, direction INTEGER)".format(name))
def copy_data(file2copy,table_name):
c.executescript("""
.separator ","
.import {} {}""".format(file2copy,table_name))
conn.commit()
try:
create_table('abc')
copy_data(r'Y:\MYPATH\somedata.csv','abc')
except Exception as e:
print e
c.close()
conn.close()
But apparently it doesn't. I get the error
near ".": syntax error
EDIT: Thanks to the below suggestion to use the subprocess module, I came up with the follwing solution.
import subprocess
sqlshell = r'c:\sqlite-tools\sqlite3' # sqlite3.exe is a shell that can be obtained from the sqlite website
process = subprocess.Popen([sqlshell], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
cmd = """.open {db}
.seperator ","
.import {csv} {tb}""".format(db='C:/Path/to/database.db', csv='C:/Path/to/file.csv', tb='table_name')
stdout, stderr = process.communicate(input=cmd)
if len(stderr):
print 'something went wrong'
Importantly, you should use '/' instead of '\' in your directory names. Also, you have to be carefull with blank spaces in your directory names.
The commands you're using in copy_data that start with . are not part of sqlite3, but the interactive shell that it ships with it. You can't use them through the sqlite3 module.
You either need to manually write the insertion step or use the subprocess module to run the shell program.
The end goal is to modify Firefox cookies.sqlite db before starting Firefox.
At present, I want to display what tables are in the db. I copied the cookies.sqlite db to my desktop. I'm working with the db on my desktop.
This is my first time using sqlite. I copied some code, but I'm not able to read what tables are in the db.
List of tables, db schema, dump etc using the Python sqlite3 API
Here is what I get in the terminal. I see the listing of the tables.
me $ ls cookies.sqlite
cookies.sqlite
me $ sqlite3 cookies.sqlite
SQLite version 3.8.10.2 2015-05-20 18:17:19
Enter ".help" for usage hints.
sqlite> .tables
moz_cookies
sqlite> .exit
me $ # I run my python script
me $ ./sqlTutorial.bash
SQLite version: 3.8.10.2
table records...
[]
more table records...
me $ cat dump.sql
BEGIN TRANSACTION;
COMMIT;
me $
python code. All I want to do is display the tables in the cookie.sqlite db.
#! /bin/python
import sqlite3 as lite
import sys
con = lite.connect('cookie.sqlite')
with con:
cur = con.cursor()
cur.execute('SELECT SQLITE_VERSION()')
data = cur.fetchone()
print ("SQLite version: %s" % data)
print ("table records...")
cursor = con.cursor()
cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")
print(cursor.fetchall())
print ("more table records...")
with open('dump.sql', 'w') as f:
for line in con.iterdump():
f.write('%s\n' % line)
As you noticed, the database name is cookies.sqlite not cookie.sqlite.
Why didn't I get an error? I though python would error out on. con = lite.connect('cookie.sqlite')
Connecting to a sqlite database either opens an existing database file, or creates a new one if the database didn't exist.
My simple test code is listed below. I created the table already and can query it using the SQLite Manager add-in on Firefox so I know the table and data exist. When I run the query in python (and using the python shell) I get the no such table error
def TroyTest(self, acctno):
conn = sqlite3.connect('TroyData.db')
curs = conn.cursor()
v1 = curs.execute('''
SELECT acctvalue
FROM balancedata
WHERE acctno = ? ''', acctno)
print v1
conn.close()
When you pass SQLite a non-existing path, it'll happily open a new database for you, instead of telling you that the file did not exist before. When you do that, it'll be empty and you'll instead get a "No such table" error.
You are using a relative path to the database, meaning it'll try to open the database in the current directory, and that is probably not where you think it is..
The remedy is to use an absolute path instead:
conn = sqlite3.connect('/full/path/to/TroyData.db')
You need to loop over the cursor to see results:
curs.execute('''
SELECT acctvalue
FROM balancedata
WHERE acctno = ? ''', acctno)
for row in curs:
print row[0]
or call fetchone():
print curs.fetchone() # prints whole row tuple
The problem is the SQL statment. you must specify the db name and after the table name...
'''SELECT * FROM db_name.table_name WHERE acctno = ? '''