So I'm using this script to save temp and humidity data on my raspberry pi, the relevant part looks something like this:
import sqlite3 as db
con = db.connect('hygolog.db')
cur = con.cursor()
#This part is commented because I used it when running the first time
#cur.execute("CREATE TABLE measure(time STR PRIMARY KEY, humidity REAL, tempertaure REAL)")
#con.commit()
while True:
cur.execute(f"""INSERT INTO measure
VALUES(datetime('now','localtime'),{round(humidity,3)},{round(temperature,3)})""")
con.commit()
And this works just fine when executing the file from the code editor (i checked and data is found in the DB after), but when I execute the file from my terminal I get the following error:
user#raspberrypi:~ $ python Desktop/hygometer.py
Traceback (most recent call last):
File "/home/user/Desktop/hygometer.py", line 19, in <module>
cur.execute(f"""INSERT INTO measure
sqlite3.OperationalError: no such table: measure
which is also a weird place to fail, why should it not find the very much existing table when executed through the terminal?
try specifying full path to hygolog.db, for example C:/Files/hygolog.db. When you run this from terminal, you could be running from a different folder, so it creates a new empty hygolog.db in that folder.
Related
I have already looked at these (and others):
disk I/O error with SQLite3 in Python 3 when writing to a database
Error: disk I/O error on a newly created database
Might be similar/related to SQLite disk I/O error (3850)
I am parsing an iOS full extraction image (about 61G) from zip file. The file is copied from the zip into a temp folder and is read from there. I do not create the database nor do anything else to it.
So running this code errors inside of VMware 16.2.1 using a Windows 10 Host and a Windows 10 virtual machine. If I run this directly on the host, it will work fine -- change "Z:\" to "D:\VM Shares" + the rest of the path.
import sqlite3
from pathlib import Path
>>> file_path = Path('Z:\\Forensics\\CTF21_Marsha_iPhoneX_FFS_Premium_2021_07_29\\xLEAPP_Reports_2021-11-23_Tuesday_115054\\temp\\filesystem2\\containers\\Shared\\SystemGroup\\30CCFC92-08E6-458A-B78B-DA920EF2EF82\\Library\\Database\\com.apple.MobileBluetooth.ledevices.other.db')
>>> db = sqlite3.connect(f'file:{file_path}?mode=ro', uri=True)
>>> cursor = db.cursor()
# The line below provides a method to check: Did I just open a sqliteDB? Errors if not
# a sqliteDB. I have seen that you can basically open a text file through the connect()
# statement and get a cursor. So this just a check before other code would run.
>>> cursor.execute("PRAGMA page_count").fetchone()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
sqlite3.OperationalError: disk I/O error
Now, if I change the connect() statement to the following:
>>> db = sqlite3.connect(f'file:{file_path}?mode=ro&immutable=1', uri=True)
You can read about immutable object on SQL Uniform Resource Identifiers (section 3.3. This seems to work leading me to believe the DB is locked somehow.
I have reboot the virtual machine and rebooted the host. I did notice copying the DB in the same directly (sometimes) works but copying to another directly (even one higher) always works.
I thought it might be done to the length of the path name. I've tried added "\?", "\\?\", or other variations for the longer paths but this does not work. The path length is under 256 anyway.
This is related to xLEAPP with the problematic code located in abstract.py.
Added a issue on Github for it - xleapp#5
Overview:
I have a function in an external file that returns a dataframe that I then save to a csv with:
df.to_csv('filepath.csv', na_rep="0", index=False)
I then try to import the csv into a postgres table using the pyscopg2 function copy_from:
try:
connect = psycopg2.connect(database = "", user = "", password = "", host = "", port = "")
except:
print("Could not connect to database")
cursor = connect.cursor()
with open("filepath", 'r') as open_csv:
next(open_csv)
try:
cursor.copy_from(open_csv, sep=",")
connect.commit()
print("Copy Complete")
except:
print("Copy Error")
cursor.close()
This results in a copy error exception in the code above (so no real detail) but there are some weird caveats:
For some reason, if I open the csv in libre office and manually save it as a text csv and then run just the above psycopg2 copy_from process, the copy works and there are no issues. So for whatever reason, in the eyes of psycopg2 copy_from, something is off with the to.csv() write that gets fixed if I just manually save the file. Manually saving the csv does not result in any visual changes so what is happening here?
Also, the above psycopg2 code snippet works without error in another file in which all dataframe manipulation is contained within the single file where the to.csv() is completed. So something about returning a dataframe from a function in an external file is off?
Fwiw, when debugging, the issue came up on the .copy_from() line so the issue has something to do with csv formatting and I cannot figure it out. I found a workaround with sqlalchemy but would like to know what I am missing here instead of ignoring the problem.
In the postgres error log, the error is: "invalid input syntax for type integer: "-1.0". This error is occurring in the last column of my table where the value is set as an INT and in the csv, the value is -1 but it is being interpreted as -1.0. Where I am confused is that if I use a COPY query to directly input the csv file into postgres, it does not have a problem. Why does it interpret the value as -1.0 through psycopg2 but not directly in postgres?
This is my first post so if more detail is needed let me know - thanks in advance for the help
I have a django & docker server running on my computer and I have created a database with code from outside this server. I am trying to access this database ('test.sqlite3') within the server.
I made sure the path was the correct one and that the file name was correct as well. When I open the database with DB Browser, I can see the tables and all my data. But I still get the following error text:
OperationalError no such table: NAMEOFTABLE
When I use the exact same code from another python IDE (spyder) it works fine. I'm guessing there's something weird going on with django?
Here is some of the code:
conn = sqlite3.connect("../test.sqlite3")
c = conn.cursor()
c.execute("SELECT firstName, lastName FROM RESOURCES")
conn.close()
(Yes, I have also tried using the absolute path and I get the same error.)
Also to be noted: I get this same error when I try to create the database file & table from within the django code (the path should then be the same but it still get the error in this case).
Update: it seems I have a problem with my path because I can't even open a text file with python and it's absolute path. So if anyone has any idea why that'd be great.
try:
f = open("/Users/XXXXX/OneDrive/XXXXX/XXXX/Autres/argon-dashboard-django/toto.txt")
# Do something with the file
except IOError:
q="File not accessible"
finally:
f.close()
always return the following error 'f referenced before assignment' and q = "File not accesible" so that means I can't even find the text file.
To answer this problem, I used two things:
I moved the sqlite3 file within the app folder and used '/app/db.sqlite3' as the path
Added ; at the ends of my SQL requests
c.execute("SELECT firstName, lastName FROM RESOURCES;")
Not sure which one solved the problem but everything works for me now.
Had a similar issue, possibly something about leaving Django Model metadata files outside of the image. I needed to synchronize the model with the DB using a run-syncdb
RUN ["python", "manage.py", "migrate"]
RUN ["python", "manage.py", "migrate", "--run-syncdb"]
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
I'm getting the following error
conn = sqlite3.connect('./mydb.db')
c = conn.cursor()
c.execute('.output ./mytable.sql')
conn.close()
c.execute('.output ./mytable.sql') sqlite3.OperationalError: near ".":
syntax error
That's because .output is a command for the command line sqlite tool. It is not a valid SQL command. Hence it cannot be used when you are using sqlite through a library, only interactively through the command prompt.
None of the shell commands listed at https://www.sqlite.org/cli.html can work as they are something totally separate from the sqlite itself. You can think of them as if they were part of a GUI program - it would not make sense to be able to access something in a GUI program through the library.
What you have to do is fetch the data yourself and parse it yourself and output in the way you want.
Another option is to call the sqlite shell and pass the commands you want it to execute. Something like:
sqlite3 < '.output FILE \n SELECT * FROM TABLE'
(this is untested...)
I have created a sqlite db and uploaded it to a hosting.
Then I'm retrieving it from my script and trying to insert some data, but execute() is returning a
DatabaseError (file is encrypted or is not a database).
urllib.urlretrieve('http://%s/%s' % (HOST, NAME_DB), NAME_DB)
con = sqlite3.connect(NAME_DB)
cur = con.cursor()
cur.execute('insert into log(date, count, average) values(date("now"), ?, ?)', (1, 1.2))
con.commit()
con.close()
Traceback (most recent call last):
File "mylog.py", line 17, in <module>
cur.execute('insert into log(date, count, average) values(date("now"), ?, ?)', (1, 1.2))
sqlite3.DatabaseError: file is encrypted or is not a database
Such error doesn't happen if I use the sqlite CLI to insert data. Could you please help me?
Version mismatch between sqlite CLI and python sqlite API? I created again my db from the script instead of the CLI. Now insert and select work from the script, but not from the CLI. $sqlite -version returns 2.8.17, while the python version is 2.7.3.
I had the same problem with a database created by C++ code using SQLite3 library which was later accessed by Python 2.7 version of SQLite3. I was unable to query the database in Python scripts. To solve the problem on my computer, I changed the version of :
C:\Python27\DLLs\sqlite3.dll
for the version found in C++ Sqlite library directory.
Okay I faced the same problem and as Visionnaire said Just replace the sqlite3.dll in the pythonXX\DLLs with sqlite3.dll in the CLI sqllite folder which originally contains sqlite3.exe and the problem was solved
I had the same problem, and I thought it's something wrong with the sqlite3 db I was working on. But it appeared, that I've accidentally overrode another sqlite3.db file (that was present in my project), as an ASCII. I was not aware that the error was coming from another db.
Pay attention :)