I am building a Python app using FBS, but part of it relies on an SQLite3 database. I have code to create this database if it doesn't find this, using a try-catch block.
When I try to run it after compiling, it not only can not find the preexisting SQLite3 file, but also won't create it. It does not display any error messages.
I have tried creating the file if it doesn't exist using this code:
try:
self.connection = sqlite3.connect(path)
self.cursor = self.connection.cursor()
except:
if not os.path.exists(path):
# Try and make the .config directory
try:
os.makedirs(".config")
except OSError as e:
if e.errno != errno.EEXIST:
raise
# Create the datastore, and close it
f = open(path, "w+")
f.close()
# And try connect to database again
return self.__connect(path)
else:
print(f"No database exists, and could not create one.\nPlease create file in app directory called: {path}\nThen restart application.")
raise
The code works find in dev, but as soon as I compile it to a Mac app, it refuses to find or create the database.
Fixed. If anyone has a similar issue please use the builtin appctxt.get_resource(file_path) method.
Related
I run a python script which do DB operations via psycopg2. But I'm facing the following error:
<class psycopg2.errors.InFailedSqlTransaction>
can't connect DB! could not create socket: Too many open files
I actually also don't have too many open files, I close always the files after reading or writing and I even don't do too many file operations.
Here my code where I do file operations:
with open(config_path) as f:
configs = json.load(f)
return configs[name]
and here:
path = getmyPath()
if not os.path.isfile(path):
return None
with open(path) as f:
data = f.read()
To solve the problem I always need to restart my script but it's in my case not a perfect solution, my script should be run for one month.
I also changed ulimit to 50000 but it still doesn't help me.
How can I solve this problem?
Here is my class for DB-Ops:
import psycopg2
class DBOps:
def __init__(self):
conString = getCons()
try:
self.connection = psycopg2.connect(conString)
self.cr = self.connection.cursor()
except (Exception, psycopg2.Error) as error:
print("can't connect DB!", error)
def select(self, query, onerow=False):
self.cr.execute(query)
records = self.cr.fetchall()
if onerow:
for row in records:
return row
#self.cr.close()
#self.connection.close()
return records
def exec(self, query):
print(query)
self.__init__()
self.cr.execute(query)
self.connection.commit()
#self.cr.close()
#self.connection.close()
return self.cr.rowcount
def close(self):
self.cr.close()
self.connection.close()
When I insert or make an update, I do following:
db = DBOps()
db.select(myQuery)
db.exec(myQuery)
Do you ever call DBOps.close()? It might be a good idea to adopt the resource manager protocol in the DBOps class and use a with statement.
I was wondering if there was a way to connect to my sqlite DB in Python and push that data to the cloud in DBHub.io.
The reason for this is that I already have my DB in DBBrowser locally and now I want to push it into the cloud.
You might try the library pydbhub, published by the DBHub.io team at their section Libraries for accessing DBHub.io via API.
The library contains a section called "Upload database" at their Further examples, that points to this main.py file.
For historical researching in StackOverflow, this is how the example looks like today:
import sys
import os
import datetime
from platform import python_version
# https://github.com/willmcgugan/rich
from rich.console import Console
from rich.theme import Theme
import pydbhub.dbhub as dbhub
if __name__ == '__main__':
custom_theme = Theme({
"info": "green",
"warning": "yellow",
"error": "bold red"
})
console = Console(theme=custom_theme)
if python_version()[0:3] < '3.7':
console.print(
"[ERROR] Make sure you have Python 3.7+ installed, quitting.\n\n", style="error")
sys.exit(1)
# Create a new DBHub.io API object
db = dbhub.Dbhub(config_file=f"{os.path.join(os.path.dirname(__file__), '..', 'config.ini')}")
# Prepare any information you want to include with the upload (eg a commit message, etc)
info = dbhub.UploadInformation(
commitmsg="An example upload",
committimestamp=datetime.datetime.fromisoformat("2021-06-01 10:00:00"),
)
try:
# Read the database file
myDB = os.path.join(os.getcwd(), "examples", "upload", 'example.db')
f = open(myDB, 'rb')
except FileNotFoundError:
print(f"File {myDB} not found. Aborting")
sys.exit(1)
except OSError:
print(f"OS error occurred trying to open {myDB}")
sys.exit(1)
except Exception as err:
print(f"Unexpected error opening {myDB} is", repr(err))
sys.exit(1)
else:
with f:
# Upload the database
res, err = db.Upload(db_name='somedb.sqlite', info=info, db_bytes=f)
if err is not None:
console.print(f"[ERROR] {err}", style="error")
sys.exit(1)
console.print(f"Database uploaded, commit: {res['commit']}", style="info")
HTH
I wanted to set the environment variables in the unix for my python scrippting project that uses cx_Oracle to connect with the database and have CRUD operations.
i have used os.environ to set the environment variables for oracle. all the libraries are present in the corresponding directory.
This is the method i have used to set the environment variables is unix
def set_environment():
os.environ["TNS_ADMIN"]="/opt/oracle/orafmw/product/11.2.0.1/client_1/network/admin"
os.environ["ORACLE_HOME"] = "opt/oracle/orafmw/product/11.2.0.1/client_1"
os.environ["LD_LIBRARY_PATH"] = "/opt/oracle/orafmw/product/11.2.0.1/client_1/lib"
os.environ["PATH"] = "$PATH:/opt/oracle/orafmw/product/11.2.0.1/client_1/bin:."
and i have called the method from the main method of my script.
def get_connect_string():
return db_username+'/'+password+'#'+host+':'+port+'/'+service_name
def main():
import os
import cx_Oracle
set_environment()
query = "SELECT * FROM SITE WHERE SITE_CODE = :1"
try:
connect_string = get_connect_string()
conn = cx_Oracle.connect(connect_string)
cur = conn.cursor()
d = cur.execute(query, ["AUS"]).fetchone()
conn.commit()
if d:
data = (([i[0] for i in cur.description]), d)
else:
data = None
except Exception as e:
print("error in operation : ", e)
conn.rollback()
finally:
conn.close()
print(data)
The error message i am getting is:
Error while trying to retrieve text for error ORA-01804
The LD_LIBRARY_PATH environment variable cannot be set from within your application. It must be set before the process is started! Otherwise, it will not take effect. The error you are getting indicates that the environment isn't set properly. Try setting the environment variables before running your script and see if that resolves the issue for you!
I have the following line code in a Python script:
sql_create_table(total_stores, "total_stores")
This is a function I created to upload tables to an Oracle database. I want to do something like this in order to record what tables were not created because the line failed to run:
Try:
sql_create_table(total_stores, "total_stores")
except:
print in a log.txt "table x could not be created in the database"
Any suggestions?
Thanks in advance!
There is the python logging module, which has a good tutorial, which even includes how to log to a file.
Very basic example:
import logging
logging.basicConfig(filename="program.log", level=logging.INFO)
…
try:
sql_create_table(total_stores, "total_stores")
except:
logging.warning("table x could not be created in the database")
You can write the log to a txt file by doing the following:
Try:
sql_create_table(total_stores, "total_stores")
except:
with open('log.txt', 'a') as log:
log.write("table x could not be created in the database")
Note, that by using 'a', we are appending to the txt file and won't be overwriting old logs.
I'm trying to export a table, contained within an Oracle 12c database, to csv format - using Python 2.7. The code I have written is shown below:
import os
import cx_Oracle
import csv
SQL = 'SELECT * FROM ORACLE_TABLE'
filename = 'C:\Temp\Python\Output.csv'
file = open(filename, 'w')
output = csv.writer(file, dialect='excel')
connection = cx_Oracle.connect('username/password#connection_name')
cursor = connection.cursor()
cursor.execute(SQL)
for i in cursor:
output.writerow(i)
cursor.close()
connection.close()
file.close()
This code yields an error in the line where I define 'connection':
ORA-12557: TNS:protocol adapter not loadable
How can I remedy this? Any help would be appreciated.
Please note: I have already encountered StackOverflow responses to very similar problems to this. However, they often suggest changing the path within environment variables - I cannot do this since I don't have appropriate administer privileges. Thanks again for your assistance.
ORA-12557 is caused by problems with the %ORACLE_HOME% on Windows. That's the usual suggestion is to change the PATH setting.
"I cannot do this since I don't have appropriate administer privileges."
In which case you don't have too many options. Perhaps you could navigate to the ORACLE_HOME directory and run your script from there. Otherwise look to see what other tools you have available: Oracle SQL Developer? TOAD? SQL*Plus?
We found that by navigating to config -> Oracle and editing the file 'tnsnames.ora' the problem can be solved. The tnsnames file appears as follows:
connection_name =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS= ... )
)
(CONNECT_DATA =
(SERVICE_NAME= ...)
)
)
By changing the first instance of connection_name to connection_name.WORLD, then typing
set ORACLE_HOME=
into the command line before executing the Python script, the above script now runs with no error.
I use ini-file to store DB connection parameters. Hope it helps.
self.mydsn = cx_Oracle.makedsn(self.parser.get('oracle', 'db'),self.parser.get('oracle', 'port'),self.parser.get('oracle', 'service_name'))
try:
self.connpool = cx_Oracle.SessionPool(user=self.parser.get('oracle', 'username'),password=self.parser.get('oracle', 'userpass'),dsn=self.mydsn,min=1,max=5,increment=1)
except Exception as e:
print e
You can use this python script for oracle csv export:
https://github.com/teopost/csv_exp