This code runs perfectly fine if I first redirect to the folder where this main.py file is located. So in cmd I just type: python main.py. In this folder is also my database "Datalog.db" located.
If I run this python file from somewhere else, I get a problem with this line of code: cur.execute(sql). So in cmd I type: python C:\Users\ [...] \main.py. I get following error: "sqlite3.OperationalError: no such table: Datalog". Later I want to include this python file in a pythonshell in node-red and there I have to define the full path of this main.py
I also tried to build an exe-file with it but then the same error occurs:"sqlite3.OperationalError: no such table: Datalog".
Apparently the connection to the database is not the issue, first my cur.execute command is not working.
I find out that I have to "include my SQLite database file in the include_files statement", but I have no idea how to do this ..
Can anybody help? I am very sorry for any inconvenience, I just started programming and this is my first post.
import sqlite3 as db
db_name = 'Datalog'
output_number = 'Output1'
output = 'hello'
timestamp = '2019-11-11 09:27:02'
db_name = f'{db_name}.db'
con = db.connect(db_name)
with con:
cur = con.cursor()
sql = f"UPDATE Datalog SET {output_number}='{output}' WHERE timestamp ='{timestamp}'"
cur.execute(sql)
con.commit()
print("### DB updated ###")
You can change db_name to the full path of your sqlite database or even better, dynamically build the path to the db (Below code assumes the db is in the same folder (directory) as the file which calls the below code ):
import os.path
db_name = os.path.dirname(os.path.abspath(__filename__)) + 'Datalog'
python looks for files in "sys.path" so you want to insert the name of the directory containing your file in that path so python can find it
here is an example:
import sys
sys.path.insert(0, r'C:\Users\Philip\Work\Bin')
where 0 is the position in the sys.path, 0 means top, 1 means second from top etc.
obviously you would replace my directory with yours
Related
I am trying to load a .csv file through mysql.connector in python. Here is the relevant portion of the script:
import os
print(os.listdir(r'C:\ProgramData\MySQL\MySQL Server 8.0\Uploads')) #returns test.csv
import_file = 'C:\\ProgramData\\MySQL\\MySQL Server 8.0\\Uploads\\test.csv'
cnx = mysql.connector.connect(user=usr,
password=passwd,
host='127.0.0.1',
database='ctr')
cursor = cnx.cursor()
load_data = (
"LOAD DATA "
"INFILE "
"'{}' "
"INTO TABLE import "
"FIELDS TERMINATED BY ',' "
"""OPTIONALLY ENCLOSED BY '"' """
"LINES TERMINATED BY '\\n' "
"IGNORE 1 LINES".format(import_file))
securefile_query = ("SHOW VARIABLES LIKE 'secure_file_priv'")
cursor.execute(securefile_query)
directory = cursor.fetchall()
print(directory) #returns C:\ProgramData\MySQL\MySQL Server 8.0\Uploads
print(import_file)
print(load_data)
cursor.execute(load_data)
Traceback of the error that follows points to cursor.execute(load_data) in my script, and the last line of the traceback is "mysql.connector.errors.DatabaseError: 1290 (HY000): The MySQL server is running with the --secure-file-priv option so it cannot execute this statement".
According to the secure_file_priv documentation:
If empty, the variable has no effect. This is not a secure setting.
If set to the name of a directory, the server limits import and export operations to work only with files in that directory. The directory must exist; the server does not create it.
If set to NULL, the server disables import and export operations.
I have confirmed multiple times that the file is in the correct directory. I have also confirmed the value of secure_file_priv using "SHOW VARIABLES". Doesn't this mean MySQL ought to be able to read test.csv from the Uploads folder? How can I read the file without disabling secure_file_priv?
Edit 17Jan2022:
Since I'm running the server locally and have the files locally, I ended up using LOAD DATA LOCAL INFILE as a workaround by adding "allow_local_infile=True" as a parameter inside cnx and then moving the source file to the same directory as the script. Setting secure-file-priv to an empty string (C:\ProgramData\MySQL\MySQL Server 8.0\my.ini) also worked.
I still don't know why I was receiving "server is running with the --secure-file-priv option" given the previous location of my files.
I want to connect my SQLite database/ or I think I'm connecting my "data.sqlite" (the blue highligted one in the picture below) with my views.py in the core folder (picture below) with:
rPath='../../Upchanges/data.sqlite'
conn = _sqlite3.connect(rPath, check_same_thread=False)
c = conn.cursor()
Path of the database I want to connect with:
Upchanges_desperate/Upchanges/data.sqlite
Path of the Python file I want to connect with the database:
Upchanges_desperate/Upchanges/core/views.py
Picture of my file orientation:
So this is my problem:
I later use these codes below in my python file:
from Upchanges.models import BlogPost( I set the __tablename__ to 'blog_post' in another python file)
#core.route('/', methods=['GET', 'POST'])
def index():
search = Blogsearch_form(request.form)
if request.method == 'POST':
c.execute("SELECT * FROM blog_post WHERE problem_name LIKE(?)", ('%' + str(search) + '%',))
results = c.fetchall()
print(results)
return render_template('blog_search_result.html', results=results)
page = request.args.get('page',1,type=int)
many_posts = BlogPost.query.order_by(BlogPost.date.desc()).paginate(page=page, per_page=10)
return render_template('index.html', many_posts=many_posts, form=search)
When I input something and click enter in the Blogsearch_form, my website shows this error: "sqlite3.OperationalError: no such table: blog_post". I also tried replacing blog_post with BlogPost but it shows the same error: "sqlite3.OperationalError: no such table: BlogPost"
I promise you I have a blog_post table in my data.sqlite database. I use it also in some codes below and my website query the data and work just fine (show everything later on in the HTML file):
page = request.args.get('page',1,type=int)
many_posts = BlogPost.query.order_by(BlogPost.date.desc()).paginate(page=page, per_page=10)
return render_template('index.html', many_posts=many_posts, form=search)
Addtionally, for more clarity, these codes below (which is in the init.py of my app.py file) are those that create my data.sqlite database:
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///' + os.path.join(basedir,
'../Upchanges/data.sqlite')
So, I really think that something is wrong with the path in the _sqlite3.connect command. I would greatly appreciate if you could help me fix the problem above.
(Please use the picture when you think about the path as it is easier to imagine)
Thank you!!
P.s/ I tried this from my previous question: https://stackoverflow.com/a/61849867/13097721 but I still couldn't fix this problem.
In fact you do not necessarily know from which directory the application has been started from.
You could test this with
import os
print("My current working directory is", os.getcwd())
However if your database is located relative to a python file, then it is better to determine the path relative to the python file's directory than trying to determine it relative to the current working directory.
You can do this for example with with
import os
MYDIR = os.path.dirname(__file__)
SQLPATH = os.path.join(MYDIR, "..", "..", "data.sqlite")
I also suggest to add following line in your existing code or in your modified code.
It will show you exactly where you point to
print("THE REAL SQLPATH is", os.path.realpath(SQLPATH))
Comment after your feedback:
I Will add more prints (and remove one "..") but you will see what you need to do with two more prints.
import os
print("MY FILE = ", os.path.realpath(__file__))
MYDIR = os.path.dirname(__file__)
print("MYDIR = ", os.path.realpath(MYDIR))
SQLPATH = os.path.join(MYDIR, "..", "data.sqlite")
print("This gives me SQLPATH = ", os.path.realpath(SQLPATH))
Comment after next Feedback
So it seems how, that the database file is now properly located.
I suggest to inspect your database.
from a shell window type:
sqlite3 /Users/kienletrung/Desktop/Upchanges_desperate/Upchanges/data.sqlite
type then commands like
.tables to list all tables .schema <tablename> to look at a tables format. or select * from tablename; to look at all rows of a table.
alternatively just type:
echo .dump | sqlite3 /Users/kienletrung/Desktop/Upchanges_desperate/Upchanges/data.sqlite > fulldump.txt
to create a full ascii dump of your database.
If you don't have the sqlite command line client installed then just install it. It should simplify debugging.
You can of course use any other tool to inspect your data base in look at what tables / structures your python code expects and what the database really contains.
My DB is located here:
D:\sqlite\db
My Python Code is located here:
D:\app\ZAKPRO\R20191121
Code looks this way:
import sqlite3
import os.path
conn = sqlite3.connect('D:\sqlite\db\db_name.db')
c = conn.cursor()
c.execute("INSERT INTO sts_meta (symbol) VALUES( 'tra' ) ")
Error-Message is:
OperationalError: no such table: sts_meta
Can anyone help to correct my code?
I separate DB and Code in different location: Is that fine and advisable?
I only find examples with relative path: Is the access with absolute path not the prefered one in python-sqlite-community? Why is that?
UPDATE
Now I have re-coded, but still facing the same issue:
filename = "dbname.db"
dir = "D:\sqlite\db"
dbpath = pathlib.Path(dir, filename)
sql = ("SELECT * from sts_meta")
conn = sqlite3.connect(dbpath)
c = conn.cursor()
c.execute(sql)
print(c.fetchall())
conn.commit()
This message I receive:
OperationalError: no such table: sts_meta
How to find out wheather I am connected to a database?
Regarding pathlib this is my db-path:
D:\sqlite\db\stocktimeseries.db
It's not that good to use a relative path.
Good practice would be to use path.join from os package in python.
And at any string which contains any characters like backslashes use r'your string'. Adding r before your string will state that the special characters contained in the following string is just a string and they serve no other purpose.
I'm not sure if that is what causes this issue. But if the table exists and you are having issues with path, then path.join can help you out.
I'm trying to export a table, contained within an Oracle 12c database, to csv format - using Python 2.7. The code I have written is shown below:
import os
import cx_Oracle
import csv
SQL = 'SELECT * FROM ORACLE_TABLE'
filename = 'C:\Temp\Python\Output.csv'
file = open(filename, 'w')
output = csv.writer(file, dialect='excel')
connection = cx_Oracle.connect('username/password#connection_name')
cursor = connection.cursor()
cursor.execute(SQL)
for i in cursor:
output.writerow(i)
cursor.close()
connection.close()
file.close()
This code yields an error in the line where I define 'connection':
ORA-12557: TNS:protocol adapter not loadable
How can I remedy this? Any help would be appreciated.
Please note: I have already encountered StackOverflow responses to very similar problems to this. However, they often suggest changing the path within environment variables - I cannot do this since I don't have appropriate administer privileges. Thanks again for your assistance.
ORA-12557 is caused by problems with the %ORACLE_HOME% on Windows. That's the usual suggestion is to change the PATH setting.
"I cannot do this since I don't have appropriate administer privileges."
In which case you don't have too many options. Perhaps you could navigate to the ORACLE_HOME directory and run your script from there. Otherwise look to see what other tools you have available: Oracle SQL Developer? TOAD? SQL*Plus?
We found that by navigating to config -> Oracle and editing the file 'tnsnames.ora' the problem can be solved. The tnsnames file appears as follows:
connection_name =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS= ... )
)
(CONNECT_DATA =
(SERVICE_NAME= ...)
)
)
By changing the first instance of connection_name to connection_name.WORLD, then typing
set ORACLE_HOME=
into the command line before executing the Python script, the above script now runs with no error.
I use ini-file to store DB connection parameters. Hope it helps.
self.mydsn = cx_Oracle.makedsn(self.parser.get('oracle', 'db'),self.parser.get('oracle', 'port'),self.parser.get('oracle', 'service_name'))
try:
self.connpool = cx_Oracle.SessionPool(user=self.parser.get('oracle', 'username'),password=self.parser.get('oracle', 'userpass'),dsn=self.mydsn,min=1,max=5,increment=1)
except Exception as e:
print e
You can use this python script for oracle csv export:
https://github.com/teopost/csv_exp
I am creating a plugin for QGIS (QGIS Plugin Builder). The relative paths to the plugin directory need to start with a colon. The icon for example is loaded like this:
icon_path = ':/plugins/into_sqlite/icon.png'
When I try to do the same for the sqlite database the database fails to load:
connection = sqlite3.connect(":/plugins/PLUGINNAME/database.sqlite")
OperationalError: unable to open database file
But when I try the same giving the absolute path it works:
connection = sqlite3.connect("/home/USERNAME/.qgis/python/plugins/PLUGINNAME/database.sqlite")
OperationalError: unable to open database file
I think that the problem is that the path starts with a colon and sqlite uses the colon as a special character to for example load a database in memore:
connection = sqlite3.connect(":memory:")
But I need to use realtive paths. How can I supply the path and escape the colon or start the path without the colon?
I now went around the problem by using this code:
path = abspath(":/plugins/PLUGINNAME/")
path_abs = path.replace(":", ".qgis2/python")
path_db = join(path_abs, "db.sqlite")
connection = sqlite3.connect(path_db)
The variables have these values:
path = "/home/USERNAME/:/plugins/PLUGINNAME"
path_abs = "/home/USERNAME/.qgis2/python/plugins/PLUGINNAME"
path_db = "/home/USERNAME/.qgis2/python/plugins/PLUGINNAME/db.sqlite"
Another suggestion:
On IRC the suggestion was:
curr_path = os.path.dirname(os.path.abspath(__file__))
self.path_db = os.path.join(curr_path, 'db.sqlite')