I have been trying to use StreamLit for a little website project I have going on and I keep getting this error:
OSError: [Errno 36] File name too long
Here is the coding
uploaded_files = st.file_uploader("Choose a file", accept_multiple_files=True)
for uploaded_file in uploaded_files:
bytes_data = uploaded_file.read()
s = bytes_data.decode("UTF-8")
SimpleDirectoryReader = download_loader("SimpleDirectoryReader")
loader = SimpleDirectoryReader(s)
documents = loader.load_data()
index = GPTSimpleVectorIndex(documents)
index.save_to_disk('index.json')
question = st.text_input("What do you want me to do with the file uploaded?")
response = index.query(question)
st.write(response)
Here is the full error:
File "/home/appuser/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
exec(code, module.__dict__)
File "/app/indextest/streamlit_app.py", line 17, in <module>
loader = SimpleDirectoryReader(s)
File ".modules/file.py", line 75, in __init__
self.input_files = self._add_files(self.input_dir)
File ".modules/file.py", line 81, in _add_files
input_files = sorted(input_dir.iterdir())
File "/usr/local/lib/python3.9/pathlib.py", line 1160, in iterdir
for name in self._accessor.listdir(self):
You are passing the entire content of the file as an argument to SimpleDirectoryReader. However, as you can see in the source code of the SimpleDirectoryReader class, the first argument should be a path to a directory:
input_dir (str): Path to the directory.
You should store the uploaded_files inside a folder, and then give this folder as an argument to SimpleDirectoryReader like this:
import os
save_folder = "files"
if not os.path.exists(save_folder):
os.makedirs(save_folder)
uploaded_files = st.file_uploader("Choose a file", accept_multiple_files=True)
for uploaded_file in uploaded_files:
bytes_data = uploaded_file.read()
s = bytes_data.decode("UTF-8")
# save the file
with open(f"{save_folder}/{uploaded_file.name}", "w") as f:
f.write(s)
# Then, once you have stored the files, load the SimpleDirectoryReader
SimpleDirectoryReader = download_loader("SimpleDirectoryReader")
loader = SimpleDirectoryReader(save_folder)
documents = loader.load_data()
Related
MySettings
from decouple import config
GOOGLE_DRIVE_STORAGE_JSON_KEY_FILE = config("GOOGLE_DRIVE_STORAGE_JSON_KEY_FILE_CONTENTS")
I'm trying to setup gdstorage for my django project but was not given a json file according to the instruction in the docs so i copied the keys and placed it in my environmental variable yet it's still throwing this error.
MyErrorMessage
gd_storage = GoogleDriveStorage()
File "C:\Users\username\.virtualenvs\vicsite-3EqYD9rF\lib\site-packages\gdstorage\storage.py", line 160, in __init__
credentials = Credentials.from_service_account_file(
File "C:\Users\username.virtualenvs\vicsite-3EqYD9rF\lib\site-packages\google\oauth2\service_account.py", line 238, in from_service_account_file
info, signer = _service_account_info.from_filename(
File "C:\Users\username.virtualenvs\vicsite-3EqYD9rF\lib\site-packages\google\auth\_service_account_info.py", line 72, in from_filename
with io.open(filename, "r", encoding="utf-8") as json_file:
FileNotFoundError: [Errno 2] No such file or directory: '78a23bb0124ac97ec7404104ae37dccffe33846a'
Try this
GOOGLE_DRIVE_STORAGE_JSON_KEY_FILE = None
GOOGLE_DRIVE_STORAGE_JSON_KEY_FILE_CONTENTS = config("GOOGLE_DRIVE_STORAGE_JSON_KEY_FILE_CONTENTS")
More details are in https://pypi.org/project/django-googledrive-storage/
I just started learning python so this might be a very basic question but here's where I'm stuck.
I'm trying to parse ALL XML files in a given folder and outputting CSV files, with the same filename as the original XML files. I've tested with single files and it works perfectly but the issue I'm having is with performing the same for all of them and having that running on a loop as it would be a perpetual script.
Here my code:
import os
import xml.etree.cElementTree as Eltree
import pandas as pd
path = r'C:/python_test'
filenames = []
for filename in os.listdir(path):
if not filename.endswith('.xml'): continue
fullname = os.path.join(path, filename)
print(fullname)
filenames.append(fullname)
cols = ["serviceName", "startDate", "endDate"]
rows = []
for filename in filenames:
xmlparse = Eltree.parse(filename)
root = xmlparse.getroot()
csvoutput=[]
for fixed in root.iter('{http://www.w3.org/2001/XMLSchema}channel'):
channel = fixed.find("channelName").text
for dyn in root.iter('programInformation'):
start = dyn.find("publishedStartTime").text
end = dyn.find("endTime").text
rows.append({"serviceName": channel, "startDate": start, "endDate": end})
df = pd.DataFrame(rows, columns=cols)
df.to_csv(csvoutput)
This is the error I'm getting:
C:/python_test\1.xml
C:/python_test\2.xml
C:/python_test\3.xml
C:/python_test\4.xml
C:/python_test\5.xml
C:/python_test\6.xml
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 49, in <module>
File "C:\Users\ragehol\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\core\generic.py", line 3466, in to_csv
return DataFrameRenderer(formatter).to_csv(
File "C:\Users\ragehol\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\io\formats\format.py", line 1105, in to_csv
csv_formatter.save()
File "C:\Users\ragehol\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\io\formats\csvs.py", line 237, in save
with get_handle(
File "C:\Users\ragehol\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\io\common.py", line 609, in get_handle
ioargs = _get_filepath_or_buffer(
File "C:\Users\ragehol\AppData\Local\Programs\Python\Python39\lib\site-packages\pandas\io\common.py", line 396, in _get_filepath_or_buffer
raise ValueError(msg)
ValueError: Invalid file path or buffer object type: <class 'list'>
Any kind of suggestions would be greatly appreciated!
Many thanks!
This is you bug:
csvoutput=[] is defined as a list. Later on you pass it as argument to df.to_csv(csvoutput). So you are passing a list to a method that looks for a file path.
I would like to decompress a bunch of .bz2 files contained in a folder (where there are also .zst files). What I am doing is the following:
destination_folder = "/destination_folder_path/"
compressed_files_path="/compressedfiles_folder_path/"
dirListing = os.listdir(compressed_files_path)
for file in dirListing:
if ".bz2" in file:
unpackedfile = bz2.BZ2File(file)
data = unpackedfile.read()
open(destination_folder, 'wb').write(data)
But I keep on getting the following error message:
Traceback (most recent call last):
File "mycode.py", line 34, in <module>
unpackedfile = bz2.BZ2File(file)
File ".../miniconda3/lib/python3.9/bz2.py", line 85, in __init__
self._fp = _builtin_open(filename, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'filename.bz2'
Why do I receive this error?
You must be sure that all the file paths you are using exist.
It is better to use the full path to the file being opened.
import os
import bz2
# this path must exist
destination_folder = "/full_path_to/folder/"
compressed_files_path = "/full_path_to_other/folder/"
# get list with filenames (strings)
dirListing = os.listdir(compressed_files_path)
for file in dirListing:
# ^ this is only filename.ext
if ".bz2" in file:
# concatenation of directory path and filename.bz2
existing_file_path = os.path.join(compressed_files_path, file)
# read the file as you want
unpackedfile = bz2.BZ2File(existing_file_path)
data = unpackedfile.read()
new_file_path = os.path.join(destination_folder, file)
with bz2.open(new_file_path, 'wb') as f:
f.write(data)
You can also use the shutil module to copy or move files.
os.path.exists
os.path.join
shutil
bz2 examples
Im trying to develop a program that can iterate over different files in the same folder. The files are all the same format but will have different names. Right now if there is only 1 file in the folder the code executes with no problems but with different files i get the error:
Traceback (most recent call last):
File "D:/Downloads/FYP/Feedback draft.py", line 24, in <module>
wb = openpyxl.load_workbook(filename)
File "C:\Users\shomi\AppData\Local\Programs\Python\Python38-32\lib\site-packages\openpyxl\reader\excel.py", line 315, in load_workbook
reader = ExcelReader(filename, read_only, keep_vba,
File "C:\Users\shomi\AppData\Local\Programs\Python\Python38-32\lib\site-packages\openpyxl\reader\excel.py", line 124, in __init__
self.archive = _validate_archive(fn)
File "C:\Users\shomi\AppData\Local\Programs\Python\Python38-32\lib\site-packages\openpyxl\reader\excel.py", line 96, in _validate_archive
archive = ZipFile(filename, 'r')
File "C:\Users\shomi\AppData\Local\Programs\Python\Python38-32\lib\zipfile.py", line 1251, in __init__
self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: 'tester2.xlsx'
The code im using is :
directory = r'D:\Downloads\FYP\TEST'
for filename in os.listdir(directory):
if filename.endswith(".xlsx"):
wb = openpyxl.load_workbook(filename)
sh1=wb['test']
doc = DocxTemplate('Assignment1feedback.docx')
context = {
'acc': acceleration
}
doc.render(context)
doc.save('D:\\Downloads\\FYP\\TEST\\' + filename + '.docx')
This is incomplete code as the full thing would be quite long but overall i want to access these excel files and then create a corresponding docx
So os.listdir only provides the basename of the directory files, which will cause problems if your working directory does not match the value of directory. If your working directory is D:\Downloads, ./file.xlsx does not exist but D:\Downloads\FYP\TEST/file.xlsx does.
You will want to use the absolute path to the file, you have two options here. You could follow #IronMan's suggestion in the their comment to produce the file path from the directory path and file basename:
import os
directory = r'D:\Downloads\FYP\TEST'
for filename in os.listdir():
wb = openpyxl.load_workbook(os.path.join(directory, filename))
This is a simple and useful approach; however, its functionality is somewhat limited and may make it harder to make changes in the future. The alternative is to use python's paathlib and scandir, and access the path directly from there:
import pathlib
directory = r'D:\Downloads\FYP\TEST'
for entry in pathlib.scandir(diectory):
wb = openpyxl.load_workbook(entry.path)
In a Folder called Assignment Parser, I've my parsing.py file along with a auth.txt file. Trying to open this auth.txt file. But getting an error that says :
(base) C:\Users\Ajay\Desktop\Python\Assignment Parser>python parsing.py
Traceback (most recent call last):
File "parsing.py", line 27, in <module>
main()
File "parsing.py", line 8, in main
file = open(file_path / "auth.txt","r")
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Ajay\\Desktop\\Python\\Assignment Parser\\auth.txt'
Code:
from pathlib import Path
import os
def main():
# read file
# C:\Users\Ajay\Desktop\Python\Assignment Parser\
file_path = Path("C:/Users/Ajay/Desktop/Python/Assignment Parser/")
file = open(file_path / "auth.txt","r")
# file = open("auth.txt", "r")
lines = file.readlines()
file.close()
Where is this going wrong? PFA for the screenprint.
Try this:
from pathlib import Path
import os
def main():
# read file
# C:\Users\Ajay\Desktop\Python\Assignment Parser\
file_path = Path("C:/Users/Ajay/Desktop/Python/Assignment Parser/")
file = open(os.path.join(file_path, "auth.txt"), "r")
# file = open("auth.txt", "r")
lines = file.readlines()
file.close()
I think the problem is in file extension, I see parsing has .py extension but auth is not
please try file = open(file_path / "auth", "r") again (just delete .txt extension)
As you have your python file in same folder as your text file. You can directly use below code.
def main():
file = open("./auth.txt")
lines = file.readlines()
file.close()
Also make sure, your syder working directory is set to this folder path "C:/Users/Ajay/Desktop/Python/Assignment Parser"