I have a function that polls a folder for new files, then loads them using numpy.loadtext when it shows up. The function is called from a while loop that runs for 30 seconds. The function works properly most of the time, but for some files, seemingly at random, I get the error IOError: [Errno 13] Permission denied: 'myfilename1.txt'. Here is the content of my function:
before = dict([(f, None) for f in os.listdir(mydir)])
while 1:
after = dict([(f, None) for f in os.listdir(mydir)])
added = [f for f in after if f not in before]
# New File
if added:
raw = numpy.loadtxt(mydir + added[0])
return raw
Any idea on why this is happening? It properly polls and reads most text files that are incoming, but sometimes spits the error and I can't find a systematic reason why.
UPDATE:
Has something to do with using the full path with loadtxt. When I change the working directory to the directory where the files are, I no longer get the permissions error.
Have you tried opening the file as read only, may be a conflict if the file is accessed by another application (or is still currently being created).
# New File
if added:
with open(mydir + added[0], 'r') as f:
raw = numpy.loadtxt(f)
You could also try some form of IOError handling which waits a little while and then tries again
import time
before = dict([(f, None) for f in os.listdir(mydir)])
added = False
while 1:
# New File
if added:
try:
raw = numpy.loadtxt(mydir + added[0])
return raw
except IOError:
time.sleep(5)
else:
after = dict([(f, None) for f in os.listdir(mydir)])
added = [f for f in after if f not in before]
I got the same error when I attempted the following:
Y = np.loadtxt("C:/Users/erios/images_3_color_15k_labeled/", dtype='int')
I.e., I passed the folder where the text was located
INstead, the following command executed with no error:
Y = np.loadtxt("C:/Users/erios/images_3_color_15k_labeled/labels_for_locations.txt", dtype='int')
In sum, specify the full name of the text file, not just the folder.
Related
I am working on a python script to unzip a folder of nested zip files. In general, it seems to be working, but when it comes to one of the files I always receive this error:
zlib.error: Error -3 while decompressing data: invalid distance too far back
I initially tried using a try and except to pass over this, but that didn't work. I then tried adding into the except clause except zipfile.BadZipFile after reading various posts on here regarding it, but I still am receiving the same error. Here is how my code looks:
import os
import zipfile
folder_path = [MY FOLDER PATH HERE]
def extract_zip(filepath_in, filepath_out, password):
if is_zip(filepath_in):
print(filepath_in)
with zipfile.ZipFile(filepath_in, 'r') as zip_file:
zip_file.extractall(filepath_out, pwd = bytes(password, 'utf-8'))
error_list = []
zip_count = 1
while zip_count > 0:
filelist = []
for root, dirs, files in os.walk(folder_path):
for file in files:
filelist.append(os.path.join(root,file))
zip_count = 0
for name in filelist:
if name[(len(name) - 4):(len(name))] == ".zip": #if file name ends in .zip
zip_count =+ 1
try:
extract_zip(name, name[0:(len(name) - 4)] , password)
os.remove(name)
except zipfile.BadZipFile as fail:
error_list.append(name), fail
pass
I am unsure of how else to either have the error ignored or if there is some other way to extract the file.
I'm using this to connect to Azure File Share and upload a file. I would like to chose what extension file will have, but I can't. I got an error shown below. If I remove .txt everything works fine. Is there a way to specify file extension while uploading it?
Error:
Exception: ResourceNotFoundError: The specified parent path does not exist.
Code:
def main(blobin: func.InputStream):
file_client = ShareFileClient.from_connection_string(conn_str="<con_string>",
share_name="data-storage",
file_path="outgoing/file.txt")
f = open('/home/temp.txt', 'w+')
f.write(blobin.read().decode('utf-8'))
f.close()
# Operation on file here
f = open('/home/temp.txt', 'rb')
string_to_upload = f.read()
f.close()
file_client.upload_file(string_to_upload)
I believe the reason you're getting this error is because outgoing folder doesn't exist in your file service share. I took your code and ran it with and without extension and in both situation I got the same error.
Then I created a folder and tried to upload the file and I was able to successfully do so.
Here's the final code I used:
from azure.storage.fileshare import ShareFileClient, ShareDirectoryClient
conn_string = "DefaultEndpointsProtocol=https;AccountName=myaccountname;AccountKey=myaccountkey;EndpointSuffix=core.windows.net"
share_directory_client = ShareDirectoryClient.from_connection_string(conn_str=conn_string,
share_name="data-storage",
directory_path="outgoing")
file_client = ShareFileClient.from_connection_string(conn_str=conn_string,
share_name="data-storage",
file_path="outgoing/file.txt")
# Create folder first.
# This operation will fail if the directory already exists.
print "creating directory..."
share_directory_client.create_directory()
print "directory created successfully..."
# Operation on file here
f = open('D:\\temp\\test.txt', 'rb')
string_to_upload = f.read()
f.close()
#Upload file
print "uploading file..."
file_client.upload_file(string_to_upload)
print "file uploaded successfully..."
Is there a way for Python to close that the file is already open file.
Or at the very least display a popup that file is open or a custom written error message popup for permission error.
As to avoid:
PermissionError: [Errno 13] Permission denied: 'C:\\zf.csv'
I've seen a lot of solutions that open a file then close it through python. But in my case. Lets say I left my csv open and then tried to run the job.
How can I make it so it closes the currently opened csv?
I've tried the below variations but none seem to work as they expect that I have already opened the csv at an earlier point through python. I suspect I'm over complicating this.
f = 'C:\\zf.csv'
file.close()
AttributeError: 'str' object has no attribute 'close'
This gives an error as there is no reference to opening of file but simply strings.
Or even..
theFile = open(f)
file_content = theFile.read()
# do whatever you need to do
theFile.close()
As well as:
fileobj=open('C:\\zf.csv',"wb+")
if not fileobj.closed:
print("file is already opened")
How do I close an already open csv?
The only workaround I can think of would be to add a messagebox, though I can't seem to get it to detect the file.
filename = "C:\\zf.csv"
if not os.access(filename, os.W_OK):
print("Write access not permitted on %s" % filename)
messagebox.showinfo("Title", "Close your CSV")
Try using a with context, which will manage the close (__exit__) operation smoothly at the end of the context:
with open(...) as theFile:
file_content = theFile.read()
You can also try to copy the file to a temporary file, and open/close/remove it at will. It requires that you have read access to the original, though.
In this example I have a file "test.txt" that is write-only (chmod 444) and it throws a "Permission denied" error if I try writing to it directly. I copy it to a temporary file that has "777" rights so that I can do what I want with it:
import tempfile, shutil, os
def create_temporary_copy(path):
temp_dir = tempfile.gettempdir()
temp_path = os.path.join(temp_dir, 'temp_file_name')
os.chmod(temp_path, 0o777); # give full access to the tempfile so we can copy
shutil.copy2(path, temp_path) # copy the original into the temp one
os.chmod(temp_path, 0o777); # replace permissions from the original file
return temp_path
path = "./test.txt" # original file
copy_path = create_temporary_copy(path) # temp copy
with open(copy_path, "w") as g: # can do what I want with it
g.write("TEST\n")
f = open("C:/Users/amol/Downloads/result.csv", "r")
print(f.readlines()) #just to check file is open
f.close()
# here you can add above print statement to check if file is closed or not. I am using python 3.5
I am parsing images from a webpage into a specif folder everything goes very well a huge part of the images are parsed into the desired folder then before the process ends until it gives this error:
IOError: [Errno 2] No such file or directory: u'C:\\Users\\pro\\Downloads\\AAA\\photos\\'
The code is something like this:
import os
save_path = raw_input("give save path. like '/home/user/dalbums'")
album = raw_input("the name of album: ")
completeName = os.path.join(save_path,album)
class X:
def saver(self, info):
path_name = os.path.join(completeName, 'photos')
if not os.path.exists(path_name):
os.makedirs(path_name)
with open(os.path.join(path_name, info), 'a') as f:
for i in lo:
f.write(lo)
If I keep only this part the error goes away but then the images goes to the wrong place:
with open(info, 'a') as f:
for i in lo:
f.write(lo)
When i try to use url https://www.google.com i get this error for the same code
InvalidSchema:
No connection adapters were found for 'javascript:void(0)'
You show different code in your question than that actually cause the error.
This is the relevant code:
with open(os.path.join(imgs_folder, My_imgs.strip()), 'wb') as f:
Since My_imgs.strip() returns an empty string, your file name is an empty string. Therefore, you try to write to directory after join the empty string to a directory name.
Here is where you create My_imgs:
My_imgs = data_fetched.split('/')[-1].split("?")[0]
For debugging you could do:
if not My_imgs.strip():
print('data_fetched:', data_fetched)
to see what data_fetched actually is.
I have a python script which polls a folder to see if a new file is added, and when a new file is added, it loads the text file into numpy.
before = dict([(f, None) for f in os.listdir('.')])
while 1:
#time.sleep(10)
after = dict([(f, None) for f in os.listdir('.')])
added = [f for f in after if f not in before]
if added:
print added[0]
try:
raw = numpy.loadtxt(added[0])
except IOError:
print "----- Error"
core.wait(0.1)
raw = numpy.loadtxt(added[0])
When I don't try catching the exception, I get an error in reading the file - "Permission denied". I am a Windows administrator, so this should not be an issue. It also only happens on some files, so I think the script might be trying to open the file before it is fully written. Is there any way to get rid of this error?