I've had a script for a while that has been running without issues however recently had a "hitch" with a temporary file that was within a directory.
The file in question started with '~$' on a windows PC so the script was erroring out on this file as it is not a proper DOCX file. The file in question was not open and occurred after being transferred of a network drive onto an external hard drive. Checking the destination drive (with hidden files on etc) did not show this file either.
I have attempted a quick fix off:
for (dirpath,dirnames,filenames) in os.walk('.'):
for filename in filenames:
if filename.endswith('.docx'):
filesList.append(os.path.join(dirpath,filename))
for file in filesList:
if file.startswith('~$'):
pass
else:
<rest of script>
However the script appears to be ignoring this to proceed then error out again, as the file is not "valid".
Does anyone know either why this isn't working or a quick solution to get it to ignore any files that are like this? I would attempt a if exists, however the file technically does exist so this wouldn't work either.
Sorry if its a bit stupid, but I am a bit stumped as to A. why its there and B. how to code around it.
In the second code block, your variable file contains the whole file path, not just the file name.
Instead skip the "bad" files in your first block instead of appending to the list:
for (dirpath,dirnames,filenames) in os.walk('.'):
for filename in filenames:
if filename.endswith('.docx'):
if not filename.startswith('~$'):
filesList.append(os.path.join(dirpath,filename))
The other option would be to check os.path.basename(file) in your second code block.
Related
I am trying to open an .npy file. However, Python keeps saying that there exists no such file or directory, even when there is one...
To make sure that it isn't an issue of giving correct path names, I changed my directory to the folder that contains the .npy file that I want. Then used list.dir() and used it to use np.load (the code is below):
os.chdir(filename_dir) #filename_dir is the directory I want to get to that contains the npy file I want)
the_path=os.path.join(os.getcwd()+os.listdir()[-1]) #i.e. I did this to make sure that the directory is correct
data=np.load(the_path)
However, I got the error, [Errno 2] No such file or directory: .......
The error occurs when I try np.load. I couldn't understand this because I explicitly made the path the_path from what Python says exists.
I think this is the problem...
Windows (by default) has a maximum path length of 260 characters. The absolute path to this file exceeds that limit. The filename alone is 143 characters.
It looks as though even if you try to access the file using a relative path (i.e., chdir() to the appropriate folder then specify just the filename), numpy is probably working out the absolute path then failing.
You have said that renaming the file to something much shorter solves the problem but is impractical due to the high numbers of files that you need to process.
There is a Windows Registry key that can be modified to enable long pathnames: Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled
Changing that may help.
Or you could buy a Mac
Try adding r. For example,
file_path = r"filePath"
This may work...
I hate to be the person to post another FileNotFoundError question, but most of them that I see are about not giving the full path to the file, that is not my problem here.
I have a number of log files in folders in ../../Data/. I create a glob of those files using
DataFiles = glob('../../Data/2021*/*.log')
I want to open each of the files in that glob, so I use
for i, file in enumerate(DataFiles):
with open(file, "r") as f:
...
etc. 99% of these open correctly and the rest of the code runs. For some reason, a few will not. I get an error like
FileNotFoundError: [Errno 2] No such file or directory: '../../Data\\20210629_081706\\20210629_081706_data.log'
The file definitely exists, that's why it was found by glob. The full path is used. And,
from pathlib import Path
Path('../../Data\\20210629_081706\\20210629_081706_data.log')
returns
WindowsPath('../../Data/20210629_081706/20210629_081706_data.log')
So does anyone know what might be happening here?
A bit late, but I had the same error when using glob in a network folder with way to many levels.
There was a particular folder where some of the files caused that error, and those files couldn't even be opened by the explorer itself:
In my case this was caused by the path being over 260 characters in length.
You can try something like suggested here to allow handling files with larger paths, or just make sure the path is short enough for the explorer to handle it.
I am trying to learn about I/O operations in python. I used open() function before to open text files (I used relative path). Now I am trying to do this on an absolute path but it keeps getting me FileNotFoundError even though the filename and directory is correct. I just want to learn why it is causing a problem. And how can I fix the issue ?
This is my code to open the input.txt file in a read mode.
file = open("D:/Audio/input.txt", 'r')
lines = file.readlines()
This is the screenshot of the directory where I have the files
Your screenshot shows that there is a text file with name input.txt (Note that .txt here is not extension but a part of file name).
But in your code you are trying to read from a text file input that doesn't exist. That is the reason you get FileNotFoundError.
Rename it to just input.
Also from your screenshot there are two text files - output.txt and output. When you add extension they become output.txt.txt and output.txt.
When you program, it is often useful to be able to see the full filename and the extension. Follow this tutorial to see how you can enable this functionnality.
I am getting a FileNotFound error when iterating through a list of files in Python on Windows.
The specific error I get looks like:
FileNotFoundError: File b'fileName.csv' does not exist
In my code, I first ask for input on where the file is located and generate a list using os (though I also tried glob):
directory = input('In what directory are your files located?')
fileList = [s for s in os.listdir(directory) if s.endswith('.csv')]
When I print the list, it does not contain byte b before any strings, as expected (but I still checked). My code seems to break at this step, which generates the error:
for file in fileList:
pd.read_csv(file) # breaks at this line
I have tried everything I could find on Stack Overflow to solve this problem. That includes:
Putting an r or b or rb right before the path string
Using both the relative and absolute file paths
Trying different variations of path separators (/, \, \\, etc.)
I've been dealing with Windows-related issues lately (since I normally work in Mac or Linux) so that was my first suspicion. I'd love another set of eyes to help me figure out where the snag is.
A2A.
Although the list was generated correctly because the full directory path was used, the working directory when the file was being run was .. You can verify this by running os.getcwd(). When later iterating through the list of file names, the program could not find those file names in the . directory, as it shouldn't. That list is just a list of file names; there was no directory tied to it so it used the current directory.
The easiest fix here is to change the directory the file is running through after the input. So,
directory = input('In what directory are your files located?')
os.chdir(directory) # Points os to given directory to avoid byte issue
If you need to access multiple directories, you could either do this right before you switch to each directory or save the full file path into your list instead.
import os, csv
f=open("C:\\tempa\\file.csv", 'wb') #write to an existing blank csv file
w=csv.writer(f)
for path, dirs, files, in os.walk("C:\\tempa"):
for filename in files:
w.writerow([filename])
running win7 64bit latest python, using anaconda spyder, pyscripter issue persists regardless of the ide.
I have some media in folders in tempa jpg, pdf and mov... and I wanted to get a file list of all of them, and the code works but it stops without any issue at row 113, nothing special with the file it stops on, no weird characters.
I could have 3 blocks of code one for each folder to work around this weird bug. but it shouldnt have an issue.. the folders are all in the root folder without going too deep in sub folders:
C:\
-tempa
-jpg
-pdf
-mov
I have heard there are issues with os.walk but I didn't expext anything weird like this.
Maybe I need an f=close?
You were examining the file before it was fully closed. (f won't be closed until, at least, it is no longer referenced by any in-scope variable name.) If you examine a file before it is closed, you may not see the final, partial, data buffer.
Use the file object's context manager to ensure that the file is flushed and closed in all cases:
import os, csv
with open("C:\\tempa\\file.csv", 'wb') as f: #write to an existing blank csv file
w=csv.writer(f)
for path, dirs, files, in os.walk("C:\\tempa"):
for filename in files:
w.writerow([filename])
# Now no need for f.close()