We Keep Coding

The 'os' module in Python says "No such file or directory" even when there definitely is one

I am trying to open an .npy file. However, Python keeps saying that there exists no such file or directory, even when there is one...
To make sure that it isn't an issue of giving correct path names, I changed my directory to the folder that contains the .npy file that I want. Then used list.dir() and used it to use np.load (the code is below):
os.chdir(filename_dir) #filename_dir is the directory I want to get to that contains the npy file I want)
the_path=os.path.join(os.getcwd()+os.listdir()[-1]) #i.e. I did this to make sure that the directory is correct
data=np.load(the_path)
However, I got the error, [Errno 2] No such file or directory: .......
The error occurs when I try np.load. I couldn't understand this because I explicitly made the path the_path from what Python says exists.

I think this is the problem...
Windows (by default) has a maximum path length of 260 characters. The absolute path to this file exceeds that limit. The filename alone is 143 characters.
It looks as though even if you try to access the file using a relative path (i.e., chdir() to the appropriate folder then specify just the filename), numpy is probably working out the absolute path then failing.
You have said that renaming the file to something much shorter solves the problem but is impractical due to the high numbers of files that you need to process.
There is a Windows Registry key that can be modified to enable long pathnames: Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled
Changing that may help.
Or you could buy a Mac

Try adding r. For example,
file_path = r"filePath"
This may work...

os.path.join accesses file preventing rename operation

I am trying to handle a JSON decode by backing up a malformed file when the decode fails, but I'm experiencing some strange behaviour that I did not expect from the os.path.join method.
The following code fails with an exception: PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'file.txt' -> 'file.txt\\.bak'
file_path = "file.txt"
try:
with open(file_path, 'r') as f:
json.load(f)
except json.JSONDecodeError as e:
os.rename(file_path, os.path.join(file_path, '.bak'))
If I change the argument like this: os.rename(file_path, file_path + '.bak') The code executes as expected without the permission error. It seems like the os.path.join method actually accesses the file rather than being a strict string operation. Is this expected behaviour?

os.path.join(file_path, '.bak')) actually will give you file.txt\\.bak like you see in the error code, but file_path + '.bak' gives you the correct file name file.txt.bak
os.path.join appends a separator between it's arguments, hence it ends up adding that separator in your case too
Example in MacOS, you can see that it adds a separator between each of it's arguments. os.path.join is more useful to append directory names, of the full filename with the directory paths.
In [4]: import os
In [5]: os.path.join('filename','.bak')
Out[5]: 'filename/.bak'
In [6]: os.path.join('folder1', 'folder2')
Out[6]: 'folder1/folder2'
The error happens since the Windows OS is trying to make a file .bak in a folder named file.txt, which isn't possible since file.txt is a plain file and not a directory, which is correct.
Using file_path+'.bak creates the file.path.bak correctly in the folder you want, hence you don't see an error there!

The error message is the key. As usually on Windows the cause (because it is being used by another process) is wrong, but the names ('file.txt' -> 'file.txt\.bak') are correct.
Join is not a string concatenation but expects that all path members except the last represent folders. So here you are trying to make a file .bak in a folder named file.txt. It is not possible because file.txt is a plain file and not a directory.
On another hand, when you use os.rename(file_path, file_path + '.bak') you are renaming file.txt to file.txt.bak in the same folder which is allowed by the underlying file system, hence no error.
So the behaviour is exactly what is expected, except for the beginning of the error message.
As I am not a core Microsoft Developper, the following is a wild guess. The number of error given by the system is limited. The rename C function received 2 strings and passed it to the system call for rename. As expected the file system generated an error but as it was neither a physical error nor a file system full error, it just choosed a permission refused cause. Which is not really wrong because it is not allowed to create folders under a plain file. But the message for that error is unfortunately because it is being used by another process which is stupid here

Ignore "temp" files?

I've had a script for a while that has been running without issues however recently had a "hitch" with a temporary file that was within a directory.
The file in question started with '~$' on a windows PC so the script was erroring out on this file as it is not a proper DOCX file. The file in question was not open and occurred after being transferred of a network drive onto an external hard drive. Checking the destination drive (with hidden files on etc) did not show this file either.
I have attempted a quick fix off:
for (dirpath,dirnames,filenames) in os.walk('.'):
for filename in filenames:
if filename.endswith('.docx'):
filesList.append(os.path.join(dirpath,filename))
for file in filesList:
if file.startswith('~$'):
pass
else:
<rest of script>
However the script appears to be ignoring this to proceed then error out again, as the file is not "valid".
Does anyone know either why this isn't working or a quick solution to get it to ignore any files that are like this? I would attempt a if exists, however the file technically does exist so this wouldn't work either.
Sorry if its a bit stupid, but I am a bit stumped as to A. why its there and B. how to code around it.

In the second code block, your variable file contains the whole file path, not just the file name.
Instead skip the "bad" files in your first block instead of appending to the list:
for (dirpath,dirnames,filenames) in os.walk('.'):
for filename in filenames:
if filename.endswith('.docx'):
if not filename.startswith('~$'):
filesList.append(os.path.join(dirpath,filename))
The other option would be to check os.path.basename(file) in your second code block.

Errno13, Permission denied when trying to read file

I have created a small python script. With that I am trying to read a txt file but my access is denied resolving to an no.13 error, here is my code:
import time
import os
destPath = 'C:\Users\PC\Desktop\New folder(13)'
for root, dirs, files in os.walk(destPath):
f=open(destPath, 'r')
.....

Based on the name, I'm guessing that destPath is a directory, not a file. You can do a os.walk or a os.listdir on the directory, but you can't open it for reading. You can only call open on a file.
Maybe you meant to call open on one or more of the items from files

1:
I take it you are trying to access a file to get what's inside but don't want to use a direct path and instead want a variable to denote the path. This is why you did the destPath I'm assuming.
From what I've experienced the issue is that you are skipping a simple step. What you have to do is INPUT the location then use os.CHDIR to go to that location. and finally you can use your 'open()'.
From there you can either use open('[direct path]','r') or destPath2 = 'something' then open(destPath2, 'r').
To summarize: You want to get the path then NAVIGATE to the path, then get the 'filename' (can be done sooner or not at all if using a direct path for this), then open the file.
2: You can also try adding an "r" in front of your path. r'[path]' for the raw line in case python is using the "\" for something else.
3: Try deleting the "c:/" and switching the / to \ or vice versa.
That's all I got, hope one of them helps! :-)

I got this issue when trying to create a file in the path -C:/Users/anshu/Documents/Python_files/Test_files . I discovered python couldn't really access the directory that was under the user's name.
So, I tried creating the file under the directory - C:/Users/anshu/Desktop .
I was able to create files in this directory through python without any issue.

Why isn't my os.rename working?

I'm trying to rename some files, but getting a baffling error*. When I run this:
if os.path.isfile(fullPath):
print 'fmf exists'
print fullPath
print newFilePath
os.rename(fullPath,newFilePath)
I get the following error:
fmf exists
(correct fullPath)
(correct newFilePath, ie. destination)
Traceback (most recent call last):
File "whatever.py", line 374, in ?
os.rename(fullPath,newFilePath)
OSError: [Errno 2] No such file or directory
Since I know that the file at fullPath exists, I'm baffled by the error. Of course, newFilePath doesn't exist, because that would be dumb. Any hints?
Thanks!
Alex
*Aren't they all?

You can still get the exception if you try to copy to a directory that does not exist.

I can't see the full inner workings of your code, so here's my two cents worth:
Your newFilePath may contain a directory that doesn't exist. If that is the case, then depending on your operating system, your program is unable to create a file in a directory that doesn't exist. That could be your error.
Hope this helps

It seems like the poster solved his problem, but I had the same symptom and the cause appeared to be different. The file I was trying to rename had just been created in a subprocess call on the previous line. If I ran my script again, I didn't have the rename problem, since the file had been created in the previous run, but if I deleted the file previously created, I would get the same problem with rename. It seems like os.rename was getting called before the subprocess was completed and the file to be renamed therefore didn't exist yet. I inserted an os.wait() after the subprocess call, and I believe that this has solved my problem.

I had the same error when my new filename contained forward slashes, which are confused with directory separators in Unix and Linux. For example, renaming a file to "4/27/2015.txt" leads to a directory that doesn't exist and results in "No such file or directory". You can solve this by replacing the forward slashes with any other acceptable character.