os.path.join accesses file preventing rename operation - python

I am trying to handle a JSON decode by backing up a malformed file when the decode fails, but I'm experiencing some strange behaviour that I did not expect from the os.path.join method.
The following code fails with an exception: PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'file.txt' -> 'file.txt\\.bak'
file_path = "file.txt"
try:
with open(file_path, 'r') as f:
json.load(f)
except json.JSONDecodeError as e:
os.rename(file_path, os.path.join(file_path, '.bak'))
If I change the argument like this: os.rename(file_path, file_path + '.bak') The code executes as expected without the permission error. It seems like the os.path.join method actually accesses the file rather than being a strict string operation. Is this expected behaviour?

os.path.join(file_path, '.bak')) actually will give you file.txt\\.bak like you see in the error code, but file_path + '.bak' gives you the correct file name file.txt.bak
os.path.join appends a separator between it's arguments, hence it ends up adding that separator in your case too
Example in MacOS, you can see that it adds a separator between each of it's arguments. os.path.join is more useful to append directory names, of the full filename with the directory paths.
In [4]: import os
In [5]: os.path.join('filename','.bak')
Out[5]: 'filename/.bak'
In [6]: os.path.join('folder1', 'folder2')
Out[6]: 'folder1/folder2'
The error happens since the Windows OS is trying to make a file .bak in a folder named file.txt, which isn't possible since file.txt is a plain file and not a directory, which is correct.
Using file_path+'.bak creates the file.path.bak correctly in the folder you want, hence you don't see an error there!

The error message is the key. As usually on Windows the cause (because it is being used by another process) is wrong, but the names ('file.txt' -> 'file.txt\.bak') are correct.
Join is not a string concatenation but expects that all path members except the last represent folders. So here you are trying to make a file .bak in a folder named file.txt. It is not possible because file.txt is a plain file and not a directory.
On another hand, when you use os.rename(file_path, file_path + '.bak') you are renaming file.txt to file.txt.bak in the same folder which is allowed by the underlying file system, hence no error.
So the behaviour is exactly what is expected, except for the beginning of the error message.
As I am not a core Microsoft Developper, the following is a wild guess. The number of error given by the system is limited. The rename C function received 2 strings and passed it to the system call for rename. As expected the file system generated an error but as it was neither a physical error nor a file system full error, it just choosed a permission refused cause. Which is not really wrong because it is not allowed to create folders under a plain file. But the message for that error is unfortunately because it is being used by another process which is stupid here

Related

Why does Python not find/recognise a file saved in the same directory as the file where the code is written?

I am quite new to working with Python files, and having a small issue. I am simply trying to print the name of a text file and its 'mode'.
Here is my code:
f = open('test.txt','r')
print(f.name)
print(f.mode)
f.close()
I have a saved a text file called 'test.txt' in the same directory as where I wrote the above code.
However, when I run the code, I get the following file not found error:
FileNotFoundError: [Errno 2] No such file or directory: 'test.txt'
Any ideas what is causing this?
I have also tried to replace the first argument above with the path of the test.txt file, but I get the same error?
open (on pretty much any operating system) doesn't care where your program lies, but from which directory you are running it. (This is not specific to python, but how file operations work, and what a current working directory is.)
So, this is expected. You need to run python from the directory that test.txt is in.
I have also tried to replace the first argument above with the path of the test.txt file, but I get the same error?
In that case, you must have mistyped the path. Make sure there's no special characters (like backslashes) that python interprets specially in there, or use the raw string format r'...' instead of just '...'.
It depend from where the python command is launched for instance :
let suppose we have this 2 files :
dir1/dir2/code.py <- your code
dir1/dir2/test.txt
if you run your python commande from the dir1 directory it will not work because it will search for dir1/test.txt
you need to run the python commande from the same directory(dir2 in the example).

FileNotFoundError: [Errno 2] No such file or directory: '9788427133310_urls.csv' [duplicate]

This question already has answers here:
open() gives FileNotFoundError / IOError: '[Errno 2] No such file or directory'
(8 answers)
Closed 7 months ago.
I'm trying to write a simple program to read a file and search for a word then print how many times that word is found in the file. Every time I type in "test.rtf" (which is the name of my document) I get this error:
Traceback (most recent call last):
File "/Users/AshleyStallings/Documents/School Work/Computer Programming/Side Projects/How many? (Python).py", line 9, in <module>
fileScan= open(fileName, 'r') #Opens file
FileNotFoundError: [Errno 2] No such file or directory: 'test.rtf'
In class last semester, I remember my professor saying you have to save the file in a specific place? I'm not sure if he really said that though, but I'm running apple OSx if that helps.
Here's the important part of my code:
fileName= input("Please enter the name of the file you'd like to use.")
fileScan= open(fileName, 'r') #Opens file
If the user does not pass the full path to the file (on Unix type systems this means a path that starts with a slash), the path is interpreted relatively to the current working directory. The current working directory usually is the directory in which you started the program. In your case, the file test.rtf must be in the same directory in which you execute the program.
You are obviously performing programming tasks in Python under Mac OS. There, I recommend to work in the terminal (on the command line), i.e. start the terminal, cd to the directory where your input file is located and start the Python script there using the command
$ python script.py
In order to make this work, the directory containing the python executable must be in the PATH, a so-called environment variable that contains directories that are automatically used for searching executables when you enter a command. You should make use of this, because it simplifies daily work greatly. That way, you can simply cd to the directory containing your Python script file and run it.
In any case, if your Python script file and your data input file are not in the same directory, you always have to specify either a relative path between them or you have to use an absolute path for one of them.
Is test.rtf located in the same directory you're in when you run this?
If not, you'll need to provide the full path to that file.
Suppose it's located in
/Users/AshleyStallings/Documents/School Work/Computer Programming/Side Projects/data
In that case you'd enter
data/test.rtf
as your file name
Or it could be in
/Users/AshleyStallings/Documents/School Work/Computer Programming/some_other_folder
In that case you'd enter
../some_other_folder/test.rtf
As noted above the problem is in specifying the path to your file.
The default path in OS X is your home directory (/Users/macbook represented by ~ in terminal ...you can change or rename the home directory with the advanced options in System Preferences > Users & Groups).
Or you can specify the path from the drive to your file in the filename:
path = "/Users/macbook/Documents/MyPython/"
myFile = path + fileName
You can also catch the File Not Found Error and give another response using try:
try:
with open(filename) as f:
sequences = pick_lines(f)
except FileNotFoundError:
print("File not found. Check the path variable and filename")
exit()
A good start would be validating the input. In other words, you can make sure that the user has indeed typed a correct path for a real existing file, like this:
import os
fileName = input("Please enter the name of the file you'd like to use.")
while not os.path.isfile(fileName):
fileName = input("Whoops! No such file! Please enter the name of the file you'd like to use.")
This is with a little help from the built in module os, That is a part of the Standard Python Library.
You might need to change your path by:
import os
path=os.chdir(str('Here_should_be_the_path_to_your_file')) #This command changes directory
This is what worked for me at least! Hope it works for you too!
Difficult to give code examples in the comments.
To read the words in the file, you can read the contents of the file, which gets you a string - this is what you were doing before, with the read() method - and then use split() to get the individual words.
Split breaks up a String on the delimiter provided, or on whitespace by default. For example,
"the quick brown fox".split()
produces
['the', 'quick', 'brown', 'fox']
Similarly,
fileScan.read().split()
will give you an array of Strings.
Hope that helps!
First check what's your file format(e.g: .txt, .json, .csv etc ),
If your file present in PWD , then just give the name of the file along with the file format inside either single('')or double("") quote and the appropriate operation mode as your requirement
e.g:
with open('test.txt','r') as f: data=f.readlines() for i in data: print(i)
If your file present in other directory, then just give the full path name where is your file is present and file name along with file format of the file inside either single('')or double("") quote and the appropriate operation mode as your requirement.
If it showing unicode error just put either r before quote of file path or else put '/' instead of ''
with open(r'C:\Users\soman\Desktop\test.txt','r') as f: data=f.readlines() for i in data: print(i)
The mistake I did was
my code :
x = open('python.txt')
print(x)
But the problem was in file directory ,I saved it as python.txt instead of just python .
So my file path was
->C:\Users\noob\Desktop\Python\Course 2\python.txt.txt
That is why it was giving a error.
Name your file without .txt it will run.

Getting \u200b in filename when using OS to scan directory for files

I'm trying to scan a directory for files using this code:
result = [os.path.join(dp, f) for dp, dn, filenames in os.walk(PATH) for f in filenames if os.path.splitext(f)[1] == '.json']
Most of them are coming back correct but a few of them are coming back with \u200b in them. When I put it in a with open() statement, I get an error message saying:
IOError: [Errno 14] Bad address
What's even stranger is that when I manually look at the file name in the directory it's stored in, there's no \u200b to be found. But if I manually try to read in the name I see printed in the directory, it says "file not found".
How do I solve this issue? The ideal solution I need is a way to read these files in with open() statements without having to manually go and rename all these files without special characters.
Thank you

Exception on extracting a zipfile, after appending to it in Python

z = zipfile.ZipFile(io.BytesIO(artifact), mode='a')
z.write("test.txt",arcname=r'bin/test.txt')
z.extractall('out')
Exception:
zipfile.BadZipFile was unhandled by user code
Message: File name in directory 'bin\test.txt' and header b'bin/test.txt' differ.
The interesting thing is if I write the file to disk, and try extract it, I get a invalid file error. This is on Win 7 by the way.
the bin folder already exists in the zipfile. Full Traceback
Actually the code works well on my Mac,and I think you should let us know that the structure of the zip file or what the variable artifact is.
Here is my advice:
Use forward slashes as path separators,when you create the zip file.
Try to print a warning not raise the exception,and check out the out folder,you will find the reason,maybe the slashes or string buffer.
Also you can read this issue.
Hope this helps.

open() method I/O error

I'm trying to write all the settings from each run of a program to a config file.
This line is giving me strange issues:
cfgfile = open(config_filename, 'w+')
I am getting I/O Errors of the 'No such file or directory' kind. This happens no matter if the flag is w+, a, etc.
I am creating the config_filename from a string stored in a dictionary, concatenated with another short string:
config_filename = my_dict['key'] + '_PARAMETERS.txt'
This is what seems to be causing the problems. I have tested with using
config_filename = 'test' + '_PARAMETERS.txt'
and it works fine, creating the file before writing to it.
Can anyone explain why this happens, I have checked I'm in the same directory and type(config_filename) throws back string as expected. Is this something to do with how dicts store data?
Thanks.
Based on your comment, it looks like you're trying to create a file with a name that includes slashes. This will fail because slashes are not allowed in file names, as they are reserved by the OS to indicate the directory structure. Colons are also not allowed in Windows file names.
Try replacing my_dict['key'] with my_dict['key'].replace('/', ''), which will remove the slashes and produce a filename like Results_150715_18:32:09_PARAMETERS.txt. For Windows, add .replace(':','') to remove the colon as well.
I should clarify that slashes are not allowed in file names, but they are allowed in file paths. The filename you were trying to use would have attempted to create a file named 15_18:32:09_PARAMETERS.txt in the 07 folder, contained in the Results_15 folder. That isn't what you wanted, but it wouldn't have failed as long as the directory structure existed (on Windows, anyway). Additionally, Windows systems don't allow colons anywhere in the path except to indicate the drive, e.g. C:.

Categories