I am trying to loop through 22 .txt files in a directory, read each lines from the file, and apply some conditions, before writing the modified lines in a newly created file in a different folder.
The issue is that the first 12 files are successfully created, but then I get a permission denied error for all subsequent files. The permissions for the files that fail are the exact same ones as the one that succeed.
Here is a simplified version of my code, excluding a bunch of if conditions for simplicity:
files = os.listdir('./folder/')
for file in files:
with open('./folder/' + file, "r") as input:
with open('./folder/mod/' + file[:-4] + '_mod.txt', "w") as output:
lines = input.readlines()
for i in range(len(lines)):
if lines[i][-7:-1] != ' },':
output.write('\n' + lines[i][:-4].replace('"',''))
output.close()
input.close()
I have tried to exclude the first 12 files completely from the source folder, and then not a single file gets written. As if the issue really lies with the .txt files themselves. These files were all generated at the same time, and are fairly simple, so I can't pinpoint any difference between them.
Here is the Stack Trace returned:
Traceback (most recent call last):
File "C:\Users\myname\Documents\project1\main.py", line 8, in <module>
with open('./folder/' + file, "r") as input:
IOError: [Errno 13] Permission denied: './folder/mod'
***EDIT
If I leave only 1 file that was successfully read and written in the source folder, the operation is successful but I still get the same permission denied error.
The stack trace says the problem is that:
`with open('./folder/' + file, "r") as input:`
is trying to open and read "folder/mod/"
os.listdir() will create a list of everything in a folder, including directories.
So since you can't "read" a directory, python is giving you a permission error.
Adding a test to see if the "file" in the for loop is actually a file should solve the problem:
Related
This question already has answers here:
PermissionError: [Errno 13] Permission denied
(16 answers)
Closed 6 months ago.
Just starting to learn some Python and I'm having an issue as stated below:
a_file = open('E:\Python Win7-64-AMD 3.3\Test', encoding='utf-8')
Traceback (most recent call last):
File "<pyshell#9>", line 1, in <module>
a_file = open('E:\Python Win7-64-AMD 3.3\Test', encoding='utf-8')
PermissionError: [Errno 13] Permission denied: 'E:\\Python Win7-64-AMD 3.3\\Test\
Seems to be a file permission error, if any one can shine some light it would be greatly appreciated.
NOTE: not sure how Python and Windows files work but I'm logged in to Windows as Admin and the folder has admin permissions.
I have tried changing .exe properties to run as Admin.
When doing;
a_file = open('E:\Python Win7-64-AMD 3.3\Test', encoding='utf-8')
...you're trying to open a directory as a file, which may (and on most non UNIX file systems will) fail.
Your other example though;
a_file = open('E:\Python Win7-64-AMD 3.3\Test\a.txt', encoding='utf-8')
should work well if you just have the permission on a.txt. You may want to use a raw (r-prefixed) string though, to make sure your path does not contain any escape characters like \n that will be translated to special characters.
a_file = open(r'E:\Python Win7-64-AMD 3.3\Test\a.txt', encoding='utf-8')
For me, I was writing to a file that is opened in Excel.
For me, I got this error when I was trying to write a file to a folder and wanted to make sure the folder existed. I accidentally used:
path = Path("path/to/my/file.txt")
path.mkdir(parents=True, exist_ok=True)
with open(path, "w") as file:
...
but the second line means "make a directory at this exact path (and make its parents too, without throwing errors for them existing already)". The third line then throws a PermissionError, because you can't use open() on a directory path, of course! The second line should have been:
path.parent.mkdir(parents=True, exist_ok=True)
I encountered this problem when I accidentally tried running my python module through the command prompt while my working directory was C:\Windows\System32 instead of the usual directory from which I run my python module
I'm attempting to write a quick python script to iterate through all csv files in the current folder and remove the header row from them then store them in a separate folder.
The current working directory has four sample csv files and the python script. Once executed the script creates the HeaderRemoved directory.
It appears that once the folder is created the code that is attempting to read the files is trying to access the folder but looking at the code I'm not sure why it would be.
I'm on a windows machine at the moment.
import csv, os, argparse, string
from ctypes import *
os.makedirs('HeaderRemoved', exist_ok=True)
# Loop through files in the current working directory
for csvFile in os.listdir('.'):
if not csvFile.endswith('.csv'):
continue # Skips non-csv files
print ('Removing header from ' + csvFile + '...')
# Read in CSV skipping the first row
csvRows = []
csvFileObj = open(csvFile)
csvReader = csv.reader(csvFileObj)
for row in csvReader:
if csvReader.line_num == 1:
continue # Skips the first row
csvRows.append(row)
csvFileObj.close()
# Write out the CSV file
csvFileObj = open (os.path.join('HeaderRemoved', csvFile), 'w', newline='')
for row in csvRows:
csvWriter.writerow(row)
csvFileObj.close()
Sample output:
Removing header from examplefile_1.csv...
Removing header from examplefile_2.csv...
Removing header from examplefile_3.csv...
Removing header from examplefile_4.csv...
Traceback (most recent call last): File "atbs_csv_parse.py", line 14, in <module>
csvFileObj = open(csvFile) PermissionError: [Errno 13] Permission denied: 'HeaderRemoved'
In my case, I had opened the csv file via Excel and ran the script. Then this Permission denied exception occurred.
Just closed the opened file and run the script again :)
In my case, the same error was because I was passing a directory name instead the file name.
Maybe could be the same problem of others.
The file in the script is opened somewhere in the system. That is the reason for getting "PermissionError : [Error 13]".
Solution:
Just close the file and run the script. You won't get the error.
As Charles Duffy commented under my original question, the issue was in fact that the lines of code for reading and writing the files had not been indented to fall within the for loop. Correcting the indentation fixed the issue and it now works as desired.
A good reminder to always check the simple things.... I got so wrapped up in why it wasn't working that I didn't even notice the lack of indentation.
If you are facing the problem where you can use the csv file with hard coded path but can't use with the windows directory or file path as the pandas or other library do not have the permission to use that object diectly, to so you have to convert it to stirng and use. I faced this issue and this solution worked for me.
from pathlib import Path
cur_dir=Path.cwd()
#cwd = os.getcwd()
csv_path=str(cur_dir)+"\\..\\Dataset\\FuelConsumptionCo2.csv"
df = pd.read_csv(csv_path)
Im trying to create a random set of files from a list of file names in python 3.
#!/usr/local/bin/python3
import random
import sys
random.seed()
print("Reading lines from ", sys.argv[1])
linecount = 0
lines = []
with open(sys.argv[1]) as f:
for line in f:
lines.append(line)
filelist = random.sample(lines, 5);
for newfile in filelist:
print("Touching file ", newfile)
open(newfile.rstrip(), "a+")
This always fails for the first file with:
$ : ./file-gen.py files.txt
Reading lines from files.txt
Touching file 62/6226320DE.pdf
Traceback (most recent call last):
File "./file-gen.py", line 19, in <module>
open(newfile.rstrip(), "a+");
FileNotFoundError: [Errno 2] No such file or directory: '62/6226320DE.pdf'
Any ideas what's missing?
I believe the problem is that the file mode a+ will create the file if its not present, but not the directory. You would have to write custom code to split the line of the filename from the path (or better still use os.path : https://docs.python.org/2/library/os.path.html, perhaps even os.path.dirname(path)
Have a look at: How to check if a directory exists and create it if necessary?
Be wary of security considerations of creating random paths in your system. Validating that the paths are inside a particular sandbox (consider someone putting an entry in your file of ../../ or /etc/passwd to get you to append random user data.. os.path.abspath can be useful - for this reason I am wary of pasting code which will just create random directories that you copy and paste in without considering that effect.
I would suggest as a first step trying to open a specific file rather than a random set of files from the input to rule out permissions and path problems.
You should also try printing os.path.getcwd() to make sure that you have write permissions there.
I am trying to make a minor modification to a python script made by my predecessor and I have bumped into a problem. I have studied programming, but coding is not my profession.
The python script processes SQL queries and writes them to an excel file, there is a folder where all the queries are kept in .txt format. The script creates a list of the queries found in the folder and goes through them one by one in a for cycle.
My problem is if I want to rename or add a query in the folder, I get a "[Errno 2] No such file or directory" error. The script uses relative path so I am puzzled why does it keep making errors for non-existing files.
queries_pathNIC = "./queriesNIC/"
def queriesDirer():
global filelist
l = 0
filelist = []
for file in os.listdir(queries_pathNIC):
if file.endswith(".txt"):
l+=1
filelist.append(file)
return(l)
Where the problem arises in the main function:
for round in range(0,queriesDirer()):
print ("\nQuery :",filelist[round])
file_query = open(queries_pathNIC+filelist[round],'r'); # problem on this line
file_query = str(file_query.read())
Contents of queriesNIC folder
00_1_Hardware_WelcomeNew.txt
00_2_Software_WelcomeNew.txt
00_3_Software_WelcomeNew_AUTORENEW.txt
The scripts runs without a problem, but if I change the first query name to
"00_1_Hardware_WelcomeNew_sth.txt" or anything different, I get the following error message:
FileNotFoundError: [Errno 2] No such file or directory: './queriesNIC/00_1_Hardware_WelcomeNew.txt'
I have also tried adding new text files to the folder (example: "00_1_Hardware_Other.txt") and the script simply skips processing the ones I added altogether and only goes with the original files.
I am using Python 3.4.
Does anyone have any suggestions what might be the problem?
Thank you
The following approach would be an improvement. The glob module can produce a list of files ending with .txt quite easily without needing to create a list.
import glob, os
queries_pathNIC = "./queriesNIC/"
def queriesDirer(directory):
return glob.glob(os.path.join(directory, "*.txt"))
for file_name in queriesDirer(queries_pathNIC):
print ("Query :", file_name)
with open(file_name, 'r') as f_query:
file_query = f_query.read()
From the sample you have given, it is not clear if you need further access to the round variable or the file list.
Currently I am trying to read in a csv file using the csv module in python. When I run the piece of code below I get an error that states the file does not exist. My first guess is that maybe, I have the file saved in the wrong place or I need to provide pyton with a file path. currently I have the file saved in C:\Documents and Settings\eag29278\My Documents\python test code\test_satdata.csv.
one side note im note sure wether having set the mode to 'rb' (read binary) was the right move.
import csv
with open('test_satdata.csv', 'rb') as csvfile:
satreader = csv.reader(csvfile, delimiter=' ', lineterminator=" ")
for row in satreader:
print ', '.join(row)
Here is the errror code.
Traceback (most recent call last):
File "C:/Python27/test code/test csv parse.py", line 2, in <module>
with open('test_satdata.csv', 'rb') as csvfile:
IOError: [Errno 2] No such file or directory: 'test_satdata.csv'
Your code is using a relative path; python is looking in the current directory (whatever that may be) to load your file. What the current directory is depends on how you started your Python script and if you executed any code that may have changed the current working directory.
Use a full absolute path instead:
path = r'C:\Documents and Settings\eag29278\My Documents\python test code\test_satdata.csv'
with open(path, 'rb') as csvfile:
Using 'rb' is entirely correct, the csv module recommends you do so:
If csvfile is a file object, it must be opened with the ‘b’ flag on platforms where that makes a difference.
Windows is such a platform.
You can hit this error when you're running a Python script from a Directory where the file is not contained.
Sounds simple to fix, put the CSV file in the same folder as the .PY file. However when you're running under an IDE like VSCode the command output might cd to another directory when it executes your python file.
PS C:\git\awesome> cd 'c:\git\awesome'; ${env:PYTHONIOENCODING}='UTF-8'; ${env:PYTHONUNBUFFERED}='1';
& 'C:\Program Files (x86)\Python37-32\python.exe' 'c:\Users\jeremy\.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\ptvsd_launcher.py'
'--default' '--client' '--host' 'localhost' '--port' '1089'
'c:\git\Project\ReadCSV.py'
See how I'm in my awesome repo and the first command is: cd 'c:\git\awesome';
Then it executes the python file: 'c:\git\Project\ReadCSV.py'
So its expecting the CSV file in 'c:\git\awesome'.
To fix it, either use the full file names or CD to the directory containing the CSV file you wish to read.
Your current guess is right: either put the file in your test code directory or point python to the right path.
Make a fresh rename of your folder. That worked for me.