With the below code I receive IOError: [Errno 13] Permission denied, and I know this is due to the output directory being a sub-folder of the input directory:
import datetime
import os
inputdir = "C:\\temp2\\CSV\\"
outputdir = "C:\\temp2\\CSV\\output\\"
keyword = "KEYWORD"
for path, dirs, files in os.walk(os.path.abspath(inputdir)):
for f in os.listdir(inputdir):
file_path = os.path.join(inputdir, f)
out_file = os.path.join(outputdir, f)
with open(file_path, "r") as fh, open(out_file, "w") as fo:
for line in fh:
if keyword not in line:
fo.write(line)
However, when I change the output folder to: outputdir = "C:\\temp2\\output\\" the code runs successfully. I want to be able to write the modified files to a sub-folder of the input directory. How would I do this without getting the 'Permission denied' error? Would the tempfile module be useful in this scenario?
os.listdir will return directory as well as file names. output is within inputdir so the with is trying to open a directory for reading/writing.
What exactly are you trying to do? path, dirs, files aren't even being used in the recursive os.walk.
Edit: I think you're looking for something like this:
import os
INPUTDIR= "c:\\temp2\\CSV"
OUTPUTDIR = "c:\\temp2\\CSV\\output"
keyword = "KEYWORD"
def make_path(p):
'''Makes sure directory components of p exist.'''
try:
os.makedirs(p)
except OSError:
pass
def dest_path(p):
'''Determines relative path of p to INPUTDIR,
and generates a matching path on OUTPUTDIR.
'''
path = os.path.relpath(p,INPUTDIR)
return os.path.join(OUTPUTDIR,path)
make_path(OUTPUTDIR)
for path, dirs, files in os.walk(INPUTDIR):
for d in dirs:
dir_path = os.path.join(path,d)
# Handle case of OUTPUTDIR inside INPUTDIR
if dir_path == OUTPUTDIR:
dirs.remove(d)
continue
make_path(dest_path(dir_path))
for f in files:
file_path = os.path.join(path, f)
out_path = dest_path(file_path)
with open(file_path, "r") as fh, open(out_path, "w") as fo:
for line in fh:
if keyword not in line:
fo.write(line)
If you are successful in writing to a output directory outside of the input traversing directory, then write it there first using the same code as above and then move it to a sub-directory within the input directory. You could use os.move for that.
Related
I'm trying to loop through files in multiple subdirectories in order to :
1- Add some text inside the files (ending with .ext)
2- Change the extension of each file from .ext to .ext2
The script works fine when I have only one subdir in the main directory, but when I try to run the script on multiple subdirs it says:
line 8, in
with open(name, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: "here the name of the subdir"
import os
directory = 'C:\\Users\\folder\\subfolders'
for dir, subdirs, files in os.walk(directory):
for name in files:
if name.endswith((".ext")):
with open(name, "r") as f:
XMLContent = f.readlines()
XMLContent.insert(6, '<XMLFormat>\n')
XMLContent.insert(40, '\n</XMLFormat>')
with open(name, "w") as f:
XMLContent = "".join(XMLContent)
f.write(XMLContent)
os.rename(os.path.join(dir, name), os.path.join(dir, name[:name.index('.ext')] +".ext1"))
Above is a screenshot of the sub dirs I have in the folder (1.Modified).
I've also created a new folder called all and put in it three folders and for each folder, I've created 2 files of .ext type.
So, I was able to write inside each file of them and change its name as well.
import os
for root, dirs, files in os.walk("/Users/ghaith/Desktop/test/all"):
for file in files:
if file.endswith('.ext'):
path = root + '/' + file
with open(path, "r") as f:
content = f.readlines()
content.insert(1, '<XMLFormat>\n')
content.insert(3, '\n</XMLFormat>')
with open(path, "w") as f:
content = "".join(content)
f.write(content)
os.rename(path, path+'2')
Output:
< XMLFormat >
< /XMLFormat >
you need to pass the directory to open the file
with open(os.path.join(directory, name), "r") as f:
But, I think the best way is use the os.listdir() to loop in the directory
for item in os.listdir(directory):
if item.endswith(".ext"):
with open(os.path.join(directory, item), "r") as r:
I am trying to open the file from folder and read it but it's not locating it. I am using Python3
Here is my code:
import os
import glob
prefix_path = "C:/Users/mpotd/Documents/GitHub/Python-Sample-
codes/Mayur_Python_code/Question/wx_data/"
target_path = open('MissingPrcpData.txt', 'w')
file_array = [os.path.abspath(f) for f in os.listdir(prefix_path) if
f.endswith('.txt')]
file_array.sort() # file is sorted list
for f_obj in range(len(file_array)):
file = os.path.abspath(file_array[f_obj])
join_file = os.path.join(prefix_path, file) #whole file path
for filename in file_array:
log = open(filename, 'r')#<---- Error is here
Error: FileNotFoundError: [Errno 2] No such file or directory: 'USC00110072.txt'
You are not giving the full path to a file to the open(), just its name - a relative path.
Non-absolute paths specify locations in relation to current working directory (CWD, see os.getcwd).
You would have to either os.path.join() correct directory path to it, or os.chdir() to the directory that the files reside in.
Also, remember that os.path.abspath() can't deduce the full path to a file just by it's name. It will only prefix its input with the path of the current working directory, if the given path is relative.
Looks like you are forgetting to modify the the file_array list. To fix this, change the first loop to this:
file_array = [os.path.join(prefix_path, name) for name in file_array]
Let me reiterate.
This line in your code:
file_array = [os.path.abspath(f) for f in os.listdir(prefix_path) if f.endswith('.txt')]
is wrong. It will not give you a list with correct absolute paths. What you should've done is:
import os
import glob
prefix_path = ("C:/Users/mpotd/Documents/GitHub/Python-Sample-"
"codes/Mayur_Python_code/Question/wx_data/")
target_path = open('MissingPrcpData.txt', 'w')
file_array = [f for f in os.listdir(prefix_path) if f.endswith('.txt')]
file_array.sort() # file is sorted list
file_array = [os.path.join(prefix_path, name) for name in file_array]
for filename in file_array:
log = open(filename, 'r')
You are using relative path where you should be using an absolute one. It's a good idea to use os.path to work with file paths. Easy fix for your code is:
prefix = os.path.abspath(prefix_path)
file_list = [os.path.join(prefix, f) for f in os.listdir(prefix) if f.endswith('.txt')]
Note that there are some other issues with your code:
In python you can do for thing in things. You did for thing in range(len(things)) it's much less readable and unnecessary.
You should use context managers when you open a file. Read more here.
I have multiple files in 7 different folder directories. All of these files have the same name, and I want to combine those files with the same name as one file, in another directory
import os
from itertools import chain
paths = (r'C:/Users/Test_folder/Input/', r'C:/Users/Test_folder/Input_2/')
for path, dirs, files in chain.from_iterable(os.walk(path) for path in paths):
for fname in paths:
for line in fname:
f = open(os.path.join(r'C:/Users/Test_folder/Test_output/', os.path.basename(fname)), 'a')
f.write ('{:}\n'.format(line))
f.close()
Error:
f = open(os.path.join(r'C:/Users/Test_folder/Test_output/', os.path.basename(fname)), 'a')
IOError: [Errno 13] Permission denied: 'C:/Users/Test_folder/Test_output/'
>>>
For issue of permisson denied
with open(os.path.join('type filename here' , os.path.basename(line)), 'w')
Or
for filename in os.listdir(src):
path = os.path.join(src, filename)
with open(path, "r") as inputFile:
content = inputFile.read()
The logic of your code is wrong:
for fname in paths should be for fname in files
for line in fname will not read the file fname line by line as fname is a string, not a file object
The permission error is due to that your code try to open a directory for appending.
Try:
import os
from itertools import chain
paths = (r'C:/Users/Test_folder/Input/', r'C:/Users/Test_folder/Input_2/')
for path, dirs, files in chain.from_iterable(os.walk(path) for path in paths):
for fname in files:
with open(os.path.join(path, fname)) as fin, open(os.path.join('C:/Users/Test_folder/Test_output/', fname), 'a') as fout:
fout.write(fin.read())
If you're using windows, re-run your ide as administrator.
There are many posts related with moving several files from one existing directory to another existing directory. Unfortunately, that has not worked for me yet, in Windows 8 and Python 2.7.
My best attempt seems to be with this code, because shutil.copy works fine, but with shutil.move I get
WindowsError: [Error 32] The process cannot access the file because it
is being used by another process
import shutil
import os
path = "G:\Tables\"
dest = "G:\Tables\Soil"
for file in os.listdir(path):
fullpath = os.path.join(path, file)
f = open( fullpath , 'r+')
dataname = f.name
print dataname
shutil.move(fullpath, dest)
del file
I know that the problem is that I don't close the files, but I've already tried it, both with del file and close.file().
Another way I tried is as follows:
import shutil
from os.path import join
source = os.path.join("G:/", "Tables")
dest1 = os.path.join("G:/", "Tables", "Yield")
dest2 = os.path.join("G:/", "Tables", "Soil")
#alternatively
#source = r"G:/Tables/"
#dest1 = r"G:/Tables/Yield"
#dest2 = r"G:/Tables/Soil"
#or
#source = "G:\\Tables\\Millet"
#dest1 = "G:\\Tables\\Millet\\Yield"
#dest2 = "G:\\Tables\\Millet\\Soil"
files = os.listdir(source)
for f in files:
if (f.endswith("harvest.out")):
shutil.move(f, dest1)
elif (f.endswith("sow.out")):
shutil.move(f, dest2)
If I use os.path.join (either using "G:" or "G:/"), then I get
WindowsError: [Error 3] The system cannot find the path specified:
'G:Tables\\Yield/*.*',
If I use forward slashes (source = r"G:/Tables/"), then I get
IOError: [Errno 2] No such file or directory: 'Pepper_harvest.out'*,
I just need one way to move files from one folder to another, that's all...
shutil.move is probably looking in the current working directory for f, rather than the source directory. Try specifying the full path.
for f in files:
if (f.endswith("harvest.out")):
shutil.move(os.path.join(source, f), dest1)
elif (f.endswith("sow.out")):
shutil.move(os.path.join(source, f), dest2)
This should work; let me know if it doesn't:
import os
import shutil
srcdir = r'G:\Tables\\'
destdir = r'G:\Tables\Soil'
for filename in os.listdir(path):
filepath = os.path.join(path, filename)
with open(filepath, 'r') as f:
dataname = f.name
print dataname
shutil.move(fullpath, dest)
The script below should open all the files inside the folder 'pruebaba' recursively but I get this error:
Traceback (most recent call last):
File
"/home/tirengarfio/Desktop/prueba.py",
line 8, in
f = open(file,'r') IOError: [Errno 21] Is a directory
This is the hierarchy:
pruebaba
folder1
folder11
test1.php
folder12
test1.php
test2.php
folder2
test1.php
The script:
import re,fileinput,os
path="/home/tirengarfio/Desktop/pruebaba"
os.chdir(path)
for file in os.listdir("."):
f = open(file,'r')
data = f.read()
data = re.sub(r'(\s*function\s+.*\s*{\s*)',
r'\1echo "The function starts here."',
data)
f.close()
f = open(file, 'w')
f.write(data)
f.close()
Any idea?
Use os.walk. It recursively walks into directory and subdirectories, and already gives you separate variables for files and directories.
import re
import os
from __future__ import with_statement
PATH = "/home/tirengarfio/Desktop/pruebaba"
for path, dirs, files in os.walk(PATH):
for filename in files:
fullpath = os.path.join(path, filename)
with open(fullpath, 'r') as f:
data = re.sub(r'(\s*function\s+.*\s*{\s*)',
r'\1echo "The function starts here."',
f.read())
with open(fullpath, 'w') as f:
f.write(data)
You're trying to open everything you see. One thing you tried to open was a directory; you need to check if an entry is a file or is a directory, and make a decision from there. (Was the error IOError: [Errno 21] Is a directory not descriptive enough?)
If it is a directory, then you'll want to make a recursive call to your function to walk over the files in that directory as well.
Alternatively, you might be interested in the os.walk function to take care of the recursive-ness for you.
os.listdir lists both files and directories. You should check if what you're trying to open really is a file with os.path.isfile