I was trying to iterate over the files in a directory like this:
import os
path = r'E:/somedir'
for filename in os.listdir(path):
f = open(filename, 'r')
... # process the file
But Python was throwing FileNotFoundError even though the file exists:
Traceback (most recent call last):
File "E:/ADMTM/TestT.py", line 6, in <module>
f = open(filename, 'r')
FileNotFoundError: [Errno 2] No such file or directory: 'foo.txt'
So what is wrong here?
It is because os.listdir does not return the full path to the file, only the filename part; that is 'foo.txt', when open would want 'E:/somedir/foo.txt' because the file does not exist in the current directory.
Use os.path.join to prepend the directory to your filename:
path = r'E:/somedir'
for filename in os.listdir(path):
with open(os.path.join(path, filename)) as f:
... # process the file
(Also, you are not closing the file; the with block will take care of it automatically).
os.listdir(directory) returns a list of file names in directory. So unless directory is your current working directory, you need to join those file names with the actual directory to get a proper absolute path:
for filename in os.listdir(path):
filepath = os.path.join(path, filename)
f = open(filepath,'r')
raw = f.read()
# ...
Here's an alternative solution using pathlib.Path.iterdir, which yields the full paths instead, removing the need to join paths:
from pathlib import Path
path = Path(r'E:/somedir')
for filename in path.iterdir():
with filename.open() as f:
... # process the file
Related
This question already has an answer here:
using os.remove() in os.walk() for loop returns FileNotFoundError
(1 answer)
Closed 11 months ago.
This is the code
import os
new_file = open("C:/Users/USER/Desktop/Coding/Python/element_search.txt", "w")
path = "C:/Users/USER/Desktop/Coding"
# This is to access sub-folders
dirs = os.listdir(path)
for root, dir, files in os.walk(path):
for file in files:
f = open(file)
content = f.read()
print(file)
And this is the error
C:\Users\USER\Desktop\Coding\Python\personal_projects\venv\Scripts\python.exe C:/Users/USER/Desktop/Coding/Python/personal_projects/element_search.py
Traceback (most recent call last):
File "C:\Users\USER\Desktop\Coding\Python\personal_projects\element_search.py", line 10, in <module>
f = open(file)
FileNotFoundError: [Errno 2] No such file or directory: 'launch.json'
But I have the file launch.json present.
The os.walk returns the walking directory as root, and the list of file names. You need to construct the full path of the file for opening, else python will search the file only in working directory.
You can construct the full path with pathlib or os.join. pathlib is the recommended option. See an edited approach below.
import os
import pathlib
new_file = open("C:/Users/USER/Desktop/Coding/Python/element_search.txt", "w")
path = "C:/Users/USER/Desktop/Coding"
# This is to access sub-folders
dirs = os.listdir(path)
# root is the parent directory and files is the list of names returned.
for root, _, files in os.walk(path): # do not use the dir as its a builtin keyword
for file in files:
# join the name and dir path, to make it readable
abs_file = pathlib.Path(root) / file # Prepare the absolute path for the file
with open(abs_file) as f: # Open with a context, so it closes after use
content = f.read()
print(content)
If you have binary files in the list, then you must provide the read as binary flag in opening like with open(abs_file, "rb")
I am trying to open the file from folder and read it but it's not locating it. I am using Python3
Here is my code:
import os
import glob
prefix_path = "C:/Users/mpotd/Documents/GitHub/Python-Sample-
codes/Mayur_Python_code/Question/wx_data/"
target_path = open('MissingPrcpData.txt', 'w')
file_array = [os.path.abspath(f) for f in os.listdir(prefix_path) if
f.endswith('.txt')]
file_array.sort() # file is sorted list
for f_obj in range(len(file_array)):
file = os.path.abspath(file_array[f_obj])
join_file = os.path.join(prefix_path, file) #whole file path
for filename in file_array:
log = open(filename, 'r')#<---- Error is here
Error: FileNotFoundError: [Errno 2] No such file or directory: 'USC00110072.txt'
You are not giving the full path to a file to the open(), just its name - a relative path.
Non-absolute paths specify locations in relation to current working directory (CWD, see os.getcwd).
You would have to either os.path.join() correct directory path to it, or os.chdir() to the directory that the files reside in.
Also, remember that os.path.abspath() can't deduce the full path to a file just by it's name. It will only prefix its input with the path of the current working directory, if the given path is relative.
Looks like you are forgetting to modify the the file_array list. To fix this, change the first loop to this:
file_array = [os.path.join(prefix_path, name) for name in file_array]
Let me reiterate.
This line in your code:
file_array = [os.path.abspath(f) for f in os.listdir(prefix_path) if f.endswith('.txt')]
is wrong. It will not give you a list with correct absolute paths. What you should've done is:
import os
import glob
prefix_path = ("C:/Users/mpotd/Documents/GitHub/Python-Sample-"
"codes/Mayur_Python_code/Question/wx_data/")
target_path = open('MissingPrcpData.txt', 'w')
file_array = [f for f in os.listdir(prefix_path) if f.endswith('.txt')]
file_array.sort() # file is sorted list
file_array = [os.path.join(prefix_path, name) for name in file_array]
for filename in file_array:
log = open(filename, 'r')
You are using relative path where you should be using an absolute one. It's a good idea to use os.path to work with file paths. Easy fix for your code is:
prefix = os.path.abspath(prefix_path)
file_list = [os.path.join(prefix, f) for f in os.listdir(prefix) if f.endswith('.txt')]
Note that there are some other issues with your code:
In python you can do for thing in things. You did for thing in range(len(things)) it's much less readable and unnecessary.
You should use context managers when you open a file. Read more here.
I need to read the contents of a file from the list of files from a directory with os.listdir. My working scriptlet is as follows:
import os
path = "/Users/Desktop/test/"
for filename in os.listdir(path):
with open(filename, 'rU') as f:
t = f.read()
t = t.split()
print(t)
print(t) gives me all the contents from all the files at once present in the directory (path).
But I like to print the contents on first file, then contents of the second and so on, until all the files are read from in dir.
Please guide ! Thanks.
You can print the file name.
Print the content after the file name.
import os
path = "/home/vpraveen/uni_tmp/temp"
for filename in os.listdir(path):
with open(filename, 'rU') as f:
t = f.read()
print filename + " Content : "
print(t)
First, you should find the path of each file using os.path.join(path, filename). Otherwise you'll loop wrong files if you change the variable path. Second, your script already provides the contents of all files starting with the first one. I added a few lines to the script to print the file path and an empty line to see where the contents end and begin:
import os
path = "/Users/Desktop/test/"
for filename in os.listdir(path):
filepath = os.path.join(path, filename)
with open(filepath, 'rU') as f:
content = f.read()
print(filepath)
print(content)
print()
os.listdir returns the name of the files only. you need to os.path.join that name with the path the files live in - otherwise python will look for them in your current working directory (os.getcwd()) and if that happens not to be the same as path python will not find the files:
import os
path = "/Users/Desktop/test/"
for filename in os.listdir(path):
print(filename)
file_path = os.path.join(path, filename)
print(file_path)
..
if you have pathlib at your disposal you can also:
from pathlib import Path
path = "/Users/Desktop/test/"
p = Path(path)
for file in p.iterdir():
if not file.is_file():
continue
print(file)
print(file.read_text())
I have a number of plain-text config files (.dta) that are spread through 27 sub-directories. I am trying to parse some information from all of them into a common document that is easier to work with.
Thus far I have:
import linecache
import csv
import os
csvout = csv.writer(open("dtaCompile.csv","wb"))
directory = os.path.join("c:\\","DirectKey")
for root,dirs,files in os.walk(directory):
for file in files:
if file.endswith(".DTA"):
f=open(file,'r')
lines = f.readlines()
description = lines[1]
articleCode = lines[2]
OS = lines[25]
SMBIOS = lines[32]
pnpID = lines[34]
cmdLine = lines[28]
csvout.writerow([SMBIOS, description, articleCode, pnpID, OS, cmdLine])
f.close()
I'm getting the following error:
Traceback (most recent call last):
File "test.py", line 11, in <module>
f=open(file,'r')
IOError: [Errno 2] No such file or directory: '000003APP.DTA'
Instead of
f=open(file,'r')
Your probaby need
f=open(os.path.join(directory, root, file),'r')
file is just the name of the file, and doesn't say anything about the path to it. you have to use os.path.join with the various components to create the full path
if file.endswith(".DTA"):
file = os.path.join(directory, root, file)
Instead of:
f=open(file,'r')
Try:
f = open(os.path.join(directory, file), "r")
My guess is that the directory you're program is executing in is not the same as the directory you're walking.
Try printing:
os.getcwd()
to see.
The script below should open all the files inside the folder 'pruebaba' recursively but I get this error:
Traceback (most recent call last):
File
"/home/tirengarfio/Desktop/prueba.py",
line 8, in
f = open(file,'r') IOError: [Errno 21] Is a directory
This is the hierarchy:
pruebaba
folder1
folder11
test1.php
folder12
test1.php
test2.php
folder2
test1.php
The script:
import re,fileinput,os
path="/home/tirengarfio/Desktop/pruebaba"
os.chdir(path)
for file in os.listdir("."):
f = open(file,'r')
data = f.read()
data = re.sub(r'(\s*function\s+.*\s*{\s*)',
r'\1echo "The function starts here."',
data)
f.close()
f = open(file, 'w')
f.write(data)
f.close()
Any idea?
Use os.walk. It recursively walks into directory and subdirectories, and already gives you separate variables for files and directories.
import re
import os
from __future__ import with_statement
PATH = "/home/tirengarfio/Desktop/pruebaba"
for path, dirs, files in os.walk(PATH):
for filename in files:
fullpath = os.path.join(path, filename)
with open(fullpath, 'r') as f:
data = re.sub(r'(\s*function\s+.*\s*{\s*)',
r'\1echo "The function starts here."',
f.read())
with open(fullpath, 'w') as f:
f.write(data)
You're trying to open everything you see. One thing you tried to open was a directory; you need to check if an entry is a file or is a directory, and make a decision from there. (Was the error IOError: [Errno 21] Is a directory not descriptive enough?)
If it is a directory, then you'll want to make a recursive call to your function to walk over the files in that directory as well.
Alternatively, you might be interested in the os.walk function to take care of the recursive-ness for you.
os.listdir lists both files and directories. You should check if what you're trying to open really is a file with os.path.isfile