We Keep Coding

Dynamic Path for Opening Files in an iterative process - Python

Looking for some helpful tips. I am trying to read through multiple .txt files present in a folder and its sub-folders and then use regular expression to extract phone numbers from them. I am able to do that using a for loop on os.walk. But when I try to read individual files in a nested loop, it keeps on throwing one error after another. Basically what i learnt so far is that it may not be opening the file as it is not finding it in the same directory and the path needs to be dynamic. I tried using {folder\subfolder\f} but didn't work, also tried os.path.join but to no avail. Any help is really appreciated.
'''
import re
import os
lst = []
for folder, subfolder, files in os.walk(top):
for f in files:
lst.append(re.findall(r'\d{3}-\d{3}-\d{4}', open(f, 'r+', encoding='utf-8', errors='ignore').read()))
return lst
'''
Resulting error:
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
<ipython-input-54-6932d4b1e2e1> in <module>
4 for folder, subfolder, files in os.walk(top):
5 for f in files:
----> 6 lst.append(re.findall(r'\d{3}-\d{3}-\d{4}', open(f, 'r+', encoding='utf-8', errors='ignore').read()))
7 return lst
FileNotFoundError: [Errno 2] No such file or directory: 'Instructions.txt'

You shouldn't join subfolder in the path since that is a list of subfolders in the actual folder you are walking through.
You should join folder and the file name from files:
open(os.path.join(folder, f), 'r+', encoding='utf-8', errors='ignore').read()))

i got this message :] File b'USA_Housing.csv' does not exist: b'USA_Housing.csv,what can i do

I already looked through all similar posts on this issue, but I couldn't find any solution. So far Pandas read through all my CSV files without any problem, however now there seems to be a problem. When I do
df = pd.read_csv('USA_Housing.csv')
I get:
FileNotFoundError: [Errno 2] File b'USA_Housing.csv' does not exist: b'USA_Housing.csv

Is the file in the same directory as the .py file you are running?
If all the files are in the same directory, you can use the 'os' package and do something like:
files = os.listdir
csv_files = [f for f in files if f[-4:] == ".csv"]
This will return a list of all files and directories in the current working folder and means you can then parse through them and load the iteratively.

File not found error in os.rename

I am trying to write a program to categorize into folders a large amount of files according to their respective groups indicated in the file name. I wrote the followin code, but when I run it it gives me a file not found error, even though the file is in the given path. I'd appreciate any help in figuring out what is wrong.
import os
old_dir = '/Users/User/Desktop/MyFolder'
for f in os.listdir(old_dir):
file_name, file_ext = os.path.splitext(f)
file_name.split('-')
split_file_name = file_name.split('-')
new_dir = os.path.join(old_dir,
'-'.join(split_file_name[:3]),
split_file_name[5],
f)
os.rename(os.path.join(old_dir, f), new_dir)
Here's the error:
Traceback (most recent call last):
File "/Users/User/Documents/Sort Files into Folders/Sort Files into Folders.py", line 19, in <module>
os.rename(os.path.join(old_dir, f), new_dir)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/User/Desktop/MyFolder/AHA35-3_30x1_12-31-7d-g1a1-ArmPro.jpg' -> '/Users/User/Desktop/MyFolder/AHA35-3_30x1_12-31/ArmPro/AHA35-3_30x1_12-31-7d-g1a1-ArmPro.jpg

os.rename does not automatically create new directories (recursively), if the new name happens to be a filename in a directory that does not exist.
To create the directories first, you can (in Python 3) use:
os.makedirs(dirname, exist_ok=True)
where dirname can contain subdirectories (existing or not).
Alternatively, use os.renames, that can handle new and intermediate directories. From the documentation:
Recursive directory or file renaming function. Works like rename(), except creation of any intermediate directories needed to make the new pathname good is attempted first

os.rename need path, so it should look like:
os.rename(path+old_name, path+new_name)

No such file or directory while trying to read files in a directory python

I have this code which lists the files in the directory and parses each one of them with my function.
paths = []
for filename in os.listdir(r"C:\Program Files (x86)\Folder\Folder"):
with open(filename) as f:
paths.append(parse_file(f))
I am getting the error:
File "find.py", line 21, in <module>
with open(filename) as f:
IOError: [Errno 2] No such file or directory: 'file.txt'
This error shows that it saw file.txt because it exist in the folder I specified in os.listdir, I have many more files there. If I delete file.txt it will show the error on a different file.
I also tried to move the files to a directory in my desktop and the script worked fine.
What is the issue I don't understand. I am pretty new to python so forgive me if its dumb question. Thank you!

os.listdir() returns filenames, not paths. Join them with the directory name to make absolute paths:
path = r"C:\Program Files (x86)\Folder\Folder"
for filename in os.listdir(path):
with open(os.path.join(path, filename)) as f:

Error Reading From A file Inside A Directory

I am trying to read files, inside a folder, reason is the number of files inside the folder is not fixed, but if there are 3 text folder, I have to read all the 3 files if 4 all the 4 text files.
Here is the code I'm trying to use, but comes up with an IOError:
for i in os.listdir("./RecordFolder"):
print i
Output is:
record1.txt
record2.txt
Now the problem is reading the files:
for files in os.listdir("./RecordFolder"):
filecontent = open(files).readlines()
for lines in filecontent:
print lines
Output:
IOError: [Errno 2] No such file or directory: 'record.txt'
Need some help here, thanks

The function os.listdir() only returns the file names, not the full paths, so you should use os.path.join() to add the directory names:
directory = "./RecordFolder"
for filename in os.listdir(directory):
with open(os.path.join(directory, filename)) as f:
for line in f:
print line
(Also note that you shouldn't use file.readlines() for simply iterating over the lines of the files, and that your code fails to close the file. These problems are fixed in the code above.)

os.listdir only returns the name of the files inside the folder. Not the entire path.
You want something like:
directory = os.path.join(os.curdir, "RecordFolder")
for files in os.listdir(directory):
filecontent = open( os.path.join(directory, files) ).readlines()
for lines in filecontent:
print lines
But this isn't the best way to get the lines from a file. Look into python context managers -- specifically:
with open(filename) as f:
for line in f:
print line
This makes sure that the file is properly closed (even in the event of an error -- barring a segmentation fault in the interpreter itself or something bizarre like that).