This question already has answers here:
How to open every file in a folder
(8 answers)
Closed 2 years ago.
I have looked at multiple questions & answers across SO, as well as other platforms pertaining to reading text files in a folder, but unfortunately none seems to work for me at the moment. I have multiple text files in a folder and would like to read them all, and put each text file as a string into a new list new_list.
path = "MyNews_AccidentDataset/News_txt.txt"
all_files = os.listdir(path)
Using this gives me all_files as a list with names of all text files
'0185_Man dies after 100ft turbine fall .txt',
'0131_Deaths_from_Working_with_Wind_Energy - Copy (5) - Copy.txt',
'0001_BENDING_WITH_THE_WIND._Modern_Power_System_N.txt']
.......
However, when I use open() to read the file,
new_list = []
for fle in all_files:
# open the file and then call .read() to get the text
with open(fle) as f:
text = f.read()
new_list.append(text)
I get the following error:-
with open(fle) as f:
FileNotFoundError: [Errno 2] No such file or directory: '0106_Car_vehicles_part_falls_on_the_roadway.txt'
although the mentioned file exists in the folder.
Any help in this regard is appreciated.
EDIT
Using complete path as in suggested comment by #bexi
for fle in all_files:
# open the file and then call .read() to get the text
with open(os.path.join(path, fle)) as f:
text = f.read()
I suppose all files ends with .txt:
new_list = []
for root, dirs, files in os.walk(<path to your folder>):
for file in files:
if file.endswith('.txt')
with open(os.path.join(root, file), 'r') as f:
text = f.read()
new_list.append(text)
Based on some other comments and answers I got UnicodeDecodeError: 'ascii' codec can't decode byte 0x93 in position 643: ordinal not in range(128). Finally, I could successfully solve the issue by setting the read mode as binary "rb" instead of "r":-
for fle in all_files:
# open the file and then call .read() to get the text
with open(os.path.join(path, fle),"rb") as f:
text = f.read()
new_list.append(text)
Related
This question already has answers here:
generalize python script to run on all files in a directory
(2 answers)
Closed 8 months ago.
i have this code that takes in a text folder and takes the 25th element in the first line of the file and place it in the 7th. However, this code opens only one text file and writes it to another but what i want that the code reads all the files in the folder and writes them in the same path.
index= 1
with open("3230c237cnc274c.txt", "r") as f:
file = f.readlines()
line = file[index].split(';')
target = line[24]
blank = line[6]
line[6] = target
line[24] = ""
file[index] = ';'.join(line)
with open("aaaaaaaaaaaaaaaa.txt", 'w') as f:
for line in file:
f.write(line)
I like to use the glob module for things like this. See if this helps:
import glob
all_text_files = glob.glob("*.txt")
for text_file in all_text_files:
with open(text_file, "r") as f:
lines = f.readlines()
# do something with the lines...
The syntax "*.txt" indicates all files ending with the .txt extension. This then returns a list of all those filenames. If your files are in a folder somewhere, you can also do "folder/*.txt", and there's a few other nice tricks with glob
This question already has answers here:
How do I append to a file?
(13 answers)
Closed 1 year ago.
I have 500 text files in one folder, each of them looks like this:
1-1,2-4,In,_,
1-2,5-9,this,_,
1-3,10-15,paper,_,
1-4,16-18,we,_,
1-5,19-26,propose,_,
1-6,27-28,a,Project[1],
I need to add one word at the end of the text in each of the files. The result I expect is:
1-1,2-4,In,_,
1-2,5-9,this,_,
1-3,10-15,paper,_,
1-4,16-18,we,_,
1-5,19-26,propose,_,
1-6,27-28,a,Project[1],
end
How do I write inside the with block?
import os
path = "All_TSV_Files"
files = [file for file in os.listdir(path) if file.endswith(".txt")]
for file in files:
with open(os.path.join(path, file), 'r',encoding='utf-8') as f:
### add "end" to end of the file
or should I use pandas data frame to do them?
Say your file is called "foo.txt", you can open it with intend of appending to it like this:
with open("foo.txt", "a") as f:
f.write("\nend")
The \n denotes a newline before inserting end.
This answer should be helpful:
Write to the last line of a text file?
Just open file in append mode (pointer will be in the ned of file) and write line.
I am trying to read files with .txt extension and wanted to append into a single txt file. I could read data. But what is the best way to write into single .txt file?
sources = ["list of paths to files you want to write from"]
dest = open("file.txt", "a")
for src in sources:
source = open(src, "r")
data = source.readlines()
for d in data:
dest.write(d)
source.close()
dest.close()
If your destination doesnt already exist you can use "w"(write) mode instead of "a"(append) mode.
Try this.
x.txt:
Python is fun
y.txt:
Hello World. Welcome to my code.
z.txt:
I know that python is popular.
Main Python file:
list_=['x.txt','y.txt','z.txt']
new_list_=[]
for i in list_:
x=open(i,"r")
re=x.read()
new_list_.append(re)
with open('all.txt',"w") as file:
for line in new_list_:
file.write(line+"\n")
After you find the filenames, if you have a lot of files you should avoid string concatenation when merging file contents because in python string concatenation comes with O(n) runtime cost. I think the code below demonstrates the full example.
import glob
# get every txt files from the current directory
txt_files = glob.iglob('./*.txt')
def get_file_content(filename):
content = ''
with open(filename, 'r') as f:
content = f.read()
return content
contents = []
for txt_file in txt_files:
contents.append(get_file_content(txt_file))
with open('complete_content.txt', 'w') as f:
f.write(''.join(contents))
This question already has answers here:
How to delete a specific line in a file?
(17 answers)
Closed 4 years ago.
Let's say I have a text file full of random lines. How can I keep a specific line and delete the others from this file, using Python?
The script should search for all .txt files from a directory/and sub directory.
Firstly, to search for all .txt files, you can use this:
import glob, os
os.chdir("/mydir")
for filePath in glob.glob("*.txt"):
...
Sourced from here
Secondly, to look through a file and find the line you want, use:
myfile = open(filePath, "r")
And look to see if your desired line is in the file.
if desiredLine in myfile:
...
Then you can close the file and reopen it in write mode.
myfile.close()
myfile = open(filePath, "w")
Then all you have to do is write the line you wanted to keep.
myfile.write(desiredLine)
Then close the file again
myfile.close()
Sourced from here
The final script ends up like this:
import glob, os
os.chdir("/mydir")
for filePath in glob.glob("*.txt"):
myfile = open(filePath, "r")
if desiredLine in myfile:
myfile.close()
myfile = open(filePath, "w")
myfile.write(desiredLine)
myfile.close()
Note, if your line doesn't exist in the file you're checking, it is left as is.
Hope this helps you
This question already has answers here:
open() gives FileNotFoundError / IOError: '[Errno 2] No such file or directory'
(8 answers)
Why doesn't calling a string method (such as .replace or .strip) modify (mutate) the string?
(3 answers)
Closed 6 months ago.
I am very new to Python so please forgive the following basic code and problem, but I have been trying to figure out what is causing the error I am getting (I have even looked at similar threads on S.O.) but can't get past my issue.
Here is what I am trying to do:
loop through a folder of CSV files
search for a 'keyword' and delete all lines containing the 'keyword'
save output to a separate folder
Here is my code:
import os, fnmatch
import shutil
src_dir = "C:/temp/CSV"
target_dir = "C:/temp/output2"
keyword = "KEYWORD"
for f in os.listdir(src_dir):
os.path.join(src_dir, f)
with open(f):
for line in f:
if keyword not in line:
write(line)
shutil.copy2(os.path.join(src_dir, f), target_dir)
Here is the error I am getting:
IOError: [Errno 2] No such file or directory: 'POS_03217_20120309_153244.csv'
I have confirmed that the folder and file do exist. What is causing the IOError and how to I resolve it? Also, is there anything else wrong with my code that would prevent me from performing the entire task?
Hmm, there are a few things going wrong here.
for f in os.listdir(src_dir):
os.path.join(src_dir, f)
You're not storing the result of join. This should be something like:
for f in os.listdir(src_dir):
f = os.path.join(src_dir, f)
This open call is is the cause of your IOError. (Because without storing the result of the join above, f was still just 'file.csv', not 'src_dir/file.csv'.)
Also, the syntax:
with open(f):
is close, but the syntax isn't quite right. It should be with open(file_name) as file_object:. Then, you use to the file_object to perform read or write operations.
And finally:
write(line)
You told python what you wanted to write, but not where to write it. Write is a method on the file object. Try file_object.write(line).
Edit: You're also clobbering your input file. You probably want to open the output file and write lines to it as you're reading them in from the input file.
See: input / output in python.
Even though #Ignacio gave you a straightforward solution, I thought I might add an answer that gives you some more details about the issues with your code...
# You are not saving this result into a variable to reuse
os.path.join(src_dir, f)
# Should be
src_path = os.path.join(src_dir, f)
# you open the file but you dont again use a variable to reference
with open(f)
# should be
with open(src_path) as fh
# this is actually just looping over each character
# in each result of your os.listdir
for line in f
# you should loop over lines in the open file handle
for line in fh
# write? Is this a method you wrote because its not a python builtin function
write(line)
# write to the file
fh.write(line)
Um...
with open(os.path.join(src_dir, f)) as fin:
for line in fin:
Also, you never output to a new file.
I solved the problem like so:
src_dir = "C:\\temp\\CSV\\"
target_dir = "C:\\temp\\output2\\"
keyword = "KEYWORD"
for f in os.listdir(src_dir):
file_name = os.path.join(src_dir, f)
out_file = os.path.join(target_dir, f)
with open(file_name, "r+") as fi, open(out_file, "w") as fo:
for line in fi:
if keyword not in line:
fo.write(line)
I got this error and fixed by appending the directory path in the loop. script not in the same directory as the files. dr1 ="~/test" directory variable
fileop=open(dr1+"/"+fil,"r")