I have a problem with reading some massive text-files.
I firstly define reading my text file as follows:
def reader(filename):
with open(filename, encoding='latin-1') as thefile:
contentsofthefile = f.read()
return contentsofthefile
Now I want to have another function, that uses the above function, such as:
def remover(filename):
a = reader(filename)
for line in a:
do this
This yields the following problem:
OSError: [Errno 63] File name too long: 'In search of lost time - CHAPTER///1 \nThe characters, plotlines, ...."
It seems that it attempts to read the entire file as the filename?
If you're going to process the file line by line, there's no reason not to read the file line by line as well. You don't really need a reader function, but it can be as simple as
def reader(filename):
return open(filename, encoding='latin1=1')
Then to use reader inside remover:
def remover(filename):
with reader(filename) as f:
for line in f:
...
remover("somefile.txt")
Related
I can't close this file, as the file is directly fed into the 'lines'-list.
I have tried closing with command lines.close() but it doesn't work.
def readfile():
lines = [line.rstrip('\n') for line in open('8ballresponses.txt', 'r')]
print(random.choice(lines))
I don't get an error, but i want to be able to close the file.
Instead of file object, lines is a list , so you can't close it. And you should store file object open('8ballresponses.txt', 'r') with a variable for closing it later:
def readfile(file_path):
test_file = open(file_path, 'r')
lines = [line.rstrip('\n') for line in test_file]
test_file.close()
print(random.choice(lines))
Or simply use with "to close a file in python, without a file object":
def readfile(file_path):
with open(file_path, 'r') as test_file:
lines = [line.rstrip('\n') for line in test_file]
print(lines)
you can use with open command. this will automatically handle all the test cases failure etc. (inbuild try except and finally in python)
below is example similiar to your code
import random
def readfile():
lines = []
with open(r"C:\Users\user\Desktop\test\read.txt",'r') as f:
lines = f.readlines()
print(random.choice(lines))
You can use try and finally block to do the work.
For example :
def readfile():
file = open('8ballresponses.txt', 'r')
try:
lines = [line.rstrip('\n') for line in file]
print(random.choice(lines))
finally:
file.close()
In this post
"When the with ends, the file will be closed. This is true even if an exception is raised inside of it."
You manually invoke the close() method of a file object explicitly or implicitly by leaving a with open(...): block. This works of course always and on any Python implementation.
Use with this will close implicitly after the block is complete
with open('8ballresponses.txt', 'r') as file:
lines = [ line.rstrip("\n") for line in file ]
I have a file that I open and i want to search through till I find a specific text phrase at the beginning of a line. I then want to overwrite that line with 'sentence'
sentence = "new text" "
with open(main_path,'rw') as file: # Use file to refer to the file object
for line in file.readlines():
if line.startswith('text to replace'):
file.write(sentence)
I'm getting:
Traceback (most recent call last):
File "setup_main.py", line 37, in <module>
with open(main_path,'rw') as file: # Use file to refer to the file object
ValueError: must have exactly one of create/read/write/append mode
How can I get this working?
You can open a file for simultaneous reading and writing but it won't work the way you expect:
with open('file.txt', 'w') as f:
f.write('abcd')
with open('file.txt', 'r+') as f: # The mode is r+ instead of r
print(f.read()) # prints "abcd"
f.seek(0) # Go back to the beginning of the file
f.write('xyz')
f.seek(0)
print(f.read()) # prints "xyzd", not "xyzabcd"!
You can overwrite bytes or extend a file but you cannot insert or delete bytes without rewriting everything past your current position.
Since lines aren't all the same length, it's easiest to do it in two seperate steps:
lines = []
# Parse the file into lines
with open('file.txt', 'r') as f:
for line in f:
if line.startswith('text to replace'):
line = 'new text\n'
lines.append(line)
# Write them back to the file
with open('file.txt', 'w') as f:
f.writelines(lines)
# Or: f.write(''.join(lines))
You can't read and write to the same file. You'd have to read from main_path, and write to another one, e.g.
sentence = "new text"
with open(main_path,'rt') as file: # Use file to refer to the file object
with open('out.txt','wt') as outfile:
for line in file.readlines():
if line.startswith('text to replace'):
outfile.write(sentence)
else:
outfile.write(line)
Not the problem with the example code, but wanted to share as this is where I wound up when searching for the error.
I was getting this error due to the chosen file name (con.txt for example) when appending to a file on Windows. Changing the extension to other possibilities resulted in the same error, but changing the file name solved the problem. Turns out the file name choice caused a redirect to the console, which resulted in the error (must have exactly one of read or write mode): Why does naming a file 'con.txt' in windows make Python write to console, not file?
In this code I tried to define a class which will write into a file.In the init method the name of the file is passed.I also defined a method named "write" to write to the file.Then I created an instance of the class and passed the value of the file name.After that, I called the write method and passed the message to write in the file.At last, I checked if the file is created and if the file has the message.Here's the code:
class Logfile(object):
def __init__(self,file_name):
self.file_name = file_name
def write(self,msg):
with open('self.file_name','w') as myFile:
myFile.write(msg)
log = Logfile('myNewFile.txt')
log.write("this is a log file.")
with open('myNewFile.txt','r') as readFile:
read_file = readFile.read()
for line in read_file:
print(line)
But, it shows an error:
FileNotFoundError: [Errno 2] No such file or directory: 'myNewFile.txt'
This python code is saved in a desktop folder called "My Folder".And when I go there, there is really no such file named "myNewFile.txt".
But, if I run the program with the checking part of the code, I mean,this part:
with open('myNewFile.txt','r') as readFile:
read_file = readFile.read()
for line in read_file:
print(line)
then, there is no error but still the "myNewFile.txt" is not created.
Can you please help me?
In the write method you should write :
with open(self.file_name, ‘w’) as my_file
because self.file_name is already a string.
log = Logfile('myNewFile.txt')
log.write("this is a log file.")
with open('myNewFile.txt','r') as readFile:
read_file = readFile.read()
for line in read_file:
print(line)
You could always try to open the file using with open(log) because you have already made the .txt file a variable. I have never tried this and have little experience with this type of code, but it could be worth a shot.
The context is the following one, I have two text file that I need to edit.
I open the first text file read it line by line and edit it but sometimes when I encounter a specific line in the first text file I need to overwritte content of the the second file.
However, each time I re-open the second text file instead of overwritting its content the below code appends it to the file...
Thanks in advance.
def edit_custom_class(custom_class_path, my_message):
with open(custom_class_path, "r+") as file:
file.seek(0)
for line in file:
if(some_condition):
file.write(mu_message)
def process_file(file_path):
with open(file_path, "r+") as file:
for line in file:
if(some_condition):
edit_custom_class(custom_class_path, my_message)
In my opinion, simultaneously reading and modifying a file is a bad thing to do. Consider using something like this. First read the file, make modifications, and then overwrite the file completely.
def modify(path):
out = []
f = open(path)
for line in f:
if some_condition:
out.append(edited_line) #make sure it has a \n at the end
else:
out.append(original_line)
f.close()
with open(path,'w') as f:
for line in out:
f.write(line)
I'm a beginner. I have written a Python program with the following pseduocode:
Define Function1.
a. This function takes a large single-fasta file (a genome) and splits it into pieces.
b. These pieces are written to a multi-fasta output file (ex. below).
Define Function2.
a. This function reads the lines of the multi-fasta file
b. Writes to an output file the fasta id followed by the length of the fasta entry.
most of the code:
from Bio import SeqIO
import io
def metagenome_simulator(genome_fasta, out_file):
outfile = open(out_file, "a+b")
fasta = SeqIO.parse(open(genome_fasta, "rU"), "fasta")
#does the split, blah, blah - I know this function works on its own already
len_file.close()
fasta.close()
return outfile
def contig_len_calculator(fasta, out_file):
outfile = io.open(out_file, "wb")
fhandle = io.open(fasta, "a+b")
outfile.write("contig_id" + "\t" + "contig_length" + "\n")
for entry in SeqIO.parse(fhandle, "fasta"):
#calculates lengths, blah, blah - i know this works independently too
outfile.close()
fhandle.close()
return
def main():
output = metagenome_simulator(sys.argv[1], sys.argv[2])
print(output)
contig_len_calculator(output, sys.argv[3])
main()
And my command (bash shell) would be:
./this_script.py genome_fasta_file split_fasta_out_file final_output_file.
The output would be two separate files, one for each function in the program. The first would be the split fasta:
>split_1
ATCG....
>split_2
ATCG....
.
.
.
And the second would be the lengths file:
>split_1 300
>split_2 550
.
.
.
This does not work. It runs Fuction1 just fine and makes the split_fasta_output file but then returns:
<open file 'out_file', mode 'a+b' at 0x7f54b8454d20>
Traceback (most recent call last):
File "./this_script.py", line 62, in <module>
main()
File "./this_script.py", line 60, in main
contig_len_calculator(output, sys.argv[3])
File "./this_script.py", line 47, in contig_len_calculator
fhandle = io.open(fasta, "a+b")
TypeError: invalid file: <open file 'out_file', mode 'a+b' at 0x7f54b8454d20>
I have no idea why it doesn't work. So my question is this: how do I properly pass a file created in one function to another function?
EDIT: Put the whole traceback error.
The problem is that metagenome_simulator returns a file descriptor, which you then try to pass into io.open. io.open takes either an integer file descriptor (some_fd.fileno()) or a path. The simple solution is then to return the path to your outfile, rather than the outfile itself.
def metagenome_simulator(genome_fasta, out_file):
... # your code as-written
return out_file
But if you like you could instead do:
def metagenome_simulator(genome_fasta, out_file):
# completely as-written, including
return outfile
def contig_len_calculator(fasta, out_file):
outfile = io.open(out_file, "wb")
fhandle = io.open(fasta.fileno(), "a+b")
...
The advantage of the first approach is that it makes the out_file and fasta arguments to contig_len_calculator have the same type, which seems sane.
The open function takes a filename and returns a file object. metagenome_simulator returns a file object. You pass this as fasta and then use open on it. But you do not need to open it since it's already an open file and not just a filename.