Say I have this simple python script:
file = open('C:\\some_text.txt')
print file.readlines()
print file.readlines()
When it is run, the first print prints a list containing the text of the file, while the second print prints a blank list. Not completely unexpected I guess. But is there a way to 'wind back' the file so that I can read it again? Or is the fastest way just to re-open it?
You can reset the file pointer by calling seek():
file.seek(0)
will do it. You need that line after your first readlines(). Note that file has to support random access for the above to work.
For small files, it's probably much faster to just keep the file's contents in memory
file = open('C:\\some_text.txt')
fileContents = file.readlines()
print fileContents
print fileContents # This line will work as well.
Of course, if it's a big file, this could put strain on your RAM.
Remember that you can always use the with statement to open and close files:
from __future__ import with_statement
with open('C:\\some_text.txt') as file:
data = file.readlines()
#File is now closed
for line in data:
print line
Related
I want to write a file that says hello guys how are you but each word must be an item of list. Here is my code. It shows nothing when I run it, when I run second time it shows item by item as I want. But when I click text file, it is written two times.
with open('stavanger.txt','r+') as f: # file closes itself with with open as filename command
words = ['hello\n','guys\n','how\n', 'are\n','you\n']
f.writelines(words)
for i in f:
x=i.rstrip().split(',')#turn text file into list and we seperate list items by comma .
print(x)
The problem is that writing to a file uses a buffer. So after the line f.writelines(words) nothing really happened. Only the buffer changed.
In effect, the file still haven't changed and the file pointer is still at the beginning of the file. So the second time you run your code you see the content printed, which leaves the file pointer at the end of the file and only then the buffer is passed to the file and you have the duplicated content.
Simply use mode='w' if you just want to write to a file...
You start reading the file from where the writing stopped. It is better to open the file first for writing, then for reading
Something like this
with open('stavanger.txt', 'w') as f: # file closes itself with with open as filename command
words = ['hello\n', 'guys\n', 'how\n', 'are\n', 'you\n']
f.writelines(words)
with open('stavanger.txt', 'r') as f:
for i in f:
x = i.rstrip().split(',') # turn text file into list and we seperate list items by comma .
print(x)
I want to load/read a text file and write it to two other text files "entirely". I will write other different data to the following of these two files later.
The problem is that the loaded file is only written to the first file, and no data from that loaded file is written to the second file.
The code I am using:
fin = open("File_Read", 'r')
fout1 = open("File_Write1", 'w')
fout2 = open("File_Write2", 'w')
fout1.write(fin.read())
fout2.write(fin.read()) #Nothing is written here!
fin.close()
fout1.close()
fout2.close()
What is happening and what is the solution?
I prefer using open instead of with open.
Thanks.
Apparently the fin.read() reads all the lines, the next fin.read() will continue from where the previous .read() ended (which is the last line). To solve this, I would simply go for:
text_fin = fin.read()
fout1.write(text_fin)
fout2.write(text_fin)
fin = open("test.txt", 'r')
data = fin.read()
fin.close()
fout1 = open("test2.txt", 'w')
fout1.write(data)
fout1.close()
fout2 = open("test3.txt", 'w')
fout2.write(data)
fout2.close()
N.B. with open is the safest and best way but at least you need to close the file as soon as there are not needed anymore.
You can try iterating through your original file line by line and appending it to both the files. You are running into the problem because file.write() method takes string argument.
fin = open("File_Read",'r')
fout1 = open("File_Write1",'a') #append permissions for line-by-line writing
fout2 = open("File_Write2",'a') #append permissions for line-by-line writing
for lines in fin:
fout1.write(lines)
fout2.write(lines)
fin.close()
fout1.close()
fout2.close()
*** NOTE: Not the most efficient solution.
I want to append a new line in the starting of 2GB+ file. I tried following code but code OUT of MEMORY
error.
myfile = open(tableTempFile, "r+")
myfile.read() # read everything in the file
myfile.seek(0) # rewind
myfile.write("WRITE IN THE FIRST LINE ")
myfile.close();
What is the way to write in a file file without getting the entire file in memory?
How to append a new line at starting of the file?
Please note, there's no way to do this with any built-in functions in Python.
You can do this easily in LINUX using tail / cat etc.
For doing it via Python we must use an auxiliary file and for doing this with very large files, I think this method is the possibility:
def add_line_at_start(filename,line_to_be_added):
f = fileinput.input(filename,inplace=1)
for xline in f:
if f.isfirstline():
print line_to_be_added.rstrip('\r\n') + '\n' + xline,
else:
print xline
NOTE:
Never try to use read() / readlines() functions when you are dealing with big files. These methods tried load the complete file into your memory
In your given code, seek function is going to take you the starting point but then everything you write would overwrite the current content
If you can afford having the entire file in memory at once:
first_line_update = "WRITE IN THE FIRST LINE \n"
with open(tableTempFile, 'r+') as f:
lines = f.readlines()
lines[0] = first_line_update
f.writelines(lines)
otherwise:
from shutil import copy
from itertools import islice, chain
# TODO: use a NamedTemporaryFile from the tempfile module
first_line_update = "WRITE IN THE FIRST LINE \n"
with open("inputfile", 'r') as infile, open("tmpfile", 'w+') as outfile:
# replace the first line with the string provided:
outfile.writelines(
(line for line in chain((first_line_update,), islice(infile,1,None)))
# if you don't want to replace the first line but to insert another line before
# this simplifies to:
#outfile.writelines(line for line in chain((first_line_update,), infile))
copy("tmpfile", "infile")
# TODO: remove temporary file
Generally, you can't do that. A file is a sequence of bytes, not a sequence of lines. This data model doesn't allow for insertions in arbitrary points - you can either replace a byte by another or append bytes at the end.
You can either:
Replace the first X bytes in the file. This could work for you if you can make sure that the first line's length will never vary.
Truncate the file, write the first line, then rewrite all the rest after it. If you can't fit all your file into the memory, then:
create a temporary file (the tempfile module will help you)
write your line to it
open your base file in r and copy its contents after the first line to the temporary file, piece-wise
close both files, then replace the input file by the temporary file
(Note that appending to the end of a file is much easier - all you need to do is open the file in the append a mode.)
I can do this using a separate file, but how do I append a line to the beginning of a file?
f=open('log.txt','a')
f.seek(0) #get to the first position
f.write("text")
f.close()
This starts writing from the end of the file since the file is opened in append mode.
In modes 'a' or 'a+', any writing is done at the end of the file, even if at the current moment when the write() function is triggered the file's pointer is not at the end of the file: the pointer is moved to the end of file before any writing. You can do what you want in two manners.
1st way, can be used if there are no issues to load the file into memory:
def line_prepender(filename, line):
with open(filename, 'r+') as f:
content = f.read()
f.seek(0, 0)
f.write(line.rstrip('\r\n') + '\n' + content)
2nd way:
def line_pre_adder(filename, line_to_prepend):
f = fileinput.input(filename, inplace=1)
for xline in f:
if f.isfirstline():
print line_to_prepend.rstrip('\r\n') + '\n' + xline,
else:
print xline,
I don't know how this method works under the hood and if it can be employed on big big file. The argument 1 passed to input is what allows to rewrite a line in place; the following lines must be moved forwards or backwards in order that the inplace operation takes place, but I don't know the mechanism
In all filesystems that I am familiar with, you can't do this in-place. You have to use an auxiliary file (which you can then rename to take the name of the original file).
To put code to NPE's answer, I think the most efficient way to do this is:
def insert(originalfile,string):
with open(originalfile,'r') as f:
with open('newfile.txt','w') as f2:
f2.write(string)
f2.write(f.read())
os.remove(originalfile)
os.rename('newfile.txt',originalfile)
Different Idea:
(1) You save the original file as a variable.
(2) You overwrite the original file with new information.
(3) You append the original file in the data below the new information.
Code:
with open(<filename>,'r') as contents:
save = contents.read()
with open(<filename>,'w') as contents:
contents.write(< New Information >)
with open(<filename>,'a') as contents:
contents.write(save)
The clear way to do this is as follows if you do not mind writing the file again
with open("a.txt", 'r+') as fp:
lines = fp.readlines() # lines is list of line, each element '...\n'
lines.insert(0, one_line) # you can use any index if you know the line index
fp.seek(0) # file pointer locates at the beginning to write the whole file again
fp.writelines(lines) # write whole lists again to the same file
Note that this is not in-place replacement. It's writing a file again.
In summary, you read a file and save it to a list and modify the list and write the list again to a new file with the same filename.
num = [1, 2, 3] #List containing Integers
with open("ex3.txt", 'r+') as file:
readcontent = file.read() # store the read value of exe.txt into
# readcontent
file.seek(0, 0) #Takes the cursor to top line
for i in num: # writing content of list One by One.
file.write(str(i) + "\n") #convert int to str since write() deals
# with str
file.write(readcontent) #after content of string are written, I return
#back content that were in the file
There's no way to do this with any built-in functions, because it would be terribly inefficient. You'd need to shift the existing contents of the file down each time you add a line at the front.
There's a Unix/Linux utility tail which can read from the end of a file. Perhaps you can find that useful in your application.
If the file is the too big to use as a list, and you simply want to reverse the file, you can initially write the file in reversed order and then read one line at the time from the file's end (and write it to another file) with file-read-backwards module
An improvement over the existing solution provided by #eyquem is as below:
def prepend_text(filename: Union[str, Path], text: str):
with fileinput.input(filename, inplace=True) as file:
for line in file:
if file.isfirstline():
print(text)
print(line, end="")
It is typed, neat, more readable, and uses some improvements python got in recent years like context managers :)
I tried a different approach:
I wrote first line into a header.csv file. body.csv was the second file. Used Windows type command to concatenate them one by one into final.csv.
import os
os.system('type c:\\\header.csv c:\\\body.csv > c:\\\final.csv')
with open("fruits.txt", "r+") as file:
file.write("bab111y")
file.seek(0)
content = file.read()
print(content)
The last line of my file is:
29-dez,40,
How can I modify that line so that it reads:
29-Dez,40,90,100,50
Note: I don't want to write a new line. I want to take the same line and put new values after 29-Dez,40,
I'm new at python. I'm having a lot of trouble manipulating files and for me every example I look at seems difficult.
Unless the file is huge, you'll probably find it easier to read the entire file into a data structure (which might just be a list of lines), and then modify the data structure in memory, and finally write it back to the file.
On the other hand maybe your file is really huge - multiple GBs at least. In which case: the last line is probably terminated with a new line character, if you seek to that position you can overwrite it with the new text at the end of the last line.
So perhaps:
f = open("foo.file", "wb")
f.seek(-len(os.linesep), os.SEEK_END)
f.write("new text at end of last line" + os.linesep)
f.close()
(Modulo line endings on different platforms)
To expand on what Doug said, in order to read the file contents into a data structure you can use the readlines() method of the file object.
The below code sample reads the file into a list of "lines", edits the last line, then writes it back out to the file:
#!/usr/bin/python
MYFILE="file.txt"
# read the file into a list of lines
lines = open(MYFILE, 'r').readlines()
# now edit the last line of the list of lines
new_last_line = (lines[-1].rstrip() + ",90,100,50")
lines[-1] = new_last_line
# now write the modified list back out to the file
open(MYFILE, 'w').writelines(lines)
If the file is very large then this approach will not work well, because this reads all the file lines into memory each time and writes them back out to the file, which is very inefficient. For a small file however this will work fine.
Don't work with files directly, make a data structure that fits your needs in form of a class and make read from/write to file methods.
I recently wrote a script to do something very similar to this. It would traverse a project, find all module dependencies and add any missing import statements. I won't clutter this post up with the entire script, but I'll show how I went about modifying my files.
import os
from mmap import mmap
def insert_import(filename, text):
if len(text) < 1:
return
f = open(filename, 'r+')
m = mmap(f.fileno(), os.path.getsize(filename))
origSize = m.size()
m.resize(origSize + len(text))
pos = 0
while True:
l = m.readline()
if l.startswith(('import', 'from')):
continue
else:
pos = m.tell() - len(l)
break
m[pos+len(text):] = m[pos:origSize]
m[pos:pos+len(text)] = text
m.close()
f.close()
Summary: This snippet takes a filename and a blob of text to insert. It finds the last import statement already present, and sticks the text in at that location.
The part I suggest paying most attention to is the use of mmap. It lets you work with files in the same manner you may work with a string. Very handy.