In regards to the related question, I'd like to know how to add text and/or lines to the beginning of a file in Python, as its been suggested that is an easier language for text/file manipulation. So, while I asked the previous, linked, question for C++, can anyone point me to how to do this with Python?
Quote from the linked (related) question:
I'd like to be able to add lines to the beginning of a file.
This program I am writing will take information from a user, and prep
it to write to a file. That file, then, will be a diff that was
already generated, and what is being added to the beginning is
descriptors and tags that make it compatible with Debian's DEP3 Patch
tagging system.
Anyone got any suggestions, or code?
Related: Adding text and lines to the beginning of a file (C++)
There are lot of similar file I/O questions recently..
In short, you need to create a new file
first, write new lines to the file
read lines from old file and write them to file
If you can guarantee each of your new lines added to the beginning is longer than the each of the corresponding original lines at the beginning, you can do it in place:
f = open('file.txt','r+')
lines = f.readlines() # read old content
f.seek(0) # go back to the beginning of the file
f.write(new_content) # write new content at the beginning
for line in lines: # write old content after new
f.write(line)
f.close()
The example above writes all the data in it's entirety after seeking it's position at beginning of file because the contents of file are over-written with new contents.
Otherwise you need to write to a new file
f = open('file.txt','r')
newf = open('newfile.txt','w')
lines = f.readlines() # read old content
newf.write(new_content) # write new content at the beginning
for line in lines: # write old content after new
newf.write(line)
newf.close()
f.close()
Something like this should work:
with open('new.txt', 'w') as new:
with open('prefix.txt') as prefix:
new.write(prefix.read())
with open('old.txt') as old:
new.write(old.read())
If old.txt or prefix.txt contain binary contents, you should add an 'rb' argument to their respective open calls and a 'b' to the flags argument for the first open() call.
Related
'r' will read a file, 'w' will write text in the file from the start, and 'a' will append. How can I open the file to read and append at the same time?
I tried these, but got errors:
open("filename", "r,a")
open("filename", "w")
open("filename", "r")
open("filename", "a")
error:
invalid mode: 'r,a'
You're looking for the r+/a+/w+ mode, which allows both read and write operations to files.
With r+, the position is initially at the beginning, but reading it once will push it towards the end, allowing you to append. With a+, the position is initially at the end.
with open("filename", "r+") as f:
# here, position is initially at the beginning
text = f.read()
# after reading, the position is pushed toward the end
f.write("stuff to append")
with open("filename", "a+") as f:
# here, position is already at the end
f.write("stuff to append")
If you ever need to do an entire reread, you could return to the starting position by doing f.seek(0).
with open("filename", "r+") as f:
text = f.read()
f.write("stuff to append")
f.seek(0) # return to the top of the file
text = f.read()
assert text.endswith("stuff to append")
(Further Reading: What's the difference between 'r+' and 'a+' when open file in python?)
You can also use w+, but this will truncate (delete) all the existing content.
Here's a nice little diagram from another SO post:
(source)
You can't do that with a textfile. Either you want to read it or you want to write to it. The a or the r specifies a seek to a particular location in the file. Specifying both is asking open to point to two different locations in the file at the same time.
Textfiles in general can't be updated in place. You can use a to add new stuff to the end but that is about it. To do what I think you want, you need to open the existing file in read mode, and open another, new file in write mode, and copy the data from the one to the other.
After that you have two files so you have to take care of deleting the old one. If that is troublesome, take a look at the module in-place.
The other alternative is to read the input file into memory, close and reopen it for writing, then write out a new version of the file. Then you don't have to delete the old copy. But if something goes wrong in the middle you will have no old input file, because you deleted it, and no new output file either, because you didn't successfully write it.
The reason for this is that textfiles are not designed for random access.
'r' will read a file, 'w' will write text in the file from the start, and 'a' will append. How can I open the file to read and append at the same time?
I tried these, but got errors:
open("filename", "r,a")
open("filename", "w")
open("filename", "r")
open("filename", "a")
error:
invalid mode: 'r,a'
You're looking for the r+/a+/w+ mode, which allows both read and write operations to files.
With r+, the position is initially at the beginning, but reading it once will push it towards the end, allowing you to append. With a+, the position is initially at the end.
with open("filename", "r+") as f:
# here, position is initially at the beginning
text = f.read()
# after reading, the position is pushed toward the end
f.write("stuff to append")
with open("filename", "a+") as f:
# here, position is already at the end
f.write("stuff to append")
If you ever need to do an entire reread, you could return to the starting position by doing f.seek(0).
with open("filename", "r+") as f:
text = f.read()
f.write("stuff to append")
f.seek(0) # return to the top of the file
text = f.read()
assert text.endswith("stuff to append")
(Further Reading: What's the difference between 'r+' and 'a+' when open file in python?)
You can also use w+, but this will truncate (delete) all the existing content.
Here's a nice little diagram from another SO post:
(source)
You can't do that with a textfile. Either you want to read it or you want to write to it. The a or the r specifies a seek to a particular location in the file. Specifying both is asking open to point to two different locations in the file at the same time.
Textfiles in general can't be updated in place. You can use a to add new stuff to the end but that is about it. To do what I think you want, you need to open the existing file in read mode, and open another, new file in write mode, and copy the data from the one to the other.
After that you have two files so you have to take care of deleting the old one. If that is troublesome, take a look at the module in-place.
The other alternative is to read the input file into memory, close and reopen it for writing, then write out a new version of the file. Then you don't have to delete the old copy. But if something goes wrong in the middle you will have no old input file, because you deleted it, and no new output file either, because you didn't successfully write it.
The reason for this is that textfiles are not designed for random access.
Similar to posting: Replace string in a specific line using python, however results were not forethcomming in my slightly different instance.
I working with python 3 on windows 7. I am attempting to batch edit some files in a directory. They are basically text files with .LIC tag. I'm not sure if that is relevant to my issue here. I am able to read the file into python without issue.
My aim is to replace a specific string on a specific line in this file.
import os
import re
groupname = 'Oldtext'
aliasname = 'Newtext'
with open('filename') as f:
data = f.readlines()
data[1] = re.sub(groupname,aliasname, data[1])
f.writelines(data[1])
print(data[1])
print('done')
When running the above code I get an UnsupportedOperation: not writable. I am having some issue writing the changes back to the file. Based on suggestion of other posts, I edited added the w option to the open('filename', "w") function. This causes all text in the file to be deleted.
Based on suggestion, the r+ option was tried. This leads to successful editing of the file, however, instead of editing the correct line, the edited line is appended to the end of the file, leaving the original intact.
Writing a changed line into the middle of a text file is not going to work unless it's exactly the same length as the original - which is the case in your example, but you've got some obvious placeholder text there so I have no idea if the same is true of your actual application code. Here's an approach that doesn't make any such assumption:
with open('filename', 'r') as f:
data = f.readlines()
data[1] = re.sub(groupname,aliasname, data[1])
with open('filename', 'w') as f:
f.writelines(data)
EDIT: If you really wanted to write only the single line back into the file, you'd need to use f.tell() BEFORE reading the line, to remember its position within the file, and then f.seek() to go back to that position before writing.
I have text file which I want to erase in Python. How do I do that?
In python:
open('file.txt', 'w').close()
Or alternatively, if you have already an opened file:
f = open('file.txt', 'r+')
f.truncate(0) # need '0' when using r+
Opening a file in "write" mode clears it, you don't specifically have to write to it:
open("filename", "w").close()
(you should close it as the timing of when the file gets closed automatically may be implementation specific)
Not a complete answer more of an extension to ondra's answer
When using truncate() ( my preferred method ) make sure your cursor is at the required position.
When a new file is opened for reading - open('FILE_NAME','r') it's cursor is at 0 by default.
But if you have parsed the file within your code, make sure to point at the beginning of the file again i.e truncate(0)
By default truncate() truncates the contents of a file starting from the current cusror position.
A simple example
As #jamylak suggested, a good alternative that includes the benefits of context managers is:
with open('filename.txt', 'w'):
pass
When using with open("myfile.txt", "r+") as my_file:, I get strange zeros in myfile.txt, especially since I am reading the file first. For it to work, I had to first change the pointer of my_file to the beginning of the file with my_file.seek(0). Then I could do my_file.truncate() to clear the file.
Writing and Reading file content
def writeTempFile(text = None):
filePath = "/temp/file1.txt"
if not text: # If not provided return file content
f = open(filePath, "r")
slug = f.read()
return slug
else:
f = open(filePath, "a") # Create a blank file
f.seek(0) # sets point at the beginning of the file
f.truncate() # Clear previous content
f.write(text) # Write file
f.close() # Close file
return text
It Worked for me
If security is important to you then opening the file for writing and closing it again will not be enough. At least some of the information will still be on the storage device and could be found, for example, by using a disc recovery utility.
Suppose, for example, the file you're erasing contains production passwords and needs to be deleted immediately after the present operation is complete.
Zero-filling the file once you've finished using it helps ensure the sensitive information is destroyed.
On a recent project we used the following code, which works well for small text files. It overwrites the existing contents with lines of zeros.
import os
def destroy_password_file(password_filename):
with open(password_filename) as password_file:
text = password_file.read()
lentext = len(text)
zero_fill_line_length = 40
zero_fill = ['0' * zero_fill_line_length
for _
in range(lentext // zero_fill_line_length + 1)]
zero_fill = os.linesep.join(zero_fill)
with open(password_filename, 'w') as password_file:
password_file.write(zero_fill)
Note that zero-filling will not guarantee your security. If you're really concerned, you'd be best to zero-fill and use a specialist utility like File Shredder or CCleaner to wipe clean the 'empty' space on your drive.
You have to overwrite the file. In C++:
#include <fstream>
std::ofstream("test.txt", std::ios::out).close();
You can also use this (based on a few of the above answers):
file = open('filename.txt', 'w')
file.close()
of course this is a really bad way to clear a file because it requires so many lines of code, but I just wrote this to show you that it can be done in this method too.
happy coding!
You cannot "erase" from a file in-place unless you need to erase the end. Either be content with an overwrite of an "empty" value, or read the parts of the file you care about and write it to another file.
Assigning the file pointer to null inside your program will just get rid of that reference to the file. The file's still there. I think the remove() function in the c stdio.h is what you're looking for there. Not sure about Python.
Since text files are sequential, you can't directly erase data on them. Your options are:
The most common way is to create a new file. Read from the original file and write everything on the new file, except the part you want to erase. When all the file has been written, delete the old file and rename the new file so it has the original name.
You can also truncate and rewrite the entire file from the point you want to change onwards. Seek to point you want to change, and read the rest of file to memory. Seek back to the same point, truncate the file, and write back the contents without the part you want to erase.
Another simple option is to overwrite the data with another data of same length. For that, seek to the exact position and write the new data. The limitation is that it must have exact same length.
Look at the seek/truncate function/method to implement any of the ideas above. Both Python and C have those functions.
This is my method:
open the file using r+ mode
read current data from the file using file.read()
move the pointer to the first line using file.seek(0)
remove old data from the file using file.truncate(0)
write new content and then content that we saved using file.read()
So full code will look like this:
with open(file_name, 'r+') as file:
old_data = file.read()
file.seek(0)
file.truncate(0)
file.write('my new content\n')
file.write(old_data)
Because we are using with open, file will automatically close.
The last line of my file is:
29-dez,40,
How can I modify that line so that it reads:
29-Dez,40,90,100,50
Note: I don't want to write a new line. I want to take the same line and put new values after 29-Dez,40,
I'm new at python. I'm having a lot of trouble manipulating files and for me every example I look at seems difficult.
Unless the file is huge, you'll probably find it easier to read the entire file into a data structure (which might just be a list of lines), and then modify the data structure in memory, and finally write it back to the file.
On the other hand maybe your file is really huge - multiple GBs at least. In which case: the last line is probably terminated with a new line character, if you seek to that position you can overwrite it with the new text at the end of the last line.
So perhaps:
f = open("foo.file", "wb")
f.seek(-len(os.linesep), os.SEEK_END)
f.write("new text at end of last line" + os.linesep)
f.close()
(Modulo line endings on different platforms)
To expand on what Doug said, in order to read the file contents into a data structure you can use the readlines() method of the file object.
The below code sample reads the file into a list of "lines", edits the last line, then writes it back out to the file:
#!/usr/bin/python
MYFILE="file.txt"
# read the file into a list of lines
lines = open(MYFILE, 'r').readlines()
# now edit the last line of the list of lines
new_last_line = (lines[-1].rstrip() + ",90,100,50")
lines[-1] = new_last_line
# now write the modified list back out to the file
open(MYFILE, 'w').writelines(lines)
If the file is very large then this approach will not work well, because this reads all the file lines into memory each time and writes them back out to the file, which is very inefficient. For a small file however this will work fine.
Don't work with files directly, make a data structure that fits your needs in form of a class and make read from/write to file methods.
I recently wrote a script to do something very similar to this. It would traverse a project, find all module dependencies and add any missing import statements. I won't clutter this post up with the entire script, but I'll show how I went about modifying my files.
import os
from mmap import mmap
def insert_import(filename, text):
if len(text) < 1:
return
f = open(filename, 'r+')
m = mmap(f.fileno(), os.path.getsize(filename))
origSize = m.size()
m.resize(origSize + len(text))
pos = 0
while True:
l = m.readline()
if l.startswith(('import', 'from')):
continue
else:
pos = m.tell() - len(l)
break
m[pos+len(text):] = m[pos:origSize]
m[pos:pos+len(text)] = text
m.close()
f.close()
Summary: This snippet takes a filename and a blob of text to insert. It finds the last import statement already present, and sticks the text in at that location.
The part I suggest paying most attention to is the use of mmap. It lets you work with files in the same manner you may work with a string. Very handy.