I have a script that reads a file and then completes tests based on that file however I am running into a problem because the file reloads after one hour and I cannot get the script to re-read the file after or at that point in time.
So:
GETS NEW FILE TO READ
Reads file
performs tests on file
GET NEW FILE TO READ (with same name - but that can change if it is part of a solution)
Reads new file
perform same tests on new file
Can anyone suggest a way to get Python to re-read the file?
Either seek to the beginning of the file
with open(...) as fin:
fin.read() # read first time
fin.seek(0) # offset of 0
fin.read() # read again
or open the file again (I'd prefer this way since you are otherwise keeping the file open for an hour doing nothing between passes)
with open(...) as fin:
fin.read() # read first time
with open(...) as fin:
fin.read() # read again
Putting this together
while True:
with open(...) as fin:
for line in fin:
# do something
time.sleep(3600)
You can move the cursor to the beginning of the file the following way:
file.seek(0)
Then you can successfully read it.
Related
I found this solution from stackoverflow but will i have to add this piece of code every time i write a code to perform read/write operations.
Or is there any long term solution available for this ?
import os
path = "E:\Python\learn"
os.chdir(path)
f = open('text.txt')
data = f.read()
print(data)
f.close()
I think you need a way to read same file multiple times in your python code. Use file.seek() to jump to a specific position in a file. However, think about whether it is really necessary to go through the file again. An Example for file.seek:
with open(name, 'r+') as file:
for line in file:
# Do Something with line
file.seek(0)
for line in file:
# Do Something with line, second time
Incase you want to re-iterate the file second time and don't want to open the file again, you can follow this syntax:
with open(name, 'r+') as file:
for line in file:
# Do Something with line
for line in file:
# Do Something with line, second time
I want to write a file that says hello guys how are you but each word must be an item of list. Here is my code. It shows nothing when I run it, when I run second time it shows item by item as I want. But when I click text file, it is written two times.
with open('stavanger.txt','r+') as f: # file closes itself with with open as filename command
words = ['hello\n','guys\n','how\n', 'are\n','you\n']
f.writelines(words)
for i in f:
x=i.rstrip().split(',')#turn text file into list and we seperate list items by comma .
print(x)
The problem is that writing to a file uses a buffer. So after the line f.writelines(words) nothing really happened. Only the buffer changed.
In effect, the file still haven't changed and the file pointer is still at the beginning of the file. So the second time you run your code you see the content printed, which leaves the file pointer at the end of the file and only then the buffer is passed to the file and you have the duplicated content.
Simply use mode='w' if you just want to write to a file...
You start reading the file from where the writing stopped. It is better to open the file first for writing, then for reading
Something like this
with open('stavanger.txt', 'w') as f: # file closes itself with with open as filename command
words = ['hello\n', 'guys\n', 'how\n', 'are\n', 'you\n']
f.writelines(words)
with open('stavanger.txt', 'r') as f:
for i in f:
x = i.rstrip().split(',') # turn text file into list and we seperate list items by comma .
print(x)
This question already has answers here:
Why can't I call read() twice on an open file?
(7 answers)
Closed 7 months ago.
I have a problem with iterating on a file. Here's what I type on the interpreter and the result:
>>> f = open('baby1990.html', 'rU')
>>> for line in f.readlines():
... print(line)
...
# ... all the lines from the file appear here ...
When I try to iterate on the same open file again I get nothing!
>>> for line in f.readlines():
... print(line)
...
>>>
There is no output at all. To solve this I have to close() the file then open it again for reading! Is that normal behavior?
Yes, that is normal behavior. You basically read to the end of the file the first time (you can sort of picture it as reading a tape), so you can't read any more from it unless you reset it, by either using f.seek(0) to reposition to the start of the file, or to close it and then open it again which will start from the beginning of the file.
If you prefer you can use the with syntax instead which will automatically close the file for you.
e.g.,
with open('baby1990.html', 'rU') as f:
for line in f:
print line
once this block is finished executing, the file is automatically closed for you, so you could execute this block repeatedly without explicitly closing the file yourself and read the file this way over again.
As the file object reads the file, it uses a pointer to keep track of where it is. If you read part of the file, then go back to it later it will pick up where you left off. If you read the whole file, and go back to the same file object, it will be like reading an empty file because the pointer is at the end of the file and there is nothing left to read. You can use file.tell() to see where in the file the pointer is and file.seek to set the pointer. For example:
>>> file = open('myfile.txt')
>>> file.tell()
0
>>> file.readline()
'one\n'
>>> file.tell()
4L
>>> file.readline()
'2\n'
>>> file.tell()
6L
>>> file.seek(4)
>>> file.readline()
'2\n'
Also, you should know that file.readlines() reads the whole file and stores it as a list. That's useful to know because you can replace:
for line in file.readlines():
#do stuff
file.seek(0)
for line in file.readlines():
#do more stuff
with:
lines = file.readlines()
for each_line in lines:
#do stuff
for each_line in lines:
#do more stuff
You can also iterate over a file, one line at a time, without holding the whole file in memory (this can be very useful for very large files) by doing:
for line in file:
#do stuff
The file object is a buffer. When you read from the buffer, that portion that you read is consumed (the read position is shifted forward). When you read through the entire file, the read position is at the end of the file (EOF), so it returns nothing because there is nothing left to read.
If you have to reset the read position on a file object for some reason, you can do:
f.seek(0)
Of course.
That is normal and sane behaviour.
Instead of closing and re-opening, you could rewind the file.
I have this code
with codecs.open("file.json", mode='a+', encoding='utf-8') as f:
I want:
1) Create file if it does not exists, and start writing from the start of file.
2) If exists, first read it and truncate it and then write something.
I found this somewhere
``r'' Open text file for reading. The stream is positioned at the
beginning of the file.
``r+'' Open for reading and writing. The stream is positioned at the
beginning of the file.
``w'' Truncate file to zero length or create text file for writing.
The stream is positioned at the beginning of the file.
``w+'' Open for reading and writing. The file is created if it does not
exist, otherwise it is truncated. The stream is positioned at
the beginning of the file.
``a'' Open for writing. The file is created if it does not exist. The
stream is positioned at the end of the file. Subsequent writes
to the file will always end up at the then current end of file,
irrespective of any intervening fseek(3) or similar.
``a+'' Open for reading and writing. The file is created if it does not
exist. The stream is positioned at the end of the file. Subse-
quent writes to the file will always end up at the then current
end of file, irrespective of any intervening fseek(3) or similar.
a+ mode suits me best but what it does that it only lets me write at end of file,
With a+ mode I have this f.seek(0) immediately after opening file, but it has no affect, it does not seek to the start of file.
Let's say you have a file a with content:
first line
second line
third line
If you need to write from the start of the file, just do:
with open('a','r+') as f:
f.write("forth line")
Output:
forth line
second line
third line
If you need to remove the current content and write from the start, do:
with open('a','r+') as f:
f.write("forth line")
f.truncate()
Output:
forth line
If you need to append after the existing file, do:
with open('a','a') as f:
f.write("forth line")
Output:
first line
second line
third line
forth line
And, as you suspected, you will not be able to seek to 0 in a+ mode. You might see details from here
Edit:
Yes, you can dump json with this configuration and still indent. Demo:
dic = {'a':1,"b":2}
import json
with open('a','r+') as f:
json.dump(dic,f, indent=2)
Output:
{
"a": 1,
"b": 2
}third line
Use os.path.isfile():
import os
if os.path.isfile(filename):
# do stuff
else:
# do other stuff
As to your second question about writing to the begging of a file, then don't use a+. See here for how to prepend to a file. I'll post the relevant bits here:
# credit goes to #eyquem. Not my code
def line_prepender(filename, line):
with open(filename, 'r+') as f:
content = f.read()
f.seek(0, 0)
f.write(line.rstrip('\r\n') + '\n' + content)
You can check if the file exists, and then branch accordingly like so:
import os.path
file_exists = os.path.isfile(filename)
if file_exists:
# do something
else:
# do something else
Hope this helps!
You can open the file using os.open to be able to seek and have more control, but you won't be able to use codecs.open or a context manager then, so it's a bit more manual labor:
import os
f = os.fdopen(os.open(filename, os.O_RDWR | os.O_CREAT), 'r+')
try:
content = f.read()
f.seek(0)
f.truncate()
f.write("Your new data")
finally:
f.close()
I want to be able to open a file, append some text to the end, and then read only the first line. I know exactly how long the first line of the file is, and the file is large enough that I don't want to read it into memory all at once. I've tried using:
with open('./output files/log.txt', 'a+') as f:
f.write('This is example text')
content = f.readline()
print(content)
but the print statement is blank. When I try using open('./output files/log.txt') or open('./output files/log.txt', 'r+') instead of open('./output files/log.txt', 'a+') this works so I know it has to do with the 'a+ argument. My problem is that I have to append to the file. How can I append to the file and still get the first line without using something like
with open('./output files/log.txt', 'a+') as f_1:
f.write('This is example text')
with open('./output files/log.txt') as f_2:
content = f_2.readline()
print(content)
When you open a file with the append flag a, it moves the file descriptor's pointer to the end of the file, so that the write call will add to the end of the file.
The readline() function reads from the current pointer of the file until the next '\n' character it reads. So when you open a file with append, and then call readline, it will try to read a line starting from the end of the file. This is why your print call is coming up blank.
You can see this in action by looking at where the file object is currently pointing, using the tell() function.
To read the first line, you'd have to make sure the file's pointer is back at the beginning of the file, which you can do using the seek function. seek takes two arguments: offset and from_what. If you omit the second argument, offset is taken from the beginning of the file. So to jump to the beginning of the file, do: seek(0).
If you want to jump back to the end of the file, you can include the from_what option. from_what=2 means take the offset from the end of the file. So to jump to the end: seek(0, 2).
Demonstration of file pointers when opened in append mode:
Example using a text file that looks like this:
the first line of the file
and the last line
Code:
with open('example.txt', 'a+') as fd:
print fd.tell() # at end of file
fd.write('example line\n')
print fd.tell() # at new end of the file after writing
# jump to the beginning of the file:
fd.seek(0)
print fd.readline()
# jump back to the end of the file
fd.seek(0, 2)
fd.write('went back to the end')
console output:
45
57
the first line of the file
new contents of example.txt:
the first line of the file
and the last line
example line
went back to the end
Edit: added jumping back to end of file
You need to go back to the start of the file using seek(0), like so:
with open('./output files/log.txt', 'a+') as f_1:
f_1.write('This is example text')
f_1.seek(0)
print(f_1.readline())