read numbers from text file python - python

I am trying to read a text file and return the contents of the text file. The textfile contains a matrix. When i run my code with the file it just prints the first line. My code looks right and i have searched online and cant seem to find the problem.
Code is:
def main():
matrix = "matrix1.txt"
print(readMatrix(matrix))
def readMatrix(matrix):
matrixFile = open(matrix, "r")
line = matrixFile.readline()
while line != "":
return line
line = matrixFile.readline()
matrixFile.close()
main()

while line != "":
return line # function ends
Maybe you mean
while line != "":
print line

return returns the value you pass it back to the caller and ends the function call. If you want to print each line, put the print statement instead of return.

You're misusing the return statement. When a function hits a return, control returns to the caller and does not return to the function. Thus, the most your function will do is read one line and return it, or close the file if the first line is empty.
Files in Python have a built-in iterator that will give you every line in the file, used like so:
with open(path) as f:
for line in f:
[do something]
Note the use of the with statement. It will automatically close the file when its block is exited, which makes it the preferred way to deal with reading/writing files.
So what you want to do could be something like
with open(path) as f:
for line in f:
if not line: # Equivalent to if line == ''
return
else: # This else is actually redundant, but here so the flow is clear
[do something]

Related

Check for text file update results in printing output twice

I have a constantly updating text file foo.txt. I want to read the last line of foo.txt only if the file updates. I have a while loop constantly opening the file and checking it. Inside the loop, I have some code that prints the last line of the file, then stores it in lastmsg.txt. On the next iteration, it checks that the last line of the file is not equal to whatever is stored in lastmsg.txt. If the two values are not equal (the file has updated and a new line has been added), it will print the last message in the file. Here is the code
import time
while True:
fileHandle = open ("foo.txt","r" )
lineList = fileHandle.readlines()
fileHandle.close()
msg = lineList[len(lineList)-1]
if(open("lastmsg.txt", "r").read() != msg):
f = open("lastmsg.txt", "w")
f.write(msg)
print(msg)
time.sleep(0.5)
This seems to work, however, it prints msg twice. So if abc is amended to the file, the output will be
abc
abc
I added f.close() line and after that, it prints msg just once.
import time
while True:
fileHandle = open ("foo.txt","r" )
lineList = fileHandle.readlines()
fileHandle.close()
msg = lineList[-1]
if(open("lastmsg.txt", "r").read() != msg):
f = open("lastmsg.txt", "w")
f.write(msg)
f.close() # <---- new line
print(msg)
time.sleep(0.5)
The below solution does make an assumption that you were only writing to the last line file as a buffer to know if you read the line before. Instead this solution just opens the file and reads to the end. Then will print any new lines that are written to the file after we read to the end.
When the file is updated the next call to readline() will retrieve the next line. this saves you reading the entire file every time.
import time
with open('query4.txt') as myfile:
#run to the end of the file we are discarding these as
#only want to tail the last line of the file when its updated
myfile.readlines()
while True:
line = myfile.readline()
if line:
print(line, end='')
time.sleep(0.5)

Python readline retrieve stdout issues

i have a text and i print each line of it.
i want to stop (for a first time just print "flag!" and in a second time to stop) the text every time the readed line is the flag
but it dont stop
code part:
import sys
path = "/somepath/story_line"
flag = "010001"
def process(line):
sys.stdout.write(line)
sys.stdout.flush()
time.sleep(2.4)
line = fileIN.readline()
with open(path, "r") as content:
if line != flag:
for line in content:
process(line)
if line == path:
print ("flag")
text part
[START]
010001
welcome traveler,
This story begins in a dark era where Evil took over the weak... Many times ago a dark force came from beyond the sky and over*** the balance of this land.
You are the hope and new strengh of this world, please choose wisely
....
....
Let the story begin
[END]
010001
GAME OVER !!!
im new to python and i tried with subprocess or to append every line into a list an parse the list but nothing do.
can someone maybe lighten this up?
I'm not sure I understood the question and the code, it looks like you are attempting to read lines from the file multiple times?
Using the "open" function you can freely iterate over the result, ie read line by line.
Here's how I'd do what you describe
import sys
import time
path = "/path/to/my/file"
flag = "010001"
def process(line):
if line.strip() == flag:
process.flagcount += 1
if process.flagcount == 1:
sys.stdout.write("flag!\n")
return process.flagcount
sys.stdout.write(line)
time.sleep(2.4)
return 0
process.flagcount = 0 #initialise a static counting attribute inside process()
#open the file
with open(path, "r") as content:
#read each line
for line in content:
#process it and if it tells us it's counted more than one flag then stop reading.
if process(line) > 1:
break
Your issue is with the way python passes variables. At the end of your function process(line), you do
line = fileIN.readline()
However, this only modifies the line within the current scope. Once you exit the function, that change is lost.
The solution is, instead of assigning to line at the end of the function, simply do
return fileIN.readline()
and, below, replace the line
process(line)
with
line = process(line)

using readline() in python to read a txt file but the second line is not read in aws comprehend api

I am reading a text file and passing it to the API, but then I am getting the result only for the first line in the file, the subsequent lines are not being read.
code below :
filename = 'c:\myfile.txt'
with open(filename) as f:
plain_text = f.readline()
response = client_comprehend.detect_entities(
Text=plain_text,
LanguageCode='en'
)
entites = list(set([x['Type'] for x in response['Entities']]))
print response
print entites
When you are doing with f.readline() it will only take the first line of the file. So if you want to go through each line of the file you have to loop through it. Otherwise if you want to read the entire file(not meant for big files) you can use f.read()
filename = 'c:\myfile.txt'
with open(filename) as f:
for plain_text in f:
response = client_comprehend.detect_entities(
Text=plain_text,
LanguageCode='en'
)
entites = list(set([x['Type'] for x in response['Entities']]))
print response
print entites
As csblo has pointed out in the comments, your readline is only reading the first line of the file because it's only being called once. readline is called once in your program as it is written, it performs the actions for the single line that has been read, and then the program closes without doing anything else.
Conveniently, file objects can be iterated over in a for loop like you would a list. Iterating over a file will return one line per iteration, as though you had called readline and assigned it to a value. Using this, your code will work when rewritten as such:
filename = 'c:\myfile.txt'
with open(filename) as f:
for plain_text_line in f:
response = client_comprehend.detect_entities(
Text=plain_text_line,
LanguageCode='en'
)
entites = list(set([x['Type'] for x in response['Entities']]))
print response
print entites
This should iterate over all lines of the file in turn.

How to edit a file in python without a temporary sacrificial file? [duplicate]

I'm using Python, and would like to insert a string into a text file without deleting or copying the file. How can I do that?
Unfortunately there is no way to insert into the middle of a file without re-writing it. As previous posters have indicated, you can append to a file or overwrite part of it using seek but if you want to add stuff at the beginning or the middle, you'll have to rewrite it.
This is an operating system thing, not a Python thing. It is the same in all languages.
What I usually do is read from the file, make the modifications and write it out to a new file called myfile.txt.tmp or something like that. This is better than reading the whole file into memory because the file may be too large for that. Once the temporary file is completed, I rename it the same as the original file.
This is a good, safe way to do it because if the file write crashes or aborts for any reason, you still have your untouched original file.
Depends on what you want to do. To append you can open it with "a":
with open("foo.txt", "a") as f:
f.write("new line\n")
If you want to preprend something you have to read from the file first:
with open("foo.txt", "r+") as f:
old = f.read() # read everything in the file
f.seek(0) # rewind
f.write("new line\n" + old) # write the new line before
The fileinput module of the Python standard library will rewrite a file inplace if you use the inplace=1 parameter:
import sys
import fileinput
# replace all occurrences of 'sit' with 'SIT' and insert a line after the 5th
for i, line in enumerate(fileinput.input('lorem_ipsum.txt', inplace=1)):
sys.stdout.write(line.replace('sit', 'SIT')) # replace 'sit' and write
if i == 4: sys.stdout.write('\n') # write a blank line after the 5th line
Rewriting a file in place is often done by saving the old copy with a modified name. Unix folks add a ~ to mark the old one. Windows folks do all kinds of things -- add .bak or .old -- or rename the file entirely or put the ~ on the front of the name.
import shutil
shutil.move(afile, afile + "~")
destination= open(aFile, "w")
source= open(aFile + "~", "r")
for line in source:
destination.write(line)
if <some condition>:
destination.write(<some additional line> + "\n")
source.close()
destination.close()
Instead of shutil, you can use the following.
import os
os.rename(aFile, aFile + "~")
Python's mmap module will allow you to insert into a file. The following sample shows how it can be done in Unix (Windows mmap may be different). Note that this does not handle all error conditions and you might corrupt or lose the original file. Also, this won't handle unicode strings.
import os
from mmap import mmap
def insert(filename, str, pos):
if len(str) < 1:
# nothing to insert
return
f = open(filename, 'r+')
m = mmap(f.fileno(), os.path.getsize(filename))
origSize = m.size()
# or this could be an error
if pos > origSize:
pos = origSize
elif pos < 0:
pos = 0
m.resize(origSize + len(str))
m[pos+len(str):] = m[pos:origSize]
m[pos:pos+len(str)] = str
m.close()
f.close()
It is also possible to do this without mmap with files opened in 'r+' mode, but it is less convenient and less efficient as you'd have to read and temporarily store the contents of the file from the insertion position to EOF - which might be huge.
As mentioned by Adam you have to take your system limitations into consideration before you can decide on approach whether you have enough memory to read it all into memory replace parts of it and re-write it.
If you're dealing with a small file or have no memory issues this might help:
Option 1)
Read entire file into memory, do a regex substitution on the entire or part of the line and replace it with that line plus the extra line. You will need to make sure that the 'middle line' is unique in the file or if you have timestamps on each line this should be pretty reliable.
# open file with r+b (allow write and binary mode)
f = open("file.log", 'r+b')
# read entire content of file into memory
f_content = f.read()
# basically match middle line and replace it with itself and the extra line
f_content = re.sub(r'(middle line)', r'\1\nnew line', f_content)
# return pointer to top of file so we can re-write the content with replaced string
f.seek(0)
# clear file content
f.truncate()
# re-write the content with the updated content
f.write(f_content)
# close file
f.close()
Option 2)
Figure out middle line, and replace it with that line plus the extra line.
# open file with r+b (allow write and binary mode)
f = open("file.log" , 'r+b')
# get array of lines
f_content = f.readlines()
# get middle line
middle_line = len(f_content)/2
# overwrite middle line
f_content[middle_line] += "\nnew line"
# return pointer to top of file so we can re-write the content with replaced string
f.seek(0)
# clear file content
f.truncate()
# re-write the content with the updated content
f.write(''.join(f_content))
# close file
f.close()
Wrote a small class for doing this cleanly.
import tempfile
class FileModifierError(Exception):
pass
class FileModifier(object):
def __init__(self, fname):
self.__write_dict = {}
self.__filename = fname
self.__tempfile = tempfile.TemporaryFile()
with open(fname, 'rb') as fp:
for line in fp:
self.__tempfile.write(line)
self.__tempfile.seek(0)
def write(self, s, line_number = 'END'):
if line_number != 'END' and not isinstance(line_number, (int, float)):
raise FileModifierError("Line number %s is not a valid number" % line_number)
try:
self.__write_dict[line_number].append(s)
except KeyError:
self.__write_dict[line_number] = [s]
def writeline(self, s, line_number = 'END'):
self.write('%s\n' % s, line_number)
def writelines(self, s, line_number = 'END'):
for ln in s:
self.writeline(s, line_number)
def __popline(self, index, fp):
try:
ilines = self.__write_dict.pop(index)
for line in ilines:
fp.write(line)
except KeyError:
pass
def close(self):
self.__exit__(None, None, None)
def __enter__(self):
return self
def __exit__(self, type, value, traceback):
with open(self.__filename,'w') as fp:
for index, line in enumerate(self.__tempfile.readlines()):
self.__popline(index, fp)
fp.write(line)
for index in sorted(self.__write_dict):
for line in self.__write_dict[index]:
fp.write(line)
self.__tempfile.close()
Then you can use it this way:
with FileModifier(filename) as fp:
fp.writeline("String 1", 0)
fp.writeline("String 2", 20)
fp.writeline("String 3") # To write at the end of the file
If you know some unix you could try the following:
Notes: $ means the command prompt
Say you have a file my_data.txt with content as such:
$ cat my_data.txt
This is a data file
with all of my data in it.
Then using the os module you can use the usual sed commands
import os
# Identifiers used are:
my_data_file = "my_data.txt"
command = "sed -i 's/all/none/' my_data.txt"
# Execute the command
os.system(command)
If you aren't aware of sed, check it out, it is extremely useful.

Remove comment lines from a file

I'm making a file type to store information from my program. The file type can include lines starting with #, like:
# This is a comment.
As shown, the # in front of a line denotes a comment.
I've written a program in Python that can read these files:
fileData = []
file = open("Tutorial.rdsf", "r")
line = file.readline()
while line != "":
fileData.append(line)
line = file.readline()
for item in list(fileData):
item.strip()
fileData = list(map(lambda s: s.strip(), fileData))
print(fileData)
As you can see, it takes the file, adds every line as an item in a list, and strips the items of \n. So far, so good.
But often these files contain comments I've made, and such the program adds them to the list.
Is there a way to delete all items in the list starting with #?
Edit: To make things a bit clearer: Comments won't be like this:
Some code:
{Some Code} #Foo
They'll be like this:
#Foo
Some code:
{Some Code}
You can process lines directly in a for loop:
with open("Tutorial.rdsf", "r") as file:
for line in file:
if line.startswith('#'):
continue # skip comments
line = line.strip()
# do more things with this line
Only put them into a list if you need random access (e.g. you need to access lines at specific indices).
I used a with statement to manage the open file, when Python reaches the end of the with block the file is automatically closed for you.
It's easy to check for leading # signs.
Change this:
while line != "":
fileData.append(line)
line = file.readline()
to this:
while line != "":
if not line.startswith("#"):
fileData.append(line)
line = file.readline()
But your program is a bit complicated for what it does. Look in the documentation where it explains about for line in file:.

Categories