i have a text and i print each line of it.
i want to stop (for a first time just print "flag!" and in a second time to stop) the text every time the readed line is the flag
but it dont stop
code part:
import sys
path = "/somepath/story_line"
flag = "010001"
def process(line):
sys.stdout.write(line)
sys.stdout.flush()
time.sleep(2.4)
line = fileIN.readline()
with open(path, "r") as content:
if line != flag:
for line in content:
process(line)
if line == path:
print ("flag")
text part
[START]
010001
welcome traveler,
This story begins in a dark era where Evil took over the weak... Many times ago a dark force came from beyond the sky and over*** the balance of this land.
You are the hope and new strengh of this world, please choose wisely
....
....
Let the story begin
[END]
010001
GAME OVER !!!
im new to python and i tried with subprocess or to append every line into a list an parse the list but nothing do.
can someone maybe lighten this up?
I'm not sure I understood the question and the code, it looks like you are attempting to read lines from the file multiple times?
Using the "open" function you can freely iterate over the result, ie read line by line.
Here's how I'd do what you describe
import sys
import time
path = "/path/to/my/file"
flag = "010001"
def process(line):
if line.strip() == flag:
process.flagcount += 1
if process.flagcount == 1:
sys.stdout.write("flag!\n")
return process.flagcount
sys.stdout.write(line)
time.sleep(2.4)
return 0
process.flagcount = 0 #initialise a static counting attribute inside process()
#open the file
with open(path, "r") as content:
#read each line
for line in content:
#process it and if it tells us it's counted more than one flag then stop reading.
if process(line) > 1:
break
Your issue is with the way python passes variables. At the end of your function process(line), you do
line = fileIN.readline()
However, this only modifies the line within the current scope. Once you exit the function, that change is lost.
The solution is, instead of assigning to line at the end of the function, simply do
return fileIN.readline()
and, below, replace the line
process(line)
with
line = process(line)
Related
I'm making a file type to store information from my program. The file type can include lines starting with #, like:
# This is a comment.
As shown, the # in front of a line denotes a comment.
I've written a program in Python that can read these files:
fileData = []
file = open("Tutorial.rdsf", "r")
line = file.readline()
while line != "":
fileData.append(line)
line = file.readline()
for item in list(fileData):
item.strip()
fileData = list(map(lambda s: s.strip(), fileData))
print(fileData)
As you can see, it takes the file, adds every line as an item in a list, and strips the items of \n. So far, so good.
But often these files contain comments I've made, and such the program adds them to the list.
Is there a way to delete all items in the list starting with #?
Edit: To make things a bit clearer: Comments won't be like this:
Some code:
{Some Code} #Foo
They'll be like this:
#Foo
Some code:
{Some Code}
You can process lines directly in a for loop:
with open("Tutorial.rdsf", "r") as file:
for line in file:
if line.startswith('#'):
continue # skip comments
line = line.strip()
# do more things with this line
Only put them into a list if you need random access (e.g. you need to access lines at specific indices).
I used a with statement to manage the open file, when Python reaches the end of the with block the file is automatically closed for you.
It's easy to check for leading # signs.
Change this:
while line != "":
fileData.append(line)
line = file.readline()
to this:
while line != "":
if not line.startswith("#"):
fileData.append(line)
line = file.readline()
But your program is a bit complicated for what it does. Look in the documentation where it explains about for line in file:.
I am trying to read a text file and return the contents of the text file. The textfile contains a matrix. When i run my code with the file it just prints the first line. My code looks right and i have searched online and cant seem to find the problem.
Code is:
def main():
matrix = "matrix1.txt"
print(readMatrix(matrix))
def readMatrix(matrix):
matrixFile = open(matrix, "r")
line = matrixFile.readline()
while line != "":
return line
line = matrixFile.readline()
matrixFile.close()
main()
while line != "":
return line # function ends
Maybe you mean
while line != "":
print line
return returns the value you pass it back to the caller and ends the function call. If you want to print each line, put the print statement instead of return.
You're misusing the return statement. When a function hits a return, control returns to the caller and does not return to the function. Thus, the most your function will do is read one line and return it, or close the file if the first line is empty.
Files in Python have a built-in iterator that will give you every line in the file, used like so:
with open(path) as f:
for line in f:
[do something]
Note the use of the with statement. It will automatically close the file when its block is exited, which makes it the preferred way to deal with reading/writing files.
So what you want to do could be something like
with open(path) as f:
for line in f:
if not line: # Equivalent to if line == ''
return
else: # This else is actually redundant, but here so the flow is clear
[do something]
I have a simple python script, where I read logfile continuosly (same as tail -f)
while True:
line = f.readline()
if line:
print line,
else:
time.sleep(0.1)
How can I make sure that I can still read the logfile, after it has been rotated by logrotate?
i.e. I need to do the same what tail -F would do.
I am using python 2.7
As long as you only plan to do this on Unix, the most robust way is probably to check so that the open file still refers to the same i-node as the name, and reopen it when that is no longer the case. You can get the i-number of the file from os.stat and os.fstat, in the st_ino field.
It could look like this:
import os, sys, time
name = "logfile"
current = open(name, "r")
curino = os.fstat(current.fileno()).st_ino
while True:
while True:
buf = current.read(1024)
if buf == "":
break
sys.stdout.write(buf)
try:
if os.stat(name).st_ino != curino:
new = open(name, "r")
current.close()
current = new
curino = os.fstat(current.fileno()).st_ino
continue
except IOError:
pass
time.sleep(1)
I doubt this works on Windows, but since you're speaking in terms of tail, I'm guessing that's not a problem. :)
You can do it by keeping track of where you are in the file and reopening it when you want to read. When the log file rotates, you notice that the file is smaller and since you reopen, you handle any unlinking too.
import time
cur = 0
while True:
try:
with open('myfile') as f:
f.seek(0,2)
if f.tell() < cur:
f.seek(0,0)
else:
f.seek(cur,0)
for line in f:
print line.strip()
cur = f.tell()
except IOError, e:
pass
time.sleep(1)
This example hides errors like file not found because I'm not sure of logrotate details such as small periods of time where the file is not available.
NOTE: In python 3, things are different. A regular open translates bytes to str and the interim buffer used for that conversion means that seek and tell don't operate properly (except when seeking to 0 or the end of file). Instead, open in binary mode ("rb") and do the decode manually line by line. You'll have to know the file encoding and what that encoding's newline looks like. For utf-8, its b"\n" (one of the reasons utf-8 is superior to utf-16, btw).
Thanks to #tdelaney and #Dolda2000's answers, I ended up with what follows. It should work on both Linux and Windows, and also handle logrotate's copytruncate or create options (respectively copy then truncate size to 0 and move then recreate file).
file_name = 'my_log_file'
seek_end = True
while True: # handle moved/truncated files by allowing to reopen
with open(file_name) as f:
if seek_end: # reopened files must not seek end
f.seek(0, 2)
while True: # line reading loop
line = f.readline()
if not line:
try:
if f.tell() > os.path.getsize(file_name):
# rotation occurred (copytruncate/create)
f.close()
seek_end = False
break
except FileNotFoundError:
# rotation occurred but new file still not created
pass # wait 1 second and retry
time.sleep(1)
do_stuff_with(line)
A limitation when using copytruncate option is that if lines are appended to the file while time-sleeping, and rotation occurs before wake-up, the last lines will be "lost" (they will still be in the now "old" log file, but I cannot see a decent way to "follow" that file to finish reading it). This limitation is not relevant with "move and create" create option because f descriptor will still point to the renamed file and therefore last lines will be read before the descriptor is closed and opened again.
Using 'tail -F
man tail
-F same as --follow=name --retr
-f, --follow[={name|descriptor}] output appended data as the file grows;
--retry keep trying to open a file if it is inaccessible
-F option will follow the name of the file not descriptor.
So when logrotate happens, it will follow the new file.
import subprocess
def tail(filename: str) -> Generator[str, None, None]:
proc = subprocess.Popen(["tail", "-F", filename], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
while True:
line = proc.stdout.readline()
if line:
yield line.decode("utf-8")
else:
break
for line in tail("/config/logs/openssh/current"):
print(line.strip())
I made a variation of the awesome above one by #pawamoy into a generator function one for my log monitoring and following needs.
def tail_file(file):
"""generator function that yields new lines in a file
:param file:File Path as a string
:type file: str
:rtype: collections.Iterable
"""
seek_end = True
while True: # handle moved/truncated files by allowing to reopen
with open(file) as f:
if seek_end: # reopened files must not seek end
f.seek(0, 2)
while True: # line reading loop
line = f.readline()
if not line:
try:
if f.tell() > os.path.getsize(file):
# rotation occurred (copytruncate/create)
f.close()
seek_end = False
break
except FileNotFoundError:
# rotation occurred but new file still not created
pass # wait 1 second and retry
time.sleep(1)
yield line
Which can be easily used like the below
import os, time
access_logfile = '/var/log/syslog'
loglines = tail_file(access_logfile)
for line in loglines:
print(line)
I am writing a python script that will go through a code file, comment out an import, build the program, and determine if each import is actually being used.
I have each component working, but for some reason when I combine them I am getting a very weird issue while writing back to the code file. It seems like whenever I use subprocess.check_output for building the code, the write commands for the file seem to always stop writing around line 315 without continuing on in the file. I would assume it was some race condition but it will always end on the same line each time I run it.
I have also tried with os.system() and had the same results. I'm not much of a python coder so maybe there is something obvious with IO that I'm missing?
Some Code:
with open(file, 'r+') as f:
content = f.read()
f.seek(0)
f.truncate()
lines = re.split("\r?\n", content);
for index, line in enumerate(lines):
result = regex.search(line)
if (result):
commentLine = '//' + line + ' //Not Needed'
lines[index] = commentLine
f.seek(0)
f.truncate()
for writeLine in lines:
f.write("%s\n" % writeLine)
returnValue = 0;
try:
returnValue = subprocess.check_output(['/usr/bin/xcodebuild', '-workspace', 'path_to_my_workspace' ,'-configuration', 'Debug', '-scheme', 'my_scheme'], stderr=subprocess.STDOUT)
except:
returnValue = 1
if (returnValue == 0): #Build succeeded!
print "Build succeeded!"
else:#Build failed
print "Build failed"
lines[index] = line
f.seek(0)
f.truncate()
for writeLine in lines:
f.write("%s\n" % writeLine)
Walkthrough:
Opens a file,
Splits it line by line searching for a regex matching #import etc etc,
Comments out this line,
Tries to build it,
If success, leave the comment,
if fail, remove the comment,
repeat for each regex match
I have a txt file with list of html/doc files, I want to download them using python and save them as 1.html, 2.doc, 3.doc, ...
http://example.com/kran.doc
http://example.com/loj.doc
http://example.com/sks.html
I've managed to create fully functional script except python will allways add question mark to the end of newly created file (if you look from linux) and if you look from windows file name would be something like 5CFB43~X
import urllib2
st = 1;
for line in open('links.txt', 'r'):
u = urllib2.urlopen(line)
ext = line.split(".")
imagefile = str(st)+"."+ext[-1]
#file created should be something.doc but its something.doc? -> notice question mark
fajl = open(imagefile, "w+")
fajl.write(u.read())
fajl.close()
print imagefile
st += 1
The line terminator is two characters, not one.
for line in open('links.txt', 'rU'):
But not anymore.
Work on line.strip() instead of line
That's because lines read this way will end up with '\n' at the end, hence the ?
Just add the following at the beginning of your loop:
if line.endswith('\n'):
line = line[:-1]
Or as AKX pointed out in the comments, just:
line = line.rstrip('\r\n')
And so you cover any kind of line ending.