I run my own business from home and started use Python 2 days ago. I'm trying to write a script that will search through my log files line by line and tell me if a system doesn't match my mandatory naming scheme. There are multiple different schemes and I want the script to look for them all. I've tried using a list (as seen below) but that won't work and then I tried with normal brackets and that gave me an error (requires left operand, not tuple). I've noted the lines that give me a problem.
#variables
tag = ["DATA-", "MARK", "MOM", "WORK-"] #THIS ONE!!!!!!
#User Input
print "Please select Day of the week"
print "1. Monday"
print "2. Tuesday"
print "3. Wednesday"
print "4. Thursday"
print "5. Friday"
print "6. Saturday"
print "7. Sunday"
day = input("> ")
#open appropriate file and check to see if 'tag' is present in each line
#then, if it doesn't, print the line out.
if day == 1:
f = open('F:\DhcpSrvLog-Mon.log', 'r')
for line in f:
if tag in line: #THIS ONE!!!!!!!!!!!!!
pass
else:
print line
Any tips or tricks would be most appreciated!
I suggest rewriting the code like this:
with open('F:\DhcpSrvLog-Mon.log', 'rU') as f:
for line in f:
for t in tag:
if t in line: break
else:
print line
Using with you automagically close the file on exit of the block, so you needn't worry about forgetting to close it. Using else: in the for loop only triggers if you don't break out of the loop earlier.
if day == 1:
f = open('F:\DhcpSrvLog-Mon.log', 'r')
for line in f:
if tag in line: #THIS ONE!!!!!!!!!!!!!
pass
else:
print line
replace with
if day == 1:
f = open('F:\DhcpSrvLog-Mon.log', 'r')
for line in f:
if [x for x in tag if x in line]: #THIS ONE!!!!!!!!!!!!!
pass
else:
print line
use any to check this. It is more efficient because it will not try all the tags if it find that one is in line.
any(x in line for x in tag)
contains_tag=False
for t in tag:
if t in line:
contains_tag=True
break # Found a match, break out of for loop
if not contains_tag:
print (line)
First you need to loop through each of the tags (e.g. for t in tag). Then you need to check if the string t is contained in line.
Since you're only looking for one tag that does match, the simplest way is to keep track with a boolean variable.
If you wanted to only look for log messages that start with that tag, you could say if line.startswith(t) instead of if t in line
Related
I want my code to be able to find what the user has asked for and print the 5 following lines. For example if the user entered "james" into the system i want it to find that name in the text file and read the 5 lines below it. Is this even possible? All i have found whilst looking through the internet is how to read specific lines.
So, you want to read a .txt file and you want to read, let's say the word James and the 5 lines after it.
Our example text file is as follows:
Hello, this is line one
The word James is on this line
Hopefully, this line will be found later,
and this line,
and so on...
are we at 5 lines yet?
ah, here we are, the 5th line away from the word James
Hopefully, this should not be found
Let's think through what we have to do.
What We Have to Do
Open the text file
Find the line where the word 'James' is
Find the next 5 lines
Save it to a variable
Print it
Solution
Let's just call our text file info.txt. You can call it whatever you want.
To start, we must open the file and save it to a variable:
file = open('info.txt', 'r') # The 'r' allows us to read it
Then, we must save the data from it to another variable, we shall do it as a list:
file_data = file.readlines()
Now, we iterate (loop through) the line with a for loop, we must save the line that 'James' is on to another variable:
index = 'Not set yet'
for x in range(len(file_data)):
if 'James' in file_data[x]:
index = x
break
if index == 'Not set yet':
print('The word "James" is not in the text file.')
As you can see, it iterates through the list, and checks for the word 'James'. If it finds it, it breaks the loop. If the index variable still is equal to what it was originally set as, it obviously has not found the word 'James'.
Next, we should find the five lines next and save it to another variable:
five_lines = [file_data[index]]
for x in range(5):
try:
five_lines.append(file_data[index + x + 1])
except:
print(f'There are not five full lines after the word James. {x + 1} have been recorded.')
break
Finally, we shall print all of these:
for i in five_lines:
print(i, end='')
Done!
Final Code
file = open('info.txt', 'r') # The 'r' allows us to read it
file_data = file.readlines()
index = 'Not set yet'
for x in range(len(file_data)):
if 'James' in file_data[x]:
index = x
break
if index == 'Not set yet':
print('The word "James" is not in the text file.')
five_lines = [file_data[index]]
for x in range(5):
try:
five_lines.append(file_data[index + x + 1])
except:
print(f'There are not five full lines after the word James. {x + 1} have been recorded.')
break
for i in five_lines:
print(i, end='')
I hope that I have been helpful.
Yeah, sure. Say the keyword your searching for ("james") is keywrd and Xlines is the number of lines after a match you want to return
def readTextandSearch1(txtFile, keywrd, Xlines):
with open(txtFile, 'r') as f: #Note, txtFile is a full path & filename
allLines = f.readlines() #Send all the lines into a list
#with automatically closes txt file at exit
temp = [] #Dim it here so you've "something" to return in event of no match
for iLine in range(0, len(allLines)):
if keywrd in allLines[iLine]:
#Found keyword in this line, want the next X lines for returning
maxScan = min(len(allLines),Xlines+1) #Use this to avoid trying to address beyond end of text file.
for iiLine in range(1, maxScan):
temp.append(allLines[iLine+iiLine]
break #On assumption of only one entry of keywrd in the file, can break out of "for iLine" loop
return temp
Then by calling readTextandSearch1() with appropriate parameters, you'll get a list back that you can print at your leisure. I'd take the return as follows:
rtn1 = readTextAndSearch1("C:\\Docs\\Test.txt", "Jimmy", 6)
if rtn1: #This checks was Jimmy within Test.txt
#Jimmy was in Test.txt
print(rtn1)
Been working at this for quite a while, and I haven't found any examples on this site or any others that seem relevant. I have a list, and what I'm trying to do is pretty simple I think. I just need to search that list to find the key word "Buffer Log". Once I find that keyword, I need to print every line from that line, until the end of the list. Any direction at all would be very much appreciated. It seems like I'm pretty close.
logfile = open('logs.txt', 'r')
readlog = logfile.readlines()
logfile.close()
lines = []
for line in readlog:
lines.append(line)
for x in lines:
if "Log Buffer" in x:
z =lines.index(x)
print(lines[z:]
Firstly, the code:
lines = []
for line in readlog:
lines.append(line)
Is unnecessary because readlog is already a list. You can try something along the lines of this:
found = False
for line in readlog: # because this is already a list
if "Log Buffer" in line:
found = True # set our logic to start printing
if found:
print(line)
You don't need the for loop in which yoú're creating one more list called lines.
You can use enumerate() to keep track of which line number you are at, while searching for the line that contains 'Log Buffer'
When you find the line that contains 'Log Buffer', just remember that line number, exit the loop, and then print all lines from readlog starting from that line number.
logfile = open('logs.txt', 'r')
readlog = logfile.readlines()
logfile.close()
for i,x in enumerate(readlog):
if 'Log Buffer' in x:
z = i # Remember the value of i for which the line had 'Log Buffer'
break # Exit the loop.
print (*readlog[z:]) # Move your print statement outside the loop.
I need to search for something in a tab delimited text file. The user is supposed to input both the file and the thing that needs searching for. The programme is then supposed to return the whole line that the user inputted word is in. I have got two models so far because I've been coming at this problem from different angles. The first programme goes as follows:
import csv
searchfile = raw_input ('Which file do you want to search? ')
try:
input_file = open (searchfile, 'rU')
except:
print "Invalid file. Please enter a correct file"
csv_file_object = csv.reader(open(searchfile, 'rb'))
header = csv_file_object.next()
data=[]
for row in csv_file_object:
data.append(row)
searchA = raw_input ('which author?')
author_search = data[0::,0] == searchA
if author_search in searchfile:
print author_search
The problem with the first programme is that this error pops up:
TypeError: list indices must be integers, not tuple
I therefore attempted this method:
import csv
searchfile = raw_input ('Which file do you want to search? ')
try:
input_file = open (searchfile, 'rU')
except:
print "Invalid file. Please enter a correct file"
with open(searchfile) as f:
reader = csv.reader(f, delimiter="\t")
d = list(reader)
searchtype = raw_input ('Search on author or journal/conference or [Q = quit]')
if searchtype == 'author':
searchdataA = raw_input ("Input author name")
if searchdataA in input_file:
print line
elif searchtype == 'journal' or 'conference' or 'journal/conference':
searchdataJ = raw_input ("input journal/conference name")
if searchdataJ in d:
print line
elif searchtype == 'Q':
print "Program left"
else:
print "please choose either author or journal/conference"
This is unable to get beyond inputting the search parameters.
Any help on where to go with either programme would be much appreciated, or if I'm completely on the wrong track then links to useful material would be great.
I think you're making this a bit more complicated than it needs to be. Since you want to print the whole line that the target word appeared on, you don't really need the CSV module. You're not doing any of the sophisticated parsing it is capable of.
searchfile = raw_input ('Which file do you want to search? ')
searchA = raw_input ('which author?')
with open(searchfile) as infile:
for line in infile:
if searchA in line:
print(' '.join(line.split()))
break # remove this if you want to print all matches instead of
# just the first one
Notice that in the when printing the line, I first split the line (which splits on whitespace by default), then rejoin the fields with two spaces between them. I think doing something like this would be a good way to go for you since you're printing tab-separated fields on the console. Reducing that extra space will make your prints a bit easier to read, but using two spaces still makes it easy to distinguish the columns from each other.
You can generalize it by prompting your user for any search term, instead of specifying "author". This may be the way to go, since your second code snippet suggests that you may want to search for other fields, like "journal" or "conference":
target_term = raw_input("Which term or phrase would you like to find?")
Since this method searches in and prints the entire line, there's no need to deal with the separate columns and different kinds of search terms. It just looks at the whole row at once and prints a matching line.
Since you are not really using a different search method, depending on if you are searching for author, journal, conference or journal/conference. So you could actually do a full text search on the line. Therefore it is wise to collect all data you need from user BEFORE processing file, so you can output just the matching lines. If user passes a rather large CSV file, then your way would take up far too much memory.
with open(searchfile, 'r') as f:
for line in f:
if line.find(searchA) > -1:
print line
This way you are looping through the file as fast as possible and prints out all matching lines.
The .find() function returns the index to where in the string he found the match and otherwise -1 if the string was not found. So from the value you could "estimate" on where the match was made, but if you really want to differentiate between author, journal, etc. then you will have to split the line. In my sample i will assume the author field to be the sixth field in the CSV line:
with open(searchfile, 'r') as f:
for line in f:
fields = line.split("\t")
if len(fields) > 5: # check length of fields array
if fields[5].find(searchA) > -1: # search straight in author field
print line # return full line
why not simply
fname = raw_input("Enter Filename")
author = raw_input("Enter Author Name:")
if author in open(fname,"rb").read():
print "match found"
if you want to see the lines you could do
print re.findall(".*%s.*"%(author),open(fname,"rb").read())
as people point out it is better form to do
with open(fname,"rb") as f:
data = print re.findall(".*%s.*"%(author),f.read())
although in CPython it will be garbage collected immediatly so its not really a problem....
The first thing that came to my mind is simply:
def check_file(file_name, author_name):
with open(file_name) as f:
content = f.readlines()
for line in content:
if author_name in line:
print "Found: ", line
Hope it can be useful.
This question already has answers here:
How to test multiple variables for equality against a single value?
(31 answers)
Closed 8 years ago.
I am trying to process a txt file I created of songs and genres, but don't know how to tell python what to look for. The txt file is formatted like so:
7/28/2012
1.1. Lovin’ You Is Fun—Country
[...]
1.100. Call Me Maybe—Teen pop, dance-pop
[]2. 7/27/2013
2.1. Crown—Hip hop
(brackets not in actual file)
I've tried writing code to find certain words in lines, but the code finds every line of the file as containing whatever string I pass it (even when that's obviously not the case). Here's the code:
try:
infilehandle = open(afile, "r")
alllines = infilehandle.read().splitlines(False)
infilehandle.close()
except:
alllines=None
print "Unable to read file {0}".format(afile)
for aline in alllines:
if aline.find("Country"):
country_count += 1
if aline=="1. 7/28/2012" or "2. 7/27/2013" or "3. 4/27/2013" or "3. 4/27/2013":
continue
else:
continue
If the code worked the way I wanted it to, it would categorize each line by the first number on the line, then search for certain strings and add to a count for those strings. Is this possible to do?
You should write:
if aline.find("Country") != -1:
or even better would be
if "Country" in aline:
Also, your second if should read:
if aline=="foo" or aline=="bar" or aline=="zom":
Find doesn't return a boolean. It returns an int. In particular, it returns -1 if it doesn't find the string.
And as jonrsharpe mentions, you are using or incorrectly, also.
try:
infilehandle = open(afile, "r")
alllines = infilehandle.read().splitlines(False)
infilehandle.close()
except:
alllines=None
print "Unable to read file {0}".format(afile)
exit(1)
for aline in alllines:
if aline.find("Country") != -1:
country_count += 1
if aline=="1. 7/28/2012" or
aline=="2. 7/27/2013" or
aline=="3. 4/27/2013" or
aline=="3. 4/27/2013":
pass # I have no idea what you're trying to do here
I got a .txt file with this list structure:
["saelyth", "somehting here", "Not needed", "2013-08-24 14:14:47"]
["anothername", "whatever 1", "Not needed neither", "2013-08-24 15:12:26"]
["athirdone", "just an example", "of the list structure", "2013-08-24 15:12:51"]
And I need to replace the second value of a specific list only if the value 1 is found in a file. The code I'm trying is this one, but so far that one is just appending data to the file instead of replacing it.
horaactual = datetime.datetime.now()
filename = "datosdeusuario.txt"
leyendo = open(filename, 'r+')
buffer = leyendo.read()
leyendo.close()
fechitaguardaips = str(horaactual)[:19]
escribiendo = open(filename, 'r+')
for line in escribiendo:
retrieved = json.loads(line)
if retrieved[0] == user.name:
retrieveddata1 = "prueba"
mythirdvalue = "not important"
escribiendo.write(json.dumps([user.name, retrieveddata1, mythirdvalue, fechitaguardaips])+"\n")
break
escribiendo.close()
I guess the fail is in the line escribiendo.write. However, I've been googling for two hours and I did get this far, but I don't find a specific call to Replace data instead of writing or appending. How can I fix this problem?
This is what happens and shouldn't :(
["saelyth", "prueba", "", "2013-08-27 13:25:14"]
["saelyth", "prueba", "", "2013-08-27 13:25:32"]
["saelyth", "prueba", "", "2013-08-27 13:26:01"]
I also got troubles understanding what a Break does as I'd like to stop infinite loops from my code (it happened to me yesterday in a different section of the program, but I was also using "For line in X" in JSON Python).
My approach is like this (based on an answer to Stack Overflow question Loading and parsing a JSON file in Python):
import json
data = []
with open('text.json', 'r+') as f:
for line in f:
data_line = json.loads(line)
if data_line[0] == 'saelyth' and '1' in data_line[1]:
data_line[1] = 'new value'
data.append(data_line)
f.seek(0)
f.writelines(["%s\n" % json.dumps(i) for i in data])
f.truncate()
Please correct me if I got your question the wrong way.
About your question with break, check Python break, continue and pass Statements.
Here is an example which you could tweak to your needs:
from contextlib import nested
filename = 'datosdeusuario.txt'
with nested( open(filename,'r'),open(filename,'w') ) as f1,f2:
for line in f1:
f2.write(line.replace('foo','bar'))
This would replace every instance of the substring foo by bar in your file, even though it does open twice the file.