Program Goal: Search a defined yamlfile (scan_dcn.yaml) and return all lines matching the search criteria as defined in the function_search_search_key() and function_search_event_type() functions.
Input File - scan_dcn.yaml:
search_dict:
[
{search_key: ["Failed to Process the file"],
event_type: "evttyp_repl_dcn_error",
event_description: "Failure to process DCN file",
priority: 50,
scan_interval: 1,
remove_dups: True,
category: "dcn",
context_begin: 0,
context_end: 1,
reportable: False,
offset: 0
},
Problem:
My program will return function_search_search_key() but will not proceed to function_search_event_type().
I would think that my problem is that I have no logic to proceed to the second function after the completion of the first.
Do I need to return a value in each function to proceed?
Python Source Code
yamlfile = open('scan_dcn.yaml', 'r')
def function_search_search_key():
search_search_key = ['{search_key:']
for line in yamlfile.readlines():
for word in search_search_key:
if word in line:
print(line)
def function_search_event_type():
search_event_type = ['event_type:']
for line in yamlfile.readlines():
for word in search_event_type:
if word in line:
print(line)
def main():
function_search_search_key()
function_search_event_type()
main()
In your first function you read the whole file with readlines. When you use readlines again in your second function you're already at the end of the file and there is no more data to read, so the for loop is not even entered.
But there's no need to read the file again for every function. Read the file outside of the functions and put it in a list. Then add a parameter to each of those functions that takes this list. In the function you can loop over the list.
def function_search_search_key(lines):
search_search_key = ['{search_key:']
for line in lines:
for word in search_search_key:
if word in line:
print(line)
def function_search_event_type(lines):
search_event_type = ['event_type:']
for line in lines:
for word in search_event_type:
if word in line:
print(line)
def main():
with open('scan_dcn.yaml', 'r') as yamlfile:
lines = yamlfile.readlines()
function_search_search_key(lines)
function_search_event_type(lines)
if __name__ = '__main__':
main()
If you ever need to change the name of the file you can do it in one place. If you open and read the file in every single function you would have to change all occurrances of the file name.
Your second function is being entered. It must if the call above it has finished.
You aren't seeing anything printed because you're attempting to loop though the same file more than once. Once you've read the file, it's exhausted. You can just re-read the file as a simple fix:
def function_search_search_key():
with open('scan_dcn.yaml', 'r') as yamlfile:
search_search_key = ['{search_key:']
for line in yamlfile.readlines():
for word in search_search_key:
if word in line:
print(line)
def function_search_event_type():
with open('scan_dcn.yaml', 'r') as yamlfile: # Read the file again
search_event_type = ['event_type:']
for line in yamlfile.readlines():
for word in search_event_type:
if word in line:
print(line)
You can read a file descriptor only once (if you don't seek to start), so you may open your file in each function
def function_search_search_key():
search_search_key = ['{search_key:']
with open('scan_dcn.yaml') as fic:
for line in fic:
for word in search_search_key:
if word in line:
print(line)
def function_search_event_type():
search_event_type = ['event_type:']
with open('scan_dcn.yaml') as fic:
for line in fic:
for word in search_event_type:
if word in line:
print(line)
Related
I'm making a joke program that has a text file storing jokes. On program load, it grabs all the lines from the file and assigns them to a jokes list Everything but the remove joke function is working. Whenever you call remove joke, it ends up re-writing every line in the text file to an empty string instead of the selected line
When running this function, it does remove the joke from the jokes list properly
def remove_joke():
for i in range(len(jokes)):
print(f"{i}\t{jokes[i]}")
remove_index = int(input("Enter the number of the joke you want to remove:\t"))
with open("jokes.txt", "r") as f:
lines = f.readlines()
with open("jokes.txt", "w") as f:
for line in lines:
print(line)
if line == jokes[remove_index]:
f.write("")
jokes.remove(jokes[remove_index])
Instead of
if line == jokes[remove_index]:
f.write("")
you want:
if line != jokes[remove_index]:
f.write(line+'\n')
Or even:
if line != jokes[remove_index]:
print(line, file=f)
I am attempting to print the file, split by line using two methods: one is using the method read on files and the second is using a for loop and splitting the files into lines. I am getting a Traceback error on the last line stating that "words" is not defined. I cannot see why this is the case.
fname = input('enter file name')
try:
fhandle = open(fname, 'r')
except:
print('file does not exist')
exit()
#store entire file in a variable called data
data = fhandle.read()
print(data)
#iterate through each line in a file handle
for line in fhandle:
line = line.strip()
words = line.split()
print(words)
When reading a file, Python keeps track of a cursor within the file. Data is read from the position of the cursor onwards, and reading moves the cursor forward to the end of the data that was read. This is so that, e.g., calling f.readline() twice will return the next line each time, rather than the first line both times.
When you call f.read(), the whole file is read, so the cursor is moved to the end of the file. Then, when you iterate through fhandle, Python only considers the lines ahead of the cursor — of which there are none. Since the object being iterated through is empty, the body of the for loop is never executed, so words is never assigned to.
You can fix this by calling fhandle.seek(0) directly before the for loop to return the cursor to the start of the file.
There is also a logical error in your program. If you want to print every line, not just the last, in your for loop, you need to indent print(words) so that it's in the for loop.
As a best practice, you should also call fhandle.close() when you're finished using the file.
words it not define because of read(), it makes for loop didn't return anything.
Python file method read() reads at most size bytes from the file. If
the read hits EOF before obtaining size bytes, then it reads only
available bytes.
When print(words) is indented in for loop, it just return nothing too. But if read() is removed while print(words) isn't indented, it'll return a list of the last line:
fname = input('enter file name')
try:
fhandle = open(fname, 'r')
except:
print('file does not exist')
exit()
# store entire file in a variable called data
# data = fhandle.read()
# print(data)
# iterate through each line in a file handle
for line in fhandle:
line = line.strip()
words = line.split()
print(words)
# ['Line', '4']
And if print(words) is indented while read() is removed, it'll return this:
fname = input('enter file name')
try:
fhandle = open(fname, 'r')
except:
print('file does not exist')
exit()
# store entire file in a variable called data
# data = fhandle.read()
# print(data)
# iterate through each line in a file handle
for line in fhandle:
line = line.strip()
words = line.split()
print(words)
# ['Line', '1']
# ['Line', '2']
# ['Line', '3']
# ['Line', '4']
I'm not sure what is your intent using split() but if you just want to print line by line using read(), your code already did that.
When using for loop, just comment or remove read() then just print line
fname = input('enter file name')
try:
fhandle = open(fname, 'r')
except:
print('file does not exist')
exit()
# store entire file in a variable called data
# data = fhandle.read()
# print(data)
# iterate through each line in a file handle
for line in fhandle:
print(line.strip())
# Line 1
# Line 2
# Line 3
# Line 4
But if you're intend to make a list consisted of each line, you can use splitlines()
fname = input('enter file name')
try:
fhandle = open(fname, 'r')
except:
print('file does not exist')
exit()
#store entire file in a variable called data
data = fhandle.read().splitlines()
print(data)
# ['Line 1', 'Line 2', 'Line 3', 'Line 4']
Hopes this help.
I want to search for particular text and replace the line if the text is present in that line.
In this code I replace line 125, but want to replace dynamically according to the text:
file = open("config.ini", "r")
lines = file.readlines()
lines[125] = "minimum_value_gain = 0.01" + '\n'
f.writelines(lines)
f.close()
How do I make it that if a line has:
minimum_value_gain =
then replace that line with:
minimum_value_gain = 0.01
There is no reason for you to manually parse a config.ini file textually. You should use configparser to make things much simpler. This library reads the file for you, and in a way converts it to a dict so processing the data is much easier. For your task you can do something like:
import configparser
config = configparser.ConfigParser()
config.read("config.ini")
for section in config:
if config.has_option(section, "minimum_value_gain"):
config.set(section, "minimum_value_gain", "0.01")
with open("config.ini", 'w') as f:
config.write(f)
Since you are replacing complete line so if statement will do the trick for you, no need to replace text
#updated make sure one line doesn't have both values
file = open("config.ini", "r")
lines=file.readlines()
newlines = []
for line in lines:
if "minimum_value_gain" in line:
line = "minimum_value_gain = 0.01" + '\n'
if "score_threshold" in line:
line = "Values you want to add"+'\n'
newlines.append(line)
f.writelines(newlines)
f.close()
Little bit messy and not optimized but get's the job the, first readlines and inserts the next_text to the given pos(line). If the line doesn't exists Raises IndexError, else writes to the file
def replace_in_file(filename: str, search_text: str, string_to_add: str) -> None:
with open(filename, "r+") as file_to_write:
lines = file_to_write.readlines()
file_to_write.seek(0)
file_to_write.truncate()
for line in lines:
if line.startswith(search_text):
line = line.rstrip("\n") + string_to_add + "\n"
file_to_write.write(line)
replace_in_file("sdf.txt", "minimum_value_gain", " = 0.01")
You can use also the regex library of Python.
Here is an example.
It is better not to read and write in the same file, that is not good practice. Write in a different file then eventually rename it.
import re
pattern = 'minimum_value_gain'
string_to_replace = 'minimum_value_gain = 0.01\n'
file = open("config.ini", "r")
fileout = open("new_config.ini", "a")
lines=file.readlines()
newlines = [string_to_replace if re.match(pattern, line) else line for line in lines]
f.close()
fileout.writelines(lines)
fileout.close()
You can rename the file afterwards :
import os
os.remove("config.ini")
os.rename("new_config.ini", "config.ini")
Set the string you would like to look for (match_string = 'example')
Have a list output_list that is empty
Use with open(x,y) as z: (this will automatically close the file after completion)
for each line in file.readlines() - run through each line of the file
The if statement adds your replacement line if the match_string is in the line, else just the adds the line
NOTE: All variables can be any name that is not reserved (don't call something just 'list')
match_string = 'example'
output_list = []
with open("config.ini", "r") as file:
for line in file.readlines():
if match_string in line:
output_list.append('minimum_value_gain = 0.01\n')
else:
output_list.append(line)
Maybe not ideal for the first introduction to Python (or more readable) - But I would have done the problem as follows:
with open('config.ini', 'r') as in_file:
out_file = ['minimum_value_gain = 0.01\n' if 'example' in line else line for line in in_file.readlines()]
To replace a specific text in a string
a = 'My name is Zano'
b = a.replace('Zano', 'Zimmer')
Is it possible to use python to skip blocks of text when writing a file from another file?
For example lets say the input file is:
This is the file I would like to write this line
I would like to skip this line
and this one...
and this one...
and this one...
but I want to write this one
and this one...
How can I write a script that allows me to skip certain lines that differ in content and size which resumes writing the lines to another file once it recognizes a certain line?
My code reads through the lines, doesn't write duplicate lines and performs some operation on the line by using dictionaries and regex.
def is_wanted(line):
#
# You have to define this!
#
# return True to keep the line, or False to discard it
def copy_some_lines(infname, outfname, wanted_fn=is_wanted):
with open(infname) as inf, open(outfname, "w") as outf:
outf.writelines(line for line in inf if wanted_fn(line))
copy_some_lines("file_a.txt", "some_of_a.txt")
In order to extend this to multi-line blocks, you can implement a finite state machine like
which would turn into something like
class BlockState:
GOOD_BLOCK = True
BAD_BLOCK = False
def __init__(self):
self.state = self.GOOD_BLOCK
def is_bad(self, line):
# *** Implement this! ***
# return True if line is bad
def is_good(self, line):
# *** Implement this! ***
# return True if line is good
def __call__(self, line):
if self.state == self.GOOD_BLOCK:
if self.is_bad(line):
self.state = self.BAD_BLOCK
else:
if self.is_good(line):
self.state = self.GOOD_BLOCK
return self.state
then
copy_some_lines("file_a.txt", "some_of_a.txt", BlockState())
Pseudo-code:
# Open input and output files, and declare the unwanted function
for line in file1:
if unwanted(line):
continue
file2.write(line)
# Close files etc...
You can read the file line by line, and have control on each line you read:
with open(<your_file>, 'r') as lines:
for line in lines:
# skip this line
# but not this one
Note that if you want to read all lines despite the content and only then manipulate it, you can:
with open(<your_file>) as fil:
lines = fil.readlines()
This should work:
SIZE_TO_SKIP = ?
CONTENT_TO_SKIP = "skip it"
with open("my/input/file") as input_file:
with open("my/output/file",'w') as output_file:
for line in input_file:
if len(line)!=SIZE_TO_SKIP and line!=CONTENT_TO_SKIP:
output_file.write(line)
I am processing a file line by line. Each line is checked, whether it contains a given text. Then, the next line needs to be assigned to the variable
i_line = iter(file)
for i_line in file:
if text in i_line:
#Go to the next line
line = next(i_line, None) #A problem
break
How to increment iterator i_line so as to point to the next line of the file? Both constructions do not work for me
next(i_line, None)
i_line.next()
Just do next(f).
Something like
>>> with open('testFile.txt', 'r') as f:
for line in f:
if 'cat' in line:
final_string = next(f)
So, for an input text file of
My Name is ABC.
I am a male.
My pet is a cat.
It's name is Ronald.
I hate rats.
The above code gives
>>> final_string
"It's name is Ronald.\n"
Use line = file.next() instead of line = next(i_line, None).
Here's a way of processing a file with next().
with open(filename) as file:
line = next(file, None)
while line:
print(line.strip())
line=next(file, None)
Using both the for line in file iterator and next would print every other line:
with open(filename) as file:
for line in file:
print(line.strip())
line = next(file,None)