We Keep Coding

Python- compare content of multiple text files with a master file line by line and print the difference

I have some issues in my program. I have been trying to come up with a script which compares text files with a master text file and the program prints out the difference.
Basically, these are network configuration and we need to compare them in bulk to make sure all devices have standard configurations. For example, the script should read each file (file1, file2..etc.) line by line and compare it with the master file (master.txt).
I am able to compare one file at a time, however, when comparing two or more files I get an "index out of range" error.
I want to compare multiple files, probably in hundreds so I need to know how to fix his loop. Understand that this could be because program trying to ready
import difflib
import sys
hosts0 = open("C:\\Users\\p1329760\\Desktop\\Personal\\Python\\Projects\\sample\\master.txt","r")
hosts1 = open("C:\\Users\\p1329760\\Desktop\\Personal\\Python\\Projects\\sample\\file1.txt","r")
hosts2 = open("C:\\Users\\p1329760\\Desktop\\Personal\\Python\\Projects\\sample\\file2.txt","r")
lines1 = hosts0.readlines()
#print(lines11)
with open('output_compare.txt', 'w') as f:
#global original_stdout
for i,lines2 in enumerate(hosts1):
if lines2 != lines1[i]:
original_stdout = sys.stdout
sys.stdout = f
print("line ", i, " in hosts1 is different \n")
print(lines2)
sys.stdout = original_stdout
else:
pass
with open('output_compare1.txt', 'w') as file:
for i,lines3 in enumerate(hosts2):
if lines3 != lines1[i]:
original_stdout = sys.stdout
sys.stdout = file
print("line ", i, " in hosts1 is different \n")
print(lines3)
sys.stdout = original_stdout
else:
pass

Hi here is what you could do:
You can have a list off all the file name
namefile = [....]
And a function which takes the file name
def compare (filename):
fileobj = open(filename)
infile = fileobj.read().split()
for i in range(0,len(infile)):
if infile[i] == masterin[i]:
pass
else:
print(...)
After that you have to open the master file
master = open( "...")
masterin = master.read().split()
After that a loop and your done
for i in namefile:
compare (i)

Replace sequences in Python

I want to make a replacement script.
It should replace str1 to str2.
My file has xml-based structure.
For example, I have:
...word1'#13#10'word2'#13#10'word3... = ...word1'#13#10'word3...
I want to remove some part of string.
I use this in a script:
Lines[i] = Lines[i].replace(key, DataBase[key])
I've already checked that "key" and "DataBase[key]" are correctly defined. If I print them into console with "print()" - it looks just like it has to.
But then script is executing it don't change sequences like this - with '#13#10'. Pairs of keys without any specific simbols works fine.
What can I do? And why it doesn't works well?
Full script:
import configparser
#import time
config = configparser.ConfigParser() # init configparser
config.optionxform = str
config.read("SocratToCortesExpress.cfg") # config file
print("Config file - readed")
filePath = config.get("PATH", "old_file") # config file - with names of files, pairs of words
DataStrings = config.items("DATA") # read pairs
DataBase = dict() # initialization of dictionary
print("Dictionary - initialized")
for Dstr in DataStrings: # old and new words for a replacement
SocratName = Dstr[0]
CortesName = Dstr[1]
DataBase[SocratName] = CortesName
print("Dictionary - fulfilled")
with open(filePath, "r", encoding='utf-8-sig') as ResultFile: # input file Lines = ResultFile.readlines()
print("Old file - uploaded")
f1 = open('logkeys.txt', 'w')
for key in DataBase.keys():
try:
f1.write('\n'+key+'\n'+DataBase[key]+'\n')
except Exception as e: #errors
f2 = open('log.txt', 'w')
f2.write('An exceptional thing happed - %s' %e)
f2.close()
f1.close()
for i in range(len(Lines)): # brutforce - all over input file
#Lines[i] = Lines[i].replace('\ufeff', '') #some weird symbol
for key in DataBase.keys():
try:
Lines[i] = Lines[i].replace(key, DataBase[key]) #replacing
except Exception as e: #errors
f2 = open('log.txt', 'w')
f2.write('An exceptional thing happed - %s' %e)
f2.close()
print("Sequences - replaced")
outFileName = config.get("PATH", "new_file") # define output file
print("Exit file - initialized")
with open(outFileName, "a", encoding='utf-8-sig') as outFile: # save
for line in Lines:
outFile.write(line)
print("OK")

Have you tried this?
>>> s = "word1'#13#10'word2'#13#10'word3"
>>> s.replace("'word2'#13#10'", '')
"word1'#13#10word3"

replace line if found or append - python

I have text that is key-value pairs separated by '='. I would like to replace the line if the key matches. if not, i would like to append it at the bottom. I've tried several ways, including:
def split_command_key_and_value(command):
if '=' in command:
command2 = command.split('=')
return command2
def test(command, path):
command2 = split_command_key_and_value(command)
pattern = command2[0]
myfile = open(path,'r') # open file handle for read
# use r'', you don't need to replace '\' with '/'
result = open(path, 'w') # open file handle for write
for line in myfile:
line = line.strip() # it's always a good behave to strip what you read from files
if pattern in line:
line = command # if match, replace line
result.write(line) # write every line
myfile.close() # don't forget to close file handle
result.close()
I know the above is just to replace text, but it deletes the text in the file, and I can't see why. Could someone point me in the right direction?
Thanks
Update:
I'm almost there, but some of my lines have similar keys, so mutiple lines are matching when only 1 should. I've tried to incorporate a regex boundary in my loop with no luck. My code is below. Does anyone have a suggestion?
There is some text in the file that isn't key-value, so I would like to skip that.
def modify(self, name, value):
comb = name + ' ' + '=' + ' ' + value + '\n'
with open('/file/', 'w') as tmpstream:
with open('/file/', 'r') as stream:
for line in stream:
if setting_name in line:
tmpstream.write(comb)
else:
tmpstream.write(line)
I think I got it. See code below.
def modify(self, name, value):
comb = name + ' ' + '=' + ' ' + value + '\n'
mylist = []
with open('/file/', 'w') as tmpstream:
with open('/file/', 'r') as stream:
for line in stream:
a = line.split()
b = re.compile('\\b'+name+'\\b')
if len(a) > 0:
if b.search(a[0]):
tmpstream.write(comb)
else:
tmpstream.write(line)
I spoke too soon. It stops at the key-value I provide. So, it only writes one line, and doesn't write the lines that don't match.
def modify(name, value):
comb = name + ' ' + '=' + ' ' + value + '\n'
mylist = []
with open('/file1', 'w') as tmpstream:
with open('/file2', 'r') as stream:
for line in stream:
a = line.split()
b = re.compile('\\b'+name+'\\b')
if len(a) > 0:
if b.search(a[0]):
tmpstream.write(comb)
else:
tmpstream.write(line)
Can anyone see the issue?

Because when you open file for writing
result = open(path, 'w') # open file handle for write
you just erase it content. Try to write in different file and after all work done replace old file with new one. Or read all data into memory and then process it and write to file.
with open(path) as f:
data = f.read()
with open(path, 'w') as f:
for l in data:
# make job here

first of all you are reading an writing the same file ...
you could first read it all and the write line by line
with open(path,'r') as f:
myfile = f.read() # read everything in the variable "myfile"
result = open(path, 'w') # open file handle for write
for line in myfile.splitlines(): # process the original file content 1 line at a time
# as before

I strongly recommend reading python's documentation on how to read and write files.
If you open an existing file in write-mode open(path, 'w'), its content will be erased:
mode can be (...) 'w' for only writing (an existing file with the same name will be erased)
To replace a line in python you can have a look at this: Search and replace a line in a file in Python
Here is one the solutions provided there adapted to your context (tested for python3):
from tempfile import mkstemp
from shutil import move
from os import close
def test(filepath, command):
# Split command into key/value
key, _ = command.split('=')
matched_key = False
# Create a temporary file
fh, tmp_absolute_path = mkstemp()
with open(tmp_absolute_path, 'w') as tmp_stream:
with open(filepath, 'r') as stream:
for line in stream:
if key in line:
matched_key = True
tmp_stream.write(command + '\n')
else:
tmp_stream.write(line)
if not matched_key:
tmp_stream.write(command + '\n')
close(fh)
move(tmp_absolute_path, filepath)
Note that with the code above every line that matches key (key=blob or blob=key) will be replaced.

File Input/Output Error?

can anyone tell me what is wrong with my code?
def count_letters(in_file, out_file):
in_file = open("in_file.txt", "r")
for line in in_file:
for x in sorted(line):
count = x.count()
out_file = open("out_file.txt", "w")
print(x, " ", count)
in_file.close()
out_file.close()
print(out_file)
it's supposed to
Takes two filenames (in_file and out_file)as arguments
Opens and reads the input file specified by in_file, and counts the number of occurrences of each letter (in a case-insensitive manner)
Writes the result in the output file specified by out_file
when I run it, it tells me that "in_file" is not defined, but I have defined it, and made an "in_file.txt."
any help would be appreciated!

You want pass "in_file.txt" as in_file variable and use it like:
in_f=open(in_file,"r")
similarly for out_file otherwise, what is the point of making a function?
But i think your error relates of not having "in_file.txt" in your current working directory, you can check with
import os
print os.getcwd()
this will show what files code can see

in_file = open(in_file, "r") # you must do the way so got an error
try this function:
def count_letters(in_file, out_file):
in_file_fd = open(in_file, "r") # open in file
out_file_fd = open(out_file, "w") # open out file
for line in in_file_fd:
for x in sorted(line):
count = line.count(x)
res = x + " " + str(count) + '\n'
print (res)
out_file_fd.write(res) # write the data into out file
in_file_fd.close() # close in_file
out_file_fd.close() # close out_file
count_letters('test.py', 'out_file')
It works for me, hope helpful.

Python: Issue when trying to read and write multiple files

This script reads and writes all the individual html files in a directory. The script reiterates, highlight and write the output.The issue is, after highlighting the last instance of the search item, the script removes all the remaining contents after the last search instance in the output of each file. Any help here is appreciated.
import os
import sys
import re
source = raw_input("Enter the source files path:")
listfiles = os.listdir(source)
for f in listfiles:
filepath = os.path.join(source+'\\'+f)
infile = open(filepath, 'r+')
source_content = infile.read()
color = ('red')
regex = re.compile(r"(\b in \b)|(\b be \b)|(\b by \b)|(\b user \b)|(\bmay\b)|(\bmight\b)|(\bwill\b)|(\b's\b)|(\bdon't\b)|(\bdoesn't\b)|(\bwon't\b)|(\bsupport\b)|(\bcan't\b)|(\bkill\b)|(\betc\b)|(\b NA \b)|(\bfollow\b)|(\bhang\b)|(\bbelow\b)", re.I)
i = 0; output = ""
for m in regex.finditer(source_content):
output += "".join([source_content[i:m.start()],
"<strong><span style='color:%s'>" % color[0:],
source_content[m.start():m.end()],
"</span></strong>"])
i = m.end()
outfile = open(filepath, 'w')
outfile.seek(0, 2)
outfile.write(output)
print "\nProcess Completed!\n"
infile.close()
outfile.close()
raw_input()

After your for loop is over, you need to include whatever is left after the last match:
...
i = m.end()
output += source_content[i:]) # Here's the end of your file
outfile = open(filepath, 'w')
...