We Keep Coding

Python - Problem in appending lines to output file with specific condition

My Problem is the following:
I have a file with lines that normally start with 'ab', condition is when line not start with ab it should be appended to previous line, but some lines are not get appended to output file
Source File:
grpid;UserGroup;Name;Description;Owner;Visibility;Members -> heading
ab;user1;name1;des1;bhalji;public
sss
ddd
fff
ab;user2;name2;des2;bhalji;private -> not appended in output
ab;user3;name3;des3;bhalji;public -> not appended in output
ab;user4;name4;des4;bhalji;private
rrr
ttt
yyy
uuu
ab;user5;name5;des5;bhalji;private
ttt
ooo
ppp
Here's is what I'm doing using python:
def grouping():
output = []
temp = []
currIdLine = ""
with( open ('usergroups.csv', 'r')) as f:
for lines in f.readlines():
line = lines.strip()
if not line:
print("Skipping empty line")
continue
if line.startswith('grpid'):
output.append(line)
continue
if line.startswith('ab'):
if temp:
output.append(currIdLine + ";" + ','.join(temp))
temp.clear()
currIdLine = line
else:
temp.append(line)
output.append(currIdLine + ";" + ','.join(temp))
#print("\n".join(output))
with open('new.csv', 'w') as f1:
for row in output:
f1.write(row + '\n')
grouping ()
Output of the above code:
grpid;UserGroup;Name;Description;Owner;Visibility;Members
ab;user1;name1;des1;bhalji;public;sss,ddd,fff
ab;user4;name4;des4;bhalji;private;rrr,ttt,yyy,uuu
ab;user5;name5;des5;bhalji;private;ttt,ooo,ppp
I hope this should be quite easy with Python but I'm not getting it right so far.
That's how the file should look at the end:
Expected Output:
grpid;UserGroup;Name;Description;Owner;Visibility;Members
ab;user1;name1;des1;bhalji;public;sss,ddd,fff
ab;user2;name2;des2;bhalji;private
ab;user3;name3;des3;bhalji;public
ab;user4;name4;des4;bhalji;private;rrr,ttt,yyy,uuu
ab;user5;name5;des5;bhalji;private;ttt,ooo,ppp

You are close. Missing an else statement to deal with the case that the line starts with ab, and the previous line started with ab.
def grouping():
output = []
temp = []
currIdLine = ""
with(open('usergroups.csv', 'r')) as f:
header = f.readline()
output.append(line.strip())
temp = []
for line in f.readlines():
if not line:
print("Skipping empty line")
continue
if line.startswith('ab'):
if temp:
output.append(currIdLine + ";" + ','.join(temp))
temp = []
else:
output.append(currIdLine)
currIdLine = line.strip()
else:
temp.append(line.strip())
if temp:
output.append(currIdLine + ";" + ','.join(temp))
else: # <-- this block is needed
output.append(currIdLine)
with open('new.csv', 'w') as f1:
for row in output:
f1.write(row + '\n')

Code that concatenates new lines does not execute else statement

with open('small_plate_data.txt', 'r') as f:
for line in f.read().split("\n"):
p = 1
op = ''
cntr = 2
num_plate = 0
if line.startswith("Plate"):
with open('Plate' + str(cntr) + '.txt', 'w+') as opf:
opf.write("Plate"+ str(num_plate) + "Data")
opf.write(op)
opf.close
op = ''
cntr += 1
p += 1
num_plate +=1
else:
op = op + '\n' + line
f.close()
The code above opens a txt file and reads it line by line. It concatenates every line together unless a line begins with 'plate', where it then takes whatever has been concatenated and then saves it to a file. The problem is that it doesn't concatenate...and never executes the else statement.

Lines missing in python

I am writing a code in python where I am removing all the text after a specific word but in output lines are missing. I have a text file in unicode which have 3 lines:
my name is test1
my name is
my name is test 2
What I want is to remove text after word "test" so I could get the output as below
my name is test
my name is
my name is test
I have written a code but it does the task but also removes the second line "my name is"
My code is below
txt = ""
with open(r"test.txt", 'r') as fp:
for line in fp.readlines():
splitStr = "test"
index = line.find(splitStr)
if index > 0:
txt += line[:index + len(splitStr)] + "\n"
with open(r"test.txt", "w") as fp:
fp.write(txt)

It looks like if there is no keyword found the index become -1.
So you are avoiding the lines w/o keyword.
I would modify your if by adding the condition as follows:
txt = ""
with open(r"test.txt", 'r') as fp:
for line in fp.readlines():
splitStr = "test"
index = line.find(splitStr)
if index > 0:
txt += line[:index + len(splitStr)] + "\n"
elif index < 0:
txt += line
with open(r"test.txt", "w") as fp:
fp.write(txt)
No need to add \n because the line already contains it.

Your code does not append the line if the splitStr is not defined.
txt = ""
with open(r"test.txt", 'r') as fp:
for line in fp.readlines():
splitStr = "test"
index = line.find(splitStr)
if index != -1:
txt += line[:index + len(splitStr)] + "\n"
else:
txt += line
with open(r"test.txt", "w") as fp:
fp.write(txt)

In my solution I simulate the input file via io.StringIO. Compared to your code my solution remove the else branch and only use one += operater. Also splitStr is set only one time and not on each iteration. This makes the code more clear and reduces possible errore sources.
import io
# simulates a file for this example
the_file = io.StringIO("""my name is test1
my name is
my name is test 2""")
txt = ""
splitStr = "test"
with the_file as fp:
# each line
for line in fp.readlines():
# cut somoething?
if splitStr in line:
# find index
index = line.find(splitStr)
# cut after 'splitStr' and add newline
line = line[:index + len(splitStr)] + "\n"
# append line to output
txt += line
print(txt)
When handling with files in Python 3 it is recommended to use pathlib for that like this.
import pathlib
file_path = pathlib.Path("test.txt")
# read from wile
with file_path.open('r') as fp:
# do something
# write back to the file
with file_path.open('w') as fp:
# do something

Suggestion:
for line in fp.readlines():
i = line.find('test')
if i != -1:
line = line[:i]

Reading a Python File to EOF while performing if statments

I am working on creating a program to concatenate rows within a file. Each file has a header, datarows labeled DAT001 to DAT113 and a trailer. Each line of concatenated rows will have DAT001 to DAT100 and 102-113 is optional. I need to print the header, concatenating DAT001-113 and when the file finds a row with DAT001 I need to start a new line concatenating DAT001-113 again. After that is all done, I will print the trailer. I have an IF statement started but it only writes the header and skips all other logic. I apologize that this is very basic - but I am struggling with reading rows over and over again without knowing how long the file might be.
I have tried the below code but it won't read or print after the header.
import pandas as pd
destinationFile = "./destination-file.csv"
sourceFile = "./TEST.txt"
header = "RHR"
data = "DPSPOS"
beg_data = "DAT001"
data2 = "DAT002"
data3 = "DAT003"
data4 = "DAT004"
data5 = "DAT005"
data6 = "DAT006"
data7 = "DAT007"
data8 = "DAT008"
data100 = "DAT100"
data101 = "DAT101"
data102 = "DAT102"
data103 = "DAT103"
data104 = "DAT104"
data105 = "DAT105"
data106 = "DAT106"
data107 = "DAT107"
data108 = "DAT108"
data109 = "DAT109"
data110 = "DAT110"
data111 = "DAT111"
data112 = "DAT112"
data113 = "DAT113"
req_data = ''
opt101 = ''
opt102 = ''
with open(sourceFile) as Tst:
for line in Tst.read().split("\n"):
if header in line:
with open(destinationFile, "w+") as dst:
dst.write(line)
elif data in line:
if beg_data in line:
req_data = line+line+line+line+line+line+line+line+line
if data101 in line:
opt101 = line
if data102 in line:
opt102 = line
new_line = pd.concat(req_data,opt101,opt102)
with open(destinationFile, "w+") as dst:
dst.write(new_line)
else:
if trailer in line:
with open(destinationFile, "w+") as dst:
dst.write(line)

Just open the output file once for the whole loop, not every time through the loop.
Check whether the line begins with DAT101. If it does, write the trailer to the current line and start a new line by printing the header.
Then for every line that begins with DAT, write it to the file in the current line.
first_line = True
with open(sourceFile) as Tst, open(destinationFile, "w+") as dst:
for line in Tst.read().split("\n"):
# start a new line when reading DAT101
if line.startswith(beg_data):
if not first_line: # need to end the current line
dst.write(trailer + '\n')
first_line = False
dst.write(header)
# copy all the lines that begin with `DAT`
if line.startswith('DAT'):
dst.write(line)
# end the last line
dst.write(trailer + '\n')

See if the following code helps make progress. It was not tested because no
Minimum Runnable Example is provided.
with open(destinationFile, "a") as dst:
# The above will keep the file open until after all the indented code runs
with open(sourceFile) as Tst:
# The above will keep the file open until after all the indented code runs
for line in Tst.read().split("\n"):
if header in line:
dst.write(line)
elif data in line:
if beg_data in line:
req_data = line + line + line + line + line + line + line + line + line
if data101 in line:
opt101 = line
if data102 in line:
opt102 = line
new_line = pd.concat(req_data, opt101, opt102)
dst.write(new_line)
else:
if trailer in line:
dst.write(line)
# With is a context manager which will automatically close the files.

Python reading data from input file

I want to read specific data from an input file. How can I read it?
For example my file has data like:
this is my first line
this is my second line.
So I just want to read first from the first line and secon from the second line.

Try the following code for your needs but please read the comments above.
# ----------------------------------------
# open text file and write reduced lines
# ----------------------------------------
#this is my first line
#this is my second line.
pathnameIn = "D:/_working"
filenameIn = "foobar.txt"
pathIn = pathnameIn + "/" + filenameIn
pathnameOut = "D:/_working"
filenameOut = "foobar_reduced.txt"
pathOut = pathnameOut + "/" + filenameOut
fileIn = open(pathIn,'r')
fileOut = open(pathOut,'w')
print(fileIn)
print(fileOut)
i = 0
# Save all reduced lines to a file.
for lineIn in fileIn.readlines():
i += 1 # number of lines read
lineOut = lineIn[11:16]
fileOut.writelines(lineOut +"\n")
print("*********************************")
print("gelesene Zeilen: " + str(i))
print("*********************************")
fileIn.close()
fileOut.close()