We Keep Coding

How does the last part of this Python code do what it does?

Essential what this code does is it takes a .txt file, replaces " " with "|"
then after it replaces it, it then goes back in and deletes any rows that start with a "|"
Im having trouble understanding what the last part of this code does.
I understand everything up until the:
output = [line for line in txt if not line.startswith("|")]
f.seek(0)
f.write("".join(output))
f.truncate()
everything before this code ^^ i understand but im not sure how this code does what its doing.
--------this is the full code----------
# This imports the correct package
import datetime
# this creates "today" as the variable that holds todays date
today = datetime.date.today().strftime("%Y%m%d")
# Read the file
with open(r'\\mlgserver04\mlgshare\DataTransfer&AuditCompliance\ARSI Calls\CallLog_ARSI_' + today + '.txt', 'r') as file:
filedata = file.read()
# Replace the target string
filedata = filedata.replace(' ', '|')
# Write the file out again
with open(r'\\mlgserver04\mlgshare\DataTransfer&AuditCompliance\ARSI Calls\CallLog_ARSI_' + today + '.txt', 'w') as file:
file.write(filedata)
# opens the file
with open(r'\\mlgserver04\mlgshare\DataTransfer&AuditCompliance\ARSI Calls\CallLog_ARSI_' + today + '.txt','r+') as f:
txt = f.readlines()
output = [line for line in txt if not line.startswith("|")]
f.seek(0)
f.write("".join(output))
f.truncate()

f.read() returns a blob of data (string or bytes)
On the other hand, f.readlines() returns a LIST of strings or bytes.
output = [line for line in txt if not line.startswith("|")]
This is the same thing as saying
output = []
for line in txt:
if not line.startswith("|"):
output.append(line)
So, make a NEW list of strings, consisting of only the ones that we want to keep.
"".join(output)
Join takes an iterable of strings, and adds them together with the delimiter (in this case, it was ""). so output[0] + "" + output[1] and so on.
Finally, write that final string back to the file.

How to use use Python to import (.tex file), create a new (.tex file) and append the new (.tex file) from the imported (.tex file)

I have several (1000+) .tex files which goes something like this:
File1.tex:
\documentclass[15pt]{article}
\begin{document}
Question1:
$f(x)=sin(x)$\\
Question2:
$f(x)=tan(x)$
\end{document}
File2.tex is similar in structure:
\documentclass[15pt]{article}
\begin{document}
Question1:
$f(x)=cos(x)$\\
Question2:
$f(x)=sec(x)$\\
Question3:
$f(x)=cot(x)$
\end{document}
What I would like to do is write a Python script that allows me to select question 1 from file1.tex and question 3 from file2.tex and compile a new file3.tex file (or PDF) with the following format:
\documentclass[15pt]{article}
\begin{document}
Question1:
$f(x)=sin(x)$\\
Question2:
$f(x)=cot(x)$
\end{document}
PS- I don't mind if I can carry out this type of work on LaTex. I just thought with Python I can eventually create a GUI.
So far I've managed to read/append a .tex file by manually typing what I want rather than creating some sort of a system that allows me to "copy" specific section of a .tex file or files into another another .tex file.

I used exactly what you had for file1 and file2.tex. I left comments throughout rather than explain step by step.
PreProcess
The preprocess involves creating an xlsx file which will have all of the names of the tex file in the first column.
import os
import xlsxwriter
workbook = xlsxwriter.Workbook('Filenames.xlsx')
worksheet = workbook.add_worksheet("FileNames")
worksheet.write(0, 0, "NameCol")
path = os.getcwd() # get path to directory
filecount = 1
for file in os.listdir(path): # for loop over files in directory
if file.split('.')[-1] == 'tex': # only open latex files
worksheet.write(filecount, 0, file)
filecount += 1
workbook.close()
Select Problems
Now you go through an list to the right like I have what problems you want out of the file.
PostProcess
Now we can run through our xlsx file and create a new latex file from it.
import pandas as pd
import math
import os
# get data
allfileqs = []
df = pd.read_excel('Filenames.xlsx')
for row in df.iterrows():
tempqs = []
for i in range(len(row[1].values) - 1):
if math.isnan(row[1].values[i + 1]):
continue
else:
tempqs.append(int(row[1].values[i + 1]))
allfileqs.append(tempqs)
print(allfileqs)
allfileqs = [["Question" + str(allfileqs[i][j]) + ':' for j in range(len(allfileqs[i]))] for i in range(len(allfileqs))]
starttex = [r'\documentclass[15pt]{article}', r'\begin{document}']
endtex = [r'\end{document}']
alloflines = []
path = os.getcwd() # get path to directory
for file in os.listdir(path): # for loop over files in directory
if file.split('.')[-1] == 'tex': # only open latex files
lf = open(file, 'r')
lines = lf.readlines()
# remove all new lines, each item is on new line we know
filt_lines = [lines[i].replace('\n', '') for i in range(len(lines)) if lines[i] != '\n']
alloflines.append(filt_lines) # save data for later
lf.close() # close file
# good now we have filtered lines
newfile = []
questcount = 1
for i in range(len(alloflines)):
for j in range(len(alloflines[i])):
if alloflines[i][j] in allfileqs[i]:
newfile.append("Question" + str(questcount) + ":")
newfile.append(alloflines[i][j + 1])
questcount += 1
# okay cool we have beg, middle (newfile) and end of tex
newest = open('file3.tex', 'w') # open as write only
starter = '\n\n'.join(starttex) + '\n' + '\n\n'.join(newfile) + '\n\n' + endtex[0]
for i in range(len(starter)):
newest.write(starter[i])
newest.close()

Writing comparison to .txt file when comparing sql files

So I'm trying to compare multiple tables using a Python script. The actual comparison is working, tested with print statements, but the write to a .txt file is not. I believe I might have an error in my syntax, though being relatively new to Python, I can't find it.
for num in range(0, 4): #runs through the database array and compares the files in each folder
comp_var = directory + server_number[size] + databases[num]
for file in os.listdir(comp_var):
for num1 in os.listdir(master + databases[num]):
var = master + databases[num] + "\\" + os.listdir(master + databases[num])[size]
for line in open(var, 'r'):
for line2 in open(comp_var + "\\" + file, 'r'):
same = set(line).intersection(line2)
print(same)
same.discard('\n')
with open('results.txt', 'w') as file_out:
for line1 in same:
file_out.write(line1)
size = size + 1
comp_var = directory + server_number[size] + databases[num]
size = 0

Your problem is that you create a new file every time you call open. You should use 'a' to append to a file, which is probably what you want.

You are overwriting the results.txt.
with open('results.txt', 'w') as file_out:
change it to:
with open('results.txt', 'a') as file_out:
from Python documentation:
'w' for only writing (an existing file with the same name will be erased), and 'a' opens the file for appending; any data written to the file is automatically added to the end.

Why loop overwriting my file instead of writing after text?

i = 1 # keep track of file number
directory = '/some/directory/'
for i in range(1, 5170): #number of files in directory
filename = directory + 'D' + str(i) + '.txt'
input = open(filename)
output = open('output.txt', 'w')
input.readline() #ignore first line
for g in range(0, 7): #write next seven lines to output.txt
output.write(input.readline())
output.write('\n') #add newline to avoid mess
output.close()
input.close()
i = i + 1
I have this code, and i am trying to get one file and rewrite it to output.txt, but when i want to attach next file, my code overwrite older file that has been attached. In result when code is complete i have something like this:
dataA[5169]=26
dataB[5169]=0
dataC[5169]=y
dataD[5169]='something'
dataE[5169]=x
data_date[5169]=2012.06.02
Instead of datas ranging from files 0 to 5169. Any tips how to fix it?

You probably want to open output.txt before your for loop (and close it after). As it is written, you overwrite the file output.txt everytime you open it. (an alternative would be to open for appending: output = open('output.txt','a'), but that's definitely not the best way to do it here ...
Of course, these days it's better to use a context manager (with statement):
i = 1 # keep track of file number <-- This line is useless in the code you posted
directory = '/some/directory/' #<-- os.path.join is better for this stuff.
with open('output.txt','w') as output:
for i in range(1, 5170): #number of files in directory
filename = directory + 'D' + str(i) + '.txt'
with open(filename) as input:
input.readline() #ignore first line
for g in range(0, 7): #write next seven lines to output.txt
output.write(input.readline())
output.write('\n') #add newline to avoid mess
i = i + 1 #<---also useless line in the code you posted

Your issue is that you open in write mode. To append to file you want to use append. See here.

Writing csv header removes data from numpy array written below

I'm trying to export data to a csv file. It should contain a header (from datastack) and restacked arrays with my data (from datastack). One line in datastack has the same length as dataset. The code below works but it removes parts of the first line from datastack. Any ideas why that could be?
s = ','.join(itertools.chain(dataset)) + '\n'
newfile = 'export.csv'
f = open(newfile,'w')
f.write(s)
numpy.savetxt(newfile, (numpy.transpose(datastack)), delimiter=', ')
f.close()

You a file with the filename 'export.csv' twice, once when you call open() and once when you call numpy.savetxt(). Thus, there are two open file handles competing for the same filename. If you pass the file handle rather than the file name to numpy.savetxt() you avoid this race condition:
s = ','.join(itertools.chain(dataset)) + '\n'
newfile = 'export.csv'
f = open(newfile,'w')
f.write(s)
numpy.savetxt(f, (numpy.transpose(datastack)), delimiter=', ')
f.close()