So I'm trying to compare multiple tables using a Python script. The actual comparison is working, tested with print statements, but the write to a .txt file is not. I believe I might have an error in my syntax, though being relatively new to Python, I can't find it.
for num in range(0, 4): #runs through the database array and compares the files in each folder
comp_var = directory + server_number[size] + databases[num]
for file in os.listdir(comp_var):
for num1 in os.listdir(master + databases[num]):
var = master + databases[num] + "\\" + os.listdir(master + databases[num])[size]
for line in open(var, 'r'):
for line2 in open(comp_var + "\\" + file, 'r'):
same = set(line).intersection(line2)
print(same)
same.discard('\n')
with open('results.txt', 'w') as file_out:
for line1 in same:
file_out.write(line1)
size = size + 1
comp_var = directory + server_number[size] + databases[num]
size = 0
Your problem is that you create a new file every time you call open. You should use 'a' to append to a file, which is probably what you want.
You are overwriting the results.txt.
with open('results.txt', 'w') as file_out:
change it to:
with open('results.txt', 'a') as file_out:
from Python documentation:
'w' for only writing (an existing file with the same name will be erased), and 'a' opens the file for appending; any data written to the file is automatically added to the end.
Related
I am trying to rename a set of 15,000 txt-files. In seperate txt-file I do have a list of the old names of these files and the new names I want them have. But not all of the 15,000 txt-files are listed in the name-list. That is where the problem starts. How do I change my code that it ignores/skips files that are not listed in the name list or those that are duplicates? The codes works fine until it reaches one of these file. Any suggestions? Thanks! This is the code I have so far:
import os
with open("rename3.txt") as fd:
for line in fd:
line = line.strip()
if len(line) == 0: continue
old, new = line.strip().split(",", 1)
os.rename(old.strip() + ".txt", new.strip() + ".txt")
import os
with open("rename3.txt") as fd:
for line in fd if not line.strip():
old, new = line.strip().split(",", 1)
if os.path.exists(old):
os.rename(old.strip() + ".txt", new.strip() + ".txt")
My code is about getting sensor values and appending them in csv file. This csv file gets too long and I want to compress it in zip file after 1000 lines. For this I want my csv file to be saved with filename as Logger1 for the first time and after this logger1 file has crossed 1000 lines it should be converted into zip file, and removed while 'i' gets incremented by 1.Now after this, the further data should be saved in the csv file with filename as logger2. How do I add 'i' along with the file name. I tried using str(i) but it didn't work as the filename is in the inverted commas itself.
export_csv = x.to_csv (r'/home/pi/Logger + str(i).csv',
index = None, mode='a', header=False)
input_file = open("Logger + str(i).csv","r+")
reader_file = csv.reader(input_file)
l = len(list(reader_file))
if (l > 1000) :
jungle_zip = zipfile.ZipFile('Logger + str(i).zip',
'w')
jungle_zip.write('Logger + str(i).csv',
compress_type=zipfile.ZIP_DEFLATED)
jungle_zip.close()
os.remove("Logger + str(i).csv")
i +=1
Try the following
'Logger' + str(i) + '.zip'
instead of
'Logger + str(i).zip'
bceause str(i) is not considered a function in the string
EDIT: The solution was simply changing how I was opening the file (thanks Primusa), not how I was replacing the information.
I am attempting to overwrite parts of a file, but my code does not work. When it runs, it completely erases everything in the file, leaving nothing behind.
Here is my code:
for fname in store:
with open(fname + ".txt", "w+") as f:
for line in f:
c = store[fname]
x = (c-1)%len(line.split(","))
y = m.floor((c-1)/len(line.split(",")))
for n in range(y):
f.readline()
ldata = line.strip("\n").split(",")
for i in range(len(ldata)):
ldata[i] = int(ldata[i])
if w == None:
ldata[x] += 1
elif w == True:
ldata[x] += 2
elif w == False:
ldata[x] -= 1
#s = line.find("\n") - 1
#f.truncate(s)
f.write(str(ldata) + "\n")
Key:
fname: a string variable corresponding to a file name without the file type.
store: a dictionary containing file name keys as string and integer values.
f: a file containing multiple lines of integer lists.
c: an integer variable used to determine the integer to be accessed.
x and y: variables with values set as the column and row (respectively) of the integer to be accessed in the file.
w: a variable stored as either a boolean or None, used to determine whether the accessed integer should increase or decrease, and by how much.
s: a currently unused integer variable used to truncate part of the file.
Example of use:
Say I have a file Foo.txt, which has the following information stored in it:
0,2,3,7
11,3,6,4
I want to increase the "6" value by 2, so I run the code with w set to True and add "Foo" : 7 to store, since "6" is the 7th number in the file (regardless of list length).
What should happen:
Foo.txt is modified and now contains:
0,2,3,7
11,3,8,4
What actually happens:
Foo.txt still exists, but now contains:
Which is to say it is empty.
What is wrong with my code? Am I handling the file incorrectly, the variable calculations, syntax, or something else entirely?
with open(fname + ".txt", "w+") as f:
Opening a file in mode "w+" truncates the file, which means it deletes everything inside of it. All of the operations you do after this statement are on an empty file.
What I would suggest would be opening the file in read mode:
with open(fname + ".txt", "r") as f:
Loading the file into memory, making your changes, and then opening the file in "w+" mode and putting the file back down.
Lets do this on the foo example:
with open("foo.txt", 'r') as f: #open in read mode
a = f.read().split(',') #breaking it apart so i can locate the 6th number easier
a[6] = str(int(a[6]) + 2) #make my modifications
a = ','.join(a) #convert it back to string
with open("foo.txt", 'w+') as f: #open in write and delete everything
f.write(a) #write my modified version of the file
Note this is a very basic example so it doesn't take into account the newlines.
i = 1 # keep track of file number
directory = '/some/directory/'
for i in range(1, 5170): #number of files in directory
filename = directory + 'D' + str(i) + '.txt'
input = open(filename)
output = open('output.txt', 'w')
input.readline() #ignore first line
for g in range(0, 7): #write next seven lines to output.txt
output.write(input.readline())
output.write('\n') #add newline to avoid mess
output.close()
input.close()
i = i + 1
I have this code, and i am trying to get one file and rewrite it to output.txt, but when i want to attach next file, my code overwrite older file that has been attached. In result when code is complete i have something like this:
dataA[5169]=26
dataB[5169]=0
dataC[5169]=y
dataD[5169]='something'
dataE[5169]=x
data_date[5169]=2012.06.02
Instead of datas ranging from files 0 to 5169. Any tips how to fix it?
You probably want to open output.txt before your for loop (and close it after). As it is written, you overwrite the file output.txt everytime you open it. (an alternative would be to open for appending: output = open('output.txt','a'), but that's definitely not the best way to do it here ...
Of course, these days it's better to use a context manager (with statement):
i = 1 # keep track of file number <-- This line is useless in the code you posted
directory = '/some/directory/' #<-- os.path.join is better for this stuff.
with open('output.txt','w') as output:
for i in range(1, 5170): #number of files in directory
filename = directory + 'D' + str(i) + '.txt'
with open(filename) as input:
input.readline() #ignore first line
for g in range(0, 7): #write next seven lines to output.txt
output.write(input.readline())
output.write('\n') #add newline to avoid mess
i = i + 1 #<---also useless line in the code you posted
Your issue is that you open in write mode. To append to file you want to use append. See here.
I'm trying to export data to a csv file. It should contain a header (from datastack) and restacked arrays with my data (from datastack). One line in datastack has the same length as dataset. The code below works but it removes parts of the first line from datastack. Any ideas why that could be?
s = ','.join(itertools.chain(dataset)) + '\n'
newfile = 'export.csv'
f = open(newfile,'w')
f.write(s)
numpy.savetxt(newfile, (numpy.transpose(datastack)), delimiter=', ')
f.close()
You a file with the filename 'export.csv' twice, once when you call open() and once when you call numpy.savetxt(). Thus, there are two open file handles competing for the same filename. If you pass the file handle rather than the file name to numpy.savetxt() you avoid this race condition:
s = ','.join(itertools.chain(dataset)) + '\n'
newfile = 'export.csv'
f = open(newfile,'w')
f.write(s)
numpy.savetxt(f, (numpy.transpose(datastack)), delimiter=', ')
f.close()