I want to save the values of different variables in a CSV file. But it prints another header every time. I don't want this, I am attaching my CSV file snapshot for your understanding. Output csv
file_orimg = open('Org_image.csv', 'a', newline='')
writer_orimg = csv.writer(file_orimg, delimiter='\t',lineterminator='\n',)
writer_orimg.writerow(["Image Name", "epsilon","MSE", "SSIM", "Prediction", "Probability"])
for i in images:
writer_orimg.writerow([i, epsilon, mse, ssim, clean_pred, clean_prob, label_idx.item()])
Try not to use writerow to write your headers. You can look at DictWriter in the CSV python module, writing headers and writing rows will be done more efficiently!
list_of_headers = ['No.', 'Image Name', 'Epsilon']
dictionary_content = {'No.': 1, 'Image Name': 'image_123', 'Epsilon': 'what?'}
w = csv.DictWriter(my_csvfile, fieldnames= list_of_headers)
w.writeheader()
w.writerow(dictionay_content)
Hope this helps, let me know if there is any rectification to be made!
Edit: Answering 'where & when should writeheader be done'
I use the os python module to determine whether the file exists, if not I'm going to create one!
if os.path.isfile(filename):
with open(filename, 'a', newline='') as my_file:
w = csv.DictWriter(my_file, fieldnames= list_of_headers)
w.writerow(dictionay_content)
else:
with open(filename, 'w', newline='') as my_file:
w = csv.DictWriter(my_file, fieldnames= list_of_headers)
w.writeheader()
w.writerow(dictionay_content)
!!! Take note of the 'a' which is to append whereas 'w' means to write. Hence appending with new rows of your data from where it left off/last occupied.
Related
I'm writing data from a PDF to a CSV. The CSV needs to have one column, with each word on a separate row.
The code below writes each word on a separate row, but also puts each letter in a separate cell.
with open('annualreport.csv', 'w', encoding='utf-8') as f:
write = csv.writer(f)
for i in keywords:
write.writerow(i)
I have also attempted the following, which writes all the words to one row, with each word in a separate column:
with open('annualreport.csv', 'w', encoding='utf-8') as f:
write = csv.writer(f)
write.writerow(keywords)
As far as I know, writerow expects an array. Thus a word is treated as an array with the individual letters -> each letter is written into a new cell.
Putting the value into a single array should fix the problem:
with open('annualreport.csv', 'w', encoding='utf-8') as f:
write = csv.writer(f)
for i in keywords:
write.writerow( [ i ] ) # <-- before: write.writerow(i)
import csv
# data to be written row-wise in csv fil
data = [['test'], [try], ['goal']]
# opening the csv file in 'w+' mode
file = open('output.csv', 'w+', newline ='')
# writing the data into the file
with file:
write = csv.writer(file)
write.writerows(data)
i'm rather new to programming and am trying to reduce the time taken to write my data into a file, and i found that the writing part is the main issue.
The following is part of my code for a machine learning program:
filename="data.csv"
f=open(filename,"w")
headers="row,open\n"
f.write(headers)
for i in range (0,55970):
score=rf.predict(edit[i].reshape(1, -1))
score=str(score).replace('[','').replace(']','')
f.write(str(i) +","+ score +"\n")
f.close()
I understand that I should be writing the data only after i have gotten all of it, but i am not sure how to go about doing it - given that i only know f.write(). Do i make a function for my prediction and return score, then create a list to store all the scores and write it in? (if that is possible)
[Edit]
score=rf.predict(edit)
with open('data.csv', 'w',newline='') as f:
writer = csv.writer(f)
writer.writerow(['row', 'open'])
for i in range(55970):
writer.writerow([i,str(score[i])])
^ added based on new suggestion. Found that i should just do the predict and then write the rows which improved the time taken significantly!
Thank you for your help!!
The CSV module is a better tool for this. More specifically, writerows() is what you are looking for.
https://docs.python.org/3/library/csv.html#csv.csvwriter.writerows
Here is an example from the docs:
import csv
with open('some.csv', 'w', newline='') as f:
writer = csv.writer(f)
writer.writerows(someiterable)
import csv
with open('data.csv', 'w') as csv_file:
writer = csv.writer(csv_file)
writer.writerow(['row_id', 'open_flag'])
for i in range(55970):
score = str(rf.predict(edit[i].reshape(1, -1)))
score.replace('[', '').replace(']', '')
writer.writerow([i, score])
I have developed a script that produces a CSV file. On inspection of the file, some cell's are being interpreted not the way I want..
E.g In my list in python, values that are '02e4' are being automatically formatted to be 2.00E+04.
table = [['aa02', 'fb4a82', '0a0009'], ['02e4, '452ca2', '0b0004']]
ofile = open('test.csv', 'wb')
for i in range(0, len(table)):
for j in range(0, len(table[i]):
ofile.write(table[i][j] + ",")
ofile.write("\n")
This gives me:
aa02 fb4a82 0a0009
2.00E+04 452ca2 0b0004
I've tried using the csv.writer instead where writer = csv.writer(ofile, ...)
and giving attributes from the lib (e.g csv.QUOTE_ALL)... but its the same output as before..
Is there a way using the CSV lib to automatically format all my values as strings before it's written?
Or is this not possible?
Thanks
Try setting the quoting parameter in your csv writer to csv.QUOTE_ALL.
See the doc for more info:
import csv
with open('myfile.csv', 'wb') as csvfile:
wtr = csv.writer(csvfile, quoting=csv.QUOTE_ALL)
wtr.writerow(...)
Although it sounds like the problem might lie with your csv viewer. Excel has a rather annoying habit of auto-formatting data like you describe.
If you want the '02e4' to show up in excel as "02e4" then annoyingly you have to write a csv with triple-double quotes: """02e4""". I don't know of a way to do this with the csv writer because it limits your quote character to a character. However, you can do something similar to your original attempt:
table = [['aa02', 'fb4a82', '0a0009'], ['02e4', '452ca2', '0b0004']]
ofile = open('test.csv', 'wb')
for i in range(0, len(table)):
for j in range(len(table[i])):
ofile.write('"""%s""",'%table[i][j])
ofile.write("\n")
If opened in a text editor your csv file will read:
"""aa02""","""fb4a82""","""0a0009""",
"""02e4""","""452ca2""","""0b0004""",
This produces the following result in Excel:
If you wanted to use any single character quotation you could use the csv module like so:
import csv
table = [['aa02', 'fb4a82', '0a0009'], ['02e4', '452ca2', '0b0004']]
ofile = open('test.csv', 'wb')
writer = csv.writer(ofile, delimiter=',', quotechar='|',quoting=csv.QUOTE_ALL)
for i in range(len(table)):
writer.writerow(table[i])
The output in the text editor will be:
|aa02|,|fb4a82|,|0a0009|
|02e4|,|452ca2|,|0b0004|
and Excel will show:
I want to write data onto several rows in a csv file. With this code I am only getting a single line written onto file. Need help.
for i in range(1,10):
with open("output.csv",'wb') as f:
writer = csv.writer(f, dialect='excel')
writer.writerow([value1, value2, value3])
You are encountering the error because it is re-writing a csv for every iteration in your loop. You should move the with open() statement outside of your loop block.
Try opening the file only once and then doing the loop:
with open("output.csv",'wb') as f:
writer = csv.writer(f, dialect='excel')
for item in list_A:
writer.writerow([value1, value2, value3])
You would need to use the w option in open as follows:
import csv
list_A = [0,0,0,0,0,0,0,0,0,0,0]
with open("output.csv",'w') as f:
writer = csv.writer(f)
for item in list_A:
writer.writerow([1,0,0,0])
My dictionary is as follows:
my_d ={'2010-01-02': [0.696083, 0.617865], '2010-01-01': [0.697253, 0.618224], '2010-01-03': [0.696083, 0.617865]}
and i wish to output like the following:
DATE,EUR,GBP
2010-01-02,0.696083,0.617865
2010-01-03,0.697253,0.618224
2010-01-03,0.696083,0.617865
i have tried this:
my_list = ["date", "EUR", "GBP"]
with open(fname, 'w',newline = '') as my_csv:
csv_writer = csv.writer(my_csv, delimiter=',')
csv_writer.writerow(my_list)
csv_writer.writerows(my_d)
but the csv file looks nothing like that. Please also be aware further up in my code the user has the ability to choose how many values each key has, so in this example it has two, in another it may have three this would also mean my_list would increase in size, but i believe the first row will already cater for that, i just need help with the rest of the rows, i need it to work for any amount of keys.
You need to loop over your dictionary items and then write to your file,also its better to use csv module for dealing with csv files :
my_list = ["date", "EUR", "GBP"]
import csv
my_d ={'2010-01-02': [0.696083, 0.617865], '2010-01-01': [0.697253, 0.618224], '2010-01-03': [0.696083, 0.617865]}
with open('fname.csv', 'wb') as csvfile:
spamwriter = csv.writer(csvfile, delimiter=',')
spamwriter.writerow(my_list)
for mydate in sorted(my_d.keys()):
spamwriter.writerow([mydate]+my_d[mydate])
Note the absence of newline='' when opening the file. This allows the file to have "normal" newline characters (usually '\n' or '\r\n')
Note that as you are in python 3.x you can open the file with w mode!