i am having troubles on this python error.
I want to save changing variables to an csv file, however while the code runs again with an different variable it overwrites the previous one. I do not have the variables predetermined, they are generated while the code runs, so every time the loop will loop the program there will a different email passed.
Here is my code:
import csv
def hello(hme):
header = ['email']
data = [hme]
with open('countries.csv', 'w', encoding='UTF8', newline='') as f:
writer = csv.writer(f)
writer.writerow(header)
writer.writerows(data)
hello(["test#icloud.com"])
Thank you!
you should open the file as append, instead of write:
'a' instead of 'w'
import csv
def hello(hme):
header = ['email']
data = [hme]
with open('countries.csv', 'a', encoding='UTF8', newline='') as f:
writer = csv.writer(f)
writer.writerow(header)
writer.writerows(data)
hello(["test#icloud.com"])
Just replace 'w' by 'a' where 'w' writes in file (override) while 'a' appends the file whenever you write in it.
with open('countries.csv', 'a', encoding='UTF8', newline='') as f:
For the header "email" just write it before you add the loop of emails to do not duplicate it
Read the file contents first; add the new data; write the data to a file.
def hello(hme):
try:
with open('countries.csv', encoding='UTF8', newline='') as f:
stuff = list(csv.reader(f))
except FileNotFoundError:
# this must be the first time the function was called
stuff = [['email']]
stuff.append([hme])
with open('countries.csv', 'w', encoding='UTF8', newline='') as f:
writer = csv.writer(f)
writer.writerows(stuff)
If your file really only has one column you don't really need to use the csv module. Just append the new line to the file.
# assumes header is present
def hello(hme):
with open('countries.csv', 'a', encoding='UTF8') as f:
f.write(hme + '\n')
Related
how can I put my first row of data in the csv under the header and not in the same row as header?
This is the results.
And down here is my coding.
import os
# ...
filename = 'C:/Desktop/GPS_Trial/Trial6/' + str(d1) + '_' + str(file_counter) +'.csv'
#check whether the file exist or not
rows_to_be_written = []
if not os.path.exists(filename):
rows_to_be_written.append(header1)
rows_to_be_written.append(header2)
rows_to_be_written.append(header3)
rows_to_be_written.append(gps)
rows_to_be_written.append(gps2)
rows_to_be_written.append(gps3)
#write the data into csv
with open(filename, 'a', newline='', encoding='UTF8') as f:
writer = csv.writer(f, delimiter=',')
writer.writerow(rows_to_be_written)
print(gps, gps2, gps3)
You write header with values in one row if it file not exists.
You should write it separately
rows_to_be_written = []
header = None
if not os.path.exists(filename):
header = [header1, header2, header3]
rows_to_be_written.append(gps)
rows_to_be_written.append(gps2)
rows_to_be_written.append(gps3)
# write the data into csv
with open(filename, 'a', newline='', encoding='UTF8') as f:
writer = csv.writer(f, delimiter=',')
if header:
writer.writerow(header)
writer.writerow(rows_to_be_written)
print(gps, gps2, gps3)
Also may be you tried write rows, but you write only one row with header in it. Then change code like this
rows_to_be_written = []
if not os.path.exists(filename):
rows_to_be_written.append([header1, header2, header3])
rows_to_be_written.append([gps, gps2, gps3])
# write the data into csv
with open(filename, 'a', newline='', encoding='UTF8') as f:
writer = csv.writer(f, delimiter=',')
for row in rows_to_be_written:
writer.writerow(row)
print(gps, gps2, gps3)
You need to add the headings separately, and only if they are not there already:
# check whether the file exist or not
if not os.path.exists(filename):
headings = [header1, header2, header3]
else:
headings = None
rows_to_be_written = [gps, gps2, gps3]
# write the data into csv
with open(filename, 'a', newline='', encoding='UTF8') as f:
writer = csv.writer(f)
# Write headings if exist
if headings != None:
writer.writerow(headings)
# Write rows
writer.writerow(rows_to_be_written)
print(gps, gps2, gps3)
I suggest you consider this approach
# Open file to see if there are headings
with open(filename, "r") as f:
try:
has_headings = csv.Sniffer().has_header(f.read(1024))
except csv.Error:
# The file seems to be empty
has_headings = False
# Open to write. In append mode ("a")
with open(filename, "a") as f:
writer = csv.writer(f)
if has_headings:
# Write the rows at the top
writer.writerow(headings_list)
# Use writerows if youe have a 2D list, else use a for loop of writer.writerow
writer.writerows(lists_of_rows)
I currently have code that is writing to a csv file, however, it keesp replacing the first row of the file and overwriting it instead of writing of it.
NOTE: I have this function inside a for loop inside main().
def write_csv_report(filename, region, data, current, server1, server2=False):
if not os.path.exists(os.path.dirname(filename)):
try:
dir = os.makedirs(os.path.dirname(filename))
header = ['region','old_dns_server', 'proposed_dns_server1', 'proposed_dns_server2']
data = [region, current, server1, server2]
with open(filename, 'a') as file:
writer = csv.writer(file)
writer.writerow(header)
writer.writerow(data)
except OSError as exc:
if exc.errno != errno.EEXIST:
raise
When I run this function in my script, it writes to the file but only one entry is added.
Is there any suggestions or solution where I can add multiple entries? I have looked at other sage questions and googled but it looks like I already have the necessary changes needed to achieve this outcome, not sure what I'm missing.
For csv.writer (and csv.reader), you need to open the file with newline=''.
You also write the header every time, which is a bit odd.
Here's a minimal example that does more-or-less what you want:
import csv
header = ['region','old_dns_server', 'proposed_dns_server1', 'proposed_dns_server2']
data1 = ['foo', 'bar', 'bar', 'quux']
data2 = ['foo1', 'bar1', 'bar1', 'quux2']
filename = 'test.csv'
with open(filename, 'w', newline='') as file:
writer = csv.writer(file)
writer.writerow(header)
writer.writerow(data1)
with open(filename, 'a', newline='') as file:
writer = csv.writer(file)
writer.writerow(data2)
giving:
region,old_dns_server,proposed_dns_server1,proposed_dns_server2
foo,bar,bar,quux
foo1,bar1,bar1,quux2
I'm new to Python and I'm trying to scrape some data and save them in a csv.
I'm trying to loop a csv with a list of URLs, read the data from each URL and write that information in another csv file
The following code is writing roughly half of the data in the cvs but is printing everything fine while it's writing
df_link = pd.read_csv('url_list')
with open('url_list.csv', newline='') as urls, open('output.csv', 'w', newline='') as output:
csv_urls = csv.reader(urls)
csv_output = csv.writer(output)
csv_output.writerow(['details','date'])
for link in df_link.iterrows():
url = link[1]['url']
browser.get(url)
soup = BeautifulSoup(browser.page_source)
csv_file = open('output.csv', 'w')
csv_writer = csv.writer(csv_file)
csv_writer.writerow(['details'])
details=[i.text for i in soup.find_all(class_='product-info-content-
block product-info')]
print('details :', details)
dt = date.today()
print('date :', dt)
csv_output.writerow([str(details).strip('[]'), dt])
csv_file.close()
Everything is being printed fine when the code is running, but not all the rows of data are being written in the output csv.
I hope someone can help.
Thank you!
It looks like you are opening output.csv twice, once in the beginning and then in the for loop. Since you are opening with the option w like csv_file = open('output.csv', 'w') it will overwrite the file every loop.
So if you move the below part out of the loop it might work better
csv_file = open('output.csv', 'w')
csv_writer = csv.writer(csv_file)
csv_writer.writerow(['details'])
I am trying to attempt something that I have not before within python.
The code below collects data from my test database and put it into a text under my headers of 'Test1','Test2','Test3'. This is working fine.
What I am trying to attempt now is to add a header (on top of the current header) and footer to the file.
python code:
file = 'file.txt'
header_names = {'t1':'Test1', 't2': 'Test2','t3':'Test3'}
with open(file, 'w', newline='') as f:
w = csv.DictWriter(f, fieldnames=header_names.keys(), restval='', extrasaction='ignore')
w.writerow(header_names)
for doc in res['test']['test']:
my_dict = doc['test']
w.writerow(my_dict)
current file output using the above code.
file.txt
Test1,Test2,Test3
Bob,john,Male
Cat,Long,female
Dog,Short,Male
Case,Fast,Male
Nice,who,Male
ideal txt output.
{header}
Filename:file.txt
date:
{data}
Test1,Test2,Test3
Bob,john,Male
Cat,Long,female
Dog,Short,Male
Case,Fast,Male
Nice,who,Male
{Footer}
this file was generated by using python.
the {header}, {data} and {footer} is not needed within the file that is just to make clear what is needed. i hope this makes sense.
Something like this
import csv
from datetime import date
# prepare some sample data
data = [['Bob', 'John', 'Male'],
['Cat', 'Long', 'Female']]
fieldnames = ['test1', 'test2', 'test3']
data = [dict(zip(fieldnames, row)) for row in data]
# actual part that writes to a file
with open('spam.txt', 'w', newline='') as f:
f.write('filename:spam.txt\n')
f.write(f'date:{date.today().strftime("%Y%m%d")}\n\n')
wrtr = csv.DictWriter(f, fieldnames = fieldnames)
wrtr.writeheader()
wrtr.writerows(data)
f.write('\nwritten with python\n')
Output in the file:
filename:spam.txt
date:20190321
test1,test2,test3
Bob,John,Male
Cat,Long,Female
written with python
Now, all that said, do you really need to write header and footer. It will just break a nicely formatted csv file and would require extra effort later on when reading it.
Or if you prefer - is the csv format what best suits your needs? Maybe using json would be better...
vardate= datetime.datetime.now().strftime("%x")
file = 'file.txt'
header_names = {'t1':'Test1', 't2': 'Test2','t3':'Test3'}
with open(file, 'w', newline='') as f:
f.seek(0,0) //This will move cursor to start position of file
f.writelines("File Name: ", file)
f.writelines("date: ", vardate)
f.writelines(".Try out next..")
w = csv.DictWriter(f, fieldnames=header_names.keys(), restval='',
extrasaction='ignore')
w.writerow(header_names)
for doc in res['test']['test']:
my_dict = doc['test']
w.writerow(my_dict)
f.seek(0,2)
f.writelines("This is generated using Python")
I've done this 100 times, but for some reason I can't get this to work. What am I doing wrong?
csv_file = open('test_a.csv', 'wb')
writer = csv.writer(csv_file)
writer.writerow('test')
OR
csv_file = open('test_a.csv', 'wb')
writer = csv.writer(csv_file)
writer.writerows(['test'])
OR
csv_file = open('test_a.csv', 'a')
writer = csv.writer(csv_file)
writer.writerow('test')
The CSV file is created, but nothing gets written.
Call a .close() on the file object or open with a context manager:
with open('test_a.csv', 'wb') as csv_file:
writer = csv.writer(csv_file)
writer.writerow(['test'])
The writerow method expects a list of values. You're giving it a string, which is iterable.. so you end up with t, e, s, t which is probably not what you want. Make sure you are passing a list:
writer.writerow(['test'])