Keep trying to write to file but the file is empty :-( I've tried so many things, feel like I'm spinning. I'm a newbie as you can tell. Any help would be greatly appreciated!
import shutil
import re
import csv
import sys # imports the sys module
infile = csv.reader(open("March9small.csv", "rU"), dialect=csv.excel_tab)
reader=csv.reader(infile)
new_rows_list=[]
for row in infile:
new_rows_list.append(row)
print new_rows_list
outfile = csv.writer(open("March9small-revised.csv","wt"), dialect=csv.excel_tab)
for i in new_rows_list:
outfile.writerow(i)
Why do you pass csv.reader to csv.reader?
Here's the simplified version without an intermediate list with nested with context managers:
import csv
with open("March9small.csv", "rU") as infile:
with open("March9small-revised.csv","wt") as outfile:
reader = csv.reader(infile, dialect=csv.excel_tab)
writer = csv.writer(outfile, dialect=csv.excel_tab)
for row in reader:
writer.writerow(row)
Make sure you understand what is going on here - if not, feel free to ask in comments.
Related
everything good?
I need some help to save this script in CSV that reads a CSV and transforms the data through a lib. I've been racking my brain for hours and I can't figure out why I can't save the CSV file.
Can anybody help me? I am a beginner in python and I am learning the tool to use in ETL processes.
import csv
from user_agents import parse
with open('UserAgent.csv', 'r') as csv_file:
csv_reader = csv.reader(csv_file)
idUser = 0
space = ' / '
for line in csv_reader:
user_agent = parse(line[0])
idUser = idUser + 1
with open('data.csv', 'w') as f:
writer = csv.writer(f)
writer.writerow(user_agent)
writer.writerow expects an iterable. Your user_agent must not be an iterable.
Try
writer.writerow( [user_agent] )
instead of
writer.writerow(user_agent)
Check if that's what you want.
This is a code:
import pandas as pd
import csv
with open('reviews.csv') as myFile:
reader = csv.reader(myFile)w
with open('bow.csv','a',newline="") as file:
handler= csv.writer(file)
for rowdata in reader:
handler.writerow({rowdata,'asd'})
Error is ValueError: I/O operation on closed file.
csv.reader() can only read from an open file. When you exit the first with block,myFile is automatically closed, so reader can't read from it any more.
You need to keep the input file open while you read from it.
import pandas as pd
import csv
with open('reviews.csv') as myFile:
reader = csv.reader(myFile)
with open('bow.csv','a',newline="") as file:
handler= csv.writer(file)
for rowdata in reader:
handler.writerow({rowdata,'asd'})
You can also open multiple files in a single with statement, so you don't need to nest them.
with open('reviews.csv') as myFile, open('bow.csv','a',newline="") as file:
reader = csv.reader(myFile)w
handler= csv.writer(file)
for rowdata in reader:
handler.writerow({rowdata,'asd'})
Who wants to help out a newbie? I keep getting either Indent errors or when I manage correct indentation I end up with an IO error because the file is already automatically closed by the Open With. What am I doing wrong?
This is my script;
import csv
infile = ('Test')
with open(infile+'.txt') as csvfile, open('output.csv', 'w', encoding='utf-8') as outfile:
lines = sorted(set(line.strip('\n').lower() for line in csvfile))
for line in lines:
writer = csv.writer(outfile, lineterminator='\n', quoting=csv.QUOTE_ALL)
writer.writerow(line)
Thanks for helping me out!
Indent the code to write out under the with statement.
You had a few spaces in there as well that were throwing out the indentation. I flattened your code then indented it out. You'll probably have to either open as read-write or open once to read and then once to write.
import csv
infile = ('Test')
with open(infile+'.txt') as csvfile, open('output.csv', 'w', encoding='utf-8') as outfile:
lines = sorted(set(line.strip('\n').lower() for line in csvfile))
for line in lines:
writer = csv.writer(outfile, lineterminator='\n', quoting=csv.QUOTE_ALL)
writer.writerow(line)
I edited the formatting on the original question. If my edit did not somehow mess things up, the likely issue is that by opening and closing both infile and outfile in the same with block you have already closed outfile when you want to write to it.
If you rearrange it like this it may be more predictable
import csv
infile = ('Test')
with open(infile+'.txt') as csvfile:
lines = sorted(set(line.strip('\n').lower() for line in csvfile))
with open('output.csv', 'w', encoding='utf-8') as outfile:
writer = csv.writer(outfile, lineterminator='\n', quoting=csv.QUOTE_ALL)
for line in lines:
writer.writerow(line)
I am trying to write to a .tsv file using python's CSV module, this is my code so far
file_name = "test.tsv"
TEMPLATE = "template.tsv"
fil = open(file_name, "w")
# Added suggested change
template = csv.DictReader(open(TEMPLATE, 'r'), delimiter='\t')
new_file = csv.DictWriter(fil, fieldnames=template.fieldnames, delimiter='\t')
new_file.writeheader()
basically TEMPLATE is a file that will contain the headers for the file, so i read the headers using DictReader and pass the fieldnames to DictWriter, as far as i know the code is fine, the file test.tsv is being created but for some reason the headers are not being written.
Any help as to why this is happening is appreciated, thanks.
DictReader's first argument should be a file object (create with open()), cf. http://docs.python.org/py3k/library/csv.html#csv.DictReader
You forgot open() for the TEMPLATE file.
import csv
file_name = "test.tsv"
TEMPLATE = "template.tsv"
fil = open(file_name, "w")
# you forgot this line, which will open the file
template_file = open(TEMPLATE, 'r')
template = csv.DictReader(template_file, delimiter='\t')
new_file = csv.DictWriter(fil, fieldnames=template.fieldnames, delimiter='\t')
new_file.writeheader()
Try to give DictReader opened file instead of file name:
csv.DictReader(open(TEMPLATE, 'r'), delimiter='\t')
Same for the writer, but opened for writing.
I have a Python script which modifies a CSV file to add the filename as the last column:
import sys
import glob
for filename in glob.glob(sys.argv[1]):
file = open(filename)
data = [line.rstrip() + "," + filename for line in file]
file.close()
file = open(filename, "w")
file.write("\n".join(data))
file.close()
Unfortunately, it also adds the filename to the header (first) row of the file. I would like the string "ID" added to the header instead. Can anybody suggest how I could do this?
Have a look at the official csv module.
Here are a few minor notes on your current code:
It's a bad idea to use file as a variable name, since that shadows the built-in type.
You can close the file objects automatically by using the with syntax.
Don't you want to add an extra column in the header line, called something like Filename, rather than just omitting a column in the first row?
If your filenames have commas (or, less probably, newlines) in them, you'll need to make sure that the filename is quoted - just appending it won't do.
That last consideration would incline me to use the csv module instead, which will deal with the quoting and unquoting for you. For example, you could try something like the following code:
import glob
import csv
import sys
for filename in glob.glob(sys.argv[1]):
data = []
with open(filename) as finput:
for i, row in enumerate(csv.reader(finput)):
to_append = "Filename" if i == 0 else filename
data.append(row+[to_append])
with open(filename,'wb') as foutput:
writer = csv.writer(foutput)
for row in data:
writer.writerow(row)
That may quote the data slightly differently from your input file, so you might want to play with the quoting options for csv.reader and csv.writer described in the documentation for the csv module.
As a further point, you might have good reasons for taking a glob as a parameter rather than just the files on the command line, but it's a bit surprising - you'll have to call your script as ./whatever.py '*.csv' rather than just ./whatever.py *.csv. Instead, you could just do:
for filename in sys.argv[1:]:
... and let the shell expand your glob before the script knows anything about it.
One last thing - the current approach you're taking is slightly dangerous, in that if anything fails when writing back to the same filename, you'll lose data. The standard way of avoiding this is to instead write to a temporary file, and, if that was successful, rename the temporary file over the original. So, you might rewrite the whole thing as:
import csv
import sys
import tempfile
import shutil
for filename in sys.argv[1:]:
tmp = tempfile.NamedTemporaryFile(delete=False)
with open(filename) as finput:
with open(tmp.name,'wb') as ftmp:
writer = csv.writer(ftmp)
for i, row in enumerate(csv.reader(finput)):
to_append = "Filename" if i == 0 else filename
writer.writerow(row+[to_append])
shutil.move(tmp.name,filename)
You can try:
data = [file.readline().rstrip() + ",id"]
data += [line.rstrip() + "," + filename for line in file]
You can try changing your code, but using the csv module is recommended. This should give you the result you want:
import sys
import glob
import csv
filename = glob.glob(sys.argv[1])[0]
yourfile = csv.reader(open(filename, 'rw'))
csv_output=[]
for row in yourfile:
if len(csv_output) != 0: # skip the header
row.append(filename)
csv_output.append(row)
yourfile = csv.writer(open(filename,'w'),delimiter=',')
yourfile.writerows(csv_output)
Use the CSV module that comes with Python.
import csv
import sys
def process_file(filename):
# Read the contents of the file into a list of lines.
f = open(filename, 'r')
contents = f.readlines()
f.close()
# Use a CSV reader to parse the contents.
reader = csv.reader(contents)
# Open the output and create a CSV writer for it.
f = open(filename, 'wb')
writer = csv.writer(f)
# Process the header.
header = reader.next()
header.append('ID')
writer.writerow(header)
# Process each row of the body.
for row in reader:
row.append(filename)
writer.writerow(row)
# Close the file and we're done.
f.close()
# Run the function on all command-line arguments. Note that this does no
# checking for things such as file existence or permissions.
map(process_file, sys.argv[1:])
You can run this as follows:
blair#blair-eeepc:~$ python csv_add_filename.py file1.csv file2.csv
you can use fileinput to do in place editing
import sys
import glob
import fileinput
for filename in glob.glob(sys.argv[1]):
for line in fileinput.FileInput(filename,inplace=1) :
if fileinput.lineno()==1:
print line.rstrip() + " ID"
else
print line.rstrip() + "," + filename