I have a script that successfully removes a column from a csv file. Currently it does this by creating a new file. I want it to write to the original file rather than create a new one.
I’ve tried this by using the r+ mode for open but it’s not working how I want. See notes below. I think r+ mode is the one I need but I’m struggling to find working examples to learn from.
my code:
import csv
in_file = "Path to Source"
out_file = "Path to Result"
with open(in_file, 'r', newline='') as inf, \
open(out_file, 'w', newline='') as outf:
reader = csv.reader(inf)
writer = csv.writer(outf)
for r in reader:
writer.writerow((r[0],r[1],r[2],r[3],r[4],r[5],r[6]))
attempt using r+ mode:
with open(in_file, 'r+', newline='') as inf:
reader = csv.reader(inf)
writer = csv.writer(inf)
for r in reader:
writer.writerow((r[0],r[1],r[2],r[3],r[4],r[5],r[6]))
This fails with the error list index out of range
From what I see, as the reader reads, the writer writes. On the same file.
Files have a 'cursor', i.e. a current position upon which they are read from/written to.
So the writer is overwriting the next row in the file after the one the reader has just read, with catastrophic consequences on the following readings.
I think the first approach is the best one: create a new file and then rename it (the original input file is deleted automatically)
import csv, os
in_file = "Path to Source"
out_file = "Path to Result"
with open(in_file, 'r', newline='') as inf, \
open(out_file, 'w', newline='') as outf:
reader = csv.reader(inf)
writer = csv.writer(outf)
for r in reader:
writer.writerow(r[:7])
os.rename(out_file, in_file)
Related
I'm trying to read sentences in a csv file, convert them to lowercase and save in other csv file.
import csv
import pprint
with open('dataset_elec_4000.csv') as f:
with open('output.csv', 'w') as ff:
data = f.read()
data = data.lower
writer = csv.writer(ff)
writer.writerow(data)
but I got error "_csv.Error: sequence expected". What should I do?
*I'm a beginner. Please be nice to me:)
You need to read over your input CSV row-by-row, and for each row, transform it, then write it out:
import csv
with open('output.csv', 'w', newline='') as f_out:
writer = csv.writer(f_out)
with open('dataset_elec_4000.csv', newline='') as f_in:
reader = csv.reader(f_in)
# comment these two lines if no input header
header = next(reader)
writer.writerow(header)
for row in reader:
# row is sequence/list of cells, so...
# select the cell with your sentence, I'm presuming it's the first cell (row[0])
data = row[0]
data = data.lower()
# need to put data back into a "row"
out_row = [data]
writer.writerow(out_row)
Python contains a module called csv for the handling of CSV files. The reader class from the module is used for reading data from a CSV file. At first, the CSV file is opened using the open() method in ‘r’ mode(specifies read mode while opening a file) which returns the file object then it is read by using the reader() method of CSV module that returns the reader object that iterates throughout the lines in the specified CSV document.
import csv
# opening the CSV file
with open('Giants.csv', mode ='r')as file:
# reading the CSV file
csvFile = csv.reader(file)
# displaying the contents of the CSV file
for lines in csvFile:
print(lines)
Who wants to help out a newbie? I keep getting either Indent errors or when I manage correct indentation I end up with an IO error because the file is already automatically closed by the Open With. What am I doing wrong?
This is my script;
import csv
infile = ('Test')
with open(infile+'.txt') as csvfile, open('output.csv', 'w', encoding='utf-8') as outfile:
lines = sorted(set(line.strip('\n').lower() for line in csvfile))
for line in lines:
writer = csv.writer(outfile, lineterminator='\n', quoting=csv.QUOTE_ALL)
writer.writerow(line)
Thanks for helping me out!
Indent the code to write out under the with statement.
You had a few spaces in there as well that were throwing out the indentation. I flattened your code then indented it out. You'll probably have to either open as read-write or open once to read and then once to write.
import csv
infile = ('Test')
with open(infile+'.txt') as csvfile, open('output.csv', 'w', encoding='utf-8') as outfile:
lines = sorted(set(line.strip('\n').lower() for line in csvfile))
for line in lines:
writer = csv.writer(outfile, lineterminator='\n', quoting=csv.QUOTE_ALL)
writer.writerow(line)
I edited the formatting on the original question. If my edit did not somehow mess things up, the likely issue is that by opening and closing both infile and outfile in the same with block you have already closed outfile when you want to write to it.
If you rearrange it like this it may be more predictable
import csv
infile = ('Test')
with open(infile+'.txt') as csvfile:
lines = sorted(set(line.strip('\n').lower() for line in csvfile))
with open('output.csv', 'w', encoding='utf-8') as outfile:
writer = csv.writer(outfile, lineterminator='\n', quoting=csv.QUOTE_ALL)
for line in lines:
writer.writerow(line)
I have a csv file and I want to transfer the raw data without the headers to a new csv file and have the rows and columns the same as the original.
IRIS_data = "IRIS_data.csv"
with open(IRIS_data, 'wb') as data:
wr = csv.writer(data, quoting=csv.QUOTE_ALL)
with open(IRIS) as f:
next(f)
for line in f:
wr.writerow(line)
The code above is my most recent attempt, when I try run it I get the following error:
a bytes-like object is required, not 'str'
It's because you opened the input file with with open(IRIS_data, 'wb'), which opens it in binary mode, and the output file with just with open(IRIS) which opens it in text mode.
In Python 3, you should open both files in text mode and specify newline='' option)—see the examples in the csv module's documentation)
To fix it, change them as follows:
with open(IRIS_data, 'w', newline='') as data:
and
with open(IRIS, newline='') as f:
However there are other issues with you code. Here's how to use those statements to get what I think you want:
import csv
IRIS = "IRIS.csv"
IRIS_data = "IRIS_data.csv"
with open(IRIS, 'r', newline='') as f, open(IRIS_data, 'w', newline='') as data:
next(f) # Skip over header in input file.
writer = csv.writer(data, quoting=csv.QUOTE_ALL)
writer.writerows(line.split() for line in f)
Contents of IRIS_data.csv file after running the script with your sample input data:
"6.4","2.8","5.6","2.2","2"
"5","2.3","3.3","1","1"
"4.9","2.5","4.5","1.7","2"
"4.9","3.1","1.5","0.1","0"
"5.7","3.8","1.7","0.3","0"
"4.4","3.2","1.3","0.2","0"
"5.4","3.4","1.5","0.4","0"
"6.9","3.1","5.1","2.3","2"
"6.7","3.1","4.4","1.4","1"
"5.1","3.7","1.5","0.4","0"
You have to encode the line you are writing like this:
wr.writerow( line.encode(”utf8”))
Also open your file using open(..., ‘wb’). This will open the file in binary mode. So you are certain the file is actually open in binary mode. Indeed it is better to now explicitly the encoding than assuming it. Enforcing encoding for both reading and writing will save you lots of trouble.
I am trying to create .csv file.
For some reason it skips line before printing entry.
Here is the output
But here is what I need
Below is code. Apparently if line != "": doesn't work
import csv
#-----------------------------------
def csv_writer(data,path):
"""
Write data to a CSV file path
"""
with open(path, "w") as csv_file:
writer = csv.writer(csv_file, delimiter=',')
for line in data:
if line != "":
writer.writerow(line)
#-----------------------------------
if __name__ == "__main__":
data = ["first_name,last_name,city".split(","),
"Tyrese,Hirthe,Strackeport".split(","),
"Jules,Dicki,Lake Nickolasville".split(","),
"Dedric,Medhurst,Stiedemannberg".split(",")
]
path = "output.csv"
csv_writer(data,path)
Some python versions (on windows) have an issue with that with open(path, "w") as csv_file:. A spurious carriage return char is inserted, creating a blank line after each line.
You have to add newline="" as stated in the documentation. Python 3:
with open(path, "w",newline="") as csv_file:
writer = csv.writer(csv_file, delimiter=',')
As for python 2:
with open(path, "wb") as csv_file:
writer = csv.writer(csv_file, delimiter=',')
see also:
portable way to write csv file in python 2 or python 3
csv writer expected byte like and space between rows
(note that latest Python versions on Windows don't need this anymore, but the documentation continues to state it)
When you open the file you need to pass the keyword argument newline with a blank string. This will prevent the newlines being added between rows. Your function should be:
def csv_writer(data,path):
"""
Write data to a CSV file path
"""
with open(path, "w", newline = '') as csv_file:
writer = csv.writer(csv_file, delimiter=',')
for line in data:
if line:
writer.writerow(line)
Note that this is only an issue on Windows.
I am generating a number of csv files dynamically, using the following code:
import csv
fieldnames = ['foo1', 'foo2', 'foo3', 'foo4']
with open(csvfilepath, 'wb') as csvfile:
csvwrite = csv.DictWriter(csvfile, delimiter=',', fieldnames=fieldnames)
csvwrite.writeheader()
for row in data:
csvwrite.writerow(row)
To save space, I want to compress them.
Using the gzip module is quite easy:
with gzip.open("foo.gz", "w") as csvfile :
csvwrite = csv.DictWriter(csvfile, delimiter=',', fieldnames=fieldnames)
csvwrite.writeheader()
for row in data:
csvwrite.writerow(row)
But I want the file in 'zip' format.
I tried the zipfile module, but I am unable to directly write files into the zip archive.
Instead, I have to write the csv file to disk, compress them in a zip file using following code, and then delete the csv file.
with ZipFile(zipfilepath, 'w') as zipfile:
zipfile.write(csvfilepath, csvfilename, ZIP_DEFLATED)
How can I write a csv file directly to a compressed zip similar to gzip?
Use the cStringIO.StringIO object to imitate a file:
with ZipFile(your_zip_file, 'w', ZIP_DEFLATED) as zip_file:
string_buffer = StringIO()
writer = csv.writer(string_buffer)
# Write data using the writer object.
zip_file.writestr(filename + '.csv', string_buffer.getvalue())
Thanks kroolik
It's done with little modification.
with ZipFile(your_zip_file, 'w', ZIP_DEFLATED) as zip_file:
string_buffer = StringIO()
csvwriter = csv.DictWriter(string_buffer, delimiter=',', fieldnames=fieldnames)
csvwrite.writeheader()
for row in cdrdata:
csvwrite.writerow(row)
zip_file.writestr(filename + '.csv', string_buffer.getvalue())
Having IOString to store every bytes in memory could be very memory consuming.
Based on the zipfile module documentation after creating a ZipFile object, all individual files has to be opened. Like this:
with ZipFile('spam.zip') as myzip:
with myzip.open('eggs.txt') as myfile:
print(myfile.read())
This example can be used for write as well...