We Keep Coding

Python Read/Write csv file

Reading and writing data from/to csv file. When I run the program its formatted correctly in the console window, however, the formatting is off in the csv file I'm writing to (has a comma after each letter). What am I missing here?
import csv
with open("WJU stats.csv", 'r') as csv_file:
csv_reader = csv.reader(csv_file)
with open('wjudata.csv', 'w') as new_file:
csv_writer = csv.writer(new_file)
for row in csv_reader:
csv_writer.writerow(row[0])
print(row[0])

The function writerow takes an iterable, for example a list, so it writes each element of the iterable to the file in a comma separated row. The thing is strings are also iterables which elements are characters. If you want a single column csv you should use
csv_writer.writerow([row[0]])

How to parse CSV files in Python?

I have downloaded the billing reports from AWS, which are in CSV format, onto my server.
Now, I have to parse those CSV files in Python so that it shows consolidated/individual cost information on day/week/monthly basis.
Can anyone please help me with this?
import csv
with open('588399947422-aws-billing-detailed-line-items-2015-09.c‌​sv') as csvfile:
readCSV = csv.reader(csvfile,delimiter=',')
for row in readCSV :
print row
CSV headers
"InvoiceID","PayerAccountId","LinkedAccountId","RecordType",‌​"RecordId","ProductN‌​ame","RateId","Subsc‌​riptionId","PricingP‌​lanId","UsageType","‌​Operation","Availabi‌​lityZone","ReservedI‌​nstance","ItemDescri‌​ption","UsageStartDa‌​te","UsageEndDate","‌​UsageQuantity","Blen‌​dedRate","BlendedCos‌​t","UnBlendedRate","‌​UnBlendedCost","Reso‌​urceId","user:Applic‌​ation Name","user:Business Unit"

Use built-in csv module.
From docs:
>>> import csv
>>> with open(path_to_your_file, 'rb') as csvfile:
... reader = csv.reader(csvfile, delimiter=',', quotechar='|')
... for row in reader: # iterate over reader line by line (line is a list of values in this case)
... print row # list of values
First, you have to open csv, the best option is to use with open(filename,'rb') as f:.
Then, instantiate reader - you have to specify delimiter (comma in most cases) and quotechar (quotes if there are some).
Then you can iterate over reader line by line.

Python - CSV file empty after rewriting using csv module

I'm attempting to rewrite specific cells in a csv file using Python.
However, whenever I try to modify an aspect of the csv file, the csv file ends up being emptied (the file contents becomes blank).
Minimal code example:
import csv
ReadFile = open("./Resources/File.csv", "rt", encoding = "utf-8-sig")
Reader = csv.reader(ReadFile)
WriteFile = open("./Resources/File.csv", "wt", encoding = "utf-8-sig")
Writer = csv.writer(WriteFile)
for row in Reader:
row[3] = 4
Writer.writerow(row)
ReadFile.close()
WriteFile.close()
'File.csv' looks like this:
1,2,3,FOUR,5
1,2,3,FOUR,5
1,2,3,FOUR,5
1,2,3,FOUR,5
1,2,3,FOUR,5
In this example, I'm attempting to change 'FOUR' to '4'.
Upon running this code, the csv file becomes empty instead.
So far, the only other question related to this that I've managed to find is this one, which does not seem to be dealing with rewriting specific cells in a csv file but instead deals with writing new rows to a csv file.
I'd be very grateful for any help anyone reading this could provide.

The following should work:
import csv
with open("./Resources/File.csv", "rt", encoding = "utf-8-sig") as ReadFile:
lines = list(csv.reader(ReadFile))
with open("./Resources/File.csv", "wt", encoding = "utf-8-sig") as WriteFile:
Writer = csv.writer(WriteFile)
for line in lines:
line[3] = 4
Writer.writerow(line)

When you open a writer with w option, it will delete the contents and start writing the file anew. The file is therefore, at the point when you start to read, empty.
Try writing to another file (like FileTemp.csv) and at the end of the program renaming FileTemp.csv to File.csv.

Inline CSV File Editing with Python

Can I modify a CSV file inline using Python's CSV library, or similar technique?
Current I am processing a file and updating the first column (a name field) to change the formatting. A simplified version of my code looks like this:
with open('tmpEmployeeDatabase-out.csv', 'w') as csvOutput:
writer = csv.writer(csvOutput, delimiter=',', quotechar='"')
with open('tmpEmployeeDatabase.csv', 'r') as csvFile:
reader = csv.reader(csvFile, delimiter=',', quotechar='"')
for row in reader:
row[0] = row[0].title()
writer.writerow(row)
The philosophy works, but I am curious if I can do an inline edit so that I'm not duplicating the file.
I've tried the follow, but this appends the new records to the end of the file instead of replacing them.
with open('tmpEmployeeDatabase.csv', 'r+') as csvFile:
reader = csv.reader(csvFile, delimiter=',', quotechar='"')
writer = csv.writer(csvFile, delimiter=',', quotechar='"')
for row in reader:
row[1] = row[1].title()
writer.writerow(row)

No, you should not attempt to write to the file you are currently reading from. You can do it if you keep seeking back after reading a row but it is not advisable, especially if you are writing back more data than you read.
The canonical method is to write to a new, temporary file and move that into place over the old file you read from.
from tempfile import NamedTemporaryFile
import shutil
import csv
filename = 'tmpEmployeeDatabase.csv'
tempfile = NamedTemporaryFile('w+t', newline='', delete=False)
with open(filename, 'r', newline='') as csvFile, tempfile:
reader = csv.reader(csvFile, delimiter=',', quotechar='"')
writer = csv.writer(tempfile, delimiter=',', quotechar='"')
for row in reader:
row[1] = row[1].title()
writer.writerow(row)
shutil.move(tempfile.name, filename)
I've made use of the tempfile and shutil libraries here to make the task easier.

There is no underlying system call for inserting data into a file. You can overwrite, you can append, and you can replace. But inserting data into the middle means reading and rewriting the entire file from the point you made your edit down to the end.
As such, the two ways to do this are either (a) slurp the entire file into memory, make your edits there, and then dump the result back to disk, or (b) open up a temporary output file where you write your results while you read the input file, and then replace the old file with the new one once you get to the end. One method uses more ram, the other uses more disk space.

If you just want to modify a csv file inline by using Python, you may just employ pandas:
import pandas as pd
df = pd.read_csv('yourfilename.csv')
# modify the "name" in row 1 as "Lebron James"
df.loc[1, 'name'] = "Lebron James"
# save the file using the same name
df.to_csv("yourfilename.csv")

How to add a header to a csv file in Python?

I've tried many solutions to add a header to my csv file, but nothing's working properly. Here they are :
I used the writerow method, but my data are overwriting the first row.
I used the DictWriter method, but I don't know how to fill it correctly. Here is my code:
csv = csv.DictWriter(open(directory +'/csv.csv', 'wt'), fieldnames = ["stuff1", "stuff2", "stuff3"], delimiter = ';')
csv.writeheader(["stuff1", "stuff2", "stuff3"])
I got a "2 arguments instead of one" error and I really don't know why.
Any advice?

All you need to do is call DictWriter.writeheader() without arguments:
with open(os.path.join(directory, 'csv.csv'), 'wb') as csvfile:
writer = csv.DictWriter(csvfile, fieldnames = ["stuff1", "stuff2", "stuff3"], delimiter = ';')
writer.writeheader()
You already told DictWriter() what your headers are.

I encountered a similar problem when writing the CSV file.
I had to read the csv file and modify some of the fields in it.
To write the header in the CSV file, i used the following code:
reader = csv.DictReader(open(infile))
headers = reader.fieldnames
with open('ChartData.csv', 'wb') as outcsv:
writer1 = csv.writer(outcsv)
writer1.writerow(headers)
and when you write the data rows, you can use a DictWriter in the following way
writer = csv.DictWriter(open("ChartData.csv", 'a' ), headers)
In the above code "a" stands for appending.
In conclusion - > use a to append data to csv after you have written your header to the same file