I have a csv file created with 6 rows, 1 column (header and 5 numbers). I want to be able to do a conversion, say from centimeters to inches, and save it in a new csv with a new header.
So far I have only been able to import the csv and read it, and print it (using print row), but how can I do the conversion? Since the numbers are saved in the csv, would I have to convert the numbers to float and then write them to a new csv? I only have 5 numbers as I want to be able to just figure out the correct code, but I will use this for a lot of numbers.
I wasn't sure where the computation would be placed either. Help please! Also, this isn't homework or the like. Im just doing this for fun.
This is the code I currently have:
import csv
with open('test.csv', 'rb') as f:
reader = csv.reader(f)
next(reader, None) # I did this to skip the header I labelled Centimeters
with open('test1.csv', 'wb') as o:
writer = csv.writer(o)
for row in reader
f.close()
o.close()
I guess I dont know how to convert the number in the rows to float and then output the values. I want to just be able to multiply the number in the row by 0.393701 so that in the new csv the header is labelled inches with the output beneath in the rows.
This should work, assuming a single column (for multiple columns the handling would differ some to output all the values, but the general concept would be the same):
import csv
with open('test.csv', 'rb') as f, open('test1.csv', 'wb') as o:
reader = csv.reader(f)
writer = csv.writer(o)
# skip the header
next(reader, None)
# print the new header
writer.writerow(['inches'])
for row in reader:
newVal = float(row[0]) * 0.393701
writer.writerow([newVal])
import csv
float_rows=[]
with open('test.csv', 'rb') as f:
reader = csv.reader(f)
next(reader, None) # I did this to skip the header I labelled Centimeters
for row in reader:
comp = [ x * 0.393701 for x in map(float,row)] # do calculations and map elements to float
float_rows.append(comp)
with open('test1.csv', 'wb') as o:
writer = csv.writer(o)
writer.writerows(float_rows) # write all computed data to new csv
No close needed, with closes the files automatically.
Using map(float,iterable) is the same as [float(x) for x in my_iterable]
Related
I have one html file containing a table in it. Total rows in the tables are around 3500. I want to read and print rows with same values. PFA Image of the html data.
I transform the data into csv where I could see same data in html format.
As shown in image. I want to print and write all the rows containing "MyData" to another CSV and then need to mail it.
I tried using Soupbeautiful but not able to get the result.
I tried using CSV and Pandas but it is not returning the expected output.
My python code is as follows;
import csv
import numpy as np
import pandas as pd
import sys
csv.field_size_limit(sys.maxsize)
df = pd.read_csv('test.csv')
data = print (df.iloc[0:5])
Another code I tried
search_string = "MyData"
with open('test.csv') as f, open('test2.csv', 'w') as g:
reader = csv.reader(f)
next(reader, None) # discard the header
writer = csv.writer(g)
for row in reader:
if row[2] == search_string:
writer.writerow(row[:2])
print(row)
When I enter complete row from info_data then it gives me that particular row but not other rows where the string "MyData" is present.
Thanks !
You are currently testing the entry for an exact match with your search string. That entry contains a JSON string, so you could use in to see if it contains search_string rather than is an exact match for it, for example:
search_string = "MyData"
with open('test.csv') as f, open('test2.csv', 'w') as g:
reader = csv.reader(f)
next(reader, None) # discard the header
writer = csv.writer(g)
for row in reader:
if search_string in row[2]:
writer.writerow(row[:2])
print(row)
You would then want to add code to further decode you JSON data.
i have a large csv file and can not load in memory at a time,i also want to add some columns at the side of csv,so i want to add one column once a time because that does not cost many memory,i use python and pandas,so what can i do for that.
here's my code.
def toCsv(filepath,lists):
i = 0
with open(filepath,'r+') as f:
reader = csv.reader(f)
writer = csv.writer(f)
for row in reader:
print lists
row.append(lists[i])
writer.writerows(row)
i = i+1
list3 = []
with open('**directory**') as csvfile:
reader = csv.DictReader(csvfile)
for row in reader:
list3.append(row)
I'm completely new to data analysis using Python, and require some assistance.
The file I'm accessing contains data from 5 people (CSV file). There are 3 columns - participant number, pre-task Score, and post-task Score.
I'm essentially trying to access this file (using csv.DictReader) and manipulate the data. By this, I mean I want to calculate the difference between the post-task score and pre-task score, for each participant, and print this to the screen.
However, I'm not sure how to do this. I can print each row to the screen, and I can save each row in a list (as I've done above) - but I'm clueless as to how I am to manipulate/deal with this data. I'm wondering if there is something better than the module I'm currently using.
Calculating the difference between the second and third columns in a CSV file can be accomplished as follows:
import csv
with open('file.csv', 'rb') as csvfile:
reader = csv.reader(csvfile, delimiter=',')
# skip the header row, remove this next line if there is no header
next(reader, None)
for row in reader:
difference = float(row[2]) - float(row[1])
print str(difference)
I am trying to append 2 data sets to my csv file. Below is my code. The code runs but my data gets appended below a set of data in the first column (i.e. col[0]). I would however like to append my data sets in separate columns at the end of file. Could I please get advice on how I might be able to do this? Thanks.
import csv
Trial = open ('Trial_test.csv', 'rt', newline = '')
reader = csv.reader(Trial)
Trial_New = open ('Trial_test.csv', 'a', newline = '')
writer = csv.writer(Trial_New, delimiter = ',')
Cortex = []
Liver = []
for col in reader:
Cortex_Diff = float(col[14])
Liver_Diff = float(col[17])
Cortex.append(Cortex_Diff)
Liver.append(Liver_Diff)
Avg_diff_Cortex = sum(Cortex)/len(Cortex)
Data1 = str(Avg_diff_Cortex)
Avg_diff_Liver = sum(Liver)/len(Liver)
Data2 = str(Avg_diff_Liver)
writer.writerows(Data1 + Data2)
Trial.close()
Trial_New.close()
I think I see what you are trying to do. I won't try to rewrite your function entirely for you, but here's a tip: assuming you are dealing with a manageable size of dataset, try reading your entire CSV into memory as a list of lists (or list of tuples), then perform your calculations on the values on this object, then write the python object back out to the new CSV in a separate block of code. You may find this article or this one of use. Naturally the official documentation should be helpful too.
Also, I would suggest using different files for input and output to make your life easier.
For example:
import csv
data = []
with open('Trial_test.csv', 'rb') as csvfile:
reader = csv.reader(csvfile, delimiter=' ', quotechar='|')
for row in reader:
data.append(row)
# now do your calculations on the 'data' object.
with open('Trial_test_new.csv', 'wb') as csvfile:
writer = csv.writer(csvfile, delimiter=' ', quotechar='|')
for row in data:
writer.writerow(row)
Something like that, anyway!
I already have written what I need for identifying and parsing the value I am seeking, I need help writing a column to the csv file (or a new csv file) with the parsed value. Here's some pseudocode / somewhat realistic Python code for what I am trying to do:
# Given a CSV file, this function creates a new CSV file with all values parsed
def handleCSVfile(csvfile):
with open(csvfile, 'rb') as file:
reader = csv.reader(file, delimiter=',', lineterminator='\n')
for row in reader:
for field in row:
if isWhatIWant(field):
parsedValue = parse(field)
# write new column to row containing parsed value
I've already written the isWhatIWant and parse functions. If I need to write a completely new csv file, then I am not sure how to have both open simultaneously and read and write from one into the other.
I'd do it like this. I'm guessing that isWhatIWant() is something that is supposed to replace a field in-place.
import csv
def handleCSVfile(infilename, outfilename):
with open(infilename, 'rb') as infile:
with open(outfilename, 'wb') as outfile:
reader = csv.reader(infile, lineterminator='\n')
writer = csv.writer(outfile, lineterminator='\n')
for row in reader:
for field_index, field in enumerate(row):
if isWhatIWant(field):
row[field_index] = parse(field)
writer.writerow(row)
This sort of pattern occurs a lot and results in really long lines. It can sometimes be helpful to break out the logic from opening and files into a different function, like this:
import csv
def load_save_csvfile(infilename, outfilename):
with open(infilename, 'rb') as infile:
with open(outfilename, 'wb') as outfile:
reader = csv.reader(infile, lineterminator='\n')
writer = csv.writer(outfile, lineterminator='\n')
read_write_csvfile(reader, writer)
def read_write_csvfile(reader, writer)
for row in reader:
for field_index, field in enumerate(row):
if isWhatIWant(field):
row[field_index] = parse(field)
writer.writerow(row)
This modularizes the code, making it easier for you to change the way the files and formats are handled from the logic independently from each other.
Additional hints:
Don't name variables file as that is a built-in function. Shadowing those names will bite you when you least expect it.
delimiter=',' is the default so you don't need to specify it explicitly.