I am trying to write a code to write multiple arrays into one single data frame in panda where I append the data frame row by row . For example I have a row of [1,2,3,4,5] and next row of [6,7,8,9,10].
I want to print it as :
1,2,3,4,5
6,7,8,9,10
in a csv file. I want to write multiple rows like this in single csv file but all codes can be found only for appending a data frame column by column. Can I write this array row by row too?
Please help.
I tried using pandas library but couldn't fine relevant command.
the next code snippet might help you:
import csv
with open('file.csv', 'w') as f:
writer = csv.writer(f)
writer.writerows([[1,2,3], [4,5,6]])
Related
I have data from a spectrometer from Ocean Optics and am trying to extract the right hand column of data to create a log intensity time graph. I'm currently having issues extracting the intensity data from the csv. This is what the data looks like:
The data intensity data starts on the 18th line of code and ends on line 3665. I only need the second column of data.
So far I have this,
import csv
with open('TEST000.csv') as csv_file:
data = csv.reader(csv_file)
for row in list(data)[18:3665]:
print(row)
But when I run the code it doesn't work and doesn't extract the data correctly.
How do I get this to work? Also any help of converting the data into a log graph would also be appreciated
you are trying to use csv.reader on a file that is not a csv. instead try something like this:
with open('TEST000.csv') as csv_file:
for row in list(csv_file)[18:3665]:
print(row)
This will give you each row and you can split the values of each entry in the list by using the split() method. Something like:
with open('TEST000.csv') as csv_file:
for row in list(csv_file)[18:3665]:
print(row.split(" ")[1])
EDIT: don't forget to close your file after you're done with csv_file.close()
Have an excel file with a column with some text in each row of this column.
I'm using pandas pd.read_excel() to open this .xlsx file. Then I would like to do the following: I would like to save every row of this column as a distinct .txt file (that would have the text from this row inside this file). Is it possible to be done via pandas?
the basic idea would be to use an iterator to loop over the rows, opening a each file and writing the value in, something like:
import pandas as pd
df = pd.read_excel('test.xlsx')
for i, value in enumerate(df['column']):
with open(f'row-{i}.txt', 'w') as fd:
fd.write(value)
I want to load a csv file into python. The csv file contains grades for a random number of students and a random number of assignments.
I want python to delete the header and the first column (Name of student) and this is my code:
with open("testgrades.csv") as f:
ncols = len(f.readline().split(','))
nrows = sum(1 for row in f)
grades = np.loadtxt("testgrades.csv", delimiter=',', skiprows=1, usecols=range(1,ncols+1))
print(document1)
The code works for columns but can't handle if I add one or more rows in the csv file?
My CSV file:
csv
And output from Python:
Output
Your csv image looks like a messed up spread sheet image. It isn't a copy of the csv file itself, which is plain text. You should be able to copy-n-paste that text to your question.
The Output image is an array, with numbers that correspond to the first 6 rows of the csv image.
Your question is not clear. I'm guessing you added the last 2 rows to the spread sheet, and are having problems loading those into numpy. I don't see anything wrong with those numbers in the spread sheet image. But if you show the actual csv file content, we might identify the problem. Maybe you aren't actually writing those added rows to the csv file.
Your code sample, with corrected indentation is:
with open("testgrades.csv") as f:
ncols = len(f.readline().split(','))
nrows = sum(1 for row in f)
grades = np.loadtxt("testgrades.csv", delimiter=',', skiprows=1, usecols=range(1,ncols+1))
print(grades)
I can see using the ncols to determine the number of columns. The usecols parameter needs an explicit list of columns, not some sort of all-but-first. You could have also gotten that number from a plain loadtxt (or genfromtxt).
But why calculate nrows? You don't appear to use it. And it isn't needed in the loadtxt. genfromtxt allows a max_rows parameter if you need to limit the number of rows read.
Python has a special module for reading and writing CSV files Python CSV
Python 2
import csv
with open('testgrades.csv', 'rb') as f:
Python 3
import csv
with open('testgrades.csv', newline='') as f:
I am trying to delete a particular row (NOT a column) in a csv file for a
class project. When I deleted columns I put:
r=row
r[22], r[21]
# and so on
So how do I specify that I want to delete rows? I am working with census data and want to get rid of that extra row of headers that are always in census tables.
Thank you for any help.
Convert your csv reader to a list and slice the appropriate indexes off:
import csv
with open('file.csv', 'rb') as f:
reader = csv.reader(f)
rows = list(reader)[1:] # no more header row
Use pandas, it's so easy to handle data and files with it. Use it to edit your data easily.
You can open your csv file and convert it to a pandas dataframe through.
df = pandas.read_csv('file.csv')
After that you can use this function.
df.drop(df.columns[[0]], axis=1)
In this example I'm deleting the row with index 0.
Pandas documentation
I got a csv file containing data in this form,
I want to extract data from column C and write them into a new csv file, like this,
So I need to do 2 things：
write 'node' and number from 1 to 22 into the first row and column (since in this case, there are 22 in one repeated cycle in the column A in input csv)
I have got data in column c extracted and write in output csv, like this,
I need to transpose those data every 22 rows one time and fill them in row starts from B2 position in excel, then B3, B4,...etc.
It's clear that I must loop through every row to do this efficiently, but I don't know how to apply the csv module in python.
Should I download the xlrd package, or can I handle this only use the built-in csv module？
I am working with python 2.7.6 and pyscripter under Windows 8.1 x64. Feel free to give me any suggestion, thanks a lot!
Read the csv python documentation.
The simple way to iterate through rows with csv reader:
import csv
X = []
spamreader = csv.reader('path_to_file/filename.csv',delimiter=',')
for row in spamreader:
X.append(row)
This creates a variable with all the csv data. The structure of your file will make it difficult to read because the cell_separator is ',' but there are also multiple commas within each cell and because of the parentheses there will be a mixture of string and numerical data that will require some cleaning. If you have access to reformatting the csv it might be easier if each cell looked like 1,2,0.01 instead of (1,2,0.01), also consider using a different delimiter between cells such as ';'.
If not expect some tedious data cleaning, and definitely read through the documentation linked above.
Edit: Try the following
import csv
X = []
with open('path_to_file/filename.csv','rb') as csvfile:
spamreader = csv.reader(csvfile,delimiter=',')
for row in spamreader:
rowTemp = []
for i in range(len(row)):
if (i+1)%3==0: #gets every third cell
rowTemp.append(row[i])
X.append(rowTemp)
This is a matrix of all the distance values. Then try:
with open('path_to_output_file/output_file.csv','wb') as csvfile:
spamwriter = csv.writer(csvfile,delimter=',')
for sublist in X:
spamwriter.writerow(sublist)
Not sure if this is exactly what you're looking for but it should be close. It ouputs a csv file that is stripped of all the node pairs