Hi
I have a code to read the csv file
import csv
d = csv.reader(open('C:/Documents and Settings/242481/My Documents/file.csv'))
for row in d:
print row
This code returns all the rows in the csv file
Is there any way i can read one row at a time.
And each time i execute the print line i need to get the next row.
Thanks in advance
Aadith
it should work the way you have it, but maybe the EOL characters are not what is expected for the system you're on. try opening it with 'rU': open('file.csv', 'rU')
to verify that it's printing one row at a time, you could print a blank line between rows:
for row in d:
print row
print
or pause it:
for row in d:
print row
raw_input('continue-> ')
For your other code, it should be something like:
def value():
infile=open("C:/Documents and Settings/242481/My Documents/file.csv", "rU")
data = [row for row in infile]
infile.close()
return data
Always close your open files. It's good practice, even though not always strictly necessary. And 'file' is a Python class name. Although you can use these any way you wish, doing so can lead to hard-to-find bugs later on.
Related
I want to delete rows from a csv file as they are processed.
My file:
Sr,Name1,Name2,Name3
1,Zname1,Zname2,Zname3
2,Yname1,Yname2,Yname3
3,Xname1,Xname2,Xname3
I want to read row by row and delete the row which has been processed.
So the file will be now:
2,Yname1,Yname2,Yname3
3,Xname1,Xname2,Xname3
The solutions which are provided on other questions are:
read the file
use next() or any other way to skip the row and write the remaining rows in an updated file
I want to delete the row from the original file which was entered in .reader() method
My code:
with open("file.txt", "r") as file
reader = csv.reader(file)
for row in reader:
#process the row
#delete the row
I have not been able to figure out how to delete/remove the row.
I want the change to be in the original file.txt because I will be running the program many times and so each time it runs, file.txt will already be present and the program will start from where it ended the last time.
Just read the csv file in memory as a list, then edit that list, and then write it back to the csv file.
lines = list()
members= input("Please enter a member's name to be deleted.")
with open('mycsv.csv', 'r') as readFile:
reader = csv.reader(readFile)
for row in reader:
lines.append(row)
for field in row:
if field == members:
lines.remove(row)
with open('mycsv.csv', 'w') as writeFile:
writer = csv.writer(writeFile)
writer.writerows(lines)
You can delete column like this:
We can use the panda pop () method to remove columns from CSV by naming the column as an argument.
Import Pandas.
Read CSV File.
Use pop() function for removing or deleting rows or columns from the CSV files.
Print Data.
You probably can find inspiration here: How to delete a specific line in a file?.
And don't forget to grant write permission when opening the file.
Since the pandas package deals with big data, there is no solution in basic Python.
You will have to import pandas.
import pandas
df=pandas.read_csv("file_name.txt")
df.set_value(0,"Name3",new_value)
df.to_csv("file_name.txt", index=False)
This code edits the cell in the 0th row and Name3 column. The 0th row is the first row below the header. Thus, Zname3 will be changed to something else. You can similarly delete a row or a cell.
I have not tried this code but it is supposed to work in the required manner.
I have the following data in a file called data.txt and would like to be able to add to the numbers at the end and replace them in the file without creating a new one:
Alfreda,art,2015,35
brook,biology,2015,3
charlie,chemistry,2015,140
dolly,Design,2015,120
Emilia,English,2015,150
Fiona,french,2015,40
Grace,Greek,2015,12
Hanna,history,2015,15
Here is the code I currently have:
with open("data.txt", "r") as f:
newline=[]
for word in f.line():
newline.append(word.replace(35,str(New))
with open("data.txt", "w") as f:
for line in newline :
f.writelines(line)
If you just want to add string to each line then update the file, this code can solve your problem but this is not optimal.
with open("data.txt", "r") as myFile:
newline=[]
# Use the readlines method to get all the lines
for line in myFile.readlines():
# Remove the \n character with the rstrip method
line = line.rstrip('\n')
newline.append(line+",35\n") # Don't forget to add \n
# Test
print newline
myFile.close()
with open("data.txt", "w") as myFile:
for line in newline :
myFile.writelines(line)
If this is not your problem, try to use the pickle module and work with objects, it will be easier.
I'm going to have to make some of your question up. If you have a file and you want to update it, the updates have to come from somewhere. The code in the question has a New variable but there is no indication of how New is supposed to get a value, or how the program is supposed to know which row to update.
I'm going to assume you have a file of updates called updates.txt that looks like this (and it is deliberately not in alphabetical order):
Emilia,45
Alfreda,35
So after your program runs the resulting file will have two rows different:
Alfreda,art,2015,70 ...this one
brook,biology,2015,3
charlie,chemistry,2015,140
dolly,Design,2015,120
Emilia,English,2015,195 ...and this one
Fiona,french,2015,40
Grace,Greek,2015,12
Hanna,history,2015,15
But the rest the same.
Since your sample data file is a .csv file I am using the Python csv module, rather than picking the data apart by hand. It doesn't matter much with simple data like this but it's a good module to know about.
import csv
marks = {}
# Read in existing data into a dictionary:
# key is name, value is a list [subject, year, score]
# like this: {"Alfreda": ["art",2015,35], ... }
# This is to make it easy to do random updates based on name
with open("data.txt", "r") as f:
for row in csv.reader(f):
name,subject,year,score = row
marks[name] = [subject,int(year),int(score)]
# Read in updates and apply each line to the corresponding entry in marks
with open("updates.txt", "r") as f:
for row in csv.reader(f):
name,added_score = row
try:
marks[name][2] += int(added_score) # for example marks["Alfreda"][2] += int("35")
except KeyError:
print(f"Name {name} not found to update, nothing done")
# Write out updated dictionary:
with open("data.txt", "w") as f:
writer = csv.writer(f,lineterminator="\n")
for name in sorted(marks.keys(), key=lambda n: n.lower()):
row=[name]+marks[name] # for example ["Alfreda"] + ["art",2015,70]
writer.writerow(row)
This line:
for name in sorted(marks.keys(), key=lambda n: n.lower()):
looks complicated but it is needed because you obviously expect the names Alfreda brook charlie dolly Emilia Fiona Grace Hanna to be in that order. But just doing the obvious
for name in sorted(marks.keys()):
will put them in the order Alfreda Emilia Fiona Grace Hanna brook charlie dolly.
In the interests of keeping the code simple and as close to your original as possible, it does no validity checks, so if this line
charlie,chemistry,2015,140
was wrongly entered as
charlie,chemistry,2015,14O
(with the letter O instead of a zero), the program will just fail. Ditto if the update file is missing a comma somewhere.
This works and will do what I think you want. But...
There are issues with the design. Your program reads in the data from data.txt, then overwrites it with new data. But suppose your program fails just after this line:
with open("data.txt", "w") as f:
Then you won't have your original data (because the call to open() truncated it), and you won't have the new data either (because you haven't written it out yet). Or suppose you accidentally run the program twice. There will be no way to tell you have done that.
You can provide some insurance against this sort of mishap by using the fileinput module, like this:
import fileinput
# Read in existing data
with fileinput.input("data.txt", inplace=True, backup=".bkp") as f:
for row in csv.reader(f):
name,subject,year,score = row
marks[name] = [subject,int(year),int(score)]
With this change, your updates will be in data.txt as before, but your original data will still be around, in a file called data.txt.bkp.
But that is just a fix. It avoids the real issue, which is that you really have a database application and you are trying to implement it using textfiles. The code above is all very well for an exercise, but it's not robust and it won't scale.
I have looked at previous answers to this question, but in each of those scenarios the questioners were asking about something specific they were doing with the file, but the problem occurs for me even when I am not.
I have a .csv file of 27,204 rows. When I open the python interpreter:
python
import csv
o = open('btc_usd1hour.csv','r')
p = csv.reader(o)
for row in p:
print(row)
I then only see roughly the last third of the document displayed to me.
Try so, at me works:
with open(name) as csvfile:
reader = csv.DictReader(csvfile)
for row in reader:
print(row)
reference:
https://docs.python.org/3.6/library/csv.html#csv.DictReader
Try the following code
import csv
fname = 'btc_usd1hour.csv'
with open(fname, newline='') as f:
reader = csv.reader(f)
for row in reader:
print(row)
It is difficult to tell what is the problem without having the sample. I guess the problem would be removed if you add that newline='' for opening the file.
Use the with construct to close the file automatically. Use the f name for a file object when no further explanation is needed. Store the file name to fname to make future modifications easier (and also for easy copying the code fragment for your later programs).
olisch may be right that the console just scrolled so fast you could not see the result. You can write the result to another text file like this:
with open(fname, newline='') as fin,\
open('output.txt', 'w') as fout:
reader = csv.reader(fin)
for row in reader:
fout.write(repr(row) + '\n')
The repr function converts the row list into its string representation. The print calls that function internally, so you will have the same result that you otherwise observe on screen.
maybe your scrollback buffer is just to short to see the whole list?
In general your csv.reader call should be working fine, except your 27k rows aren't extremly long so that you might be able to hit any 64bit boundaries, which would be quite uncommon.
len(o) might be interesting to see.
I have two programs in Python. One writes a customer's information to a CSV. The other accesses it. When the first has written it, I can open the CSV file (in Excel) and see that it has been written correctly. However for the other program to access the new data in the CSV file I have to manually open it and save it (in Excel) otherwise it doesn't work. Does anyone know why this may be?
Edit:
This writes to it (from first program):
f = open('details.csv', 'at', newline=''); csv_f = csv.reader(f)
csv_w.writerow(clientList)
f.close()
And this reads it (second program):
f = open('details.csv', 'rt', newline=''); csv_f = csv.reader(f)
for row in csv_f:
name.append(row[0])
I get this error when trying to append row[0] to a list.
Traceback (most recent call last):
File "C:\Users\Dan\Desktop\Garden Centre\work.py", line 8, in <module>
name.append(row[0])
IndexError: list index out of range
I have seen a number of such problems stemming from differnt platform line endings. Under Python 2, you might try the "universal line endings" file reading mode:
with open('data.csv', 'rU') as f:
for row in csv.reader(f):
print row
Because Excel does often use the old Mac (\r) and Windows (\r\n) standards, which can get in the csv module's way, esp. on a Unix or Mac platform where Python expects the Unix standard line ending (\n). Python 3 is generally smarter about this (and other file/string encoding issues), so generally doesn't need a special mode.
I found an answer after hours of trying. In Excel, each item in an 'empty' row contains '' for the largest number of items in any row. Python doesn't write it like that to the CSV and instead only one item on an empty row contains None. As it iterated through each row, there was no first item to add to the list on the empty rows.
I had to manually add an extra '' to the list that is wrote by the first program.
Here is my code:
import csv
with open(outputfile, 'a') as csvfile:
filewrite = csv.writer(csvfile, delimiter=',', quoting=csv.QUOTE_MINIMAL)
filewrite.writerow([hostname])
if (CiscoSyslog == 0):
filewrite.writerow(['Syslog'])
if (CiscoSNMP == 0):
filewrite.writerow(['SNMP'])
At the moment, it outputs data in a new row, as it finds it. Basically, the ultimate goal is to generate a CSV file with bunch of hostnames that are missing Syslog and/or SNMP configuration. The way I want it to look in the end is, for example:
router1,Syslog
router2,SNMP
router3,Syslog,SNMP
As you can see, at the begging I have the file open and it prints out the hostname, then, my goal at the moment is: I would like to keep the row "open", so I can keep appending more data as the "if" statements keep finding a match. And ultimately it produces something similar to the example I have shown above.
I tried researching on this forum, and other forums, but I keep finding examples of how to append to a new row in CSV, or I keep finding comments by random people, how CSV library does not support adding a new column.
Some help would be very much appreciated. Thank you!
You're going to want to construct your row entirely before trying to write it. Try this:
import csv
with open(outputfile, 'a') as csvfile:
filewrite = csv.writer(csvfile, delimiter=',', quoting=csv.QUOTE_MINIMAL)
row = [hostname]
if (CiscoSyslog == 0):
row.append('Syslog')
if (CiscoSNMP == 0):
row.append('SNMP')
filewrite.writerow(row)
The argument of cvs.writerow is a list, the list of cells.
So you can build the row before outputing it, like :
row=[]
if something:
row.append("cell1valA")
else:
row.append("cell1valB")
row.append("cell2")
row.append("cell3")
row.append("cell4")
filewrite.writerow(row)