I need to read each csv file from a predefined dir, for each csv in that dir I need to take each row and write it to a new csv file.
Currently, I have this code snippet that reads a specific csv file and loops on each row.
import csv
with open('E:\EE\EE\TocsData\CSAT\csat_20140331.csv', 'rb') as csvfile:
reader = csv.reader(csvfile, delimiter=',', quotechar='|')
for row in reader:
# write row to a seperate csv file
Try this:
import glob
import csv
with open('somefile.out', 'wb') as out:
writer = csv.writer(out, delimiter=',')
for inf in glob.glob(r'E:\EE\EE\TocsData\CSAT\*.csv'):
with open(inf, 'rb') as csv_input:
reader = csv.reader(csv_input, delimiter=',', quotechar='|')
for row in reader:
writer.write(row)
Something similar to this should work. Loop through all files in the directory, read from each and write all rows to the new file.
myDirectory = "E:\EE\EE\TocsData\CSAT\\"
myNewCSV= "E:\myNewCSV.csv"
with open(myNewCSV,'ab') as w: #append bytes:'ab', write bytes:'wb' - whichever you want.
for myFile in os.listdir(myDirectory):
absolutePath = myDirectory + str(myFile)
writer = csv.writer(w)
with open(absolutePath, 'rb') as r:
reader = csv.reader(r)
for row in reader:
writer.writerow(row)
Related
How can I remove a row from csv file. This is the code and I want to delete row 1 of my csv file. I added del row[1] but it does not do anything. The program runs without error but does not delete row 1.
import csv
with open('grades.csv', 'r') as file:
grades_reader = csv.reader(file, delimiter=',')
row_num = 1
for row in grades_reader:
print('Row #{}:'.format(row_num), row)
row_num += 1
del row[1]
One approach is to write the content to a temp file and then rename it
Ex:
import csv
import os
with open('grades.csv', 'r') as file, open('grades_out.csv', 'w', newline='') as outfile:
grades_reader = csv.reader(file, delimiter=',')
grades_reader_out = csv.writer(outfile, delimiter=',')
header = next(grades_reader) # Header
next(grades_reader) # Skip first row
grades_reader_out.writerow(header) #writer Header
for row in grades_reader:
grades_reader_out.writerow(row)
# Rename
os.rename(..., ...)
I'm sure this is a really easy question but I can't seem to find any information on it.
I have a very large CSV file which I need to insert a row directly after the header which helps with another code that reads the csv and joins it to a parcel shapefile.
I have the code to append the row of data that I want, but it will only go to the last line. I cannot figure out how to get the code to insert my row immediately after the header row. Here is my code:
import os
import csv
insert_row = '"AAAAAAAAAAAAAAAAAAA","**********","**********","**********","**********","**********","**********","**","**********","**********","****","**********",999999,9999,00'
os.chdir(r"D:\PROPERTY\PINELLAS\Data_20201001_t")
with open("owner_mail.csv", 'r') as csv_file, open("owner_mail.csv", 'a', newline = "") as new_file:
csv_reader = csv.reader(csv_file)
csv_writer = csv.writer(new_file)
csv_writer.writerow(insert_row)
So that's it. I just need the insert_row line of data to be in row position number 2 instead of at the end of the file. Thank you.
You can't just insert a row in the middle of a file unless replacing data of exactly the same length. You have to read the entire file, edit it, and re-write it.
Something like this should work:
import csv
# This must be an iterable not a string
insert_row = "AAAAAAAAAAAAAAAAAAA","**********","**********","**********","**********","**********","**********","**","**********","**********","****","**********",999999,9999,00
with open("owner_mail.csv", 'r') as csv_file, open("owner_mail_updated.csv", 'w', newline = "") as new_file:
csv_reader = csv.reader(csv_file)
csv_writer = csv.writer(new_file)
header = next(csv_reader)
csv_writer.writerow(header)
csv_writer.writerow(insert_row)
for line in csv_reader:
csv_writer.writerow(line)
If the CSV file is not too large to fit entirely in memory than you can read all the lines at once, edit them, and write them back out to the same file. It's riskier if there is a problem. Safer to write to a new file, then delete original and rename if no errors:
import csv
# This must be an iterable not a string
insert_row = "AAAAAAAAAAAAAAAAAAA","**********","**********","**********","**********","**********","**********","**","**********","**********","****","**********",999999,9999,00
with open("owner_mail.csv", 'r') as csv_file:
rows = list(csv.reader(csv_file))
rows.insert(1,insert_row) # insert after header row
with open("owner_mail.csv", 'w') as csv_file:
w = csv.writer(csv_file)
w.writerows(rows)
Please try this:
import os
import csv
insert_row = '"AAAAAAAAAAAAAAAAAA","**********","**********","**********","**********","**********","**********","**","**********","**********","****","**********",999999,9999,00'
with open("owner_mail.csv", 'r') as csv_file, open("owner_mail.csv", 'w') as new_file:
csv_reader = csv.reader(csv_file)
reader = list(csv_reader)
reader.insert(1,insert_row)
csv_writer = csv.writer(new_file)
csv_writer.writerows(reader)
I have a database of tweets in csv format which looks like this - screen of csv database - and I need to perform the following task with this file using Python code:
Search for certain words/phrases in tweets (text of a tweet is in the column C) and if the tweet has this word/phrase I'm looking for, I need to write the whole row with this tweet to a new csv file
and (if possible) to delete this tweet from the old csv file or create a new one without it.
I hope I made it clear.
You can use the csv library to read the file and do your search on each row one at a time.
import csv
with open('out.csv', 'w', newline='') as output_file:
writer = csv.writer(output_file, delimiter=',', quotechar='|', quoting=csv.QUOTE_MINIMAL)
with open('test.csv') as csv_file:
csv_read = csv.reader(csv_file, delimiter=',')
for row in csv_read:
if "a" in row[2]:
writer.writerow(row)
Here is a link to the python docs: https://docs.python.org/3/library/csv.html. Hope this helps.
EDIT: If you want more than one search term, use any() on a list comprehension.
import csv
with open('out.csv', 'w', newline='') as output_file:
writer = csv.writer(output_file, delimiter=',', quotechar='|', quoting=csv.QUOTE_MINIMAL)
with open('input.csv') as csv_file:
csv_read = csv.reader(csv_file, delimiter=',')
for row in csv_read:
search_terms = ["term1", "term2"]
if any([term in row[2] for term in search_terms]):
writer.writerow(row)
I have a csv with two columns of data. I want to extract data from one column and write to a text file with single-quote on each element and separated by a comma. For example, I have this..
taxable_entity_id,id
45efc167-9254-406c-b5a8-6aef91a73dd9,331999
5ae97680-f489-4182-9dcb-eb07a73fab15,103507
00018d93-ae71-4367-a0da-f252cea4dfa2,32991
I want all the taxable_entity_ids in a text file like this
'45efc167-9254-406c-b5a8-6aef91a73dd9','5ae97680-f489-4182-9dcb-eb07a73fab15','00018d93-ae71-4367-a0da-f252cea4dfa2'
without any space between two elements, separated by a comma.
Edit:
This is what i tried..
import csv
with open("Taxable_entity_those_who_filed_G1_M_July_but_not_in_Aug.csv", 'r') as csv_File:
reader = csv.DictReader(csv_File)
with open("te_id.csv", 'w') as text_file:
writer = csv.writer(text_file, quotechar='\'', quoting=csv.QUOTE_MINIMAL)
for row in reader:
writer.writerow(row["taxable_entity_id"])
# print(row["taxable_entity_id"])
text_file.close()
csv_File.close()
and this is what I got..
4,5,e,f,c,1,6,7,-,9,2,5,4,-,4,0,6,c,-,b,5,a,8,-,6,a,e,f,9,1,a,7,3,d,d,9
5,a,e,9,7,6,8,0,-,f,4,8,9,-,4,1,8,2,-,9,d,c,b,-,e,b,0,7,a,7,3,f,a,b,1,5
0,0,0,1,8,d,9,3,-,a,e,7,1,-,4,3,6,7,-,a,0,d,a,-,f,2,5,2,c,e,a,4,d,f,a,2
You were close. Simply as you want one single line in the output file, you should write it at once by using a comprehension:
import csv
with open("Taxable_entity_those_who_filed_G1_M_July_but_not_in_Aug.csv", 'r') as csv_File:
reader = csv.DictReader(csv_File)
with open("te_id.csv", 'w') as text_file:
# use QUOTE_ALL to force the quoting
writer = csv.writer(text_file, quotechar='\'', quoting=csv.QUOTE_ALL)
writer.writerow((row["taxable_entity_id"] for row in reader))
And do not use close as you have (correctly) used with.
try that
import pandas as pd
df = pd.read_csv('nameoffile.csv',delimiter = ',')
X = df[0].values
f = open('newfile.txt','w')
for i in X:
f.write(X[i] + ',')
f.close()
It's seems a little odd that you basically want a one row csv file for the taxable_entity_ids, but certain possible. You also don't need to explicitly close() the open files because the with context manager will do it for you automatically.
You also need to open the CSV file with newline='' as shown in all the examples in the csv module's documentation.
Lastly, if you want the all the fields to be quoted you need to use quoting=csv.QUOTE_ALL instead of quoting=csv.QUOTE_MINIMAL.
import csv
inp_filename = "Taxable_entity_those_who_filed_G1_M_July_but_not_in_Aug.csv"
outp_filename = "te_id.csv"
with open(outp_filename, 'w', newline='') as text_file, \
open(inp_filename, 'r', newline='') as csv_File:
reader = csv.DictReader(csv_File)
writer = csv.writer(text_file, quotechar="'", quoting=csv.QUOTE_ALL)
taxable_entity_ids = (row["taxable_entity_id"] for row in reader)
writer.writerow(taxable_entity_ids)
print('done')
How would I go about correcting this code, so that I can view the contents of the CSV?
import csv
def csv_to_list("jo.csv", delimiter=','):
with open("jo.csv", 'r') as csv_con:
reader = csv.reader(csv_con, delimiter=delimiter)
return list(reader)
I don't know what you are trying to do but the proper usage of csv.reader is:
import csv
with open("jo.csv", 'r') as csv_con:
reader = csv.reader(csv_con, delimiter=delimiter)
for row in reader:
# Process rows here
print(', '.join(row))
One of the goals of csv.reader is not to load the whole file in the reader but to access it row by row.