So I'm trying to combine two different csv files into a single one and I've done that. The two csv files are of students in school who are present in 1 and absent in another.
I need to put the date the file was created at the top of the new csv and have each grade of the present students on a new page or after 3 blank rows.
Also on each new page or after each 3 blanks i want to have the name or the teacher, the date on which the file was created and the grade.
import csv
with open('inschool.csv', encoding="cp437") as f:
reader = csv.reader(f)
in_school = list(reader)
with open('notinschool.csv', encoding="cp437") as f:
reader = csv.reader(f)
not_in_school = list(reader)
for grade, name, status, hr_teacher in not_in_school:
print(grade, name, status, hr_teacher)
for grade, name, status, hr_teacher in in_school:
print(grade, name, status, hr_teacher)
iFile = open('inschool.csv', encoding="cp437")
reader = csv.reader(iFile)
IFILE = open('notinschool.csv', encoding="cp437")
READER = csv.reader(IFILE)
oFile = open('combined.csv','wt',encoding="cp437")
writer = csv.writer(oFile, delimiter='|', quoting=csv.QUOTE_ALL)
for row in READER:
writer.writerow(row)
writer.writerow("[]")
for row in reader:
writer.writerow(row)
writer.writerow("[]")
The code which i tried for the 3 blank rows had this ending but it gave 3 blank rows/lines after each students name instead of after each grade.
iFile = open('Inschool.csv',)
reader = csv.reader(iFile)
IFILE = open('notinschool.csv')
READER = csv.reader(IFILE)
oFile = open('combined.csv','wb')
writer_a = csv.writer(oFile, delimiter='|', quoting=csv.QUOTE_ALL)
writer_b = csv.writer(oFile, delimiter='|', quoting=csv.QUOTE_ALL, lineterminator="\n\n\n\n")
for row in READER:
writer_a.writerow(row)
writer_b.writerow([])
for row in reader:
writer_b.writerow(row)
I would appreciate it if someone could help me. Thanks.
You can do it really easy in the terminal. Just cd to the directory and do the command cat inschool.csv notinschool.csv > combined.csv
If you want to do it in Python I would do:
in_file1 = open("inschool.csv","r").read().split("\n")
in_file2 = open("notinschool.csv","r").read().split("\n")
out_file = open("combined.csv","w")
for line in in_file1:
if line:
out_file.write(line + "\n")
for line in in_file2:
if line:
out_file.write(line + "\n")
reading files the way above isn't the most efficient, but if they are small it doesnt really matter and it's easier to visualize what's happening. you can use your input file method with this b/c the concept stays the same :)
I just got into using this module called pandas and it is for DataFrames. They are much easier to use, process, navigate through, and merge than parsing text files.
Related
I started learning python and was wondering if there was a way to create multiple files from unique values of a column. I know there are 100's of ways of getting it done through pandas. But I am looking to have it done through inbuilt libraries. I couldn't find a single example where its done through inbuilt libraries.
Here is the sample csv file data:
uniquevalue|count
a|123
b|345
c|567
d|789
a|123
b|345
c|567
Sample output file:
a.csv
uniquevalue|count
a|123
a|123
b.csv
b|345
b|345
I am struggling with looping on unique values in a column and then print them out. Can someone explain with logic how to do it ? That will be much appreciated. Thanks.
import csv
from collections import defaultdict
header = []
data = defaultdict(list)
DELIMITER = "|"
with open("inputfile.csv", newline="") as csvfile:
reader = csv.reader(csvfile, delimiter=DELIMITER)
for i, row in enumerate(reader):
if i == 0:
header = row
else:
key = row[0]
data[key].append(row)
for key, value in data.items():
filename = f"{key}.csv"
with open(filename, "w", newline="") as f:
writer = csv.writer(f, delimiter=DELIMITER)
rows = [header] + value
writer.writerows(rows)
import csv
with open('sample.csv', newline='') as csvfile:
reader = csv.reader(csvfile)
for row in reader:
with open(f"{row[0]}.csv", 'a') as inner:
writer = csv.writer(
inner, delimiter='|',
fieldnames=('uniquevalue', 'count')
)
writer.writerow(row)
the task can also be done without using csv module. the lines of the file are read, and with read_file.read().splitlines()[1:] the newline characters are stripped off, also skipping the header line of the csv file. with a set a unique collection of inputdata is created, that is used to count number of duplicates and to create the output files.
with open("unique_sample.csv", "r") as read_file:
items = read_file.read().splitlines()[1:]
for line in set(items):
with open(line[:line.index('|')] + '.csv', 'w') as output:
output.write((line + '\n') * items.count(line))
def usunPsa(self, ImiePsa):
with open('schronisko.csv', 'rb') as input, open('schronisko.csv', 'wb') as output:
writer = csv.writer(output)
for row in csv.reader(input):
if row[0] == ImiePsa:
writer.writerow(row)
with open(self.plik, 'r') as f:
print(f.read())
Dsac;Chart;2;2020-11-04
Dsac;Chart;3;2020-11-04
Dsac;Chart;4;2020-11-04
Lala;Chart;4;2020-11-04
Sda;Chart;4;2020-11-04
Sda;X;4;2020-11-04
Sda;Y;4;2020-11-04
pawel;Y;4;2020-11-04`
If I use usunPsa("pawel") every line gets removed.
Following code earse my whole csv file instead only one line with given ImiePsa,
What may be the problem there?
I found the problem. row[0] in your code returns the entire row, that means the lines are not parsed correctly. After a bit of reading, I found that csv.reader has a parammeter called delimiter to sepcify the delimiter between columns.
Adding that parameter solves your problem, but not all problems though.
The code that worked for me (just in case you still want to use your original code)
import csv
def usunPsa(ImiePsa):
with open('asd.csv', 'rb') as input, open('schronisko.csv', 'wb') as output:
writer = csv.writer(output)
for row in csv.reader(input, delimiter=';'):
if row[0] == ImiePsa:
writer.writerow(row)
usunPsa("pawel")
Notice that I changed the output filename. If you want to keep the filename the same however, you have to use Hamza Malik's answer.
Just read the csv file in memory as a list, then edit that list, and then write it back to the csv file.
lines = list()
members= input("Please enter a member's name to be deleted.")
with open('mycsv.csv', 'r') as readFile:
reader = csv.reader(readFile)
for row in reader:
lines.append(row)
for field in row:
if field == members:
lines.remove(row)
with open('mycsv.csv', 'w') as writeFile:
writer = csv.writer(writeFile)
writer.writerows(lines)
I'm new to python and I struggling with this code. Have 2 file, 1st file is text file containing email addresses (one each line), 2nd file is csv file with 5-6 columns. Script should take search input from file1 and search in file 2, the output should be stored in another csv file (only first 3 columns) see example below. Also I have copied a script that I was working on. If there is a better/efficient script then please let me know. Thank you, appreciate your help.
File1 (output.txt)
rrr#company.com
eee#company.com
ccc#company.com
File2 (final.csv)
Sam,Smith,sss#company.com,admin
Eric,Smith,eee#company.com,finance
Joe,Doe,jjj#company.com,telcom
Chase,Li,ccc#company.com,IT
output (out_name_email.csv)
Eric,Smith,eee#company.com
Chase,Li,ccc#company.com
Here is the script
import csv
outputfile = 'C:\\Python27\\scripts\\out_name_email.csv'
inputfile = 'C:\\Python27\\scripts\\output.txt'
datafile = 'C:\\Python27\\scripts\\final.csv'
names=[]
with open(inputfile) as f:
for line in f:
names.append(line)
with open(datafile, 'rb') as fd, open(outputfile, 'wb') as fp_out1:
writer = csv.writer(fp_out1, delimiter=",")
reader = csv.reader(fd, delimiter=",")
headers = next(reader)
for row in fd:
for name in names:
if name in line:
writer.writerow(row)
Load the emails into a set for O(1) lookup:
with open(inputfile) as fin:
emails = set(line.strip() for line in fin)
Then loop over the rows once, and check it exists in emails - no need to loop over each possible match for each row:
# ...
for row in reader:
if row[1] in emails:
writer.writerow(row)
If you're not doing anything else, then you can make it:
writer.writerows(row for row in reader if row[1] in emails)
A couple of notes, in your original code you're not using the csv.reader object reader - you're looping over fd and you appear to have some naming issues with names and line and row...
I have 2 files named input.csv (composed of one column count ) and output.csv (composed of one column id).
I want to paste my count column in output.csv, just after the id column.
Here is my snippet :
with open ("/home/julien/input.csv", "r") as csvinput:
with open ("/home/julien/excel/output.csv", "a") as csvoutput:
writer = csv.writer(csvoutput, delimiter = ";")
for row in csv.reader(csvinput, delimiter = ";"):
if row[0] != "":
result = row[0]
else:
result = ""
row.append(result)
writer.writerow(row)
But it doesn't work.
I've been searching the problem for many hours but I'v got no solution. Would you have any tricks to solve my problem ?
Thanks! Julien
You need to work with three files, two for reading and one for writing.
This should work.
import csv
in_1_name = "/home/julien/input.csv"
in_2_name = "/home/julien/excel/output.csv"
out_name = "/home/julien/excel/merged.csv"
with open(in_1_name) as in_1, open(in_2_name) as in_2, open(out_name, 'w') as out:
reader1 = csv.reader(in_1, delimiter=";")
reader2 = csv.reader(in_2, delimiter=";")
writer = csv.writer(out, delimiter=";")
for row1, row2 in zip(reader1, reader2):
if row1[0] and row2[0]:
writer.writerow([row1[0], row2[0]])
You write the row for each column:
row.append(result)
writer.writerow(row)
Dedent the last line to write only once:
row.append(result)
writer.writerow(row)
Open both files for input.
Open a new file for output.
In a loop, read a line from each, formatting an output line, which is then written to the output file
close all the files
Programmatically copy your output file on top of the input file
"output.csv".
Done
If anyone was given two tables, merging them by using first column of each is very easy. With my library pyexcel, you do the merge just like merging tables:
>>> from pyexcel import Reader,Writer
>>> f1=Reader("input.csv", delimiter=';')
>>> f2=Reader("output.csv", delimiter=';')
>>> columns = [f1.column_at(0), f2.column_at(0)]
>>> f3=Writer("merged.csv", delimiter=';')
>>> f3.write_columns(columns)
>>> f3.close()
I have no knowledge of python.
What i want to be able to do is create a script that will edit a CSV file so that it will wrap every field in column 3 around quotes. I haven't been able to find much help, is this quick and easy to do? Thanks.
column1,column2,column3
1111111,2222222,333333
This is a fairly crude solution, very specific to your request (assuming your source file is called "csvfile.csv" and is in C:\Temp).
import csv
newrow = []
csvFileRead = open('c:/temp/csvfile.csv', 'rb')
csvFileNew = open('c:/temp/csvfilenew.csv', 'wb')
# Open the CSV
csvReader = csv.reader(csvFileRead, delimiter = ',')
# Append the rows to variable newrow
for row in csvReader:
newrow.append(row)
# Add quotes around the third list item
for row in newrow:
row[2] = "'"+str(row[2])+"'"
csvFileRead.close()
# Create a new CSV file
csvWriter = csv.writer(csvFileNew, delimiter = ',')
# Append the csv with rows from newrow variable
for row in newrow:
csvWriter.writerow(row)
csvFileNew.close()
There are MUCH more elegant ways of doing what you want, but I've tried to break it down into basic chunks to show how each bit works.
I would start by looking at the csv module.
import csv
filename = 'file.csv'
with open(filename, 'wb') as f:
reader = csv.reader(f)
for row in reader:
row[2] = "'%s'" % row[2]
And then write it back in the csv file.