Related
I would like to read each row of the csv file and match each word in the row with a list of strings. If any of the strings appears in the row, then write that string at the end of the line separated by comma.
The code below doesn't give me what I want.
file = 'test.csv'
read_files = open(file)
lines=read_files.read()
text_lines = lines.split("\n")
name=''
with open('testnew2.csv','a') as f:
for line in text_lines:
line=str(line)
#words = line.split()
with open('names.csv', 'r') as fd:
reader = csv.reader(fd, delimiter=',')
for row in reader:
if row[0] in line:
name=row
print(name)
f.write(line+","+name[0]+'\n')
A sample of test.csv would look like this:
A,B,C,D
ABCD,,,
Total,Robert,,
Name,Annie,,
Total,Robert,,
And the names.csv would look:
Robert
Annie
Amanda
The output I want is:
A,B,C,D,
ABCD,,,,
Total,Robert,,,Robert
Name,Annie,,,Annie
Total,Robert,,,Robert
Currently the code will get rid of lines that don't result in a match, so I got:
Total,Robert,,,Robert
Name,Annie,,,Annie
Total,Robert,,,Robert
Process each line by testing row[1] and appending the 5th column, then writing it. The name list isn't really a csv. If it's really long use a set for lookup. Read it only once for efficiency as well.
import csv
with open('names.txt') as f:
names = set(f.read().strip().splitlines())
# newline='' per Python 3 csv documentation...
with open('input.csv',newline='') as inf:
with open('output.csv','w',newline='') as outf:
r = csv.reader(inf)
w = csv.writer(outf)
for row in r:
row.append(row[1] if row[1] in names else '')
w.writerow(row)
Output:
A,B,C,D,
ABCD,,,,
Total,Robert,,,Robert
Name,Annie,,,Annie
Total,Robert,,,Robert
I think the problem is you're only writing when the name is in the row. To fix that move the writing call outside of the if conditional:
if row[0] in line:
name=row
print(name)
f.write(line+","+name[0]+'\n')
I'm guessing that print statement is for testing purposes?
EDIT: On second thought, you may need to move name='' inside the loop as well so it is reset after each row is written, that way you don't get names from matched rows bleeding into unmatched rows.
EDIT: Decided to show an implementation that should avoid the (possible) problem of two matched names in a row:
EDIT: Changed name=row and the call of name[0] in f.write() to name=row[0] and a call of name in f.write()
file = 'test.csv'
read_files = open(file)
lines=read_files.read()
text_lines = lines.split("\n")
with open('testnew2.csv','a') as f:
for line in text_lines:
name=''
line=str(line)
#words = line.split()
with open('names.csv', 'r') as fd:
reader = csv.reader(fd, delimiter=',')
match=False
while match == False:
for row in reader:
if row[0] in line:
name=row[0]
print(name)
match=True
f.write(line+","+name+'\n')
Try this as well:
import csv
myFile = open('testnew2.csv', 'wb+')
writer = csv.writer(myFile)
reader2 = open('names.csv').readlines()
with open('test.csv') as File1:
reader1 = csv.reader(File1)
for row in reader1:
name = ""
for record in reader2:
record = record.replace("\n","")
if record in row:
row.append(record)
writer.writerow(row)
break
I have a CSV file which looks like this:
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_02,983,0,Prod,983
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_03,124,0,Prod ,124
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_04,206,0,Prod,206
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_05,983,0,Prod ,983
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_06,564,0,Prod,564
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_07,189,0,Prod ,189
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_08,168,0,Prod,168
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_09,570,0,Prod ,570
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_10,189,0,Prod,189
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_11,204,0,Prod ,204
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_12,189,2,Prod,187
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_13,568,0,Prod ,568
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_14,204,0,Prod,204
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_15,142,0,Prod ,142
File,2/13/2017,domain\test_roi,root_user,ntsrv1,/vol/vol_ntsrv1_16,168,0,Prod,168
I want to add to a list the 4th column (root_user) and the 7th column (where the numbers are written). Any suggestions how?
import csv
four_col, seven_col = [], []
with open(file='test.csv', mode='r', encoding='utf-8') as csvfile:
spamreader = csv.reader(csvfile, delimiter=',', quotechar='"')
# firstline = csvfile.readline() # if csv have header uncomment it
for row in spamreader:
four_col.append(row[3])
seven_col.append(row[6])
With this csv file you can read it also setting the spamreader as:
spamreader = csv.reader(csvfile, dialect='excel')
but I wrote you the more generic way if the file don't uses commas for delimiter.
It's pretty simple this way:
fourth_column_list = []
seventh_column_list = []
with open(my_csv_file, 'r') as infile:
parsed = (x.split(',') for line in infile) # get all parsed columns
for parsed_line in parsed: # iterate over parsed lines
fourth_column_list.append(parsed_line[3]) # append 4th column
seventhth_column_list.append(parsed_line[6]) # append 7th column
I am trying to read a file with below data
Et1, Arista2, Ethernet1
Et2, Arista2, Ethernet2
Ma1, Arista2, Management1
I need to read the file replace Et with Ethernet and Ma with Management. At the end of them the digit should be the same. The actual output should be as follows
Ethernet1, Arista2, Ethernet1
Ethernet2, Arista2, Ethernet2
Management1, Arista2, Management1
I tried a code with Regular expressions, I am able to get to the point I can parse all Et1, Et2 and Ma1. But unable to replace them.
import re
with open('test.txt','r') as fin:
for line in fin:
data = re.findall(r'\A[A-Z][a-z]\Z\d[0-9]*', line)
print(data)
The output looks like this..
['Et1']
['Et2']
['Ma1']
import re
#to avoid compile in each iteration
re_et = re.compile(r'^Et(\d+),')
re_ma = re.compile(r'^Ma(\d+),')
with open('test.txt') as fin:
for line in fin:
data = re_et.sub('Ethernet\g<1>,', line.strip())
data = re_ma.sub('Management\g<1>,', data)
print(data)
This example follows Joseph Farah's suggestion
import csv
file_name = 'data.csv'
output_file_name = "corrected_data.csv"
data = []
with open(file_name, "rb") as csvfile:
reader = csv.reader(csvfile, delimiter=',')
for row in reader:
data.append(row)
corrected_data = []
for row in data:
tmp_row = []
for col in row:
if 'Et' in col and not "Ethernet" in col:
col = col.replace("Et", "Ethernet")
elif 'Ma' in col and not "Management" in col:
col = col.replace("Ma", "Management")
tmp_row.append(col)
corrected_data.append(tmp_row)
with open(output_file_name, "wb") as csvfile:
writer = csv.writer(csvfile, delimiter=',')
for row in corrected_data:
writer.writerow(row)
print data
Here are the steps you should take:
Read each line in the file
Separate each line into smaller list items using the comments as delimiters
Use str.replace() to replace the characters with the words you want; keep in mind that anything that says "Et" (including the beginning of the word "ethernet") will be replaced, so remember to account for that. Same goes for Ma and Management.
Roll it back into one big list and put it back in the file with file.write(). You may have to overwrite the original file.
I have a csv file with column A that has dates. I want to search the dates and return the corresponding row in array (VERY IMPT) format. How would I do that using python? Thank you.
excel file:
A B C
1 12202014 403 302
2 12212014 312 674
3 12222014 193 310
input:
Search csv file for 12212014
output:
[12212014,312,674]
attempt code:
date = '12212014'
with open('dates.csv', 'rt') as f:
reader = csv.reader(f, delimiter=',')
for row in reader:
if date == row[0]:
print "found the date"
How do I return the row instead of just a print statement?
The basic idea is to create a dictionary/mapping date->line and get the value by the key:
import csv
data = {}
with open('test.csv', 'r') as f:
reader = csv.reader(f)
for line in reader:
data[line[0]] = line
date_to_find = '12212014'
print data.get(date_to_find, 'Date not found')
prints:
['12212014', '312', '674']
This helps if you need the mapping afterwards to find the dates in it.
Another approach is to use generator expression to iterate over lines in a file until we find the date:
import csv
date_to_find = '12212014'
with open('test.csv', 'r') as f:
print next((line for line in csv.reader(f) if line[0] == date_to_find),
'Date not found')
prints:
['12212014', '312', '674']
Hope that helps.
There is a lot of examples of reading csv data using python, like this one:
import csv
with open('some.csv', newline='') as f:
reader = csv.reader(f)
for row in reader:
print(row)
I only want to read one line of data and enter it into various variables. How do I do that? I've looked everywhere for a working example.
My code only retrieves the value for i, and none of the other values
reader = csv.reader(csvfile, delimiter=',', quotechar='"')
for row in reader:
i = int(row[0])
a1 = int(row[1])
b1 = int(row[2])
c1 = int(row[2])
x1 = int(row[2])
y1 = int(row[2])
z1 = int(row[2])
To read only the first row of the csv file use next() on the reader object.
with open('some.csv', newline='') as f:
reader = csv.reader(f)
row1 = next(reader) # gets the first line
# now do something here
# if first row is the header, then you can do one more next() to get the next row:
# row2 = next(f)
or :
with open('some.csv', newline='') as f:
reader = csv.reader(f)
for row in reader:
# do something here with `row`
break
you could get just the first row like:
with open('some.csv', newline='') as f:
csv_reader = csv.reader(f)
csv_headings = next(csv_reader)
first_line = next(csv_reader)
You can use Pandas library to read the first few lines from the huge dataset.
import pandas as pd
data = pd.read_csv("names.csv", nrows=1)
You can mention the number of lines to be read in the nrows parameter.
Just for reference, a for loop can be used after getting the first row to get the rest of the file:
with open('file.csv', newline='') as f:
reader = csv.reader(f)
row1 = next(reader) # gets the first line
for row in reader:
print(row) # prints rows 2 and onward
From the Python documentation:
And while the module doesn’t directly support parsing strings, it can easily be done:
import csv
for row in csv.reader(['one,two,three']):
print row
Just drop your string data into a singleton list.
The simple way to get any row in csv file
import csv
csvfile = open('some.csv','rb')
csvFileArray = []
for row in csv.reader(csvfile, delimiter = '.'):
csvFileArray.append(row)
print(csvFileArray[0])
To print a range of line, in this case from line 4 to 7
import csv
with open('california_housing_test.csv') as csv_file:
data = csv.reader(csv_file)
for row in list(data)[4:7]:
print(row)
I think the simplest way is the best way, and in this case (and in most others) is one without using external libraries (pandas) or modules (csv). So, here is the simple answer.
""" no need to give any mode, keep it simple """
with open('some.csv') as f:
""" store in a variable to be used later """
my_line = f.nextline()
""" do what you like with 'my_line' now """