Can you help me with this problem?
I`m new in programming and want to find out how to create a matrix, which looks like this:
matrix = {"hello":["one","two","three"],
"world": ["five","six","seven"],
"goodbye":["one","two","three"]}
I want to import a csv, which has all the strings (one, two three,...) in it and I tried with the split method, but I`m not getting there...
Another problems are the names of the categories (hello, world, goodbye)
Do you have any suggestions?
have you looked into the csv module?
https://docs.python.org/2/library/csv.html
import csv
TEST_TEXT = """\
hello,one,two,three
world,four,five,six
goodbye,one,two,three"""
TEST_FILE = TEST_TEXT.split("\n")
#file objects iterate over newlines anyway
#so this is how it would be when opening a file
#this would be the minimum needed to use the csv reader object:
for row in csv.reader(TEST_FILE):
print(row)
#or to get a list of all the rows you can use this:
as_list = list(csv.reader(TEST_FILE))
#splitting off the first element and using it as the key in a dictionary
dict_I_call_matrix = {row[0]:row[1:] for row in csv.reader(TEST_FILE)}
print(dict_I_call_matrix)
without_csv = [row.split(",") for row in TEST_FILE] #...row in TEST_TEXT.split("\n")]
matrix = [row[1:] for row in without_csv]
labels = [row[0] for row in without_csv]
Related
Im trying to read just a single cell, in order to bring in the date to use elsewhere.
Using pandas I get an error If I try to do this, generally just that the dataframe cant be read because it expects a workable dataframe and not a single cell value prior to the actual convertable dataframe far below the initial line. How can I just get the cell i.e. [A2]
CSV example
Yes, pandas is not very good at files with inconsistent format (like varying number of columns). For that purpose, I recommend you should use csv the standard library.
The code below should give you the desired value.
import csv
row = 2; col = 1 # A2 = (2,1) cell
with open("yourfile.csv") as f:
reader = csv.reader(f)
for i in range(row):
row = next(reader)
value = row[col-1] # because python index starts at zero
print(value)
Demo (using string instead of a file):
import csv
row = 2; col = 1 # A2 = (2,1) cell
input_str = ["a,b", "c,d,e", "f,g,h,i"]
reader = csv.reader(input_str)
for i in range(row):
row = next(reader)
value = row[col-1] # because python index starts at zero
print(value)
To be able to access csv like a two-dimensional object you need to convert it first to 2D python List
import csv
with open("imdb.csv") as f:
csv_as_list = list(csv.reader(f, delimiter=","))
print(csv_as_list[3][2])
The format for accessing element is csv_as_list[row_index][column_index].
Is there a better way to create a list or a numpy array from this csv file? What I'm asking is how to do it and parse more gracefully than I did in the code below.
fname = open("Computers discovered recently by discovery method.csv").readlines()
lst = [elt.strip().split(",")[8:] for elt in fname if elt != "\n"][4:]
lst2 = []
for row in lst:
print(row)
if row[0].startswith("SMZ-") or row[0].startswith("MTR-"):
lst2.append(row)
print(*lst2, sep = "\n")
You can always use Pandas. As an example,
import pandas as pd
import numpy as np
df = pd.read_csv('pandas_dataframe_importing_csv/example.csv')
To convert it, you will have to convert it to your favorite numeric type. I guess you can write the whole thing in one line:
result = numpy.array(list(df)).astype("float")
You can also do the following:
from numpy import genfromtxt
my_data = genfromtxt('my_file.csv', delimiter=',')
You can use pandas and specify header column to make it work correctly on you sample file
import pandas as pd
df = pd.read_csv('Computers discovered recently by discovery method.csv', header=2)
You can check your content using:
>>> df.head()
You can check headers using
>>> df.columns
And to convert it to numpy array you can use
>>> np_arr = df.values
It comes with a lot of options to parse and read csv files. For more information please check the docs
I am not sure what you want but try this
import csv
with open("Computers discovered recently by discovery method.csv", 'r') as f:
reader = csv.reader(f)
ll = list(reader)
print (ll)
this should read the csv line by line and store it as a list
You should never parse CSV structures manually unless you want to tackle all possible exceptions and CSV format oddities. Python has you covered in that regard with its csv module.
The main problem, in your case, stems from your data - there seems to be two different CSV structures in a single file so you first need to find where your second structure begins. Plus, from your code, it seems you want to filter out all columns before Details_Table0_Netbios_Name0 and include only rows whose Details_Table0_Netbios_Name0 starts with SMZ- or MTR-. So something like:
import csv
with open("Computers discovered recently by discovery method.csv") as f:
reader = csv.reader(f) # create a CSV reader
for row in reader: # skip the lines until we encounter the second CSV structure/header
if row and row[0] == "Header_Table0_Netbios_Name0":
break
index = row.index("Details_Table0_Netbios_Name0") # find where your columns begin
result = [] # storage for the rows we're interested in
for row in reader: # read the rest of the CSV row by row
if row and row[index][:4] in {"SMZ-", "MTR-"}: # only include these rows
result.append(row[index:]) # trim and append to the `result` list
print(result[10]) # etc.
# ['MTR-PC0BXQE6-LB', 'PR2', 'anisita', 'VALUEADDCO', 'VALUEADDCO', 'Heartbeat Discovery',
# '07.12.2017 17:47:51', '13']
should do the trick.
Sample Code
import csv
csv_file = 'sample.csv'
with open(csv_file) as fh:
reader = csv.reader(fh)
for row in reader:
print(row)
sample.csv
name,age,salary
clado,20,25000
student,30,34000
sam,34,32000
I imported my CSV File and made the data into an array. Now I was wondering, what can I do so that I'm able to print a specific value in the array? For instance if I wanted the value in the 2nd row, 2nd column.
Also how would I go about adding the two values together? Thanks.
import csv
import numpy as np
f = open("Test.csv")
csv_f = csv.reader(f)
for row in csv_f:
print(np.array(row))
f.close()
There is no need to use csv module.
This code reads csv file and prints value of cell in second row and second column. I am assuming that fields are separated by commas.
with open("Test.csv") as fo:
table = [row.split(",") for row in fo.read().replace("\r", "").split("\n")]
print table[1][1]
So, I grabbed a dataset ("Company Funding Records") from here. Then, I just rewrote a little...
#!/usr/bin/python
import csv
#import numpy as np
csvaslist = []
f = open("TechCrunchcontinentalUSA.csv")
csv_f = csv.reader(f)
for row in csv_f:
# print(np.array(row))
csvaslist.append(row)
f.close()
# Now your data is in a dict. Everything past this point is just playing
# Add together a couple of arbitrary values...
print int(csvaslist[2][7]) + int(csvaslist[11][7])
# Add using a conditional...
print "\nNow let's see what Facebook has received..."
fbsum = 0
for sublist in csvaslist:
if sublist[0] == "facebook":
print sublist
fbsum += int(sublist[7])
print "Facebook has received", fbsum
I've commented lines at a couple points to show what's being used and what was unneeded. Notice at the end that referring to a particular datapoint is simply a matter of referencing what is, effectively, original_csv_file[line_number][field_on_that_line], and then recasting as int, float, whatever you need. This is because the csv file has been changed to a list of lists.
To get specific values within your array/file, and add together:
import csv
f = open("Test.csv")
csv_f = list(csv.reader(f))
#returns the value in the second row, second column of your file
print csv_f[1][1]
#returns sum of two specific values (in this example, value of second row, second column and value of first row, first column
sum = int(csv_f[1][1]) + int(csv_f[0][0])
print sum
this is my first post but I am hoping you can tell me how to perform a calculation and insert the value within a csv data file.
For each row I want to be able to be able to take each 'uniqueclass' and sum the scores achieved in column 12. See example data below;
text1,Data,Class,Uniqueclass1,data1,data,2,data2,data3,data4,data5,175,12,data6,data7
text1,Data,Class,Uniqueclass1,data1,data,2,data2,data3,data4,data5,171,18,data6,data7
text1,Data,Class,Uniqueclass2,data1,data,4,data2,data3,data4,data5,164,5,data6,data7
text1,Data,Class,Uniqueclass2,data1,data,4,data2,data3,data4,data5,121,21.5,data6,data7
text2,Data,Class,Uniqueclass2,data1,data,4,data2,data3,data4,data5,100,29,data6,data7
text2,Data,Class,Uniqueclass2,data1,data,4,data2,data3,data4,data5,85,21.5,data6,data7
text3,Data,Class,Uniqueclass3,data1,data,3,data2,data3,data4,data5,987,35,data6,data7
text3,Data,Class,Uniqueclass3,data1,data,3,data2,data3,data4,data5,286,18,data6,data7
text3,Data,Class,Uniqueclass3,data1,data,3,data2,data3,data4,data5,003,5,data6,data7
So for instance the first Uniqueclass lasts for the first two rows. I would like to be able to therefore insert a subsquent value on that row which would be '346'(the sum of both 175 & 171.) The resultant would look like this:
text1,Data,Class,Uniqueclass1,data1,data,2,data2,data3,data4,data5,175,12,data6,data7,346
text1,Data,Class,Uniqueclass1,data1,data,2,data2,data3,data4,data5,171,18,data6,data7,346
I would like to be able to do this for each of the uniqueclass'
Thanks SMNALLY
I always like the defaultdict class for this type of thing.
Here would be my attempt:
from collections import defaultdict
class_col = 3
data_col = 11
# Read in the data
with open('path/to/your/file.csv', 'r') as f:
# if you have a header on the file
# header = f.readline().strip().split(',')
data = [line.strip().split(',') for line in f]
# Sum the data for each unique class.
# assuming integers, replace int with float if needed
count = defaultdict(int)
for row in data:
count[row[class_col]] += int(row[data_col])
# Append the relevant sum to the end of each row
for row in xrange(len(data)):
data[row].append(str(count[data[row][class_col]]))
# Write the results to a new csv file
with open('path/to/your/new_file.csv', 'w') as nf:
nf.write('\n'.join(','.join(row) for row in data))
I am trying to read the lines of a text file into a list or array in python. I just need to be able to individually access any item in the list or array after it is created.
The text file is formatted as follows:
0,0,200,0,53,1,0,255,...,0.
Where the ... is above, there actual text file has hundreds or thousands more items.
I'm using the following code to try to read the file into a list:
text_file = open("filename.dat", "r")
lines = text_file.readlines()
print lines
print len(lines)
text_file.close()
The output I get is:
['0,0,200,0,53,1,0,255,...,0.']
1
Apparently it is reading the entire file into a list of just one item, rather than a list of individual items. What am I doing wrong?
You will have to split your string into a list of values using split()
So,
lines = text_file.read().split(',')
EDIT:
I didn't realise there would be so much traction to this. Here's a more idiomatic approach.
import csv
with open('filename.csv', 'r') as fd:
reader = csv.reader(fd)
for row in reader:
# do something
You can also use numpy loadtxt like
from numpy import loadtxt
lines = loadtxt("filename.dat", comments="#", delimiter=",", unpack=False)
So you want to create a list of lists... We need to start with an empty list
list_of_lists = []
next, we read the file content, line by line
with open('data') as f:
for line in f:
inner_list = [elt.strip() for elt in line.split(',')]
# in alternative, if you need to use the file content as numbers
# inner_list = [int(elt.strip()) for elt in line.split(',')]
list_of_lists.append(inner_list)
A common use case is that of columnar data, but our units of storage are the
rows of the file, that we have read one by one, so you may want to transpose
your list of lists. This can be done with the following idiom
by_cols = zip(*list_of_lists)
Another common use is to give a name to each column
col_names = ('apples sold', 'pears sold', 'apples revenue', 'pears revenue')
by_names = {}
for i, col_name in enumerate(col_names):
by_names[col_name] = by_cols[i]
so that you can operate on homogeneous data items
mean_apple_prices = [money/fruits for money, fruits in
zip(by_names['apples revenue'], by_names['apples_sold'])]
Most of what I've written can be speeded up using the csv module, from the standard library. Another third party module is pandas, that lets you automate most aspects of a typical data analysis (but has a number of dependencies).
Update While in Python 2 zip(*list_of_lists) returns a different (transposed) list of lists, in Python 3 the situation has changed and zip(*list_of_lists) returns a zip object that is not subscriptable.
If you need indexed access you can use
by_cols = list(zip(*list_of_lists))
that gives you a list of lists in both versions of Python.
On the other hand, if you don't need indexed access and what you want is just to build a dictionary indexed by column names, a zip object is just fine...
file = open('some_data.csv')
names = get_names(next(file))
columns = zip(*((x.strip() for x in line.split(',')) for line in file)))
d = {}
for name, column in zip(names, columns): d[name] = column
This question is asking how to read the comma-separated value contents from a file into an iterable list:
0,0,200,0,53,1,0,255,...,0.
The easiest way to do this is with the csv module as follows:
import csv
with open('filename.dat', newline='') as csvfile:
spamreader = csv.reader(csvfile, delimiter=',')
Now, you can easily iterate over spamreader like this:
for row in spamreader:
print(', '.join(row))
See documentation for more examples.
Im a bit late but you can also read the text file into a dataframe and then convert corresponding column to a list.
lista=pd.read_csv('path_to_textfile.txt', sep=",", header=None)[0].tolist()
example.
lista=pd.read_csv('data/holdout.txt',sep=',',header=None)[0].tolist()
Note: the column name of the corresponding dataframe will be in the form of integers and i choose 0 because i was extracting only the first column
Better this way,
def txt_to_lst(file_path):
try:
stopword=open(file_path,"r")
lines = stopword.read().split('\n')
print(lines)
except Exception as e:
print(e)