I am very new to Python, I need to read numbers from a file and store them in a matrix like I would do it in fortran or C;
for i
for j
data[i][j][0]=read(0)
data[i][j][1]=read(1)
data[i][j][2]=read(2)
...
...
How can I do the same in Python? I read a bit but got confused with tuples and similar things
If you could point me to a similar example it would be great
thanks
Python doesn't come with multi-dimensional arrays, though you can add them through the popular numpy third-party package. If you want to avoid third-party packages, what you would do in Python would be to use a list of lists of lists (each "list" being a 1-D "vector-like" sequence, which can hold items of any type).
For example:
data = [ [ [0 for i in range(4)] for j in range(5)] for k in range(6)]
this makes a list of 6 items which are lists of 5 items which are lists of 4 0's -- i.e., a 6 x 5 x 4 "3D matrix" which you could then address the way you want,
for i in range(6):
for j in range(5):
data[i][j][0]=read(0)
data[i][j][1]=read(1)
data[i][j][2]=read(2)
to initialize the first three of the four items on each most-nested sublist with calls to that mysterious function read which presumably you want to write yourself (I have no idea what it's supposed to do -- not "read and return the next number" since it takes a mysterious argument, but, then what?).
It depends on your file format, but take a look on:
Link
and
http://docs.scipy.org/doc/scipy/reference/tutorial/io.html
You may want to use numpy and use the built in function for using I/O, in particular loadtxt.
http://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html
There are a lot of addictional functions to handle I/O:
http://docs.scipy.org/doc/numpy/reference/routines.io.html
A simple example would be:
data = []
with open(_filename_, 'r') as f:
for line in f:
data.append([int(x) for x in line.split()])
A way to extend the list in a form to work like matrix. I have gone through other codes of matrix in python, all are using comprehensions to first initialize a list of required size and then update the values (which takes a little more time).Let the r represents row and c for column.
r = input('Enter row size: ')
c = input('Enter column size: ')
m = []
for i in range(r):
m.append([])
for j in range(c):
m[i].append(input())
for i in m:
print i
Here, you can input the elements of matrix as it was in 'C' or equivalent languages. Hope this may help someone a different view of implementing matrices.
Matrices are two dimensional structures. In plain Python, the most natural representation of a matrix is as a list of lists.
So, you can write a row matrix as:
[[1, 2, 3, 4]]
And write a column matrix as:
[[1],
[2],
[3],
[4]]
This extends nicely to m x n matrices as well:
[[10, 20],
[30, 40],
[50, 60]]
See matfunc.py for an example of how to develop a full matrix package in pure Python.
The documentation for it is here.
And here is a worked-out example of doing matrix multiplication in plain python using a list-of-lists representation:
>>> from pprint import pprint
>>> def mmul(A, B):
nr_a, nc_a = len(A), len(A[0])
nr_b, nc_b = len(B), len(B[0])
if nc_a != nr_b:
raise ValueError('Mismatched rows and columns')
return [[sum(A[i][k] * B[k][j] for k in range(nc_a))
for j in range(nc_b)] for i in range(nr_a)]
>>> A = [[1, 2, 3, 4]]
>>> B = [[1],
[2],
[3],
[4]]
>>> pprint(mmul(A, B))
[[30]]
>>> pprint(mmul(B, A), width=20)
[[1, 2, 3, 4],
[2, 4, 6, 8],
[3, 6, 9, 12],
[4, 8, 12, 16]]
As another respondent mentioned, if you get serious about doing matrix work, it would behoove you to install numpy which has direct support for many matrix operations:
Related
I have a list of lists, where each inner list represents a row in a spreadsheet. With my current data structure, how can I perform an operation on each element on an inner list with the same index ( which amounts to basically performing operations down a column in a spreadsheet.)
Here is an example of what I am looking for (in terms of addition)
>>> lisolis = [[1,2,3], [4,5,6], [7,8,9]]
>>> sumindex = [1+4+7, 2+5+8, 3+6+9]
>>> sumindex = [12, 15, 18]
This problem can probably be solved with slicing, but I'm unable to see how to do that cleanly. Is there a nifty tool/library out there that can accomplish this for me?
Just use zip:
sumindex = [sum(elts) for elts in zip(*lisolis)]
#tzaman has a good solution for lists, but since you have also put numpy in the tags, there's an even simpler solution if you have a numpy 2D array:
>>> inport numpy
>>> a = numpy.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
>>> a.sum(axis=0)
array([12, 15, 18])
This should be faster if you have large arrays.
>>> sumindex = numpy.array(lisolis).sum(axis=0).tolist()
>>import pandas as pd
>>df = pd.DataFrame([[1,2,3], [4,5,6], [7,8,9]], columns=['A','B','C'])
>>df.sum()
A 12
B 15
C 18
The list(), map(), zip(), and sum() functions make short work of this problem:
>>> list(map(sum, zip(*lisolis)))
[12, 15, 18]
First I'll fill out the lists of what we want to add up:
>>> lisolis = [[1,2,3], [4,5,6], [7,8,9]]
Then create an empty list for my sums:
>>> sumindex = []
Now I'll start a loop inside a loop. I'm going to add the numbers that are in same positions of each little list (that's the "y" loop) and I'm going to do that for each position (that's the "x" loop).
>>> for x in range(len(lisolis[0])):
z = 0
for y in range(len(lisolis)):
z += lisolis[y][x]
sumindex.append(z)
range(len(lisolis[0])) gives me the length of the little lists so I know how many positions there are to add up, and range(len(lisolis)) gives me the amount of little lists so I know how many numbers need to be added up for any particular position in the list.
"z" is a placecard holder. Each number in a particular list position is going to be added to "z" until they're summed up. After that, I'll put the value of "z" into the next slot in sumindex.
To see the results, I would then type:
>>>sumindex
which would give me:
[12, 15, 18]
Apologies if this has already been asked, but I searched quite a bit and couldn't find quite the right solution. I'm new to python, but I'll try to be as clear as possible. In short, I have a list of arrays in the following format resulting from a joining a multiprocessing pool:
array = [[[1,2,3], 5, 47, 2515],..... [[4,5,6], 3, 35, 2096]]]
and I want to get all values from the first array element to form a new array in the following form:
print(new_array)
[1,2,3,4,5,6]
In my code, I was trying to get the first value through this function:
new_array = array[0][0]
but this only returns the first value as such:
print(new_array)
[1,2,3]
I also tried np.take after converting the array into a np array:
array = np.array(array)
new_array = np.take(results,0)
print(new_array)
[1,2,3]
I have tried a number of np functions (concatenate, take, etc.) to try and iterate this over the list, but get back the following error (presumably because the size of the array changes):
ValueError: autodetected range of [[], [1445.0, 1445.0, -248.0, 638.0, -108.0, 649.0]] is not finite
Thanks for any help!
You can achieve it without numpy using reduce:
from functools import reduce
l = [[[1,2,3], 5, 47, 2515], [[4,5,6], 3, 35, 2096]]
res = reduce(lambda a, b: [*a, *b], [x[0] for x in l])
Output
[1, 2, 3, 4, 5, 6]
Maybe it is worth mentioning that [*a, *b] is a way to concatenate lists in python, for example:
[*[1, 2, 3], *[4, 5, 6]] # [1, 2, 3, 4, 5, 6]
You could also use itertools' chain() function to flatten an extraction of the first subArray in each element of the list:
from itertools import chain
result = list(chain(*[sub[0] for sub in array]))
I am working through some code trying to understand some Python mechanics, which I just do not get. I guess it is pretty simple and I also now, what it does, but i do not know how it works. I understand the normal use of for-loops but this here... I do not know.
Remark: I know some Python, but I am not an expert.
np.array([[[S[i,j]] for i in range(order+1)] for j in range(order+1)])
The second piece of code, I have problems with is this one:
for i in range(len(u)):
for j in range(len(v)):
tmp+=[rm[i,j][k]*someFuction(name,u[i],v[j])[k] for k in range(len(rm[i,j])) if rm[i,j][k]]
How does the innermost for-loop work? And also what does the if do here?
Thank you for your help.
EDIT: Sorry that the code is so unreadable, I just try to understand it myself. S, rm are numpy matrices, someFunction returns an array with scalar entries, andtmp is just a help variable
There are quite a few different concepts inside your code. Let's start with the most basic ones. Python lists and numpy arrays have different methodologies for indexation. Also you can build a numpy array by providing it a list:
S_list = [[1,2,3], [4,5,6], [7,8,9]]
S_array = np.array(S_list)
print(S_list)
print(S_array)
print(S_list[0][2]) # indexing element 2 from list 0
print(S_array[0,2]) # indexing element at position 0,2 of 2-dimensional array
This results in:
[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
[[1 2 3]
[4 5 6]
[7 8 9]]
3
3
So for your first line of code:
np.array([[[S[i,j]] for i in range(order+1)] for j in range(order+1)])
You are building a numpy array by providing it a list. This list is being built with the concept of list comprehension. So the code inside the np.array(...) method:
[[[S[i,j]] for i in range(order+1)] for j in range(order+1)]
... is equivalent to:
order = 2
full_list = []
for j in range(order+1):
local_list = []
for i in range(order+1):
local_list.append(S_array[i, j])
full_list.append(local_list)
print(full_list)
This results in:
[[1, 4, 7], [2, 5, 8], [3, 6, 9]]
As for your second snippet its important to notice that although typically numpy arrays have very specific and constant (for all the array) cell types you can actually give the data type object to a numpy array. So creating a 2-dimensional array of lists is possible. It is also possible to create a 3-dimensional array. Both are compatible with the indexation rm[i,j][k]. You can check this in the following example:
rm = np.array(["A", 3, [1,2,3]], dtype="object")
print(rm, rm[2][0]) # Acessing element 0 of list at position 2 of the array
rm2 = np.zeros((3, 3, 3))
print(rm2[0, 1][2]) # This is also valid
The following code:
[rm[i,j][k]*someFuction(name,u[i],v[j])[k] for k in range(len(rm[i,j])) if rm[i,j][k]]
... could be written as such:
some_list = []
for k in range(len(rm[i,j])):
if rm[i, j][k]: # Expecting a boolean value (or comparable)
a_list = rm[i,j][k]*someFuction(name,u[i],v[j])
some_list.append(a_list[k])
The final detail is the tmp+=some_list. When you sum two list they'll be concatenated as can been seen in this simple example:
tmp = []
tmp += [1, 2, 3]
print(tmp)
tmp += [4, 5, 6]
print(tmp)
Which results in this:
[1, 2, 3]
[1, 2, 3, 4, 5, 6]
Also notice that multiplying a list by a number will effectively be the same as summing the list several times. So 2*[1,2] will result in [1,2,1,2].
Its a list comprehension, albeit a pretty unreadable one. That was someome doing something very 'pythonic' in spite of readablity. Just look up list comprehensions and try to rewrite it yourself as a traditional for loop. list comprehensions are very useful, not sure I would have gone that route here.
The syntax for a list comprehension is
[var for var in iterable if optional condition]
So this bottom line can be rewritten like so:
for k in range(len(rm[i,j]):
if rm[i,j][k]:
tmp+= rm[i,j][k]*someFunction(name,u[i],v[j])[k]
I know that list aliasing is an issue in Python, but I can't figure out a way around it.
def zeros(A):
new_mat = A
for i in range(len(A)):
for j in range(len(A[i])):
if A[i][j]==0:
for b in range(len(A)):
new_mat[b][j] = 0
else:
new_mat[i][j] = A[i][j]
return A
Even though I don't change the values of A at all, when I return A, it is still modified:
>>> Matrix = [[1,2,3],[5,0,78],[7,3,45]]
>>> zeros(Matrix)
[[1, 0, 3], [5, 0, 78], [7, 0, 45]]
Is this list aliasing? If so, how do you modify elements of a 2D array without aliasing occurring? Thanks so muhc <3.
new_mat = A does not create a new matrix. You have merely given a new name to the object you also knew as A. If it's a list of lists of numbers, you might want to use copy.deepcopy to create a full copy, and if it's a numpy array you can use the copy method.
new_mat = A[:]
This creates a copy of the list instead of just referencing it. Give it a try.
[:] just specifies a slice from beginning to end. You could have [1:] and it would be from element 1 to the end, or [1:4] and it would be element 1 to 4, for instance. Check out list slicing regarding that.
This might help someone else. just do this.
import copy
def zeros(A):
new_mat = copy.deepcopy(A)
QUICK EXPLANATION
If you want A to be the original matrix your function should be like this. Just use np.copy() function
def zeros(A):
new_mat = np.copy(A) #JUST COPY IT WITH np.copy() function
for i in range(len(A)):
for j in range(len(A[i])):
if A[i][j]==0:
for b in range(len(A)):
new_mat[b][j] = 0
else:
new_mat[i][j] = A[i][j]
return A
THOROUGH EXPLANATION
Let's say we don't want numpy ndarray a to have aliasing. For example, we want to prevent a to change any of its values when we take (for example) a slice of it, we assign this slice to b and then we modify one element of b. We want to AVOID this:
import numpy as np
a = np.array([[1,2,3],[4,5,6],[7,8,9]])
b = a[0]
b[2] = 50
a
Out[]:
array([[ 1, 2, 50],
[ 4, 5, 6],
[ 7, 8, 9]])
Now you might think that treating a numpy-ndarray object as if it was a list object might solve our problem. But it DOES NOT WORK either:
%reset #delete all previous variables
a = np.array([[1,2,3],[4,5,6],[7,8,9]])
b = a[:][0] #this is how you would have copied a slice of "a" if it was a list
b[2] = 50
a
Out[]:
array([[ 1, 2, 50], #problem persists
[ 4, 5, 6],
[ 7, 8, 9]])
The problem gets solved, in this case, if you think of numpy arrays as different objects than python lists. In order to copy numpy arrays you can't do copy = name_array_to_be_copied[:], but copy = np.copy(name_array_to_be_copied). Therefore, this would solve our aliasing problem:
%reset #delete all previous variables
import numpy as np
a = np.array([[1,2,3],[4,5,6],[7,8,9]])
b = np.copy(a)[0] #this is how you copy numpy ndarrays. Calling np.copy() function
b[2] = 50
a
Out[]:
array([[1, 2, 3], #problem solved
[4, 5, 6],
[7, 8, 9]])
P.S. Watch out with the zeros() function. Even after fixing the aliasing issue, your function does not convert to 0 the columns in the new_matrix who have at least one zero in the same column for the matrix A (this is what I think you wanted to acomplish by seeing the incorrectly reported output of your function [[1, 0, 3], [5, 0, 78], [7, 0, 45]], since it actually yields [[1,0,3],[5,0,78],[7,3,45]]). If you want that you can try this:
def zeros_2(A):
new_mat = np.copy(A)
for i in range(len(A[0])): #I assume each row has same length.
if 0 in new_mat[:,i]:
new_mat[:,i] = 0
print(new_mat)
return A
I'd like to be able to use list comprehension syntax to work with NumPy arrays easily.
For instance, I would like something like the below obviously wrong code to just reproduce the same array.
>>> X = np.random.randn(8,4)
>>> [[X[i,j] for i in X] for j in X[i]]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: arrays used as indices must be of integer (or boolean) type
What is the easy way to do this, to avoid using range(len(X)?
First, you should not be using NumPy arrays as lists of lists.
Second, let's forget about NumPy; your listcomp doesn't make any sense in the first place, even for lists of lists.
In the inner comprehension, for i in X is going to iterate over the rows in X. Those rows aren't numbers, they're lists (or, in NumPy, 1D arrays), so X[i] makes no sense whatsoever. You may have wanted i[j] instead.
In the outer comprehension, for j in X[i] has the same problem, but is has an even bigger problem: there is no i value. You have a comprehension looping over each i inside this comprehension.
If you're confused by a comprehension, write it out as an explicit for statement, as explained in the tutorial section on List Comprehensions:
tmp = []
for j in X[i]:
tmp.append([X[i,j] for i in X])
… which expands to:
tmp = []
for j in X[i]:
tmp2 = []
for i in X:
tmp2.append(X[i,j])
tmp.append(tmp2)
… which should make it obvious what's wrong here.
I think what you wanted was:
[[cell for cell in row] for row in X]
Again, turn it back into explicit for statements:
tmp = []
for row in X;
tmp2 = []
for cell in row:
tmp2.append(cell)
tmp.append(tmp2)
That's obviously right.
Or, if you really want to use indexing (but you don't):
[[X[i][j] for j in range(len(X[i]))] for i in range(len(X))]
So, back to NumPy. In NumPy terms, that last version is:
[[X[i,j] for j in range(X.shape[1])] for i in range(X.shape[0])]
… and if you want to go in column-major order instead of row-major, you can (unlike with a list of lists):
[[X[i,j] for i in range(X.shape[0])] for j in range(X.shape[1])]
… but that will of course transpose the array, which isn't what you wanted to do.
The one thing you can't do is mix up column-major and row-major order in the same expression, because you end up with nonsense.
Of course the right way to make a copy of an array is to use the copy method:
X.copy()
Just as the right way to transpose an array is:
X.T
The easy way is to not do this. Use numpy's implicit vectorization instead. For example, if you have arrays A and B as follows:
A = numpy.array([[1, 3, 5],
[2, 4, 6],
[9, 8, 7]])
B = numpy.array([[5, 3, 5],
[3, 5, 3],
[5, 3, 5]])
then the following code using list comprehensions:
C = numpy.array([[A[i, j] * B[i, j] for j in xrange(A.shape[1])]
for i in xrange(A.shape[0])])
can be much more easily written as
C = A * B
It'll also run much faster. Generally, you will produce faster, clearer code if you don't use list comprehensions with numpy than if you do.
If you really want to use list comprehensions, standard Python list-comprehension-writing techniques apply. Iterate over the elements, not the indices:
C = numpy.array([[a*b for a, b in zip(a_row, b_row)]
for a_row, b_row in zip(A, B)]
Thus, your example code would become
numpy.array([[elem for elem in x_row] for x_row in X])
Another option (though not necessarily performant) is to rethink your problem as a map instead of a comprehension and write a ufunc:
http://docs.scipy.org/doc/numpy/reference/ufuncs.html
You can call functional-lite routines like:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.apply_over_axes.html
http://docs.scipy.org/doc/numpy/reference/generated/numpy.vectorize.html
Etc.
Do you mean following?
>>> [[X[i,j] for j in range(X.shape[1])] for i in range(X.shape[0])]
[[0.62757350000000001, -0.64486080999999995, -0.18372566000000001, 0.78470704000000002],
[1.78209799, -1.336448459999999 9, -1.3851422200000001, -0.49668994],
[-0.84148266000000005, 0.18864597999999999, -1.1135151299999999, -0.40225053999999 999],
[0.93852824999999995, 0.24652238000000001, 1.1481637499999999, -0.70346624999999996],
[0.83842508000000004, 1.0058 697599999999, -0.91267403000000002, 0.97991269000000003],
[-1.4265273000000001, -0.73465904999999998, 0.6684284999999999 8, -0.21551155],
[-1.1115614599999999, -1.0035033200000001, -0.11558254, -0.4339924],
[1.8771354, -1.0189299199999999, - 0.84754008000000003, -0.35387946999999997]]
Using numpy.ndarray.copy:
>>> X.copy()
array([[ 0.6275735 , -0.64486081, -0.18372566, 0.78470704],
[ 1.78209799, -1.33644846, -1.38514222, -0.49668994],
[-0.84148266, 0.18864598, -1.11351513, -0.40225054],
[ 0.93852825, 0.24652238, 1.14816375, -0.70346625],
[ 0.83842508, 1.00586976, -0.91267403, 0.97991269],
[-1.4265273 , -0.73465905, 0.6684285 , -0.21551155],
[-1.11156146, -1.00350332, -0.11558254, -0.4339924 ],
[ 1.8771354 , -1.01892992, -0.84754008, -0.35387947]])