how can I change the values of the diagonal of a matrix in numpy?
I checked Numpy modify ndarray diagonal, but the function there is not implemented in numpy v 1.3.0.
lets say we have a np.array X and I want to set all values of the diagonal to 0.
Did you try numpy.fill_diagonal? See the following answer and this discussion. Or the following from the documentation (although currently broken):
http://docs.scipy.org/doc/numpy/reference/generated/numpy.fill_diagonal.html
If you're using a version of numpy that doesn't have fill_diagonal (the right way to set the diagonal to a constant) or diag_indices_from, you can do this pretty easily with array slicing:
# assuming a 2d square array
n = mat.shape[0]
mat[range(n), range(n)] = 0
This is much faster than an explicit loop in Python, because the looping happens in C and is potentially vectorized.
One nice thing about this is that you can also fill a diagonal with a list of elements, rather than a constant value (like diagflat, but for modifying an existing matrix rather than making a new one). For example, this will set the diagonal of your matrix to 0, 1, 2, ...:
# again assuming 2d square array
n = mat.shape[0]
mat[range(n), range(n)] = range(n)
If you need to support more array shapes, this is more complicated (which is why fill_diagonal is nice...):
m[list(zip(*map(range, m.shape)))] = 0
(The list call is only necessary in Python 3, where zip returns an iterator.)
You can use numpy.diag_indices_from() to get the indices of the diagonal elements of your array. Then set the value of those indices.
X[np.diag_indices_from(X)] = 0.
Example:
>>> import numpy as np
>>> X = np.random.rand(5, 5)
>>> print(X)
[[0.59480384 0.20133725 0.59147423 0.22640441 0.40898203]
[0.65230581 0.57055258 0.97009881 0.58535275 0.32036626]
[0.71524332 0.73424734 0.92461381 0.38704119 0.08147428]
[0.18931865 0.97366736 0.11482649 0.82793141 0.13347333]
[0.47402986 0.73329347 0.18892479 0.11883424 0.78718883]]
>>> X[np.diag_indices_from(X)] = 0
>>> print(X)
[[0. 0.20133725 0.59147423 0.22640441 0.40898203]
[0.65230581 0. 0.97009881 0.58535275 0.32036626]
[0.71524332 0.73424734 0. 0.38704119 0.08147428]
[0.18931865 0.97366736 0.11482649 0. 0.13347333]
[0.47402986 0.73329347 0.18892479 0.11883424 0. ]]
Here's another good way to do this. If you want a one-dimensional view of the array's main diagonal use:
A.ravel()[:A.shape[1]**2:A.shape[1]+1]
For the i'th superdiagonal use:
A.ravel()[i:max(0,A.shape[1]-i)*A.shape[1]:A.shape[1]+1]
For the i'th subdiagonal use:
A.ravel()[A.shape[1]*i:A.shape[1]*(i+A.shape[1]):A.shape[1]+1]
Or in general, for the i'th diagonal where the main diagonal is 0, the subdiagonals are negative and the superdiagonals are positive, use:
A.ravel()[max(i,-A.shape[1]*i):max(0,(A.shape[1]-i))*A.shape[1]:A.shape[1]+1]
These are views and not copies, so they will run faster for extracting a diagonal, but any changes made to the new array object will apply to the original array.
On my machine these run faster than the fill_diagonal function when setting the main diagonal to a constant, but that may not always be the case. They can also be used to assign an array of values to a diagonal instead of just a constant.
Notes: for small arrays it may be faster to use the flat attribute of the NumPy array.
If speed is a major issue it could be worth it to make A.shape[1] a local variable.
Also, if the array is not contiguous, ravel() will return a copy, so, in order to assign values to a strided slice, it will be necessary to creatively slice the original array used to generate the strided slice (if it is contiguous) or to use the flat attribute.
Also, it was originally planned that in NumPy 1.10 and later the 'diagonal' method of arrays will return a view instead of a copy.
That change hasn't yet been made though, but hopefully at some point this trick to get a view will no longer be necessary.
See http://docs.scipy.org/doc/numpy-dev/reference/generated/numpy.diagonal.html
def replaceDiagonal(matrix, replacementList):
for i in range(len(replacementList)):
matrix[i][i] = replacementList[i]
Where size is n in an n x n matrix.
>>> a = numpy.random.rand(2,2)
>>> a
array([[ 0.41668355, 0.07982691],
[ 0.60790982, 0.0314224 ]])
>>> a - numpy.diag(numpy.diag(a))
array([[ 0. , 0.07982691],
[ 0.60790982, 0. ]])
You can do the following.
Assuming your matrix is 4 * 4 matrix.
indices_diagonal = np.diag_indices(4)
yourarray[indices_diagonal] = Val
Related
Here's the question and the example given:
You are given a 2-d array A of size NxN containing floating-point
numbers. The array represents pairwise correlation between N elemenets
with A[i,j] = A[j,i] = corr(i,j) and A[i,i] = 1.
Write a Python program using NumPy to find the index of the highest
correlated element for each element and finally print the sum of all
these indexes.
Example: The array A = [[1, 0.3, 0.4], [0.4,1,0.5],[0.1,0.6,1]]. Then, the indexes of the highest correlated elements for each element
are [3, 3, 2]. the sum of these indexes is 8.
I'm having trouble understanding the question, but the example makes my confusion worse. With each array inside A having only 3 values, and A itself having only three arrays inside how can any "index of the highest correlated elements" being greater than 2 if numpy is zero indexed?
Does anyone understand the question?
To reiterate, the example is wrong in multiple ways.
Correlation matrices are by definition symmetric, yet the example is not:
array([[1. , 0.3, 0.4],
[0.4, 1. , 0.5],
[0.1, 0.6, 1. ]])
Also you are right, numpy arrays (like everything else I know in Python that supports indexing) are zero-indexed. So the solution is off by one.
The exercise wants you to find the index j of the random variable with the greatest correlation for each random variable with index i. Obviously excluding itself (the correlation coefficient of 1 on the diagonal).
Here is one way to do that given your numpy array a:
np.where(a != 1, a, 0).argmax(axis=1)
Here np.where produces an array identical to a except we replace the ones with zeroes. This is based on the assumption that if i != j, the correlation is always < 1. If that does not hold, the solution will obviously be wrong.
Then argmax gives the indices of the greatest values in each row. Although, in an actual correlation matrix, axis=0 would work just as well, since it would be... you know... symmetrical.
The result is array([2, 2, 1]). To get the sum, you just add a .sum() at the end.
EDIT:
Now that I think about it, the assumption is too strong. Here is a better way:
b = a.copy()
np.fill_diagonal(b, -1)
b.argmax(axis=1)
Now we only assume that actual correlations can never be < 0, which I think is reasonable. If you don't care about mutating the original array, you could obviously omit the copy and fill the diagonal of a with -1. instead.
I am trying to 'expand' an array (generate a new array with proportionally more elements in all dimensions). I have an array with known numbers (let's call it X) and I want to make it j times bigger (in each dimension).
So far I generated a new array of zeros with more elements, then I used broadcasting to insert the original numbers in the new array (at fixed intervals).
Finally, I used linspace to fill the gaps, but this part is actually not directly relevant to the question.
The code I used (for n=3) is:
import numpy as np
new_shape = (np.array(X.shape) - 1 ) * ratio + 1
new_array = np.zeros(shape=new_shape)
new_array[::ratio,::ratio,::ratio] = X
My problem is that this is not general, I would have to modify the third line based on ndim. Is there a way to use such broadcasting for any number of dimensions in my array?
Edit: to be more precise, the third line would have to be:
new_array[::ratio,::ratio] = X
if ndim=2
or
new_array[::ratio,::ratio,::ratio,::ratio] = X
if ndim=4
etc. etc. I want to avoid having to write code for each case of ndim
p.s. If there is a better tool to do the entire process (such as 'inner-padding' that I am not aware of, I will be happy to learn about it).
Thank you
array = array[..., np.newaxis] will add another dimension
This article might help
You can use slice notation -
slicer = tuple(slice(None,None,ratio) for i in range(X.ndim))
new_array[slicer] = X
Build the slicing tuple manually. ::ratio is equivalent to slice(None, None, ratio):
new_array[(slice(None, None, ratio),)*new_array.ndim] = ...
I wanna print the index of the row containing the minimum element of the matrix
my matrix is matrix = [[22,33,44,55],[22,3,4,12],[34,6,4,5,8,2]]
and the code
matrix = [[22,33,44,55],[22,3,4,12],[34,6,4,5,8,2]]
a = np.array(matrix)
buff_min = matrix.argmin(axis = 0)
print(buff_min) #index of the row containing the minimum element
min = np.array(matrix[buff_min])
print(str(min.min(axis=0))) #print the minium of that row
print(min.argmin(axis = 0)) #index of the minimum
print(matrix[buff_min]) # print all row containing the minimum
after running, my result is
1
3
1
[22, 3, 4, 12]
the first number should be 2, because the minimum is 2 in the third list ([34,6,4,5,8,2]), but it returns 1. It returns 3 as minimum of the matrix.
What's the error?
I am not sure which version of Python you are using, i tested it for Python 2.7 and 3.2 as mentioned your syntax for argmin is not correct, its should be in the format
import numpy as np
np.argmin(array_name,axis)
Next, Numpy knows about arrays of arbitrary objects, it's optimized for homogeneous arrays of numbers with fixed dimensions. If you really need arrays of arrays, better use a nested list. But depending on the intended use of your data, different data structures might be even better, e.g. a masked array if you have some invalid data points.
If you really want flexible Numpy arrays, use something like this:
np.array([[22,33,44,55],[22,3,4,12],[34,6,4,5,8,2]], dtype=object)
However this will create a one-dimensional array that stores references to lists, which means that you will lose most of the benefits of Numpy (vector processing, locality, slicing, etc.).
Also, to mention if you can resize your numpy array thing might work, i haven't tested it, but by the concept that should be an easy solution. But i will prefer use a nested list in this case of input matrix
Does this work?
np.where(a == a.min())[0][0]
Note that all rows of the matrix need to contain the same number of elements.
Suppose I have an N*M*X-dimensional array "data", where N and M are fixed, but X is variable for each entry data[n][m].
(Edit: To clarify, I just used np.array() on the 3D python list which I used for reading in the data, so the numpy array is of dimensions N*M and its entries are variable-length lists)
I'd now like to compute the average over the X-dimension, so that I'm left with an N*M-dimensional array. Using np.average/mean with the axis-argument doesn't work, so the way I'm doing it right now is just iterating over N and M and appending the manually computed average to a new list, but that just doesn't feel very "python":
avgData=[]
for n in data:
temp=[]
for m in n:
temp.append(np.average(m))
avgData.append(temp)
Am I missing something obvious here? I'm trying to freshen up my python skills while I'm at it, so interesting/varied responses are more than welcome! :)
Thanks!
What about using np.vectorize:
do_avg = np.vectorize(np.average)
data_2d = do_avg(data)
data = np.array([[1,2,3],[0,3,2,4],[0,2],[1]]).reshape(2,2)
avg=np.zeros(data.shape)
avg.flat=[np.average(x) for x in data.flat]
print avg
#array([[ 2. , 2.25],
# [ 1. , 1. ]])
This still iterates over the elements of data (nothing un-Pythonic about that). But since there's nothing special about the shape or axes of data, I'm just using data.flat. While appending to Python list, with numpy it is better to assign values to the elements of an existing array.
There are fast numeric methods to work with numpy arrays, but most (if not all) work with simple numeric dtypes. Here the array elements are object (either list or array), numpy has to resort to the usual Python iteration and list operations.
For this small example, this solution is a bit faster than Zwicker's vectorize. For larger data the two solutions take about the same time.
Here is a small code to illustrate the problem.
A = array([[1,2], [1,0], [5,3]])
f_of_A = f(A) # this is precomputed and expensive
values = array([[1,2], [1,0]])
# location of values in A
# if I just had 1d values I could use numpy.in1d here
indices = array([0, 1])
# example of operation type I need (recalculating f_of_A as needed is not an option)
f_of_A[ indices ]
So, basically I think I need some equivalent to in1d for higher dimensions. Does such a thing exist? Or is there some other approach?
Looks like there is also a searchsorted() function, but that seems to work for 1d arrays also. In this example I used 2d points, but any solution would need to work for 3d points also.
Okay, this is what I came up with.
To find the value of one multi-dimensional index, let's say ii = np.array([1,2]), we can do:
n.where((A == ii).all(axis=1))[0]
Let's break this down, we have A == ii, which will give element-wise comparisons with ii for each row of A. We want an entire row to be true, so we add .all(axis=1) to collapse them. To find where these indices happen, we plug this into np.where and get the first value of the tuple.
Now, I don't have a fast way to do this with multiple indices yet (although I have a feeling there is one). However, this will get the job done:
np.hstack([np.where((A == values[i]).all(axis=1))[0] for i in xrange(len(values))])
This basically just calls the above, for each value of values, and concatenates the result.
Update:
Here is for the multi-dimensional case (all in one go, should be fairly fast):
np.where((np.expand_dims(A, -1) == values.T).all(axis=1).any(axis=1))[0]
You can use np.in1d over a view of your original array with all coordinates collapsed into a single variable of dtype np.void:
import numpy as np
A = np.array([[1,2], [1,0], [5,3]])
values = np.array([[1,2], [1,0]])
# Make sure both arrays are contiguous and have common dtype
common_dtype = np.common_type(A, values)
a = np.ascontiguousarray(A, dtype=common_dtype)
vals = np.ascontiguousarray(values, dtype=common_dtype)
a_view = A.view((np.void, A.dtype.itemsize*A.shape[1])).ravel()
values_view = values.view((np.void,
values.dtype.itemsize*values.shape[1])).ravel()
Now each item of a_view and values_view is all coordinates for one point packed together, so you can do whatever 1D magic you would use. I don't see how to use np.in1d to find indices though, so I would go the np.searchsorted route:
sort_idx = np.argsort(a_view)
locations = np.searchsorted(a_view, values_view, sorter=sort_idx)
locations = sort_idx[locations]
>>> locations
array([0, 1], dtype=int64)