I want to extract off-block-diagonal elements from a block-diagonal matrix, i.e.
import numpy as np
import scipy as sp
A = np.array([
[1, 1, 1],
[1, 1, 1],
[1, 1, 1]
])
B = np.array([
[2, 2],
[2, 2]
])
C = np.array([
[3]
])
D = sp.linalg.block_diag(A, B, C)
print(D)
>>> array([[1, 1, 1, 0, 0, 0],
[1, 1, 1, 0, 0, 0],
[1, 1, 1, 0, 0, 0],
[0, 0, 0, 2, 2, 0],
[0, 0, 0, 2, 2, 0],
[0, 0, 0, 0, 0, 3]])
So, I need to extract the elements over the diagonal that do not belong to blocks, i.e. that ones which are zeros in D.
How to achieve that?
A straightforwad solution based on loops, can one make it better avoiding loops?
Taking upper triangle and then taking non-zero values is not what I want as it will not allow me to get indices for the original block matrix D, but only for it's upper triangle.
def block_triu_indices(block_sizes=None):
n = np.sum(block_sizes)
blocks = []
for block_size in block_sizes:
blocks.append(np.ones((block_size, block_size)))
A = sp.linalg.block_diag(*blocks)
row_idx = []
col_idx = []
for i in range(n):
for j in range(i+1, n):
if A[i,j]==0:
row_idx.append(i)
col_idx.append(j)
return (np.array(row_idx), np.array(col_idx))
block_triu_idx = block_triu_indices([3, 2, 1])
print(block_triu_idx)
>>> (array([0, 0, 0, 1, 1, 1, 2, 2, 2, 3, 4]),
array([3, 4, 5, 3, 4, 5, 3, 4, 5, 5, 5]))
print(D[block_triu_idx])
>>> array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
Matrix A:
A = np.array([[3, 0, 0, 8, 3],
[9, 3, 2, 2, 6],
[5, 5, 4, 2, 8],
[3, 8, 7, 1, 2],
[3, 9, 1, 5, 5]])
Matrix B: values in each row means the index of each row in matrix A.
B = np.array([[1, 2],
[3, 4],
[1, 3],
[0, 1],
[2, 3]])
We will set values in A whose index are in B to 1, others to 0.
Then the result will be:
A = np.array([[0, 1, 1, 0, 0],
[0, 0, 0, 1, 1],
[0, 1, 0, 1, 0],
[1, 1, 0, 0, 0],
[0, 0, 1, 1, 0]])
I don't want to use for loop, how can I do it with numpy?
We can index using arrays. For axis0, we just make a range for 0-len(B) to cover each row. Then for axis1, we transpose B to represent all the column indices we want to access.
>>> C = np.zeros_like(A)
>>> C[np.arange(len(B)), B.T] = 1
>>> C
array([[0, 1, 1, 0, 0],
[0, 0, 0, 1, 1],
[0, 1, 0, 1, 0],
[1, 1, 0, 0, 0],
[0, 0, 1, 1, 0]])
>>> B = np.array([[1, 2],
... [3, 4],
... [1, 3],
... [0, 1],
... [2, 3]])
Convenient but a bit wasteful
>>> np.identity(5,int)[B].sum(1)
array([[0, 1, 1, 0, 0],
[0, 0, 0, 1, 1],
[0, 1, 0, 1, 0],
[1, 1, 0, 0, 0],
[0, 0, 1, 1, 0]])
More economical but also more typing
>>> out = np.zeros((5,5),int)
>>> out[np.c_[:5],B] = 1
>>> out
array([[0, 1, 1, 0, 0],
[0, 0, 0, 1, 1],
[0, 1, 0, 1, 0],
[1, 1, 0, 0, 0],
[0, 0, 1, 1, 0]])
I have three (n,n) arrays that I need to combine in a very specific way, in order to yield n*n new arrays, that have to be combined into one big array.
I essentially need to take one element from each array and create a new (3,3) array, wherein the diagonal is the three elements (the rest is empty) and then combine these new arrays into one.
It's a bit difficult to explain properly. I've attempted to give an example below which hopefully gives an idea of what I'm trying to do.
Example: Given three (2,3) arrays:
a = np.array([[2,5,9], [7,2,4]])
b = np.array([[3,6,2], [1,6,8]])
c = np.array([[8,7,4], [9,3,1]])
create six arrays with the elements from a, b, and c as the diagonals:
T1 = ([[ 2, 0, 0],
[ 0, 3, 0],
[ 0, 0, 8]])
T2 = ([[ 5, 0, 0],
[ 0, 6, 0],
[ 0, 0, 7]])
T3 = ([[ 9, 0, 0],
[ 0, 2, 0],
[ 0, 0, 4]])
T4 = ([[ 7, 0, 0],
[ 0, 1, 0],
[ 0, 0, 9]])
T5 = ([[ 2, 0, 0],
[ 0, 6, 0],
[ 0, 0, 3])
T6 = ([[ 4, 0, 0],
[ 0, 8, 0],
[ 0, 0, 1]])
combine the six arrays to yield
array([[ 2, 0, 0, 5, 0, 0, 9, 0, 0],
[ 0, 3, 0, 0, 6, 0, 0, 2, 0],
[ 0, 0, 8, 0, 0, 7, 0, 0, 4],
[ 7, 0, 0, 2, 0, 0, 4, 0, 0],
[ 0, 1, 0, 0, 6, 0, 0, 8, 0],
[ 0, 0, 9, 0, 0, 3, 0, 0, 1]])
as in
array([[ T1, T2, T3],
[ T4, T5, T6]])
*The six arrays are not needed in themselves as separate arrays, only the final array is needed. I've just chosen this route as it makes it a bit more apparent what the final one consists of.
It can be done with einsum:
ABC = np.array((a,b,c))
i,j,k = ABC.shape
out = np.zeros((i*j,i*k),ABC.dtype)
np.einsum("jiki->ijk",out.reshape(j,i,k,i))[...] = ABC
out
# array([[2, 0, 0, 5, 0, 0, 9, 0, 0],
# [0, 3, 0, 0, 6, 0, 0, 2, 0],
# [0, 0, 8, 0, 0, 7, 0, 0, 4],
# [7, 0, 0, 2, 0, 0, 4, 0, 0],
# [0, 1, 0, 0, 6, 0, 0, 8, 0],
# [0, 0, 9, 0, 0, 3, 0, 0, 1]])
Explanation:
What does the reshape do?
axis 2 (size k)
/-----------------------\
axis 3 (size i)
/-----\ /-----\ /-----\
a s / a s / [[2, 0, 0, 5, 0, 0, 9, 0, 0],
x i | x i | [0, 3, 0, 0, 6, 0, 0, 2, 0],
i z | i z \ [0, 0, 8, 0, 0, 7, 0, 0, 4],
s e | s e / [7, 0, 0, 2, 0, 0, 4, 0, 0],
| | [0, 1, 0, 0, 6, 0, 0, 8, 0],
0 j \ 1 i \ [0, 0, 9, 0, 0, 3, 0, 0, 1]]
It isolates the 3x3 diagonal matrices into axes 1,3.
What does einsum do here?
It maps the axes of the reshaped out to those of ABC;
"jiki->ijk" means that axis 0 ("j") maps to axis 1, axes 1 and 3 ("i") map to axis 0, and axis 2 ("k") maps to axis 2.
Mapping two axes to one (as with "i") has the special meaning of taking the diagonal.
einsum creates a writeable view, so all that's left to do is assigning ABC to that.
Note: that we use the same letters i,j,k for the shape and for the einsum spec doesn't syntactically mean anything, it just makes the thing a lot more readable.
We can combine the 3 arrays with stack (or np.array):
In [65]: a = np.array([[2,5,9], [7,2,4]])
...: b = np.array([[3,6,2], [1,6,8]])
...: c = np.array([[8,7,4], [9,3,1]])
In [66]: abc = np.stack((a,b,c))
In [67]: abc.shape
Out[67]: (3, 2, 3)
One 'column' of abc is one of your diagonals:
In [68]: abc[:,0,0]
Out[68]: array([2, 3, 8])
Make a target array to hold all 6 diagonals:
In [69]: TT = np.zeros((6,3,3),int)
We can then set one diagonal with:
In [70]: idx=np.arange(3)
In [71]: TT[0,idx,idx] = abc[:,0,0]
In [72]: TT
Out[72]:
array([[[2, 0, 0],
[0, 3, 0],
[0, 0, 8]],
...
To set all 6 we need an array that matches this shape:
In [74]: TT[:,idx,idx].shape
Out[74]: (6, 3)
Reshape abc. The result is (3,6). Transpose to make a (6,3):
In [75]: abc.reshape(3,6)
Out[75]:
array([[2, 5, 9, 7, 2, 4],
[3, 6, 2, 1, 6, 8],
[8, 7, 4, 9, 3, 1]])
In [76]: TT[:,idx,idx] = abc.reshape(3,6).T
In [77]: TT
Out[77]:
array([[[2, 0, 0],
[0, 3, 0],
[0, 0, 8]],
[[5, 0, 0],
[0, 6, 0],
[0, 0, 7]],
[[9, 0, 0],
[0, 2, 0],
[0, 0, 4]],
[[7, 0, 0],
[0, 1, 0],
[0, 0, 9]],
[[2, 0, 0],
[0, 6, 0],
[0, 0, 3]],
[[4, 0, 0],
[0, 8, 0],
[0, 0, 1]]])
Rearrange elements with reshapes and transpose:
In [82]: TT.reshape(2,3,3,3).transpose(0,2,1,3).reshape(6,9)
Out[82]:
array([[2, 0, 0, 5, 0, 0, 9, 0, 0],
[0, 3, 0, 0, 6, 0, 0, 2, 0],
[0, 0, 8, 0, 0, 7, 0, 0, 4],
[7, 0, 0, 2, 0, 0, 4, 0, 0],
[0, 1, 0, 0, 6, 0, 0, 8, 0],
[0, 0, 9, 0, 0, 3, 0, 0, 1]])
I came up that, step by step. You may want to recreate those steps for yourself. I won't take up the space here.
There may be more direct ways of creating this, but I think the steps are instructive.
Here is one way to do that with advanced indexing:
import numpy as np
a = np.array([[2, 5, 9], [7, 2, 4]])
b = np.array([[3, 6, 2], [1, 6, 8]])
c = np.array([[8, 7, 4], [9, 3, 1]])
# Put all input arrays together
abc = np.stack([a, b, c])
# Works with any shape and number of arrays
n, r, c = abc.shape
# Row and column index grid
ii, jj = np.ogrid[:r, :c]
# Shift row and column indices over submatrices of result
idx = np.arange(n)[:, np.newaxis, np.newaxis]
row_idx = ii * n + idx
col_idx = jj * n + idx
# Broadcast indices
row_idx, col_idx = np.broadcast_arrays(row_idx, col_idx)
# Make output
out = np.zeros((n * r, n * c), abc.dtype)
out[row_idx, col_idx] = abc
print(out)
# [[2 0 0 5 0 0 9 0 0]
# [0 3 0 0 6 0 0 2 0]
# [0 0 8 0 0 7 0 0 4]
# [7 0 0 2 0 0 4 0 0]
# [0 1 0 0 6 0 0 8 0]
# [0 0 9 0 0 3 0 0 1]]
I am unsure as to why you would need to do this, but I believe that I have answered your question anyway. The code is roughly commented, and the variable names are slightly odd, however, it does what you wanted it to do and it does it in the way you suggested above. The code is not very efficient or fast, though it could be cleaned up and made much faster. It takes the arrays you want to convert into the larger output array, makes them the diagonals of 6 3x3 arrays, and then inserts them into the required spot in the output array.
# Import numpy
import numpy as np
# Create your arrays
a = np.array([[2,5,9], [7,2,4]])
b = np.array([[3,6,2], [1,6,8]])
c = np.array([[8,7,4], [9,3,1]])
# Make them into a list
abc = []
abc.append(a)
abc.append(b)
abc.append(c)
# Create an array that will contain T1, T2, ...
arrays = []
for i in range(6):
arr = np.ndarray(shape=(3, 3))
# Fill the array with zeros
for x in range(3):
for y in range(3):
arr[x][y] = 0
for j in range(3):
arr[j][j] = abc[j][0 if i < 3 else 1][i % 3]
arrays.append(arr)
# Combine the arrays into one, in the way specified
bigarr = np.ndarray(shape=(6, 9))
offsetX = 0
offsetY = 0
arr = 0
# Loop over all of the arrays (T1, T2, etc.)
for arr in range(6):
for i in range(3):
for j in range(3):
bigarr[i + offsetX][j + offsetY] = arrays[arr][i][j]
# Offset the place the arrays will be inserted
offsetY += 3
if offsetY >= 9:
offsetY = 0
offsetX += 3
# The final output is bigarr
print(bigarr)
I hope this answers your question, and if not helps you find another answer.
I want to cluster non-zero locations in a NumPy 2D array for MSER detection. Then I want to find the number of points in each cluster and remove those clusters which do not have number of points between some x and y (10 and 300).
I have tried clustering them by searching with neighbouring points but the method fails for concave-shaped non-zero clusters.
[[0, 1, 0, 0, 1],
[0, 1, 1, 1, 1],
[0, 0, 0, 0, 0],
[1, 1, 0, 1, 1],
[1, 0, 0, 1, 1]]
should output, for x=4 and y=5 (both included)
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 1, 1],
[0, 0, 0, 1, 1]]
I'm not sure I have understood your question correctly, but I think scikit-image's label and regionprops could get the job done.
In [6]: import numpy as np
In [7]: from skimage import measure, regionprops
In [8]: img = np.array([[0, 7, 0, 0, 7],
...: [0, 9, 1, 1, 4],
...: [0, 0, 0, 0, 0],
...: [2, 1, 0, 2, 1],
...: [1, 0, 0, 6, 4]])
...:
In [9]: arr = measure.label(img > 0)
In [10]: arr
Out[10]:
array([[0, 1, 0, 0, 1],
[0, 1, 1, 1, 1],
[0, 0, 0, 0, 0],
[2, 2, 0, 3, 3],
[2, 0, 0, 3, 3]])
In [11]: print('Label\t# pixels')
...: for region in measure.regionprops(arr):
...: print(f"{region['label']}\t{region['area']}")
...:
Label # pixels
1 6
2 3
3 4
In python, I have a matrix and I want to find the two largest elements in every row and every column and change their values to 1 (seperately, I mean get two matrices where one of them modified the rows and the other modified the cols).
The main goal is to get a corresponding matrix with zeros everywhere except those ones I've put in the 2 largest element of each row and column (using np.where(mat == 1, 1, 0).
I'm trying to use the np.argpartition in order to do so but without success.
Please help.
See image below.
Here's an approach with np.argpartition -
idx_row = np.argpartition(-a,2,axis=1)[:,:2]
out_row = np.zeros(a.shape,dtype=int)
out_row[np.arange(idx_row.shape[0])[:,None],idx_row] = 1
idx_col = np.argpartition(-a,2,axis=0)[:2]
out_col = np.zeros(a.shape,dtype=int)
out_col[idx_col,np.arange(idx_col.shape[1])] = 1
Sample input, output -
In [40]: a
Out[40]:
array([[ 3, 7, 1, -5, 14, 2, 8],
[ 5, 8, 1, 4, -3, 3, 10],
[11, 3, 5, 1, 9, 2, 5],
[ 6, 4, 12, 6, 1, 15, 4],
[ 8, 2, 0, 1, -2, 3, 5]])
In [41]: out_row
Out[41]:
array([[0, 0, 0, 0, 1, 0, 1],
[0, 1, 0, 0, 0, 0, 1],
[1, 0, 0, 0, 1, 0, 0],
[0, 0, 1, 0, 0, 1, 0],
[1, 0, 0, 0, 0, 0, 1]])
In [42]: out_col
Out[42]:
array([[0, 1, 0, 0, 1, 0, 1],
[0, 1, 0, 1, 0, 1, 1],
[1, 0, 1, 0, 1, 0, 0],
[0, 0, 1, 1, 0, 1, 0],
[1, 0, 0, 0, 0, 0, 0]])
Alternatively, if you are into compact codes, we can skip the initialization and use broadcasting to get the outputs from idx_row and idx_col directly, like so -
out_row = (idx_row[...,None] == np.arange(a.shape[1])).any(1).astype(int)
out_col = (idx_col[...,None] == np.arange(a.shape[0])).any(0).astype(int).T