Replacement for numpy.apply_along_axis in CuPy - python

I have a NumPy-based neural network that I am trying to port to CuPy. I have a function as follows:
import numpy as np
def tensor_diag(x): return np.apply_along_axis(np.diag, -1, x)
# Usage: (x is a matrix, i.e. a 2-tensor)
def sigmoid_prime(x): return tensor_diag(sigmoid(x) * (1 - sigmoid(x)))
This works using NumPy, but CuPy does not have an analogue for the function (it is unsupported as of 8th May 2020). How can I emulate this behaviour in CuPy?


In [284]: arr = np.arange(24).reshape(2,3,4)
np.diag takes a 1d array, and returns a 2d with the values on the diagonal. apply_along_axis just iterates on all dimensions except the last, and passes the last, one array at a time to diag:
In [285]: np.apply_along_axis(np.diag,-1,arr)
Out[285]:
array([[[[ 0, 0, 0, 0],
[ 0, 1, 0, 0],
[ 0, 0, 2, 0],
[ 0, 0, 0, 3]],
[[ 4, 0, 0, 0],
[ 0, 5, 0, 0],
[ 0, 0, 6, 0],
[ 0, 0, 0, 7]],
[[ 8, 0, 0, 0],
[ 0, 9, 0, 0],
[ 0, 0, 10, 0],
[ 0, 0, 0, 11]]],
[[[12, 0, 0, 0],
[ 0, 13, 0, 0],
[ 0, 0, 14, 0],
[ 0, 0, 0, 15]],
[[16, 0, 0, 0],
[ 0, 17, 0, 0],
[ 0, 0, 18, 0],
[ 0, 0, 0, 19]],
[[20, 0, 0, 0],
[ 0, 21, 0, 0],
[ 0, 0, 22, 0],
[ 0, 0, 0, 23]]]])
In [286]: _.shape
Out[286]: (2, 3, 4, 4)
I could do the same mapping with:
In [287]: res = np.zeros((2,3,4,4),int)
In [288]: res[:,:,np.arange(4),np.arange(4)] = arr
check with the apply result:
In [289]: np.allclose(_285, res)
Out[289]: True
Or for a more direct copy of apply, use np.ndindex to generate all the i,j tuple pairs to iterate over the first 2 dimensions of arr:
In [298]: res = np.zeros((2,3,4,4),int)
In [299]: for ij in np.ndindex(2,3):
...: res[ij]=np.diag(arr[ij])
...:
In [300]: np.allclose(_285, res)
Out[300]: True

Related

Fastest way to expand the values of a numpy matrix in diagonal blocks

I'm searching for a fast way for resize the matrix in a special way, without using for-loops:
I have a squared Matrix:
matrix = [[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9,10],
[11,12,13,14,15],
[16,17,18,19,20],
[21,22,23,24,25]]
and my purpose is to resize it 3 (or n) times, where the values are diagonal blocks in the matrix and other values are zeros:
goal_matrix = [[ 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0],
[ 0, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0],
[ 0, 0, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5],
[ 6, 0, 0, 7, 0, 0, 8, 0, 0, 9, 0, 0,10, 0, 0],
[ 0, 6, 0, 0, 7, 0, 0, 8, 0, 0, 9, 0, 0,10, 0],
[ 0, 0, 6, 0, 0, 7, 0, 0, 8, 0, 0, 9, 0, 0,10],
[11, 0, 0,12, 0, 0,13, 0, 0,14, 0, 0,15, 0, 0],
[ 0,11, 0, 0,12, 0, 0,13, 0, 0,14, 0, 0,15, 0],
[ 0, 0,11, 0, 0,12, 0, 0,13, 0, 0,14, 0, 0,15],
[16, 0, 0,17, 0, 0,18, 0, 0,19, 0, 0,20, 0, 0],
[ 0,16, 0, 0,17, 0, 0,18, 0, 0,19, 0, 0,20, 0],
[ 0, 0,16, 0, 0,17, 0, 0,18, 0, 0,19, 0, 0,20],
[21, 0, 0,22, 0, 0,23, 0, 0,24, 0, 0,25, 0, 0],
[ 0,21, 0, 0,22, 0, 0,23, 0, 0,24, 0, 0,25, 0],
[ 0, 0,21, 0, 0,22, 0, 0,23, 0, 0,24, 0, 0,25]]
It should do something like this question, but without unnecessary zero padding.
Is there any mapping, padding or resizing function for doing this in a fast way?

IMO, it is inappropriate to reject the for loop blindly. Here I provide a solution without the for loop. When n is small, its performance is better than that of #MichaelSzczesny and #SalvatoreDanieleBianco solutions:
def mechanic(mat, n):
ar = np.zeros((*mat.shape, n * n), mat.dtype)
ar[..., ::n + 1] = mat[..., None]
return ar.reshape(
*mat.shape,
n,
n
).transpose(0, 3, 1, 2).reshape([s * n for s in mat.shape])
This solution obtains the expected output through a slice assignment, then transpose and reshape, but copies will occur in the last step of reshaping, making it inefficient when n is large.
After a simple test, I found that the solution that simply uses the for loop has the best performance:
def mechanic_for_loop(mat, n):
ar = np.zeros([s * n for s in mat.shape], mat.dtype)
for i in range(n):
ar[i::n, i::n] = mat
return ar
Next is a benchmark test using perfplot. The test functions are as follows:
import numpy as np
def mechanic(mat, n):
ar = np.zeros((*mat.shape, n * n), mat.dtype)
ar[..., ::n + 1] = mat[..., None]
return ar.reshape(
*mat.shape,
n,
n
).transpose(0, 3, 1, 2).reshape([s * n for s in mat.shape])
def mechanic_for_loop(mat, n):
ar = np.zeros([s * n for s in mat.shape], mat.dtype)
for i in range(n):
ar[i::n, i::n] = mat
return ar
def michael_szczesny(mat, n):
return np.einsum(
'ij,kl->ikjl',
mat,
np.eye(n, dtype=mat.dtype)
).reshape([s * n for s in mat.shape])
def salvatore_daniele_bianco(mat, n):
repeated_matrix = mat.repeat(n, axis=0).repeat(n, axis=1)
col_ids, row_ids = np.meshgrid(
np.arange(repeated_matrix.shape[0]),
np.arange(repeated_matrix.shape[1])
)
repeated_matrix[(col_ids % n) - (row_ids % n) != 0] = 0
return repeated_matrix
functions = [
mechanic,
mechanic_for_loop,
michael_szczesny,
salvatore_daniele_bianco
]
Resize times unchanged, array size changes:
if __name__ == '__main__':
from itertools import accumulate, repeat
from operator import mul
from perfplot import bench
bench(
functions,
list(accumulate(repeat(2, 11), mul)),
lambda n: (np.arange(n * n).reshape(n, n), 5),
xlabel='ar.shape[0]'
).show()
Output:
Resize times changes, array size unchanged:
if __name__ == '__main__':
from itertools import accumulate, repeat
from operator import mul
from perfplot import bench
ar = np.arange(25).reshape(5, 5)
bench(
functions,
list(accumulate(repeat(2, 11), mul)),
lambda n: (ar, n),
xlabel='resize times'
).show()
Output:

Input:
matrix = np.array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9,10],
[11,12,13,14,15],
[16,17,18,19,20],
[21,22,23,24,25]])
Solution:
repeated_matrix = matrix.repeat(3, axis=0).repeat(3, axis=1)
col_ids, row_ids = np.meshgrid(np.arange(repeated_matrix.shape[0]), np.arange(repeated_matrix.shape[1]))
repeated_matrix[(col_ids%3)-(row_ids%3)!=0]=0
Output (repeated_matrix):
array([[ 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0, 0],
[ 0, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5, 0],
[ 0, 0, 1, 0, 0, 2, 0, 0, 3, 0, 0, 4, 0, 0, 5],
[ 6, 0, 0, 7, 0, 0, 8, 0, 0, 9, 0, 0, 10, 0, 0],
[ 0, 6, 0, 0, 7, 0, 0, 8, 0, 0, 9, 0, 0, 10, 0],
[ 0, 0, 6, 0, 0, 7, 0, 0, 8, 0, 0, 9, 0, 0, 10],
[11, 0, 0, 12, 0, 0, 13, 0, 0, 14, 0, 0, 15, 0, 0],
[ 0, 11, 0, 0, 12, 0, 0, 13, 0, 0, 14, 0, 0, 15, 0],
[ 0, 0, 11, 0, 0, 12, 0, 0, 13, 0, 0, 14, 0, 0, 15],
[16, 0, 0, 17, 0, 0, 18, 0, 0, 19, 0, 0, 20, 0, 0],
[ 0, 16, 0, 0, 17, 0, 0, 18, 0, 0, 19, 0, 0, 20, 0],
[ 0, 0, 16, 0, 0, 17, 0, 0, 18, 0, 0, 19, 0, 0, 20],
[21, 0, 0, 22, 0, 0, 23, 0, 0, 24, 0, 0, 25, 0, 0],
[ 0, 21, 0, 0, 22, 0, 0, 23, 0, 0, 24, 0, 0, 25, 0],
[ 0, 0, 21, 0, 0, 22, 0, 0, 23, 0, 0, 24, 0, 0, 25]])
basically you can define your custom function to do this on any matrix like:
def what_you_whant(your_matrix, n_repeats):
repeated_matrix = your_matrix.repeat(n_repeats, axis=0).repeat(n_repeats, axis=1)
col_ids, row_ids = np.meshgrid(np.arange(repeated_matrix.shape[1]), np.arange(repeated_matrix.shape[0]))
repeated_matrix[(col_ids%n_repeats)-(row_ids%n_repeats)!=0]=0
return repeated_matrix

As Michael Szczesny suggested in his comment:
The fastest way is to use the einsum, and multiplicate the matrix with the identification matrix with size of the block and reshape it to the expanded size:
np.einsum('ij,kl->ikjl', matrix, np.eye(3)).reshape(len(matrix) * 3, -1)
another more straight forward answer (but ~4x slower) is to use the Kronecker product. Again multiplying the matrix with the identity matrix:
np.kron(matrix, np.eye(3))

How to count non zero rows in a N-d tensor?

I need to find the number of non zero rows and put them in a 1D tensor(kind of vector).
For an example:
tensor = [
[
[1, 2, 3, 4, 0, 0, 0],
[4, 5, 6, 7, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0]
],
[
[4, 3, 2, 1, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0]
],
[
[0, 0, 0, 0, 0, 0, 0],
[4, 5, 6, 7, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0]
]
]
the tensor shape will be [None,45,7] in a real application, but here it is [3,2,7].
So I need to find the number of non-zero rows in dimension 1 and keep them in a 1d tensor.
non_zeros = [2,1,1] #result for the above tensor
I need to do it in TensorFlow, if it is in NumPy, I would have done it.
Can anyone help me with this?
Thanks in advance

You can use tf.math.count_nonzero combined with tf.reduce_sum
>>> tf.math.count_nonzero(tf.reduce_sum(tensor,axis=2),axis=1)
<tf.Tensor: shape=(3,), dtype=int64, numpy=array([2, 1, 1])>

Try this code:
t = tf.math.not_equal(tensor, 0)
t = tf.reduce_any(t, -1)
t = tf.cast(t, tf.int32)
t = tf.reduce_sum(t, -1)

Numpy remove duplicate columns with values greater than 0

I've the following array.
array([[ 0, 0, 0, 0, 0, 3],
[ 4, 4, 0, 0, 0, 0],
[ 0, 0, 0, 23, 0, 0]])
I am looking to find the unique values column wise such that my result is.
array([[ 0, 0, 0, 0, 3],
[ 4, 0, 0, 0, 0],
[ 0, 0, 23, 0, 0]])
The unique should only be applied to columns without 0 values i.e all columns which has 0 as their value should remain. Also I've to make sure that the indices of the columns is not changed. They remain at their place.
I've already tried the following.
np.unique(a,axis=1, return_index=True)
But this gives me
(array([[ 0, 0, 0, 3],
[ 0, 0, 4, 0],
[ 0, 23, 0, 0]]), array([2, 3, 0, 5]))
There are two problems in this result. The column indices are moved and the columns with only 0 values are also merged.

This will accomplish what you want:
import numpy as np
import pandas as pd
x = np.array([[ 0, 0, 0, 0, 0, 3],
[ 4, 4, 0, 0, 0, 0],
[ 0, 0, 0, 23, 0, 0]])
df = pd.DataFrame(x.T)
row_sum = np.sum(df, axis=1)
df1 = df[row_sum != 0].drop_duplicates()
df0 = df[row_sum == 0]
y = pd.concat([df1, df0]).sort_index().values.T
y
array([[ 0, 0, 0, 0, 3],
[ 4, 0, 0, 0, 0],
[ 0, 0, 23, 0, 0]])
By summing the columns (or the rows after transposing) you can identify which ones contain all zeros, and filter them out before dropping the duplicates. Then you can re-combine them and sort by the index to get the desired output.

multi label classification confusion matrix have wrong number of labels

i am feeding in y_test and y_pred to a confusion matrix. My data is for multi label classification so the row values are one hot encodings.
my data has 30 labels but after feeding into the confusion matrix, the output only has 11 rows and cols which is confusing me. I thought i should have a 30X30.
Their formats are numpy arrays. (y_test and y_pred are dataframes of which i convert to numpy arrays using dataframe.values)
y_test.shape
(8680, 30)
y_test
array([[1, 0, 0, ..., 0, 0, 0],
[1, 0, 0, ..., 0, 0, 0],
[1, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]])
y_pred.shape
(8680, 30)
y_pred
array([[1, 0, 0, ..., 0, 0, 0],
[1, 0, 0, ..., 0, 0, 0],
[1, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]])
I transform them to confusion matrix usable format:
y_test2 = y_test.argmax(axis=1)
y_pred2 = y_pred.argmax(axis=1)
conf_mat = confusion_matrix(y_test2, y_pred2)
here is what my confusion matrix look like:
conf_mat.shape
(11, 11)
conf_mat
array([[4246, 77, 13, 72, 81, 4, 6, 3, 0, 0, 4],
[ 106, 2010, 20, 23, 21, 0, 5, 2, 0, 0, 0],
[ 143, 41, 95, 32, 10, 3, 14, 1, 1, 1, 2],
[ 101, 1, 0, 351, 36, 0, 0, 0, 0, 0, 0],
[ 346, 23, 7, 10, 746, 5, 6, 4, 3, 3, 2],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])
Why does my confusion matrix only have 11 X 11 shape? shouldn't it be 30X30?

I think you are not quit clear the definition of confusion_matrix
y_true = [2, 0, 2, 2, 0, 1]
y_pred = [0, 0, 2, 2, 0, 2]
confusion_matrix(y_true, y_pred)
array([[2, 0, 0],
[0, 0, 1],
[1, 0, 2]])
Which in data frame is
pd.DataFrame(confusion_matrix(y_true, y_pred),columns=[0,1,2],index=[0,1,2])
Out[245]:
0 1 2
0 2 0 0
1 0 0 1
2 1 0 2
The column and index are the category of input.
You have (11,11), which means you only have 11 categories in your data

All this means is that some labels are unused.
y_test.any(axis=0)
y_pred.any(axis=0)
Should show that only 11 of the columns have any 1s in them.
Here's what it would look like if that was not the case:
from sklearn.metrics import confusion_matrix
y_test = np.zeros((8680, 30))
y_pred = np.zeros((8680, 30))
y_test[np.arange(8680), np.random.randint(0, 30, 8680)] = 1
y_pred[np.arange(8680), np.random.randint(0, 30, 8680)] = 1
y_test2 = y_test.argmax(axis=1)
y_pred2 = y_pred.argmax(axis=1)
confusion_matrix(y_test2, y_pred2).shape # (30, 30)

Use one array to filter out another array

How to use one array filter out another array with non-zero value?
from numpy import array
a = array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
b = array([[0, 0, 1, 0, 0],
[0, 0, 2, 0, 0],
[0, 0, 3, 0, 0],
[0, 0, 4, 0, 0],
[0, 0, 5, 0, 0]])
Expected result:
array([[ 0, 0, 2, 0, 0],
[ 0, 0, 7, 0, 0],
[ 0, 0, 12, 0, 0],
[ 0, 0, 17, 0, 0],
[ 0, 0, 22, 0, 0]])
Thank you

The easiest way if you want a new array would be np.where with 3 arguments:
>>> import numpy as np
>>> np.where(b, a, 0)
array([[ 0, 0, 2, 0, 0],
[ 0, 0, 7, 0, 0],
[ 0, 0, 12, 0, 0],
[ 0, 0, 17, 0, 0],
[ 0, 0, 22, 0, 0]])
If you want to change a in-place you could instead use boolean indexing based on b:
>>> a[b == 0] = 0
>>> a
array([[ 0, 0, 2, 0, 0],
[ 0, 0, 7, 0, 0],
[ 0, 0, 12, 0, 0],
[ 0, 0, 17, 0, 0],
[ 0, 0, 22, 0, 0]])

One line solution:
a * (b != 0)

Categories