We Keep Coding

How can I multiply matrices of matrices elementwise using numpy (and not for loops)?

I have some 4-dimensional numpy arrays for which the easiest visualisation is a matrix of arbitrary size (not necessarily square) in which each element is a 2x2 square matrix. I would like to standard matrix multiply (#) the 2x2 matrices of the large matrices elementwise (producing another matrix of the same dimension of 2x2 matrices). The eventual hope is to parallelize this process using CuPy so I want to do this without resorting to looping over every element of the bigger matrix.
Any help would be appreciated.
Example of what I mean:
x = np.array([[ [[1,0], [0, 1]], [[2,2], [2, 1]] ]])
y = np.array([[ [[1,3], [0, 1]], [[2,0], [0, 2]] ]])
xy = np.array([[ [[1,3], [0, 1]], [[4,4], [4, 2]] ]])
[[ [[1, 0], [[2, 2] x [[ [[1, 3], [[2, 0]
[0, 1]] , [2, 1]] ]] [0, 1]] , [0, 2]] ]]
=> [[ [[1, 3], [[4, 4]
[0, 1]] , [4, 2]] ]]
In this example the 2 'large' matrices are 1x2 matrices where each of the 2 elements are 2x2 matrices. I have tried to lay it out in a manner that makes it clear what is going on as well as using standard 4d numpy arrays.
Edited in line with comments.

As Homer512 stated in a comment, np.matmul, aka the # operator, will handle this scenario (see the numpy docs). You will need to make sure your 2 x 2 matrices are in the last dimensions.
import numpy as np
a1 = np.array([[1, 0], [0, 1]])
a2 = np.array([[2, 2], [2, 1]])
a = np.array([a1, a2])
b1 = [[1, 3], [0, 1]]
b2 = [[2, 0], [0, 2]]
b = np.array([b1, b2])
x = np.array([a, b])
print(a # b)
Output:
[[[1 3]
[0 1]]
[[4 4]
[4 2]]]

How would I achieve this "row in A * all rows in B by col in A" multiplication in NumPy without a loop?

Say I have two matrices, A and B:
A = np.array([[1, 3, 2],
[2, 2, 3],
[3, 1, 1]])
B = np.array([[0, 1, 0],
[1, 1, 0],
[1, 1, 1]])
I want to take one column in A and multiply it by each column in B element-wise, then proceed to the next column in A. So, using just one column as an example, I will use A[:,0] (values 1,2,3), and multiply it by each column in B to get this:
array([[0, 1, 0],
[2, 2, 0],
[3, 3, 3]])
I've implemented this using np.einsum like so:
np.einsum('i,ij->ij',A[:,0],B)
I then want to generate a 3D matrix with the depth dimension corresponding to the multiplication by each column in A, which I implemented using a for loop:
np.stack([np.einsum('i,ij->ij',A[:,i],B) for i in range(0,A.shape[1])])
This returns my desired array:
array([[[0, 1, 0],
[2, 2, 0],
[3, 3, 3]],
[[0, 3, 0],
[2, 2, 0],
[1, 1, 1]],
[[0, 2, 0],
[3, 3, 0],
[1, 1, 1]]])
How would I go about doing this without the loop? Can this be done purely with np.einsum? Is there another function in NumPy that will do this more simply?

Here's a simple way:
A.T[:,:,None]*B
adding the last None in indexing creates a new axis which is then used for broadcasting the elementwise multiplication.

How about this code?
A.T.reshape(3, 3, 1) * B
Reshaping ndarray can make doing many things...

Keeping with your usage of einsum:
np.einsum('ij,ik->jik', A, B)

List as index in matlab

There is this method written in Matlab that I want to translate into Python. However, I don't understand how to interpret the notation of indexing the sparse matrix M with a row of the matrix faces. What would be the equivalent in Python?
M = spalloc(size(template,1), size(template,1), 10*size(template,1));
for i = 1:size(faces,1)
v = faces(i,:); % faces is a Nx3 matrix
...
M(v, v) = M(v, v) + WIJ; % WIJ is some 3x3 matrix

#Eric Yu` uses a dense numpy array:
In [239]: A=np.array([[1,2,3],[3,4,5],[5,6,7]])
In [240]: A
Out[240]:
array([[1, 2, 3],
[3, 4, 5],
[5, 6, 7]])
In [241]: v=[0,1]
this indexing selects rows:
In [242]: A[v]
Out[242]:
array([[1, 2, 3],
[3, 4, 5]])
and from that select columns:
In [243]: A[v][:,v]
Out[243]:
array([[1, 2],
[3, 4]])
But A[v] is a copy, not a view, so assignment will fail:
In [244]: A[v][:,v] = 0
In [245]: A
Out[245]:
array([[1, 2, 3],
[3, 4, 5],
[5, 6, 7]])
===
To properly index a block of a numpy array, use ix_ (or equivalent) to create indexing arrays that broadcast against each other to define the block:
In [247]: np.ix_(v,v)
Out[247]:
(array([[0],
[1]]), array([[0, 1]]))
In [248]: A[np.ix_(v,v)]
Out[248]:
array([[1, 2],
[3, 4]])
In [249]: A[np.ix_(v,v)]=0
In [250]: A
Out[250]:
array([[0, 0, 3],
[0, 0, 5],
[5, 6, 7]])
Without the ix_ transform, indexing with [v,v] selects a diagonal:
In [251]: A[v,v]
Out[251]: array([0, 0])
MATLAB M(v,v) indexes the block. Indexing the diagonal on the other hand requires use of sub2idx (or something like that). This is a case where MATLAB's indexing notation makes one task easy, and the other more complex. numpy does the reverse.
===
What I wrote is applicable to sparse matrices as well
In [253]: M=sparse.lil_matrix(np.array([[1,2,3],[3,4,5],[5,6,7]]))
In [254]: M
Out[254]:
<3x3 sparse matrix of type '<class 'numpy.int64'>'
with 9 stored elements in LInked List format>
The diagonal selection:
In [255]: M[v,v]
Out[255]:
<1x2 sparse matrix of type '<class 'numpy.int64'>'
with 2 stored elements in LInked List format>
In [256]: _.A
Out[256]: array([[1, 4]], dtype=int64)
Note that this matrix is (1,2), still 2d, in the style of MATLAB matrices.
block selection:
In [258]: M[np.ix_(v,v)]
Out[258]:
<2x2 sparse matrix of type '<class 'numpy.int64'>'
with 4 stored elements in LInked List format>
In [259]: _.A
Out[259]:
array([[1, 2],
[3, 4]], dtype=int64)
In [260]: M[np.ix_(v,v)]=0
In [261]: M.A
Out[261]:
array([[0, 0, 3],
[0, 0, 5],
[5, 6, 7]], dtype=int64)
sparse.csr_matrix will index in the same way (with some differences in the assignment step).

import numpy as np
A=[[1,2,3],[3,4,5],[5,6,7]]
M=np.array(A)
v=[0,1]
M[v][:,v]
the result is:
array([[1, 2],
[3, 4]])

Find indices of each bin using numpy

I'm encountering a problem that I hope you can help me solve.
I have a 2D numpy array which I want to divide into bins by value. Then I need to know the exact initial indices of all the numbers in each bin.
For example, consider the matrix
[[1,2,3], [4,5,6], [7,8,9]]
and the bin array
[0,2,4,6,8,10].
Then the element first element ([0,0]) should be stored in one bin, the next two elements ([0,1],[0,2]) should be stored in another bin and so on. The desired output looks like this:
[[[0,0]],[[0,1],[0,2]],[[1,0],[1,1]],[[1,2],[2,0]],[[2,1],[2,2]]]
Even though I tried several numpy functions, I'm not able to do this in an elegant way. The best attempt might be
>>> a = [[1,2,3], [4,5,6], [7,8,9]]
>>> bins = [0,2,4,6,8,10]
>>> bin_in_mat = np.digitize(a, bins, right=False)
>>> bin_in_mat
array([[1, 2, 2],
[3, 3, 4],
[4, 5, 5]])
>>> indices = np.argwhere(bin_in_mat)
>>> indices
array([[0, 0],
[0, 1],
[0, 2],
[1, 0],
[1, 1],
[1, 2],
[2, 0],
[2, 1],
[2, 2]])
but this doesn't solve my problem. Any suggestions?

You need to leave numpy and use a loop for this - it's not capable of representing your result:
bin_in_mat = np.digitize(a, bins, right=False)
bin_contents = [np.argwhere(bin_in_mat == i) for i in range(len(bins))]
>>> for b in bin_contents:
... print(repr(b))
array([], shape=(0, 2), dtype=int64)
array([[0, 0]], dtype=int64)
array([[0, 1],
[0, 2]], dtype=int64)
array([[1, 0],
[1, 1]], dtype=int64)
array([[1, 2],
[2, 0]], dtype=int64)
array([[2, 1],
[2, 2]], dtype=int64)
Note that digitize is a bad choice for large integer input (until 1.15), and is faster and more correct as bin_in_mat = np.searchsorted(bins, a, side='left')

Converting a list of numpy array to a single int numpy array

I have a list of numpy arrays, that I want to convert into a single int numpy array.
For example if I have 46 4 x 4 numpy arrays in a list of dimension 2 x 23, I want to convert it into a single integer numpy array of 2 x 23 x 4 x 4 dimension. I have found a way to do this by going through every single element and using numpy.stack(). Is there any better way?

You can simply use np.asarray like so
import numpy as np
list_of_lists = [[np.random.normal(0, 1, (4, 4)) for _ in range(23)]
for _ in range(2)]
a = np.asarray(list_of_lists)
a.shape
The function will infer the shape of the list of lists for you and create an appropriate array.

Stack works for me:
In [191]: A,B,C = np.zeros((2,2),int),np.ones((2,2),int),np.arange(4).reshape(2,
...: 2)
In [192]: x = [[A,B,C],[C,B,A]]
In [193]:
In [193]: x
Out[193]:
[[array([[0, 0],
[0, 0]]), array([[1, 1],
[1, 1]]), array([[0, 1],
[2, 3]])], [array([[0, 1],
[2, 3]]), array([[1, 1],
[1, 1]]), array([[0, 0],
[0, 0]])]]
In [194]: np.stack(x)
Out[194]:
array([[[[0, 0],
[0, 0]],
[[1, 1],
[1, 1]],
[[0, 1],
[2, 3]]],
[[[0, 1],
[2, 3]],
[[1, 1],
[1, 1]],
[[0, 0],
[0, 0]]]])
In [195]: _.shape
Out[195]: (2, 3, 2, 2)
stack views x as a list of 2 items, and applies np.asarray to each.
In [198]: np.array(x[0]).shape
Out[198]: (3, 2, 2)
Then adds a dimension, (1,3,2,2), and concatenates on the first axis.
In this case np.array(x) works just as well
In [201]: np.array(x).shape
Out[201]: (2, 3, 2, 2)