Say I have three numpy arrays and I want to perform a calculation over them:
a = np.array([[1,2,3,4,5,6,7],[1,2,3,4,5,6,7],[1,2,3,4,5,6,7],[1,2,3,4,5,6,7],
[1,2,3,4,5,6,7]]) #shape is (5,7)
b = np.array([[11],[12],[11],[12],[11]]) #shape is (5,1)
c = np.array([[10],[20],[30],[40],[50],[60],[70]]) #shape is (5,1)
The calculation is: 10 + (b(rows) * (c . a(rows)))
Where c . a is the dot product of C and the row of a.
By rows, I mean doing it as a vector where I need my result to be (7,1) (one row per each column I have on a)
I'm trying to do something like:
result = 10 + (b[:][:] * (np.dot(c.T, a[:]) + b))
But this fails the np.dot operation with shapes being misaligned for that numpy.dot operation. I'm trying to figure out how to perform the calculation above as a one-liner (no for loops) in a way that Python will interpret the vectorized operation, especially for that np.dot part.
Any hints?
Thanks for your time
EDIT: this is a for loop that solves my problem. I'd like to replace that for loop with one Python line.
iBatchSize = a.shape[0]
iFeatureCount = a.shape[1]
result = np.zeros((iBatchSize,1))
for i in range(iBatchSize):
for j in range(iFeatureCount):
result [i] = 10 + (b[i][0] * (np.dot(c.T, a[i]) + b))
EDIT 2: Corrected array a with the correct array
EDIT 3: Corrected expected shape for result
In [31]: a = np.array([[1,2],[2,3],[3,4],[4,5],[5,6],[6,7],[7,8]]) #shape is (5,7)
...: b = np.array([[11],[12],[11],[12],[11]]) #shape is (5,1)
...: c = np.array([[10],[20],[30],[40],[50],[60],[70]]) #shape is (7,1)
In [32]: a.shape, b.shape, c.shape
Out[32]: ((7, 2), (5, 1), (7, 1))
a.shape does not match the comment.
In [33]: iBatchSize = a.shape[0]
...: iFeatureCount = a.shape[1]
...:
...: result = np.zeros((iBatchSize,1))
...:
...: for i in range(iBatchSize):
...: for j in range(iFeatureCount):
...: result [i] = 10 + (b[i][0] * (np.dot(c.T, a[i]) + b))
...:
Traceback (most recent call last):
File "<ipython-input-33-717691add3dd>", line 8, in <module>
result [i] = 10 + (b[i][0] * (np.dot(c.T, a[i]) + b))
File "<__array_function__ internals>", line 6, in dot
ValueError: shapes (1,7) and (2,) not aligned: 7 (dim 1) != 2 (dim 0)
np.dot is raising that error. It expects the last of first arg to match with the 2nd to the last (or only) of second arg:
In [34]: i
Out[34]: 0
In [35]: c.T.shape
Out[35]: (1, 7)
In [37]: a[i].shape
Out[37]: (2,)
This dot works:
In [38]: np.dot(c.T,a).shape # (1,7) with (7,2) => (1,2)
Out[38]: (1, 2)
====
With the correct a,
10 + (b[i][0] * (np.dot(c.T, a[i]) + b))
is (5,1) array (because of the +b), which can't be put in result[i].
===
a simple dot of a and c produces a (5,1) which can be combined with b (either with + or * or both), resulting in a (5,1) array:
In [68]: np.dot(a,c).shape
Out[68]: (5, 1)
In [69]: b*(np.dot(a,c)+b)
Out[69]:
array([[15521],
[16944],
[15521],
[16944],
[15521]])
Related
My code is as follows doing the jacobi decomposition to solve a matrix
def iterative_solver(M, b, Vinit, kmax, epsmax):
"""
Provides an approximation of M^{-1}b using the proposed iterative method
----------
parameters:
M : matrix in the equation MV = b to solve (numpy array of size N,N)
b : vector on the right-hand-side of the equation MV = b (numpy array of size N)
Vinit : initial vector of the iterative method (numpy array of size N)
kmax : maximum number of iterations, stops if k reaches kmax (integer)
epsmax : tolerance on the residual, stops if eps reaches epsmax (float)
returns:
V : resulting vector at the end of the iterations (numpy array of size N)
eps : vector composed of the residuals eps at every iteration (numpy array of size k)
k : number of iterations performed before the algorithm stops (integer)
"""
eps = np.inf
N = len(M)
d = list()
for i in range(N):
d.append(M[i,i])
D = np.diag(d)
R = D - M
Dinv = np.linalg.inv(D)
V = Vinit
k =0
print(f"V: {np.shape(V)} M: {np.shape(M)} b: {np.shape(b)}")
while k <= kmax and eps >= epsmax:
A = np.matmul((-M), V)+ b
print(f"np.shape(A): {np.shape(A)}")
V = V + np.matmul(Dinv,A)
print(f"np.shape(V): {np.shape(V)}")
#A = np.matmul(R,V)
#V = np.matmul(Dinv, A + b)
k += 1
eps = np.linalg.norm(M*V.T-b)
return V, eps, k
M = np.mat([[3,1,0],[-1, -5, 1], [0,2,4]])
b = np.array([4,-5,6]).T
Vinit = np.random.rand(3).T
kmax = 100
epsmax = 1e-10
print(f"solved: {iterative_solver(M, b, Vinit, kmax, epsmax)}")
However it gives me the output
V: (3,) M: (3, 3) b: (3,)
np.shape(A): (1, 3)
Which is very weird since a shape (3,3) matrix times a (3,) vector plus a (3,) vector make a (1,3) shape in numpy?
The shape business is buried in your code, but I think this is what you are referring to:
V: (3,) M: (3, 3) b: (3,)
A = np.matmul((-M), V)+ b
matmul of a (3,3) with (3,) produces a (3,). Add a (3,) should stay (3,)
But then you change V with V + np.matmul(Dinv,A). You print its shape, but I don't see that.
In [224]: V = np.arange(3)
...: M = np.ones((3, 3))
...: b = np.ones(3)
In [225]: V, M, b
Out[225]:
(array([0, 1, 2]),
array([[1., 1., 1.],
[1., 1., 1.],
[1., 1., 1.]]),
array([1., 1., 1.]))
In [226]: np.matmul((-M), V) + b
Out[226]: array([-2., -2., -2.])
With your code:
In [23]: iterative_solver(M, b, Vinit, kmax, epsmax)
V: (3,) M: (3, 3) b: (3,)
np.shape(A): (1, 3)
Traceback (most recent call last):
Input In [23] in <module>
iterative_solver(M, b, Vinit, kmax, epsmax)
Input In [4] in iterative_solver
V = V + np.matmul(Dinv, A)
ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 1 is different from 3)
I tried:
In [25]: -M # Vinit
Out[25]: matrix([[-2.66022252, 2.20102655, -2.90845177]])
It took me a bit to notice that this is np.matrix! That by definition is 2d, hence the (1,3) shape.
If I pass a np.array matrix instead:
In [28]: iterative_solver(M.A, b, Vinit, kmax, epsmax)
V: (3,) M: (3, 3) b: (3,)
(3,)
np.shape(A): (3,)
...
Out[28]: (array([1., 1., 1.]), 13.856406460551018, 101)
edit
np.matrix is a subclass of ndarray. It defines its own * and ** as matrix operations, as opposed to elementwise. And it enforces a 2d shape. If necessary it adds a leading dimension (numpy is 'C' order by default)
In [150]: M = np.matrix(np.arange(9).reshape(3, 3))
In [151]: v = np.arange(3)
Using its own *:
In [152]: M * v
Traceback (most recent call last):
Input In [152] in <module>
M * v
File /usr/local/lib/python3.8/dist-packages/numpy/matrixlib/defmatrix.py:218 in __mul__
return N.dot(self, asmatrix(other))
File <__array_function__ internals>:180 in dot
ValueError: shapes (3,3) and (1,3) not aligned: 3 (dim 1) != 1 (dim 0)
The (3,) v has been 'promoted' to matrix (1,3).
matmul/# is a recent addition (np.matrix is old, and on its way out):
In [153]: M # v
Out[153]: matrix([[ 5, 14, 23]])
In [154]: M.dot(v)
Out[154]: matrix([[ 5, 14, 23]])
It appears that both of these perform the multiply as though they were ndarray, and converts the result to matrix.
If both are np.matrix:
In [155]: V = np.matrix(v)
In [156]: V
Out[156]: matrix([[0, 1, 2]])
In [157]: M * V.T
Out[157]:
matrix([[ 5],
[14],
[23]])
M#V.T does the same thing.
I wish to compute the dot product between two 3D tensors along the first dimension. I tried the following einsum notation:
import numpy as np
a = np.random.randn(30).reshape(3, 5, 2)
b = np.random.randn(30).reshape(3, 2, 5)
# Expecting shape: (3, 5, 5)
np.einsum("ijk,ikj->ijj", a, b)
Sadly it returns this error:
ValueError: einstein sum subscripts string includes output subscript 'j' multiple times
I went with Einstein sum after I failed at it with np.tensordot. Ideas and follow up questions are highly welcome!
Your two dimensions of size 5 and 5 do not correspond to the same axes. As such you need to use two different subscripts to designate them. For example, you can do:
>>> res = np.einsum('ijk,ilm->ijm', a, b)
>>> res.shape
(3, 5, 5)
Notice you are also required to change the subscript for axes of size 2 and 2. This is because you are computing the batched outer product (i.e. we iterate on two axes at the same time), not a dot product (i.e. we iterate simultaneously on the two axes).
Outer product:
>>> np.einsum('ijk,ilm->ijm', a, b)
Dot product over subscript k, which is axis=2 of a and axis=1 of b:
>>> np.einsum('ijk,ikm->ijm', a, b)
which is equivalent to a#b.
dot product ... along the first dimension is a bit unclear. Is the first dimension a 'batch' dimension, with 3 dot's on the rest? Or something else?
In [103]: a = np.random.randn(30).reshape(3, 5, 2)
...: b = np.random.randn(30).reshape(3, 2, 5)
In [104]: (a#b).shape
Out[104]: (3, 5, 5)
In [105]: np.einsum('ijk,ikl->ijl',a,b).shape
Out[105]: (3, 5, 5)
#Ivan's answer is different:
In [106]: np.einsum('ijk,ilm->ijm', a, b).shape
Out[106]: (3, 5, 5)
In [107]: np.allclose(np.einsum('ijk,ilm->ijm', a, b), a#b)
Out[107]: False
In [108]: np.allclose(np.einsum('ijk,ikl->ijl', a, b), a#b)
Out[108]: True
Ivan's sums the k dimension of one, and l of the other, and then does a broadcasted elementwise. That is not matrix multiplication:
In [109]: (a.sum(axis=-1,keepdims=True)* b.sum(axis=1,keepdims=True)).shape
Out[109]: (3, 5, 5)
In [110]: np.allclose((a.sum(axis=-1,keepdims=True)* b.sum(axis=1,keepdims=True)),np.einsum('ijk,ilm->ijm', a,
...: b))
Out[110]: True
Another test of the batch processing:
In [112]: res=np.zeros((3,5,5))
...: for i in range(3):
...: res[i] = a[i]#b[i]
...: np.allclose(res, a#b)
Out[112]: True
I'm trying to make a clean connection between the dimensions in a numpy array and the dimensions of a matrix via classical linear algebra. Suppose the following:
In [1] import numpy as np
In [2] rand = np.random.RandomState(42)
In [3] a = rand.rand(3,2)
In [4] a
Out[4]:
array([[0.61185289, 0.13949386],
[0.29214465, 0.36636184],
[0.45606998, 0.78517596]])
In [5]: a[np.newaxis,:,:]
Out[5]:
array([[[0.61185289, 0.13949386],
[0.29214465, 0.36636184],
[0.45606998, 0.78517596]]])
In [6]: a[:,np.newaxis,:]
Out[6]:
array([[[0.61185289, 0.13949386]],
[[0.29214465, 0.36636184]],
[[0.45606998, 0.78517596]]])
In [7]: a[:,:,np.newaxis]
Out[7]:
array([[[0.61185289],
[0.13949386]],
[[0.29214465],
[0.36636184]],
[[0.45606998],
[0.78517596]]])
My questions are as follows:
Is is correct to say that the dimensions of a are 3 X 2? In other words, a 3 X 2 matrix?
Is it correct to say that the dimensions of a[np.newaxis,:,:] are 1 X 3 X 2? In other words, a matrix containing a 3 X 2 matrix?
Is it correct to say that the dimensions of a[:,np.newaxis,:] are 3 X 1 X 2? In other words a matrix containing 3 1 X 2 matrices?
Is it correct to say that the dimensions of a[:,:,np.newaxis] are 3 X 2 X1? In other words a matrix containing 3 matrices each of which contain 2 1 X 1 matrices?
yes
yes
yes
three 2x1 matrices each of which contains one vector of size 1
Just find out using .shape:
import numpy as np
rand = np.random.RandomState(42)
# 1.
a = rand.rand(3, 2)
print(a.shape, a, sep='\n', end='\n\n')
# 2.
b = a[np.newaxis, :, :]
print(b.shape, b, sep='\n', end='\n\n')
# 3.
c = a[:, np.newaxis, :]
print(c.shape, c, sep='\n', end='\n\n')
# 4.a
d = a[:, :, np.newaxis]
print(d.shape, d, sep='\n', end='\n\n')
# 4.b
print(d[0].shape, d[0], sep='\n', end='\n\n')
print(d[0, 0].shape, d[0, 0])
output:
(3, 2)
[[0.37454012 0.95071431]
[0.73199394 0.59865848]
[0.15601864 0.15599452]]
(1, 3, 2)
[[[0.37454012 0.95071431]
[0.73199394 0.59865848]
[0.15601864 0.15599452]]]
(3, 1, 2)
[[[0.37454012 0.95071431]]
[[0.73199394 0.59865848]]
[[0.15601864 0.15599452]]]
(3, 2, 1)
[[[0.37454012]
[0.95071431]]
[[0.73199394]
[0.59865848]]
[[0.15601864]
[0.15599452]]]
(2, 1)
[[0.37454012]
[0.95071431]]
(1,) [0.37454012]
Very similar to https://math.stackexchange.com/q/3615927/419686, but different.
I have 2 matrices (A with shape (5,2,3) and B with shape (6,3,8)), and I want to perform some kind of multiplication in order to take a new matrix with shape (5,6,2,8).
Python code:
import numpy as np
np.random.seed(1)
A = np.random.randint(0, 10, size=(5,2,3))
B = np.random.randint(0, 10, size=(6,3,8))
C = np.zeros((5,6,2,8))
for i in range(A.shape[0]):
for j in range(B.shape[0]):
C[i,j] = A[i].dot(B[j])
Is it possible to do the above operation without using a loop?
In [52]: np.random.seed(1)
...: A = np.random.randint(0, 10, size=(5,2,3))
...: B = np.random.randint(0, 10, size=(6,3,8))
...:
...: C = np.zeros((5,6,2,8))
...: for i in range(A.shape[0]):
...: for j in range(B.shape[0]):
...: C[i,j] = A[i].dot(B[j])
...:
np.dot does broadcast the outer dimensions:
In [53]: D=np.dot(A,B)
In [54]: C.shape
Out[54]: (5, 6, 2, 8)
In [55]: D.shape
Out[55]: (5, 2, 6, 8)
The axes order is different, but we can easily change that:
In [56]: np.allclose(C, D.transpose(0,2,1,3))
Out[56]: True
In [57]: np.allclose(C, np.swapaxes(D,1,2))
Out[57]: True
From the np.dot docs:
dot(a, b)[i,j,k,m] = sum(a[i,j,:] * b[k,:,m])
Use np.einsum which is very powerful:
C = np.einsum('aij, bjk -> abik', A, B)
I have an array A (shape = (a, 1)) and matrix B (shape = (b1, b2)). Want to multiply the latter by each element of the former to generate a tridimensional array (shape = (a, b1, b2)).
Is there a vectorized way to do this?
import numpy as np
A = np.random.rand(3, 1)
B = np.random.rand(5, 4)
C = np.array([ a * B for a in A ])
There are several ways you can achieve this.
One is using np.dot, note that it will be necessary to introduce a second axis in B so both ndarrays can be multiplied:
C = np.dot(A,B[:,None])
print(C.shape)
# (3, 5, 4)
Using np.multiply.outer, as #divakar suggests:
C = np.multiply.outer(A,B)
print(C.shape)
# (3, 5, 4)
Or you could also use np.einsum:
C = np.einsum('ij,kl->ikl', A, B)
print(C.shape)
# (3, 5, 4)