Related
I need to divide a 2D matrix into a set of 2D patches with a certain stride, then multiply every patch by its center element and sum the elements of each patch.
It feels not unlike a convolution where a separate kernel is used for every element of the matrix.
Below is a visual illustration.
The elements of the result matrix are calculated like this:
The result should look like this:
Here's a solution I came up with:
window_shape = (2, 2)
stride = 1
# Matrix
m = np.arange(1, 17).reshape((4, 4))
# Pad it once per axis to make sure the number of views
# equals the number of elements
m_padded = np.pad(m, (0, 1))
# This function divides the array into `windows`, from:
# https://stackoverflow.com/questions/45960192/using-numpy-as-strided-function-to-create-patches-tiles-rolling-or-sliding-w#45960193
w = window_nd(m_padded, window_shape, stride)
ww, wh, *_ = w.shape
w = w.reshape((ww * wh, 4)) # Two first dimensions multiplied is the number of rows
# Tile each center element for element-wise multiplication
m_tiled = np.tile(m.ravel(), (4, 1)).transpose()
result = (w * m_tiled).sum(axis = 1).reshape(m.shape)
In my view it's not very efficient as a few arrays are allocated in the intermediary steps.
What is a better or more efficient way to accomplish this?
Try scipy.signal.convolve
from scipy.signal import convolve
window_shape = (2, 2)
stride = 1
# Matrix
m = np.arange(1, 17).reshape((4, 4))
# Pad it once per axis to make sure the number of views
# equals the number of elements
m_padded = np.pad(m, (0, 1))
output = convolve(m_padded, np.ones(window_shape), 'valid') * m
print(output)
Output:
array([[ 14., 36., 66., 48.],
[150., 204., 266., 160.],
[414., 500., 594., 336.],
[351., 406., 465., 256.]])
3D numpy array A contains a series (in this example, I am choosing 3) of 2D numpy array D of shape 2 x 2. The D matrix is as follows:
D = np.array([[1,2],[3,4]])
A is initialized and assigned as below:
idx = np.arange(3)
A = np.zeros((3,2,2))
A[idx,:,:] = D # This gives A = [[[1,2],[3,4]],[[1,2],[3,4]],\
# [[1,2],[3,4]]]
# In mathematical notation: A = {D, D, D}
Now, essentially what I require after the execution of the codes is:
Mathematically, A = {D^0, D^1, D^2} = {D0, D1, D2}
where D0 = [[1,0],[0,1]], D1 = [[1,2],[3,4]], D2=[[7,10],[15,22]]
Is it possible to apply power to each matrix element in A without using a for-loop? I would be doing larger matrices with more in the series.
I had defined, n = np.array([0,1,2]) # corresponding to powers 0, 1 and 2 and tried
Result = np.power(A,n) but I do not get the desired output.
Is there are an efficient way to do it?
Full code:
D = np.array([[1,2],[3,4]])
idx = np.arange(3)
A = np.zeros((3,2,2))
A[idx,:,:] = D # This gives A = [[[1,2],[3,4]],[[1,2],[3,4]],\
# [[1,2],[3,4]]]
# In mathematical notation: A = {D, D, D}
n = np.array([0,1,2])
Result = np.power(A,n) # ------> Not the desired output.
A cumulative product exists in numpy, but not for matrices. Therefore, you need to make your own 'matcumprod' function. You can use np.dot for this, but np.matmul (or #) is specialized for matrix multiplication.
Since you state your powers always go from 0 to some_power, I suggest the following function:
def matcumprod(D, upto):
Res = np.empty((upto, *D.shape), dtype=A.dtype)
Res[0, :, :] = np.eye(D.shape[0])
Res[1, :, :] = D.copy()
for i in range(1,upto):
Res[i, :, :] = Res[i-1,:,:] # D
return Res
By the way, a loop often times outperforms a built-in numpy function if the latter uses a lot of memory, so don't fret over it if your powers stay within bounds...
Alright, i spent a lot of time on this problem but could not seem to find a vectorized solution in the way you'd like. So i would like to instead first propose a basic solution, and then perhaps an optimization if you require finding continuous powers.
The function you're looking for is called numpy.linalg.matrix_power
import numpy as np
D = np.matrix([[1,2],[3,4]])
idx = np.arange(3)
A = np.zeros((3,2,2))
A[idx,:,:] = D # This gives A = [[[1,2],[3,4]],[[1,2],[3,4]],\
# [[1,2],[3,4]]]
# In mathematical notation: A = {D, D, D}
np.zeros(A.shape)
n = np.array([0,1,2])
result = [np.linalg.matrix_power(D, i) for i in n]
np.array(result)
#Output:
array([[[ 1, 0],
[ 0, 1]],
[[ 1, 2],
[ 3, 4]],
[[ 7, 10],
[15, 22]]])
However, if you notice, you end up calculating multiple powers for the same base matrix. We could instead utilize the intermediate results and go from there, using numpy.linalg.multi_dot
def all_powers_arr_of_matrix(A):
result = np.zeros(A.shape)
result[0] = np.linalg.matrix_power(A[0], 0)
for i in range(1, A.shape[0]):
result[i] = np.linalg.multi_dot([result[i - 1], A[i]])
return result
result = all_powers_arr_of_matrix(A)
#Output:
array([[[ 1., 0.],
[ 0., 1.]],
[[ 1., 2.],
[ 3., 4.]],
[[ 7., 10.],
[15., 22.]]])
Also, we can avoid creating the matrix A entirely, saving some time.
def all_powers_matrix(D, *rangeargs): #end exclusive
''' Expects 2D matrix.
Use as all_powers_matrix(D, end) or
all_powers_matrix(D, start, end)
'''
if len(rangeargs) == 1:
start = 0
end = rangeargs[0]
elif len(rangeargs) == 2:
start = rangeargs[0]
end = rangeargs[1]
else:
print("incorrect args")
return None
result = np.zeros((end - start, *D.shape))
result[0] = np.linalg.matrix_power(A[0], start)
for i in range(start + 1, end):
result[i] = np.linalg.multi_dot([result[i - 1], D])
return result
return result
result = all_powers_matrix(D, 3)
#Output:
array([[[ 1., 0.],
[ 0., 1.]],
[[ 1., 2.],
[ 3., 4.]],
[[ 7., 10.],
[15., 22.]]])
Note that you'd need to add error handling if you decide to use these functions as-is.
To calculate power of matrix D, one way could be to find the eigenvalues and right eigenvectors of it with np.linalg.eig and then raise the power of the diagonal matrix as it is easier, then after some manipulation, you can use two np.einsum to calculate A
#get eigvalues and eigvectors
eigval, eigvect = np.linalg.eig(D)
# to check how it works, you can do:
print (np.dot(eigvect*eigval,np.linalg.inv(eigvect)))
#[[1. 2.]
# [3. 4.]]
# so you get back on D
#use power as ufunc of outer with n on the eigenvalues to get all the one you want
arrp = np.power.outer( eigval, n).T
#apply_along_axis to create the diagonal matrix along the last axis
diagp = np.apply_along_axis( np.diag, axis=-1, arr=arrp)
#finally use two np.einsum to calculate with the subscript to get what you want
A = np.einsum('lij,jk -> lik',
np.einsum('ij,kjl -> kil',eigvect,diagp), np.linalg.inv(eigvect)).round()
print (A)
print (A.shape)
#[[[ 1. 0.]
# [-0. 1.]]
#
# [[ 1. 2.]
# [ 3. 4.]]
#
# [[ 7. 10.]
# [15. 22.]]]
#
#(3, 2, 2)
I don't have a full solution, but there are some things I wanted to mention which are a bit too long for the comments.
You might first look into addition chain exponentiation if you are computing big powers of big matrices. This is basically asking how many matrix multiplications are required to compute A^k for a given k. For instance A^5 = A(A^2)^2 so you need to only three matrix multiplies: A^2 and (A^2)^2 and A(A^2)^2. This might be the simplest way to gain some efficiency, but you will probably still have to use explicit loops.
Your question is also related to the problem of computing Ax, A^2x, ... , A^kx for a given A and x. This is an active area of research right now (search "matrix powers kernel"), since computing such a sequence efficiently is useful for parallel/communication avoiding Krylov subspace methods. If you're looking for a very efficient solution to your problem it might be worth looking into some of the results about this.
suppose I have this 2d array A:
[[0,0,0,0],
[0,0,0,0],
[0,0,0,0],
[0,0,0,4]]
and I want to sum B:
[[1,2,3]
[4,5,6]
[7,8,9]]
centered on A[0][0] so the result would be:
array_sum(A,B,0,0) =
[[5,6,0,4],
[8,9,0,0],
[0,0,0,0],
[2,0,0,5]]
I was thinking that I should make a function that compares if its on a boundary and then adjust the index for that:
def array_sum(A,B,i,f):
...
if i == 0 and j == 0:
A[-1][-1] = A[-1][-1]+B[0][0]
...
else:
A[i-1][j-1] = A[i][j]+B[0][0]
A[i][j] = A[i][j]+B[1][1]
A[i+1][j+1] = A[i][j]+B[2][2]
...
but I don't know if there is a better way of doing that, I was reading about broadcasting or maybe using convolute for that, but I'm not sure if there is a better way to do that.
Assuming B.shape is all odd numbers, you can use np.indices, manipulate them to point where you want, and use np.add.at
def array_sum(A, B, loc = (0, 0)):
A_ = A.copy()
ix = np.indices(B.shape)
new_loc = np.array(loc) - np.array(B.shape) // 2
new_ix = np.mod(ix + new_loc[:, None, None],
np.array(A.shape)[:, None, None])
np.add.at(A_, tuple(new_ix), B)
return A_
Testing:
array_sum(A, B)
Out:
array([[ 5., 6., 0., 4.],
[ 8., 9., 0., 7.],
[ 0., 0., 0., 0.],
[ 2., 3., 0., 5.]])
As a rule of thumb slice indexing is faster (~2x) than fancy indexing. This appears to be true even for the small example in OP. Downside: the code is slightly more complicated.
import numpy as np
from numpy import s_ as _
from itertools import product, starmap
def wrapsl1d(N, n, c):
# check in 1D whether a patch of size n centered at c in a vector
# of length N fits or has to be wrapped around
# return appropriate slice objects for both vector and patch
assert n <= N
l = (c - n//2) % N
h = l + n
# return list of pairs (index into A, index into patch)
# 2 pairs if we wrap around, otherwise 1 pair
return [_[l:h, :]] if h <= N else [_[l:, :N-l], _[:h-N, n+N-h:]]
def use_slices(A, patch, center=(0, 0)):
slAptch = product(*map(wrapsl1d, A.shape, patch.shape, center))
# the product now has elements [(idx0A, idx0ptch), (idx1A, idx1ptch)]
# transpose them:
slAptch = starmap(zip, slAptch)
out = A.copy()
for sa, sp in slAptch:
out[sa] += patch[sp]
return out
I am realizing Exponentiation of a matrix using FOR:
import numpy as np
fl=2
cl=2
fl2=fl
cl2=cl
M = random.random((fl,cl))
M2 = M
Result = np.zeros((fl,cl))
Temp = np.zeros((fl,cl))
itera = 2
print('Matriz A:\n',M)
print('Matriz AxA:\n',M2)
for i in range (0,itera):
for a in range(0,fl):
for b in range (0,cl):
Result[a,b]+=M[a,b]*M[a,b]
temp[a,b]=Result[a,b]
Res[a,k]=M[a,b]
print('Potencia:\n',temp)
print('Matriz:\n', Result)
The error is that it does not perform well the multiplication in Result[a,b]+=M[a,b]*M[a,b] and when I save it in a temporary matrix to multiply it with the original matrix, it does not make the next jump in for i in range (0,itera):
I know I can perform the function np.matmul
but I try to do it with the FOR loop
Example
You're looking for np.linalg.matrix_power.
If you're using numpy, don't use a for loop, use a vectorized operation.
arr = np.arange(16).reshape((4,4))
np.linalg.matrix_power(arr, 3)
array([[ 1680, 1940, 2200, 2460],
[ 4880, 5620, 6360, 7100],
[ 8080, 9300, 10520, 11740],
[11280, 12980, 14680, 16380]])
Which is the same as the explicit multiplication:
arr # arr # arr
>>> np.array_equal(arr # arr # arr, np.linalg.matrix_power(arr, 3))
True
Since you asked
If you really want a naive solution using loops, we can put together the pieces quite easily. First we need a way to actually multiple the matrices. There are options that beat n^3 complexity, this answer is not going to do that. Here is a basic matrix multiplication function:
def matmultiply(a, b):
res = np.zeros(a.shape)
size = a.shape[0]
for i in range(size):
for j in range(size):
for k in range(size):
res[i][j] += a[i][k] * b[k][j]
return res
Now you need an exponential function. This function takes a matrix and a power, and raises a matrix to that power.
def loopy_matrix_power(a, n):
res = np.identity(a.shape[0])
while n > 0:
if n % 2 == 0:
a = matmultiply(a, a)
n /= 2
else:
res = matmultiply(res, a)
n -= 1
return res
In action:
loopy_matrix_power(arr, 3)
array([[ 1680., 1940., 2200., 2460.],
[ 4880., 5620., 6360., 7100.],
[ 8080., 9300., 10520., 11740.],
[11280., 12980., 14680., 16380.]])
There are some problems here:
you do not reset the result matrix after multiplication is done, hence you keep adding more values; and
you never assign the result back to m to perform a next generation of multiplications.
Naive power implementation
I think it is also better to "encapsulate" matrix multiplication in a separate function, like:
def matmul(a1, a2):
m, ka = a1.shape
kb, n = a2.shape
if ka != kb:
raise ValueError()
res = np.zeros((m, n))
for i in range(m):
for j in range(n):
d = 0.0
for k in range(ka):
d += a1[i,k] * a2[k,j]
res[i, j] = d
return res
Then we can calculate the power of this matrix with:
m2 = m
for i in range(topow-1):
m = matmul(m, m2)
Note that we can not use m here as the only matrix. Since if we write m = matmul(m, m), then m is now m2. But that means that if we perform the multiplication a second time, we get m4 instead of m3.
This then produces the expected results:
>>> cross = np.array([[1,0,1],[0,1,0], [1,0,1]])
>>> matmul(cross, cross)
array([[2., 0., 2.],
[0., 1., 0.],
[2., 0., 2.]])
>>> matmul(cross, matmul(cross, cross))
array([[4., 0., 4.],
[0., 1., 0.],
[4., 0., 4.]])
>>> matmul(cross, matmul(cross, matmul(cross, cross)))
array([[8., 0., 8.],
[0., 1., 0.],
[8., 0., 8.]])
Logarithmic power multiplication
The above can calculate the Mn in O(n) (linear time), but we can do better, we can calculate this matrix in logarithmic time: we do this by looking if the power is 1, if it is, we simply return the matrix, if it is not, we check if the power is even, if it is even, we multiply the matrix with itself, and calculate the power of that matrix, but with the power divided by two, so M2 n=(M×M)n. If the power is odd, we do more or less the same, except that we multiply it with the original value for M: M2 n + 1=M×(M×M)n. Like:
def matpow(m, p):
if p <= 0:
raise ValueError()
if p == 1:
return m
elif p % 2 == 0: # even
return matpow(matmul(m, m), p // 2)
else: # odd
return matmul(m, matpow(matmul(m, m), p // 2))
The above can be written more elegantly, but I leave this as an exercise :).
Note however that using numpy arrays for scalar comuputations is typically less efficient than using the matrix multiplication (and other functions) numpy offers. These are optimized, and are not interpreted, and typically outperform Python equivalents significantly. Therefore I would really advice you to use these. The numpy functions are also tested, making it less likely that there are bugs in it.
TL;DR:
What is the theano.scan equivalent of:
M = np.arange(9).reshape(3, 3)
for i in range(M.shape[0]):
for j in range(M.shape[1]):
M[i, j] += 5
M
possibly (if doable) without using nested scans?
Note that this question does not want to be specifically about how to apply an operation elementwise to a matrix, but more generally on how to implement with theano.scan a nested looping construct like the above.
Long version:
theano.scan (or equivalently in this case, theano.map) allows to map a function looping through multiple indices, by simply providing a sequence of elements to the sequences arguments, with something like
import theano
import theano.tensor as T
M = T.dmatrix('M')
def map_func(i, j, matrix):
return matrix[i, j] + i * j
results, updates = theano.scan(map_func,
sequences=[T.arange(M.shape[0]), T.arange(M.shape[1])],
non_sequences=[M])
f = theano.function(inputs=[M], outputs=results)
f(np.arange(9).reshape(3, 3))
#
which is roughly equivalent to a python loop of the form:
M = np.arange(9).reshape(3, 3)
for i, j in zip(np.arange(M.shape[0]), np.arange(M.shape[1])):
M[i, j] += 5
M
which increases by 5 all the elements in the diagonal of M.
But what if I want to find the theano.scan equivalent of:
M = np.arange(9).reshape(3, 3)
for i in range(M.shape[0]):
for j in range(M.shape[1]):
M[i, j] += 5
M
possibly without nesting scan?
One way is of course to flatten the matrix, scan through the flattened elements, and then reshape it to the original shape, with something like
import theano
import theano.tensor as T
M = T.dmatrix('M')
def map_func(i, X):
return X[i] + .5
M_flat = T.flatten(M)
results, updates = theano.map(map_func,
sequences=T.arange(M.shape[0] * M.shape[1]),
non_sequences=M_flat)
final_M = T.reshape(results, M.shape)
f = theano.function([M], final_M)
f([[1, 2], [3, 4]])
but is there a better way that doesn't involve explicitly flattening and reshaping the matrix?
Here is an example on how this kind of thing can be achieve using nested theano.scan calls.
In this example we add the number 3.141 to every element of a matrix, effectively simulating in a convoluted way the output of H + 3.141:
H = T.dmatrix('H')
def fn2(col, row, matrix):
return matrix[row, col] + 3.141
def fn(row, matrix):
res, updates = theano.scan(fn=fn2,
sequences=T.arange(matrix.shape[1]),
non_sequences=[row, matrix])
return res
results, updates = theano.scan(fn=fn,
sequences=T.arange(H.shape[0]),
non_sequences=[H])
f = theano.function([H], results)
f([[0, 1], [2, 3]])
# array([[ 3.141, 4.141],
# [ 5.141, 6.141]])
As another example, let us add to each element of a matrix the product of its row and column indices:
H = T.dmatrix('H')
def fn2(col, row, matrix):
return matrix[row, col] + row * col
def fn(row, matrix):
res, updates = theano.scan(fn=fn2,
sequences=T.arange(matrix.shape[1]),
non_sequences=[row, matrix])
return res
results, updates = theano.scan(fn=fn,
sequences=T.arange(H.shape[0]),
non_sequences=[H])
f = theano.function([H], results)
f(np.arange(9).reshape(3, 3))
# Out[2]:array([[ 0., 1., 2.],
# [ 3., 5., 7.],
# [ 6., 9., 12.]])