Related
I want to create a convolver function without using the convolve function of NumPy that get 4 elements:
convolver_1(j, matrices_list, filter_matrix, stride_size)
The function should return the output after performing a convolution operation with filter_matrix, with a stride of size stride_size, over matrix in index j at matrices_list, filter_matrix has lower dimensions than the matrices in matrices_list.
Let's go through a simple implementation of np.convolve whose documentation can be found here.
import numpy as np
def convolve_1d(a, filter):
N, M = len(a), len(filter)
assert N >= M # assumption in the question
# in the full mode (default of np.convolve), result length is N+M-1
# therefore, pad should be M-1 on each side
# N-M+2p+1 == N+M-1 => p = M-1
result_length = N+M-1
result = np.zeros(result_length) # allocating memory for result
p = M-1
padded_a = np.pad(a,p)
flipped_filter = np.flip(filter)
for i in range(result_length):
result[i] = np.sum(np.multiply(flipped_filter, padded_a[i:i+M]))
return result
a = np.array([1,2,3,4])
filter = np.array([1,-1,3])
convolve_1d(a, filter)
results in
array([ 1., 1., 4., 7., 5., 12.])
which is the same as the result for np.convolve(a, filter).
So, it basically pads the input array with zeros, flips the filter and sums the element-wise multiplication of two arrays.
I am not sure about the index that you mentioned; the result is a 1d array and you can index its elements.
To add stride to this function, we need to modify the result_length and multiply the stride to the iterator:
def convolve_1d_strided(a, filter, stride):
N, M = len(a), len(filter)
assert N >= M # assumption in the question
# in the full mode (default of np.convolve), result length is N+M-1
# therefore, pad should be M-1 on each side
# N-M+2p+1 == N+M-1 => p = M-1
result_length = (N+M-1)//stride
result = np.zeros(result_length) # allocating memory for result
p = M-1
padded_a = np.pad(a,p)
flipped_filter = np.flip(filter)
for i in range(result_length):
result[i] = np.sum(np.multiply(flipped_filter, padded_a[i*stride:i*stride+M]))
return result
a = np.array([1,2,3,4])
filter = np.array([1,-1,3])
convolve_1d_strided(a, filter, 2)
array([1., 4., 5.])
Hope it helps and if that is what you liked to see, I am happy to expand it to two dimensions.
For 1D arrays:
import numpy as np
from numpy.lib.stride_tricks import as_strided
def conv1d(A, kernel, stride, reverse_kernel=True, mode='full'):
if reverse_kernel:
kernel = kernel[::-1]
if mode == 'full':
A = np.pad(A, kernel.shape[0] - 1)
#else: convolution in 'valid' mode
# Sliding-window view of A
output_size = (A.shape[0] - kernel.shape[0])//stride + 1
A_w = as_strided(A, shape=(output_size, kernel.shape[0]),
strides=(stride*A.strides[0], A.strides[0]))
# Return convolution of A with kernel
return np.sum(A_w * kernel, axis=1)
Here A = matrices_list[j]. Note that in Deep Learning the filters in convolution are not reversed.
Given a function like my_function(x,y) that takes two ndarrays x and y as an input and outputs a scalar:
def my_function(x,y):
perm = np.take(x, y)
return np.sum((np.power(2, perm) - 1) / (np.log2(np.arange(3, k + 3))))
I want to find a way to apply it to two matrices r and p
r = np.asarray([[5,6,7],[8,9,10]])
p = np.asarray([[2,1,0],[0,2,1]])
in such a way that an ndarray is returned with the values
np.asarray([my_function([5,6,7],[2,1,0]), my_function([8,9,10],[0,2,1])
You can slightly modify your function to use take_along_axis instead of take, which will allow you to adapt to the 2D solution.
def my_function_2d(x, y, k=1):
t = np.take_along_axis(x, y, -1)
u = np.power(2, t) - 1
v = np.log2(np.arange(3, k+3))
return (u / v).sum(-1)
my_function_2d(r, p, k=1)
array([ 139.43547554, 1128.73332914])
Validation
In [96]: k = 1
In [97]: my_function([5,6,7],[2,1,0])
Out[97]: 139.4354755392921
In [98]: my_function([8,9,10],[0,2,1])
Out[98]: 1128.7333291393375
This will also still work on the 1D case:
In [145]: my_function_2d(r[0], p[0], k=1)
Out[145]: 139.4354755392921
This approach generalizes to the N-dimensional case:
In [157]: r = np.random.randint(1, 5, (2, 2, 2, 2, 2, 3))
In [158]: p = np.random.randint(0, r.shape[-1], r.shape)
In [159]: my_function_2d(r, p, k=3)
Out[159]:
array([[[[[ 8.34718483, 14.25597598],
[12.25597598, 19.97868221]],
[[12.97868221, 4.68481893],
[ 2.42295943, 1.56160631]]],
[[[23.42409467, 9.82346582],
[10.93124418, 16.42409467]],
[[23.42409467, 1.56160631],
[ 3.68481893, 10.68481893]]]],
[[[[15.97868221, 10.93124418],
[ 5.40752517, 14.93124418]],
[[ 4.14566566, 6.34718483],
[14.93124418, 3.68481893]]],
[[[ 9.20853795, 13.39462286],
[23.42409467, 3.82346582]],
[[23.42409467, 9.85293763],
[ 4.56160631, 10.93124418]]]]])
I assume you realize your approach doesn't work for all inputs and ks, there are some shape requirements
You can try either map or a list comprehension with zip as following. Please note that I took k=1 to have a running code as you did not specify k
def my_function(x,y):
k=1
perm = np.take(x, y)
return np.sum((np.power(2, perm) - 1) / (np.log2(np.arange(3, k + 3))))
r = np.asarray([[5,6,7],[8,9,10]])
p = np.asarray([[2,1,0],[0,2,1]])
result = np.asarray([my_function(i, j) for i, j in zip(r, p)])
print (result)
# [ 139.43547554 1128.73332914]
You can use np.vectorize with the signature keyword:
k = 3
np.vectorize(my_function, signature='(i),(i)->()')(r, p)
# array([124.979052 , 892.46280834])
3D numpy array A contains a series (in this example, I am choosing 3) of 2D numpy array D of shape 2 x 2. The D matrix is as follows:
D = np.array([[1,2],[3,4]])
A is initialized and assigned as below:
idx = np.arange(3)
A = np.zeros((3,2,2))
A[idx,:,:] = D # This gives A = [[[1,2],[3,4]],[[1,2],[3,4]],\
# [[1,2],[3,4]]]
# In mathematical notation: A = {D, D, D}
Now, essentially what I require after the execution of the codes is:
Mathematically, A = {D^0, D^1, D^2} = {D0, D1, D2}
where D0 = [[1,0],[0,1]], D1 = [[1,2],[3,4]], D2=[[7,10],[15,22]]
Is it possible to apply power to each matrix element in A without using a for-loop? I would be doing larger matrices with more in the series.
I had defined, n = np.array([0,1,2]) # corresponding to powers 0, 1 and 2 and tried
Result = np.power(A,n) but I do not get the desired output.
Is there are an efficient way to do it?
Full code:
D = np.array([[1,2],[3,4]])
idx = np.arange(3)
A = np.zeros((3,2,2))
A[idx,:,:] = D # This gives A = [[[1,2],[3,4]],[[1,2],[3,4]],\
# [[1,2],[3,4]]]
# In mathematical notation: A = {D, D, D}
n = np.array([0,1,2])
Result = np.power(A,n) # ------> Not the desired output.
A cumulative product exists in numpy, but not for matrices. Therefore, you need to make your own 'matcumprod' function. You can use np.dot for this, but np.matmul (or #) is specialized for matrix multiplication.
Since you state your powers always go from 0 to some_power, I suggest the following function:
def matcumprod(D, upto):
Res = np.empty((upto, *D.shape), dtype=A.dtype)
Res[0, :, :] = np.eye(D.shape[0])
Res[1, :, :] = D.copy()
for i in range(1,upto):
Res[i, :, :] = Res[i-1,:,:] # D
return Res
By the way, a loop often times outperforms a built-in numpy function if the latter uses a lot of memory, so don't fret over it if your powers stay within bounds...
Alright, i spent a lot of time on this problem but could not seem to find a vectorized solution in the way you'd like. So i would like to instead first propose a basic solution, and then perhaps an optimization if you require finding continuous powers.
The function you're looking for is called numpy.linalg.matrix_power
import numpy as np
D = np.matrix([[1,2],[3,4]])
idx = np.arange(3)
A = np.zeros((3,2,2))
A[idx,:,:] = D # This gives A = [[[1,2],[3,4]],[[1,2],[3,4]],\
# [[1,2],[3,4]]]
# In mathematical notation: A = {D, D, D}
np.zeros(A.shape)
n = np.array([0,1,2])
result = [np.linalg.matrix_power(D, i) for i in n]
np.array(result)
#Output:
array([[[ 1, 0],
[ 0, 1]],
[[ 1, 2],
[ 3, 4]],
[[ 7, 10],
[15, 22]]])
However, if you notice, you end up calculating multiple powers for the same base matrix. We could instead utilize the intermediate results and go from there, using numpy.linalg.multi_dot
def all_powers_arr_of_matrix(A):
result = np.zeros(A.shape)
result[0] = np.linalg.matrix_power(A[0], 0)
for i in range(1, A.shape[0]):
result[i] = np.linalg.multi_dot([result[i - 1], A[i]])
return result
result = all_powers_arr_of_matrix(A)
#Output:
array([[[ 1., 0.],
[ 0., 1.]],
[[ 1., 2.],
[ 3., 4.]],
[[ 7., 10.],
[15., 22.]]])
Also, we can avoid creating the matrix A entirely, saving some time.
def all_powers_matrix(D, *rangeargs): #end exclusive
''' Expects 2D matrix.
Use as all_powers_matrix(D, end) or
all_powers_matrix(D, start, end)
'''
if len(rangeargs) == 1:
start = 0
end = rangeargs[0]
elif len(rangeargs) == 2:
start = rangeargs[0]
end = rangeargs[1]
else:
print("incorrect args")
return None
result = np.zeros((end - start, *D.shape))
result[0] = np.linalg.matrix_power(A[0], start)
for i in range(start + 1, end):
result[i] = np.linalg.multi_dot([result[i - 1], D])
return result
return result
result = all_powers_matrix(D, 3)
#Output:
array([[[ 1., 0.],
[ 0., 1.]],
[[ 1., 2.],
[ 3., 4.]],
[[ 7., 10.],
[15., 22.]]])
Note that you'd need to add error handling if you decide to use these functions as-is.
To calculate power of matrix D, one way could be to find the eigenvalues and right eigenvectors of it with np.linalg.eig and then raise the power of the diagonal matrix as it is easier, then after some manipulation, you can use two np.einsum to calculate A
#get eigvalues and eigvectors
eigval, eigvect = np.linalg.eig(D)
# to check how it works, you can do:
print (np.dot(eigvect*eigval,np.linalg.inv(eigvect)))
#[[1. 2.]
# [3. 4.]]
# so you get back on D
#use power as ufunc of outer with n on the eigenvalues to get all the one you want
arrp = np.power.outer( eigval, n).T
#apply_along_axis to create the diagonal matrix along the last axis
diagp = np.apply_along_axis( np.diag, axis=-1, arr=arrp)
#finally use two np.einsum to calculate with the subscript to get what you want
A = np.einsum('lij,jk -> lik',
np.einsum('ij,kjl -> kil',eigvect,diagp), np.linalg.inv(eigvect)).round()
print (A)
print (A.shape)
#[[[ 1. 0.]
# [-0. 1.]]
#
# [[ 1. 2.]
# [ 3. 4.]]
#
# [[ 7. 10.]
# [15. 22.]]]
#
#(3, 2, 2)
I don't have a full solution, but there are some things I wanted to mention which are a bit too long for the comments.
You might first look into addition chain exponentiation if you are computing big powers of big matrices. This is basically asking how many matrix multiplications are required to compute A^k for a given k. For instance A^5 = A(A^2)^2 so you need to only three matrix multiplies: A^2 and (A^2)^2 and A(A^2)^2. This might be the simplest way to gain some efficiency, but you will probably still have to use explicit loops.
Your question is also related to the problem of computing Ax, A^2x, ... , A^kx for a given A and x. This is an active area of research right now (search "matrix powers kernel"), since computing such a sequence efficiently is useful for parallel/communication avoiding Krylov subspace methods. If you're looking for a very efficient solution to your problem it might be worth looking into some of the results about this.
suppose I have this 2d array A:
[[0,0,0,0],
[0,0,0,0],
[0,0,0,0],
[0,0,0,4]]
and I want to sum B:
[[1,2,3]
[4,5,6]
[7,8,9]]
centered on A[0][0] so the result would be:
array_sum(A,B,0,0) =
[[5,6,0,4],
[8,9,0,0],
[0,0,0,0],
[2,0,0,5]]
I was thinking that I should make a function that compares if its on a boundary and then adjust the index for that:
def array_sum(A,B,i,f):
...
if i == 0 and j == 0:
A[-1][-1] = A[-1][-1]+B[0][0]
...
else:
A[i-1][j-1] = A[i][j]+B[0][0]
A[i][j] = A[i][j]+B[1][1]
A[i+1][j+1] = A[i][j]+B[2][2]
...
but I don't know if there is a better way of doing that, I was reading about broadcasting or maybe using convolute for that, but I'm not sure if there is a better way to do that.
Assuming B.shape is all odd numbers, you can use np.indices, manipulate them to point where you want, and use np.add.at
def array_sum(A, B, loc = (0, 0)):
A_ = A.copy()
ix = np.indices(B.shape)
new_loc = np.array(loc) - np.array(B.shape) // 2
new_ix = np.mod(ix + new_loc[:, None, None],
np.array(A.shape)[:, None, None])
np.add.at(A_, tuple(new_ix), B)
return A_
Testing:
array_sum(A, B)
Out:
array([[ 5., 6., 0., 4.],
[ 8., 9., 0., 7.],
[ 0., 0., 0., 0.],
[ 2., 3., 0., 5.]])
As a rule of thumb slice indexing is faster (~2x) than fancy indexing. This appears to be true even for the small example in OP. Downside: the code is slightly more complicated.
import numpy as np
from numpy import s_ as _
from itertools import product, starmap
def wrapsl1d(N, n, c):
# check in 1D whether a patch of size n centered at c in a vector
# of length N fits or has to be wrapped around
# return appropriate slice objects for both vector and patch
assert n <= N
l = (c - n//2) % N
h = l + n
# return list of pairs (index into A, index into patch)
# 2 pairs if we wrap around, otherwise 1 pair
return [_[l:h, :]] if h <= N else [_[l:, :N-l], _[:h-N, n+N-h:]]
def use_slices(A, patch, center=(0, 0)):
slAptch = product(*map(wrapsl1d, A.shape, patch.shape, center))
# the product now has elements [(idx0A, idx0ptch), (idx1A, idx1ptch)]
# transpose them:
slAptch = starmap(zip, slAptch)
out = A.copy()
for sa, sp in slAptch:
out[sa] += patch[sp]
return out
I am realizing Exponentiation of a matrix using FOR:
import numpy as np
fl=2
cl=2
fl2=fl
cl2=cl
M = random.random((fl,cl))
M2 = M
Result = np.zeros((fl,cl))
Temp = np.zeros((fl,cl))
itera = 2
print('Matriz A:\n',M)
print('Matriz AxA:\n',M2)
for i in range (0,itera):
for a in range(0,fl):
for b in range (0,cl):
Result[a,b]+=M[a,b]*M[a,b]
temp[a,b]=Result[a,b]
Res[a,k]=M[a,b]
print('Potencia:\n',temp)
print('Matriz:\n', Result)
The error is that it does not perform well the multiplication in Result[a,b]+=M[a,b]*M[a,b] and when I save it in a temporary matrix to multiply it with the original matrix, it does not make the next jump in for i in range (0,itera):
I know I can perform the function np.matmul
but I try to do it with the FOR loop
Example
You're looking for np.linalg.matrix_power.
If you're using numpy, don't use a for loop, use a vectorized operation.
arr = np.arange(16).reshape((4,4))
np.linalg.matrix_power(arr, 3)
array([[ 1680, 1940, 2200, 2460],
[ 4880, 5620, 6360, 7100],
[ 8080, 9300, 10520, 11740],
[11280, 12980, 14680, 16380]])
Which is the same as the explicit multiplication:
arr # arr # arr
>>> np.array_equal(arr # arr # arr, np.linalg.matrix_power(arr, 3))
True
Since you asked
If you really want a naive solution using loops, we can put together the pieces quite easily. First we need a way to actually multiple the matrices. There are options that beat n^3 complexity, this answer is not going to do that. Here is a basic matrix multiplication function:
def matmultiply(a, b):
res = np.zeros(a.shape)
size = a.shape[0]
for i in range(size):
for j in range(size):
for k in range(size):
res[i][j] += a[i][k] * b[k][j]
return res
Now you need an exponential function. This function takes a matrix and a power, and raises a matrix to that power.
def loopy_matrix_power(a, n):
res = np.identity(a.shape[0])
while n > 0:
if n % 2 == 0:
a = matmultiply(a, a)
n /= 2
else:
res = matmultiply(res, a)
n -= 1
return res
In action:
loopy_matrix_power(arr, 3)
array([[ 1680., 1940., 2200., 2460.],
[ 4880., 5620., 6360., 7100.],
[ 8080., 9300., 10520., 11740.],
[11280., 12980., 14680., 16380.]])
There are some problems here:
you do not reset the result matrix after multiplication is done, hence you keep adding more values; and
you never assign the result back to m to perform a next generation of multiplications.
Naive power implementation
I think it is also better to "encapsulate" matrix multiplication in a separate function, like:
def matmul(a1, a2):
m, ka = a1.shape
kb, n = a2.shape
if ka != kb:
raise ValueError()
res = np.zeros((m, n))
for i in range(m):
for j in range(n):
d = 0.0
for k in range(ka):
d += a1[i,k] * a2[k,j]
res[i, j] = d
return res
Then we can calculate the power of this matrix with:
m2 = m
for i in range(topow-1):
m = matmul(m, m2)
Note that we can not use m here as the only matrix. Since if we write m = matmul(m, m), then m is now m2. But that means that if we perform the multiplication a second time, we get m4 instead of m3.
This then produces the expected results:
>>> cross = np.array([[1,0,1],[0,1,0], [1,0,1]])
>>> matmul(cross, cross)
array([[2., 0., 2.],
[0., 1., 0.],
[2., 0., 2.]])
>>> matmul(cross, matmul(cross, cross))
array([[4., 0., 4.],
[0., 1., 0.],
[4., 0., 4.]])
>>> matmul(cross, matmul(cross, matmul(cross, cross)))
array([[8., 0., 8.],
[0., 1., 0.],
[8., 0., 8.]])
Logarithmic power multiplication
The above can calculate the Mn in O(n) (linear time), but we can do better, we can calculate this matrix in logarithmic time: we do this by looking if the power is 1, if it is, we simply return the matrix, if it is not, we check if the power is even, if it is even, we multiply the matrix with itself, and calculate the power of that matrix, but with the power divided by two, so M2 n=(M×M)n. If the power is odd, we do more or less the same, except that we multiply it with the original value for M: M2 n + 1=M×(M×M)n. Like:
def matpow(m, p):
if p <= 0:
raise ValueError()
if p == 1:
return m
elif p % 2 == 0: # even
return matpow(matmul(m, m), p // 2)
else: # odd
return matmul(m, matpow(matmul(m, m), p // 2))
The above can be written more elegantly, but I leave this as an exercise :).
Note however that using numpy arrays for scalar comuputations is typically less efficient than using the matrix multiplication (and other functions) numpy offers. These are optimized, and are not interpreted, and typically outperform Python equivalents significantly. Therefore I would really advice you to use these. The numpy functions are also tested, making it less likely that there are bugs in it.