I am trying to compute many dot products efficiently. This post gets really close to what I am trying to do, but I can't quite get it to work. I have a large list of matrices (a), and a list of vectors (b). I want to do a series of dot product operations between them. This is what works now:
import numpy as np
a # shape (15000,4,4)
b # shape (15000,4)
out = np.empty((15000,4))
for i in range(15000):
out[i] = np.dot(a[i],b[i])
All my attempts to adapt np.tensordot or np.einsum from the linked post have failed to give me what I want. If anyone sees how to do this I would really appreciate it.
Einstein summation works just fine:
>>> a = np.random.randn(100, 4, 4)
>>> b = np.random.randn(100, 4)
>>> foo = np.einsum('ijk,ik->ij', a, b)
>>> bar = np.zeros_like(foo)
>>> for i, (ai, bi) in enumerate(zip(a, b)):
bar[i] = np.dot(ai, bi)
>>> np.allclose(foo, bar)
True
To explain the summation a bit, note that you're contracting the last axis of b. So you can think about doing each inner product as if by np.einsum('jk,k->j', a[0], b[0]). But we're doing one for each element of a and b, thus the inclusion of the first axis, which is not contracted. Hence, ijk,ik->ij.
Related
I have three NumPy arrays of ints, same number of columns, arbitrary number of rows each. I am interested in all instances where a row of the first one plus a row of the second one gives a row of the third one ([3, 1, 4] + [1, 5, 9] = [4, 6, 13]).
Here is a pseudo-code:
for i, j in rows(array1), rows(array2):
if i + j is in rows(array3):
somehow store the rows this occured at (eg. (1,2,5) if 1st row of
array1 + 2nd row of array2 give 5th row of array3)
I will need to run this for very big matrices so I have two questions:
(1) I can write the above using nested loops but is there a quicker way, perhaps list comprehensions or itertools?
(2) What is the fastest/most memory-efficient way to store the triples? Later I will need to create a heatmap using two as coordinates and the first one as the corresponding value eg. point (2,5) has value 1 in the pseudo-code example.
Would be very grateful for any tips - I know this sounds quite simple but it needs to run fast and I have very little experience with optimization.
edit: My ugly code was requested in comments
import numpy as np
#random arrays
A = np.array([[-1,0],[0,-1],[4,1], [-1,2]])
B = np.array([[1,2],[0,3],[3,1]])
C = np.array([[0,2],[2,3]])
#triples stored as numbers with 2 coordinates in a otherwise-zero matrix
output_matrix = np.zeros((B.shape[0], C.shape[0]), dtype = int)
for i in range(A.shape[0]):
for j in range(B.shape[0]):
for k in range(C.shape[0]):
if np.array_equal((A[i,] + B[j,]), C[k,]):
output_matrix[j, k] = i+1
print(output_matrix)
We can leverage broadcasting to perform all those summations and comparison in a vectorized manner and then use np.where on it to get the indices corresponding to the matching ones and finally index and assign -
output_matrix = np.zeros((B.shape[0], C.shape[0]), dtype = int)
mask = ((A[:,None,None,:] + B[None,:,None,:]) == C).all(-1)
I,J,K = np.where(mask)
output_matrix[J,K] = I+1
(1) Improvements
You can use sets for the final result in the third matrix, as a + b = c must hold identically. This already replaces one nested loop with a constant-time lookup. I will show you an example of how to do this below, but we first ought to introduce some notation.
For a set-based approach to work, we need a hashable type. Lists will thus not work, but a tuple will: it is an ordered, immutable structure. There is, however, a problem: tuple addition is defined as appending, that is,
(0, 1) + (1, 0) = (0, 1, 1, 0).
This will not do for our use-case: we need element-wise addition. As such, we subclass the built-in tuple as follows,
class AdditionTuple(tuple):
def __add__(self, other):
"""
Element-wise addition.
"""
if len(self) != len(other):
raise ValueError("Undefined behaviour!")
return AdditionTuple(self[idx] + other[idx]
for idx in range(len(self)))
Where we override the default behaviour of __add__. Now that we have a data-type amenable to our problem, let's prepare the data.
You give us,
A = [[-1, 0], [0, -1], [4, 1], [-1, 2]]
B = [[1, 2], [0, 3], [3, 1]]
C = [[0, 2], [2, 3]]
To work with. I say,
from types import SimpleNamespace
A = [AdditionTuple(item) for item in A]
B = [AdditionTuple(item) for item in B]
C = {tuple(item): SimpleNamespace(idx=idx, values=[])
for idx, item in enumerate(C)}
That is, we modify A and B to use our new data-type, and turn C into a dictionary which supports (amortised) O(1) look-up times.
We can now do the following, eliminating one loop altogether,
from itertools import product
for a, b in product(enumerate(A), enumerate(B)):
idx_a, a_i = a
idx_b, b_j = b
if a_i + b_j in C: # a_i + b_j == c_k, identically
C[a_i + b_j].values.append((idx_a, idx_b))
Then,
>>>print(C)
{(2, 3): namespace(idx=1, values=[(3, 2)]), (0, 2): namespace(idx=0, values=[(0, 0), (1, 1)])}
Where for each value in C, you get the index of that value (as idx), and a list of tuples of (idx_a, idx_b) whose elements of A and B together sum to the value at idx in C.
Let us briefly analyse the complexity of this algorithm. Redefining the lists A, B, and C as above is linear in the length of the lists. Iterating over A and B is of course in O(|A| * |B|), and the nested condition computes the element-wise addition of the tuples: this is linear in the length of the tuples themselves, which we shall denote k. The whole algorithm then runs in O(k * |A| * |B|).
This is a substantial improvement over your current O(k * |A| * |B| * |C|) algorithm.
(2) Matrix plotting
Use a dok_matrix, a sparse SciPy matrix representation. Then you can use any heatmap-plotting library you like on the matrix, e.g. Seaborn's heatmap.
Let's presume that we have numpy arrays A.shape = (1, 2) and B.shape = (2, 2) and a count X.
If X = 1, we want the result = A.dot(B)
If X = 2, we want the result = (A.dot(B)).dot(B)
If X = 3, we want the result = ((A.dot(B)).dot(B)).dot(B)
How can we write a function in Python that gives us the correct result based 9n X? Tried to loop through the dot products based upon X, but the code is very messy and not working properly.
One liner using matrix multiplication commutativity and matrix_power:
A.dot(np.linalg.matrix_power(B, X))
I am searching for a faster and maybe more elegant way to compute the following:
I have a matrix A and I want to compute the row-wise dot product of A. Herby I want to compute Ai.T * Ai, whereby index i indicates the ith row of matrix A.
import numpy as np
A=np.arange(40).reshape(20,2)
sol=[np.dot(A[ii,:].reshape(1,2).T,A[ii,:].reshape(1,2)) for ii in range(20)]
This results in a matrix of shape np.shape(sol) #=(20,2,2)
I already had a look at np.einsum, but could not make it work so far.
If there only exists a solution, where all 20 2x2 matrices are summed, this is also okay, since I want to sum it anyway in the end :)
Thanks
Using np.dot -
A.T.dot(A)
Using np.einsum -
np.einsum('ij,ik->jk',A,A)
Sample run -
>>> A=np.arange(40).reshape(20,2)
>>> sol=[np.dot(A[ii,:].reshape(1,2).T,A[ii,:].reshape(1,2)) for ii in range(20)]
>>> sol = np.array(sol)
>>> sol.sum(0)
array([[ 9880, 10260],
[10260, 10660]])
>>> A.T.dot(A)
array([[ 9880, 10260],
[10260, 10660]])
>>> np.einsum('ij,ik->jk',A,A)
array([[ 9880, 10260],
[10260, 10660]])
If the result must be a 20 element array, I think you need -
np.einsum('ij,ik->i',A,A)
I have a question based an how to "call" a specific cell in an array, while looping over another array.
Assume, there is an array a:
a = [[a1 a2 a3],[b1 b2 b3]]
and an array b:
b = [[c1 c2] , [d1 d2]]
Now, I want to recalculate the values in array b, by using the information from array a. In detail, each value of array b has to be recalculated by multiplication with the integral of the gauss-function between the borders given in array a. but for the sake of simplicity, lets forget about the integral, and assume a simple calculation is necessary in the form of:
c1 = c1 * (a2-a1) ; c2 = c2 * (a3 - a2) and so on,
with indices it might look like:
b[i,j] = b[i,j] * (a[i, j+1] - a[i,j])
Can anybody tell me how to solve this problem?
Thank you very much and best regards,
Marc
You can use zip function within a nested list comprehension :
>>> [[k*(v[1]-v[0]) for k,v in zip(v,zip(s,s[1:]))] for s,v in zip(a,b)]
zip(s,s[1:]) will gave you the desire pairs of elements that you want, for example :
>>> s =[4, 5, 6]
>>> zip(s,s[1:])
[(4, 5), (5, 6)]
Demo :
>>> b =[[7, 8], [6, 0]]
>>> a = [[1,5,3],[4 ,0 ,6]]
>>> [[k*(v[1]-v[0]) for k,v in zip(v,zip(s,s[1:]))] for s,v in zip(a,b)]
[[28, -16], [-24, 0]]
you can also do this really cleanly with numpy:
import numpy as np
a, b = np.array(a), np.array(b)
np.diff(a) * b
First I would split your a table in a table of lower bound and one of upper bound to work with aligned tables and improve readability :
lowerBounds = a[...,:-1]
upperBounds = a[...,1:]
Define the Gauss function you provided :
def f(x, gs_wdth = 1., mean=0.):
return 1./numpy.sqrt(2*numpy.pi)*gs_wdth * numpy.exp(-(x-mean)**2/(2*gs_wdth**2))
Then, use a nditer (see Iterating Over Arrays) to efficientely iterate over the arrays :
it = numpy.nditer([b, lowerBounds, upperBounds],
op_flags=[['readwrite'], ['readonly'], ['readonly']])
for _b, _lb, _ub in it:
multiplier = scipy.integrate.quad(f, _lb, _ub)[0]
_b[...] *= multiplier
print b
This does the job required in your post, and should be computationnaly efficient. Note that b in modified "in-place" : original values are lost but there is no memory overshoot during calculation.
I want to do something like this:
a = # multi-dimensional numpy array
ares = # multi-dim array, same shape as a
a.shape
>>> (45, 72, 37, 24) # the relevant point is that all dimension are different
v = # 1D numpy array, i.e. a vector
v.shape
>>> (37) # note that v has the same length as the 3rd dimension of a
for i in range(37):
ares[:,:,i,:] = a[:,:,i,:]*v[i]
I'm thinking there has to be a more compact way to do this with numpy, but I haven't figured it out. I guess I could replicate v and then calculate a*v, but I am guessing there is something better than that too. So I need to do element wise multiplication "over a given axis", so to speak. Anyone know how I can do this? Thanks. (BTW, I did find a close duplicate question, but because of the nature of the OP's particular problem there, the discussion was very short and got tracked into other issues.)
Here is one more:
b = a * v.reshape(-1, 1)
IMHO, this is more readable than transpose, einsum and maybe even v[:, None], but pick the one that suits your style.
You can automatically broadcast the vector against the outermost axis of an array. So, you can transpose the array to swap the axis you want to the outside, multiply, then transpose it back:
ares = (a.transpose(0,1,3,2) * v).transpose(0,1,3,2)
You can do this with Einstein summation notation using numpy's einsum function:
ares = np.einsum('ijkl,k->ijkl', a, v)
I tend to do something like
b = a * v[None, None, :, None]
where I think I'm officially supposed to write np.newaxis instead of None.
For example:
>>> import numpy as np
>>> a0 = np.random.random((45,72,37,24))
>>> a = a0.copy()
>>> v = np.random.random(37)
>>> for i in range(len(v)):
... a[:,:,i,:] *= v[i]
...
>>> b = a0 * v[None,None,:,None]
>>>
>>> np.allclose(a,b)
True