We Keep Coding

How to multiply a tensor with a vector?

I have 2 tensors
a = torch.tensor([1,2])
b = torch.tensor([[[10,20],
[30,40]],
[[1,2],
[3,4]]])
and I would like to combine them in such a way that
a ? b = tensor([[[10,20],
[30,40]],
[[ 2, 4],
[ 6, 8]]])
(and then sum over the 0th dimension, in the end I want to do a weighted sum)
I've tried:
""" no idea how to interpret that """
a # b
tensor([[ 70, 100],
[ 7, 10]])
b # a
tensor([[ 50, 110],
[ 5, 11]])
for i in range(b.size()[0]): # works but I don't think this will work with autograd
b[i] *= a[i]
a * b # multiplies right side by 2
tensor([[[10, 40],
[30, 80]],
[[ 1, 4],
[ 3, 8]]])
a.unsqueeze(1) # multiplies bottom side by 2
tensor([[[10, 20],
[60, 80]],
[[ 1, 2],
[ 6, 8]]])
a.unsqueeze(2) * b # dimension out of range

This should workc = a.unsqueeze(1).unsqueeze(1) * b

You can also try below code:
a = torch.tensor([1,2])
b = torch.tensor([[[10,20],
[30,40]],
[[1,2],
[3,4]]])
print((a.view(-1, 1)*torch.flatten(b, 1)).view(b.shape))
output:
tensor([[[10, 20],
[30, 40]],
[[ 2, 4],
[ 6, 8]]])
Here, we are basically carrying out below steps:
Reshaping a to a 2d tensor of size [a.shape[0],1], i.e. [2, 1] in above case.
Then, we are using torch.flatten() to flatten tensor b starting from the first dimension(i.e. start_dim=1). Here, end_dim=-1 by default. Resultant size is [2, 4].
Performing element-wise multiplication.
Finally, reshaping the result to shape same as original tensor b, i.e [2, 2, 2].

Interesting -- I tried a few different broadcasting tricks and didn't see any obvious wins, so the simple version:
b[0] *= a[0]
b[1] *= a[1]
c = b

How to create a symmetric matrix from a numpy 1D array the most efficient way

I have a 1d np.array. Its length may vary according to user input, but it will always stay single-dimesional.
Please advise if there is an efficient way to create a symmetric 2d np.array from it? By 'symmetric' I mean that its elements will be according to the rule k[i, j] = k[j, i].
I realise it is possible to do with a python for loop and lists, but that is very inefficient.
Many thanks in advance!
EXAMPLE:
For example, we have x = np.array([1, 2, 3]). The desired result should be
M = np.array([[1, 2, 3],
[2, 1, 2],
[3, 2, 1])

Interpretation #1
Seems like you are reusing elements at each row. So, with that sort of idea, an implementation using broadcasting would be -
def symmetricize(arr1D):
ID = np.arange(arr1D.size)
return arr1D[np.abs(ID - ID[:,None])]
Sample run -
In [170]: arr1D
Out[170]: array([59, 21, 70, 10, 42])
In [171]: symmetricize(arr1D)
Out[171]:
array([[59, 21, 70, 10, 42],
[21, 59, 21, 70, 10],
[70, 21, 59, 21, 70],
[10, 70, 21, 59, 21],
[42, 10, 70, 21, 59]])
Interpretation #2
Another interpretation I had when you would like to assign the elements from the input 1D array into a symmetric 2D array without re-use, such that we would fill in the upper triangular part once and then replicate those on the lower triangular region by keeping symmetry between the row and column indices. As such, it would only work for a specific size of it. So, as a pre-processing step, we need to perform that error-checking. After we are through the error-checking, we will initialize an output array and use row and column indices of a triangular array to assign values once as they are and once with swapped indices to assign values in the other triangular part, thus giving it the symmetry effect.
It seemed like Scipy's squareform should do be able to do this task, but from the docs, it doesn't look like it supports filling up the diagonal elements with the input array elements. So, let's give our solution a closely-related name.
Thus, we would have an implementation like so -
def squareform_diagfill(arr1D):
n = int(np.sqrt(arr1D.size*2))
if (n*(n+1))//2!=arr1D.size:
print "Size of 1D array not suitable for creating a symmetric 2D array!"
return None
else:
R,C = np.triu_indices(n)
out = np.zeros((n,n),dtype=arr1D.dtype)
out[R,C] = arr1D
out[C,R] = arr1D
return out
Sample run -
In [179]: arr1D = np.random.randint(0,9,(12))
In [180]: squareform_diagfill(arr1D)
Size of 1D array not suitable for creating a symmetric 2D array!
In [181]: arr1D = np.random.randint(0,9,(10))
In [182]: arr1D
Out[182]: array([0, 4, 3, 6, 4, 1, 8, 6, 0, 5])
In [183]: squareform_diagfill(arr1D)
Out[183]:
array([[0, 4, 3, 6],
[4, 4, 1, 8],
[3, 1, 6, 0],
[6, 8, 0, 5]])

What you're looking for is a special Toeplitz matrix and easy to generate with scipy
from numpy import concatenate, zeros
from scipy.linalg import toeplitz
toeplitz([1,2,3])
array([[1, 2, 3],
[2, 1, 2],
[3, 2, 1]])
another special matrix interpretation can be using Hankel matrix, which will give you minimum dimension square matrix for a given array.
from scipy.linalg import hankel
a=[1,2,3]
t=int(len(a)/2)+1
s=t-2+len(a)%2
hankel(a[:t],a[s:])
array([[1, 2],
[2, 3]])

how to vectorize this python code to make it more efficient? (in speed)

I am trying to vectorize my code using numpy modules, the original code is like:
m = [M[i,:9].dot(N[:9,i]) for i in xrange(9)]
and I have improved the code as:
m = np.diagonal(M[:9,:9].dot(N[:9,:9]))
yet this will lead to some unnecessary calculations (especially when the index is much greater than 9). What can I do to improve the efficiency further?
Edit: Basically what I intend to do is to calculate the diagonal elements of the dot product of two matrixes M and N.

You can use np.einsum as we need to keep the first axis of M aligned with the second axis of N, while reducing/losing the leftover axes from the inputs. Thus, we would have an einsum based solution, like so:
m = np.einsum('ij,ji->i',M[:,:9],N[:9,:])

Assuming that both matrices are square and have the same dimensions, this is what we can do:
By the definition of matrix multiplication, the result you want to calculate is basically a vector of dot products. The first element of the result vector is the dot product of M's first row with N's first column, followed by the dot product of M's second row with N's second column, etc.
We can express this calculation by transposing N, then multiplying M and NT element-wise, then adding up each row. This gives us a resulting column vector. This is the code:
Nprime = np.transpose(N[:,:9])
product = np.multiply(M[:9,:], Nprime)
result = np.sum(product, axis=1)
By Divakar's suggestion, it can be condensed into one line:
result = (M[:,:9] * (N[:9,:].T)).sum(1)
Example (with size 3 instead of 9):
import numpy as np
>>> M = np.array(
[[1, 2, 3],
[6, 5, 4],
[8, 7, 9]])
>>> N = np.array(
[[0, 9, 7],
[2, 4, 5],
[6, 8, 1]])
>>> M.dot(N)
array([[22, 41, 20],
[34, 106, 71],
[68, 172, 100]])
>>> np.diagonal(M.dot(N))
array([22, 106, 100])
# ^ The reference answer
>>> Nprime = np.transpose(N)
array([[0, 2, 6],
[9, 4, 8],
[7, 5, 1]])
>>> product = np.multiply(M, Nprime)
array([[ 0, 4, 18],
[54, 20, 32],
[56, 35, 9]])
>>> result = np.sum(product, axis=1)
array([22, 106, 100])

how to broadcast certain processing in numpy arrays

Let's say I have an numpy array
import numpy as np
a = np.array([[1,2,3],[3,4,5]])
and I defined a function to process the data, e.g, get the product of a vector:
def vp(v):
p = 1
for i in v:
p = p * i
return p
and how can I easily broadcast the function to all the vectors in a by something like the map function for the list, e.g. vp(a) will give me [6, 60]?
what if a is a 3D or even 4D array, is there a good way to broadcast such customized function?

I think the core of your question relies on creating custom functions to apply over multidimensional datasets. For that you would use numpy.apply_along_axis().
a = array([[1, 2, 3],
[3, 4, 5]])
np.apply_along_axis(arr = a, func1d=vp, axis=1)
> array([ 6, 60])
Yes this also works with N-Dimentional datasets.
c = array([[[ 1, 2],
[ 3, 3],
[ 4, 5]],
[[16, 17],
[18, 18],
[19, 20]]])
np.apply_along_axis(vp, axis=1, arr=c)
> array([[ 12, 30],
[5472, 6120]])

How to apply function of multiple matrices to each element of a list of matrices with python numpy?

For sake of simplicity lets define the function to do only matrix multiplication:
f(matrix1, matrix2):
#assume that matrix1.shape == np.transpose(matrix2).shape
#both are 1 dimensional so this returns a scalar
return matrix1 * matrix2
Now lets say I want to run this function a bunch of times for getting a sum:
- matrix1 - different each time
- matrix2 - same each time
I could write a for loop:
matrix_a1 = np.matrix([1,2])
matrix_a2 = np.matrix([3,4])
matrix_list = [matrix_a1, matix_a2]
matrixb = np.matrix([5,6],[7,8])
total = 0
for matrix in matrix_list
total+= f(matrix, matrixb)
I want to write it like this:
sum(f(matrix_list, matrixb))
But this doesn't work because it tries to do matrix multiplication between matrix_list and matrixb instead of iterating over matrix_list.
How to I iterate over the matrix_list using numpy?

You convert your list of matrices to a multidimensional array.
That will be easier if you step out of the comfort of the matrix object and go with plain arrays. Your first function, when given two arrays, can be rewritten with np.dot as:
def f(array1, array2) :
return np.dot(array1, array2)
You can now do:
>>> array_a1 = np.array([1, 2])
>>> array_a2 = np.array([3, 4])
>>> array_a = np.array([array_a1, array_a2])
>>> array_a
array([[1, 2],
[3, 4]])
>>> array_b = np.array([[5, 6], [7, 8]])
>>> array_b
array([[5, 6],
[7, 8]])
>>> f(array_a, array_b)
array([[19, 22],
[43, 50]])
>>> np.sum(f(array_a, array_b), axis=0)
array([62, 72])
You could even do :
>>> array_a1 = np.array([[1, 2], [3, 4]])
>>> array_a2 = np.array([[5, 6], [7, 8]])
>>> array_a = np.array([array_a1, array_a2])
>>> array_b = np.array([[9, 10], [11, 12]])
>>> np.sum(f(array_a, array_b), axis=0)
array([[142, 156],
[222, 244]])