Related
I have the following two PyTorch tensors A and B.
A = torch.tensor(np.array([40, 42, 38]), dtype = torch.float64)
tensor([40., 42., 38.], dtype=torch.float64)
B = torch.tensor(np.array([[[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5],[1,2,3,4,5]], [[4,5,6,7,8],[4,5,6,7,8],[4,5,6,7,8],[4,5,6,7,8],[4,5,6,7,8]], [[7,8,9,10,11],[7,8,9,10,11],[7,8,9,10,11],[7,8,9,10,11],[7,8,9,10,11]]]), dtype = torch.float64)
tensor([[[ 1., 2., 3., 4., 5.],
[ 1., 2., 3., 4., 5.],
[ 1., 2., 3., 4., 5.],
[ 1., 2., 3., 4., 5.],
[ 1., 2., 3., 4., 5.]],
[[ 4., 5., 6., 7., 8.],
[ 4., 5., 6., 7., 8.],
[ 4., 5., 6., 7., 8.],
[ 4., 5., 6., 7., 8.],
[ 4., 5., 6., 7., 8.]],
[[ 7., 8., 9., 10., 11.],
[ 7., 8., 9., 10., 11.],
[ 7., 8., 9., 10., 11.],
[ 7., 8., 9., 10., 11.],
[ 7., 8., 9., 10., 11.]]], dtype=torch.float64)
Tensor A is of shape:
torch.Size([3])
Tensor B is of shape:
torch.Size([3, 5, 5])
How do I multiply tensor A with tensor B (using broadcasting) in such a way for eg. the first value in tensor A (ie. 40.) is multiplied with all the values in the first 'nested' tensor in tensor B, ie.
tensor([[[ 1., 2., 3., 4., 5.],
[ 1., 2., 3., 4., 5.],
[ 1., 2., 3., 4., 5.],
[ 1., 2., 3., 4., 5.],
[ 1., 2., 3., 4., 5.]],
and so on for the other 2 values in tensor A and the other two nested tensors in tensor B, respectively.
I could do this multiplication (via broadcasting) with numpy arrays if A and B are arrays of both shape (3,) - ie. A*B - but I can't seem to figure out a counterpart of this with PyTorch tensors. Any help would really be appreciated.
When applying broadcasting in pytorch (as well as in numpy) you need to start at the last dimension (check out https://pytorch.org/docs/stable/notes/broadcasting.html). If they do not match you need to reshape your tensor. In your case they can't directly be broadcasted:
[3] # the two values in the last dimensions are not one and do not match
[3, 5, 5]
Instead you can redefine A = A[:, None, None] before muliplying such that you get shapes
[3, 1, 1]
[3, 5, 5]
which satisfies the conditions for broadcasting.
I have tensorA of size 10x4x9x2, the other tensorB is of size 10x5x2 that contains values from tensorA. Now, how can i find the index of each element in tensorB in tensorA.
Example:
First 2 elements of TensorA:
[[[[ 4., 1.],
[ 1., 2.],
[ 2., 5.],
[ 5., 3.],
[ 3., 11.],
[11., 10.],
[10., -1.],
[-1., -1.],
[-1., -1.]],
[[12., 13.],
[13., 9.],
[ 9., 7.],
[ 7., 5.],
[ 5., 3.],
[ 3., 4.],
[ 4., 1.],
[ 1., 0.],
[ 0., -1.]],
...... so on
Fist 2 elements of TensorB:
[[[ 2., 5.],
[ 5., 7.],
[ 7., 9.],
[ 9., 10.],
[10., 12.]],
[[ 0., 1.],
[ 1., 2.],
[ 2., 5.],
[ 5., -1.],
[-1., -1.]],
Now in tensorB the first element is [2,5] included in the first 5x2 matrix (dimension 0).
so the element should be matched against dimension 0 in tensorA. And the output should be index
0,0,2 since it is the 3rd element.
You can compare the rows that are equal, sum along the last axis, and check that sum against the size of the searched tensor. Then the nonzero function will get you the indices you're looking for.
Since for the example tensors you have given, TensorB[0, 0] is [2., 5.], that looks like:
((TensorA == TensorB[0, 0]).sum(dim=3) == 2).nonzero()
This will return a tensor of [[0, 0, 2]] if that is the only matching row. If you don't want to hard-code 2 (the size of the searched tensor), you can use:
((TensorA == TensorB[0, 0]).sum(dim=3) == TensorB[0, 0].size()[0]).nonzero()
I have a 2d numpy array my_array that starts out like this:
array([[1., 2., 3., 4., 5., 6., 7., 8., 9.],
[1., 2., 3., 4., 5., 6., 7., 8., 9.],
[1., 2., 3., 4., 5., 6., 7., 8., 9.],
[1., 2., 3., 4., 5., 6., 7., 8., 9.],
[1., 2., 3., 4., 5., 6., 7., 8., 9.],
[1., 2., 3., 4., 5., 6., 7., 8., 9.],
[1., 2., 3., 4., 5., 6., 7., 8., 9.],
[1., 2., 3., 4., 5., 6., 7., 8., 9.],
[1., 2., 3., 4., 5., 6., 7., 8., 9.]])
But after some processing which is irrelevant now looks like this:
array([[1., 2., 0., 4., 5., 6., 0., 8., 9.],
[0., 2., 0., 0., 5., 6., 7., 8., 9.],
[0., 2., 0., 4., 5., 0., 7., 0., 9.],
[1., 2., 0., 4., 5., 6., 7., 8., 9.],
[1., 2., 3., 4., 5., 0., 7., 8., 9.],
[0., 2., 0., 4., 5., 6., 0., 8., 9.],
[1., 2., 0., 4., 5., 6., 7., 8., 9.],
[1., 2., 0., 4., 5., 6., 7., 8., 9.],
[1., 2., 0., 4., 5., 6., 7., 8., 0.]])
As you can see, some of the items have been "zeroed out" quite randomly, but only the value of 3 was left with only 1 item that isn't zero. I'm looking for a function that takes this array and returns the index / row number that has the value 3 (or any other value that only appears once and only once in the array).
To explain this differently:
I first have to figure out if there is such an item that only appears once (in this example the answer is yes and that item is the number 3), and then I need to return its row number (in this case 4 since the only line with 3 in it is: my_array[4])
I have successfully done that with iterating over the array, item by item, and counting the number of times each number appears (and returning only the item whose count is 1) and then iterating over everything a second time to find the correct index / row number of where that item is located.
This seems very inefficient, especially if the array will be larger. Is there a better way in numpy to do this?
EDIT: if the number that appears only once is 0 that shouldn't count, i'm only looking for the "column" that was zeroed-out completely except 1 item in it
Try using the numpy.count_nonzero method
numpy.count_nonzero(arr, axis=0)
This will count the non-zero values columnwise
Here's a Demo
I will leave the rest to you. Good Luck
Edit I wasn't even using the mask, you can just use the first and last lines:
x = np.array([[1., 2., 0., 4., 5., 6., 0., 8., 9.],
[0., 2., 0., 0., 5., 6., 7., 8., 9.],
[0., 2., 0., 4., 5., 0., 7., 0., 9.],
[1., 2., 0., 4., 5., 6., 7., 8., 9.],
[1., 2., 3., 4., 5., 0., 7., 8., 9.],
[0., 2., 0., 4., 5., 6., 0., 8., 9.],
[1., 2., 0., 4., 5., 6., 7., 8., 9.],
[1., 2., 0., 4., 5., 6., 7., 8., 9.],
[1., 2., 0., 4., 5., 6., 7., 8., 0.]])
res = (x == 3)
print(np.where(res * x)[0])
Output:
[4]
The full response to np.where() is:
(array([4], dtype=int64), array([2], dtype=int64))
So if you wanted both the column and the row number, you could use both of these.
Suppose we create an numpy array like this:
x = np.linspace(1,5,5).reshape(-1,1)
which results in this:
array([[ 1.],
[ 2.],
[ 3.],
[ 4.],
[ 5.]])
now we add the transpose of this array to it:
x + x.T
which results in this:
array([[ 2., 3., 4., 5., 6.],
[ 3., 4., 5., 6., 7.],
[ 4., 5., 6., 7., 8.],
[ 5., 6., 7., 8., 9.],
[ 6., 7., 8., 9., 10.]])
I don't understand this because the two arrays have different dimensions (5x1 and 1x5) and I learned in linear algebra that we can only sum up matrices when they have the same dimension.
Edit: OK thanks, got it
Here, we have
x = array([[ 1.],[ 2.],[ 3.],[ 4.],[ 5.]])
x.T = array([[ 1., 2., 3., 4., 5.]])
Now you are trying to add two matrices of different dimensions (1 X 5) and (5 X 1).
The way numpy handles this is by copying elements in each row of 1st matrix across its columns to match a number of columns of 2nd matrix and copying elements in each column of 2nd matrix across its rows to match no. of rows of 1st matrix. This gives you 2 5 X 5 matrix which can be added together.
The element wise addition is done as
array([[ 1., 1., 1., 1., 1.],[ 2., 2., 2., 2., 2.,],[ 3., 3., 3., 3., 3.,],[4., 4., 4., 4., 4.],[ 5., 5., 5., 5., 5.,]]) + array([[ 1., 2., 3., 4., 5.],[ 1., 2., 3., 4., 5.],[ 1., 2., 3., 4., 5.],[ 1., 2., 3., 4., 5.],[ 1., 2., 3., 4., 5.]])
which produces the result
array([[ 2., 3., 4., 5., 6.],
[ 3., 4., 5., 6., 7.],
[ 4., 5., 6., 7., 8.],
[ 5., 6., 7., 8., 9.],
[ 6., 7., 8., 9., 10.]])
I would like to get several values whose i have the coordinates.
My coordinates are given by "Coord" (shape : (3, 3, 2, 3) : X and Y during 3 times and with 2 because of 2 coordinates) and my values are given by "Values" (shape : (3, 3, 3) for 3 times)
In other words, i would like to concatenate values in time with "slices" for each positions...
I dont know how to undertake that...Here there is a little part of the arrays.
import numpy as np
Coord = np.array([[[[ 4., 6., 10.],
[ 1., 3., 7.]],
[[ 3., 5., 9.],
[ 1., 3., 7.]],
[[ 2., 4., 8.],
[ 1., 3., 7.]]],
[[[ 4., 6., 10.],
[ 2., 4., 8.]],
[[ 3., 5., 9.],
[ 2., 4., 8.]],
[[ 2., 4., 8.],
[ 2., 4., 8.]]],
[[[ 4., 6., 10.],
[ 3., 5., 9.]],
[[ 3., 5., 9.],
[ 3., 5., 9.]],
[[ 2., 4., 8.],
[ 3., 5., 9.]]]])
Values = np.array([[[-4.24045246, 0.97551048, -5.78904502],
[-3.24218504, 0.9771782 , -4.79103141],
[-2.24390519, 0.97882129, -3.79298771]],
[[-4.24087775, 1.97719843, -5.79065966],
[-3.24261128, 1.97886271, -4.7926441 ],
[-2.24433235, 1.98050192, -3.79459845]],
[[-4.24129055, 2.97886284, -5.79224713],
[-3.24302502, 2.98052345, -4.79422942],
[-2.24474697, 2.98215901, -3.79618161]]])
EDIT LATER
I try in case of a simplified problem (without time first). I have used a "for loop" but
somes errors seems subsist...do you think it s the best way to treat this problem? because my arrays are important... 400x300x100
Coord3 = np.array([[[ 2, 2.],
[ 0., 1.],
[ 0., 2.]],
[[ 1., 0.],
[ 2., 1.],
[ 1., 2.]],
[[ 2., 0.],
[ 1., 1.],
[ 0., 0.]]])
Coord3 = Coord3.astype(int)
Values2 = np.array([[0., 1., 2.],
[3., 4., 5.],
[6., 7., 8.]])
b = np.zeros((3,3))
for i in range(Values2.shape[0]):
for j in range(Values2.shape[1]):
b[Coord3[i,j,0], Coord3[i,j,1]] = Values2[i,j]
b
Your second example is relatively easy to do with fancy indexing:
b = np.zeros((3,3), values2.dtype)
b[coord3[..., 0], coord3[..., 1]] = values2
The origial problem is a bit harder to do, but I think this takes care of it:
coord = coord.astype(int)
x_size = coord[..., 0, :].max() + 1
y_size = coord[..., 1, :].max() + 1
# x_size, y_size = coord.max(axis=(0, 1, 3)) + 1
nt = coord.shape[3]
b = np.zeros((x_size, y_size, nt), values.dtype)
b[coord[..., 0, :], coord[..., 1, :], np.arange(nt)] = values