Assume I have a matrix like
a = np.array([[[ 1, 2], [ 3, 4]],
[[ 5, 6], [ 7, 8]],
[[ 9, 10], [11, 12]],
[[13, 14], [15, 16]]])
The shape is (4, 2, 2). I want to sum the first two and the 2nd two matrices to each other. Final output size should have shape (2, 2, 2) and the output should be
output = np.array([[[ 6, 8], [10, 12]],
[[22, 24], [26, 28]]])
You can see my attempt below:
import numpy as np
a = np.array([[[ 1, 2], [ 3, 4]],
[[ 5, 6], [ 7, 8]],
[[ 9, 10], [11, 12]],
[[13, 14], [15, 16]]])
output = np.add(a[:2], a[2:])
Break the first dimension up into two using a reshape, sum along the second axis:
a.reshape(2, 2, *a.shape[1:]).sum(axis=1)
Your current approach is equivalent to a.reshape(2, 2, *a.shape[1:]).sum(axis=0). The correct way would be to slice every other row of the entire array, rather than every other block of the entire array:
a[::2] + a[1::2]
The latter approach does not generalize well. If you had to add up say every block of seven, you would get
a[::7] + a[1::7] + a[2::7] + a[3::7] + ... + a[6::7]
The former approach is quite flexible, however:
a.reshape(-1, 7, *a.shape[1:]).sum(axis=1)
Related
I'm trying to combine n matrices in a 3-dimensional PyTorch tensor of shape (n, i, j) into a single 2-dimensional matrix of shape (i, j*n). Here's a simple example where n=2, i=2, j=2:
m = torch.tensor([[[2, 3],
[5, 7]],
[[11, 13],
[17, 19]]])
m.reshape(2, 4)
I was hoping this would produce:
tensor([[ 2, 3, 11, 13],
[ 5, 7, 17, 19]])
But instead it produced:
tensor([[ 2, 3, 5, 7],
[11, 13, 17, 19]])
How do I do this? I tried torch.cat and torch.stack, but they require tuples of tensors. I could try and create tuples, but that seems inefficient. Is there a better way?
To combine n + j with reshape you need them consequent in shape. One can fix it with swapaxes:
m = torch.tensor([[[2, 3],
[5, 7]],
[[11, 13],
[17, 19]]])
m=m.swapaxes( 0,1 )
m.reshape(2, 4)
tensor([[ 2, 3, 11, 13],
[ 5, 7, 17, 19]])
Let's say I have an array that contains 2 times 3x3-array:
a = np.arange(2 * 3 * 3).reshape(2, 3, 3)
print(a)
Output:
array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17]]])
Now I would like to have the upper triangle of each of the 2 arrays. I know I can achieve it through the following two:
np.array([aa[np.triu_indices(3)] for aa in a])
# or
a.T[np.tril_indices(3)].T
Output:
array([[ 0, 1, 2, 4, 5, 8],
[ 9, 10, 11, 13, 14, 17]])
However, I know that list comprehension is slow so I'd rather not use it. And the transpose + tril makes it difficult to understand what it does at first sight. I had hoped that one of the following options would work, but none of them did:
a[:, np.triu_indices(3)] # totally different output
a[np.arange(len(a)), np.triu_indices(3)] # error
a[np.indices(a.shape)[0], np.triu_indices(3)] # error
Is there an elegant and fast way to do it?
Let's say I have an array X of shape (6, 2) like this:
import numpy as np
X = np.array([[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12]])
I want to reshape it to an array of shape (3, 2, 2), so I did this:
X.reshape(3, 2, 2)
And got:
array([[[ 1, 2],
[ 3, 4]],
[[ 5, 6],
[ 7, 8]],
[[ 9, 10],
[11, 12]]])
However, I need my data in a different format. To be precise, I want to end up wth:
array([[[ 1, 2],
[ 7, 8]],
[[ 3, 4],
[ 9, 10]],
[[ 5, 6],
[11, 12]]])
Should I be using reshape for this or something else? What's the best way to do this in Numpy?
You have to set the order option:
>>> X.reshape(3, 2, 2, order='F')
array([[[ 1, 2],
[ 7, 8]],
[[ 3, 4],
[ 9, 10]],
[[ 5, 6],
[11, 12]]])
âFâ means to read / write the elements using Fortran-like index order, with the first index changing fastest, and the last index changing slowest.
see: https://numpy.org/doc/stable/reference/generated/numpy.reshape.html
You need to specify order;
X.reshape(3, 2, 2, order='F')
should work
A functional equivalent to the order='F' reshape:
In [31]: x.reshape(2,3,2).transpose(1,0,2)
Out[31]:
array([[[ 1, 2],
[ 7, 8]],
[[ 3, 4],
[ 9, 10]],
[[ 5, 6],
[11, 12]]])
In [32]: x.reshape(2,3,2).transpose(1,0,2).strides
Out[32]: (16, 48, 8)
Without the transpose the strides would be (48,16,8).
A thing that's a bit tricky about this layout is that the last dimension remains in 'C' order. It's the just first two dimension that are switched.
The full 'F' layout would be
In [33]: x = np.arange(1,13).reshape(3,2,2,order='F')
In [34]: x
Out[34]:
array([[[ 1, 7],
[ 4, 10]],
[[ 2, 8],
[ 5, 11]],
[[ 3, 9],
[ 6, 12]]])
I have been stuck with a question about slicing numpy array for a while.
Below is an array I have right now:
a = np.array([[[ 1, 2],
[ 3, 4],
[ 5, 6]],
[[ 7, 8],
[ 9, 10],
[11, 12]]]
How can I use slicing to get an array like the following?
np.array([[[ 1, 2]],
[[ 9, 10],
[11, 12]]]
I have tried a[[0,1],[0,[1,2]] however it didn't work and gave an error:
ValueError: setting an array element with a sequence.
Thank you in advance!
The exact thing you give as your desired output is not possible, since arrays have to be "hyper-rectangles", so X[0].shape has to be the same as X[1].shape.
What you can do is:
a[[0,1,1],[0,1,2]]
# array([[ 1, 2],
# [ 9, 10],
# [11, 12]])
You can do this, for example:
import numpy as np
a = np.array([[[ 1, 2], [ 3, 4], [ 5, 6]], [[ 7, 8], [ 9, 10], [11, 12]]])
print(np.array([[a[0, 0 ,: ], a[1, 1 ,:], a[1, 2 ,: ]]]))
Result:
[[[ 1 2]
[ 9 10]
[11 12]]]
You can apply two operations separably and merge them afterwards:
np.array((a[0,0:1].tolist(), a[1,1:].tolist()))
# array([[[1, 2]], [[9, 10], [11, 12]]], dtype=object)
I have a 2-d numpy array of shape NxM which represents M contiguous samples from N different sequences. I need to present patches of L samples (L << M) covering the entire dataset as a 2-d numpy array. There is too much data to construct a new dataset by simply copying of all the patches.
If there was a single sequence, it would be very straight forward to generate the overlapping patches without copying any data using the as_strided trick:
patches = np.lib.stride_tricks.as_strided(data, shape(N*M-L+1,L), strides=(8,8))
The problem with this approach for my data is that it produces patches that overlap separate sequences.
I can also see how to generate a 3-d array of shape N,M-L+1,L using something like:
patches = np.lib.stride_ticks.as_strided(data, shape(N,M-L+1,L), strides=(8*M,8,8))
This produces the correct patches, but I am not sure how to collapse the first two dimensions into one.
There are obviously several SO answers related to as_strided, but I could not find any that address these particular requirements.
Any ideas are appreciated.
Edit: Short example follows
Here is an example of using as_strided to make a 3-d array that almost accomplishes the task:
>>> a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
>>> a
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
>>> b = np.lib.stride_tricks.as_strided(a, shape=(3, 3, 2), strides=(32,8,8))
>>> b
array([[[ 1, 2],
[ 2, 3],
[ 3, 4]],
[[ 5, 6],
[ 6, 7],
[ 7, 8]],
[[ 9, 10],
[10, 11],
[11, 12]]])
>>>
The issue with trying to flatten this 3-d array into 2-d as suggested by #Divakar is that the reshaping produces the correct data but does so by making a copy which creates an unmanageable amount of data for the actual problem at hand:
>>> c = b.reshape(-1,b.shape[-1])
>>> c
array([[ 1, 2],
[ 2, 3],
[ 3, 4],
[ 5, 6],
[ 6, 7],
[ 7, 8],
[ 9, 10],
[10, 11],
[11, 12]])
>>> b[0][0][0] = 9999
>>> c
array([[ 1, 2],
[ 2, 3],
[ 3, 4],
[ 5, 6],
[ 6, 7],
[ 7, 8],
[ 9, 10],
[10, 11],
[11, 12]])
>>>