Let's say I have a 3-D array:
[[[0,1,2],
[0,1,2],
[0,1,2]],
[[3,4,5],
[3,4,5],
[3,4,5]]]
And I want to rearrange this by the columns:
[[0,1,2,3,4,5],
[0,1,2,3,4,5],
[0,1,2,3,4,5]]
What would be an elegant python numpy code for doing this for essentially a 3-D np.array of arbitrary shape and depth?
Could there be a fast method that bypasses for loop? All the approaches I made were terribly adhoc and brute they were basically too slow and useless...
Thanks!!
Swap axes and reshape -
a.swapaxes(0,1).reshape(a.shape[1],-1)
Sample run -
In [115]: a
Out[115]:
array([[[0, 1, 2],
[0, 1, 2],
[0, 1, 2]],
[[3, 4, 5],
[3, 4, 5],
[3, 4, 5]]])
In [116]: a.swapaxes(0,1).reshape(a.shape[1],-1)
Out[116]:
array([[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5]])
Using einops:
einops.rearrange(a, 'x y z -> y (x z) ')
And I would recommend to give meaningful names to axes (instead of x y z) depending on the context (e.g. time, height, etc.). This will make it easy to understand what the code does
In : einops.rearrange(a, 'x y z -> y (x z) ')
Out:
array([[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5]])
Related
It's easy to understand the concept of Transpose in 2-D array. I reall can not understand How the transpose of high-dimensional arrays works.
For example
c = np.indices([4,5]).T.reshape(20,1,2)
d = np.indices([4,5]).reshape(20,1,2)
np.all(c==d) # output is False
Why are the outputs of C and D inconsistent?
In [143]: c = np.indices([4,5])
In [144]: c
Out[144]:
array([[[0, 0, 0, 0, 0],
[1, 1, 1, 1, 1],
[2, 2, 2, 2, 2],
[3, 3, 3, 3, 3]],
[[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4]]])
In [145]: c.shape
Out[145]: (2, 4, 5)
In [146]: c.T.shape
Out[146]: (5, 4, 2)
Look at one 2d array from the size 2 dimension:
In [150]: c[0,:,:]
Out[150]:
array([[0, 0, 0, 0, 0],
[1, 1, 1, 1, 1],
[2, 2, 2, 2, 2],
[3, 3, 3, 3, 3]])
In [151]: c.T[:,:,0]
Out[151]:
array([[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3]])
The 2nd is the usual 2d transpose, a (5,4) array.
MATLAB doesn't do transpose on 3d arrays, at least it doesn't call it such. It may have a way making such a change. numpy, using a general shape/strides multidimensional implementation, easily generalizes the 2d transpose - to 1d or 3d or more.
I have a multidimension array that looks like this:
my_array = np.arange(2)[:,None,None] *np.arange(4)[:, None]*np.arange(8)
I am looking for a multidimensional equivalent of the 2-D argmax
In particular, I am looking for argmax of maxima along axis = 2. I tried reshaping first, but reshaping will completely destroy the original indices information of the entire array, so it probably won't work. I have no clue how to do it and need helps from you guys. Thank you in advance
EDIT: Desire output is:
[(0,0,0),(1,3,1),(1,3,2),(1,3,3),(1,3,4),(1,3,5),(1,3,6),(1,3,7)]
This exactly is the array of the indices of maxima along axis = 2
For finding such argmax indices along the last axis of a 3D ndarray, we can use something along these lines -
In [66]: idx = my_array.reshape(-1,my_array.shape[-1]).argmax(0)
In [67]: r,c = np.unravel_index(idx,my_array.shape[:-1])
In [68]: l = np.arange(len(idx))
In [69]: np.c_[r,c,l]
Out[69]:
array([[0, 0, 0],
[1, 3, 1],
[1, 3, 2],
[1, 3, 3],
[1, 3, 4],
[1, 3, 5],
[1, 3, 6],
[1, 3, 7]])
To extend this to a generic ndarray -
In [99]: R = np.unravel_index(idx,my_array.shape[:-1])
In [104]: np.hstack((np.c_[R],l[:,None]))
Out[104]:
array([[0, 0, 0],
[1, 3, 1],
[1, 3, 2],
[1, 3, 3],
[1, 3, 4],
[1, 3, 5],
[1, 3, 6],
[1, 3, 7]])
This question already has answers here:
Quick way to upsample numpy array by nearest neighbor tiling [duplicate]
(3 answers)
Closed 3 years ago.
Let's say I have a numpy array:
x = np.array([[1, 2],
[3, 4]]
What is the easiest way to expand the elements into submatrices?
An intermediary result could look like this:
x = np.array([[[[1, 1],[1, 1]], [[2, 2],[2, 2]]],
[[[3, 3],[3, 3]], [[4, 4],[4, 4]]]]
And the desired result:
x = np.array([[1, 1, 2, 2],
[1, 1, 2, 2],
[3, 3, 4, 4],
[3, 3, 4, 4]]
You can use two repeats over the desired axes:
In [34]: np.repeat(np.repeat(x, 2, 1), 2, 0)
Out[34]:
array([[1, 1, 2, 2],
[1, 1, 2, 2],
[3, 3, 4, 4],
[3, 3, 4, 4]])
Or as a faster approach (more suitable for larger arrays and repeat numbers) you can use as_strided:
In [43]: from numpy.lib.stride_tricks import as_strided
In [44]: x, y = arr.shape
In [45]: xs, ys = arr.strides
In [46]: result = as_strided(arr, (x, 2, y, 2), (xs, 0, ys, 0))
In [47]: result.reshape(x*2, y*2)
Out[47]:
array([[1, 1, 2, 2],
[1, 1, 2, 2],
[3, 3, 4, 4],
[3, 3, 4, 4]])
You can use numpy.repeat for the task. It has an axis argument.
>>> a = np.array([[1, 2], [3, 4]])
>>> a
array([[1, 2],
[3, 4]])
>>> np.repeat(a, 2)
array([1, 1, 2, 2, 3, 3, 4, 4])
>>> np.repeat(a, 2, axis=1)
array([[1, 1, 2, 2],
[3, 3, 4, 4]])
>>> np.repeat(np.repeat(a, 2, axis=1), 2, axis=0)
array([[1, 1, 1, 1],
[2, 2, 2, 2],
[3, 3, 3, 3],
[4, 4, 4, 4]])
I am trying to figure out a fully vectorised way to compute the co-variance matrix for a 2D numpy array for a given base kernel function. For example if the input is X = [[a,b],[c,d]] for a kernel function k(x_1,x_2) the covariance matrix will be
K=[[k(a,a),k(a,b),k(a,c),k(a,d)],
[k(b,a),k(b,b),k(b,c),k(b,d)],
[k(c,a),k(c,b),k(c,c),k(c,d)],
[k(d,a),k(d,b),k(d,c),k(d,d)]].
how do I go about doing this? I am confused as to how to repeat the values and then apply the function and what might be the most efficient way of doing this.
You can use np.meshgrid to get two matrices with values for the first and second parameter to the k function.
In [8]: X = np.arange(4).reshape(2,2)
In [9]: np.meshgrid(X, X)
Out[9]:
[array([[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3]]),
array([[0, 0, 0, 0],
[1, 1, 1, 1],
[2, 2, 2, 2],
[3, 3, 3, 3]])]
You can then just pass these matrices to the k function:
In [10]: k = lambda x1, x2: (x1-x2)**2
In [11]: X1, X2 = np.meshgrid(X, X)
In [12]: k(X1, X2)
Out[12]:
array([[0, 1, 4, 9],
[1, 0, 1, 4],
[4, 1, 0, 1],
[9, 4, 1, 0]])
Here's another way
k(X.reshape(-1, 1), X.reshape(1, -1))
I have a numpy array of shape (1429,1) where each row itself is a numpy array of shape (3,100) where l may vary from row to row.
How can I reshape this array by flattening each row such that the resulting numpy array will have the shape (1429, 300)?
I guess your initial array's shape is (1429, 3, 100), if that's true, you can change it's shape as below:
import numpy as np
a = a.flatten().reshape((1429, 300)) #a is the initial numpy array
The type of your embedding structure is probably object. It's just a collection of references on 1429 numpy.ndarrays.
As an exemple :
a=np.empty((1429,1),object)
for x in a :
x[0]=np.random.rand(3,100)
In [19]: a.shape,a.dtype
Out[19]: ((1429, 1), dtype('O'))
In [20]: a[0,0].shape
Out[20]: (3, 100)
The structure is probably not contiguous. To obtain a block containing all your data, you must reconstruct it to obtain the good layout :
b=np.array([x.ravel() for x in a.ravel()])
In [21]: b.shape
Out[21]: (1429, 300)
ravel discard unwanted dimensions.
Assuming it is an object dtype array with shape (1429,1), and all elements are 2d of shape (3,100), a good way to 'flatten' is to use concatenate or stack.
np.stack(arr.ravel()).reshape(-1,300)
I use arr.ravel() so the array looks like a (1429) element list to stack. stack then concatenates the elements, creating a (1429, 3, 100) array. The reshape then converts that to (1429, 300).
In [939]: arr = np.empty((5,1),object)
In [940]: arr[:,0] = [np.arange(6).reshape(2,3) for _ in range(5)]
In [941]: arr
Out[941]:
array([[array([[0, 1, 2],
[3, 4, 5]])],
[array([[0, 1, 2],
[3, 4, 5]])],
[array([[0, 1, 2],
[3, 4, 5]])],
[array([[0, 1, 2],
[3, 4, 5]])],
[array([[0, 1, 2],
[3, 4, 5]])]], dtype=object)
In [942]: np.stack(arr.ravel())
Out[942]:
array([[[0, 1, 2],
[3, 4, 5]],
[[0, 1, 2],
[3, 4, 5]],
[[0, 1, 2],
[3, 4, 5]],
[[0, 1, 2],
[3, 4, 5]],
[[0, 1, 2],
[3, 4, 5]]])
In [943]: np.stack(arr.ravel()).reshape(-1,6)
Out[943]:
array([[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5]])
np.stack with the default axis=0 is the same as np.array(...).
Or with concatenate
In [950]: np.concatenate(arr.ravel(),axis=0)
Out[950]:
array([[0, 1, 2],
[3, 4, 5],
[0, 1, 2],
[3, 4, 5],
[0, 1, 2],
[3, 4, 5],
[0, 1, 2],
[3, 4, 5],
[0, 1, 2],
[3, 4, 5]])
In [951]: np.concatenate(arr.ravel(),axis=0).reshape(5,6)
Out[951]:
array([[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5]])