I have 3D numpy array.
import numpy as np
X = np.arange(12).reshape(2, 2, 3)
print(X)
[[[ 0 1 2]
[ 3 4 5]]
[[ 6 7 8]
[ 9 10 11]]]
I would like to vectorize the following for all 2D array in 3D array. For example, for 1st 2D array:
ss = np.array(np.meshgrid(*X[0]), dtype=object).T.reshape(-1,2)
print(ss)
[[0 3]
[0 4]
[0 5]
[1 3]
[1 4]
[1 5]
[2 3]
[2 4]
[2 5]]
I tried following:
def f(x):
return np.array(np.meshgrid(*x), dtype=object).T.reshape(-1,2)
ff = np.apply_along_axis(f, 0, X)
print(ff)
Here's a generic solution that uses one-loop and scales to generic shapes. It assigns into an initialized array and broadcasts to replicate values, where it achieves memory efficiency. It works for any length along the second axis of X. Hence, the implementation would be -
def meshgrid_2D_blocks(X):
m,n,r = X.shape
out_shp = [m]+[r]*n+[n]
out = np.empty(out_shp,dtype=X.dtype)
# Assign each block iteratively
shp = [-1]+[1]*n
for i in range(n):
shp[i+1] = r
out[...,i] = X[:,i].reshape(shp)
shp[i+1] = 1
return out.reshape(m,-1,n)
Sample runs
Case #1 : Second axis of length=2
In [167]: X = np.arange(12).reshape(2, 2, 3)
In [168]: X
Out[168]:
array([[[ 0, 1, 2],
[ 3, 4, 5]],
[[ 6, 7, 8],
[ 9, 10, 11]]])
In [169]: meshgrid_2D_blocks(X)
Out[169]:
array([[[ 0, 3],
[ 0, 4],
[ 0, 5],
[ 1, 3],
[ 1, 4],
[ 1, 5],
[ 2, 3],
[ 2, 4],
[ 2, 5]],
[[ 6, 9],
[ 6, 10],
[ 6, 11],
[ 7, 9],
[ 7, 10],
[ 7, 11],
[ 8, 9],
[ 8, 10],
[ 8, 11]]])
Case #2 : Second axis of length=3
In [170]: X = np.arange(12).reshape(2, 3, 2)
In [171]: X
Out[171]:
array([[[ 0, 1],
[ 2, 3],
[ 4, 5]],
[[ 6, 7],
[ 8, 9],
[10, 11]]])
In [172]: meshgrid_2D_blocks(X)
Out[172]:
array([[[ 0, 2, 4],
[ 0, 2, 5],
[ 0, 3, 4],
[ 0, 3, 5],
[ 1, 2, 4],
[ 1, 2, 5],
[ 1, 3, 4],
[ 1, 3, 5]],
[[ 6, 8, 10],
[ 6, 8, 11],
[ 6, 9, 10],
[ 6, 9, 11],
[ 7, 8, 10],
[ 7, 8, 11],
[ 7, 9, 10],
[ 7, 9, 11]]])
This is one way to achieve that:
import numpy as np
X = np.arange(12).reshape(2, 2, 3)
out = np.stack(np.broadcast_arrays(X[:, 0, :, None], X[:, 1, None, :]), -1).reshape(len(X), -1, 2)
print(out)
# [[[ 0 3]
# [ 0 4]
# [ 0 5]
# [ 1 3]
# [ 1 4]
# [ 1 5]
# [ 2 3]
# [ 2 4]
# [ 2 5]]
#
# [[ 6 9]
# [ 6 10]
# [ 6 11]
# [ 7 9]
# [ 7 10]
# [ 7 11]
# [ 8 9]
# [ 8 10]
# [ 8 11]]]
Related
I'm currently porting a MATLAB library over to Python. As of right now, I am trying to keep the code as one-to-one as possible. I'm noticing some differences between reshape in Matlab vs Python that is causing some issues.
I've heard people talk about the difference in 'C' and 'Fortran' order. How numpy defaults to 'C' order and Matlab uses 'Fortran'. Below are two Python examples using both orders.
>>> a = np.arange(12).reshape((2,3,2))
>>> a
array([[[ 0, 1],
[ 2, 3],
[ 4, 5]],
[[ 6, 7],
[ 8, 9],
[10, 11]]])
>>> b = np.arange(12).reshape((2,3,2), order='F')
>>> b
array([[[ 0, 6],
[ 2, 8],
[ 4, 10]],
[[ 1, 7],
[ 3, 9],
[ 5, 11]]])
Below is the matlab/octave equivalent to the above python code.
octave:12> a = reshape((0:11), [3,2,2])
a =
ans(:,:,1) =
0 3
1 4
2 5
ans(:,:,2) =
6 9
7 10
8 11
Notice that each example yields a different result.
These examples are meant to illustrate the discrepancy that I'm referring to. The datasets that I'm working on in my project are significantly larger. I need to be able to reshape arrays in Python and be confident that it is performing the same reshape operations as it would in Matlab. Any help would be appreciated.
Why are you using a (2,3,2) shape in one, and (3,2,2) in the other?
In [82]: arr = np.arange(12).reshape((3,2,2), order='F')
In [83]: arr
Out[83]:
array([[[ 0, 6],
[ 3, 9]],
[[ 1, 7],
[ 4, 10]],
[[ 2, 8],
[ 5, 11]]])
In [84]: arr[:,:,0]
Out[84]:
array([[0, 3],
[1, 4],
[2, 5]])
In [85]: arr[:,:,1]
Out[85]:
array([[ 6, 9],
[ 7, 10],
[ 8, 11]])
===
Looking a strides may help identify the differences between c and f orders
In [86]: arr.shape
Out[86]: (3, 2, 2)
In [87]: arr.strides
Out[87]: (8, 24, 48)
Notice how the smallest steps, 1 element (8 bytes) is taken in first dimension.
Contrast that with a C order:
In [89]: np.arange(12).reshape(2,2,3)
Out[89]:
array([[[ 0, 1, 2],
[ 3, 4, 5]],
[[ 6, 7, 8],
[ 9, 10, 11]]])
In [90]: np.arange(12).reshape(2,2,3).strides
Out[90]: (48, 24, 8)
===
OK lets try the (2,3,2) shape:
>> a = reshape((0:11),[2,3,2])
a =
ans(:,:,1) =
0 2 4
1 3 5
ans(:,:,2) =
6 8 10
7 9 11
Samething with order 'F':
In [94]: arr = np.arange(12).reshape((2,3,2), order='F')
In [95]: arr
Out[95]:
array([[[ 0, 6],
[ 2, 8],
[ 4, 10]],
[[ 1, 7],
[ 3, 9],
[ 5, 11]]])
In [96]: arr[:,:,0]
Out[96]:
array([[0, 2, 4],
[1, 3, 5]])
>> squeeze(a(1,:,:))
ans =
0 6
2 8
4 10
In [98]: arr[0,:,:]
Out[98]:
array([[ 0, 6],
[ 2, 8],
[ 4, 10]])
I would like to control how array.reshape() populates the new array. For example
a = np.arange(12).reshape(3,4)
## array([[ 0, 1, 2, 3],
## [ 4, 5, 6, 7],
## [ 8, 9, 10, 11]])
but what I would like to be able to is populate the array columnwise with something like:
a = np.arange(9).reshape(3,3, 'columnwise')
## array([[ 0, 3, 6, 9],
## [ 1, 4, 7, 10],
## [ 2, 5, 8, 11]])
Use np.transpose.
import numpy as np
print(np.arange(9).reshape(3,3).transpose())
Output:
[[0 3 6]
[1 4 7]
[2 5 8]]
In [22]: np.arange(12).reshape(3,4, order='F')
Out[22]:
array([[ 0, 3, 6, 9],
[ 1, 4, 7, 10],
[ 2, 5, 8, 11]])
If you take a transpose of the original matrix, you will get your desired effect.
import numpy as np
a = np.arange(6).reshape(3,3).tranpose()
OR
a = np.arange(6).reshape(3,3).T
I have a data set like this
a = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
How can I reshape this into shape (3,2,2) so that a[:,0,0] = [1,2,3]?
you can use two steps:
step1.
In [28]: b1 = np.reshape(a,(3,4), order='F')
In [29]: b1
Out[29]:
array([[ 1, 4, 7, 10],
[ 2, 5, 8, 11],
[ 3, 6, 9, 12]])
use order='F' means to read / write the elements using Fortran-like index order, with the first index changing fastest, and the last index changing slowest. numpy.reshape
setp2
In [30]: c = b1.reshape(3,2,2)
In [31]: c
Out[31]:
array([[[ 1, 4],
[ 7, 10]],
[[ 2, 5],
[ 8, 11]],
[[ 3, 6],
[ 9, 12]]])
get the final result:
In [34]: c[:,0,0]
Out[34]: array([1, 2, 3])
In [30]: a=np.arange(1,13)
In [31]: a
Out[31]: array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])
Since you want to keep the first 3 values 'together', we could start with a reshape like:
In [32]: a.reshape(2,2,3)
Out[32]:
array([[[ 1, 2, 3],
[ 4, 5, 6]],
[[ 7, 8, 9],
[10, 11, 12]]])
and then swap a couple of the axes:
In [33]: a.reshape(2,2,3).transpose(2,0,1)
Out[33]:
array([[[ 1, 4],
[ 7, 10]],
[[ 2, 5],
[ 8, 11]],
[[ 3, 6],
[ 9, 12]]])
In [34]: _[:,0,0]
Out[34]: array([1, 2, 3])
Or with a different transpose:
In [35]: a.reshape(2,2,3).transpose(2,1,0)
Out[35]:
array([[[ 1, 7],
[ 4, 10]],
[[ 2, 8],
[ 5, 11]],
[[ 3, 9],
[ 6, 12]]])
transpose() with an argument, (also invoked with .T) does the same thing.
So your question is a bit ambiguous.
So does the reshape with order F mentioned in the other answer:
In [37]: a.reshape(3,2,2, order='F')
Out[37]:
array([[[ 1, 7],
[ 4, 10]],
[[ 2, 8],
[ 5, 11]],
[[ 3, 9],
[ 6, 12]]])
(though the two step, a.reshape(3,4, order='F').reshape(3,2,2) produces my first result Out[33]).
Given this array:
>>> a
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11]])
How can I select [[4,5], [7,8]]? a[0::2, 1:;2] doesn't work
>>> a
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11]])
>>> a[1:3,1:3]
array([[4, 5],
[7, 8]])
The first 1:3 is to select row 1 & 2. The second 1:3 is to select column 1 & 2.
>>> arr = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])
>>> arr
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
I am deleting the 3rd column as
>>> np.hstack(((np.delete(arr, np.s_[2:], 1)),(np.delete(arr, np.s_[:3],1))))
array([[ 1, 2, 4],
[ 5, 6, 8],
[ 9, 10, 12]])
Are there any better way ?
Please consider this to be a novice question.
If you ever want to delete more than one columns, you just pass indices of columns you want deleted as a list, like this:
>>> a = np.arange(12).reshape(3,4)
>>> a
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
>>> np.delete(a, [1,3], axis=1)
array([[ 0, 2],
[ 4, 6],
[ 8, 10]])
>>> import numpy as np
>>> arr = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])
>>> np.delete(arr, 2, axis=1)
array([[ 1, 2, 4],
[ 5, 6, 8],
[ 9, 10, 12]])
Something like this:
In [7]: x = range(16)
In [8]: x = np.reshape(x, (4, 4))
In [9]: x
Out[9]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
In [10]: np.delete(x, 1, 1)
Out[10]:
array([[ 0, 2, 3],
[ 4, 6, 7],
[ 8, 10, 11],
[12, 14, 15]])