Related
I have a 3D matrix x_test of size (100, 33, 66) and I want to change its dimensions to (100, 66, 33).
What is the most efficient way to do this using python3.5? I look for something along those lines:
y = x_test.transpose()
You can pass the desired dimensions to the function np.transpose using in your case np.transpose(x_test, (0, 2, 1)).
For example,
import numpy as np
x_test = np.arange(30).reshape(3, 2, 5)
print(x_test)
print(x_test.shape)
This will print
[[[ 0 1 2 3 4]
[ 5 6 7 8 9]]
[[10 11 12 13 14]
[15 16 17 18 19]]
[[20 21 22 23 24]
[25 26 27 28 29]]]
(3, 2, 5)
Now, you can transpose the matrix with the command from above
y = np.transpose(x_test, (0, 2, 1))
print(y)
print(y.shape)
which will give
[[[ 0 5]
[ 1 6]
[ 2 7]
[ 3 8]
[ 4 9]]
[[10 15]
[11 16]
[12 17]
[13 18]
[14 19]]
[[20 25]
[21 26]
[22 27]
[23 28]
[24 29]]]
(3, 5, 2)
Apart from transpose (see #Cleb's answer) there are also swapaxes and moveaxis:
import numpy as np
mock = np.arange(30).reshape(2,3,5)
mock.swapaxes(1,2)
# array([[[ 0, 5, 10],
[ 1, 6, 11],
[ 2, 7, 12],
[ 3, 8, 13],
[ 4, 9, 14]],
[[15, 20, 25],
[16, 21, 26],
[17, 22, 27],
[18, 23, 28],
[19, 24, 29]]])
np.moveaxis(mock,2,1)
# array([[[ 0, 5, 10],
[ 1, 6, 11],
[ 2, 7, 12],
[ 3, 8, 13],
[ 4, 9, 14]],
[[15, 20, 25],
[16, 21, 26],
[17, 22, 27],
[18, 23, 28],
[19, 24, 29]]])
np.rot90 is another option. I confess I do not yet understand the axes = (a, b) notation and sort through all combinations from (0, 1) to (2, 1) to find what I want. Using x_test above, note its original shape (3, 2 ,5):
x2 = np.rot90(x_test, axes = (0, 1))
array([[[ 5, 6, 7, 8, 9],
[15, 16, 17, 18, 19],
[25, 26, 27, 28, 29]],
[[ 0, 1, 2, 3, 4],
[10, 11, 12, 13, 14],
[20, 21, 22, 23, 24]]])
x2.shape
(2, 3, 5)
I created a sample array:
a = np.arange(18).reshape(9,2)
On printing, I get this as output:
[[ 0 1]
[ 2 3]
[ 4 5]
[ 6 7]
[ 8 9]
[10 11]
[12 13]
[14 15]
[16 17]]
On executing this reshaping:
b = a.reshape(2,3,3).swapaxes(0,2)
I get:
[[[ 0 9]
[ 3 12]
[ 6 15]]
[[ 1 10]
[ 4 13]
[ 7 16]]
[[ 2 11]
[ 5 14]
[ 8 17]]]
I went through this question, but it does not solve my problem.
Reshape an array in NumPy
The documentation is not useful either.
https://docs.scipy.org/doc/numpy/reference/generated/numpy.swapaxes.html
I need to know how the swapping is working(which is x-axis, y-axis, z-axis). A diagrammatic explanation would be most helpful.
Here is my understanding of swapaxes
Suppose you have an array
In [1]: arr = np.arange(16).reshape((2, 2, 4))
In [2]: arr
Out[2]:
array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7]],
[[ 8, 9, 10, 11],
[12, 13, 14, 15]]])
And the shape of arr is (2, 2, 4), for the value 7, you can get the value by
In [3]: arr[0, 1, 3]
Out[3]: 7
There are 3 axes 0, 1 and 2, now, we swap axis 0 and 2
In [4]: arr_swap = arr.swapaxes(0, 2)
In [5]: arr_swap
Out[5]:
array([[[ 0, 8],
[ 4, 12]],
[[ 1, 9],
[ 5, 13]],
[[ 2, 10],
[ 6, 14]],
[[ 3, 11],
[ 7, 15]]])
And as you can guess, the index of 7 is (3, 1, 0), with axis 1 unchanged,
In [6]: arr_swap[3, 1, 0]
Out[6]: 7
So, now from the perspective of the index, swapping axis is just change the index of values. For example
In [7]: arr[0, 0, 1]
Out[7]: 1
In [8]: arr_swap[1, 0, 0]
Out[8]: 1
In [9]: arr[0, 1, 2]
Out[9]: 6
In [9]: arr_swap[2, 1, 0]
Out[9]: 6
So, if you feel difficult to get the swapped-axis array, just change the index, say arr_swap[2, 1, 0] = arr[0, 1, 2].
Start with the reshape
In [322]: a = np.arange(18).reshape(2,3,3)
In [323]: a
Out[323]:
array([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],
[[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17]]])
This displays as 2 planes, and each plane is a 3x3. Is that part clear? The fact that the array was shaped (9,2) at one point isn't significant. Reshaping doesn't change the order of elements.
Apply the swapaxes. Shape is now (3,3,2). 3 planes, each is 3x2. This particular swap is the same as a transpose
np.arange(18).reshape(2,3,3).transpose(2,1,0)
The middle axis is unchanged. There are still columns of [0,3,6], [9,12,15], etc.
It may be easier to visualize the change with 3 different sized axes
In [335]: a=np.arange(2*3*4).reshape(2,3,4)
In [336]: a
Out[336]:
array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]],
[[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]]])
In [337]: a.swapaxes(0,2)
Out[337]:
array([[[ 0, 12],
[ 4, 16],
[ 8, 20]],
[[ 1, 13],
[ 5, 17],
[ 9, 21]],
[[ 2, 14],
[ 6, 18],
[10, 22]],
[[ 3, 15],
[ 7, 19],
[11, 23]]])
Notice what happens when I flatten the array
In [338]: a.swapaxes(0,2).ravel()
Out[338]:
array([ 0, 12, 4, 16, 8, 20, 1, 13, 5, 17, 9, 21, 2, 14, 6, 18, 10,
22, 3, 15, 7, 19, 11, 23])
the order of terms has been shuffled. As created it was [0,1,2,3...]. Now the 1 is the 6th term (2x3).
Under the covers numpy actually performs the swap or transpose by changing shape, strides and order, without changing the data buffer (i.e. it's a view). But further reshaping, including raveling, forces it to make a copy. But that might be more confusing than helpful at this stage.
In numpy axes are numbered. Terms like x,y,z or planes, rows, columns may help you map those on to constructs that you can visualize, but they aren't 'built-in'. Describing the swap or transpose in words is tricky.
Given a square array D of dimension(4x4):
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
how can you, for each sub-matrix d of size (2x2) create another set of matrices D*, such that
D* = [...,
[[ 1, 2, 1, 2],
[ 3, 4, 3, 4],
[ 1, 2, 1, 2],
[ 3, 4, 3, 4]]
]
I then wish to build another square array out of D*, D**, such that:
D** = [[ 1, 2, 1, 2, 5, 6, 5, 6],
[ 3, 4, 3, 4, 7, 8, 7, 8],
[ 1, 2, 1, 2, 5, 6, 5, 6],
[ 3, 4, 3, 4, 7, 8, 7, 8],
[ 9, 10, 9, 10, 13, 14, 13, 14],
[ 11, 12, 11, 12, 15, 16, 15, 16],
[ 9, 10, 9, 10, 13, 14, 13, 14],
[ 11, 12, 11, 12, 15, 16, 15, 16]]]
My actual starting matrix D's dimension is 184x184, so I found that for-loops were too slow to achieve this. Is this too computationally intensive for numpy? Or is there a way to achieve this elegantly an efficiently?
Here is an example of the foor-loop pseudo-code:
segments = [(0,0), (2,2), (0,2), (2, 0)]
for seg in segments:
actual_seg = D[seg[0]:seg[0]+2, seg[1]:seg[1]+2]
D*.append(numpy.kron(numpy.ones((2, 2), dtype=int), actual_seg))
Given D and looking to expand each such (2x2) submatrix, one approach using a combination of np.tile and np.repeat would be -
m,n = D.shape
out = np.repeat(np.tile(D.reshape(m//2,2,n//2,2),2),2,axis=0).reshape(2*m,2*n)
Sample run -
In [116]: D
Out[116]:
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
In [117]: m,n = D.shape
In [118]: np.repeat(np.tile(D.reshape(m//2,2,n//2,2),2),2,axis=0).reshape(2*m,2*n)
Out[118]:
array([[ 1, 2, 1, 2, 5, 6, 5, 6],
[ 3, 4, 3, 4, 7, 8, 7, 8],
[ 1, 2, 1, 2, 5, 6, 5, 6],
[ 3, 4, 3, 4, 7, 8, 7, 8],
[ 9, 10, 9, 10, 13, 14, 13, 14],
[11, 12, 11, 12, 15, 16, 15, 16],
[ 9, 10, 9, 10, 13, 14, 13, 14],
[11, 12, 11, 12, 15, 16, 15, 16]])
from __future__ import print_function
import numpy as np
def fn(m_):
rows, cols = m_.shape
assert rows%2==0 and cols%2==0
return np.reshape(
np.transpose(
np.tile(
np.transpose(
np.reshape(
m_,(rows//2,2, cols//2,2)
),(0,2,1,3)),(2,2)), (0,2,1,3)
),
(rows*2,cols*2)
)
m = np.array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
print(fn(m))
output:
[[ 1 2 1 2 5 6 5 6]
[ 3 4 3 4 7 8 7 8]
[ 1 2 1 2 5 6 5 6]
[ 3 4 3 4 7 8 7 8]
[ 9 10 9 10 13 14 13 14]
[11 12 11 12 15 16 15 16]
[ 9 10 9 10 13 14 13 14]
[11 12 11 12 15 16 15 16]]
To reverse the last row is the first, we can write:
import numpy as np
a = np.arange(20)
a = a.reshape(4,5)
c = a[::-1,:]
print c
c:
[[15 16 17 18 19]
[10 11 12 13 14]
[ 5 6 7 8 9]
[ 0 1 2 3 4]]
But how does the slicing reverse use the last column last line be the first before?
I got just a line this way. But how do I arrive until the beginning of the matrix with this statement?
a[-1, -1::-1]
a:
[19 18 17 16 15]
You can reverse both the rows and columns of the 2D array by using the slice ::-1 in each axis:
>>> a[::-1, ::-1]
array([[19, 18, 17, 16, 15],
[14, 13, 12, 11, 10],
[ 9, 8, 7, 6, 5],
[ 4, 3, 2, 1, 0]])
A couple of other ways:
Reverse before making it 2d:
In [928]: np.arange(20)[::-1].reshape(4,5)
Out[928]:
array([[19, 18, 17, 16, 15],
[14, 13, 12, 11, 10],
[ 9, 8, 7, 6, 5],
[ 4, 3, 2, 1, 0]])
Reverse the values and copy them back in with flat.
In [929]: a=np.arange(20).reshape(4,5)
In [930]: a.flat[::-1]
Out[930]:
array([19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3,
2, 1, 0])
In [931]: a.flat[:]=a.flat[::-1]
In [932]: a
Out[932]:
array([[19, 18, 17, 16, 15],
[14, 13, 12, 11, 10],
[ 9, 8, 7, 6, 5],
[ 4, 3, 2, 1, 0]])
Consider an array of the following form (just an example):
[[ 0 1]
[ 2 3]
[ 4 5]
[ 6 7]
[ 8 9]
[10 11]
[12 13]
[14 15]
[16 17]]
It's shape is [9,2]. Now I want to transform the array so that each column becomes a shape [3,3], like this:
[[ 0 6 12]
[ 2 8 14]
[ 4 10 16]]
[[ 1 7 13]
[ 3 9 15]
[ 5 11 17]]
The most obvious (and surely "non-pythonic") solution is to initialise an array of zeroes with the proper dimension and run two for-loops where it will be filled with data. I'm interested in a solution that is language-conform...
a = np.arange(18).reshape(9,2)
b = a.reshape(3,3,2).swapaxes(0,2)
# a:
array([[ 0, 1],
[ 2, 3],
[ 4, 5],
[ 6, 7],
[ 8, 9],
[10, 11],
[12, 13],
[14, 15],
[16, 17]])
# b:
array([[[ 0, 6, 12],
[ 2, 8, 14],
[ 4, 10, 16]],
[[ 1, 7, 13],
[ 3, 9, 15],
[ 5, 11, 17]]])
numpy has a great tool for this task ("numpy.reshape") link to reshape documentation
a = [[ 0 1]
[ 2 3]
[ 4 5]
[ 6 7]
[ 8 9]
[10 11]
[12 13]
[14 15]
[16 17]]
`numpy.reshape(a,(3,3))`
you can also use the "-1" trick
`a = a.reshape(-1,3)`
the "-1" is a wild card that will let the numpy algorithm decide on the number to input when the second dimension is 3
so yes.. this would also work:
a = a.reshape(3,-1)
and this:
a = a.reshape(-1,2)
would do nothing
and this:
a = a.reshape(-1,9)
would change the shape to (2,9)
There are two possible result rearrangements (following example by #eumiro). Einops package provides a powerful notation to describe such operations non-ambigously
>> a = np.arange(18).reshape(9,2)
# this version corresponds to eumiro's answer
>> einops.rearrange(a, '(x y) z -> z y x', x=3)
array([[[ 0, 6, 12],
[ 2, 8, 14],
[ 4, 10, 16]],
[[ 1, 7, 13],
[ 3, 9, 15],
[ 5, 11, 17]]])
# this has the same shape, but order of elements is different (note that each paer was trasnposed)
>> einops.rearrange(a, '(x y) z -> z x y', x=3)
array([[[ 0, 2, 4],
[ 6, 8, 10],
[12, 14, 16]],
[[ 1, 3, 5],
[ 7, 9, 11],
[13, 15, 17]]])