We Keep Coding

Numpy "Fortran"-like reshape?

Let's say I have an array X of shape (6, 2) like this:
import numpy as np
X = np.array([[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12]])
I want to reshape it to an array of shape (3, 2, 2), so I did this:
X.reshape(3, 2, 2)
And got:
array([[[ 1, 2],
[ 3, 4]],
[[ 5, 6],
[ 7, 8]],
[[ 9, 10],
[11, 12]]])
However, I need my data in a different format. To be precise, I want to end up wth:
array([[[ 1, 2],
[ 7, 8]],
[[ 3, 4],
[ 9, 10]],
[[ 5, 6],
[11, 12]]])
Should I be using reshape for this or something else? What's the best way to do this in Numpy?

You have to set the order option:
>>> X.reshape(3, 2, 2, order='F')
array([[[ 1, 2],
[ 7, 8]],
[[ 3, 4],
[ 9, 10]],
[[ 5, 6],
[11, 12]]])
‘F’ means to read / write the elements using Fortran-like index order, with the first index changing fastest, and the last index changing slowest.
see: https://numpy.org/doc/stable/reference/generated/numpy.reshape.html

You need to specify order;
X.reshape(3, 2, 2, order='F')
should work

A functional equivalent to the order='F' reshape:
In [31]: x.reshape(2,3,2).transpose(1,0,2)
Out[31]:
array([[[ 1, 2],
[ 7, 8]],
[[ 3, 4],
[ 9, 10]],
[[ 5, 6],
[11, 12]]])
In [32]: x.reshape(2,3,2).transpose(1,0,2).strides
Out[32]: (16, 48, 8)
Without the transpose the strides would be (48,16,8).
A thing that's a bit tricky about this layout is that the last dimension remains in 'C' order. It's the just first two dimension that are switched.
The full 'F' layout would be
In [33]: x = np.arange(1,13).reshape(3,2,2,order='F')
In [34]: x
Out[34]:
array([[[ 1, 7],
[ 4, 10]],
[[ 2, 8],
[ 5, 11]],
[[ 3, 9],
[ 6, 12]]])

Create meshgrids from 2D slices off 3D array

I have 3D numpy array.
import numpy as np
X = np.arange(12).reshape(2, 2, 3)
print(X)
[[[ 0 1 2]
[ 3 4 5]]
[[ 6 7 8]
[ 9 10 11]]]
I would like to vectorize the following for all 2D array in 3D array. For example, for 1st 2D array:
ss = np.array(np.meshgrid(*X[0]), dtype=object).T.reshape(-1,2)
print(ss)
[[0 3]
[0 4]
[0 5]
[1 3]
[1 4]
[1 5]
[2 3]
[2 4]
[2 5]]
I tried following:
def f(x):
return np.array(np.meshgrid(*x), dtype=object).T.reshape(-1,2)
ff = np.apply_along_axis(f, 0, X)
print(ff)

Here's a generic solution that uses one-loop and scales to generic shapes. It assigns into an initialized array and broadcasts to replicate values, where it achieves memory efficiency. It works for any length along the second axis of X. Hence, the implementation would be -
def meshgrid_2D_blocks(X):
m,n,r = X.shape
out_shp = [m]+[r]*n+[n]
out = np.empty(out_shp,dtype=X.dtype)
# Assign each block iteratively
shp = [-1]+[1]*n
for i in range(n):
shp[i+1] = r
out[...,i] = X[:,i].reshape(shp)
shp[i+1] = 1
return out.reshape(m,-1,n)
Sample runs
Case #1 : Second axis of length=2
In [167]: X = np.arange(12).reshape(2, 2, 3)
In [168]: X
Out[168]:
array([[[ 0, 1, 2],
[ 3, 4, 5]],
[[ 6, 7, 8],
[ 9, 10, 11]]])
In [169]: meshgrid_2D_blocks(X)
Out[169]:
array([[[ 0, 3],
[ 0, 4],
[ 0, 5],
[ 1, 3],
[ 1, 4],
[ 1, 5],
[ 2, 3],
[ 2, 4],
[ 2, 5]],
[[ 6, 9],
[ 6, 10],
[ 6, 11],
[ 7, 9],
[ 7, 10],
[ 7, 11],
[ 8, 9],
[ 8, 10],
[ 8, 11]]])
Case #2 : Second axis of length=3
In [170]: X = np.arange(12).reshape(2, 3, 2)
In [171]: X
Out[171]:
array([[[ 0, 1],
[ 2, 3],
[ 4, 5]],
[[ 6, 7],
[ 8, 9],
[10, 11]]])
In [172]: meshgrid_2D_blocks(X)
Out[172]:
array([[[ 0, 2, 4],
[ 0, 2, 5],
[ 0, 3, 4],
[ 0, 3, 5],
[ 1, 2, 4],
[ 1, 2, 5],
[ 1, 3, 4],
[ 1, 3, 5]],
[[ 6, 8, 10],
[ 6, 8, 11],
[ 6, 9, 10],
[ 6, 9, 11],
[ 7, 8, 10],
[ 7, 8, 11],
[ 7, 9, 10],
[ 7, 9, 11]]])

This is one way to achieve that:
import numpy as np
X = np.arange(12).reshape(2, 2, 3)
out = np.stack(np.broadcast_arrays(X[:, 0, :, None], X[:, 1, None, :]), -1).reshape(len(X), -1, 2)
print(out)
# [[[ 0 3]
# [ 0 4]
# [ 0 5]
# [ 1 3]
# [ 1 4]
# [ 1 5]
# [ 2 3]
# [ 2 4]
# [ 2 5]]
#
# [[ 6 9]
# [ 6 10]
# [ 6 11]
# [ 7 9]
# [ 7 10]
# [ 7 11]
# [ 8 9]
# [ 8 10]
# [ 8 11]]]

Reshape numpy array in z axis

I have a data set like this
a = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
How can I reshape this into shape (3,2,2) so that a[:,0,0] = [1,2,3]?

you can use two steps:
step1.
In [28]: b1 = np.reshape(a,(3,4), order='F')
In [29]: b1
Out[29]:
array([[ 1, 4, 7, 10],
[ 2, 5, 8, 11],
[ 3, 6, 9, 12]])
use order='F' means to read / write the elements using Fortran-like index order, with the first index changing fastest, and the last index changing slowest. numpy.reshape
setp2
In [30]: c = b1.reshape(3,2,2)
In [31]: c
Out[31]:
array([[[ 1, 4],
[ 7, 10]],
[[ 2, 5],
[ 8, 11]],
[[ 3, 6],
[ 9, 12]]])
get the final result:
In [34]: c[:,0,0]
Out[34]: array([1, 2, 3])

In [30]: a=np.arange(1,13)
In [31]: a
Out[31]: array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])
Since you want to keep the first 3 values 'together', we could start with a reshape like:
In [32]: a.reshape(2,2,3)
Out[32]:
array([[[ 1, 2, 3],
[ 4, 5, 6]],
[[ 7, 8, 9],
[10, 11, 12]]])
and then swap a couple of the axes:
In [33]: a.reshape(2,2,3).transpose(2,0,1)
Out[33]:
array([[[ 1, 4],
[ 7, 10]],
[[ 2, 5],
[ 8, 11]],
[[ 3, 6],
[ 9, 12]]])
In [34]: _[:,0,0]
Out[34]: array([1, 2, 3])
Or with a different transpose:
In [35]: a.reshape(2,2,3).transpose(2,1,0)
Out[35]:
array([[[ 1, 7],
[ 4, 10]],
[[ 2, 8],
[ 5, 11]],
[[ 3, 9],
[ 6, 12]]])
transpose() with an argument, (also invoked with .T) does the same thing.
So your question is a bit ambiguous.
So does the reshape with order F mentioned in the other answer:
In [37]: a.reshape(3,2,2, order='F')
Out[37]:
array([[[ 1, 7],
[ 4, 10]],
[[ 2, 8],
[ 5, 11]],
[[ 3, 9],
[ 6, 12]]])
(though the two step, a.reshape(3,4, order='F').reshape(3,2,2) produces my first result Out[33]).

numpy: fancy indexing "unexpected behaviour" -- fancy indexing seems to give results that are the "transpose" of what would be expected intuitively

I am a bit confused about the behaviour of fancy indexing, see:
>>> t = np.arange(2*2*3).reshape((2, 2, 3))
>>> t
array([[[ 0, 1, 2],
[ 3, 4, 5]],
[[ 6, 7, 8],
[ 9, 10, 11]]])
>>> t[1, :, [1, 2]]
array([[ 7, 10],
[ 8, 11]])
I thought that after indexing with t[1, :, [1, 2]] I would have had the array:
array([[ 7, 8],
[10, 11]])
but instead I get the tranpose, as can be seen above.
Also, consider the following:
>>> t[:, :, [1, 2]][1]
array([[ 7, 8],
[10, 11]])
This doesn't follow the pattern of behaviour we just noted as being unintuitive...this behaves "as expected". Why?
Why do I get this behaviour, and how can I get the behaviour that I expected?

Per the docs,
In simple cases (i.e. one indexing array
and N - 1 slice objects) it does exactly what you would expect (concatenation of
repeated application of basic slicing).
In this case, the indexing array is [1, 2], and the slice objects are 1 and : (or more precisely, slice(1,2), and slice(None)):
So the result is the concatenation of the slices
In [43]: t[1,:,1]
Out[43]: array([ 7, 10])
In [44]: t[1,:,2]
Out[44]: array([ 8, 11])
Also note that the shape of t[1, :, [1,2]] will be (2,2) since the scalar 1
removes the 0 axis and the : spans all of axis 1 (which has length 2), and [1,2]
has length 2. So as you run over the last (i.e. second) axis of the result, you get the arrays
array([ 7, 10]) and array([ 8, 11]).
In [45]: t[1, :, [1,2]]
Out[45]:
array([[ 7, 10],
[ 8, 11]])
The easiest way to get the result you want is to use basic slicing,
In [45]: t[1, :, 1:3]
Out[45]:
array([[ 7, 8],
[10, 11]])
Another way, which uses "fancy" integer indexing is:
In [121]: t[1, [(0,0),(1,1)], [1,2]]
Out[121]:
array([[ 7, 8],
[10, 11]])
or (using broadcasting)
In [154]: t[1, [[0],[1]], [1,2]]
Out[154]:
array([[ 7, 8],
[10, 11]])
This might actually be closer to what you want, since it is generalizable to the case where your indexing array is some arbitrary list like [1, 5, 9, 10].
In [157]: t = np.arange(2*2*11).reshape(2,2,11)
In [158]: t[1, [[0],[1]], [1,5,9,10]]
Out[158]:
array([[23, 27, 31, 32],
[34, 38, 42, 43]])
The same rule applies to
In [101]: t[:, :, [1, 2]][1]
Out[101]:
array([[ 7, 8],
[10, 11]])
First note that the shape of t[:, :, [1, 2]] will be (2,2,2). The result will be the concatenation of the basic slices
In [102]: t[:, :, 1]
Out[102]:
array([[ 1, 4],
[ 7, 10]])
In [103]: t[:, :, 2]
Out[103]:
array([[ 2, 5],
[ 8, 11]])
So as you run over the last (i.e. third) axis of the result, you get the arrays
array([[ 1, 4], [ 7, 10]]) and array([[ 2, 5], [ 8, 11]]).
In [107]: np.allclose(t[:, :, [1,2]], np.dstack([np.array([[ 1, 4], [ 7, 10]]), np.array([[ 2, 5], [ 8, 11]])]))
Out[107]: True

It seems like the behavior changes when there's a non-slicing in any previous axes:
>>> import numpy as np
>>> t = np.arange(2*2*3).reshape((2, 2, 3))
>>> t[1:2, :, [1, 2]] # 1 -> 1:2 (slicing)
array([[[ 7, 8],
[10, 11]]])
>>> t[1][:, [1, 2]]
array([[ 7, 8],
[10, 11]])

Row-wise scaling with Numpy

I have an array H of dimension MxN, and an array A of dimension M . I want to scale H rows with array A. I do it this way, taking advantage of element-wise behaviour of Numpy
H = numpy.swapaxes(H, 0, 1)
H /= A
H = numpy.swapaxes(H, 0, 1)
It works, but the two swapaxes operations are not very elegant, and I feel there is a more elegant and consise way to achieve the result, without creating temporaries. Would you tell me how ?

I think you can simply use H/A[:,None]:
In [71]: (H.swapaxes(0, 1) / A).swapaxes(0, 1)
Out[71]:
array([[ 8.91065496e-01, -1.30548362e-01, 1.70357901e+00],
[ 5.06027691e-02, 3.59913305e-01, -4.27484490e-03],
[ 4.72868136e-01, 2.04351398e+00, 2.67527572e+00],
[ 7.87239835e+00, -2.13484271e+02, -2.44764975e+02]])
In [72]: H/A[:,None]
Out[72]:
array([[ 8.91065496e-01, -1.30548362e-01, 1.70357901e+00],
[ 5.06027691e-02, 3.59913305e-01, -4.27484490e-03],
[ 4.72868136e-01, 2.04351398e+00, 2.67527572e+00],
[ 7.87239835e+00, -2.13484271e+02, -2.44764975e+02]])
because None (or newaxis) extends A in dimension (example link):
In [73]: A
Out[73]: array([ 1.1845468 , 1.30376536, -0.44912446, 0.04675434])
In [74]: A[:,None]
Out[74]:
array([[ 1.1845468 ],
[ 1.30376536],
[-0.44912446],
[ 0.04675434]])

You just need to reshape A so that it will broad cast properly:
A = A.reshape((-1, 1))
so:
In [21]: M
Out[21]:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 14],
[15, 16, 17],
[18, 19, 20]])
In [22]: A
Out[22]: array([1, 2, 3, 4, 5, 6, 7])
In [23]: M / A.reshape((-1, 1))
Out[23]:
array([[0, 1, 2],
[1, 2, 2],
[2, 2, 2],
[2, 2, 2],
[2, 2, 2],
[2, 2, 2],
[2, 2, 2]])