I have a big two-dimensional array like this:
array([[ 1, 2, 3, 4, 5, 6, 7, 8],
[ 9,10,11,12,13,14,15,16],
[17,18,19,20,21,22,23,24],
[25,26,27,28,29,30,31,32],
[33,34,35,36,37,38,39,40],
[41,42,43,44,45,46,47,48],
....])
and I need to convert it into:
array([ 1, 9,17, 2,10,18, 3,11,19, 4,12,20, 5,13,21, 6,14,22, 7,15,23, 8,16,24],
[25,33,41,26,34,42,27,35,43,28,36,44,29,37,45,30,38,46,31,39,47,32,40,48],
...
Note that this should only be a demonstration what it should do.
The original array contains only boolean values and has the size of 512x8. In my example, I order only 3 rows with 8 elements into one row but what I really need are respectively 32 rows with 8 elements.
I am really sorry, but after 30 minutes of writing, this is the only description I got of my problem. I hope it is enough.
I think you can achieve your desired result using two reshape operations and a transpose:
x = np.array([[ 1, 2, 3, 4, 5, 6, 7, 8],
[ 9,10,11,12,13,14,15,16],
[17,18,19,20,21,22,23,24],
[25,26,27,28,29,30,31,32],
[33,34,35,36,37,38,39,40],
[41,42,43,44,45,46,47,48]])
y = x.reshape(2, 3, 8).transpose(0, 2, 1).reshape(2, -1)
print(repr(y))
# array([[ 1, 9, 17, 2, 10, 18, 3, 11, 19, 4, 12, 20, 5, 13, 21, 6, 14,
# 22, 7, 15, 23, 8, 16, 24],
# [25, 33, 41, 26, 34, 42, 27, 35, 43, 28, 36, 44, 29, 37, 45, 30, 38,
# 46, 31, 39, 47, 32, 40, 48]])
To break that down a bit:
#hpaulj's first reshape operation gives us this:
x1 = x.reshape(2, 3, 8)
print(repr(x1))
# array([[[ 1, 2, 3, 4, 5, 6, 7, 8],
# [ 9, 10, 11, 12, 13, 14, 15, 16],
# [17, 18, 19, 20, 21, 22, 23, 24]],
# [[25, 26, 27, 28, 29, 30, 31, 32],
# [33, 34, 35, 36, 37, 38, 39, 40],
# [41, 42, 43, 44, 45, 46, 47, 48]]])
print(x1.shape)
# (2, 3, 8)
In order to get the desired output we need to 'collapse' this array along the second dimension (with size 3), then along the third dimension (with size 8).
The easiest way to achieve this sort of thing is to first transpose the
array so that the dimensions you want to collapse along are ordered from first to last:
x2 = x1.transpose(0, 2, 1) # you could also use `x2 = np.rollaxis(x1, 1, 3)`
print(repr(x2))
# array([[[ 1, 9, 17],
# [ 2, 10, 18],
# [ 3, 11, 19],
# [ 4, 12, 20],
# [ 5, 13, 21],
# [ 6, 14, 22],
# [ 7, 15, 23],
# [ 8, 16, 24]],
# [[25, 33, 41],
# [26, 34, 42],
# [27, 35, 43],
# [28, 36, 44],
# [29, 37, 45],
# [30, 38, 46],
# [31, 39, 47],
# [32, 40, 48]]])
print(x2.shape)
# (2, 8, 3)
Finally I can use reshape(2, -1) to collapse the array over the last two dimensions. The -1 causes numpy to infer the appropriate size in the last dimension based on the number of elements in x.
y = x2.reshape(2, -2)
Looks like a starting point is to reshape it, for example
In [49]: x.reshape(2,3,8)
Out[49]:
array([[[ 1, 2, 3, 4, 5, 6, 7, 8],
[ 9, 10, 11, 12, 13, 14, 15, 16],
[17, 18, 19, 20, 21, 22, 23, 24]],
[[25, 26, 27, 28, 29, 30, 31, 32],
[33, 34, 35, 36, 37, 38, 39, 40],
[41, 42, 43, 44, 45, 46, 47, 48]]])
.ravel(order='F') doesn't get it right, so I think we need to swap some axes before flattening. It will need to be a copy.
Using #ali_m's transpose:
In [65]: x1=x.reshape(2,3,8)
In [66]: x1.transpose(0,2,1).flatten()
Out[66]:
array([ 1, 9, 17, 2, 10, 18, 3, 11, 19, 4, 12, 20, 5, 13, 21, 6, 14,
22, 7, 15, 23, 8, 16, 24, 25, 33, 41, 26, 34, 42, 27, 35, 43, 28,
36, 44, 29, 37, 45, 30, 38, 46, 31, 39, 47, 32, 40, 48])
oops - there's an inner layer of nesting that's easy to miss
array([1,9,17,2,10,18,3,11,19,4,12,20,5,13,21,6,14,22,7,15,23,8,16,24],
[25,33,41,26,34,42,27,35,43,28,36,44,29,37,45,30,38,46,31,39,47,32,40,4],
You are missing a [] set. So #ali_m got it right.
I'm tempted to delete this, but my trial and error might be instructive.
Related
Let's assume that I have input of 4x4 image with 3 channels with following pixel values:
And I want to make it to 12 x 9 matrix of image patches like this (using 2x2 kernel on a 4x4 image):
How can I achieve this using numpy?
Thank you for your help.
Assuming 4x4x3 as input and 12x9 as output
from scipy.signal import convolve
import numpy as np
# creating the 4x4x3 input image
a = np.arange( 1,16+1).reshape(4,4)
b = np.arange(17,32+1).reshape(4,4)
c = np.arange(33,48+1).reshape(4,4)
i_4x4x3 = np.dstack((a, b, c))
# creating four 2x2 kernels
mask_tl = np.array([0,0,0,1]).reshape(2,2)
mask_tr = np.array([0,0,1,0]).reshape(2,2)
mask_bl = np.array([0,1,0,0]).reshape(2,2)
mask_br = np.array([1,0,0,0]).reshape(2,2)
mask_tl = mask_tl[:,:,None]
mask_tr = mask_tr[:,:,None]
mask_bl = mask_bl[:,:,None]
mask_br = mask_br[:,:,None]
# convolving the input with all four kernels
tl = convolve(i_4x4x3, mask_tl, mode='valid')
tr = convolve(i_4x4x3, mask_tr, mode='valid')
bl = convolve(i_4x4x3, mask_bl, mode='valid')
br = convolve(i_4x4x3, mask_br, mode='valid')
i = np.dstack((
tl.reshape(-1,3),
tr.reshape(-1,3),
bl.reshape(-1,3),
br.reshape(-1,3)))
i=i.reshape(i.shape[0],-1).transpose()
display(a,b,c)
display(i)
Output:
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]])
array([[17, 18, 19, 20],
[21, 22, 23, 24],
[25, 26, 27, 28],
[29, 30, 31, 32]])
array([[33, 34, 35, 36],
[37, 38, 39, 40],
[41, 42, 43, 44],
[45, 46, 47, 48]])
array([[ 1, 2, 3, 5, 6, 7, 9, 10, 11],
[ 2, 3, 4, 6, 7, 8, 10, 11, 12],
[ 5, 6, 7, 9, 10, 11, 13, 14, 15],
[ 6, 7, 8, 10, 11, 12, 14, 15, 16],
[17, 18, 19, 21, 22, 23, 25, 26, 27],
[18, 19, 20, 22, 23, 24, 26, 27, 28],
[21, 22, 23, 25, 26, 27, 29, 30, 31],
[22, 23, 24, 26, 27, 28, 30, 31, 32],
[33, 34, 35, 37, 38, 39, 41, 42, 43],
[34, 35, 36, 38, 39, 40, 42, 43, 44],
[37, 38, 39, 41, 42, 43, 45, 46, 47],
[38, 39, 40, 42, 43, 44, 46, 47, 48]])
I can initialize a numpy array and reshape it at the time of creation.
test = np.arange(32).reshape(4, 8)
which produces this:
array([[ 0, 1, 2, 3, 4, 5, 6, 7],
[ 8, 9, 10, 11, 12, 13, 14, 15],
[16, 17, 18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29, 30, 31]])
... but I'd like to know how to start the sequential numbering at a given point, say at 13 rather than at 0. How is that done in numpy?
I've looked for answers and found something somewhat similar but it seems there would be a numpy command to do this.
arange takes an optional start argument.
start = 13 # Any number works here
np.arange(start, start + 32).reshape(4, 8)
# array([[13, 14, 15, 16, 17, 18, 19, 20],
# [21, 22, 23, 24, 25, 26, 27, 28],
# [29, 30, 31, 32, 33, 34, 35, 36],
# [37, 38, 39, 40, 41, 42, 43, 44]])
I have a 2d array
test = np.arange(25).reshape((5, 5))
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
for which I need a "checkboard" like selector. Basically, I want to select odd elements in the first row, even elements in the second row, odd elements in the third row etc..
In this case (because the row numbers are odd) I can get that via
test.flatten()[::2]
Out[22]: array([ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24])
which is exactly "every other element" in both dimensions. But if we try that with
test2 = np.arange(16).reshape((4, 4))
test2
Out[23]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
test2.flatten()[::2]
Out[27]: array([ 0, 2, 4, 6, 8, 10, 12, 14])
unsurprisingly, it doesn't work out.
What's a general way of generating the type of selection that I want, no matter the shape of the (2d) matrix? Is it easily (or even possible) to extend to 3d matrices?
I do prefer efficient (vectorized or cython-based) approaches.
Bonus: 3D:
test3 = np.arange(25*3).reshape((3, 5, 5))
Out[30]:
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]],
[[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39],
[40, 41, 42, 43, 44],
[45, 46, 47, 48, 49]],
[[50, 51, 52, 53, 54],
[55, 56, 57, 58, 59],
[60, 61, 62, 63, 64],
[65, 66, 67, 68, 69],
[70, 71, 72, 73, 74]]])
Here it's not about odd and even rows, but rather about neighboring elements. No two neighboring elements should have the same color (be part of the same bool selector).
That is, if we select test3[0, ...].flatten()[::2] we get [0, ... 24]. The neighboring element of 0 in the first dimension is 25, so we don't want that. The expected output is
np.hstack((test3[0, ...].flatten()[::2], test3[1, ...].flatten()[1::2], test3[2, ...].flatten()[::2], ))
Out[42]:
array([ 0, 2, 4, 6, 8, 10, 12, 14, 17, 19, 21, 23, 25, 27, 29, 31, 32,
34, 36, 38, 40, 42, 44, 46])
Here's one vectorized solution -
def checker_select(a):
m,n = a.shape[-2:]
i,j = np.ogrid[:m,:n]
mask = (i+j)%2==0
return a.reshape(-1,m,n)[:,mask].ravel()
Probably a faster one with array-initialization -
def checker_select_v2(a):
m,n = a.shape[-2:]
mask = np.zeros((m,n), dtype=bool)
mask[::2,::2] = 1
mask[1::2,1::2] = 1
return a.reshape(-1,m,n)[:,mask].ravel()
Sample run for 2D -
In [117]: a
Out[117]:
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
In [118]: checker_select(a)
Out[118]: array([ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24])
Sample run for 3D -
In [144]: a
Out[144]:
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[ 10, 11, 12, 13, 14],
[ 15, 16, 17, 18, 19],
[ 20, 21, 22, 23, 24]],
[[ 25, 26, 27, 28, 29],
[ 30, 31, 32, 33, 34],
[ 35, 36, 37, 38, 39],
[ 40, 41, 42, 43, 44],
[ 45, 46, 47, 48, 49]],
....
In [145]: checker_select(a)
Out[145]:
array([ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24,
25, 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49,
....
You can do that in plain old python
mat = np.arange(16).reshape((4, 4))
result = []
for row in range(len(mat)):
for col in range(len(mat[row])):
if row % 2 == 0 and col % 2 == 0:
result.append(mat[row][col])
elif row % 2 == 1 and col % 2 == 1:
result.append(mat[row][col])
print(result)
Output
[0, 2, 5, 7, 8, 10, 13, 15]
If you are looking for a more pythonic way.
n = 6
m = 4
mat = np.arange(n * m).reshape((n, m))
flat = mat.flatten()
result = [flat[i] for i in range(len(flat)) if i // m % 2 == i % m % 2]
print(result)
Output
[0, 2, 5, 7, 8, 10, 13, 15, 16, 18, 21, 23]
Example using basic lists and plain python.
matrix = [
[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]
]
result = (row[i % 2::2] for i, row in enumerate(matrix))
flattened_result = [cell for filtered_row in result for cell in filtered_row]
print(flattened_result)
How would one (efficiently) do the following:
x = np.arange(49)
x2 = np.reshape(x, (7,7))
x2
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34],
[35, 36, 37, 38, 39, 40, 41],
[42, 43, 44, 45, 46, 47, 48]])
From here I want to roll a couple of things.
I want to roll 0,7,14,21 etc so 14 comes to top.
Then the same with 4,11,18,25 etc so 39 comes to top.
Result should be:
x2
array([[14, 1, 2, 3, 39, 5, 6],
[21, 8, 9, 10, 46, 12, 13],
[28, 15, 16, 17, 4, 19, 20],
[35, 22, 23, 24, 11, 26, 27],
[42, 29, 30, 31, 18, 33, 34],
[ 0, 36, 37, 38, 25, 40, 41],
[ 7, 43, 44, 45, 32, 47, 48]])
I looked up numpy.roll, here and google but couldn't find how one would do this.
For horizontal rolls, I could do:
np.roll(x2[0], 3, axis=0)
x3
array([4, 5, 6, 0, 1, 2, 3])
But how do I return the full array with this roll change as a new copy?
Roll with a negative shift:
x2[:, 0] = np.roll(x2[:, 0], -2)
Roll with a positive shift:
x2[:, 4] = np.roll(x2[:, 4], 2)
gives:
>>>x2
array([[14, 1, 2, 3, 39, 5, 6],
[21, 8, 9, 10, 46, 12, 13],
[28, 15, 16, 17, 4, 19, 20],
[35, 22, 23, 24, 11, 26, 27],
[42, 29, 30, 31, 18, 33, 34],
[ 0, 36, 37, 38, 25, 40, 41],
[ 7, 43, 44, 45, 32, 47, 48]])
Here's a way to roll multiple columns in one go with advanced-indexing -
# Params
cols = [0,4] # Columns to be rolled
dirn = [2,-2] # Offset with direction as sign
n = x2.shape[0]
x2[:,cols] = x2[np.mod(np.arange(n)[:,None] + dirn,n),cols]
Sample run -
In [45]: x2
Out[45]:
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34],
[35, 36, 37, 38, 39, 40, 41],
[42, 43, 44, 45, 46, 47, 48]])
In [46]: cols = [0,4,5] # Columns to be rolled
...: dirn = [2,-2,4] # Offset with direction as sign
...: n = x2.shape[0]
...: x2[:,cols] = x2[np.mod(np.arange(n)[:,None] + dirn,n),cols]
...:
In [47]: x2 # Three columns rolled
Out[47]:
array([[14, 1, 2, 3, 39, 33, 6],
[21, 8, 9, 10, 46, 40, 13],
[28, 15, 16, 17, 4, 47, 20],
[35, 22, 23, 24, 11, 5, 27],
[42, 29, 30, 31, 18, 12, 34],
[ 0, 36, 37, 38, 25, 19, 41],
[ 7, 43, 44, 45, 32, 26, 48]])
You have to overwrite the column
e.g.:
x2[:,0] = np.roll(x2[:,0], 3)
See here a useful method for shifting a 2D array in all 4 directions (up, down, left, right):
def image_shift_roll(img, x_shift, y_roll):
img_roll = img.copy()
img_roll = np.roll(img_roll, -y_roll, axis = 0) # Positive y rolls up
img_roll = np.roll(img_roll, x_roll, axis = 1) # Positive x rolls right
return img_roll
I have a 4x4 array and wonder if there's a way to randomly sample a 2x2 square from it at any location, allowing for the square to wrapped around when it reaches an edge.
For example:
>> A = np.arange(16).reshape(4,-1)
>> start_point = A[0,3]
start_point = 3
the square would be [[15, 12], [3,0]]
I've generalized (a little...) my answer, permitting a rectangular input array
and even a rectangular random sample
def rand_sample(arr, nr, nc):
"sample a nr x nc submatrix starting from a random element of arr, with wrap"
r, c = arr.shape
i, j = np.random.randint(r), np.random.randint(c)
r = np.arange(i, i+nr) % r
c = np.arange(j, j+nc) % c
return arr[r][:,c]
You may want to check that arr is a 2D array
You could use np.lib.pad with 'wrap' option to have a padded version of the input array that has wrapped around elements at the end along the rows and columns. Finally, index into the padded array to get the desired output. Here's the implementation -
import numpy as np
# Input array and start row-col indices and neighbourhood extent
A = np.arange(42).reshape(6,-1)
start_pt = [0,3] # start_point
N = 5 # neighbourhood extent
# Pad boundary with wrapped elements
A_pad = np.lib.pad(A, ((0,N-1),(0,N-1)), 'wrap')
# Final output after indexing into padded array
out = A_pad[start_pt[0]:start_pt[0]+N,start_pt[1]:start_pt[1]+N]
Sample run -
In [192]: A
Out[192]:
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34],
[35, 36, 37, 38, 39, 40, 41]])
In [193]: start_pt
Out[193]: [0, 3]
In [194]: A_pad
Out[194]:
array([[ 0, 1, 2, 3, 4, 5, 6, 0, 1, 2, 3],
[ 7, 8, 9, 10, 11, 12, 13, 7, 8, 9, 10],
[14, 15, 16, 17, 18, 19, 20, 14, 15, 16, 17],
[21, 22, 23, 24, 25, 26, 27, 21, 22, 23, 24],
[28, 29, 30, 31, 32, 33, 34, 28, 29, 30, 31],
[35, 36, 37, 38, 39, 40, 41, 35, 36, 37, 38],
[ 0, 1, 2, 3, 4, 5, 6, 0, 1, 2, 3],
[ 7, 8, 9, 10, 11, 12, 13, 7, 8, 9, 10],
[14, 15, 16, 17, 18, 19, 20, 14, 15, 16, 17],
[21, 22, 23, 24, 25, 26, 27, 21, 22, 23, 24]])
In [195]: out
Out[195]:
array([[ 3, 4, 5, 6, 0],
[10, 11, 12, 13, 7],
[17, 18, 19, 20, 14],
[24, 25, 26, 27, 21],
[31, 32, 33, 34, 28]])