Efficient Numpy sampling of subarray in a wrapped 2D array - python

I have a 4x4 array and wonder if there's a way to randomly sample a 2x2 square from it at any location, allowing for the square to wrapped around when it reaches an edge.
For example:
>> A = np.arange(16).reshape(4,-1)
>> start_point = A[0,3]
start_point = 3
the square would be [[15, 12], [3,0]]


I've generalized (a little...) my answer, permitting a rectangular input array
and even a rectangular random sample
def rand_sample(arr, nr, nc):
"sample a nr x nc submatrix starting from a random element of arr, with wrap"
r, c = arr.shape
i, j = np.random.randint(r), np.random.randint(c)
r = np.arange(i, i+nr) % r
c = np.arange(j, j+nc) % c
return arr[r][:,c]
You may want to check that arr is a 2D array


You could use np.lib.pad with 'wrap' option to have a padded version of the input array that has wrapped around elements at the end along the rows and columns. Finally, index into the padded array to get the desired output. Here's the implementation -
import numpy as np
# Input array and start row-col indices and neighbourhood extent
A = np.arange(42).reshape(6,-1)
start_pt = [0,3] # start_point
N = 5 # neighbourhood extent
# Pad boundary with wrapped elements
A_pad = np.lib.pad(A, ((0,N-1),(0,N-1)), 'wrap')
# Final output after indexing into padded array
out = A_pad[start_pt[0]:start_pt[0]+N,start_pt[1]:start_pt[1]+N]
Sample run -
In [192]: A
Out[192]:
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34],
[35, 36, 37, 38, 39, 40, 41]])
In [193]: start_pt
Out[193]: [0, 3]
In [194]: A_pad
Out[194]:
array([[ 0, 1, 2, 3, 4, 5, 6, 0, 1, 2, 3],
[ 7, 8, 9, 10, 11, 12, 13, 7, 8, 9, 10],
[14, 15, 16, 17, 18, 19, 20, 14, 15, 16, 17],
[21, 22, 23, 24, 25, 26, 27, 21, 22, 23, 24],
[28, 29, 30, 31, 32, 33, 34, 28, 29, 30, 31],
[35, 36, 37, 38, 39, 40, 41, 35, 36, 37, 38],
[ 0, 1, 2, 3, 4, 5, 6, 0, 1, 2, 3],
[ 7, 8, 9, 10, 11, 12, 13, 7, 8, 9, 10],
[14, 15, 16, 17, 18, 19, 20, 14, 15, 16, 17],
[21, 22, 23, 24, 25, 26, 27, 21, 22, 23, 24]])
In [195]: out
Out[195]:
array([[ 3, 4, 5, 6, 0],
[10, 11, 12, 13, 7],
[17, 18, 19, 20, 14],
[24, 25, 26, 27, 21],
[31, 32, 33, 34, 28]])

Related

How can two items be randomly selected and moved from one list to another list?

I want a method to select two elements (as sub-lists) from one list and then move them to another list, while removing them from the first list.
My code:
A=[[17, 4, 10, 18, 11], [23, 16, 24, 29, 19, 12, 22], [15, 2, 32, 30, 20, 7, 25], [33, 21, 5, 13, 6, 28, 26, 31], [27, 8, 9, 14, 3]]
B = random.sample(A, 2)
print("B",B)
My results:
B [[27, 8, 9, 14, 3], [33, 21, 5, 13, 6, 28, 26, 31]]
Expected results:
A [[17, 4, 10, 18, 11], [23, 16, 24, 29, 19, 12, 22], [15, 2, 32, 30, 20, 7, 25]]
B [[27, 8, 9, 14, 3], [33, 21, 5, 13, 6, 28, 26, 31]]

You could randomly select an index from A and pop/append from one list to the other
import random
A = [[17, 4, 10, 18, 11], [23, 16, 24, 29, 19, 12, 22], [15, 2, 32, 30, 20, 7, 25], [33, 21, 5, 13, 6, 28, 26, 31], [27, 8, 9, 14, 3]]
B = []
for _ in range(2):
B.append(A.pop(random.randrange(0, len(A))))
Result
>>> A
[[15, 2, 32, 30, 20, 7, 25], [33, 21, 5, 13, 6, 28, 26, 31], [27, 8, 9, 14, 3]]
>>> B
[[23, 16, 24, 29, 19, 12, 22], [17, 4, 10, 18, 11]]
Note that you need to re-evaluate len(A) within your loop when you call randrange since the size is decreasing as you pop elements.

A not too performant solution:
import random
A = [
[17, 4, 10, 18, 11],
[23, 16, 24, 29, 19, 12, 22],
[15, 2, 32, 30, 20, 7, 25],
[33, 21, 5, 13, 6, 28, 26, 31],
[27, 8, 9, 14, 3],
]
B = []
for i in random.sample(range(len(A)), 2):
B.append(A[i])
A.remove(A[i])
A, B
>>>
([[17, 4, 10, 18, 11],
[23, 16, 24, 29, 19, 12, 22],
[33, 21, 5, 13, 6, 28, 26, 31]],
[[15, 2, 32, 30, 20, 7, 25], [27, 8, 9, 14, 3]])
A more effective one:
idx = sorted(random.sample(range(len(A)), 2))
A_new = []
B = []
for i, a in enumerate(A):
if i in idx:
B.append(a)
idx = idx[1:]
else:
A.append(a)
A = A_new

A very simple way would be to just iterate over the elements you have in B and removing them from A after the sampling.
for el in B:
A.remove(el)
>>> A
[[15, 2, 32, 30, 20, 7, 25], [33, 21, 5, 13, 6, 28, 26, 31], [27, 8, 9, 14, 3]]
>>> B
[[23, 16, 24, 29, 19, 12, 22], [17, 4, 10, 18, 11]]

one possible solution
import random
A=[[17, 4, 10, 18, 11], [23, 16, 24, 29, 19, 12, 22], [15, 2, 32, 30, 20, 7, 25], [33, 21, 5, 13, 6, 28, 26, 31], [27, 8, 9, 14, 3]]
B_with_index = random.sample((list(enumerate(A))), 2)
B = []
for element in B_with_index:
B.append(element[1])
A.pop(element[0])

Please explain unravel_index when order='F'

I understood unravel_index in general but don't get how this example works :
np.unravel_index([31, 41, 13], (7,6), order='F')

C layout:
In [447]: np.arange(42).reshape(7,6)
Out[447]:
array([[ 0, 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17],
[18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35],
[36, 37, 38, 39, 40, 41]])
In [448]: np.unravel_index([31, 41, 13], (7,6), order='C')
Out[448]: (array([5, 6, 2]), array([1, 5, 1]))
F layout:
In [450]: np.arange(42).reshape(7,6,order='F')
Out[450]:
array([[ 0, 7, 14, 21, 28, 35],
[ 1, 8, 15, 22, 29, 36],
[ 2, 9, 16, 23, 30, 37],
[ 3, 10, 17, 24, 31, 38],
[ 4, 11, 18, 25, 32, 39],
[ 5, 12, 19, 26, 33, 40],
[ 6, 13, 20, 27, 34, 41]])
In [451]: np.unravel_index([31, 41, 13], (7,6), order='F')
Out[451]: (array([3, 6, 6]), array([4, 5, 1]))

"Checkbord-like" binary selection

I have a 2d array
test = np.arange(25).reshape((5, 5))
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
for which I need a "checkboard" like selector. Basically, I want to select odd elements in the first row, even elements in the second row, odd elements in the third row etc..
In this case (because the row numbers are odd) I can get that via
test.flatten()[::2]
Out[22]: array([ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24])
which is exactly "every other element" in both dimensions. But if we try that with
test2 = np.arange(16).reshape((4, 4))
test2
Out[23]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
test2.flatten()[::2]
Out[27]: array([ 0, 2, 4, 6, 8, 10, 12, 14])
unsurprisingly, it doesn't work out.
What's a general way of generating the type of selection that I want, no matter the shape of the (2d) matrix? Is it easily (or even possible) to extend to 3d matrices?
I do prefer efficient (vectorized or cython-based) approaches.
Bonus: 3D:
test3 = np.arange(25*3).reshape((3, 5, 5))
Out[30]:
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]],
[[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39],
[40, 41, 42, 43, 44],
[45, 46, 47, 48, 49]],
[[50, 51, 52, 53, 54],
[55, 56, 57, 58, 59],
[60, 61, 62, 63, 64],
[65, 66, 67, 68, 69],
[70, 71, 72, 73, 74]]])
Here it's not about odd and even rows, but rather about neighboring elements. No two neighboring elements should have the same color (be part of the same bool selector).
That is, if we select test3[0, ...].flatten()[::2] we get [0, ... 24]. The neighboring element of 0 in the first dimension is 25, so we don't want that. The expected output is
np.hstack((test3[0, ...].flatten()[::2], test3[1, ...].flatten()[1::2], test3[2, ...].flatten()[::2], ))
Out[42]:
array([ 0, 2, 4, 6, 8, 10, 12, 14, 17, 19, 21, 23, 25, 27, 29, 31, 32,
34, 36, 38, 40, 42, 44, 46])

Here's one vectorized solution -
def checker_select(a):
m,n = a.shape[-2:]
i,j = np.ogrid[:m,:n]
mask = (i+j)%2==0
return a.reshape(-1,m,n)[:,mask].ravel()
Probably a faster one with array-initialization -
def checker_select_v2(a):
m,n = a.shape[-2:]
mask = np.zeros((m,n), dtype=bool)
mask[::2,::2] = 1
mask[1::2,1::2] = 1
return a.reshape(-1,m,n)[:,mask].ravel()
Sample run for 2D -
In [117]: a
Out[117]:
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
In [118]: checker_select(a)
Out[118]: array([ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24])
Sample run for 3D -
In [144]: a
Out[144]:
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[ 10, 11, 12, 13, 14],
[ 15, 16, 17, 18, 19],
[ 20, 21, 22, 23, 24]],
[[ 25, 26, 27, 28, 29],
[ 30, 31, 32, 33, 34],
[ 35, 36, 37, 38, 39],
[ 40, 41, 42, 43, 44],
[ 45, 46, 47, 48, 49]],
....
In [145]: checker_select(a)
Out[145]:
array([ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24,
25, 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49,
....

You can do that in plain old python
mat = np.arange(16).reshape((4, 4))
result = []
for row in range(len(mat)):
for col in range(len(mat[row])):
if row % 2 == 0 and col % 2 == 0:
result.append(mat[row][col])
elif row % 2 == 1 and col % 2 == 1:
result.append(mat[row][col])
print(result)
Output
[0, 2, 5, 7, 8, 10, 13, 15]
If you are looking for a more pythonic way.
n = 6
m = 4
mat = np.arange(n * m).reshape((n, m))
flat = mat.flatten()
result = [flat[i] for i in range(len(flat)) if i // m % 2 == i % m % 2]
print(result)
Output
[0, 2, 5, 7, 8, 10, 13, 15, 16, 18, 21, 23]

Example using basic lists and plain python.
matrix = [
[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]
]
result = (row[i % 2::2] for i, row in enumerate(matrix))
flattened_result = [cell for filtered_row in result for cell in filtered_row]
print(flattened_result)

Numpy roll vertical in 2d array

How would one (efficiently) do the following:
x = np.arange(49)
x2 = np.reshape(x, (7,7))
x2
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34],
[35, 36, 37, 38, 39, 40, 41],
[42, 43, 44, 45, 46, 47, 48]])
From here I want to roll a couple of things.
I want to roll 0,7,14,21 etc so 14 comes to top.
Then the same with 4,11,18,25 etc so 39 comes to top.
Result should be:
x2
array([[14, 1, 2, 3, 39, 5, 6],
[21, 8, 9, 10, 46, 12, 13],
[28, 15, 16, 17, 4, 19, 20],
[35, 22, 23, 24, 11, 26, 27],
[42, 29, 30, 31, 18, 33, 34],
[ 0, 36, 37, 38, 25, 40, 41],
[ 7, 43, 44, 45, 32, 47, 48]])
I looked up numpy.roll, here and google but couldn't find how one would do this.
For horizontal rolls, I could do:
np.roll(x2[0], 3, axis=0)
x3
array([4, 5, 6, 0, 1, 2, 3])
But how do I return the full array with this roll change as a new copy?

Roll with a negative shift:
x2[:, 0] = np.roll(x2[:, 0], -2)
Roll with a positive shift:
x2[:, 4] = np.roll(x2[:, 4], 2)
gives:
>>>x2
array([[14, 1, 2, 3, 39, 5, 6],
[21, 8, 9, 10, 46, 12, 13],
[28, 15, 16, 17, 4, 19, 20],
[35, 22, 23, 24, 11, 26, 27],
[42, 29, 30, 31, 18, 33, 34],
[ 0, 36, 37, 38, 25, 40, 41],
[ 7, 43, 44, 45, 32, 47, 48]])

Here's a way to roll multiple columns in one go with advanced-indexing -
# Params
cols = [0,4] # Columns to be rolled
dirn = [2,-2] # Offset with direction as sign
n = x2.shape[0]
x2[:,cols] = x2[np.mod(np.arange(n)[:,None] + dirn,n),cols]
Sample run -
In [45]: x2
Out[45]:
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12, 13],
[14, 15, 16, 17, 18, 19, 20],
[21, 22, 23, 24, 25, 26, 27],
[28, 29, 30, 31, 32, 33, 34],
[35, 36, 37, 38, 39, 40, 41],
[42, 43, 44, 45, 46, 47, 48]])
In [46]: cols = [0,4,5] # Columns to be rolled
...: dirn = [2,-2,4] # Offset with direction as sign
...: n = x2.shape[0]
...: x2[:,cols] = x2[np.mod(np.arange(n)[:,None] + dirn,n),cols]
...:
In [47]: x2 # Three columns rolled
Out[47]:
array([[14, 1, 2, 3, 39, 33, 6],
[21, 8, 9, 10, 46, 40, 13],
[28, 15, 16, 17, 4, 47, 20],
[35, 22, 23, 24, 11, 5, 27],
[42, 29, 30, 31, 18, 12, 34],
[ 0, 36, 37, 38, 25, 19, 41],
[ 7, 43, 44, 45, 32, 26, 48]])

You have to overwrite the column
e.g.:
x2[:,0] = np.roll(x2[:,0], 3)

See here a useful method for shifting a 2D array in all 4 directions (up, down, left, right):
def image_shift_roll(img, x_shift, y_roll):
img_roll = img.copy()
img_roll = np.roll(img_roll, -y_roll, axis = 0) # Positive y rolls up
img_roll = np.roll(img_roll, x_roll, axis = 1) # Positive x rolls right
return img_roll

Re-order a numpy array python

I have a big two-dimensional array like this:
array([[ 1, 2, 3, 4, 5, 6, 7, 8],
[ 9,10,11,12,13,14,15,16],
[17,18,19,20,21,22,23,24],
[25,26,27,28,29,30,31,32],
[33,34,35,36,37,38,39,40],
[41,42,43,44,45,46,47,48],
....])
and I need to convert it into:
array([ 1, 9,17, 2,10,18, 3,11,19, 4,12,20, 5,13,21, 6,14,22, 7,15,23, 8,16,24],
[25,33,41,26,34,42,27,35,43,28,36,44,29,37,45,30,38,46,31,39,47,32,40,48],
...
Note that this should only be a demonstration what it should do.
The original array contains only boolean values and has the size of 512x8. In my example, I order only 3 rows with 8 elements into one row but what I really need are respectively 32 rows with 8 elements.
I am really sorry, but after 30 minutes of writing, this is the only description I got of my problem. I hope it is enough.

I think you can achieve your desired result using two reshape operations and a transpose:
x = np.array([[ 1, 2, 3, 4, 5, 6, 7, 8],
[ 9,10,11,12,13,14,15,16],
[17,18,19,20,21,22,23,24],
[25,26,27,28,29,30,31,32],
[33,34,35,36,37,38,39,40],
[41,42,43,44,45,46,47,48]])
y = x.reshape(2, 3, 8).transpose(0, 2, 1).reshape(2, -1)
print(repr(y))
# array([[ 1, 9, 17, 2, 10, 18, 3, 11, 19, 4, 12, 20, 5, 13, 21, 6, 14,
# 22, 7, 15, 23, 8, 16, 24],
# [25, 33, 41, 26, 34, 42, 27, 35, 43, 28, 36, 44, 29, 37, 45, 30, 38,
# 46, 31, 39, 47, 32, 40, 48]])
To break that down a bit:
#hpaulj's first reshape operation gives us this:
x1 = x.reshape(2, 3, 8)
print(repr(x1))
# array([[[ 1, 2, 3, 4, 5, 6, 7, 8],
# [ 9, 10, 11, 12, 13, 14, 15, 16],
# [17, 18, 19, 20, 21, 22, 23, 24]],
# [[25, 26, 27, 28, 29, 30, 31, 32],
# [33, 34, 35, 36, 37, 38, 39, 40],
# [41, 42, 43, 44, 45, 46, 47, 48]]])
print(x1.shape)
# (2, 3, 8)
In order to get the desired output we need to 'collapse' this array along the second dimension (with size 3), then along the third dimension (with size 8).
The easiest way to achieve this sort of thing is to first transpose the
array so that the dimensions you want to collapse along are ordered from first to last:
x2 = x1.transpose(0, 2, 1) # you could also use `x2 = np.rollaxis(x1, 1, 3)`
print(repr(x2))
# array([[[ 1, 9, 17],
# [ 2, 10, 18],
# [ 3, 11, 19],
# [ 4, 12, 20],
# [ 5, 13, 21],
# [ 6, 14, 22],
# [ 7, 15, 23],
# [ 8, 16, 24]],
# [[25, 33, 41],
# [26, 34, 42],
# [27, 35, 43],
# [28, 36, 44],
# [29, 37, 45],
# [30, 38, 46],
# [31, 39, 47],
# [32, 40, 48]]])
print(x2.shape)
# (2, 8, 3)
Finally I can use reshape(2, -1) to collapse the array over the last two dimensions. The -1 causes numpy to infer the appropriate size in the last dimension based on the number of elements in x.
y = x2.reshape(2, -2)

Looks like a starting point is to reshape it, for example
In [49]: x.reshape(2,3,8)
Out[49]:
array([[[ 1, 2, 3, 4, 5, 6, 7, 8],
[ 9, 10, 11, 12, 13, 14, 15, 16],
[17, 18, 19, 20, 21, 22, 23, 24]],
[[25, 26, 27, 28, 29, 30, 31, 32],
[33, 34, 35, 36, 37, 38, 39, 40],
[41, 42, 43, 44, 45, 46, 47, 48]]])
.ravel(order='F') doesn't get it right, so I think we need to swap some axes before flattening. It will need to be a copy.
Using #ali_m's transpose:
In [65]: x1=x.reshape(2,3,8)
In [66]: x1.transpose(0,2,1).flatten()
Out[66]:
array([ 1, 9, 17, 2, 10, 18, 3, 11, 19, 4, 12, 20, 5, 13, 21, 6, 14,
22, 7, 15, 23, 8, 16, 24, 25, 33, 41, 26, 34, 42, 27, 35, 43, 28,
36, 44, 29, 37, 45, 30, 38, 46, 31, 39, 47, 32, 40, 48])
oops - there's an inner layer of nesting that's easy to miss
array([1,9,17,2,10,18,3,11,19,4,12,20,5,13,21,6,14,22,7,15,23,8,16,24],
[25,33,41,26,34,42,27,35,43,28,36,44,29,37,45,30,38,46,31,39,47,32,40,4],
You are missing a [] set. So #ali_m got it right.
I'm tempted to delete this, but my trial and error might be instructive.

Categories