Slice 2d array into smaller 2d arrays - python
Is there a way to slice a 2d array in numpy into smaller 2d arrays?
Example
[[1,2,3,4], -> [[1,2] [3,4]
[5,6,7,8]] [5,6] [7,8]]
So I basically want to cut down a 2x4 array into 2 2x2 arrays. Looking for a generic solution to be used on images.
There was another question a couple of months ago which clued me in to the idea of using reshape and swapaxes. The h//nrows makes sense since this keeps the first block's rows together. It also makes sense that you'll need nrows and ncols to be part of the shape. -1 tells reshape to fill in whatever number is necessary to make the reshape valid. Armed with the form of the solution, I just tried things until I found the formula that works.
You should be able to break your array into "blocks" using some combination of reshape and swapaxes:
def blockshaped(arr, nrows, ncols):
"""
Return an array of shape (n, nrows, ncols) where
n * nrows * ncols = arr.size
If arr is a 2D array, the returned array should look like n subblocks with
each subblock preserving the "physical" layout of arr.
"""
h, w = arr.shape
assert h % nrows == 0, f"{h} rows is not evenly divisible by {nrows}"
assert w % ncols == 0, f"{w} cols is not evenly divisible by {ncols}"
return (arr.reshape(h//nrows, nrows, -1, ncols)
.swapaxes(1,2)
.reshape(-1, nrows, ncols))
turns c
np.random.seed(365)
c = np.arange(24).reshape((4, 6))
print(c)
[out]:
[[ 0 1 2 3 4 5]
[ 6 7 8 9 10 11]
[12 13 14 15 16 17]
[18 19 20 21 22 23]]
into
print(blockshaped(c, 2, 3))
[out]:
[[[ 0 1 2]
[ 6 7 8]]
[[ 3 4 5]
[ 9 10 11]]
[[12 13 14]
[18 19 20]]
[[15 16 17]
[21 22 23]]]
I've posted an inverse function, unblockshaped, here, and an N-dimensional generalization here. The generalization gives a little more insight into the reasoning behind this algorithm.
Note that there is also superbatfish's
blockwise_view. It arranges the
blocks in a different format (using more axes) but it has the advantage of (1)
always returning a view and (2) being capable of handling arrays of any
dimension.
It seems to me that this is a task for numpy.split or some variant.
e.g.
a = np.arange(30).reshape([5,6]) #a.shape = (5,6)
a1 = np.split(a,3,axis=1)
#'a1' is a list of 3 arrays of shape (5,2)
a2 = np.split(a, [2,4])
#'a2' is a list of three arrays of shape (2,5), (2,5), (1,5)
If you have a NxN image you can create, e.g., a list of 2 NxN/2 subimages, and then divide them along the other axis.
numpy.hsplit and numpy.vsplit are also available.
There are some other answers that seem well-suited for your specific case already, but your question piqued my interest in the possibility of a memory-efficient solution usable up to the maximum number of dimensions that numpy supports, and I ended up spending most of the afternoon coming up with possible method. (The method itself is relatively simple, it's just that I still haven't used most of the really fancy features that numpy supports so most of the time was spent researching to see what numpy had available and how much it could do so that I didn't have to do it.)
def blockgen(array, bpa):
"""Creates a generator that yields multidimensional blocks from the given
array(_like); bpa is an array_like consisting of the number of blocks per axis
(minimum of 1, must be a divisor of the corresponding axis size of array). As
the blocks are selected using normal numpy slicing, they will be views rather
than copies; this is good for very large multidimensional arrays that are being
blocked, and for very large blocks, but it also means that the result must be
copied if it is to be modified (unless modifying the original data as well is
intended)."""
bpa = np.asarray(bpa) # in case bpa wasn't already an ndarray
# parameter checking
if array.ndim != bpa.size: # bpa doesn't match array dimensionality
raise ValueError("Size of bpa must be equal to the array dimensionality.")
if (bpa.dtype != np.int # bpa must be all integers
or (bpa < 1).any() # all values in bpa must be >= 1
or (array.shape % bpa).any()): # % != 0 means not evenly divisible
raise ValueError("bpa ({0}) must consist of nonzero positive integers "
"that evenly divide the corresponding array axis "
"size".format(bpa))
# generate block edge indices
rgen = (np.r_[:array.shape[i]+1:array.shape[i]//blk_n]
for i, blk_n in enumerate(bpa))
# build slice sequences for each axis (unfortunately broadcasting
# can't be used to make the items easy to operate over
c = [[np.s_[i:j] for i, j in zip(r[:-1], r[1:])] for r in rgen]
# Now to get the blocks; this is slightly less efficient than it could be
# because numpy doesn't like jagged arrays and I didn't feel like writing
# a ufunc for it.
for idxs in np.ndindex(*bpa):
blockbounds = tuple(c[j][idxs[j]] for j in range(bpa.size))
yield array[blockbounds]
You question practically the same as this one. You can use the one-liner with np.ndindex() and reshape():
def cutter(a, r, c):
lenr = a.shape[0]/r
lenc = a.shape[1]/c
np.array([a[i*r:(i+1)*r,j*c:(j+1)*c] for (i,j) in np.ndindex(lenr,lenc)]).reshape(lenr,lenc,r,c)
To create the result you want:
a = np.arange(1,9).reshape(2,1)
#array([[1, 2, 3, 4],
# [5, 6, 7, 8]])
cutter( a, 1, 2 )
#array([[[[1, 2]],
# [[3, 4]]],
# [[[5, 6]],
# [[7, 8]]]])
Some minor enhancement to TheMeaningfulEngineer's answer that handles the case when the big 2d array cannot be perfectly sliced into equally sized subarrays
def blockfy(a, p, q):
'''
Divides array a into subarrays of size p-by-q
p: block row size
q: block column size
'''
m = a.shape[0] #image row size
n = a.shape[1] #image column size
# pad array with NaNs so it can be divided by p row-wise and by q column-wise
bpr = ((m-1)//p + 1) #blocks per row
bpc = ((n-1)//q + 1) #blocks per column
M = p * bpr
N = q * bpc
A = np.nan* np.ones([M,N])
A[:a.shape[0],:a.shape[1]] = a
block_list = []
previous_row = 0
for row_block in range(bpc):
previous_row = row_block * p
previous_column = 0
for column_block in range(bpr):
previous_column = column_block * q
block = A[previous_row:previous_row+p, previous_column:previous_column+q]
# remove nan columns and nan rows
nan_cols = np.all(np.isnan(block), axis=0)
block = block[:, ~nan_cols]
nan_rows = np.all(np.isnan(block), axis=1)
block = block[~nan_rows, :]
## append
if block.size:
block_list.append(block)
return block_list
Examples:
a = np.arange(25)
a = a.reshape((5,5))
out = blockfy(a, 2, 3)
a->
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
out[0] ->
array([[0., 1., 2.],
[5., 6., 7.]])
out[1]->
array([[3., 4.],
[8., 9.]])
out[-1]->
array([[23., 24.]])
For now it just works when the big 2d array can be perfectly sliced into equally sized subarrays.
The code bellow slices
a ->array([[ 0, 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17],
[18, 19, 20, 21, 22, 23]])
into this
block_array->
array([[[ 0, 1, 2],
[ 6, 7, 8]],
[[ 3, 4, 5],
[ 9, 10, 11]],
[[12, 13, 14],
[18, 19, 20]],
[[15, 16, 17],
[21, 22, 23]]])
p ang q determine the block size
Code
a = arange(24)
a = a.reshape((4,6))
m = a.shape[0] #image row size
n = a.shape[1] #image column size
p = 2 #block row size
q = 3 #block column size
block_array = []
previous_row = 0
for row_block in range(blocks_per_row):
previous_row = row_block * p
previous_column = 0
for column_block in range(blocks_per_column):
previous_column = column_block * q
block = a[previous_row:previous_row+p,previous_column:previous_column+q]
block_array.append(block)
block_array = array(block_array)
If you want a solution that also handles the cases when the matrix is
not equally divided, you can use this:
from operator import add
half_split = np.array_split(input, 2)
res = map(lambda x: np.array_split(x, 2, axis=1), half_split)
res = reduce(add, res)
Here is a solution based on unutbu's answer that handle case where matrix cannot be equally divided. In this case, it will resize the matrix before using some interpolation. You need OpenCV for this. Note that I had to swap ncols and nrows to make it works, didn't figured why.
import numpy as np
import cv2
import math
def blockshaped(arr, r_nbrs, c_nbrs, interp=cv2.INTER_LINEAR):
"""
arr a 2D array, typically an image
r_nbrs numbers of rows
r_cols numbers of cols
"""
arr_h, arr_w = arr.shape
size_w = int( math.floor(arr_w // c_nbrs) * c_nbrs )
size_h = int( math.floor(arr_h // r_nbrs) * r_nbrs )
if size_w != arr_w or size_h != arr_h:
arr = cv2.resize(arr, (size_w, size_h), interpolation=interp)
nrows = int(size_w // r_nbrs)
ncols = int(size_h // c_nbrs)
return (arr.reshape(r_nbrs, ncols, -1, nrows)
.swapaxes(1,2)
.reshape(-1, ncols, nrows))
a = np.random.randint(1, 9, size=(9,9))
out = [np.hsplit(x, 3) for x in np.vsplit(a,3)]
print(a)
print(out)
yields
[[7 6 2 4 4 2 5 2 3]
[2 3 7 6 8 8 2 6 2]
[4 1 3 1 3 8 1 3 7]
[6 1 1 5 7 2 1 5 8]
[8 8 7 6 6 1 8 8 4]
[6 1 8 2 1 4 5 1 8]
[7 3 4 2 5 6 1 2 7]
[4 6 7 5 8 2 8 2 8]
[6 6 5 5 6 1 2 6 4]]
[[array([[7, 6, 2],
[2, 3, 7],
[4, 1, 3]]), array([[4, 4, 2],
[6, 8, 8],
[1, 3, 8]]), array([[5, 2, 3],
[2, 6, 2],
[1, 3, 7]])], [array([[6, 1, 1],
[8, 8, 7],
[6, 1, 8]]), array([[5, 7, 2],
[6, 6, 1],
[2, 1, 4]]), array([[1, 5, 8],
[8, 8, 4],
[5, 1, 8]])], [array([[7, 3, 4],
[4, 6, 7],
[6, 6, 5]]), array([[2, 5, 6],
[5, 8, 2],
[5, 6, 1]]), array([[1, 2, 7],
[8, 2, 8],
[2, 6, 4]])]]
I publish my solution. Notice that this code doesn't' actually create copies of original array, so it works well with big data. Moreover, it doesn't crash if array cannot be divided evenly (but you can easly add condition for that by deleting ceil and checking if v_slices and h_slices are divided without rest).
import numpy as np
from math import ceil
a = np.arange(9).reshape(3, 3)
p, q = 2, 2
width, height = a.shape
v_slices = ceil(width / p)
h_slices = ceil(height / q)
for h in range(h_slices):
for v in range(v_slices):
block = a[h * p : h * p + p, v * q : v * q + q]
# do something with a block
This code changes (or, more precisely, gives you direct access to part of an array) this:
[[0 1 2]
[3 4 5]
[6 7 8]]
Into this:
[[0 1]
[3 4]]
[[2]
[5]]
[[6 7]]
[[8]]
If you need actual copies, Aenaon code is what you are looking for.
If you are sure that big array can be divided evenly, you can use numpy splitting tools.
to add to #Aenaon answer and his blockfy function, if you are working with COLOR IMAGES/ 3D ARRAY here is my pipeline to create crops of 224 x 224 for 3 channel input
def blockfy(a, p, q):
'''
Divides array a into subarrays of size p-by-q
p: block row size
q: block column size
'''
m = a.shape[0] #image row size
n = a.shape[1] #image column size
# pad array with NaNs so it can be divided by p row-wise and by q column-wise
bpr = ((m-1)//p + 1) #blocks per row
bpc = ((n-1)//q + 1) #blocks per column
M = p * bpr
N = q * bpc
A = np.nan* np.ones([M,N])
A[:a.shape[0],:a.shape[1]] = a
block_list = []
previous_row = 0
for row_block in range(bpc):
previous_row = row_block * p
previous_column = 0
for column_block in range(bpr):
previous_column = column_block * q
block = A[previous_row:previous_row+p, previous_column:previous_column+q]
# remove nan columns and nan rows
nan_cols = np.all(np.isnan(block), axis=0)
block = block[:, ~nan_cols]
nan_rows = np.all(np.isnan(block), axis=1)
block = block[~nan_rows, :]
## append
if block.size:
block_list.append(block)
return block_list
then extended above to
for file in os.listdir(path_to_crop): ### list files in your folder
img = io.imread(path_to_crop + file, as_gray=False) ### open image
r = blockfy(img[:,:,0],224,224) ### crop blocks of 224 x 224 for red channel
g = blockfy(img[:,:,1],224,224) ### crop blocks of 224 x 224 for green channel
b = blockfy(img[:,:,2],224,224) ### crop blocks of 224 x 224 for blue channel
for x in range(0,len(r)):
img = np.array((r[x],g[x],b[x])) ### combine each channel into one patch by patch
img = img.astype(np.uint8) ### cast back to proper integers
img_swap = img.swapaxes(0, 2) ### need to swap axes due to the way things were proceesed
img_swap_2 = img_swap.swapaxes(0, 1) ### do it again
Image.fromarray(img_swap_2).save(path_save_crop+str(x)+"bounding" + file,
format = 'jpeg',
subsampling=0,
quality=100) ### save patch with new name etc
Related
Pad list of arrays with zeros in order all arrays to have the same size
I have created this array(or I think its a list) that consist of many arrays that are different size and that is the reason I put dtype = object . m = [data[a:b] for a, b in zip(z[0:-1:2], z[1:-1:2])] array = np.array(m, dtype=object) I need to pad each array with zero so that they have the same size (lets say size=smax) and become a "proper" array. My definitions are a little off and I am sorry in advance
You can do this using np.pad on each row. For example: import numpy as np data = np.arange(10) z = [0, 2, 1, 4, 6, 10, 8, 9] m = [data[a:b] for a, b in zip(z[0:-1:2], z[1:-1:2])] max_length = max(len(row) for row in m) result = np.array([np.pad(row, (0, max_length-len(row))) for row in m]) print(result) # [[0 1 0 0] # [1 2 3 0] # [6 7 8 9]]
Convert c-order index into f-order index in Python
I am trying to find a solution to the following problem. I have an index in C-order and I need to convert it into F-order. To explain simply my problem, here is an example: Let's say we have a matrix x as: x = np.arange(1,5).reshape(2,2) print(x) array([[1, 2], [3, 4]]) Then the flattened matrix in C order is: flat_c = x.ravel() print(flat_c) array([1, 2, 3, 4]) Now, the value 3 is at the 2nd position of the flat_c vector i.e. flat_c[2] is 3. If I would flatten the matrix x using the F-order, I would have: flat_f = x.ravel(order='f') array([1, 3, 2, 4]) Now, the value 3 is at the 1st position of the flat_f vector i.e. flat_f[1] is 3. I am trying to find a way to get the F-order index knowing the dimension of the matrix and the corresponding index in C-order. I tried using np.unravel_index but this function returns the matrix positions...
We can use a combination of np.ravel_multi_index and np.unravel_index for a ndarray supported solution. Hence, given array shape s of input array a and c-order index c_idx, it would be -
s = a.shape
f_idx = np.ravel_multi_index(np.unravel_index(c_idx,s)[::-1],s[::-1])
So, the idea is pretty simple. Use np.unravel_index to get c-based indices in n-dim, then get flattened-linear index in fortran order by using np.ravel_multi_index on flipped shape and those flipped n-dim indices to simulate fortran behavior.
Sample runs on 2D -
In [321]: a
Out[321]:
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14]])
In [322]: s = a.shape
In [323]: c_idx = 6
In [324]: np.ravel_multi_index(np.unravel_index(c_idx,s)[::-1],s[::-1])
Out[324]: 4
In [325]: c_idx = 12
In [326]: np.ravel_multi_index(np.unravel_index(c_idx,s)[::-1],s[::-1])
Out[326]: 8
Sample run on 3D array -
In [336]: a
Out[336]:
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14]],
[[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29]]])
In [337]: s = a.shape
In [338]: c_idx = 21
In [339]: np.ravel_multi_index(np.unravel_index(c_idx,s)[::-1],s[::-1])
Out[339]: 9
In [340]: a.ravel('F')[9]
Out[340]: 21
Suppose your matrix is of shape (nrow,ncol). Then the 1D index when unraveled in C style for the (irow,icol) entry is given by idxc = ncol*irow + icol In the above equation, you know idxc. Then, icol = idxc % ncol Now you can find irow irow = (idxc - icol) / ncol Now you know both irow and icol. You can use them to get the F index. I think the F index will be given by idxf = nrow*icol + irow Please double-check my math, I might have got something wrong... For the 3D case, if your array has dimensions [n1][n2][n3], then the unraveled C-index for [i1][i2][i3] is idxc = n2*n3*i1 + n3*i2+i3 Using modulo operations similar to the 2D case, we can recover i1,i2,i3 and then convert to unraveled F index, i.e. n3*i2 + i3 = idxc % (n2*n3) i3 = (n3*i2+i3) % n3 i2 = ((n3*i2+i3) - i3) /n3 i1 = (idxc-(n3+i2+i3)) /(n2*n3) F index would be: idxf = i1 + n1*i2 +n1*n2*i3 Please check my math.
In simple cases you may also get away with transposing and ravelling the array: import numpy as np x = np.arange(2 * 2).reshape(2, 2) print(x) # [[0 1] # [2 3]] print(x.ravel()) # [0 1 2 3] print(x.transpose().ravel()) # [0 2 1 3] x = np.arange(2 * 3 * 4).reshape(2, 3, 4) print(x) # [[[ 0 1 2 3] # [ 4 5 6 7] # [ 8 9 10 11]] # [[12 13 14 15] # [16 17 18 19] # [20 21 22 23]]] print(x.ravel()) # [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23] print(x.transpose().ravel()) # [ 0 12 4 16 8 20 1 13 5 17 9 21 2 14 6 18 10 22 3 15 7 19 11 23]
Intersect multiple 2D np arrays for determining zones
Using this small reproducible example, I've so far been unable to generate a new integer array from 3 arrays that contains unique groupings across all three input arrays. The arrays are related to topographic properties: import numpy as np asp = np.array([8,1,1,2,7,8,2,3,7,6,4,3,6,5,5,4]).reshape((4,4)) #aspect slp = np.array([9,10,10,9,9,12,12,9,10,11,11,9,9,9,9,9]).reshape((4,4)) #slope elv = np.array([13,14,14,13,14,15,16,14,14,15,16,14,13,14,14,13]).reshape((4,4)) #elevation The idea is that the geographic contours are broken into 3 different properties using GIS routines: 1-8 for aspect (1=north facing, 2=northeast facing, etc.) 9-12 for slope (9=gentle slope...12=steepest slope) 13-16 for elevation (13=lowest elevations...16=highest elevations) The small graphic below attempts to depict the kind of result I'm after (array shown in lower left). Note, the "answer" given in the graphic is but one possible answer. I'm not concerned about the final arrangement of integers in the resulting array so long as the final array contains an integer at each row/column index that identifies unique groupings. For example, the array indexes at [0,1] and [0,2] have the same aspect, slope, and elevation and therefore receive the same integer identifier in the resulting array. Does numpy have a built in routine for this kind of thing?
Each location in the grid is associated with a tuple composed of one value from asp, slp and elv. For example, the upper left corner has tuple (8,9,13). We would like to map this tuple to a number which uniquely identifies this tuple. One way to do that would be to think of (8,9,13) as the index into the 3D array np.arange(9*13*17).reshape(9,13,17). This particular array was chosen to accommodate the largest values in asp, slp and elv: In [107]: asp.max()+1 Out[107]: 9 In [108]: slp.max()+1 Out[108]: 13 In [110]: elv.max()+1 Out[110]: 17 Now we can map the tuple (8,9,13) to the number 1934: In [113]: x = np.arange(9*13*17).reshape(9,13,17) In [114]: x[8,9,13] Out[114]: 1934 If we do this for each location in the grid, then we get a unique number for each location. We could end right here, letting these unique numbers serve as labels. Or, we can generate smaller integer labels (starting at 0 and increasing by 1) by using np.unique with return_inverse=True: uniqs, labels = np.unique(vals, return_inverse=True) labels = labels.reshape(vals.shape) So, for example, import numpy as np asp = np.array([8,1,1,2,7,8,2,3,7,6,4,3,6,5,5,4]).reshape((4,4)) #aspect slp = np.array([9,10,10,9,9,12,12,9,10,11,11,9,9,9,9,9]).reshape((4,4)) #slope elv = np.array([13,14,14,13,14,15,16,14,14,15,16,14,13,14,14,13]).reshape((4,4)) #elevation x = np.arange(9*13*17).reshape(9,13,17) vals = x[asp, slp, elv] uniqs, labels = np.unique(vals, return_inverse=True) labels = labels.reshape(vals.shape) yields array([[11, 0, 0, 1], [ 9, 12, 2, 3], [10, 8, 5, 3], [ 7, 6, 6, 4]]) The above method works fine as long as the values in asp, slp and elv are small integers. If the integers were too large, the product of their maximums could overflow the maximum allowable value one can pass to np.arange. Moreover, generating such a large array would be inefficient. If the values were floats, then they could not be interpreted as indices into the 3D array x. So to address these problems, use np.unique to convert the values in asp, slp and elv to unique integer labels first: indices = [ np.unique(arr, return_inverse=True)[1].reshape(arr.shape) for arr in [asp, slp, elv] ] M = np.array([item.max()+1 for item in indices]) x = np.arange(M.prod()).reshape(M) vals = x[indices] uniqs, labels = np.unique(vals, return_inverse=True) labels = labels.reshape(vals.shape) which yields the same result as shown above, but works even if asp, slp, elv were floats and/or large integers. Finally, we can avoid the generation of np.arange: x = np.arange(M.prod()).reshape(M) vals = x[indices] by computing vals as a product of indices and strides: M = np.r_[1, M[:-1]] strides = M.cumprod() indices = np.stack(indices, axis=-1) vals = (indices * strides).sum(axis=-1) So putting it all together: import numpy as np asp = np.array([8,1,1,2,7,8,2,3,7,6,4,3,6,5,5,4]).reshape((4,4)) #aspect slp = np.array([9,10,10,9,9,12,12,9,10,11,11,9,9,9,9,9]).reshape((4,4)) #slope elv = np.array([13,14,14,13,14,15,16,14,14,15,16,14,13,14,14,13]).reshape((4,4)) #elevation def find_labels(*arrs): indices = [np.unique(arr, return_inverse=True)[1] for arr in arrs] M = np.array([item.max()+1 for item in indices]) M = np.r_[1, M[:-1]] strides = M.cumprod() indices = np.stack(indices, axis=-1) vals = (indices * strides).sum(axis=-1) uniqs, labels = np.unique(vals, return_inverse=True) labels = labels.reshape(arrs[0].shape) return labels print(find_labels(asp, slp, elv)) # [[ 3 7 7 0] # [ 6 10 12 4] # [ 8 9 11 4] # [ 2 5 5 1]]
This can be done using numpy.unique() and then a mapping like: Code: combined = 10000 * asp + 100 * slp + elv unique = dict(((v, i + 1) for i, v in enumerate(np.unique(combined)))) combined_unique = np.vectorize(unique.get)(combined) Test Code: import numpy as np asp = np.array([8, 1, 1, 2, 7, 8, 2, 3, 7, 6, 4, 3, 6, 5, 5, 4]).reshape((4, 4)) # aspect slp = np.array([9, 10, 10, 9, 9, 12, 12, 9, 10, 11, 11, 9, 9, 9, 9, 9]).reshape((4, 4)) # slope elv = np.array([13, 14, 14, 13, 14, 15, 16, 14, 14, 15, 16, 14, 13, 14, 14, 13]).reshape((4, 4)) combined = 10000 * asp + 100 * slp + elv unique = dict(((v, i + 1) for i, v in enumerate(np.unique(combined)))) combined_unique = np.vectorize(unique.get)(combined) print(combined_unique) Results: [[12 1 1 2] [10 13 3 4] [11 9 6 4] [ 8 7 7 5]]
This seems like a similar problem to labeling unique regions in an image. This is a function I've written to do this, though you would first need to concatenate your 3 arrays to 1 3D array. def labelPix(pix): height, width, _ = pix.shape pixRows = numpy.reshape(pix, (height * width, 3)) unique, counts = numpy.unique(pixRows, return_counts = True, axis = 0) unique = [list(elem) for elem in unique] labeledPix = numpy.zeros((height, width), dtype = int) offset = 0 for index, zoneArray in enumerate(unique): index += offset zone = list(zoneArray) zoneArea = (pix == zone).all(-1) elementsArray, numElements = scipy.ndimage.label(zoneArea) elementsArray[elementsArray!=0] += offset labeledPix[elementsArray!=0] = elementsArray[elementsArray!=0] offset += numElements return labeledPix This will label unique 3-value combinations, while also assigning separate labels to zones which have the same 3-value combination, but are not in contact with one another. asp = numpy.array([8,1,1,2,7,8,2,3,7,6,4,3,6,5,5,4]).reshape((4,4)) #aspect slp = numpy.array([9,10,10,9,9,12,12,9,10,11,11,9,9,9,9,9]).reshape((4,4)) #slope elv = numpy.array([13,14,14,13,14,15,16,14,14,15,16,14,13,14,14,13]).reshape((4,4)) #elevation pix = numpy.zeros((4,4,3)) pix[:,:,0] = asp pix[:,:,1] = slp pix[:,:,2] = elv print(labelPix(pix)) returns: [[ 0 1 1 2] [10 12 3 4] [11 9 6 4] [ 8 7 7 5]]
Here's a plain Python technique using itertools.groupby. It requires the input to be 1D lists, but that shouldn't be a major issue. The strategy is to zip the lists together, along with an index number, then sort the resulting columns. We then group identical columns together, ignoring the index number when comparing columns. Then we gather the index numbers from each group, and use them to build the final output list.
from itertools import groupby
def show(label, seq):
print(label, ' '.join(['{:2}'.format(u) for u in seq]))
asp = [8, 1, 1, 2, 7, 8, 2, 3, 7, 6, 4, 3, 6, 5, 5, 4]
slp = [9, 10, 10, 9, 9, 12, 12, 9, 10, 11, 11, 9, 9, 9, 9, 9]
elv = [13, 14, 14, 13, 14, 15, 16, 14, 14, 15, 16, 14, 13, 14, 14, 13]
size = len(asp)
a = sorted(zip(asp, slp, elv, range(size)))
groups = sorted([u[-1] for u in g] for _, g in groupby(a, key=lambda t:t[:-1]))
final = [0] * size
for i, g in enumerate(groups, 1):
for j in g:
final[j] = i
show('asp', asp)
show('slp', slp)
show('elv', elv)
show('out', final)
output
asp 8 1 1 2 7 8 2 3 7 6 4 3 6 5 5 4
slp 9 10 10 9 9 12 12 9 10 11 11 9 9 9 9 9
elv 13 14 14 13 14 15 16 14 14 15 16 14 13 14 14 13
out 1 2 2 3 4 5 6 7 8 9 10 7 11 12 12 13
There's no need to do that second sort, we could just use a plain list comp
groups = [[u[-1] for u in g] for _, g in groupby(a, key=lambda t:t[:-1])]
or generator expression
groups = ([u[-1] for u in g] for _, g in groupby(a, key=lambda t:t[:-1]))
I only did it so that my output matches the output in the question.
Here's one way to solve this problem using a dictionary based lookup. from collections import defaultdict import itertools group_dict = defaultdict(list) idx_count = 0 for a, s, e in np.nditer((asp, slp, elv)): asp_tuple = (a.tolist(), s.tolist(), e.tolist()) if asp_tuple not in group_dict: group_dict[asp_tuple] = [idx_count+1] idx_count += 1 else: group_dict[asp_tuple].append(group_dict[asp_tuple][-1]) list1d = list(itertools.chain(*list(group_dict.values()))) np.array(list1d).reshape(4, 4) # result array([[ 1, 2, 2, 3], [ 4, 5, 6, 7], [ 7, 8, 9, 10], [11, 12, 12, 13]])
Efficently multiply a matrix with itself after offsetting it by one in numpy
I am trying to write a function that takes a matrix A, then offsets it by one, and does element wise matrix multiplication on the shared area. Perhaps an example will help. Suppose I have the matrix:
A = np.array([[1,2,3],[4,5,6],[7,8,9]])
What i'd like returned is:
(1*2) + (4*5) + (7*8) = 78
The following code does it, but inefficently:
import numpy as np
A = np.array([[1,2,3],[4,5,6],[7,8,9]])
Height = A.shape[0]
Width = A.shape[1]
Sum1 = 0
for y in range(0, Height):
for x in range(0,Width-2):
Sum1 = Sum1 + \
A.item(y,x)*A.item(y,x+1)
print("%d * %d"%( A.item(y,x),A.item(y,x+1)))
print(Sum1)
With output:
1 * 2
4 * 5
7 * 8
78
Here is my attempt to write the code more efficently with numpy:
import numpy as np
A = np.array([[1,2,3],[4,5,6],[7,8,9]])
print(np.sum(np.multiply(A[:,0:-1], A[:,1:])))
Unfortunately, this time I get 186. I am at a loss where did I go wrong. i'd love someone to either correcty me or offer another way to implement this.
Thank you.
In this 3 column case, you are just multiplying the 1st 2 columns, and taking the sum: A[:,:2].prod(1).sum() Out[36]: 78 Same as (A[:,0]*A[:,1]).sum() Now just how does that generalize to more columns? In your original loop, you can cut out the row iteration by taking the sum of this list: [A[:,x]*A[:,x+1] for x in range(0,A.shape[1]-2)] Out[40]: [array([ 2, 20, 56])] Your description talks about multiplying the shared area; what direction are you doing the offset? From the calculation it looks like the offset is negative. A[:,:-1] Out[47]: array([[1, 2], [4, 5], [7, 8]]) If that is the offset logic, than I could rewrite my calculation as A[:,:-1].prod(1).sum() which should work for many more columns. =================== Your 2nd try: In [3]: [A[:,:-1],A[:,1:]] Out[3]: [array([[1, 2], [4, 5], [7, 8]]), array([[2, 3], [5, 6], [8, 9]])] In [6]: A[:,:-1]*A[:,1:] Out[6]: array([[ 2, 6], [20, 30], [56, 72]]) In [7]: _.sum() Out[7]: 186 In other words instead of 1*2, you are calculating [1,2]*[2*3]=[2,6]. Nothing wrong with that, if that's you you really intend. The key is being clear about 'offset' and 'overlap'.
Slicing python matrix into quadrants
Suppose I have the following matrix in python: [[1,2,3,4], [5,6,7,8], [9,10,11,12], [13,14,15,16]] I want to slice it into the following matrices (or quadrants/corners): [[1,2], [5,6]] [[3,4], [7,8]] [[9,10], [13,14]] [[11,12], [15,16]] Is this supported with standard slicing operators in python or is it necessary to use an extended library like numpy?
If you are always working with a 4x4 matrix: a = [[1 ,2 , 3, 4], [5 ,6 , 7, 8], [9 ,10,11,12], [13,14,15,16]] top_left = [a[0][:2], a[1][:2]] top_right = [a[0][2:], a[1][2:]] bot_left = [a[2][:2], a[3][:2]] bot_right = [a[2][2:], a[3][2:]] You could also do the same for an arbitrary size matrix: h = len(a) w = len(a[1]) top_left = [a[i][:w // 2] for i in range(h // 2)] top_right = [a[i][w // 2:] for i in range(h // 2)] bot_left = [a[i][:w // 2] for i in range(h // 2, h)] bot_right = [a[i][w // 2:] for i in range(h // 2, h)]
The question is already answered, but I think this solution is more general. It can also be used numpy.split and list comprehension in the following way: import numpy as np A = np.array([[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]) B = [M for SubA in np.split(A,2, axis = 0) for M in np.split(SubA,2, axis = 1)] Getting: >>>[array([[1, 2],[5, 6]]), array([[3, 4],[7, 8]]), array([[ 9, 10],[13, 14]]), array([[11, 12],[15, 16]])] Now if you want to have them assigned into different variables, just: C1,C2,C3,C4 = B Have a look to numpy.split doc. Changing the parameter indices_or_sections you can get a higher number of splits.
>>> a = [[1,2,3,4], [5,6,7,8], [9,10,11,12], [13,14,15,16]] >>> x = map(lambda x:x[:2], a) >>> x [[1, 2], [5, 6], [9, 10], [13, 14]] >>> y = map(lambda x: x[2:], a) >>> y [[3, 4], [7, 8], [11, 12], [15, 16]] >>> x[:2] + y[:2] + x[2:] + y[2:] [[1, 2], [5, 6], [3, 4], [7, 8], [9, 10], [13, 14], [11, 12], [15, 16]]
Although the answers can provide the required solution. These are not applicable for the arrays in different sizes. If you have a NumPy array in size of (6x7), then these methods will not create a solution. I have prepared a solution for myself and want to share it here.
Note: In my solution, there will be overlaps due to the different axis sizes.
I have created this solution to divide an astronomical image into four quadrants. I, then, use these quadrants to calculate the mean and median in an annulus.
import numpy as np
def quadrant_split2d(array):
"""Example function for identifiying the elements of quadrants in an array.
array:
A 2D NumPy array.
Returns:
The quadrants. True for the members of the quadrants, False otherwise.
"""
Ysize = array.shape[0]
Xsize = array.shape[1]
y, x = np.indices((Ysize,Xsize))
if not (Xsize==Ysize)&(Xsize % 2 == 0): print ('There will be overlaps')
sector1=(x<Xsize/2)&(y<Ysize/2)
sector2=(x>Xsize/2-1)&(y<Ysize/2)
sector3=(x<Xsize/2)&(y>Ysize/2-1)
sector4=(x>Xsize/2-1)&(y>Ysize/2-1)
sectors=(sector1,sector2,sector3,sector4)
return sectors
You can test the function with the different type of arrays.
For example:
test = np.arange(42).reshape(6,7)
print ('Original array:\n', test)
sectors = quadrant_split2d(test)
print ('Sectors:')
for ss in sectors: print (test[ss])
This will give us the following sectors:
[ 0 1 2 3 7 8 9 10 14 15 16 17]
[ 3 4 5 6 10 11 12 13 17 18 19 20]
[21 22 23 24 28 29 30 31 35 36 37 38]
[24 25 26 27 31 32 33 34 38 39 40 41]