Write functions resilient to variable dimension array - python

I'm struggling when writing a function that would seemlessly apply to any numpy arrays whatever its dimension.
At one point in my code, I have boolean arrays that I consider as mask for other arrays (0 = not passing, 1 = passing).
I would like to "enlarge" those mask arrays by overriding zeros adjacent to ones on a defined range.
Example :
input = [0,0,0,0,0,1,0,0,0,0,1,0,0,0]
enlarged_by_1 = [0,0,0,0,1,1,1,0,0,1,1,1,0,0]
enlarged_by_2 = [0,0,0,1,1,1,1,1,1,1,1,1,1,0]
input = [[0,0,0,1,0,0,1,0],
[0,1,0,0,0,0,0,0],
[0,0,0,0,0,0,1,0]]
enlarged_by_1 = [[0,0,1,1,1,1,1,1],
[1,1,1,0,0,0,0,0],
[0,0,0,0,0,1,1,1]]
This is pretty straighforward when inputs are 1D.
However, I would like this function to take seemlessy 1D, matrix, 3D, and so on.
So for a matrix, the same logic would be applied to each lines.
I read about ellipsis, but it does not seem to be applicable in my case.
Flattening the input applying the logic and reshaping the array would lead to possible contamination between individual arrays.
I do not want to go through testing the shape of input numpy array / recursive function as it does not seems very clean to me.
Would you have some suggestions ?


The operation that you are described seems very much like a convolution operation followed by clipping to ensure that values remain 0 or 1.
For your example input:
import numpy as np
input = np.array([0,0,0,0,0,1,0,0,0,0,1,0,0,0], dtype=int)
print(input)
def enlarge_ones(x, k):
mask = np.ones(2*k+1, dtype=int)
return np.clip(np.convolve(x, mask, mode='same'), 0, 1).astype(int)
print(enlarge_ones(input, k=1))
print(enlarge_ones(input, k=3))
which yields
[0 0 0 0 0 1 0 0 0 0 1 0 0 0]
[0 0 0 0 1 1 1 0 0 1 1 1 0 0]
[0 0 1 1 1 1 1 1 1 1 1 1 1 1]
numpy.convolve only works for 1-d arrays. However, one can imagine a for loop over the number of array dimensions and another for loop over each array. In other words, for a 2-d matrix first operate on every row and then on every column. You get the idea for nd-array with more dimensions. In other words the enlarge_ones would become something like:
def enlarge_ones(x, k):
n = len(x.shape)
if n == 1:
mask = np.ones(2*k+1, dtype=int)
return np.clip(np.convolve(x, mask, mode='same')[:len(x)], 0, 1).astype(int)
else:
x = x.copy()
for d in range(n):
for i in np.ndindex(x.shape[:-1]):
x[i] = enlarge_ones(x[i], k) # x[i] is 1-d
x = x.transpose(list(range(1, n)) + [0])
return x
Note the use of np.transpose to rotate the dimensions so that np.convolve is applied to the 1-d along each dimension. This is exactly n times, which returns the matrix to original shape at the end.
x = np.zeros((3, 5, 7), dtype=int)
x[1, 2, 2] = 1
print(x)
print(enlarge_ones(x, k=1))
[[[0 0 0 0 0 0 0]
[0 0 0 0 0 0 0]
[0 0 0 0 0 0 0]
[0 0 0 0 0 0 0]
[0 0 0 0 0 0 0]]
[[0 0 0 0 0 0 0]
[0 0 0 0 0 0 0]
[0 0 1 0 0 0 0]
[0 0 0 0 0 0 0]
[0 0 0 0 0 0 0]]
[[0 0 0 0 0 0 0]
[0 0 0 0 0 0 0]
[0 0 0 0 0 0 0]
[0 0 0 0 0 0 0]
[0 0 0 0 0 0 0]]]
[[[0 0 0 0 0 0 0]
[0 1 1 1 0 0 0]
[0 1 1 1 0 0 0]
[0 1 1 1 0 0 0]
[0 0 0 0 0 0 0]]
[[0 0 0 0 0 0 0]
[0 1 1 1 0 0 0]
[0 1 1 1 0 0 0]
[0 1 1 1 0 0 0]
[0 0 0 0 0 0 0]]
[[0 0 0 0 0 0 0]
[0 1 1 1 0 0 0]
[0 1 1 1 0 0 0]
[0 1 1 1 0 0 0]
[0 0 0 0 0 0 0]]]

Related

Fill the secondary diagonal with respect to a point?

The problem is that I have a point, say P = (p1,p2), in a 2x2 numpy array in Python. Now using the point P I want to fill the all the entries in the secondary diagonal passing through with that point.
So what it looks like is:
arr = [0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0
0,0,0,0,0]
P = (1,4)
arr = [0,0,0,0,0
0,0,0,0,1
0,0,0,1,0
0,0,1,0,0
0,1,0,0,0]
or let's say P = (3,0):
arr = [0,0,0,1,0
0,0,1,0,0
0,1,0,0,0
1,0,0,0,0
0,0,0,0,0]
The array with ones is the final result required.

You can slice the array using the indices and feed it to np.fliplr and p.fill_diagonal to get the reversed diagonal
arr = np.zeros(shape=(5, 5), dtype=int)
p = (...)
np.fill_diagonal(np.fliplr(arr[p[0]:, :p[1]+1]), 1)
print(arr)
Output
p = 1, 4
[[0 0 0 0 0]
[0 0 0 0 1]
[0 0 0 1 0]
[0 0 1 0 0]
[0 1 0 0 0]]
p = 0, 3
[[0 0 0 1 0]
[0 0 1 0 0]
[0 1 0 0 0]
[1 0 0 0 0]
[0 0 0 0 0]]

How to resize a Numpy array to add/replace rows with combinations determined by the values in each row of the array

So I have a program that at some point creates random arrays and I have perform an operation which is to add rows while replacing other rows based on the values found in the rows. One of the random arrays will look something like this but keep in mind that it could randomly vary in size ranging from 3x3 up to 10x10:
0 2 0 1
1 0 0 1
1 0 2 1
2 0 1 2
For every row that has at least one value equal to 2 I need to remove/replace the row and add some more rows. The number of rows added will depend on the number of combinations possible of 0s and 1s where the number of digits is equal to the number of 2s counted in each row. Each added row will introduce one of these combinations in the positions where the 2s are located. The result that I'm looking for will look like this:
0 1 0 1 # First combination to replace 0 2 0 1
0 0 0 1 # Second combination to replace 0 2 0 1 (Only 2 combinations, only one 2)
1 0 0 1 # Stays the same
1 0 1 1 # First combination to replace 1 0 2 1
1 0 0 1 # Second combination to replace 1 0 2 1 (Only 2 combinations, only one 2)
0 0 1 0 # First combination to replace 2 0 1 2
0 0 1 1 # Second combination to replace 2 0 1 2
1 0 1 1 # Third combination to replace 2 0 1 2
1 0 1 0 # Fourth combination to replace 2 0 1 2 (4 combinations, there are two 2s)
If you know a Numpy way of accomplishing this I will be grateful.

You can try the following. Create a sample array:
import numpy as np
np.random.seed(5)
a = np.random.randint(0, 3, (4, 4))
print(a)
This gives:
[[2 1 2 2]
[0 1 0 0]
[2 0 2 0]
[0 1 1 0]]
Compute the output array:
ts = (a == 2).sum(axis=1)
r = np.hstack([np.array(np.meshgrid(*[[0, 1]] * t)).reshape(t, -1).T.ravel() for t in ts if t])
out = np.repeat(a, 2**ts, axis=0)
out[out == 2] = r
print(out)
Result:
[[0 1 0 0]
[0 1 0 1]
[1 1 0 0]
[1 1 0 1]
[0 1 1 0]
[0 1 1 1]
[1 1 1 0]
[1 1 1 1]
[0 1 0 0]
[0 0 0 0]
[1 0 0 0]
[0 0 1 0]
[1 0 1 0]
[0 1 1 0]]

Not the prettiest code but it does the job. You could clean up the itertools calls but this lets you see how it works.
import numpy as np
import itertools
X = np.array([[0, 2, 0, 1],
[1, 0, 0, 1],
[1, 0, 2, 1],
[2, 0, 1, 2]])
def add(X_,Y):
if Y.size == 0:
Y = X_
else:
Y = np.vstack((Y, X_))
return(Y)
Y = np.array([])
for i in range(len(X)):
if 2 not in X[i,:]:
Y = add(X[i,:], Y)
else:
a = np.where(X[i,:]==2)[0]
n = [[i for i in itertools.chain([1, 0])] for _ in range(len(a))]
m = list(itertools.product(*n))
for j in range(len(m)):
M = 1 * X[i,:]
u = list(m[j])
for k in range(len(a)):
M[a[k]] = u[k]
Y = add(M, Y)
print(Y)
#[[0 1 0 1]
# [0 0 0 1]
# [1 0 0 1]
# [1 0 1 1]
# [1 0 0 1]
# [1 0 1 1]
# [1 0 1 0]
# [0 0 1 1]
# [0 0 1 0]]

Can i change value of Adjacency Matrix on Python Programming?

I have problem with my code, i cannot print or change value of Adjacency matrix, can you help me sir?
i have code like this
for i in range(len(A.todense())):
for j in A[i].todense()*1:
print(j)
and the output is
[[0 0 0 0 0 1 1 0]]
[[0 0 0 0 0 0 0 1]]
[[0 0 0 0 0 1 0 0]]
[[0 0 0 0 0 1 0 0]]
[[0 0 0 0 0 0 0 1]]
[[1 0 1 1 0 0 1 0]]
[[1 0 0 0 0 1 0 0]]
[[0 1 0 0 1 0 0 0]]
and want to change zero to one, or one to zero, but i cannot print or change with A.todense()[i][j]. Can you help me to change the value of adjacency matrix? Thanks you
top up saldo paypal

You can't change value in Adjancency matrix, so that you have to convert to numpy or other array type that can change value.
One Solution Below
A = np.array(A.data)
for i in range(len(A)):
for j in range(len(A[i])):
if A[i][j] == 0:
A[i][j] = 1
else:
A[i][j] = 0
That will change values and then convert to Adjancency matrix.

counting objects using scikit-image label

My goal is to count objects in a binary array, using Python. I am applying the scikit-image measure.label, to to count objects(should be 1's) in the the array, despite reading the documentation-link, I am getting results that cannot be explained.
a=np.array(np.matrix('0 1 0 0 1;0 1 0 0 0; 0 0 0 0 0;0 0 0 0 1'))
print(a)
img=measure.label(a)
propsa = measure.regionprops(img)
length = len(propsa)
print ('length='+str(length))
for label in propsa:
print (label.centroid)
>>>
[[0 1 0 0 1]
[0 1 0 0 0]
[0 0 0 0 0]
[0 0 0 0 1]]
length=3
(0.5, 1.0)
(0.0, 4.0)
(3.0, 4.0)
When background is selected to be zero,
a=np.array(np.matrix('0 1 0 0 1;0 1 0 0 0; 0 0 0 0 0;0 0 0 0 1'))
print(a)
img=measure.label(a, background=0)
propsa = measure.regionprops(img)
length = len(propsa)
print ('length='+str(length))
for label in propsa:
print (label.centroid)
>>>
[[0 1 0 0 1]
[0 1 0 0 0]
[0 0 0 0 0]
[0 0 0 0 1]]
length=2
(0.0, 4.0)
(3.0, 4.0)
Why is there inconsistent? From my understanding the label function labels "0" as -1 background?!
Following question
It appears that measure.label of the CSV file example counts two objects a large one and its cavity. Sure enough when I inquiry the coordinates of the cavity of img I get a value of 2. Which means the cavity is the second object. Why is an aggregate of zeros counted as an object, and is there away around it?
length=2
(214.23444957510378, 505.25546156532539)
(238.77173913043478, 740.28260869565213)
>>> img[238,740]
2
>>>

For debug purposes it is useful to print the full labeled image. With background=0:
>>> print(img)
[[-1 0 -1 -1 1]
[-1 0 -1 -1 -1]
[-1 -1 -1 -1 -1]
[-1 -1 -1 -1 2]]
The background is correctly labeled as -1. But when you call regionprops on it, it only returns RegionProperties objects for labels 1 and 2 because, as stated in the docs for regionprops(),
label_image : (N, M) ndarray
Labeled input image. Labels with value 0 are ignored.
Therefore the first area that has label 0 is ignored.
When background is not specified, the 0-filled area is has the label 0 and therefore ignored by regionprops(), giving the output of the remaining three 1-filled regions:
>>> print(img)
[[0 1 0 0 2]
[0 1 0 0 0]
[0 0 0 0 0]
[0 0 0 0 3]]

Ok here is simple solution that struck me.
I can simply define background=0, and img=img+1.
The problem was, when label is applied on the matrix with a background=0 the 0 values are changed to -1 and if I have a group of ones they are reduced to 0.Therefore by adding a 1, I adjust the img object to background=0 and any group of numbers that is not 0 will be counted.
Here is an example of what I mean:
import matplotlib
matplotlib.use('Gtk3Agg')
import numpy as np
from skimage import filters, morphology, measure
np.set_printoptions(threshold=np.nan)
a=np.array(np.matrix('0 1 0 0 1;0 1 0 0 0; 0 0 0 0 0;0 0 0 1 1'))
print(a)
img=measure.label(a, background=0)
print('img=')
print (img)
#adjusting +1
img=img+1
print('img+1=')
print (img)
propsa = measure.regionprops(img)
length = len(propsa)
print ('length='+str(length))
for label in propsa:
print (label.centroid)
The code returns the following.
>>>
(4, 5)
[[0 1 0 0 1]
[0 1 0 0 0]
[0 0 0 0 0]
[0 0 0 1 1]]
img=
[[-1 0 -1 -1 1]
[-1 0 -1 -1 -1]
[-1 -1 -1 -1 -1]
[-1 -1 -1 2 2]]
img+1=
[[0 1 0 0 2]
[0 1 0 0 0]
[0 0 0 0 0]
[0 0 0 3 3]]
length=3
(0.5, 1.0)
(0.0, 4.0)
(3.0, 3.5)

bounding box of numpy array

Suppose you have a 2D numpy array with some random values and surrounding zeros.
Example "tilted rectangle":
import numpy as np
from skimage import transform
img1 = np.zeros((100,100))
img1[25:75,25:75] = 1.
img2 = transform.rotate(img1, 45)
Now I want to find the smallest bounding rectangle for all the nonzero data. For example:
a = np.where(img2 != 0)
bbox = img2[np.min(a[0]):np.max(a[0])+1, np.min(a[1]):np.max(a[1])+1]
What would be the fastest way to achieve this result? I am sure there is a better way since the np.where function takes quite a time if I am e.g. using 1000x1000 data sets.
Edit: Should also work in 3D...

You can roughly halve the execution time by using np.any to reduce the rows and columns that contain non-zero values to 1D vectors, rather than finding the indices of all non-zero values using np.where:
def bbox1(img):
a = np.where(img != 0)
bbox = np.min(a[0]), np.max(a[0]), np.min(a[1]), np.max(a[1])
return bbox
def bbox2(img):
rows = np.any(img, axis=1)
cols = np.any(img, axis=0)
rmin, rmax = np.where(rows)[0][[0, -1]]
cmin, cmax = np.where(cols)[0][[0, -1]]
return rmin, rmax, cmin, cmax
Some benchmarks:
%timeit bbox1(img2)
10000 loops, best of 3: 63.5 µs per loop
%timeit bbox2(img2)
10000 loops, best of 3: 37.1 µs per loop
Extending this approach to the 3D case just involves performing the reduction along each pair of axes:
def bbox2_3D(img):
r = np.any(img, axis=(1, 2))
c = np.any(img, axis=(0, 2))
z = np.any(img, axis=(0, 1))
rmin, rmax = np.where(r)[0][[0, -1]]
cmin, cmax = np.where(c)[0][[0, -1]]
zmin, zmax = np.where(z)[0][[0, -1]]
return rmin, rmax, cmin, cmax, zmin, zmax
It's easy to generalize this to N dimensions by using itertools.combinations to iterate over each unique combination of axes to perform the reduction over:
import itertools
def bbox2_ND(img):
N = img.ndim
out = []
for ax in itertools.combinations(reversed(range(N)), N - 1):
nonzero = np.any(img, axis=ax)
out.extend(np.where(nonzero)[0][[0, -1]])
return tuple(out)
If you know the coordinates of the corners of the original bounding box, the angle of rotation, and the centre of rotation, you could get the coordinates of the transformed bounding box corners directly by computing the corresponding affine transformation matrix and dotting it with the input coordinates:
def bbox_rotate(bbox_in, angle, centre):
rmin, rmax, cmin, cmax = bbox_in
# bounding box corners in homogeneous coordinates
xyz_in = np.array(([[cmin, cmin, cmax, cmax],
[rmin, rmax, rmin, rmax],
[ 1, 1, 1, 1]]))
# translate centre to origin
cr, cc = centre
cent2ori = np.eye(3)
cent2ori[:2, 2] = -cr, -cc
# rotate about the origin
theta = np.deg2rad(angle)
rmat = np.eye(3)
rmat[:2, :2] = np.array([[ np.cos(theta),-np.sin(theta)],
[ np.sin(theta), np.cos(theta)]])
# translate from origin back to centre
ori2cent = np.eye(3)
ori2cent[:2, 2] = cr, cc
# combine transformations (rightmost matrix is applied first)
xyz_out = ori2cent.dot(rmat).dot(cent2ori).dot(xyz_in)
r, c = xyz_out[:2]
rmin = int(r.min())
rmax = int(r.max())
cmin = int(c.min())
cmax = int(c.max())
return rmin, rmax, cmin, cmax
This works out to be very slightly faster than using np.any for your small example array:
%timeit bbox_rotate([25, 75, 25, 75], 45, (50, 50))
10000 loops, best of 3: 33 µs per loop
However, since the speed of this method is independent of the size of the input array, it can be quite a lot faster for larger arrays.
Extending the transformation approach to 3D is slightly more complicated, in that the rotation now has three different components (one about the x-axis, one about the y-axis and one about the z-axis), but the basic method is the same:
def bbox_rotate_3d(bbox_in, angle_x, angle_y, angle_z, centre):
rmin, rmax, cmin, cmax, zmin, zmax = bbox_in
# bounding box corners in homogeneous coordinates
xyzu_in = np.array(([[cmin, cmin, cmin, cmin, cmax, cmax, cmax, cmax],
[rmin, rmin, rmax, rmax, rmin, rmin, rmax, rmax],
[zmin, zmax, zmin, zmax, zmin, zmax, zmin, zmax],
[ 1, 1, 1, 1, 1, 1, 1, 1]]))
# translate centre to origin
cr, cc, cz = centre
cent2ori = np.eye(4)
cent2ori[:3, 3] = -cr, -cc -cz
# rotation about the x-axis
theta = np.deg2rad(angle_x)
rmat_x = np.eye(4)
rmat_x[1:3, 1:3] = np.array([[ np.cos(theta),-np.sin(theta)],
[ np.sin(theta), np.cos(theta)]])
# rotation about the y-axis
theta = np.deg2rad(angle_y)
rmat_y = np.eye(4)
rmat_y[[0, 0, 2, 2], [0, 2, 0, 2]] = (
np.cos(theta), np.sin(theta), -np.sin(theta), np.cos(theta))
# rotation about the z-axis
theta = np.deg2rad(angle_z)
rmat_z = np.eye(4)
rmat_z[:2, :2] = np.array([[ np.cos(theta),-np.sin(theta)],
[ np.sin(theta), np.cos(theta)]])
# translate from origin back to centre
ori2cent = np.eye(4)
ori2cent[:3, 3] = cr, cc, cz
# combine transformations (rightmost matrix is applied first)
tform = ori2cent.dot(rmat_z).dot(rmat_y).dot(rmat_x).dot(cent2ori)
xyzu_out = tform.dot(xyzu_in)
r, c, z = xyzu_out[:3]
rmin = int(r.min())
rmax = int(r.max())
cmin = int(c.min())
cmax = int(c.max())
zmin = int(z.min())
zmax = int(z.max())
return rmin, rmax, cmin, cmax, zmin, zmax
I've essentially just modified the function above using the rotation matrix expressions from here - I haven't had time to write a test-case yet, so use with caution.

Here is an algorithm to calculate the bounding box for N dimensional arrays,
def get_bounding_box(x):
""" Calculates the bounding box of a ndarray"""
mask = x == 0
bbox = []
all_axis = np.arange(x.ndim)
for kdim in all_axis:
nk_dim = np.delete(all_axis, kdim)
mask_i = mask.all(axis=tuple(nk_dim))
dmask_i = np.diff(mask_i)
idx_i = np.nonzero(dmask_i)[0]
if len(idx_i) != 2:
raise ValueError('Algorithm failed, {} does not have 2 elements!'.format(idx_i))
bbox.append(slice(idx_i[0]+1, idx_i[1]+1))
return bbox
which can be used with 2D, 3D, etc arrays as follows,
In [1]: print((img2!=0).astype(int))
...: bbox = get_bounding_box(img2)
...: print((img2[bbox]!=0).astype(int))
...:
[[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0]
[0 0 0 0 0 0 1 1 1 1 1 1 1 1 0 0 0 0 0 0]
[0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0]
[0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0]
[0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0]
[0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0]
[0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0]
[0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0]
[0 0 0 0 0 0 1 1 1 1 1 1 1 1 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]]
[[0 0 0 0 0 0 1 1 0 0 0 0 0 0]
[0 0 0 0 0 1 1 1 1 0 0 0 0 0]
[0 0 0 0 1 1 1 1 1 1 0 0 0 0]
[0 0 0 1 1 1 1 1 1 1 1 0 0 0]
[0 0 1 1 1 1 1 1 1 1 1 1 0 0]
[0 1 1 1 1 1 1 1 1 1 1 1 1 0]
[1 1 1 1 1 1 1 1 1 1 1 1 1 1]
[1 1 1 1 1 1 1 1 1 1 1 1 1 1]
[0 1 1 1 1 1 1 1 1 1 1 1 1 0]
[0 0 1 1 1 1 1 1 1 1 1 1 0 0]
[0 0 0 1 1 1 1 1 1 1 1 0 0 0]
[0 0 0 0 1 1 1 1 1 1 0 0 0 0]
[0 0 0 0 0 1 1 1 1 0 0 0 0 0]
[0 0 0 0 0 0 1 1 0 0 0 0 0 0]]
Although replacing the np.diff and np.nonzero calls by one np.where might be better.

I was able to squeeze out a little more performance by replacing np.where with np.argmax and working on a boolean mask.
def bbox(img):
img = (img > 0)
rows = np.any(img, axis=1)
cols = np.any(img, axis=0)
rmin, rmax = np.argmax(rows), img.shape[0] - 1 - np.argmax(np.flipud(rows))
cmin, cmax = np.argmax(cols), img.shape[1] - 1 - np.argmax(np.flipud(cols))
return rmin, rmax, cmin, cmax
This was about 10µs faster for me than the bbox2 solution above on the same benchmark. There should also be a way to just use the result of argmax to find the non-zero rows and columns, avoiding the extra search done by using np.any, but this may require some tricky indexing that I wasn't able to get working efficiently with simple vectorized code.

I know this post is old and has already been answered, but I believe I've identified an optimized approach for large arrays and arrays loaded as np.memmaps.
I was using ali_m's response that was optimized by Allen Zelener for smaller ndarrays, but this approach turns out to be quite slow for np.memmaps.
Below is my implementation that has extremely similar performance speeds to ali_m's approach approach for arrays that fit in the working memory, but that far outperforms when bounding large arrays or np.memmaps.
import numpy as np
from numba import njit, prange
#njit(parallel=True, nogil=True, cache=True)
def bound(volume):
"""
Bounding function to bound large arrays and np.memmaps
volume: A 3D np.array or np.memmap
"""
mins = np.array(volume.shape)
maxes = np.zeros(3)
for z in prange(volume.shape[0]):
for y in range(volume.shape[1]):
for x in range(volume.shape[2]):
if volume[z,y,x]:
if z < mins[0]:
mins[0] = z
elif z > maxes[0]:
maxes[0] = z
if y < mins[1]:
mins[1] = y
elif y > maxes[1]:
maxes[1] = y
if x < mins[2]:
mins[2] = x
elif x > maxes[2]:
maxes[2] = x
return mins, maxes
My approach is somewhat inefficient in the sense that it just iterates over every point rather than flattening the arrays over specific dimensions. However, I found flattening np.memmaps using np.any() with a dimension argument to be quite slow. I tried using numba to speed up the flattening, but it doesn't support np.any() with arguments. As such, I came to my iterative approach that seems to perform quite well.
On my computer (2019 16" MacBook Pro, 6-core i7, 16 GB 2667 MHz DDR4), I'm able to bound a np.memmap with a shape of (1915, 4948, 3227) in ~33 seconds, as opposed to the ali_m approach that takes around ~250 seconds.
Not sure if anyone will ever see this, but hopefully it helps in the niche cases of needing to bound np.memmaps.

Categories