I have a numpy array ys_big_seg which has the following shape: (146, 128, 128). It contains pixel masks which values can be 0 or 1. 1 if the pixel is in a given category otherwise 0. I have to scale it to binary mask. So I want to iterate through the (128, 128) matrices and split it to (8, 8) matrices and then based on the smaller matrices values (if every element is 0 then 0, if every element is 1 then 1, if there are mixed values then randomly 0 or 1) substitute these smaller matrices with given values to reduce the (128, 128) matrices to (16, 16).
How can I solve this problem?
I hope it makes sense, sorry for my English.
I think this does what you're looking for:
>>> x.shape
(146, 128, 128)
>>> mask = x.reshape(-1, 16, 16, 8, 8).sum(axis=(3, 4)) >= 32
>>> mask.shape
(146, 16, 16)
Any 8x8 block with a mixture of 0s and 1s will result in a 1 if the total sum is >= 32 (i.e., half or more of the values values are 1), so it's not quite randomly chosen.
Obviously, a sum of 0 (all elements in an 8x8 block are 0) will "fail" that criteria and be 0, and a sum of 64 (all elements in an 8x8 block are 1) will "pass" and end up as a 1. If your matrices are a lot more sparse, you could lower the threshold from 32.
Since you're using this array as a mask, you can leave the 1s and 0s as their boolean counterparts. But if you plan to use the mask as a binary array, then you can easily add .astype(int).
Related
I am trying to mask an array (called dataset) in python:
The array has the following size (5032, 48, 48). Basically these are 5032 48x48 images. But some of the images may not contain any data, so there might only be 0's there. These are the ones I want to mask.
I tried the following: (dataset[:] == 0).all(axis=0).
When I print the shape of the above operation I get (5032, 48) which is not what I want. I expected (5032, ).
I am not sure what I am doing wrong.
I wanted to create a mask with the size (5032, ) which has True (if there is at least one value in the 48x48 array that is nonzero) and False (if there are only zero values in the 48x48 array) values.
Thanks for your help
Kind of a hacky way, but just sum across the last two axis and check if the sum is zero.
nonzero_images = images[np.where(np.sum(images, axis = (1, 2)) == 0)]
You can try something like
# sample data - 3 nonzeros and 2 zeros
dataset = np.concatenate([np.ones((3, 48, 48)), np.zeros((2, 48, 48))])
new = dataset[np.unique(np.where(dataset.all(axis=1))[0])]
print(f'Dataset Shape: {dataset.shape}\nNew Shape: {new.shape}')
# Dataset Shape: (5, 48, 48)
# New Shape: (3, 48, 48)
I have a numpy array arr_seg_labs which has the following shape: (1735, 128, 128).
It contains pixel masks between 1 and 10 and also contains zeros and 255 (background).
I want to remove those (128, 128) matrices which not contain the given category identifier (9) and to keep those which contain at least one 9.
I made a mask (horse_mask) for this, but I don't know how can I continue this thread to filter this numpy array
CAT_IDX_HORSE = 9
horse_mask = arr_seg_labs == CAT_IDX_HORSE
IIUC you can use masks and indexing as:
CAT_IDX_HORSE = 9
mask = (a == CAT_IDX_HORSE ).sum((1, 2))
result = a[mask != 0]
I have a 512x512 image array and I want to perform operations on 8x8 blocks. At the moment I have something like this:
output = np.zeros(512, 512)
for i in range(0, 512, 8):
for j in rangerange(0, 512, 8):
a = input[i:i+8, j:j+8]
b = some_other_array[i:i+8, j:j+8]
output[i:i+8, j:j+8] = np.dot(a, b)
where a & b are 8x8 blocks derived from the original array. I would like to speed up this code by using vectorised operations. I have reshaped my inputs like this:
input = input.reshape(64, 8, 64, 8)
some_other_array = some_other_array.reshape(64, 8, 64, 8)
How could I perform a dot product on only axes 1 & 3 to output an array of shape (64, 8, 64, 8)?
I have tried np.tensordot(input, some_other_array, axes=([0, 1], [2, 3])) which gives the correct output shape, but the values do not match the output from the loop above. I've also looked at np.einsum but I haven't come across a simple example with what I'm trying to achieve.
As you suspected, np.einsum can take care of this. If input and some_other_array have shapes (64, 8, 64, 8), then if you write
output = np.einsum('ijkl,ilkm->ijkm', input, some_other_array)
then output will also have shape (64, 8, 64, 8), where matrix multiplication (i.e. np.dot) has been done only on axes 1 and 3.
The string argument to np.einsum looks complicated, but really it's a combination of two things. First, matrix multiplication is given by jl,lm->jm (see e.g. this answer on einsum). Second, we don't want to do anything to axis 0 and 2, so for them I just write ik,ik->ik. Combining the two gives ijkl,ilkm->ijkm.
They'll work if you reorder them a bit. If input and some_other_array are both shaped (64,8,64,8), then:
input = input.transpose(0,2,1,3)
some_other_array = some_other_array.transpose(0,2,1,3)
This will reorder them to 64,64,8,8. At this point you can compute a matrix multiplication. Do note that you need matmul to compute the block products, and not dot, which will try to multiply the entire matrices.
output = input # some_other_array
output = output.transpose(0,2,1,3)
output = output.reshape(512,512)
I'm trying to pad a 4D numpy array. This array is just a collection of 3D arrays, for example, its size can be interpreted as (samples, height, width, depth). The array size before padding is (1682160, 21, 200, 3). I would like to pad the rows in this array by 10 pixels in each side, hence my final padded array size would be (1682160, 21, 220, 3). I currently use the numpy pad function this way:
matrix = np.pad(matrix, ((0, 0), (0, 0), (10, 10), (0,0)), mode='reflect')
I rewrite the padded array matrix into the same variable since it is float16 datatype, with a size of roughly 40 GB. Writing into a new matrix results in a memory error. The execution time for the above command is roughly 700 seconds.
I would like to know if there is a faster way of implementing this in python, maybe in a vectorized fashion?
How can I eliminate a dummy dimension in python numpy ndarray?
For example, suppose that A.shape = (0, 1325, 3),
then how can eliminate '0' dimension so that A.shape = (1325,3).
Both 'np.sqeeze(A)' or 'A.reshape(A.shape[1:])' don't work.
You can't eliminate that 0 dimension. A dimension of length 0 is not a "dummy" dimension. It really means length 0. Since the total number of elements in the array (which you can check with a.size) is the product of the shape attribute, an array with shape (0, 1325, 3) contains 0 elements, while an array with shape (1325, 3) contains 3975 elements. If there was a way to eliminate the 0 dimension, where would that data come from?
If your array is supposed to contain data, then you probably need to look at how that array was created in the first place.