I have a ndarray of shape (68, 64, 64) called 'prediction'. These dimensions correspond to image_number, height, width. For each image, I have a tuple of length two that contains coordinates that corresponds to a particular location in each 64x64 image, for example (12, 45). I can stack these coordinates into another Numpy ndarray of shape (68,2) called 'locations'.
How can I construct a slice object or construct the necessary advanced indexing indices to access these locations without using a loop? Looking for help on the syntax. Using pure Numpy matrixes without loops is the goal.
Working loop structure
Import numpy as np
# example code with just ones...The real arrays have 'real' data.
prediction = np.ones((68,64,64), dtype='float32')
locations = np.ones((68,2), dtype='uint32')
selected_location_values = np.empty(prediction.shape[0], dtype='float32')
for index, (image, coordinates) in enumerate(zip(prediction, locations)):
selected_locations_values[index] = image[coordinates]
Desired approach
selected_location_values = np.empty(prediction.shape[0], dtype='float32')
correct_indexing = some_function_here(locations). # ?????
selected_locations_values = predictions[correct_indexing]
A straightforward indexing should work:
img = np.arange(locations.shape[0])
r = locations[:, 0]
c = locations[:, 1]
selected_locations_values = predictions[img, r, c]
Fancy indexing works by selecting elements of the indexed array that correspond to the shape of the broadcasted indices. In this case, the indices are quite straightforward. You just need the range to tell you what image each location corresponds to.
Related
I'm currently learning about broadcasting in Numpy and in the book I'm reading (Python for Data Analysis by Wes McKinney the author has mentioned the following example to "demean" a two-dimensional array:
import numpy as np
arr = np.random.randn(4, 3)
print(arr.mean(0))
demeaned = arr - arr.mean(0)
print(demeaned)
print(demeand.mean(0))
Which effectively causes the array demeaned to have a mean of 0.
I had the idea to apply this to an image-like, three-dimensional array:
import numpy as np
arr = np.random.randint(0, 256, (400,400,3))
demeaned = arr - arr.mean(2)
Which of course failed, because according to the broadcasting rule, the trailing dimensions have to match, and that's not the case here:
print(arr.shape) # (400, 400, 3)
print(arr.mean(2).shape) # (400, 400)
Now, i have gotten it to work mostly, by substracting the mean from every single index in the third dimension of the array:
demeaned = np.ones(arr.shape)
for i in range(3):
demeaned[...,i] = arr[...,i] - means
print(demeaned.mean(0))
At this point, the returned values are very close to zero and i think, that's a precision error. Am i actually right with this thought or is there another caveat, that i missed?
Also, this doesn't seam to be the cleanest, most 'numpy'-way to achieve what i wanted to achieve. Is there a function or a principle that i can make use of to improve the code?
As of numpy version 1.7.0, np.mean, and several other functions, accept a tuple in their axis parameter. This means that you can perform the operation on the planes of the image all at once:
m = arr.mean(axis=(0, 1))
This mean will have shape (3,), with one element for each plane of the image.
If you want to subtract the means of each pixel individually, you have to remember that broadcasting aligns shape tuples on the right edge. That means that you need to insert an extra dimension:
n = arr.mean(axis=2)
n = n.reshape(*n.shape, 1)
Or
n = arr.mean(axis=2)[..., None]
Try np.apply_along_axis().
np.apply_along_axis(lambda x: x - np.mean(x), 2, arr)
Output: you get the array of the same shape where each cell is demeaned in the dimension you want (the second parameter, here it is 2).
I am analyzing some image represented datasets using keras. I am stuck that I have two different dimensions of images. Please see the snapshot. Features has 14637 images having dimension (10,10,3) and features2 has dimension (10,10,100)
Is there any way that I can merge/concatenate these two data together.?
If features and features2 contain the features of the same batch of images, that is features[i] is the same image of features2[i] for each i, then it would make sense to group the features in a single array using the numpy function concatenate():
newArray = np.concatenate((features, features2), axis=3)
Where 3 is the axis along which the arrays will be concatenated. In this case, you'll end up with a new array having dimension (14637, 10, 10, 103).
However, if they refer to completely different batches of images and you would like to merge them on the first axis such that the 14637 images of features2 are placed after the first 14637 image, then, there no way you can end up with an array, since numpy array are structured as matrix, non as a list of objects.
For instance, if you try to execute:
> a = np.array([[0, 1, 2]]) // shape = (1, 3)
> b = np.array([[0, 1]]) // shape = (1, 2)
> c = np.concatenate((a, b), axis=0)
Then, you'll get:
ValueError: all the input array dimensions except for the concatenation axis must match exactly
since you are concatenating along axis = 0 but axis 1's dimensions differ.
If dealing with numpy arrays, you should be able to use concatenate method and specify the axis, along which the data should be merged. Basically: np.concatenate((array_a, array_b), axis=2)
I think it would be better if you use class.
class your_class:
array_1 = []
array_2 = []
final_array = []
for x in range(len(your_previous_one_array)):
temp_class = your_class
temp_class.array_1 = your_previous_one_array
temp_class.array_2 = your_previous_two_array
final_array.append(temp_class)
I am working on a segmentation problem where given an image, each RGB value corresponds to a class label. The problem I have is to efficiently map RGB values from an image (numpy array) to a corresponding class label image.
Let's provide the following simplified example:
color2IdMap
{(100,0,100):0, (0,200,0):2}
labelOld
array([[[100,0,100],
[0,200,0]],
[[100,0,100],
[0,200,0]]], dtype=uint8)
(in a real example the colorIdMap will have about 20 entries and labelOld will be an array of shape: (1024,512,3))
Now I want the result to be the following mapped array. with shape: (1024,512)
labelNew
array([[ 0, 2],
[ 0, 2]])
I attempted to do this with loops and list comprehensions but both methods are quite slow (about ~10seconds per image, which is a big number for 250K images). And I am wondering if there is a faster way of doing it.
Attempted method 1:
labelNew = np.empty((1052,1914), dtype=np.uint8)
for i in range(1052):
for j in range(1914):
labelNew[i, j] = color2IdMap[tuple(labelOld[i, j])]
Attempted method 2:
labelNew = [[color2IdMap[tuple(x)] for x in y] for y in labelOld]
So, my question is if there is any faster and more efficient way of doing this?
Here's one approach based on dimensionality-reduction -
# Get keys and values
k = np.array(list(color2IdMap.keys()))
v = np.array(list(color2IdMap.values()))
# Setup scale array for dimensionality reduction
s = 256**np.arange(3)
# Reduce k to 1D
k1D = k.dot(s)
# Get sorted k1D and correspondingly re-arrange the values array
sidx = k1D.argsort()
k1Ds = k1D[sidx]
vs = v[sidx]
# Reduce image to 2D
labelOld2D = np.tensordot(labelOld, s, axes=((-1),(-1)))
# Get the positions of 1D sorted keys and get the correspinding values by
# indexing into re-arranged values array
out = vs[np.searchsorted(k1Ds, labelOld2D)]
Alternatively, we could use sidx as sorter input arg for np.searchsorted to get the final output -
out = v[sidx[np.searchsorted(k1D, labelOld2D, sorter=sidx)]]
Assume the RGB value map is like this(store in a Python dict):
color_dict = {(128,128,128):(255,255,255),
(128,256,128):(255,128,255),
}
The RGB value remap operation can be done using np.where() and np.all():
cvt_img = np.zeros_like(img)
for rgb_src, rgb_dst in color_dict.items():
rgb_src = np.array(rgb_src)
rgb_dst = np.array(rgb_dst)
idx = np.where(np.all(img == rgb_src, axis=-1))
cvt_img[idx] = rgb_dst
What is the best way to transform an 1D array that contains rgb data into a 3D RGB array ?
If the array was in this order, it would be easy, (a single reshape)
RGB RGB RGB RGB...
However my array is in the form,
RRRR...GGGG....BBBB
or sometimes even,
GGGG....RRRR....BBBB (result still should be RGB not GRB)
I could of course derive some Python way to achieve this, I even did try a numpy solution, it works but It is obviously a bad solution, I wonder what is the best way, maybe a built-in numpy function ?
My solution:
for i in range(len(video_string) // 921600 - 1): # Consecutive frames iterated over.
frame = video_string[921600 * i: 921600 * (i + 1)] # One frame
array = numpy.fromstring(frame, dtype=numpy.uint8) # Numpy array from one frame.
r = array[:307200].reshape(480, 640)
g = array[307200:614400].reshape(480, 640)
b = array[614400:].reshape(480, 640)
rgb = numpy.dstack((b, r, g)) # Bring them together as 3rd dimention
Don't let the for loop confuse you, I just have frames concatenated to each other in a string, like a video, which is not a part of the question.
What did not help me: In this question, r, g, b values are already 2d arrays so not helping my situation.
Edit1: Desired array shape is 640 x 480 x 3
Reshape to 2D, transpose and then reshape back to 3D for RRRR...GGGG....BBBB form -
a1D.reshape(3,-1).T.reshape(height,-1,3) # assuming height is given
Or use reshape with Fortran order and then swap axes -
a1D.reshape(-1,height,3,order='F').swapaxes(0,1)
Sample run -
In [146]: np.random.seed(0)
In [147]: a = np.random.randint(11,99,(4,2,3)) # original rgb image
In [148]: a1D = np.ravel([a[...,0].ravel(), a[...,1].ravel(), a[...,2].ravel()])
In [149]: height = 4
In [150]: np.allclose(a, a1D.reshape(3,-1).T.reshape(height,-1,3))
Out[150]: True
In [151]: np.allclose(a, a1D.reshape(-1,height,3,order='F').swapaxes(0,1))
Out[151]: True
For GGGG....RRRR....BBBB form, simply append : [...,[1,0,2]].
I have a function in Python that returns a numpy.mat of shape (100, 1). I am calling this function 4 times in a loop and would like to take the resulting 4 matricies and create a matrix of shape (100, 4). I have looked for sometime at numpy.append, numpy.concatenate, and numpy.insert but have not been able to get this working.
Here is a short SSCCE of my issue
zeros = np.zeros(shape=(100, 4))
for i in range(1, 5):
np.append(zeros, np.empty(shape=(100, 1)))
print(zeros)
Where zeros should results in a matrix of shape (100, 4) with "junk" values from each of the calls to numpy.empty and not all 0..
Do something along these lines -
zeros = np.zeros(shape=(100, 4))
for i in range(1, 5):
data = np.random.rand(100,1) # func that returns (100,1) shaped array
zeros[:,i-1] = data.ravel()
In place of ravel(), we could also use : data[:,0] or np.squeeze(data), basic idea is to feed a 1D array there, because the LHS zeros[:,i-1] expects a 1D array there.
As an alternative, inside the loop, we could also do -
zeros[:,[i-1]] = data
Thus, with that list of column index [i-1] instead of i-1, we are keeping the dimensions into which data is to be assigned (keeps as 2D) and that allows us to feed in data, which is also 2D without any change.