Somewhat Randomly create 3D points given 2 images
The goal is to create a set of n 3D coordinates (seeds) from 2 images. n could be any where from 100 - 1000 points.
I have 2 pure black and white images whose heights are the same and the widths variable. The size of the images can be as big as 1000x1000 pixels. I read them into numpy arrays and flattened the rgb codes to 1's (black) and zeros (white).
Here is example from processing 2 very small images:
In [6]: img1
Out[6]:
array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 0, 0, 0, 0, 0, 0, 1, 1],
[1, 1, 0, 0, 0, 0, 0, 0, 1, 1],
[1, 1, 0, 0, 0, 0, 0, 0, 1, 1],
[1, 1, 0, 0, 0, 0, 0, 0, 1, 1],
[1, 1, 0, 0, 0, 0, 0, 0, 1, 1],
[1, 1, 0, 0, 0, 0, 0, 0, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1]], dtype=uint8)
In [8]: img2
Out[8]:
array([[0, 0, 0, 0, 0, 0, 1, 0, 0, 0],
[0, 0, 0, 0, 1, 1, 1, 0, 0, 0],
[0, 0, 0, 1, 1, 0, 0, 1, 0, 0],
[0, 0, 1, 1, 0, 0, 0, 1, 0, 0],
[1, 1, 0, 0, 0, 0, 0, 0, 1, 1],
[1, 1, 0, 0, 0, 0, 0, 0, 1, 1],
[0, 0, 1, 0, 0, 0, 1, 1, 0, 0],
[0, 0, 1, 0, 0, 1, 1, 0, 0, 0],
[0, 0, 0, 1, 1, 1, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0, 0, 0, 0, 0]], dtype=uint8)
Next, I create an index array to map all locations of black pixels for each image like so:
In [10]: np.transpose(np.nonzero(img1))
Out[10]:
array([[0, 0],
[0, 1],
[0, 2],
[0, 3],
[0, 4],
[0, 5],
[0, 6],
...
I then want to extend each 2D black pixel for each image into 3D space. Where those 3D points intersect, I want to randomly grab n number of 3D ponts (seeds). Furthermore, as an enhancement, it would be even better if I could disperse these 3d points somewhat evenly in the 3d space to avoid 'clustering' of points where there are areas of greater black pixel density. But I haven't been able to wrap my head around that process yet.
Here's a visualization of the set up:
What I've tried below seems to work on very small images but slows to a halt as the images get bigger. The bottleneck seems to occur where I assign common_points.
img1_array = process_image("Images/nhx.jpg", nheight)
img2_array = process_image("Images/ku.jpg", nheight)
img1_black = get_black_pixels(img1_array)
img2_black = get_black_pixels(img2_array)
# create all img1 3D points:
img1_3d = []
z1 = len(img2_array[1]) # number of img2 columns
for pixel in img1_black:
for i in range(z1):
img1_3d.append((pixel[0], pixel[1], i)) # (img1_row, img1_col, img2_col)
# create all img2 3D points:
img2_3d = []
z2 = len(img1_array[1]) # number of img1 columns
for pixel in img2_black:
for i in range(z2):
img2_3d.append((pixel[0], pixel[1], i)) # (img2_row, img2_col, img1_col)
# get all common 3D points
common_points = [x for x in img1_3d if x in img2_3d]
# get num_seeds number of random common_points
seed_indices = np.random.choice(len(common_points), num_seeds, replace=False)
seeds = []
for index_num in seed_indices:
seeds.append(common_points[index_num])
Questions:
How can I avoid the bottleneck? I haven't been able to come up with a numpy solution.
Is there a better solution, in general, to how I am coding this?
Any thoughts on how I could somewhat evenly disperse seeds?
Update Edit:
Based on Luke's algorithm, I've come up with the following working code. Is this the correct implementation? Could this be improved upon?
img1_array = process_image("Images/John.JPG", 500)
img2_array = process_image("Images/Ryan.jpg", 500)
img1_black = get_black_pixels(img1_array)
# img2_black = get_black_pixels(img2_array)
density = 0.00001
seeds = []
for img1_pixel in img1_black:
row = img1_pixel[0]
img2_row = np.array(np.nonzero(img2_array[row])) # array of column numbers where there is a black pixel
if np.any(img2_row):
for img2_col in img2_row[0]:
if np.random.uniform(0, 1) < density:
seeds.append([row, img1_pixel[1], img2_col])
The bottleneck is because you're comparing every 3D point in the "apple" shaded area to every 3D point in the "orange" shaded area, which is a huge number of comparisons. You could speed it up by a factor of imgHeight by only looking at points in the same row. You could also speed it up by storing img2_3d as a set instead of a list, because calling "in" on a set is much faster (it's an O(1) operation instead of an O(n) operation).
However, it's better to completely avoid making lists of all 3D points. Here's one solution:
Choose an arbitrary density parameter, call it Density. Try Density = 0.10 to fill in 10% of the intersection points.
For each black pixel in Apple, loop through the black pixels in the same row of Orange. If (random.uniform(0,1) < Density), create a 3D point at (applex, orangex, row) or whatever the correct arrangement is for your coordinate system.
That algorithm will sample evenly, so 3D areas with more black will have more samples. If I understand your last question, you want to sample more densely in areas with less black (though I'm not sure why). To do that you could:
Do a Gaussian blur of the inverse of your two images (OpenCV has functions for this), and multiply each times 0.9 and add 0.1. You now have an image that has a higher value where the image is more white.
Do the algorithm above, but for each pixel pair in step 2, set Density = blurredOrangePixel * blurredApplePixel. Thus, your selection density will be higher in white regions.
I would try the basic algorithm first though; I think it will look better.
Related
I have a quick question regarding rigid transformation between two 2D numpy arrays. I have tried several methods from opencv but none return interesting result and I guess that my problem is not too complicated, so maybe I am looking in the wrong direction and I will need your precious help.
So I have two 2D numpy arrays of the same size filled with 0 and 1, like this one:
[[0, 0, 0, 1, 0, 0, 0, 1, 0],
[0, 1, 0, 0, 0, 1, 0, 0, 1],
[1, 0, 0, 1, 0, 0, 1, 0, 0],
[0, 0, 1, 0, 0, 0, 0, 1, 0]]
When there is a 1 it means that I have a point at location (x,y) and 0, means there is nothing.
So at least, I can consider this matrix as a cloud of points that can be drawn in a graph.
I have a second array with same size as the previous one but where the 1 elements have been translated in one direction (all the 1 elements are translated in the same direction and with egal number of translations). It means that some of the 1 element will be out of the array, while some other 1 elements will appear in the free space leaves by the translation, for example second matrix can look like this :
[[1, 0, 1, 0, 1, 0, 1, 0, 1],
[[0, 0, 0, 1, 0, 0, 0, 1, 0],
[0, 1, 0, 0, 0, 1, 0, 0, 1],
[1, 0, 0, 1, 0, 0, 1, 0, 0]]
So first matrix has been translated down of 1 row. First row is new and the three rows below are common in the two matrix. The last row disappears in the second matrix because of the translation. Translation can be in any direction, but it is a rigid transformation (keep distance between points).
Is there a clever method to estimate the best warp matrix between this two arrays ?
Thanks a lot for your help
How do I isolate the bars with no white fill, in the image below that represents a bar chart?
I'm looking for a solution which will work for any variation of this image. You can assume that the format will be the same, but some features like the gaps in the gridlines/axis line might be in different places.
I've tried detecting various features of the bars, like the 26x3px ends of the bars, or the top-left and bottom-right corners. For example, using masks like the following for top-left:
bar_top_kernel: Numpy = np.array([
[1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 1, 1, 1, 1, 1, 1, 1],
[1, 1, 0, 0, 0, 0, 0, 0, 0],
], dtype='uint8')
or
bar_top_kernel: Numpy = np.array([
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
[0, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[0, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[0, 1, 1, 1, 1, 1, 1, 1, 1, 1],
[0, 1, 1, 0, 0, 0, 0, 0, 0, 0],
], dtype='uint8')
But depending on what I try, I either get missed corners or false positives because of how the ends of the bars interact with the gridlines.
I've tried removing the gridlines first. But due to the interaction of the bar ends and the gridlines, I tend to get pieces left over which interfere with the feature detection.
I'm starting to think this might not be possible without some kind of ML approach, but I'm hoping someone will spot a clever trick to achieve this.
(please click to see full size)
I ended up figuring out a simple trick:
Invert the colours.
Find and fill any contour with: bounding box width > bar width. This basically isolates the bars.
Find contours again and find pairs of contours whose centres of mass are close together. This gives the bars small bars that are split in 2 by the tick marks.
Fill rectangle that encompasses each pair to fix the splits.
Given a matrix containing a polygon mask (here a small and simplistic case):
array([[0, 0, 0, 0, 0, 0, 0],
[0, 0, 1, 1, 0, 0, 0],
[0, 0, 1, 1, 1, 0, 0],
[0, 1, 1, 1, 1, 1, 0],
[0, 1, 1, 1, 1, 1, 0],
[0, 0, 1, 1, 1, 0, 0],
[0, 0, 0, 0, 0, 0, 0]])
The outline is extracted with skimage.segmentation.find_boundaries(), giving:
array([[0, 0, 0, 0, 0, 0, 0],
[0, 0, 1, 1, 0, 0, 0],
[0, 0, 1, 0, 1, 0, 0],
[0, 1, 0, 0, 0, 1, 0],
[0, 1, 0, 0, 0, 1, 0],
[0, 0, 1, 1, 1, 0, 0],
[0, 0, 0, 0, 0, 0, 0]])
The outline's [row,column] (i.e. [y,x]) coordinates are then extracted giving:
outline = array([[2,2],[1,2],[1,3],[2,4],[3,5],[4,5],[5,4],[5,3],[5,2],[4,1],[3,1]])
These coordinates are then pruned to a minimal set that define the polygon (i.e. the vertices), giving:
vertices = array([[2,2],[1,2],[1,3],[3,5],[4,5],[5,4],[5,2],[4,1],[3,1]])
(Which corresponds to:)
array([[0, 0, 0, 0, 0, 0, 0],
[0, 0, 1, 1, 0, 0, 0],
[0, 0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 1, 0],
[0, 1, 0, 0, 0, 1, 0],
[0, 0, 1, 0, 1, 0, 0],
[0, 0, 0, 0, 0, 0, 0]])
Is there a fast way using numpy/scipy/skimage/etc to get the outline coordinates (the array outline above) given the vertex coordinates (the array vertices above)?
Further, after getting back the outline coordinates, is there a good numpy/scipy/skimage way to get back the coordinates of all points in the original polygon mask?
Given 2 vertices in a polygon v1, v2 we can get all the points p which are part of the line from v1 to v2 using a line rasterization algorithm. A very fast algorithm for this is Bresenham's line drawing algorithm. After this you can apply this algorithm for each pair of adject vertices in the polygon. Although, I can not guarantee that the outline will be exactly the one in the original polygon since the rasterization algorithm will give the best set of points for the given line, not the one you have in the original algorithm (consider them compression errors).
For the filling algorithm, they are called polygon rasterization algorithm, but I can't help you here since I don't know which are best/fastest.
I have a sparse matrix stored on disk in coordinate format, (triplet format).
I would like to read chunks of the matrix into memory, using scipy.sparse, however, when doing this, scipy will always assume a dense matrix indexing from 0,0, regardless of the chunk.
This means, for example, that for the last 'chunk' in the sparse matrix scipy will interpret as being a huge matrix that only has some values in the bottom right corner.
How can I correctly handle the chunks so that when doing toarray to create a dense matrix it only creates the subset corresponding to that chunk?
The reason for doing this is that, even sparse, the matrix is too large for memory (approx 600 million 32bit floating point values) and to display on screen (as the matrix represents a geospatial raster) I need to convert it to a dense matrix to store in a geospatial format (e.g. geotiff).
You should be able tweak the row and col values when building the subset. For example:
In [84]: row=np.arange(10)
In [85]: col=np.random.randint(0,6,row.shape)
In [86]: data=np.ones(row.shape,dtype=int)*2
In [87]: M=sparse.coo_matrix((data,(row,col)),shape=(10,6))
In [88]: M.A
Out[88]:
array([[0, 0, 2, 0, 0, 0],
[0, 0, 0, 0, 0, 2],
[0, 0, 0, 2, 0, 0],
[0, 0, 2, 0, 0, 0],
[0, 0, 2, 0, 0, 0],
[0, 2, 0, 0, 0, 0],
[2, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 2, 0],
[0, 0, 0, 2, 0, 0],
[0, 0, 0, 0, 0, 2]])
To build a matrix with a subset of the rows use:
In [89]: M1=sparse.coo_matrix((data[5:],(row[5:]-5,col[5:])),shape=(5,6))
In [90]: M1.A
Out[90]:
array([[0, 2, 0, 0, 0, 0],
[2, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 2, 0],
[0, 0, 0, 2, 0, 0],
[0, 0, 0, 0, 0, 2]])
You'll have to decide whether you want to specify the shape for M1, or let it deduce it from the range of row and col.
If these coordinates are not sorted, or you also want to take a subrange of col, things could get more complicated. But I think this captures the basic idea.
following on from " Adding different sized/shaped displaced NumPy matrices ", I want to extend the code, so that I can do the addition at the center
import numpy as np
#two 3d arrays, of different size.
# b1 is variable in size, for the purposes of illustration, lets make it 5x5x5
b1 = np.zeros((5,5,5), dtype=np.int)
# b2 is a fixed size matrix in practice of 10x10x10, but for purposes of illustration,
# its 3x3x3
b2 = np.ones((3,3,3), dtype=np.int)
# want to add b1 to b2, but at a specific location.
# Here, for the purposes of illustration I use the location 2x2x2 in b1 so that the added values fit in the
# lower back corner of b1.
pos_v, pos_h, pos_z = 2, 2, 2 # the 3d coordinates. I want b2 to be centered at 3,3,3 in b1, so this is a bit of a hack to make it fit
v_range1 = slice(max(0, pos_v), max(min(pos_v + b2.shape[0], b1.shape[0]), 0))
h_range1 = slice(max(0, pos_h), max(min(pos_h + b2.shape[1], b1.shape[1]), 0))
z_range1 = slice(max(0, pos_z), max(min(pos_z + b2.shape[2], b1.shape[2]), 0))
v_range2 = slice(max(0, -pos_v), min(-pos_v + b1.shape[0], b2.shape[0]))
h_range2 = slice(max(0, -pos_h), min(-pos_h + b1.shape[1], b2.shape[1]))
z_range2 = slice(max(0, -pos_z), min(-pos_z + b1.shape[2], b2.shape[2]))
b1[v_range1, h_range1, z_range1] += b2[v_range2, h_range2, z_range2]
this gives a result for b1 as
>> b1
array([[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]],
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]],
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 1, 1, 1],
[0, 0, 1, 1, 1],
[0, 0, 1, 1, 1]],
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 1, 1, 1],
[0, 0, 1, 1, 1],
[0, 0, 1, 1, 1]],
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 1, 1, 1],
[0, 0, 1, 1, 1],
[0, 0, 1, 1, 1]]])
but ideally, I'd like to do the addition so that the center of b2 (the 3x3x3 matrix) is placed at the center of the actual target 3,3,3. Rather than the corner of the matrix at 2,2,2. The result in this example will be the same (because I artificially moved the target which is 3,3,3 to 2,2,2 so that it will hit the same position), but on the actual code I use it on (where the data within the matrix is not aligned to the matrix axes, it is oblique), the result would be quite different if I did a similar hack.
In practice, my "b1" matrix will be of variable size, but much larger than 5x5x5, and "b2" will be 10x10x10 or larger. I have the precise location coordinates in my target matrix (b1) which is the center of where I want the center of data from b2 to be stamped (basically I'm working on 3d volume image data, and I want to put a sphere at a coordinate in a 3d volume). As I said, the data within the matrix is not aligned to the matrix axes, so modifying the target coordinate position by offsetting it for the size of b2 isn't the greatest of ideas (but one I considered). This might be being pedantic, but I would prefer getting it right from the start.
Can anyone offer any advice?