TLDR: I am looking to fit a triangle to a boomerang-shaped object in order to detect its "head", potentially using python's opencv.
I have a collection of boomerang-shape objects (see image below), whose size and internal angles vary. Additionally, sometimes the "boomerangs" (unlike a real boomerang) can be asymmetric with one leg longer than the other, and can have defects and holes along their legs.
I can accurately extract the contours of these shapes, and now am trying to detect the direction the boomerang is facing (defined as the direction the "pointy" edge, the one marked by brown dots in the image below).
My plan so far was to use opencv's convex defects method to detect the internal angle, and from there detect the direction. However, my "boomerangs" are not perfect - they sometimes have holes and defects along their legs that confuse the convex defect algorithm.
My question is: is there a way to find the best-fit triangle (much like the best fit ellipse) that would fit the boomerang?
Related
I have a task of identifying the number of bacterial colonies on a relatively diverse set of top-down photos of a Petri dish located on a table. The basic process is the following:
detect the Petri dish on the image, crop everything outside of it;
apply binary thresholding which should result in a black background and white colonies or clusters thereof;
use simple blob detector or watershed to identify the colonies, highlight them on the source image and output their count.
Input example 1 Input example 2 Input example 3: an edge case
Problem 1
The table around the Petri dish isn't smooth and contains spots so I usually use Hough transform to detect the dish and remove everything outside of it. The problem is that there are light reflections near the edge of the Petri dish represented as rings with their radius on par with that of the dish edge, as well as other reflections that obscure the view of the colonies and affect the thresholding applied. So I need reliable code for detecting the innermost circle that has roughly the same centre as the outer border of the Petri dish and doesn't contain any further reflections, i.e. cropping at the outer border is sub-optimal.
Cropping attempt
fig. 1. Grab first detected circle with a radius within the range of [int(image.shape[1]/4),int(image.shape[1]/2)] from circle Hough Transform, use a mask and crop to [x+r:x-r,y+r:y-r]
Problem 2.1
The colonies have a colour usually close to the colour of the background (the agar) and in different areas these can overlap (e.g. colony colour in a section A has the same colour as the background in a section B). This renders the method of general thresholding useless. Different photos having different brightness is an issue as well in the context of the binary method and its rigid parameters - for some images a param of (184,255) is useful while on others only a setting as low as (120,255) results in something half-usable.
Gaussian blur, pyrmeanshift, binary threshold
fig 2.1. Gaussian blur (3,3) + pyrmeanshift (6,27) + binary threshold (205,255)
Problem 2.2
The bacterial colonies have round shapes which sometimes form clusters of overlapping circles so the simple blob detector tends to ignore those. The algorithm is supposed to detect the cluster and identify how many colonies (circles) are in it. To tackle this, I've tried Euclidean distance transform coupled with watershed as an alternative to simple blob detector but this needs to be fed a clean image not containing anything other than the colonies themselves, so a robust threshold algorithm is required for removing all the light reflections and eliminating the background's (agar's) gradient. There are also many spots on the Petri dish usually smaller in size than the colonies and not really round - these should be ignored by the detector algorithm. I've heard of adaptive thresholding used for overcoming the problem of a varied background but this tends to convert non-colony small spots on the dish into full-fledged circles which isn't very optimal.
Adaptive threshold - Gaussian method
fig 2.2. Adaptive threshold (Gaussian C)
An attempt at detecting colonies on input example 2
fig 3. A failed attempt at using watershed with distance transform, demonstrating that this algorithm requires a well cleaned-up and properly thresholded input
I'm interested to know whether this task is feasible in the context of a varied collection of photos taken in different lighting conditions as well as different colonies having different sizes and colours. If so, what ways are there to approach this?
I am trying to scan an object with a laser to extract 3D point clouds. There are 2 cameras and 1 laser in my setup. What I do is giving nonzero points in masks to OpenCV's triangulatePoints function as projPoints arg. Since both numbers of points must be the same for triangulatePoints function and there are 2 masks, if one mask has more nonzero points than the other, I basically downsize it to other's size by doing this:
l1 = len(pts1)
l2 = len(pts2)
newPts1 = pts1[0:l2]
Is there a good way for matching left and right frame nonzero points?
First, if your images normally look like that, your sensors are deeply saturated, and consequently your 3D ranges are either worthless or much less accurate than they could be.
Second, you should aim for matching one point per rectified scanline on each image of the pair, rather than a set of points. The whole idea of using a laser stripe is to get a well focused beam of light on as small a spot or band as possible, so you can probe the surface in detail.
For best accuracy, the peak-finding should be done independently on each scanline of the original (distorted and not rectified) images, so it is not affected by the interpolation used by the undistortion and stereo rectification procedures. Rather, you would use the geometrical undistortion and stereo rectification transforms to map the peaks detected in original images into the rectified ones.
There are several classical algorithms for peak-finding with laser stripe-based triangulation methods, you may find this other answer of mine useful.
Last, if your setup is expected to be as in the picture, with the laser stripe illuminating two orthogonal planes in addition to the object of interest, then you do not need to use stereo at all: you can solve for the 3D plane spanned by the laser stripe projector and triangulate by intersecting that plane with each ray back-projecting the peaks of the image of the laser stripe on the object. This is similar to one of the methods J. Y. Bouguet used in his old Ph.D. thesis on desktop photography (here is a summary by S. Seitz). One implementation using a laser striper is detailed in this patent. This method is surprisingly accurate: with it we achieved approximately 0.2mm accuracy in a cubic foot of volume using a dinky 640x480 CCD video camera back in 1999. Patent has expired, so you are free to enjoy it.
I am looking a robust way to extract the contour of the front panel of a washing machine. Or just get 4 corner points of the front panel.
I've tried color masking but didn't find stable results.
Here some examples:
Three potential options:
Get a bunch of images of the machines, manually determine a label saying where the door is, and then train a convolutional neural network to regress those parameters per image.
Treat each image as a separate optimization problem, where the goal is to estimate the parameters of the best rectangle most likely to correspond to the front panel. So our model is theta = (p_1, p_2, p_3, p_4), the four 2D locations of the panel in the image. We need an energy function E to minimize wrt theta (e.g., using gradient descent with momentum, or RANSAC). There are a number of terms you can use, just as some ideas:
a. At least some of the corners should be "corner-like": run a simple corner detector, and define an energy E_corner which penalizes distance to the closest corner.
b. At least some of the edges (between p_1 and p_2 or p_3, for example) should be "edge-like": compute the gradient magnitude of the image M = || \nabla I || and enforce that along the panel edge the values of M should be larger, using an energy E_edge. E.g., for x,y along an edge, let E_edge(x,y)=1/(1+M(x,y)) (Robust losses tend to be better here though).
c. Use the fact that each door is actually a projected 3D rectangle: e.g., see this question. An interesting idea is to start with a rectangle (representing the panel) and instead of regressing the p_i's, instead regress the parameters of an affine transform or even perspective projection transform (though this requires the algorithm estimate depth), that maps the starting rectangle to one in the image. You can then regularize the parameters of the estimated transform to prevent unlikely transforms from being output.
d. Use knowledge of what must be inside the rectangle. For instance, given the four corners, you can determine the ellipse defining the round door to the machine. The appearance statistics within that ellipse should be somewhat unique, as well as the edges/image gradient at the door boundary; hence you can define an energy term encouraging the model to choose corners such that the interior has a dark elliptical object on a white background.
Overall, this approach is similar to snakes, or active contour models, which might be worth looking into for you I think. However, energy-minimizing snakes tend not to consider the inside of the region they enclose; hence, some variant of the Mumford-Shah functional could be a useful addition (though note smoothness of the "door region" is not entirely desirable in your case).
If all your machines are very similar or nearly the same (as the ones you've posted are), it might actually be best to estimate a homography between the images. (See also here or here). Since the front of the machine is nearly planar, the fronts of different images must be related by a homography. Then knowing where the front panel is in one image will tell you where it is in all of them. For instance, check out the OpenCV tutorial for homographies, where they show how to undo the perspective transform of a planar surface allowing you to do a perspective warp of one image to another (here, one projected machine panel to another template one).
I have been trying Python+OpenCV for quite long time already and followed many tutorials in order to identify particles in the following image:
My ultimate goal is to identify every particle, from there I will be able to e.g. count number of particles, calculate a size distribution, etc.
I have already tried to customize many examples several sites.
I got good hints based on:
How to define the markers for Watershed in OpenCV?
Counting particles using image processing in python
Although I was not able to achieve decent results.
How can I identify particles in this image using Python and OpenCV?
IMO, the only hope to get meaningful results is to use the fact that the particles are round. By using some homogeneity criterion, you could find candidate particle centers, and from these grow contours in such a way that they remain round and stop at edges. An option could be to draw rays from the seed point, find the closest edge points and use a robust fit of a circle or an ellipse.
Reject the shapes that are too far from roundness. This should allow you to find the unoccluded particles. Then you can continue the game from other seed points, this time growing contours that can be occluded by the already detected particles. (When an edge is hit, if it is known to belong to a particle, ignore it.)
Let's pretend the goal is to get an estimated number of particles. Also, let's assume those particles are spheres.
With that being said it should be possible to build a model, based on highlight, shadow, halftone to make the final result as accurate as it can be.
With that being said a simple proof of the concept based on highlight segmentation can be verified.
Initial result doesn't seem to be promising, but a tiny change of the contrast improves it:
Should be enough to get estimated number of stones and apply more advanced models for identified regions.
I'm looking to draw outlines of 2D-closed irregular shapes with curved edges, and then compute both (a) the center of gravity (centroid) of the shape's area, and (b) the center of gravity of the shape's perimeter (i.e. the centroid of, e.g., a wire wrapped tightly around the outside of the shape). I have a late-beginner's level proficiency with Python and Matlab.
(a) and (b) are easy enough given a polygon's vertices: the centroid of a polygon's area is given by the equation here, and the centroid of a polygon's perimeter is just the average of all the line segments' midpoints weighted by the line segments' lengths. I've already written some functions to do this.
The trouble I'm having is getting/approximating those vertices from any of the ways I know how to draw a closed shape with curved edges. The best solution I've come up with so far is to use something like this matplotlib-based script to draw the curvy shape, and then call path.to_polygons(), which converts Path objects to polygons — but does so with surprisingly low resolution, such that the resulting approximation is quite poor (and too poor for my purposes — I'd like to compute those centroids fairly precisely).
So, I'm looking to either (i) find some way to increase the resolution of .to_polygons (about 10-fold), which would be satisfactory for my purposes, or (ii) try some new strategy entirely. One option would be to draw the shapes using something like Adobe Illustrator and then get an approximation of their vertices via some plugin or maybe an image processing toolbox (but I have no clue how to do either of those things). Another would be to draw the shape using some toolbox/library that already has built-in functions for finding the centroids of areas and perimeters (I've seen some having the former, but none with the latter). But I'm sure there are many other options out there that I haven't considered.