I am attempting to use machine learning (namely random forests) for image segmentation. The classifier utilizes a number of different pixel level features to classify pixels as either edge pixels or non edge pixels. I recently applied my classifier to a set of images that are pretty difficult to segment even manually (Image segmentation based on edge pixel map) and am still working on obtaining reasonable contours from the resulting probability map. I also applied the classifier to an easier set of images and am obtaining quite good predicted outlines (Rand index > 0.97) when I adjust the threshold to 0.95. I am interested in improving the segmentation result by filtering contours extracted from the probability map.
Here is the original image:
The expert outlines:
The probability map generated from my classifier:
This can be further refined when I convert the image to binary based on a threshold of 0.95:
I tried filling holes in the probability map, but that left me with a lot of noise and sometimes merged nearby cells. I also tried contour finding in openCV but this didn't work either as many of these contours are not completely connected - a few pixels will be missing here and there in the outlines.
Edit: I ended up using Canny edge detection on the probability map.
The initial image seems to be well contrasted and I guess we can simply threshold to obtain a good estimate of the cells. Here is a morphological area based filtering of the thresholded image:
Threshold:
Area based opening filter(this needs to be set based on your dataset of cells under study):
Area based closing filter(this needs to be set based on your dataset of cells under study):
Contours using I-Erosion(I):
Code snippet:
C is input image
C10 = C>10; %threshold depends on the average contrast in your dataset
C10_areaopen = bwareaopen(C10,2500); %area filters average remove small components that are not cells
C10_areaopenclose = ~bwareaopen(~C10_areaopen,100); %area filter fills holes
se = strel('disk',1);
figure, imshow(C10_areaopenclose-imerode(C10_areaopenclose,se)) %inner contour
To get smoother shapes I guess fine opening operations can be performed on the filtered images, thus removing any concave parts of the cells. Also for cells that are attached one could use the distance function and the watershed over the distance function to obtain segmentations of the cells: http://www.ias-iss.org/ojs/IAS/article/viewFile/862/765
I guess this can be also used on your probability/confidence maps to perform nonlinear area based filtering.
Related
I have a task of identifying the number of bacterial colonies on a relatively diverse set of top-down photos of a Petri dish located on a table. The basic process is the following:
detect the Petri dish on the image, crop everything outside of it;
apply binary thresholding which should result in a black background and white colonies or clusters thereof;
use simple blob detector or watershed to identify the colonies, highlight them on the source image and output their count.
Input example 1 Input example 2 Input example 3: an edge case
Problem 1
The table around the Petri dish isn't smooth and contains spots so I usually use Hough transform to detect the dish and remove everything outside of it. The problem is that there are light reflections near the edge of the Petri dish represented as rings with their radius on par with that of the dish edge, as well as other reflections that obscure the view of the colonies and affect the thresholding applied. So I need reliable code for detecting the innermost circle that has roughly the same centre as the outer border of the Petri dish and doesn't contain any further reflections, i.e. cropping at the outer border is sub-optimal.
Cropping attempt
fig. 1. Grab first detected circle with a radius within the range of [int(image.shape[1]/4),int(image.shape[1]/2)] from circle Hough Transform, use a mask and crop to [x+r:x-r,y+r:y-r]
Problem 2.1
The colonies have a colour usually close to the colour of the background (the agar) and in different areas these can overlap (e.g. colony colour in a section A has the same colour as the background in a section B). This renders the method of general thresholding useless. Different photos having different brightness is an issue as well in the context of the binary method and its rigid parameters - for some images a param of (184,255) is useful while on others only a setting as low as (120,255) results in something half-usable.
Gaussian blur, pyrmeanshift, binary threshold
fig 2.1. Gaussian blur (3,3) + pyrmeanshift (6,27) + binary threshold (205,255)
Problem 2.2
The bacterial colonies have round shapes which sometimes form clusters of overlapping circles so the simple blob detector tends to ignore those. The algorithm is supposed to detect the cluster and identify how many colonies (circles) are in it. To tackle this, I've tried Euclidean distance transform coupled with watershed as an alternative to simple blob detector but this needs to be fed a clean image not containing anything other than the colonies themselves, so a robust threshold algorithm is required for removing all the light reflections and eliminating the background's (agar's) gradient. There are also many spots on the Petri dish usually smaller in size than the colonies and not really round - these should be ignored by the detector algorithm. I've heard of adaptive thresholding used for overcoming the problem of a varied background but this tends to convert non-colony small spots on the dish into full-fledged circles which isn't very optimal.
Adaptive threshold - Gaussian method
fig 2.2. Adaptive threshold (Gaussian C)
An attempt at detecting colonies on input example 2
fig 3. A failed attempt at using watershed with distance transform, demonstrating that this algorithm requires a well cleaned-up and properly thresholded input
I'm interested to know whether this task is feasible in the context of a varied collection of photos taken in different lighting conditions as well as different colonies having different sizes and colours. If so, what ways are there to approach this?
I am trying to compare simple geometric images on a white background. These images should be labeled similar (Ex1 Ex2 and Ex1 Ex2). Due to the nature of the objects, Feature Matching does not generate enough accurate matches to determine similarity and matchshapes does not take into account internal features. I would like to detect when the outer contour has additives on it that distort the shape when compared to the corresponding image. Is it possible to determine the smoothness / roughness of a contour using opencv?
I have an image of a leaf on a poorly white screen and need to find out just the leaf:
The problem is that it contains greenish shadows, so simple color-based methods are not enough. Thus, I need some preprocessing to achieve an uniform background and to enhance important features. The goal is to delete undesired noise and distortions to apply an Expectation Maximization algorithm.
I tried playing with OpenCV fastNlMeansDenoising, but it is not sufficient.
UPDATE FOR CLARIFICATION
When segmenting with color it is imperative to use the right color domains that exclude undesired noise. So we used Saturation and Value color components for leaf segmentation. This was useful for posterior segmentation using Expectation-Maximization (EM). I used OpenCV to convert the original images into the HSV domain. Then, using NumPy, I extracted the Saturation and Value components, which were fed to the EM algorithm.
Once images were converted to HSV and the desired channels were extracted, I applied EM to the color domain in order to cluster the pixels into one of 2 possible groups: leaf and non-leaf groups. However, due to that greenish the EM is returning a wrong result.
I cropped license plates but they have some borders I want to remove the borders to segment characters, I tried to use Hough transform but It's not a promising approach. Here is the samples of license plates:
Is there any simple way to do that?
I have a naïve solution for one image. You have to tune some parameters to generalize it for the other images.
I chose the third image due to its clarity.
1. Threshold
In such cases the first step is to reach an optimal threshold, where all the letters/numbers of interest are converted to same pixel values. As a result I got the following:
2. Finding Contour and Bounding Region
Now I found the external contour present in the image to retain the letter/numbers. After finding it I found the bounding rectangle for the corresponding contour:
3. Cropping
Next I used the parameters returned from bounding the contour and used them to crop the image:
VOILA! There you have your region of interest!
Note:
This approach would work if all the images are taken in a similar manner and for the same color space. The second image provided has a different color. Hence you will have to alter the threshold parameters to segment your ROI properly.
You can also perform some morphological operations on the threshold image to obtain a better ROI.
I am in the process of putting together an OpenCV script to analyze immunohistochemically stained heart tissue. Our staining procedure renders cell types expressing certain proteins in their plasma membranes with pigments visible under a light microscope, which we use to photograph the images.
So far, I've succeded in segmenting the images to different layers based on color range using a modified version of the frequently cited color segmentation script available through the OpenCV community(http://opencv-python-tutroals.readthedocs.org/en/latest/py_tutorials/py_imgproc/py_colorspaces/py_colorspaces.html).
A screen shot of the original image:
B-Cell layer displayed:
At this point, I would like to calculate the ratio of area of B-Cells to unstained tissue. This operation prompted an extraction of the background cell layer as such based on color range:
Obviously, these results leave much to be desired.
Does anyone have ideas of how to approach this problem? Again, I would like to segment the background tissue (transparent) layer, which is unfortunately fairly sponge-like in texture. My goal is to create a mask representive of the area of unstained tissue. It seems a blur technique is necessary to fill the gaps in the tissue, but the loss in accuracy this approach entails is obvious.
In the sample image, the channels look highly correlated. If you apply decorrelation-stretching to the image you should be able to see more detail. Here in my blog post I've implemented decorrelation-stretching in C++ (unfortualtely not Python).
Using the sample code in the blog I did the following to segment the cell region:
dstretch the CIE Lab image with following targetMean and tergetSigma.
float mu[3] = {128.0f, 128.0f, 128.0f};
float sd[3] = {128.0f, 5.0f, 5.0f};
Mat mean = Mat(3, 1, CV_32F, mu);
Mat sigma = Mat(3, 1, CV_32F, sd);
Convert the dstretched CIE Lab image back to BGR.
Erode this BGR image with a 3x3 rectangular structuring element once.
Apply kmeans clustering to this eroded image with k = 2.
I don't know how good this segmentation is. I think it is possible to get a better segmentation by trying different values for the above parameters (mean, sigma, structuring element size and number of times the image is eroded).
(Following images are not to the original scale)
Original:
dstretched CIE Lab converted back to BGR:
Eroded:
kmeans with k = 2: