I'm currently trying to figure out how to detect a text paragraph on an image in order to remove it.
I get an input image, which is similar to the image given above. From there on I want to detect the body of the comment/the message of the comment. Likes, Username and Avatar are not needed and should be ignored. The body should then be removed from the comment, but the rest should stay.
I added a threshold so far and found the contours. The problem is that the comment body does not get detected as one part, but rather as various contours. How do I combine them? Furthermore, I then want to remove it from the image as soon as I found its contour. The background color is RGB(17, 17, 17), is there a way of painting over it or how does it work in OpenCv? I'm quite new to it.
img = cv2.imread("Comment.png")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, threshold = cv2.threshold(gray, 80, 255, cv2.THRESH_BINARY)
contours, _ = cv2.findContours(threshold, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
The result shoud look like this
Help is appreciated, thanks in advance!
The idea is really simple. Use morphology to isolate the text you want to detect. Using this image, create a mask to delete the region of interest in the input image and produce a final image. All via morphology. My answer is in C++, but the implementation is really easy:
//Read input image:
std::string imagePath = "C://opencvImages//commentImage.png";
cv::Mat imageInput= cv::imread( imagePath );
//Convert it to grayscale:
cv::Mat grayImg;
cv::cvtColor( imageInput, grayImg, cv::COLOR_BGR2GRAY );
//Get binary image via Otsu:
cv::threshold( grayImg, grayImg, 0, 255 , cv::THRESH_OTSU );
Up until this point, you have generated the binary image. Now, let's dilate the image using a rectangular structuring element (SE) wider than taller. The idea is that I want to join all the text horizontally AND vertically (just a little bit). If you see the input image, the “TEST132212” text is just a little bit separated from the comment, enough to survive the dilate operation, it seems. Let's see, here, I'm using a SE of size 9 x 6 with 2 iterations:
cv::Mat morphKernel = cv::getStructuringElement( cv::MORPH_RECT, cv::Size(9, 6) );
int morphIterations = 2;
cv::morphologyEx( grayImg, grayImg, cv::MORPH_DILATE, morphKernel, cv::Point(-1,-1), morphIterations );
This is the result:
I got a unique block where the original comment was - Nice! Now, this is the largest blob in the image. If I subtract it to the original binary image, I should generate a mask that will successfully isolate everything that is not the “comment” blob:
cv::Mat bigBlob = findBiggestBlob( grayImg );
I get this:
Now, the binary mask generation:
cv::Mat binaryMask = grayImg - bigBlob;
//Use the binaryMask to produce the final image:
cv::Mat resultImg;
imageInput.copyTo( resultImg, binaryMask );
Produces the masked image:
Now, you should have noted the findBiggestBlob function. This is a function I've made that returns the biggest blob in a binary image. The idea is just to compute all the contours in the input image, calculate their area and store the contour with the largest area of the bunch. This is the C++ implementation:
//Function to get the largest blob in a binary image:
cv::Mat findBiggestBlob( cv::Mat &inputImage ){
cv::Mat biggestBlob = inputImage.clone();
int largest_area = 0;
int largest_contour_index=0;
std::vector< std::vector<cv::Point> > contours; // Vector for storing contour
std::vector<cv::Vec4i> hierarchy;
// Find the contours in the image
cv::findContours( biggestBlob, contours, hierarchy,CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE );
for( int i = 0; i< (int)contours.size(); i++ ) {
//Find the area of the contour
double a = cv::contourArea( contours[i],false);
//Store the index of largest contour:
if( a > largest_area ){
largest_area = a;
largest_contour_index = i;
}
}
//Once you get the biggest blob, paint it black:
cv::Mat tempMat = biggestBlob.clone();
cv::drawContours( tempMat, contours, largest_contour_index, cv::Scalar(0),
CV_FILLED, 8, hierarchy );
//Erase the smaller blobs:
biggestBlob = biggestBlob - tempMat;
tempMat.release();
return biggestBlob;
}
Edit: Since the posting of the answer, I've been learning Python. Here's the Python equivalent of the C++ code:
import cv2
import numpy as np
# Set image path
path = "D://opencvImages//"
fileName = "commentImage.png"
# Read Input image
inputImage = cv2.imread(path+fileName)
# Convert BGR to grayscale:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Threshold via Otsu + bias adjustment:
threshValue, binaryImage = cv2.threshold(grayscaleImage, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)
# Set kernel (structuring element) size:
kernelSize = (9, 6)
# Set operation iterations:
opIterations = 2
# Get the structuring element:
morphKernel = cv2.getStructuringElement(cv2.MORPH_RECT, kernelSize)
# Perform Dilate:
openingImage = cv2.morphologyEx(binaryImage, cv2.MORPH_DILATE, morphKernel, None, None, opIterations, cv2.BORDER_REFLECT101)
# Find the big contours/blobs on the filtered image:
biggestBlob = openingImage.copy()
contours, hierarchy = cv2.findContours(biggestBlob, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
contoursPoly = [None] * len(contours)
boundRect = []
largestArea = 0
largestContourIndex = 0
# Loop through the contours, store the biggest one:
for i, c in enumerate(contours):
# Get the area for the current contour:
currentArea = cv2.contourArea(c, False)
# Store the index of largest contour:
if currentArea > largestArea:
largestArea = currentArea
largestContourIndex = i
# Once you get the biggest blob, paint it black:
tempMat = biggestBlob.copy()
# Draw the contours on the mask image:
cv2.drawContours(tempMat, contours, largestContourIndex, (0, 0, 0), -1, 8, hierarchy)
# Erase the smaller blobs:
biggestBlob = biggestBlob - tempMat
# Generate the binary mask:
binaryMask = openingImage - biggestBlob
# Use the binaryMask to produce the final image:
resultImg = cv2.bitwise_and(inputImage, inputImage, mask = binaryMask)
cv2.imshow("Result", resultImg)
cv2.waitKey(0)
Related
I am a newbie in android, and open CV both. However, I am trying to take an image from the camera, convert it into the desired format, and pass it to the tflite model.
Code for capturing image, and applying image processing to it.
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
float mh = mRGBA.height();
float cw = (float) Resources.getSystem().getDisplayMetrics().widthPixels;
float scale = mh / cw * 0.7f;
mRGBA = inputFrame.rgba();
frame = classifier.processMat(mRGBA);
Mat temp = new Mat();
Mat temp3= new Mat();
if (!isDebug) {
if (counter == CLASSIFY_INTERVAL) {
Imgproc.cvtColor(frame, frame, Imgproc.COLOR_RGBA2GRAY);
Core.rotate(frame, frame, Core.ROTATE_90_CLOCKWISE);
Imgproc.GaussianBlur(frame, frame, new Size(5, 5), 0);
Imgproc.adaptiveThreshold(frame, frame, 255, Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C, Imgproc.THRESH_BINARY_INV , 3, 2);
Bitmap bmsp = null;
runInterpreter();
counter = 0;
} else {
counter++;
}
}
Imgproc.rectangle(mRGBA,
new Point(mRGBA.cols() / 2f - (mRGBA.cols() * scale / 2),
mRGBA.rows() / 2f - (mRGBA.cols() * scale / 2)),
new Point(mRGBA.cols() / 2f + (mRGBA.cols() * scale / 2),
mRGBA.rows() / 2f + (mRGBA.cols() * scale / 2)),
new Scalar(0, 255, 0), 1);
if (isEdge) {
mRGBA = classifier.debugMat(mRGBA);
}
System.gc();
return mRGBA;
}
My output looks like this image, but I want the hand to be filled with white color before passing it to model. Can somebody suggest?
The main issue is that the result of adaptiveThreshold has gaps in the external edge, so you can't use it as input to findContours.
I think that using GaussianBlur makes things worst, because it blurs the edge between the hand and the background.
You may use the following stages:
Convert frame to Grayscale.
Apply adaptiveThreshold with large kernel size (I used size 51).
Using a large kernel size, keeps a thick edge line without gaps (except from a small gap at the fingernail).
Find contours.
Find the contour with the maximum area.
Draw the contour (fill with solid value of 255) on a zeros image.
There is a problem: the inner part of the hand is not filled due to the weird shape of the contour.
For complete the filling:
Find the center of the contour, and fill it using floodFill.
Here is a Python code sample:
import numpy as np
import cv2
frame = cv2.imread("hand.jpg") # Read image from file (for testing).
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # Use BGR to Gray conversion (not RGBA, because image is read from file)
# Apply adaptiveThreshold with large filter size.
thres_gray = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 51, 2)
# Find contours (external contours)
cnts, hier = cv2.findContours(thres_gray, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
# Find contour with the maximum area
c = max(cnts, key=cv2.contourArea)
res = np.zeros_like(gray) # Create new zeros images for storing the result.
# Fill the contour with white color - draw the filled contour on res image.
cv2.drawContours(res, [c], -1, 255, -1)
# Compute the center of the contour
# https://www.pyimagesearch.com/2016/02/01/opencv-center-of-contour/
M = cv2.moments(c)
cX = int(M["m10"] / M["m00"])
cY = int(M["m01"] / M["m00"])
# Use floodFill for filling the center of the contour
cv2.floodFill(res, None, (cX, cY), 255)
# Show images for testing
cv2.imshow('thres_gray', thres_gray)
cv2.imshow('res', res)
cv2.waitKey()
cv2.destroyAllWindows()
Results:
thres_gray:
res before floodFill:
res after floodFill:
JAVA implementation:
package myproject;
import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.CvType;
import org.opencv.core.Scalar;
import org.opencv.core.Point;
import org.opencv.core.MatOfPoint;
import org.opencv.imgproc.Imgproc;
import org.opencv.imgproc.Moments;
import org.opencv.imgcodecs.Imgcodecs;
import java.util.List;
import java.util.ArrayList;
class Sample {
static { System.loadLibrary(Core.NATIVE_LIBRARY_NAME); }
public static void main(String[] args) {
Mat frame = Imgcodecs.imread("hand.jpg");
Mat gray = new Mat();
Mat thres_gray = new Mat();
Imgproc.cvtColor(frame, gray, Imgproc.COLOR_BGR2GRAY);
//Apply adaptiveThreshold with large filter size.
Imgproc.adaptiveThreshold(gray, thres_gray, 255, Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C, Imgproc.THRESH_BINARY_INV, 51, 2);
List<MatOfPoint> contours = new ArrayList<>();
Mat hierarchy = new Mat();
//Find contours
Imgproc.findContours(thres_gray, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_NONE);
//Find contour with the maximum area
//https://stackoverflow.com/questions/38759925/how-to-find-largest-contour-in-java-opencv
double maxVal = 0;
int maxValIdx = 0;
for (int contourIdx = 0; contourIdx < contours.size(); contourIdx++)
{
double contourArea = Imgproc.contourArea(contours.get(contourIdx));
if (maxVal < contourArea)
{
maxVal = contourArea;
maxValIdx = contourIdx;
}
}
Mat res = Mat.zeros(gray.size(), CvType.CV_8UC1); //Create new zeros images for storing the result.
Imgproc.drawContours(res, contours, maxValIdx, new Scalar(255), -1);
//Compute the center of the contour
//https://www.pyimagesearch.com/2016/02/01/opencv-center-of-contour/
Moments M = Imgproc.moments(contours.get(maxValIdx));
int cX = (int)(M.get_m10() / M.get_m00());
int cY = (int)(M.get_m01() / M.get_m00());
//Use floodFill for filling the center of the contour.
Mat mask = Mat.zeros(res.rows() + 2, res.cols() + 2, CvType.CV_8UC1);
Imgproc.floodFill(res, mask, new Point(cX, cY), new Scalar(255));
Imgcodecs.imwrite("res.png", res);
}
}
I have this image that contains text (numbers and alphabets) in it. I want to get the location of all the text and numbers present in this image. Also I want to extract all the text as well.
How do I get the coordinates as well as the all the text (numbers and alphabets) in my image? For eg 10B, 44, 16, 38, 22B etc
Here's a potential approach using morphological operations to filter out non-text contours. The idea is:
Obtain binary image. Load image, grayscale, then
Otsu's threshold
Remove horizontal and vertical lines. Create horizontal and vertical kernels using cv2.getStructuringElement() then remove lines with cv2.drawContours()
Remove diagonal lines, circle objects, and curved contours. Filter using contour area cv2.contourArea()
and contour approximation cv2.approxPolyDP()
to isolate non-text contours
Extract text ROIs and OCR. Find contours and filter for ROIs then OCR using
Pytesseract.
Removed horizontal lines highlighted in green
Removed vertical lines
Removed assorted non-text contours (diagonal lines, circular objects, and curves)
Detected text regions
import cv2
import numpy as np
import pytesseract
pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract.exe"
# Load image, grayscale, Otsu's threshold
image = cv2.imread('1.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
clean = thresh.copy()
# Remove horizontal lines
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (15,1))
detect_horizontal = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
cnts = cv2.findContours(detect_horizontal, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(clean, [c], -1, 0, 3)
# Remove vertical lines
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,30))
detect_vertical = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, vertical_kernel, iterations=2)
cnts = cv2.findContours(detect_vertical, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(clean, [c], -1, 0, 3)
cnts = cv2.findContours(clean, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
# Remove diagonal lines
area = cv2.contourArea(c)
if area < 100:
cv2.drawContours(clean, [c], -1, 0, 3)
# Remove circle objects
elif area > 1000:
cv2.drawContours(clean, [c], -1, 0, -1)
# Remove curve stuff
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.02 * peri, True)
x,y,w,h = cv2.boundingRect(c)
if len(approx) == 4:
cv2.rectangle(clean, (x, y), (x + w, y + h), 0, -1)
open_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (2,2))
opening = cv2.morphologyEx(clean, cv2.MORPH_OPEN, open_kernel, iterations=2)
close_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3,2))
close = cv2.morphologyEx(opening, cv2.MORPH_CLOSE, close_kernel, iterations=4)
cnts = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
x,y,w,h = cv2.boundingRect(c)
area = cv2.contourArea(c)
if area > 500:
ROI = image[y:y+h, x:x+w]
ROI = cv2.GaussianBlur(ROI, (3,3), 0)
data = pytesseract.image_to_string(ROI, lang='eng',config='--psm 6')
if data.isalnum():
cv2.rectangle(image, (x, y), (x + w, y + h), (36,255,12), 2)
print(data)
cv2.imwrite('image.png', image)
cv2.imwrite('clean.png', clean)
cv2.imwrite('close.png', close)
cv2.imwrite('opening.png', opening)
cv2.waitKey()
Alright, here's another possible solution. I know you work with Python - I work with C++. I'll give you some ideas and hopefully, if you desire so, you will be able to implement this answer.
The main idea is to not use pre-processing at all (at least not at the initial stage) and instead focus on each target character, get some properties, and filter every blob according to these properties.
I'm trying to not use pre-processing because: 1) Filters and morphological stages could degrade the quality of the blobs and 2) your target blobs appear to exhibit some characteristics that we could exploit, mainly: aspect ratio and area.
Check it out, the numbers and letters all appear to be taller than wider… furthermore, they appear to vary within a certain area value. For example, you want to discard objects "too wide" or "too big".
The idea is that I'll filter everything that does not fall within pre-calculated values. I examined the characters (numbers and letters) and came with minimum, maximum area values and a minimum aspect ratio (here, the ratio between height and width).
Let's work on the algorithm. Start by reading the image and resizing it to half the dimensions. Your image is way too big. Convert to grayscale and get a binary image via otsu, here's in pseudo-code:
//Read input:
inputImage = imread( "diagram.png" );
//Resize Image;
resizeScale = 0.5;
inputResized = imresize( inputImage, resizeScale );
//Convert to grayscale;
inputGray = rgb2gray( inputResized );
//Get binary image via otsu:
binaryImage = imbinarize( inputGray, "Otsu" );
Cool. We will work with this image. You need to examine every white blob, and apply a "properties filter". I’m using connected components with stats to loop trough each blob and get its area and aspect ratio, in C++ this is done as follows:
//Prepare the output matrices:
cv::Mat outputLabels, stats, centroids;
int connectivity = 8;
//Run the binary image through connected components:
int numberofComponents = cv::connectedComponentsWithStats( binaryImage, outputLabels, stats, centroids, connectivity );
//Prepare a vector of colors – color the filtered blobs in black
std::vector<cv::Vec3b> colors(numberofComponents+1);
colors[0] = cv::Vec3b( 0, 0, 0 ); // Element 0 is the background, which remains black.
//loop through the detected blobs:
for( int i = 1; i <= numberofComponents; i++ ) {
//get area:
auto blobArea = stats.at<int>(i, cv::CC_STAT_AREA);
//get height, width and compute aspect ratio:
auto blobWidth = stats.at<int>(i, cv::CC_STAT_WIDTH);
auto blobHeight = stats.at<int>(i, cv::CC_STAT_HEIGHT);
float blobAspectRatio = (float)blobHeight/(float)blobWidth;
//Filter your blobs…
};
Now, we will apply the properties filter. This is just a comparison with the pre-calculated thresholds. I used the following values:
Minimum Area: 40 Maximum Area:400
MinimumAspectRatio: 1
Inside your for loop, compare the current blob properties with these values. If the tests are positive, you "paint" the blob black. Continuing inside the for loop:
//Filter your blobs…
//Test the current properties against the thresholds:
bool areaTest = (blobArea > maxArea)||(blobArea < minArea);
bool aspectRatioTest = !(blobAspectRatio > minAspectRatio); //notice we are looking for TALL elements!
//Paint the blob black:
if( areaTest || aspectRatioTest ){
//filtered blobs are colored in black:
colors[i] = cv::Vec3b( 0, 0, 0 );
}else{
//unfiltered blobs are colored in white:
colors[i] = cv::Vec3b( 255, 255, 255 );
}
After the loop, construct the filtered image:
cv::Mat filteredMat = cv::Mat::zeros( binaryImage.size(), CV_8UC3 );
for( int y = 0; y < filteredMat.rows; y++ ){
for( int x = 0; x < filteredMat.cols; x++ )
{
int label = outputLabels.at<int>(y, x);
filteredMat.at<cv::Vec3b>(y, x) = colors[label];
}
}
And… that's pretty much it. You filtered all the elements that are not similar to what you are looking for. Running the algorithm you get this result:
I've additionally found the Bounding Boxes of the blobs to better visualize the results:
As you see, some elements are miss-detected. You can refine the "properties filter" to better identify the characters you are looking for. A deeper solution, involving a little bit of machine learning, requires the construction of an "ideal feature vector", extracting features from the blobs, and comparing both vectors via a similarity measure. You can also apply some post-processing to improve the results...
One method is to use sliding window (It is expensive).
Determine the size of the characters in the image (all characters are of same size as seen in the image) and set the size of the window. Try tesseract for the detection (The input image requires pre processing). If a window detects characters consecutively, then store the coordinates of the window. Merge the coordinates and get the region on the characters.
I am new to OpenCV and Python and I have been encountering a problem in removing noises in my input image. I only wanted to extract the nucleus of a WBC so I used addition to highlight the nucleus and used thresholding to remove the RBCs in the image. I successfully removed the RBCs but the platelets are not removed and some lines appeared in the borders. I also tried using dilation, erosion, opening and closing to denoise the image but the nucleus gets destroyed.
Here is my code:
img = cv2.imread('1.bmp')
img_2 = cv2.imread('1.bmp')
input_img = cv2.addWeighted(img, 0.55, img_2, 0.6, 0)
retval, threshold = cv2.threshold(input_img, 158, 255, cv2.THRESH_BINARY)
threshold = cv2.cvtColor(threshold, cv2.COLOR_BGR2GRAY)
retval2, threshold2 = cv2.threshold(threshold, 0, 255,
cv2.THRESH_BINARY+cv2.THRESH_OTSU)
blur2 = cv2.medianBlur(threshold2,5)
Here is the original image:
After Thresholding:
If the nucleus of a WBC as you have highlighted is always the largest contour before thresholding, I would suggest using findContours to store it alone and remove the smaller blobs like this:
vector<vector<Point>>contours; //Vector for storing contour
vector<Vec4i> hierarchy;
//Find the contours in the image
findContours(input_img, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);
for (int i = 0; i< contours.size(); i++) // iterate through each contour.
{
double a = contourArea(contours[i], false); // Find the area of contour
if (a>largest_area){
largest_area = a;
//Store the index of largest contour
largest_contour_index = i;
// Find the bounding rectangle for biggest contour
bounding_rect = boundingRect(contours[i]);
}
}
Scalar color(255, 255, 255);
// Draw the largest contour using the previously stored index.
Mat dst;
drawContours(dst, contours, largest_contour_index, color, CV_FILLED, 8, hierarchy);
My code is C++ but you can find python examples: How to detect and draw contours using OpenCV in Python?
I have an image to process.I need detect all the circles in the image.Here is it.
And here is my code.
import cv2
import cv2.cv as cv
img = cv2.imread(imgpath)
cv2.imshow("imgorg",img)
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
cv2.imshow("gray",gray)
ret,thresh = cv2.threshold(gray, 199, 255, cv.CV_THRESH_BINARY_INV)
cv2.imshow("thresh",thresh)
cv2.waitKey(0)
cv2.destrotAllWindows()
Then,I got a image like this.
And I tried to use erode and dilate to divided them into single.But it doesnt work.My question is how to divide these contacted circles into single,so i can detect them.
According to #Micka's idea,I tried to process the image in following way,and here is my code.
import cv2
import cv2.cv as cv
import numpy as np
def findcircles(img,contours):
minArea = 300;
minCircleRatio = 0.5;
for contour in contours:
area = cv2.contourArea(contour)
if area < minArea:
continue
(x,y),radius = cv2.minEnclosingCircle(contour)
center = (int(x),int(y))
radius = int(radius)
circleArea = radius*radius*cv.CV_PI;
if area/circleArea < minCircleRatio:
continue;
cv2.circle(img, center, radius, (0, 255, 0), 2)
cv2.imshow("imggg",img)
img = cv2.imread("a.png")
cv2.imshow("org",img)
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
ret,threshold = cv2.threshold(gray, 199, 255,cv. CV_THRESH_BINARY_INV)
cv2.imshow("threshold",threshold)
blur = cv2.medianBlur(gray,5)
cv2.imshow("blur",blur)
laplacian=cv2.Laplacian(blur,-1,ksize = 5,delta = -50)
cv2.imshow("laplacian",laplacian)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(7,7))
dilation = cv2.dilate(laplacian,kernel,iterations = 1)
cv2.imshow("dilation", dilation)
result= cv2.subtract(threshold,dilation)
cv2.imshow("result",result)
contours, hierarchy = cv2.findContours(result,cv2.RETR_LIST,cv2.CHAIN_APPROX_NONE)
findcircles(gray,contours)
But I dont get the same effect as #Micka's.I dont know which step is wrong.
Adapting the idea of #jochen I came to this:
extract the full circle mask as you've done (I called it fullForeground )
from your colored image, compute grayscale, blur (median blur size 7) it and and extract edges, for example with cv::Laplacian
This laplacian thresholded > 50 gives:
cv::Laplacian(blurred, lap, 0, 5); // no delta
lapMask = lap > 50; // thresholding to values > 50
This one dilated once gives:
cv::dilate(lapMask, dilatedThresholdedLaplacian, cv::Mat()); // dilate the edge mask once
Now subtraction fullForeground - dilatedThresholdedLaplacian (same as and_not operator for this type of masks) gives:
from this you can compute contours. For each contour you can compute the area and compare it to the area of an enclosing circle, giving this code and result:
std::vector<std::vector<cv::Point> > contours;
cv::findContours(separated.clone(), contours, CV_RETR_LIST, CV_CHAIN_APPROX_NONE);
double minArea = 500;
double minCircleRatio = 0.5;
for(unsigned int i=0; i<contours.size(); ++i)
{
double cArea = cv::contourArea(contours[i]);
if(cArea < minArea) continue;
//filteredContours.push_back(contours[i]);
//cv::drawContours(input, contours, i, cv::Scalar(0,255,0), 1);
cv::Point2f center;
float radius;
cv::minEnclosingCircle(contours[i], center, radius);
double circleArea = radius*radius*CV_PI;
if(cArea/circleArea < minCircleRatio) continue;
cv::circle(input, center, radius, cv::Scalar(0,0,255),2);
}
here is another image showing the coverage:
hope this helps
I think the first mistake ist the value of thesh.
In your example the command cv2.threshold converts all white areas to black and everything else to white. I would suggest using a smaller value for thesh so that all black pixel get converted to white and all white or "colored" pixels (inside the circles) get converted to black or vise versa. The value of thesh should be a little bigger than the brightest of the black pixels.
See opencv docu for threshold for more information.
Afterwards I would let opencv find all contours in the thresholded image and filter them for "valid" circles, e.g. by size and shape.
If that is not sufficiant you could segment the inner circle from the rest of the image: First compute threasholdImageA with all white areas colored black. Then compute threasholdImageB with all the black areas being black. Afterwards combine both, threasholdImageA and threasholdImageB, (e.g. with numpy.logical_and) to have a binary image with only the inner circle being white and the rest black. Of course the values for the threshold have to be chosen wisely to get the specific result.
That way also circles where the inner part directly touches the background will be segmented.
I'm working on opencv problem to figure out which circles are filled. However, sometimes edge of circles are cause of false positive. It makes my wonder if I can remove these circles by turning all pixels white that have high R value in RGB. My approach is to create a mask of pixels that are pinkish and then subtract mask from original image to remove circles. As of now I am getting black mask. I'm doing something wrong. Please guide.
rgb = cv2.imread(img, cv2.CV_LOAD_IMAGE_COLOR)
rgb_filtered = cv2.inRange(rgb, (200, 0, 90), (255, 110, 255))
cv2.imwrite('mask.png',rgb_filtered)
Here is my solution. Unfortunately it's in C++ too and this is how it works:
threshold the image to find out which parts are background (white paper)
find the circles by extracting contours.
now each contour is assumed to be a circle, so compute the minimum circle enclosing that contour. No parameter tuning necessary if the input is ok (that means every circle is a single contour, so circle may not be connected by drawing for example)
check for each circle, whether there are more foreground (drawing) or background (white paper) pixel inside (by some ratio threshold).
int main()
{
cv::Mat colorImage = cv::imread("countFilledCircles.png");
cv::Mat image = cv::imread("countFilledCircles.png", CV_LOAD_IMAGE_GRAYSCALE);
// threshold the image!
cv::Mat thresholded;
cv::threshold(image,thresholded,0,255,CV_THRESH_BINARY_INV | CV_THRESH_OTSU);
// save threshold image for demonstration:
cv::imwrite("countFilledCircles_threshold.png", thresholded);
// find outer-contours in the image these should be the circles!
cv::Mat conts = thresholded.clone();
std::vector<std::vector<cv::Point> > contours;
std::vector<cv::Vec4i> hierarchy;
cv::findContours(conts,contours,hierarchy, CV_RETR_EXTERNAL, CV_C HAIN_APPROX_SIMPLE, cv::Point(0,0));
// colors in which marked/unmarked circle outlines will be drawn:
cv::Scalar colorMarked(0,255,0);
cv::Scalar colorUnMarked(0,0,255);
// each outer contour is assumed to be a circle
// TODO: you could first find the mean radius of all assumed circles and try to find outlier (dirt etc in the image)
for(unsigned int i=0; i<contours.size(); ++i)
{
cv::Point2f center;
float radius;
// find minimum circle enclosing the contour
cv::minEnclosingCircle(contours[i],center,radius);
bool marked = false;
cv::Rect circleROI(center.x-radius, center.y-radius, center.x+radius, center.y+radius);
//circleROI = circleROI & cv::Rect(0,0,image.cols, image.rows);
// count pixel inside the circle
float sumCirclePixel = 0;
float sumCirclePixelMarked = 0;
for(int j=circleROI.y; j<circleROI.y+circleROI.height; ++j)
for(int i=circleROI.x; i<circleROI.x+circleROI.width; ++i)
{
cv::Point2f current(i,j);
// test if pixel really inside the circle:
if(cv::norm(current-center) < radius)
{
// count total number of pixel in the circle
sumCirclePixel = sumCirclePixel+1.0f;
// and count all pixel in the circle which hold the segmentation threshold
if(thresholded.at<unsigned char>(j,i))
sumCirclePixelMarked = sumCirclePixelMarked + 1.0f;
}
}
const float ratioThreshold = 0.5f;
if(sumCirclePixel)
if(sumCirclePixelMarked/sumCirclePixel > ratioThreshold) marked = true;
// draw the circle for demonstration
if(marked)
cv::circle(colorImage,center,radius,colorMarked,1);
else
cv::circle(colorImage,center,radius,colorUnMarked,1);
}
cv::imshow("thres", thresholded);
cv::imshow("colorImage", colorImage);
cv::imwrite("countFilledCircles_output.png", colorImage);
cv::waitKey(-1);
}
giving me these results:
after otsu thresholding:
final image:
I've tried to come up with a solution in Python. Basically the process is the following:
Gaussian blur to reduce noise.
Otsu's threshold.
Find contours that have no parents, those contours should be the circles.
Check the ratio of white-to-black pixels inside each contour.
You may need to tune up the white ratio threshold to fit your application. I've used 0.7 as it seems a reasonable value.
import cv2
import numpy
# Read image and apply gaussian blur
img = cv2.imread("circles.png", cv2.CV_LOAD_IMAGE_GRAYSCALE)
img = cv2.GaussianBlur(img, (5, 5), 0)
# Apply OTSU thresholding and reverse it so the circles are in the foreground (white)
_, otsu = cv2.threshold(img, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
otsu = cv2.bitwise_not(otsu).astype("uint8")
# Find contours that have no parent
contours, hierarchy = cv2.findContours(numpy.copy(otsu), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
parent_contours = [contours[idx] for idx, val in enumerate(hierarchy[0]) if val[3] == -1]
# Loop through all contours to check the ratio of white to black pixels inside each one
filled_circles_contours = list()
for contour in parent_contours:
contour_mask = numpy.zeros(img.shape).astype("uint8")
cv2.drawContours(contour_mask, [contour], -1, 1, thickness=-1)
white_len_mask = len(cv2.findNonZero(contour_mask))
white_len_thresholded = len(cv2.findNonZero(contour_mask * otsu))
white_ratio = float(white_len_thresholded) / white_len_mask
if white_ratio > 0.7:
filled_circles_contours.append(contour)
# Show image with detected circles
cv2.drawContours(img, filled_circles_contours, -1, (0, 0, 0), thickness=2)
cv2.namedWindow("Result")
cv2.imshow("Result", img)
cv2.waitKey(0)
This is the result I obtained from applying the code above to your image:
Here's how I did it:
Convert to grayscale, apply gaussian blur to remove noises
Apply otsu thresholding, it's quite good to separate fore and background, you should read about it
Apply Hough circle transform to find candidate circles, sadly this requires heavy tuning. Maybe watershed segmentation is a better alternative
Extract the ROI from the candidate circles, and find the ratio of black and white pixels.
Here's my sample result:
When we draw our result on original image:
Here's the sample code (sorry in C++):
void findFilledCircles( Mat& img ){
Mat gray;
cvtColor( img, gray, CV_BGR2GRAY );
/* Apply some blurring to remove some noises */
GaussianBlur( gray, gray, Size(5, 5), 1, 1);
/* Otsu thresholding maximizes inter class variance, pretty good in separating background from foreground */
threshold( gray, gray, 0.0, 255.0, CV_THRESH_OTSU );
erode( gray, gray, Mat(), Point(-1, -1), 1 );
/* Sadly, this is tuning heavy, adjust the params for Hough Circles */
double dp = 1.0;
double min_dist = 15.0;
double param1 = 40.0;
double param2 = 10.0;
int min_radius = 15;
int max_radius = 22;
/* Use hough circles to find the circles, maybe we could use watershed for segmentation instead(?) */
vector<Vec3f> found_circles;
HoughCircles( gray, found_circles, CV_HOUGH_GRADIENT, dp, min_dist, param1, param2, min_radius, max_radius );
/* This is just to draw coloured circles on the 'originally' gray image */
vector<Mat> out = { gray, gray, gray };
Mat output;
merge( out, output );
float diameter = max_radius * 2;
float area = diameter * diameter;
Mat roi( max_radius, max_radius, CV_8UC3, Scalar(255, 255, 255) );
for( Vec3f circ: found_circles ) {
/* Basically we extract the region of the circles, and count the ratio of black pixels (0) and white pixels (255) */
Mat( gray, Rect( circ[0] - max_radius, circ[1] - max_radius, diameter, diameter ) ).copyTo( roi );
float filled_percentage = 1.0 - 1.0 * countNonZero( roi ) / area;
/* If more than half is filled, then maybe it's filled */
if( filled_percentage > 0.5 )
circle( output, Point2f( circ[0], circ[1] ), max_radius, Scalar( 0, 0, 255), 3 );
else
circle( output, Point2f( circ[0], circ[1] ), max_radius, Scalar( 255, 255, 0), 3 );
}
namedWindow("");
moveWindow("", 0, 0);
imshow("", output );
waitKey();
}