We Keep Coding

Detect different shapes in noisy binary image

I want to detect the circle and the five squares in this image:
This is the relevant part of the code I currently use:
# detect shapes in black-white RGB formatted cv2 image
def detect_shapes(img, approx_poly_accuracy=APPROX_POLY_ACCURACY):
res_dict = {
"rectangles": [],
"squares": []
}
vis = img.copy()
shape = img.shape
height, width = shape[0], shape[1]
total_area = height * width
# Morphological closing: get rid of holes
# img = cv2.morphologyEx(img, cv2.MORPH_CLOSE, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5)))
# Morphological opening: get rid of extensions at the border of the objects
# img = cv2.morphologyEx(img, cv2.MORPH_OPEN, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (121, 121)))
img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
# cv2.imshow('intermediate', img)
# cv2.waitKey(0)
contours, hierarchy = cv2.findContours(img, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
logging.info("Number of found contours for shape detection: {0}".format(len(contours)))
# vis = img.copy()
# cv2.drawContours(vis, contours, -1, (0, 255, 0), 2)
cv2.imshow('vis', vis)
cv2.waitKey(0)
for contour in contours:
area = cv2.contourArea(contour)
if area < MIN_SHAPE_AREA:
logging.warning("Area too small: {0}. Skipping.".format(area))
continue
if area > MAX_SHAPE_AREA_RATIO * total_area:
logging.warning("Area ratio too big: {0}. Skipping.".format(area / total_area))
continue
approx = cv2.approxPolyDP(contour, approx_poly_accuracy * cv2.arcLength(contour, True), True)
cv2.drawContours(vis, [approx], -1, (0, 0, 255), 2)
la = len(approx)
# find the center of the shape
M = cv2.moments(contour)
if M['m00'] == 0.0:
logging.warning("Unable to compute shape center! Skipping.")
continue
x = int(M['m10'] / M['m00'])
y = int(M['m01'] / M['m00'])
if la < 3:
logging.warning("Invalid shape detected! Skipping.")
continue
if la == 3:
logging.info("Triangle detected at position {0}".format((x, y)))
elif la == 4:
logging.info("Quadrilateral detected at position {0}".format((x, y)))
if approx.shape != (4, 1, 2):
raise ValueError("Invalid shape before reshape to (4, 2): {0}".format(approx.shape))
approx = approx.reshape(4, 2)
r_check, data = check_rect_or_square(approx)
blob_data = {"position": (x, y), "approx": approx}
blob_data.update(data)
if r_check == 2:
res_dict["squares"].append(blob_data)
elif r_check == 1:
res_dict["rectangles"].append(blob_data)
elif la == 5:
logging.info("Pentagon detected at position {0}".format((x, y)))
elif la == 6:
logging.info("Hexagon detected at position {0}".format((x, y)))
else:
logging.info("Circle, ellipse or arbitrary shape detected at position {0}".format((x, y)))
cv2.drawContours(vis, [contour], -1, (0, 255, 0), 2)
cv2.imshow('vis', vis)
cv2.waitKey(0)
logging.info("res_dict: {0}".format(res_dict))
return res_dict
The problem is: if I set the approx_poly_accuracy parameter too high, the circle is detected as a polygon (Hexagon or Octagon, for example). If I set it too low, the squares are not detected as squares, but as Pentagons, for example:
The red lines are the approximated contours, the green lines are the original contours. The text is detected as a completely wrong contour, it should never be approximated to this level (I don't care about the text so much, but if it is detected as a polygon with less than 5 vertices, it will be a false positive).
For a human, it is obvious that the left object is a circle and that the five objects on the right are squares, so there should be a way to make the computer realize that with high accuracy too. How do I modify this code to properly detect all objects?
What I already tried:
Apply filters like MedianFilter. It made things worse, because the rounded edges of the squares promoted them being detected as a polygon with more than four vertices.
Variate the approx_poly_accuracy parameter. There is no value that fits my purposes, considering that some other images might even have a some more noise.
Find an implementation of the RDP algorithm that allows me to specify an EXACT number of output points. This would allow me to to compute the suggested polygons for a certain number of points (for example in the range 3..10) and then calculate (A_1 + A_2) / A_common - 1 to use the area instead of the arc length as an accuracy, which would probably lead to a better result. I have not yet found a good implementation for that. I will now try to use a numerical solver method to dynamically figure out the correct epsilon parameter for RDP. The approach is not really clean and efficient though. I will post the results here as soon as available. If someone has a better approach, please let me know.

A possible approach would involve the calculation of some blob descriptors and filter blobs according to those properties. For example, you can compute the blob's aspect ratio, the (approximated) number of vertices and area. The steps are very straightforward:
Load the image and convert it to grayscale.
(Invert) Threshold the image. Let’s make sure the blobs are colored in white.
Get the binary image’s contours.
Compute two features: aspect ratio and number of vertices
Filter the blobs based on those features
Let’s see the code:
# Imports:
import cv2
import numpy as np
# Load the image:
fileName = "yh6Uz.png"
path = "D://opencvImages//"
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
# Prepare a deep copy of the input for results:
inputImageCopy = inputImage.copy()
# Grayscale conversion:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Threshold via Otsu:
_, binaryImage = cv2.threshold(grayscaleImage, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
# Find the blobs on the binary image:
contours, hierarchy = cv2.findContours(binaryImage, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
# Store the bounding rectangles here:
circleData = []
squaresData = []
Alright. So far, I’ve loaded, thresholded and computed contours on the input image. Additionally, I’ve prepared two lists to store the bounding box of the squares and the circle. Let’s create the feature filter:
for i, c in enumerate(contours):
# Get blob perimeter:
currentPerimeter = cv2.arcLength(c, True)
# Approximate the contour to a polygon:
approx = cv2.approxPolyDP(c, 0.04 * currentPerimeter, True)
# Get polygon's number of vertices:
vertices = len(approx)
# Get the polygon's bounding rectangle:
(x, y, w, h) = cv2.boundingRect(approx)
# Compute bounding box area:
rectArea = w * h
# Compute blob aspect ratio:
aspectRatio = w / h
# Set default color for bounding box:
color = (0, 0, 255)
I loop through each contour and calculate the current blob’s perimeter and polygon approximation. This info is used to approximately compute the blob vertices. The aspect ratio calculation is very easy. I first get the blob’s bounding box and get its dimensions: top left corner (x, y), width and height. The aspect ratio is just the width divided by the height.
The squares and the circle a very compact. These means that their aspect ratio should be close to 1.0. However, the squares have exactly 4 vertices, while the (approximated) circle has more. I use this info to build a very basic feature filter. It first checks aspect ratio, area and then number of vertices. I use the difference between the ideal feature and the real feature. The parameter delta adjusts the filter tolerance. Be sure to also filter tiny blobs, use the area for this:
# Set minimum tolerable difference between ideal
# feature and actual feature:
delta = 0.15
# Set the minimum area:
minArea = 400
# Check features, get blobs with aspect ratio
# close to 1.0 and area > min area:
if (abs(1.0 - aspectRatio) < delta) and (rectArea > minArea):
print("Got target blob.")
# If the blob has 4 vertices, it is a square:
if vertices == 4:
print("Target is square")
# Save bounding box info:
tempTuple = (x, y, w, h)
squaresData.append(tempTuple)
# Set green color:
color = (0, 255, 0)
# If the blob has more than 6 vertices, it is a circle:
elif vertices > 6:
print("Target is circle")
# Save bounding box info:
tempTuple = (x, y, w, h)
circleData.append(tempTuple)
# Set blue color:
color = (255, 0, 0)
# Draw bounding rect:
cv2.rectangle(inputImageCopy, (int(x), int(y)), (int(x + w), int(y + h)), color, 2)
cv2.imshow("Rectangles", inputImageCopy)
cv2.waitKey(0)
This is the result. The squares are identified with a green rectangle and the circle with a blue one. Additionally, the bounding boxes are stored in squaresData and circleData respectively:

Area of a closed contour on a plot using python openCV

I am attempting to find the area inside an arbitrarily-shaped closed curve plotted in python (example image below). So far, I have tried to use both the alphashape and polygon methods to acheive this, but both have failed. I am now attempting to use OpenCV and the floodfill method to count the number of pixels inside the curve and then I will later convert that to an area given the area that a single pixel encloses on the plot.
Example image:
testplot.jpg
In order to do this, I am doing the following, which I adapted from another post about OpenCV.
import cv2
import numpy as np
# Input image
img = cv2.imread('testplot.jpg', cv2.IMREAD_GRAYSCALE)
# Dilate to better detect contours
temp = cv2.dilate(temp, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3)))
# Find largest contour
cnts, _ = cv2.findContours(255-temp, cv2.RETR_TREE , cv2.CHAIN_APPROX_NONE) #255-img and cv2.RETR_TREE is to account for how cv2 expects the background to be black, not white, so I convert the background to black.
largestCnt = [] #I expect this to yield the blue contour
for cnt in cnts:
if (len(cnt) > len(largestCnt)):
largestCnt = cnt
# Determine center of area of largest contour
M = cv2.moments(largestCnt)
x = int(M["m10"] / M["m00"])
y = int(M["m01"] / M["m00"])
# Initial mask for flood filling, should cover entire figure
width, height = temp.shape
mask = img2 = np.ones((width + 2, height + 2), np.uint8) * 255
mask[1:width, 1:height] = 0
# Generate intermediate image, draw largest contour onto it, flood fill this contour
temp = np.zeros(temp.shape, np.uint8)
temp = cv2.drawContours(temp, largestCnt, -1, 255, cv2.FILLED)
_, temp, mask, _ = cv2.floodFill(temp, mask, (x, y), 255)
temp = cv2.morphologyEx(temp, cv2.MORPH_OPEN, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3)))
area = cv2.countNonZero(temp) #Number of pixels encircled by blue line
I expect from this to get to a place where I have the same image as above, but with the center of the contour filled in white and the background and original blue contour in black. I end up with this:
result.jpg
While this at first glance appears to have accurately turned the area inside the contour white, the white area is actually larger than the area inside the contour and so the result I get is overestimating the number of pixels inside it.
Any input on this would be greatly appreciated. I am fairly new to OpenCV so I may have misunderstood something.
EDIT:
Thanks to a comment below, I made some edits and this is now my code, with edits noted:
import cv2
import numpy as np
# EDITED INPUT IMAGE: Input image
img = cv2.imread('testplot2.jpg', cv2.IMREAD_GRAYSCALE)
# EDIT: threshold
_, temp = cv2.threshold(img, 250, 255, cv2.THRESH_BINARY_INV)
# EDIT, REMOVED: Dilate to better detect contours
# Find largest contour
cnts, _ = cv2.findContours(temp, cv2.RETR_EXTERNAL , cv2.CHAIN_APPROX_NONE)
largestCnt = [] #I expect this to yield the blue contour
for cnt in cnts:
if (len(cnt) > len(largestCnt)):
largestCnt = cnt
# Determine center of area of largest contour
M = cv2.moments(largestCnt)
x = int(M["m10"] / M["m00"])
y = int(M["m01"] / M["m00"])
# Initial mask for flood filling, should cover entire figure
width, height = temp.shape
mask = img2 = np.ones((width + 2, height + 2), np.uint8) * 255
mask[1:width, 1:height] = 0
# Generate intermediate image, draw largest contour, flood filled
temp = np.zeros(temp.shape, np.uint8)
temp = cv2.drawContours(temp, largestCnt, -1, 255, cv2.FILLED)
_, temp, mask, _ = cv2.floodFill(temp, mask, (x, y), 255)
temp = cv2.morphologyEx(temp, cv2.MORPH_OPEN, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3)))
area = cv2.countNonZero(temp) #Number of pixels encircled by blue line
I input a different image with the axes and the frame that python adds by default removed for ease. I get what I expect at the second step, so this image. However, in the enter image description here both the original contour and the area it encircles appear to have been made white, whereas I want the original contour to be black and only the area it encircles to be white. How might I acheive this?

The problem is your opening operation at the end. This morphological operation includes a dilation at the end that expands the white contour, increasing its area. Let’s try a different approach where no morphology is involved. These are the steps:
Convert your image to grayscale
Apply Otsu’s thresholding to get a binary image, let’s work with black and white pixels only.
Apply a first flood-fill operation at image location (0,0) to get rid of the outer white space.
Filter small blobs using an area filter
Find the “Curve Canvas” (The white space that encloses the curve) and locate and store its starting point at (targetX, targetY)
Apply a second flood-fill al location (targetX, targetY)
Get the area of the isolated blob with cv2.countNonZero
Let’s take a look at the code:
import cv2
import numpy as np
# Set image path
path = "C:/opencvImages/"
fileName = "cLIjM.jpg"
# Read Input image
inputImage = cv2.imread(path+fileName)
inputCopy = inputImage.copy()
# Convert BGR to grayscale:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Threshold via Otsu + bias adjustment:
threshValue, binaryImage = cv2.threshold(grayscaleImage, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)
This is the binary image you get:
Now, let’s flood-fill at the corner located at (0,0) with a black color to get rid of the first white space. This step is very straightforward:
# Flood-fill background, seed at (0,0) and use black color:
cv2.floodFill(binaryImage, None, (0, 0), 0)
This is the result, note how the first big white area is gone:
Let’s get rid of the small blobs applying an area filter. Everything below an area of 100 is gonna be deleted:
# Perform an area filter on the binary blobs:
componentsNumber, labeledImage, componentStats, componentCentroids = \
cv2.connectedComponentsWithStats(binaryImage, connectivity=4)
# Set the minimum pixels for the area filter:
minArea = 100
# Get the indices/labels of the remaining components based on the area stat
# (skip the background component at index 0)
remainingComponentLabels = [i for i in range(1, componentsNumber) if componentStats[i][4] >= minArea]
# Filter the labeled pixels based on the remaining labels,
# assign pixel intensity to 255 (uint8) for the remaining pixels
filteredImage = np.where(np.isin(labeledImage, remainingComponentLabels) == True, 255, 0).astype('uint8')
This is the result of the filter:
Now, what remains is the second white area, I need to locate its starting point because I want to apply a second flood-fill operation at this location. I’ll traverse the image to find the first white pixel. Like this:
# Get Image dimensions:
height, width = filteredImage.shape
# Store the flood-fill point here:
targetX = -1
targetY = -1
for i in range(0, width):
for j in range(0, height):
# Get current binary pixel:
currentPixel = filteredImage[j, i]
# Check if it is the first white pixel:
if targetX == -1 and targetY == -1 and currentPixel == 255:
targetX = i
targetY = j
print("Flooding in X = "+str(targetX)+" Y: "+str(targetY))
There’s probably a more elegant, Python-oriented way of doing this, but I’m still learning the language. Feel free to improve the script (and share it here). The loop, however, gets me the location of the first white pixel, so I can now apply a second flood-fill at this exact location:
# Flood-fill background, seed at (targetX, targetY) and use black color:
cv2.floodFill(filteredImage, None, (targetX, targetY), 0)
You end up with this:
As you see, just count the number of non-zero pixels:
# Get the area of the target curve:
area = cv2.countNonZero(filteredImage)
print("Curve Area is: "+str(area))
The result is:
Curve Area is: 1510

Here is another approach using Python/OpenCV.
Read Input
convert to HSV colorspace
Threshold on color range of blue
Find the largest contour
Get its area and print that
draw the contour as a white filled contour on black background
Save the results
Input:
import cv2
import numpy as np
# read image as grayscale
img = cv2.imread('closed_curve.jpg')
# convert to HSV
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
#select blu color range in hsv
lower = (24,128,115)
upper = (164,255,255)
# threshold on blue in hsv
thresh = cv2.inRange(hsv, lower, upper)
# get largest contour
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)
area = cv2.contourArea(c)
print("Area =",area)
# draw filled contour on black background
result = np.zeros_like(thresh)
cv2.drawContours(result, [c], -1, 255, cv2.FILLED)
# save result
cv2.imwrite("closed_curve_thresh.jpg", thresh)
cv2.imwrite("closed_curve_result.jpg", result)
# view result
cv2.imshow("threshold", thresh)
cv2.imshow("result", result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Threshold Image:
Result Filled Contour On Black Background:
Area Result:
Area = 2347.0

How to order order cv2 contours in a grid which may have some distortion?

I have wrote a python file to detect contours in a cv2 grid and order them by going down the columns from left to right. (See grid1 image below).
This is fairly trivial to sort I have pulled the top left corner of the contour and sorted by its x then by its y coordinate then use the sorted corners to sort the contour list. This works fine when the grid is perfectly straight.
Now if the grid has distortion then this no longer works looking at grid2 we can see that the x coordinate of the top left corner of the piece labelled 2 is less than x coordinate of the topleft corner of the piece labelled 1 (as shown by the green line).
Hence when I apply my sorting function that worked for grid1 it sorts by x then y and consequently the piece labelled 2 is incorrectly ordered to be the first element of the sorted contours instead of the second which it should be.
I am looking for a good method to sort both cases correctly.
Anyone have a suggestion(s)?

You may based your ordering selection based both on the distance of a corner for the origin, and the relative corner position.
Find contours and hierarchy.
Keep contours with no child (based on hierarchy).
Find corners of bounding rectangles.
Analyze the corners based on the following conditions (or find more simple conditions):
Top left contours is the one with the minimum distance of top left corner.
Bottom right contours is the one with the maximum distance of top left corner.
Other two contours can be separated by maximum x and maximum y (after eliminating the top left and bottom right).
The solution below, draws bounding rectangles in colors for testing:
Red
Green
Blue
Yellow
Here is a working code sample (please read the comments):
import numpy as np
import cv2
# Read input image as Grayscale
img = cv2.imread('img.png', cv2.IMREAD_GRAYSCALE)
# Convert img to uint8 binary image with values 0 and 255
# All black pixels goes to 0, and other pixels goes to 255
ret, thresh_gray = cv2.threshold(img, 1, 255, cv2.THRESH_BINARY)
# Find contours in thresh_gray.
cnts, hiers = cv2.findContours(thresh_gray, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)[-2:] # [-2:] indexing takes return value before last (due to OpenCV compatibility issues).
corners = [] # List of corners
dist = np.array([]) # Array of distance from axes origin
# Iterate cnts and hiers, find bounding rectangles, and add corners to a list
for c, h in zip(cnts, hiers[0]):
# If contours has no child
if h[2] == -1:
# Get bounding rectangle
x, y, w, h = cv2.boundingRect(c)
# Append corner to list of corners - format is corners[i] holds a tuple: ((x0, y0), (x1, y1))
p0 = (x, y)
p1 = (x+w, y+h)
corners.append((p0, p1))
# Distance of corners from origin
d = np.array([np.linalg.norm(p0), np.linalg.norm(p1)])
if dist.size == 0:
dist = d
else:
dist = np.vstack((dist, d))
top_left = np.argmin(dist[:,0]) # Index of top left corner (assume minimum distance from origin)
bottom_right = np.argmax(dist[:,1]) # Index of top bottom right corner (assume maximum distance from origin)
tmp_corners = np.array(corners)
tmp_corners[top_left, :, :] = np.array(((0,0), (0,0))) #Ignore top_left corners
tmp_corners[bottom_right, :, :] = np.array(((0,0), (0,0))) #Ignore bottom_right corners
bottom_left = np.argmax(tmp_corners[:,1,1]) #Maximum y is bottom left
tmp_corners[bottom_left, :, :] = np.array(((0,0), (0,0))) #Ignore bottom_left corners
top_right = np.argmax(tmp_corners[:,1,0]) #Maximum x is top right
# Convert Grayscale to BGR (just for testing - for drawing rectangles in green color).
out = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
# Draw rectangles (for testing)
# 1. Red
# 2. Green
# 3. Blue
# 4. Yellow
cv2.rectangle(out, corners[top_left][0], corners[top_left][1], (0, 0, 255), thickness = 2)
cv2.rectangle(out, corners[bottom_left][0], corners[bottom_left][1], (0, 255, 0), thickness = 2)
cv2.rectangle(out, corners[top_right][0], corners[top_right][1], (255, 0, 0), thickness = 2)
cv2.rectangle(out, corners[bottom_right][0], corners[bottom_right][1], (0, 255, 255), thickness = 2)
cv2.imwrite('out.png', out) #Save out to file (for testing).
# Show result (for testing).
cv2.imshow('thresh_gray', thresh_gray)
cv2.imshow('out', out)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:

Convert ellipse in image to circle (warp ellipse to circle like a polygon warping to rectangle)

I have an image of an ellipse and if an image has ellipse i am finding it using findcontours() and then i want to convert this ellipse to a circle.
see the example
and i want transform each of them to like this
First I have applied canny edge detection.Then on this image findcontour() is applied.
I have found the ellipse using findcontours() to get all contours and get the required elliptical contour and then the i am using fitellipse() to get center, rotation angle and major and minor axis of ellipse.
I have then tried to rotate the image by the rotated angle and then scale height and width of image w.r.t minor and major axis(i.e. making major axis and minor axis length same) but also I am not getting the proper circular object image as above. There will be some rotation left/it will still be like an ellipse which will be near to circle or so.
_, contours, _ = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
for c in contours:
if len(c) >= 5:
a = cv2.fitEllipse(c)
(x, y), (MA, ma), angle = a
area = (math.pi * MA * ma)/4
if abs(x-image.shape[0]/2) <= 2 and abs(y-image.shape[1]/2) <= 2 and (area - cv2.contourArea(c)) < 50:
screenCount = c
width, height = MA, ma
centerX, centerY = x, y
ellipseAngle = angle
print(width, height, centerX, centerY, ellipseAngle)
# cv2.drawContours(img, c, -1, (0, 255, 0), 4)
cv2.ellipse(img, a, (0, 0, 255), 2, 8)
cv2.imshow("ellipse", img)
break
img = image.copy()
if ellipseAngle < 90:
rotatedImg = imutils.rotate(img, ellipseAngle)
else:
rotatedImg = imutils.rotate(img, -(ellipseAngle - 90))
Then i have scaled as per major and minor axis
after applying findcontour() i got these 2 contours for 1st image in the post
from these any contour is fine right? i am using the first contour from countour as per the code and fitellipse() gives me this ellipse
EDITED - If there are any better approach to solve this problem it would be helpful.

There are several issues I can see in the code:
You are using an edge detection algorithm, and getting the contours of the result. This is OK in principle, but it leads to an image that has two contours: one for the inner edge and one for the outer edge of the edge detection result. It is easier to just threshold the image and obtain a single edge. Though if the image gets more complex an edge detection might be relevant. Indeed, any one of the two contours you obtain should be useful.
The line if abs(x-image.shape[0]/2) <= 2 and abs(y-image.shape[1]/2) <= 2 and (area - cv2.contourArea(c)) < 50 is very restrictive, it didn't trigger for the second image for me.
Rotating by -(ellipseAngle - 90) if the angle is negative is strange. You should rotate all ellipses in the same way.
The code below produces a circular output for both ellipse images shown in the question. I think that the ellipse parameters determined this way are not very precise, but it looks like they're good enough for this application.
import cv2
import numpy as np
img = cv2.imread('im1.png',0)
_, thresh = cv2.threshold(img, 128, 255, type=cv2.THRESH_BINARY_INV)
_, contours, _ = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
params = cv2.fitEllipse(contours[0])
angle = params[2]
scale = params[1]
scale = scale[0]/scale[1]
M = cv2.getRotationMatrix2D((img.shape[0]/2, img.shape[1]/2), angle, 1)
# Let's add the scaling too:
M[:,0:2] = np.array([[1,0],[0,scale]]) # M[:,0:2]
M[1,2] = M[1,2] * scale # This moves the ellipse so it doesn't end up outside the image (it's not correct to keep the ellipse in the middle of the image)
out = cv2.warpAffine(img, M, img.shape, borderValue=255)
cv2.imshow('out',out)
cv2.waitKey()
Using PyDIP (I'm an author) you can get a more precise measure in the idealized case of the OP by not thresholding, and using the grey-values around the edges of the ellipse to get a more precise fit. We compute the second order central moments of the image, and derive the ellipse parameters from those. It is important here that the background is exactly 0, and that the foreground (ellipse pixels) are uniform in intensity except at the edge, where the intermediate grey-values add information about the sub-pixel location of the edge.
import PyDIP as dip
import numpy as np
img = -dip.ImageRead('im1.png').TensorElement(0) # We use the inverted first channel
params = dip.Moments(img).secondOrder
M = np.array([[params[0],params[2]],[params[2],params[1]]])
d, V = np.linalg.eig(M)
d = np.sqrt(d)
scale = d[0]/d[1]
angle = np.arctan2(V[1,0],V[0,0])
img = dip.Rotation2D(img, -angle)
img = dip.Resampling(img, [scale, 1])
img.Show()

How to find the centre of these sometimes-overlapping circles

As part of a project I'm working on, I need to find the centre-point of some "blobs" in an image using OpenCV with Python.
I'm having a bit of trouble with it, and would truly appreciate any help or insight :)
My current method is to: get the contours of the images, overlay ellipses on those, use the blob detector to find the centre of each of these.
This works fairly well, but occasionally I have extraneous blobs that I need to ignore, and sometimes the blobs are touching each-other.
Here's an example of when it goes well:
Good source image:
After extracting contours:
With the blobs detected:
And when it goes poorly (you can see that it's incorrectly overlayed an ellipse over three blobs, and detected one that I don't want):
Bad source image:
After extracting contours:
With the blobs detected:
This is the code I currently use. I'm unsure of any other option.
def process_and_detect(img_path):
img = cv2.imread(path)
imgray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(imgray, 50, 150, 0)
im2, contours, hierarchy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
drawn_img = np.zeros(img.shape, np.uint8)
min_area = 50
min_ellipses = []
for cnt in contours:
if cv2.contourArea(cnt) >= min_area:
ellipse = cv2.fitEllipse(cnt)
cv2.ellipse(drawn_img,ellipse,(0,255,0),-1)
plot_img(drawn_img, size=12)
# Change thresholds
params = cv2.SimpleBlobDetector_Params()
params.filterByColor = True
params.blobColor = 255
params.filterByCircularity = True
params.minCircularity = 0.75
params.filterByArea = True
params.minArea = 150
# Set up the detector
detector = cv2.SimpleBlobDetector_create(params)
# Detect blobs.
keypoints = detector.detect(drawn_img)
for k in keypoints:
x = round(k.pt[0])
y = round(k.pt[1])
line_length = 20
cv2.line(img, (x-line_length, y), (x+line_length, y), (255, 0, 0), 2)
cv2.line(img, (x, y-line_length), (x, y+line_length), (255, 0, 0), 2)
plot_img(img, size=12)
Thank you so much for reading this far, I sincerely hope someone can help me out, or point me in the right direction. Thanks!

Blob detector
Currently, your implementation is redundant. From the SimpleBlobDetector() docs:
The class implements a simple algorithm for extracting blobs from an image:
Convert the source image to binary images by applying thresholding with several thresholds from minThreshold (inclusive) to maxThreshold (exclusive) with distance thresholdStep between neighboring thresholds.
Extract connected components from every binary image by findContours() and calculate their centers.
Group centers from several binary images by their coordinates. Close centers form one group that corresponds to one blob, which is controlled by the minDistBetweenBlobs parameter.
From the groups, estimate final centers of blobs and their radiuses and return as locations and sizes of keypoints.
So you're implementing part of the steps already, which might give some unexpected behavior. You could try playing with the parameters to see if you can figure out some that work for you (try creating trackbars to play with the parameters and get live results of your algorithm with different blob detector parameters).
Modifying your pipeline
However, you've already got most of your own pipeline written, so you can easily remove the blob detector and implement your own algorithm. If you simply drop your threshold a bit, you can easily get clearly marked circles, and then blob detection is as simple as contour detection. If you have a separate contour for each blob, then you can calculate the centroid of the contour with moments(). For example:
def process_and_detect(img_path):
img = cv2.imread(img_path)
imgray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(imgray, 100, 255, cv2.THRESH_BINARY)
contours = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)[1]
line_length = 20
for c in contours:
if cv2.contourArea(c) >= min_area:
M = cv2.moments(c)
x = int(M['m10']/M['m00'])
y = int(M['m01']/M['m00'])
cv2.line(img, (x-line_length, y), (x+line_length, y), (255, 0, 0), 2)
cv2.line(img, (x, y-line_length), (x, y+line_length), (255, 0, 0), 2)
Getting more involved
This same pipeline can be used to automatically loop through threshold values so you don't have to guess and hardcode those values. Since the blobs all seem roughly the same size, you can loop through until all contours have roughly the same area. You could do this for e.g. by finding the median contour size, defining some percentage of that median size above and below that you'll allow, and checking if all the contours detected fit in those bounds.
Here's an animated gif of what I mean. Notice that the gif stops once the contours are separated:
Then you can simply find the centroids of those separated contours. Here's the code:
def process_and_detect(img_path):
img = cv2.imread(img_path)
imgray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
for thresh_val in range(0, 255):
# threshold and detect contours
thresh = cv2.threshold(imgray, thresh_val, 255, cv2.THRESH_BINARY)[1]
contours = cv2.findContours(thresh,
cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)[1]
# filter contours by area
min_area = 50
filtered_contours = [c for c in contours
if cv2.contourArea(c) >= min_area]
area_contours = [cv2.contourArea(c) for c in filtered_contours]
# acceptable deviation from median contour area
median_area = np.median(area_contours)
dev = 0.3
lowerb = median_area - dev*median_area
upperb = median_area + dev*median_area
# break when all contours are within deviation from median area
if ((area_contours > lowerb) & (area_contours < upperb)).all():
break
# draw center location of blobs
line_length = 8
cross_color = (255, 0, 0)
for c in filtered_contours:
M = cv2.moments(c)
x = int(M['m10']/M['m00'])
y = int(M['m01']/M['m00'])
cv2.line(img, (x-line_length, y), (x+line_length, y), cross_color, 2)
cv2.line(img, (x, y-line_length), (x, y+line_length), cross_color, 2)
Note that here I looped through all possible threshold values with range(0, 255) to give 0, 1, ..., 254 but really you could start higher and skip through a few values at a time with, say, range(50, 200, 5) to get 50, 55, ..., 195 which would of course be much faster.

The "standard" approach for such blob-splitting problem is by means of the watershed transform. It can be applied on the binary image, using a transform distance, or directly on the grayscale image.
Oversegmentation problems can make it tricky, but it seems that your case will not suffer from that.
To find the center, I would usually recommend a weighted average of the pixel coordinates to get a noise reduction effect, but in this case I would probably go for the location of the maximum intensity, which won't be influenced by the deformation of the shape.
Here is what you get with a grayscale watershed (the region intensity is the average). Contrary to what I initially thought, there is some fragmentation due to irregularities in the blobs
You can improve with a little of lowpass filtering before segmentation.