Related
Let's imagine the following newspaper article needs to be analyzed regarding the amount of columns (the solution should be 3 text columns). I tried to retrieve the amount of columns using the cv2 library with python and found the following suggestion on StackOverflow: Detect number of rows and columns in table image with OpenCV
However, as the table of that solution is well structured, the amount of columns and rows can be extracted quite easily. Based on that solution, here is what I came up with:
import numpy as np
from imutils import contours
import cv2
# Load image, grayscale, Gaussian blur, Otsu's threshold
image = cv2.imread('example_newspaper_article.png')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (5,5), 0)
thresh = cv2.threshold(blur, 240, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Find contours and remove text inside cells
cnts = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
area = cv2.contourArea(c)
if area < 10000:
cv2.drawContours(thresh, [c], -1, (255, 255, 255), 30)
# Invert image
invert = thresh
offset, old_cY, first = 10, 0, True
visualize = cv2.cvtColor(invert, cv2.COLOR_GRAY2BGR)
# Find contours, sort from top-to-bottom and then sum up column/rows
cnts = cv2.findContours(invert, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
(cnts, _) = contours.sort_contours(cnts, method="top-to-bottom")
for c in cnts:
# Find centroid
M = cv2.moments(c)
cX = int(M["m10"] / M["m00"])
cY = int(M["m01"] / M["m00"])
# New row
if (abs(cY) - abs(old_cY)) > offset:
if first:
row, table = [], []
first = False
old_cY = cY
table.append(row)
row = []
# Cell in same row
if ((abs(cY) - abs(old_cY)) <= offset) or first:
row.append(1)
# Uncomment to visualize
#cv2.circle(visualize, (cX, cY), 10, (36, 255, 12), -1)
#cv2.imshow('visualize', visualize)
#cv2.waitKey(200)
print('Rows: {}'.format(len(table)))
print('Columns: {}'.format(len(table[1])))
cv2.imshow('invert', invert)
cv2.imshow('thresh', thresh)
cv2.waitKey()
I thought, that increasing the thickness argument of the drawContours-method would help somehow, but unfortunately that does not do the trick. The result looks like this:
I assume, that drawing rectangles over the text area would be more helpful?
Does anyone know a solution and could help me out?
Thanks in advance!
Whenever there's such a task, I tend to count pixels along the y-axis, and try to find (large) differences between neighbouring columns. That'd be my complete pipeline:
Convert image to grayscale; inverse binary threshold using Otsu's to get white pixels on black background.
Do some morphological closing, here using a large vertical line kernel to connect all pixels in the same column.
Count all white pixels; calculate the absolute difference between neighbouring columns.
Find peaks in that "signal" – manually or, as shown here, by using scipy.signal.find_peaks. The peaks identify the start and end of each text column, so the number of text columns is half the number of peaks.
Here's the whole code including some visualization:
import cv2
import matplotlib.pyplot as plt # Only for visualization output
import numpy as np
from scipy import signal
from skimage import io # Only for web grabbing images
# Read image from web (attention: RGB order here, scikit-image)
image = io.imread('https://i.stack.imgur.com/jbAeZ.png')
# Convert image to grayscale
gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
# Inverse binary threshold by Otsu's
thr = cv2.threshold(gray, 0, 255, cv2.THRESH_OTSU + cv2.THRESH_BINARY_INV)[1]
# Morphological closing with large vertical line kernel
thr_mod = cv2.morphologyEx(thr, cv2.MORPH_CLOSE, np.ones((image.shape[0], 1)))
# Count white pixels along y-axis
y_count = np.sum(thr_mod / 255, 0)
# Calculate absolute difference between neighbouring x-axis values
y_count_diff = np.abs(np.diff(y_count))
# Find peaks in that "signal"
peaks = signal.find_peaks(y_count_diff, distance=50)[0]
# Number of columns is half the number of found peaks
n_cols = np.int(peaks.shape[0] / 2)
# Text output
print('Number of columns: ' + str(n_cols))
# Some visualization output
plt.figure(0)
plt.subplot(221)
plt.imshow(image)
plt.title('Original image')
plt.subplot(222)
plt.imshow(thr_mod, cmap='gray')
plt.title('Thresholded, morphlogically closed image')
plt.subplot(223)
plt.plot(y_count)
plt.plot(peaks, y_count[peaks], 'r.')
plt.title('Summed white pixels along y-axis')
plt.subplot(224)
plt.plot(y_count_diff)
plt.plot(peaks, y_count_diff[peaks], 'r.')
plt.title('Absolute difference in summed white pixels')
plt.tight_layout()
plt.show()
The textual output:
Number of columns: 3
The visualization output:
Limitations: If your image is tilted, etc. you might get bad results. If you have a lot of (large) images crossing text columns, you also might get bad results. In general, you'll need to adapt the details in the given implementation to meet your actual requirements (no more examples were given).
----------------------------------------
System information
----------------------------------------
Platform: Windows-10-10.0.16299-SP0
Python: 3.8.5
Matplotlib: 3.3.1
NumPy: 1.19.1
OpenCV: 4.4.0
SciPy: 1.5.2
----------------------------------------
You could prep the image a little differently before searching for columns. For example you can connect the text horizontally first (with some morphological operation). That will give you contours with a certain height (the heading will be connected vertically as one contour per row and the text in columns will be connected as one contour per row). Then search for all contours and draw bounding rectangles over the ones that are higher than certain value you set (can be calculated or set manually). After that perform the morphological operation again with bigger kernel (horizontal and vertical) so you get all the remaining text connected if it is close together.
Here is an example code:
import cv2
import numpy as np
img = cv2.imread("columns.png") # read image
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # grayscale transform
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)[1] # OTSU thresold
kernel = np.ones((5, 10), dtype=np.uint8) # kernel for first closing procedure (connect blobs in x direction)
closing = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel) # closing
cv2.imwrite("closing1.png", closing)
contours = cv2.findContours(closing, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0] # search for contours
heights = [] # all of contours heights
for cnt in contours:
x, y, w, h = cv2.boundingRect(cnt) # bounding rectangles height, width and coordinates
heights.append(h) # append height of one contours
boundary = np.mean(heights, axis=0) # mean of heights will serve as boundary but
# this will probably not be the case on other samples - you would need to make
# a function to determin this boundary or manualy set it
# iterate through contours
for cnt in contours:
x, y, w, h = cv2.boundingRect(cnt) # bounding rectangles height, width and coordinates
if h > boundary: # condition - contour must be higher than height boundary
cv2.rectangle(closing, (x, y), (x+w, y+h), (0, 0, 0), -1) # draw filled rectangle on the closing image
cv2.imwrite("closing1-filled.png", closing)
kernel = np.ones((25, 25), dtype=np.uint8) # kernel for second closing (connect blobs in x and y direction)
closing = cv2.morphologyEx(closing, cv2.MORPH_CLOSE, kernel) # closing again
cv2.imwrite("closing2.png", closing)
contours = cv2.findContours(closing, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0] # search for contours again
# iterate through contours
print("Number of columns: ", len(contours)) # this is the number of columns
for cnt in contours:
x, y, w, h = cv2.boundingRect(cnt) # this are height, width and coordinates of the columns
cv2.rectangle(img, (x, y), (x+w, y+h), (0, 255, 0), 3) # draw bouning rectangle on original image
cv2.imwrite("result.png", img)
cv2.imshow("img", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
Number of columns: 3
Step 1:
Step 2:
Step 3:
I'm currently working on an algorithm to detect bacterial centroids in microscopy images.
This question is a continuation of: OpenCV/Python — Matching Centroid Points of Bacteria in Two Images: Python/OpenCV — Matching Centroid Points of Bacteria in Two Images
I am using a modified version of the program proposed by Rahul Kedia.
https://stackoverflow.com/a/63049277/13696853
Currently, the issues in segmentation I am working on are:
Low Contrast
Clustering
The images below are sampled a second apart. However, in the latter image, one of the bacteria does not get detected.
Bright-field Image #1
Bright-Field Image #2
Bright-Field Contour Image #1
Bright-Field Contour Image #2
Bright-Field Image #1 (Unsegmented)
Bright-Field Image #2 (Unsegmented)
I want to know, given that I can successfully determine bacterial centroids in an image, can I use the data to intelligently look for the same bacteria in the subsequent image?
I haven't been able to find anything substantial online; I believe SIFT/SURF would likely be ineffective as the bacteria have the same appearance. Moreover, I am looking for specific points in the images. You can view my program below. Insert a specific path as indicated if you'd like to run the program.
import cv2
import numpy as np
import os
kernel = np.array([[0, 0, 1, 0, 0],
[0, 1, 1, 1, 0],
[1, 1, 1, 1, 1],
[0, 1, 1, 1, 0],
[0, 0, 1, 0, 0]], dtype=np.uint8)
def e_d(image, it):
image = cv2.erode(image, kernel, iterations=it)
image = cv2.dilate(image, kernel, iterations=it)
return image
path = r"[INSERT PATH]"
img_files = [file for file in os.listdir(path)]
def segment_index(index: int):
segment_file(img_files[index])
def segment_file(img_file: str):
img_path = path + "\\" + img_file
print(img_path)
img = cv2.imread(img_path)
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# Applying adaptive mean thresholding
th = cv2.adaptiveThreshold(img, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY_INV, 11, 2)
# Removing small noise
th = e_d(th.copy(), 1)
# Finding contours with RETR_EXTERNAL flag and removing undesired contours and
# drawing them on a new image.
cnt, hie = cv2.findContours(th, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cntImg = th.copy()
for contour in cnt:
x, y, w, h = cv2.boundingRect(contour)
# Eliminating the contour if its width is more than half of image width
# (bacteria will not be that big).
if w > img.shape[1] / 2:
continue
cntImg = cv2.drawContours(cntImg, [cv2.convexHull(contour)], -1, 255, -1)
# Removing almost all the remaining noise.
# (Some big circular noise will remain along with bacteria contours)
cntImg = e_d(cntImg, 3)
# Finding new filtered contours again
cnt2, hie2 = cv2.findContours(cntImg, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
# Now eliminating circular type noise contours by comparing each contour's
# extent of overlap with its enclosing circle.
finalContours = [] # This will contain the final bacteria contours
for contour in cnt2:
# Finding minimum enclosing circle
(x, y), radius = cv2.minEnclosingCircle(contour)
center = (int(x), int(y))
radius = int(radius)
# creating a image with only this circle drawn on it(filled with white colour)
circleImg = np.zeros(img.shape, dtype=np.uint8)
circleImg = cv2.circle(circleImg, center, radius, 255, -1)
# creating a image with only the contour drawn on it(filled with white colour)
contourImg = np.zeros(img.shape, dtype=np.uint8)
contourImg = cv2.drawContours(contourImg, [contour], -1, 255, -1)
# White pixels not common in both contour and circle will remain white
# else will become black.
union_inter = cv2.bitwise_xor(circleImg, contourImg)
# Finding ratio of the extent of overlap of contour to its enclosing circle.
# Smaller the ratio, more circular the contour.
ratio = np.sum(union_inter == 255) / np.sum(circleImg == 255)
# Storing only non circular contours(bacteria)
if ratio > 0.55:
finalContours.append(contour)
finalContours = np.asarray(finalContours)
# Finding center of bacteria and showing it.
bacteriaImg = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
for bacteria in finalContours:
M = cv2.moments(bacteria)
cx = int(M['m10'] / M['m00'])
cy = int(M['m01'] / M['m00'])
bacteriaImg = cv2.circle(bacteriaImg, (cx, cy), 5, (0, 0, 255), -1)
cv2.imshow("bacteriaImg", bacteriaImg)
cv2.waitKey(0)
# Segment Each Image
for i in range(len(img_files)):
segment_index(i)
Edit #1: Applying frmw42's approach, this image seems to get lost. I have tried adjusting a number of parameters but the image does not seem to show up.
Bright-Field Image #3
Bright-Field Image #4
Here is my Python/OpenCV code to extract your bacteria. I simply threshold, then get the contours and draw filled contours for those within a certain area range. I will let you do any further processing that you want. I simply viewed each step to make sure I have tuned the arguments appropriately before moving to the next step.
Input 1:
Input 2:
import cv2
import numpy as np
# read image
#img = cv2.imread("bacteria1.png")
img = cv2.imread("bacteria2.png")
# convert img to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gray = 255 - gray
# do adaptive threshold on inverted gray image
thresh = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 21, 5)
result = np.zeros_like(img)
contours = cv2.findContours(thresh , cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for cntr in contours:
area = cv2.contourArea(cntr)
if area > 600 and area < 1100:
cv2.drawContours(result, [cntr], 0, (255,255,255), -1)
# write results to disk
#cv2.imwrite("bacteria_filled_contours1.png", result)
cv2.imwrite("bacteria_filled_contours2.png", result)
# display it
cv2.imshow("thresh", thresh)
cv2.imshow("result", result)
cv2.waitKey(0)
Result 1:
Result 2:
Adjust as desired.
It would seem that adaptive threshold is not able to handle all your various images. I suspect nothing simple will. You may need to use AI with training. Nevertheless, this works for your images: 1, 2 and 4 in Python/OpenCV. I make no guarantee that it will work for any of your other images.
First I found a simple threshold that seems to work, but brings in other regions. So since all your bacteria have similar shapes and range of orientations, I fit and ellipse to your bacteria and get the orientation of the major axis and filter the contours with area and angle.
import cv2
import numpy as np
# read image
#img = cv2.imread("bacteria1.png")
#img = cv2.imread("bacteria2.png")
img = cv2.imread("bacteria4.png")
# convert img to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gray = 255 - gray
# median filter
#gray = cv2.medianBlur(gray, 1)
# do simple threshold on inverted gray image
thresh = cv2.threshold(gray, 170, 255, cv2.THRESH_BINARY)[1]
result = np.zeros_like(img)
contours = cv2.findContours(thresh , cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for cntr in contours:
area = cv2.contourArea(cntr)
if area > 600 and area < 1100:
ellipse = cv2.fitEllipse(cntr)
(xc,yc),(d1,d2),angle = ellipse
if angle > 90:
angle = angle - 90
else:
angle = angle + 90
print(angle,area)
if angle >= 150 and angle <= 250:
cv2.drawContours(result, [cntr], 0, (255,255,255), -1)
# write results to disk
#cv2.imwrite("bacteria_filled_contours1.png", result)
#cv2.imwrite("bacteria_filled_contours2.png", result)
cv2.imwrite("bacteria_filled_contours4.png", result)
# display it
cv2.imshow("thresh", thresh)
cv2.imshow("result", result)
cv2.waitKey(0)
Result for image 1:
Result for image 2:
Result for image 4:
You might explore noise reduction before thresholding. I had some success with using some of ImageMagick tools and there is a Python version called Python Wand that uses ImageMagick.
I have pictures of apple slices that have been soaked in an iodine solution. The goal is to segment the apples into individual regions of interest and evaluate the starch level of each one. This is for a school project so my goal is to test different methods of segmentation and objectively find the best solution whether it be a single technique or a combination of multiple techniques.
The problem is that so far I have only come close on one method. That method is using HoughCircles. I had originally planned to use the Watershed method, Morphological operations, or simple thresholding. This plan failed when I couldn't modify any of them to work.
The original images look similar to this, with varying shades of darkness of the apple
I've tried removing the background tray using cv2.inRange with HSV values, but it doesn't work well with darker apples.
This is what the HoughCircles produced on the original image with a grayscale and median blur applied, also with an attempted mask of the tray.
Any advice or direction on where to look next would be greatly appreciated. I can supply the code I'm using if that will help.
Thank you!
EDIT 1 : Adding some code and clarifying the question
Thank you for the responses. My real question is are there any other methods of segmentation that this scenario lends itself well to? I would like to try a couple different methods and compare results on a large set of photos. My next in line to try is using k-means segmentation. Also I'll add some code below to show what I've tried so far.
HSV COLOR FILTERING
import cv2
import numpy as np
# Load image
image = cv2.imread('ApplePic.jpg')
# Set minimum and max HSV values to display
lower = np.array([0, 0, 0])
upper = np.array([105, 200, 255])
# Create HSV Image and threshold into a range.
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, lower, upper)
maskedImage = cv2.bitwise_and(image, image, mask=mask)
# Show Image
cv2.imshow('HSV Mask', image)
cv2.waitKey(0)
HoughCircles
# import the necessary packages
import numpy as np
import argparse
import cv2
import os
directory = os.fsencode('Photos\\Sample N 100')
for file in os.listdir(directory):
filename = os.fsdecode(file)
if filename.endswith('.jpg'):
# Load the image
image = cv2.imread('Photos\\Sample N 100\\' + filename)
# Calculate scale
scale_factor = 800 / image.shape[0]
width = int(image.shape[1] * scale_factor)
height = 800
dimension = (width, height)
min_radius = int((width / 10) * .8)
max_radius = int((width / 10) * 1.2)
# Resize image
image = cv2.resize(image, dimension, interpolation=cv2.INTER_AREA)
# Copy Image
output = image.copy()
# Grayscale Image
gray = cv2.medianBlur(cv2.cvtColor(image, cv2.COLOR_BGR2GRAY), 5)
# Detect circles in image
circles = cv2.HoughCircles(gray, cv2.HOUGH_GRADIENT, 1, min_radius * 2, 4, 60, 20, min_radius, max_radius)
# ensure at least some circles were found
if circles is not None:
# convert the (x, y) coordinates and radius of the circles to integers
circles = np.round(circles[0, :]).astype("int")
# loop over the (x, y) coordinates and radius of the circles
for (x, y, r) in circles:
# draw the circle in the output image, then draw a rectangle
# corresponding to the center of the circle
cv2.circle(output, (x, y), r, (0, 255, 0), 4)
cv2.rectangle(output, (x - 5, y - 5), (x + 5, y + 5), (0, 128, 255), -1)
cv2.putText(output, '(' + str(x) + ',' + str(y) + ',' + str(r) + ')', (x, y),
cv2.FONT_HERSHEY_COMPLEX_SMALL, 1, 255)
# show the output image
cv2.imshow("output", np.hstack([image, output, maskedImage]))
cv2.waitKey(0)
continue
else:
continue
An alternative approach to segmenting the apples is to perform Kmeans color segmentation before thresholding then using contour filtering to isolate the apple objects:
Apply Kmeans color segmentation. We load the image, resize smaller using imutils.resize then apply Kmeans color segmentation. Depending on the number of clusters, we can segment the image into the desired number of colors.
Obtain binary image. Next we convert to grayscale, Gaussian blur and Otsu's threshold.
Filter using contour approximation. We filter out non-circle contours and small noise.
Morphological operations. We perform a morph close to fill adjacent contours
Draw minimum enclosing circles using contour area as filter. We find contours and draw the approximated circles. For this we use two sections, one where there was a good threshold and another where we approximate the radius.
Kmeans color quantization with clusters=3 and binary image
Morph close and result
The "good" contours that had the radius automatically calculated using cv2.minEnclosingCircle is highlighted in green while the approximated contours are highlighted in teal. These approximated contours were not segmented well from the thresholding process so we average the "good" contours radius and use that to draw the circle.
Code
import cv2
import numpy as np
import imutils
# Kmeans color segmentation
def kmeans_color_quantization(image, clusters=8, rounds=1):
h, w = image.shape[:2]
samples = np.zeros([h*w,3], dtype=np.float32)
count = 0
for x in range(h):
for y in range(w):
samples[count] = image[x][y]
count += 1
compactness, labels, centers = cv2.kmeans(samples,
clusters,
None,
(cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10000, 0.0001),
rounds,
cv2.KMEANS_RANDOM_CENTERS)
centers = np.uint8(centers)
res = centers[labels.flatten()]
return res.reshape((image.shape))
# Load image, resize smaller, perform kmeans, grayscale
# Apply Gaussian blur, Otsu's threshold
image = cv2.imread('1.jpg')
image = imutils.resize(image, width=600)
kmeans = kmeans_color_quantization(image, clusters=3)
gray = cv2.cvtColor(kmeans, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (9,9), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
# Filter out contours not circle
cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.04 * peri, True)
if len(approx) < 4:
cv2.drawContours(thresh, [c], -1, 0, -1)
# Morph close
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
close = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel, iterations=3)
# Find contours and draw minimum enclosing circles
# using contour area as filter
approximated_radius = 63
cnts = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
area = cv2.contourArea(c)
x,y,w,h = cv2.boundingRect(c)
# Large circles
if area > 6000 and area < 15000:
((x, y), r) = cv2.minEnclosingCircle(c)
cv2.circle(image, (int(x), int(y)), int(r), (36, 255, 12), 2)
# Small circles
elif area > 1000 and area < 6000:
((x, y), r) = cv2.minEnclosingCircle(c)
cv2.circle(image, (int(x), int(y)), approximated_radius, (200, 255, 12), 2)
cv2.imshow('kmeans', kmeans)
cv2.imshow('thresh', thresh)
cv2.imshow('close', close)
cv2.imshow('image', image)
cv2.waitKey()
I am tasked to build a license plate detection system and my code does not work if the plate has the same colour of the paint of car (background).
Take a look at this picture below.
I have tried a variety of edge detection technique and my findings are they hardly work.
Here is my image processing pipeline:
Extract the gray channel from the image.
Reduce noise with Iterative Bilaterial Filtering
Detect edges with Adaptive Thresholding
Dilate the edges slightly
Locate contours based on some heuristics.
The edge detection part performed miserably around the plate region.
The pipeline works good and I am able to detect license plates if the car is has a different paint colour than the plate.
Code
def rectangleness(hull):
rect = cv2.boundingRect(hull)
rectPoints = np.array([[rect[0], rect[1]],
[rect[0] + rect[2], rect[1]],
[rect[0] + rect[2], rect[1] + rect[3]],
[rect[0], rect[1] + rect[3]]])
intersection_area = cv2.intersectConvexConvex(np.array(rectPoints), hull)[0]
rect_area = cv2.contourArea(rectPoints)
rectangleness = intersection_area/rect_area
return rectangleness
def preprocess(image):
image = imutils.resize(image, 1000)
# Attenuate shadows by using H channel instead of converting to gray directly
imgHSV = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
_, _, gray = cv2.split(imgHSV)
# Reduce noise while preserve edge with Iterative Bilaterial Filtering
blur = cv2.bilateralFilter(gray, 11, 6, 6)
# Detect edges by thresholding
edge = cv2.adaptiveThreshold(blur, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 11, 5)
# Dilate edges, kernel size cannot be too big as some fonts are very closed to the edge of the plates
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2, 2))
dilated = cv2.dilate(edge, kernel)
# Detect contours
edge, contours, _ = cv2.findContours(dilated, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
# Loop through contours and select the most probable ones
contours = sorted(contours, key = cv2.contourArea, reverse=True)[:10]
for contour in contours:
perimeter = cv2.arcLength(contour, closed=True)
approximate = cv2.approxPolyDP(contour, 0.02*perimeter, closed=True)
if len(approximate) == 4:
(x, y, w, h) = cv2.boundingRect(approximate)
whRatio = w / h
# Heuristics:
# 1. Width of plate should at least be 2x greater than height
# 2. Width of contour should be more than 5 (eliminate false positive)
# 3. Height must not be too small
# 4. Polygon must resemble a rectangle
if (2.0 < whRatio < 6.0) and (w > 5.0) and (h > 20):
hull = cv2.convexHull(approximate, returnPoints=True)
if rectangleness(hull) > 0.75:
print("X Y {} {}".format(x, y))
print("Height: {}".format(h))
print("Width : {}".format(w))
print("Ratio : {}\n".format(w/h))
cv2.drawContours(image, [approximate], -1, (0, 255, 0), 2)
cv2.imshow("Edge", edge)
cv2.imshow("Frame", image)
cv2.waitKey(0)
cv2.destroyAllWindows()
You can use cv2.morphologyEx for making the plate region become more visible. Next step is to find contours and set reasonable conditions to extract the contour that contains the plate. If you want, you can have a look at this github repository where my friend and I show detailed steps about license plate detection and recognition.
import cv2
import numpy as np
img = cv2.imread("a.png")
imgBlurred = cv2.GaussianBlur(img, (7, 7), 0)
gray = cv2.cvtColor(imgBlurred, cv2.COLOR_BGR2GRAY) # convert to gray
sobelx = cv2.Sobel(gray, cv2.CV_8U, 1, 0, ksize=3) # sobelX to get the vertical edges
ret,threshold_img = cv2.threshold(sobelx, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)
morph_img_threshold = threshold_img.copy()
element = cv2.getStructuringElement(shape=cv2.MORPH_RECT, ksize=(22, 3))
cv2.morphologyEx(src=threshold_img, op=cv2.MORPH_CLOSE, kernel=element,
dst=morph_img_threshold)
cv2.imshow("img", img)
cv2.imshow("sobelx", sobelx)
cv2.imshow("morph_img_threshold", morph_img_threshold)
cv2.waitKey()
cv2.destroyAllWindows()
As part of a project I'm working on, I need to find the centre-point of some "blobs" in an image using OpenCV with Python.
I'm having a bit of trouble with it, and would truly appreciate any help or insight :)
My current method is to: get the contours of the images, overlay ellipses on those, use the blob detector to find the centre of each of these.
This works fairly well, but occasionally I have extraneous blobs that I need to ignore, and sometimes the blobs are touching each-other.
Here's an example of when it goes well:
Good source image:
After extracting contours:
With the blobs detected:
And when it goes poorly (you can see that it's incorrectly overlayed an ellipse over three blobs, and detected one that I don't want):
Bad source image:
After extracting contours:
With the blobs detected:
This is the code I currently use. I'm unsure of any other option.
def process_and_detect(img_path):
img = cv2.imread(path)
imgray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(imgray, 50, 150, 0)
im2, contours, hierarchy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
drawn_img = np.zeros(img.shape, np.uint8)
min_area = 50
min_ellipses = []
for cnt in contours:
if cv2.contourArea(cnt) >= min_area:
ellipse = cv2.fitEllipse(cnt)
cv2.ellipse(drawn_img,ellipse,(0,255,0),-1)
plot_img(drawn_img, size=12)
# Change thresholds
params = cv2.SimpleBlobDetector_Params()
params.filterByColor = True
params.blobColor = 255
params.filterByCircularity = True
params.minCircularity = 0.75
params.filterByArea = True
params.minArea = 150
# Set up the detector
detector = cv2.SimpleBlobDetector_create(params)
# Detect blobs.
keypoints = detector.detect(drawn_img)
for k in keypoints:
x = round(k.pt[0])
y = round(k.pt[1])
line_length = 20
cv2.line(img, (x-line_length, y), (x+line_length, y), (255, 0, 0), 2)
cv2.line(img, (x, y-line_length), (x, y+line_length), (255, 0, 0), 2)
plot_img(img, size=12)
Thank you so much for reading this far, I sincerely hope someone can help me out, or point me in the right direction. Thanks!
Blob detector
Currently, your implementation is redundant. From the SimpleBlobDetector() docs:
The class implements a simple algorithm for extracting blobs from an image:
Convert the source image to binary images by applying thresholding with several thresholds from minThreshold (inclusive) to maxThreshold (exclusive) with distance thresholdStep between neighboring thresholds.
Extract connected components from every binary image by findContours() and calculate their centers.
Group centers from several binary images by their coordinates. Close centers form one group that corresponds to one blob, which is controlled by the minDistBetweenBlobs parameter.
From the groups, estimate final centers of blobs and their radiuses and return as locations and sizes of keypoints.
So you're implementing part of the steps already, which might give some unexpected behavior. You could try playing with the parameters to see if you can figure out some that work for you (try creating trackbars to play with the parameters and get live results of your algorithm with different blob detector parameters).
Modifying your pipeline
However, you've already got most of your own pipeline written, so you can easily remove the blob detector and implement your own algorithm. If you simply drop your threshold a bit, you can easily get clearly marked circles, and then blob detection is as simple as contour detection. If you have a separate contour for each blob, then you can calculate the centroid of the contour with moments(). For example:
def process_and_detect(img_path):
img = cv2.imread(img_path)
imgray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(imgray, 100, 255, cv2.THRESH_BINARY)
contours = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)[1]
line_length = 20
for c in contours:
if cv2.contourArea(c) >= min_area:
M = cv2.moments(c)
x = int(M['m10']/M['m00'])
y = int(M['m01']/M['m00'])
cv2.line(img, (x-line_length, y), (x+line_length, y), (255, 0, 0), 2)
cv2.line(img, (x, y-line_length), (x, y+line_length), (255, 0, 0), 2)
Getting more involved
This same pipeline can be used to automatically loop through threshold values so you don't have to guess and hardcode those values. Since the blobs all seem roughly the same size, you can loop through until all contours have roughly the same area. You could do this for e.g. by finding the median contour size, defining some percentage of that median size above and below that you'll allow, and checking if all the contours detected fit in those bounds.
Here's an animated gif of what I mean. Notice that the gif stops once the contours are separated:
Then you can simply find the centroids of those separated contours. Here's the code:
def process_and_detect(img_path):
img = cv2.imread(img_path)
imgray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
for thresh_val in range(0, 255):
# threshold and detect contours
thresh = cv2.threshold(imgray, thresh_val, 255, cv2.THRESH_BINARY)[1]
contours = cv2.findContours(thresh,
cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)[1]
# filter contours by area
min_area = 50
filtered_contours = [c for c in contours
if cv2.contourArea(c) >= min_area]
area_contours = [cv2.contourArea(c) for c in filtered_contours]
# acceptable deviation from median contour area
median_area = np.median(area_contours)
dev = 0.3
lowerb = median_area - dev*median_area
upperb = median_area + dev*median_area
# break when all contours are within deviation from median area
if ((area_contours > lowerb) & (area_contours < upperb)).all():
break
# draw center location of blobs
line_length = 8
cross_color = (255, 0, 0)
for c in filtered_contours:
M = cv2.moments(c)
x = int(M['m10']/M['m00'])
y = int(M['m01']/M['m00'])
cv2.line(img, (x-line_length, y), (x+line_length, y), cross_color, 2)
cv2.line(img, (x, y-line_length), (x, y+line_length), cross_color, 2)
Note that here I looped through all possible threshold values with range(0, 255) to give 0, 1, ..., 254 but really you could start higher and skip through a few values at a time with, say, range(50, 200, 5) to get 50, 55, ..., 195 which would of course be much faster.
The "standard" approach for such blob-splitting problem is by means of the watershed transform. It can be applied on the binary image, using a transform distance, or directly on the grayscale image.
Oversegmentation problems can make it tricky, but it seems that your case will not suffer from that.
To find the center, I would usually recommend a weighted average of the pixel coordinates to get a noise reduction effect, but in this case I would probably go for the location of the maximum intensity, which won't be influenced by the deformation of the shape.
Here is what you get with a grayscale watershed (the region intensity is the average). Contrary to what I initially thought, there is some fragmentation due to irregularities in the blobs
You can improve with a little of lowpass filtering before segmentation.