I want to get the time from an analog clock. Right now I'm stuck a bit, I managed to get the segmented image (altough I couldn't remove the bottom part of it...), and did a Canny detection. The problem I have is, well the bottom part I couldn't remove, and the detection of the clock hands. My goal is to detect the hands in a way I can calculate the angles and then the time from those angles. I know that I need Hough Line Transform, but I don't really understand how it works, how to set the parameters.
The original, segmented and the Canny detected pictures:
This is the code I'm using to get there:
img = cv2.imread('clock.jpg')
cv2.imshow('img', img)
cv2.waitKey(0)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blur = cv2.medianBlur(gray, 5)
cv2.imshow('blur', blur)
cv2.waitKey(0)
circles = cv2.HoughCircles(blur, cv2.HOUGH_GRADIENT, 1, 20, param1=20, param2=100, minRadius=0, maxRadius=0)
detected_cricles = np.uint16(np.around(circles))
circle = detected_cricles[0][0]
x = circle[0]
y = circle[1]
r = circle[2]
rect = (x - r, y - r, x+r, y+(r-10))
mask = np.zeros(img.shape[:2], dtype = np.uint8)
bgdModel = np.zeros((1,65), np.float64)
fgdModel = np.zeros((1,65), np.float64)
cv2.grabCut(img, mask, rect, bgdModel, fgdModel, 1, cv2.GC_INIT_WITH_RECT)
mask2 = np.where((mask == 1) + (mask == 3), 255, 0).astype('uint8')
segmented = cv2.bitwise_and(img, img, mask=mask2)
cv2.imshow('segmented', segmented)
cv2.waitKey(0)
blur = cv2.GaussianBlur(segmented, (11,11), 0)
cv2.imshow('blur2', blur)
cv2.waitKey(0)
canny = cv2.Canny(blur, 30, 150, None, 3)
cv2.imshow('canny', canny)
cv2.waitKey(0)
Here is one way using HoughLinesP in Python/OpenCV. The approach uses thresholding, contours and thinning before getting the Hough Lines. I will leave it to you to compute the angles from the line end points.
Input:
import cv2
import numpy as np
from skimage.morphology import skeletonize
# Read image
img = cv2.imread('clock.jpg')
hh, ww = img.shape[:2]
# convert to gray
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# threshold
thresh = cv2.threshold(gray, 128, 255, cv2.THRESH_BINARY)[1]
# invert so shapes are white on black background
thresh = 255 - thresh
# get contours and save area
cntrs_info = []
contours = cv2.findContours(thresh, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
index=0
for cntr in contours:
area = cv2.contourArea(cntr)
cntrs_info.append((index,area))
index = index + 1
# sort contours by area
def takeSecond(elem):
return elem[1]
cntrs_info.sort(key=takeSecond, reverse=True)
# get third largest contour
arms = np.zeros_like(thresh)
index_third = cntrs_info[2][0]
cv2.drawContours(arms,[contours[index_third]],0,(1),-1)
#arms=cv2.ximgproc.thinning(arms)
arms_thin = skeletonize(arms)
arms_thin = (255*arms_thin).clip(0,255).astype(np.uint8)
# get hough lines and draw on copy of input
result = img.copy()
lineThresh = 15
minLineLength = 20
maxLineGap = 100
max
lines = cv2.HoughLinesP(arms_thin, 1, np.pi/180, lineThresh, None, minLineLength, maxLineGap)
for [line] in lines:
x1 = line[0]
y1 = line[1]
x2 = line[2]
y2 = line[3]
cv2.line(result, (x1,y1), (x2,y2), (0,0,255), 2)
# save results
cv2.imwrite('clock_thresh.jpg', thresh)
cv2.imwrite('clock_arms.jpg', (255*arms).clip(0,255).astype(np.uint8))
cv2.imwrite('clock_arms_thin.jpg', arms_thin)
cv2.imwrite('clock_lines.jpg', result)
cv2.imshow('thresh', thresh)
cv2.imshow('arms', (255*arms).clip(0,255).astype(np.uint8))
cv2.imshow('arms_thin', arms_thin)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Thresholded image:
Contour of arms:
Thinned (skeleton):
Hough Line Segments on input:
Here's another possible solution. We will try to segment the clocks hands and run them through Hough's line transform to detect the lines. Now, this detection will yield all the possible straight lines that pass through the clock hands' pixels - producing multiple lines. You can try to play with the line transform parameters to narrow the result to the target lines, but you will probably end up with a cluster of lines. I will try to cluster these lines using K-Means to get only two lines regardless of the output of Hough's line transform. These are the steps:
Get a binary mask of the image to isolate the clock hands
Apply some morphology to get rid of the noise
Run the binary mask through Hough's line detection
Use K-means on the multiple lines to get only 2 (average) lines (one per clock hand)
Let's see the code:
# Imports
import cv2
import numpy as np
# Read image
imagePath = "D://opencvImages//"
inputImage = cv2.imread(imagePath+"orFGl.jpg")
# Store deep copy for results:
originalImg = inputImage.copy()
# Convert BGR back to grayscale:
grayInput = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Threshold via Otsu + bias adjustment:
threshValue, binaryImage = cv2.threshold(grayInput, 0, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
The first bit is trivial and produces this binary mask:
We can get rid of the small elements via some morphology. Let's apply an erosion followed by a dilation to filter everything but the larger components - the clock hands:
# Set morph operation iterations:
opIterations = 1
# Get the structuring element:
structuringElement = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 3))
# Perform Erode:
erodeImg = cv2.morphologyEx(binaryImage, cv2.MORPH_ERODE, structuringElement, None, None, opIterations, cv2.BORDER_REFLECT101)
# Perform Dilate:
dilateImg = cv2.morphologyEx(erodeImg, cv2.MORPH_DILATE, structuringElement, None, None, opIterations, cv2.BORDER_REFLECT101)
This produces this image:
Very nice, almost all the noise is gone. Let's run this directly through the line detection and check out what kind of results we get. Additionally, I've prepared some lists to store every starting (x1, y1) and ending (x2, y2) point of the lines:
# Set HoughLinesP parameters:
lineThresh = 50
minLineLength = 20
maxLineGap = 100
# Run the line detection:
lines = cv2.HoughLinesP(dilateImg, 1, np.pi/180, lineThresh, None, minLineLength, maxLineGap)
# Prepare some lists to store every coordinate of the detected lines:
X1 = []
X2 = []
Y1 = []
Y2 = []
# Store and draw the lines:
for [currentLine] in lines:
# First point:
x1 = currentLine[0]
y1 = currentLine[1]
X1.append(x1)
Y1.append(y1)
# Second point:
x2 = currentLine[2]
y2 = currentLine[3]
X2.append(x2)
Y2.append(y2)
# Draw the lines:
cv2.line(originalImg, (x1,y1), (x2,y2), (0,0,255), 2)
cv2.imshow("Lines", originalImg)
cv2.waitKey(0)
This is the result:
As you can see, there are multiple lines. Luckily, these lines are clustered in two very discernible groups: the left hand and the right hand. If we cluster the four coordinates into two groups, we can get the average starting and ending points of each hand. This can be done by applying a clustering algorithm, in this case K-Means. K-means will need four arrays holding the data to produce two cluster centers. Before giving it our data we need to reshape it the way K-means expects it:
# Reshape the arrays for K-means
X1 = np.array(X1)
Y1 = np.array(Y1)
X2 = np.array(X2)
Y2 = np.array(Y2)
X1dash = X1.reshape(-1,1)
Y1dash = Y1.reshape(-1,1)
X2dash = X2.reshape(-1,1)
Y2dash = Y2.reshape(-1,1)
# Stack the data
Z = np.hstack((X1dash, Y1dash, X2dash, Y2dash))
# K-means operates on 32-bit float data:
floatPoints = np.float32(Z)
# Set the convergence criteria and call K-means:
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0)
# Set the desired number of clusters
K = 2
ret, label, center = cv2.kmeans(floatPoints, K, None, criteria, 10, cv2.KMEANS_RANDOM_CENTERS)
The results are in the center array. Here we gave out final pair of lines. Let's loop through it and draw them on the original image:
# Loop through the center points
# and draw the lines:
for p in range(len(center)):
# Get line points:
print(center[p])
x1 = int(center[p][0])
y1 = int(center[p][1])
x2 = int(center[p][2])
y2 = int(center[p][3])
cv2.line(originalImg, (x1, y1), (x2, y2), (0, 255, 0), 1)
cv2.imshow("Lines", originalImg)
cv2.waitKey(0)
This is the final pair of lines (in green):
Related
I have an image and I've done some pre-processing on the that image. Below I showed my preprocessing:
img= cv2.imread("...my_drive...\\image_69.tif",0)
median=cv2.medianBlur(img,13)
ret, th = cv2.threshold(median, 0 , 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)
kernel=np.ones((3,15),np.uint8)
closing1 = cv2.morphologyEx(th, cv2.MORPH_CLOSE, kernel, iterations=2)
kernel=np.ones((1,31),np.uint8)
closing2 = cv2.morphologyEx(closing1, cv2.MORPH_CLOSE, kernel)
kernel=np.ones((1,13),np.uint8)
opening1= cv2.morphologyEx(closing2, cv2.MORPH_OPEN, kernel, iterations=2)
So, basically I used "Threshold filtering" , "closing" and "opening" and the result looks like this:
Please note that when I used type(opening1), I got numpy.ndarray. So the image at this step is numpy array with 1021 x 1024 size.
Then I labeled my image:
label_image=measure.label(opening1, connectivity=opening1.ndim)
props= measure.regionprops_table (label_image, properties=['label', "area", "coords"])
and the result looks like this
Please note that when I used type(label_image), I got numpy.ndarray. So the image at this step is numpy array with 1021 x 1024 size.
As you can see, currently the image has 6 labels. Some of these labels are short and small pieces, so I tried to keep top 2 label based on area
slc=label_image
rps=regionprops(slc)
areas=[r.area for r in rps]
id=np.argsort(props["area"])[::-1]
new_slc=np.zeros_like(slc)
for i in id[:2]:
new_slc[tuple(rps[i].coords.T)]=i+1
Now the result looks like this:
It looks like I was successful in keeping 2 top regions (please note that by changing id[:2] you can select thickest white layer or thin layer). Now:
What I want to do: I want to find the average thickness of these two regions
Also, please note that I know each of my pixels is 314 nm
Can anyone here advise how I can do this task?
Original photo: Below I showed low quality of my original image, so you have better understanding as why I did all the pre-processing
you can also access the original photo here : https://www.mediafire.com/file/20h66aq83edy1h7/img.7z/file
Here is one way to do that in Python/OpenCV.
Read the input
Convert to gray
Threshold to binary
Get the contours and filter on area so that we have only the two primary lines
Sort by area
Select the first (smaller and thinner) contour
Draw it white filled on a black background
Get its skeleton
Get the points of the skeleton
Fit a line to the points and get the rotation angle of the skeleton
Loop over each of the two contours and draw them white filled on a black background. Then rotate to horizontal lines. Then get the vertical thickness of the lines from the average thickness along each column using np.count_nonzero() and print the value.
Save intermediate images
Input:
import cv2
import numpy as np
import skimage.morphology
import skimage.transform
import math
# read image
img = cv2.imread('lines.jpg')
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# threshold
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)[1]
# get contours
new_contours = []
img2 = np.zeros_like(thresh, dtype=np.uint8)
contour_img = thresh.copy()
contour_img = cv2.merge([contour_img,contour_img,contour_img])
contours = cv2.findContours(thresh , cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for cntr in contours:
area = cv2.contourArea(cntr)
if area > 1000:
cv2.drawContours(contour_img, [cntr], 0, (0,0,255), 1)
cv2.drawContours(img2, [cntr], 0, (255), -1)
new_contours.append(cntr)
# sort contours by area
cnts_sort = sorted(new_contours, key=lambda x: cv2.contourArea(x), reverse=False)
# select first (smaller) sorted contour
first_contour = cnts_sort[0]
contour_first_img = np.zeros_like(thresh, dtype=np.uint8)
cv2.drawContours(contour_first_img, [first_contour], 0, (255), -1)
# thin smaller contour
thresh1 = (contour_first_img/255).astype(np.float64)
skeleton = skimage.morphology.skeletonize(thresh1)
skeleton = (255*skeleton).clip(0,255).astype(np.uint8)
# get skeleton points
pts = np.column_stack(np.where(skeleton.transpose()==255))
# fit line to pts
(vx,vy,x,y) = cv2.fitLine(pts, cv2.DIST_L2, 0, 0.01, 0.01)
#print(vx,vy,x,y)
x_axis = np.array([1, 0]) # unit vector in the same direction as the x axis
line_direction = np.array([vx, vy]) # unit vector in the same direction as your line
dot_product = np.dot(x_axis, line_direction)
[angle_line] = (180/math.pi)*np.arccos(dot_product)
print("angle:", angle_line)
# loop over each sorted contour
# draw contour filled on black background
# rotate
# get mean thickness from np.count_non-zeros
black = np.zeros_like(thresh, dtype=np.uint8)
i = 1
for cnt in cnts_sort:
cnt_img = black.copy()
cv2.drawContours(cnt_img, [cnt], 0, (255), -1)
cnt_img_rot = skimage.transform.rotate(cnt_img, angle_line, resize=False)
thickness = np.mean(np.count_nonzero(cnt_img_rot, axis=0))
print("line ",i,"=",thickness)
i = i + 1
# save resulting images
cv2.imwrite('lines_thresh.jpg',thresh)
cv2.imwrite('lines_filtered.jpg',img2)
cv2.imwrite('lines_small_contour_skeleton.jpg',skeleton )
# show thresh and result
cv2.imshow("thresh", thresh)
cv2.imshow("contours", contour_img)
cv2.imshow("lines_filtered", img2)
cv2.imshow("first_contour", contour_first_img)
cv2.imshow("skeleton", skeleton)
cv2.waitKey(0)
cv2.destroyAllWindows()
Threshold image:
Contour image:
Filtered contour image:
Skeleton image:
Angle (in degrees) and Thicknesses (in pixels):
angle: 3.1869032185349733
line 1 = 8.79219512195122
line 2 = 49.51609756097561
To get the thickness in nm, multiply thickness in pixels by your 314 nm/pixel.
ADDITION
If I start with your tiff image, the following shows my preprocessing, which is similar to yours.
import cv2
import numpy as np
import skimage.morphology
import skimage.transform
import math
# read image
img = cv2.imread('lines.tif')
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# threshold
thresh = cv2.threshold(gray, 128, 255, cv2.THRESH_BINARY)[1]
# apply morphology
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,5))
morph = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel)
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (29,1))
morph = cv2.morphologyEx(morph, cv2.MORPH_CLOSE, kernel)
# get contours
new_contours = []
img2 = np.zeros_like(gray, dtype=np.uint8)
contour_img = gray.copy()
contour_img = cv2.merge([contour_img,contour_img,contour_img])
contours = cv2.findContours(morph , cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for cntr in contours:
area = cv2.contourArea(cntr)
if area > 1000:
cv2.drawContours(contour_img, [cntr], 0, (0,0,255), 1)
cv2.drawContours(img2, [cntr], 0, (255), -1)
new_contours.append(cntr)
# sort contours by area
cnts_sort = sorted(new_contours, key=lambda x: cv2.contourArea(x), reverse=False)
# select first (smaller) sorted contour
first_contour = cnts_sort[0]
contour_first_img = np.zeros_like(morph, dtype=np.uint8)
cv2.drawContours(contour_first_img, [first_contour], 0, (255), -1)
# thin smaller contour
thresh1 = (contour_first_img/255).astype(np.float64)
skeleton = skimage.morphology.skeletonize(thresh1)
skeleton = (255*skeleton).clip(0,255).astype(np.uint8)
# get skeleton points
pts = np.column_stack(np.where(skeleton.transpose()==255))
# fit line to pts
(vx,vy,x,y) = cv2.fitLine(pts, cv2.DIST_L2, 0, 0.01, 0.01)
#print(vx,vy,x,y)
x_axis = np.array([1, 0]) # unit vector in the same direction as the x axis
line_direction = np.array([vx, vy]) # unit vector in the same direction as your line
dot_product = np.dot(x_axis, line_direction)
[angle_line] = (180/math.pi)*np.arccos(dot_product)
print("angle:", angle_line)
# loop over each sorted contour
# draw contour filled on black background
# rotate
# get mean thickness from np.count_non-zeros
black = np.zeros_like(thresh, dtype=np.uint8)
i = 1
for cnt in cnts_sort:
cnt_img = black.copy()
cv2.drawContours(cnt_img, [cnt], 0, (255), -1)
cnt_img_rot = skimage.transform.rotate(cnt_img, angle_line, resize=False)
thickness = np.mean(np.count_nonzero(cnt_img_rot, axis=0))
print("line ",i,"=",thickness)
i = i + 1
# save resulting images
cv2.imwrite('lines_thresh2.jpg',thresh)
cv2.imwrite('lines_morph2.jpg',morph)
cv2.imwrite('lines_filtered2.jpg',img2)
cv2.imwrite('lines_small_contour_skeleton2.jpg',skeleton )
# show thresh and result
cv2.imshow("thresh", thresh)
cv2.imshow("morph", morph)
cv2.imshow("contours", contour_img)
cv2.imshow("lines_filtered", img2)
cv2.imshow("first_contour", contour_first_img)
cv2.imshow("skeleton", skeleton)
cv2.waitKey(0)
cv2.destroyAllWindows()
Threshold image:
Morphology image:
Filtered Lines image:
Skeleton image:
Angle (degrees) and Thickness (pixels):
angle: 3.206927978669998
line 1 = 9.26171875
line 2 = 49.693359375
Use Deskew to straighten up the image.
Then, count the pixels of each column of the color of the label you want to measure then divide it by the number of columns to get the average thickness
This can be done with various tools in scipy. Assume you have the image here:
I = PIL.Image.open("input.jpg")
img = np.array(I).mean(axis=2)
mask = img==255 # or some kind of thresholding
imshow(mask) #note this is a binary image, the green coloring is due to some kind of rendering artifact or aliasing
If one zooms in they can see split up regions
To get around that we can dilate the mask
from scipy import ndimage as ni
mask1 = ni.binary_dilation(mask, iterations=2)
imshow(mask1)
Now, we can find connected regions, and find the top regions with the most pixels, which should be the two lines of interest:
lab, nlab = ni.label(mask1)
max_labs = np.argsort([ (lab==i).sum() for i in range(1, nlab+1)])[::-1]+1
imshow(lab==max_labs[0])
and imshow(lab==max_labs[1])
Working with the first line as an example:
from scipy.stats import linregress
y0,x0 = np.where(lab==max_labs[0])
l0 = linregress( x0, y0)
xi,yi = np.arange(img.shape[3]), np.arange(img.shape[3])*l0.slope + l0.intercept
plot( xi, yi, 'r--')
Interpolate along this region at different y-intercepts and compute the average signal along each line
from scipy.interpolate import RectBivariateSpline
img0 = img.copy()
img0[~(lab==max_labs[0])] = 0 # set everything outside this line region to 0
rbv = RectBivariateSpline(np.arange(img.shape[0]), np.arange(img.shape[1]), img0)
prof0 = [rbv.ev(yi+i, xi).mean() for i in np.arange(-300,300)] # pick a wide window here (-300,300), can be more technical, but not necessary
plot(prof0)
Use your favorite method to compute the FWHM of this profile, then multiply by your pixel-to-nanometers factor.
I would just use a Gaussian fit to compute fwhm
xvals = np.arange(len(prof0))
yvals = np.array(prof0)
def func(p, xvals, yvals):
mu,var, amp = p
model = np.exp(-(xvals-mu)**2/2/var)*amp
resid = (model-yvals)**2
return resid.sum()
from scipy.optimize import minimize
x0 = 300,200,255 # initial estimate of mu, variance, amplitude
fit_gauss = minimize(func, x0=x0, args=(xvals, yvals), method='Nelder-Mead')
mu, var, amp = fit_gauss.x
fwhm = 2.355 * np.sqrt(var)
# display using matplotlib plot /hlines
plot( xvals, yvals)
plot( xvals, amp*np.exp(-(xvals-mu)**2/2/var) )
hlines(amp*0.5, mu-fwhm/2., mu+fwhm/2, color='r')
legend(("profile","fit gauss","fwhm=%.2f pix" % fwhm))
Finally, thickness=fwhm*314, or about 13 microns.
Following the exact same approach for the second line (lab==max_labs[1]) gives a thickness of about 2.2 microns:
Note, I was using interactive plotting to do this example, hence calls to imshow , plot etc. are meant motly as a reference to the reader. One may need to take extra steps to recreate the exact images I've uploaded (zooming etc).
I am trying to convert the result of a skeletonization into a set of line segments, where the vertices correspond to the junction points of the skeleton. The shape is not a closed polygon and it may be somewhat noisy (the segments are not as straight as they should be).
Here is an example input image:
And here are the points I want to retrieve:
I have tried using the harris corner detector, but it has trouble in some areas even after trying to tweak the algorithm's parameters (such as the angled section on the bottom of the image). Here are the results:
Do you know of any method capable of doing this? I am using python with mostly OpenCV and Numpy but I am not bound to any library. Thanks in advance.
Edit: I've gotten some good responses regarding the junction points, I am really grateful. I would also appreciate any solutions regarding extracting line segments from the junction points. I think #nathancy's answer could be used to extract line segments by subtracting the masks with the intersection mask, but I am not sure.
My approach is based on my previous answer here. It involves convolving the image with a special kernel. This convolution identifies the end-points of the lines, as well as the intersections. This will result in a points mask containing the pixel that matches the points you are looking for. After that, apply a little bit of morphology to join possible duplicated points. The method is sensible to the corners produced by the skeleton.
This is the code:
import cv2
import numpy as np
# image path
path = "D://opencvImages//"
fileName = "Repn3.png"
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
inputImageCopy = inputImage.copy()
# Convert to grayscale:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Compute the skeleton:
skeleton = cv2.ximgproc.thinning(grayscaleImage, None, 1)
# Threshold the image so that white pixels get a value of 10 and
# black pixels a value of 0:
_, binaryImage = cv2.threshold(skeleton, 128, 10, cv2.THRESH_BINARY)
# Set the convolution kernel:
h = np.array([[1, 1, 1],
[1, 10, 1],
[1, 1, 1]])
# Convolve the image with the kernel:
imgFiltered = cv2.filter2D(binaryImage, -1, h)
So far I convolved the skeleton image with my special kernel. You can inspect the image produced and search for the numerical values at the corners and intersections.
This is the output so far:
Next, identify a corner or an intersection. This bit is tricky, because the threshold value depends directly on the skeleton image, which sometimes doesn't produce good (close to straight) corners:
# Create list of thresholds:
thresh = [130, 110, 40]
# Prepare the final mask of points:
(height, width) = binaryImage.shape
pointsMask = np.zeros((height, width, 1), np.uint8)
# Perform convolution and create points mask:
for t in range(len(thresh)):
# Get current threshold:
currentThresh = thresh[t]
# Locate the threshold in the filtered image:
tempMat = np.where(imgFiltered == currentThresh, 255, 0)
# Convert and shape the image to a uint8 height x width x channels
# numpy array:
tempMat = tempMat.astype(np.uint8)
tempMat = tempMat.reshape(height,width,1)
# Accumulate mask:
pointsMask = cv2.bitwise_or(pointsMask, tempMat)
This is the binary mask:
Let's dilate to join close points:
# Set kernel (structuring element) size:
kernelSize = 3
# Set operation iterations:
opIterations = 4
# Get the structuring element:
morphKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelSize, kernelSize))
# Perform Dilate:
pointsMask = cv2.morphologyEx(pointsMask, cv2.MORPH_DILATE, morphKernel, None, None, opIterations, cv2.BORDER_REFLECT101)
This is the output:
Now simple extract external contours. Get their bounding boxes and calculate their centroid:
# Look for the outer contours (no children):
contours, _ = cv2.findContours(pointsMask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# Store the points here:
pointsList = []
# Loop through the contours:
for i, c in enumerate(contours):
# Get the contours bounding rectangle:
boundRect = cv2.boundingRect(c)
# Get the centroid of the rectangle:
cx = int(boundRect[0] + 0.5 * boundRect[2])
cy = int(boundRect[1] + 0.5 * boundRect[3])
# Store centroid into list:
pointsList.append( (cx,cy) )
# Set centroid circle and text:
color = (0, 0, 255)
cv2.circle(inputImageCopy, (cx, cy), 3, color, -1)
font = cv2.FONT_HERSHEY_COMPLEX
cv2.putText(inputImageCopy, str(i), (cx, cy), font, 0.5, (0, 255, 0), 1)
# Show image:
cv2.imshow("Circles", inputImageCopy)
cv2.waitKey(0)
This is the result. Some corners are missed, you might one to improve the solution before computing the skeleton.
Here's a simple approach, the idea is:
Obtain binary image. Load image, convert to grayscale, Gaussian blur, then Otsu's threshold.
Obtain horizontal and vertical line masks. Create horizontal and vertical structuring elements with cv2.getStructuringElement then perform cv2.morphologyEx to isolate the lines.
Find joints. We cv2.bitwise_and the two masks together to get the joints. The idea is that the intersection points on the two masks are the joints.
Find centroid on joint mask. We find contours then calculate the centroid.
Find leftover endpoints. Endpoints do not correspond to an intersection so to find those, we can use the Shi-Tomasi Corner Detector
Horizontal and vertical line masks
Results (joints in green and endpoints in blue)
Code
import cv2
import numpy as np
# Load image, grayscale, Gaussian blur, Otsus threshold
image = cv2.imread('1.png')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (3,3), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
# Find horizonal lines
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5,1))
horizontal = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=1)
# Find vertical lines
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,5))
vertical = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, vertical_kernel, iterations=1)
# Find joint intersections then the centroid of each joint
joints = cv2.bitwise_and(horizontal, vertical)
cnts = cv2.findContours(joints, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
# Find centroid and draw center point
x,y,w,h = cv2.boundingRect(c)
centroid, coord, area = cv2.minAreaRect(c)
cx, cy = int(centroid[0]), int(centroid[1])
cv2.circle(image, (cx, cy), 5, (36,255,12), -1)
# Find endpoints
corners = cv2.goodFeaturesToTrack(thresh, 5, 0.5, 10)
corners = np.int0(corners)
for corner in corners:
x, y = corner.ravel()
cv2.circle(image, (x, y), 5, (255,100,0), -1)
cv2.imshow('thresh', thresh)
cv2.imshow('joints', joints)
cv2.imshow('horizontal', horizontal)
cv2.imshow('vertical', vertical)
cv2.imshow('image', image)
cv2.waitKey()
I am trying to detect all of the overlapping circle/ellipses shapes in this image all of which have digits on them. I have tried different types of image processing techniques using OpenCV, however I cannot detect the shapes that overlap the tree. I have tried erosion and dilation however it has not helped.
Any pointers on how to go about this would be great. I have attached my code below
original = frame.copy()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (3, 3), 0)
canny = cv2.Canny(blurred, 120, 255, 1)
kernel = np.ones((5, 5), np.uint8)
dilate = cv2.dilate(canny, kernel, iterations=1)
# Find contours
cnts = cv2.findContours(dilate, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
image_number = 0
for c in cnts:
x, y, w, h = cv2.boundingRect(c)
cv2.rectangle(frame, (x, y), (x + w, y + h), (36, 255, 12), 2)
ROI = original[y:y + h, x:x + w]
cv2.imwrite("ROI_{}.png".format(image_number), ROI)
image_number += 1
cv2.imshow('canny', canny)
cv2.imshow('image', frame)
cv2.waitKey(0)
Here's a possible solution. I'm assuming that the target blobs (the saucer-like things) are always labeled - that is, they always have a white number inside them. The idea is to create a digits mask, because their size and color seem to be constant. I use the digits as guide to obtain sample pixels of the ellipses. Then, I convert these BGR pixels to HSV, create a binary mask and use that info to threshold and locate the ellipses. Let's check out the code:
# imports:
import cv2
import numpy as np
# image path
path = "D://opencvImages//"
fileName = "4dzfr.png"
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
# Deep copy for results:
inputImageCopy = inputImage.copy()
# Convert RGB to grayscale:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Get binary image via Otsu:
binaryImage = np.where(grayscaleImage >= 200, 255, 0)
# The above operation converted the image to 32-bit float,
# convert back to 8-bit uint
binaryImage = binaryImage.astype(np.uint8)
The first step is to make a mask of the digits. I also created a deep copy of the BGR image. The digits are close to white (That is, an intensity close to 255). I use 200 as threshold and obtain this result:
Now, let's locate these contours on this binary mask. I'm filtering based on aspect ratio, as the digits have a distinct aspect ratio close to 0.70. I'm also filtering contours based on hierarchy - as I'm only interested on external contours (the ones that do not have children). That's because I really don't need contours like the "holes" inside the digit 8:
# Find the contours on the binary image:
contours, hierarchy = cv2.findContours(binaryImage, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
# Store the sampled pixels here:
sampledPixels = []
# Look for the outer bounding boxes (no children):
for i, c in enumerate(contours):
# Get the contour bounding rectangle:
boundRect = cv2.boundingRect(c)
# Get the dimensions of the bounding rect:
rectX = boundRect[0]
rectY = boundRect[1]
rectWidth = boundRect[2]
rectHeight = boundRect[3]
# Compute the aspect ratio:
aspectRatio = rectWidth / rectHeight
# Create the filtering threshold value:
delta = abs(0.7 - aspectRatio)
epsilon = 0.1
# Get the hierarchy:
currentHierarchy = hierarchy[0][i][3]
# Prepare the list of sampling points (One for the ellipse, one for the circle):
samplingPoints = [ (rectX - rectWidth, rectY), (rectX, rectY - rectHeight) ]
# Look for the target contours:
if delta < epsilon and currentHierarchy == -1:
# This list will hold both sampling pixels:
pixelList = []
# Get sampling pixels from the two locations:
for s in range(2):
# Get sampling point:
sampleX = samplingPoints[s][0]
sampleY = samplingPoints[s][1]
# Get sample BGR pixel:
samplePixel = inputImageCopy[sampleY, sampleX]
# Store into temp list:
pixelList.append(samplePixel)
# convert list to tuple:
pixelList = tuple(pixelList)
# Save pixel value:
sampledPixels.append(pixelList)
Ok, there area a couple of things happening in the last snippet of code. We want to sample pixels from both the ellipse and the circle. We will use two sampling locations that are function of each digit's original position. These positions are defined in the samplingPoints tuple. For the ellipse, I'm sampling at a little before the top right position of the digit. For the circle, I'm sapling directly above the top right position - we end up with two pixels for each figure.
You'll notice I'm doing a little bit of data type juggling, converting lists to tuples. I want these pixels stored as a tuple for convenience. If I draw bounding rectangles of the digits, this would be the resulting image:
Now, let's loop through the pixel list, convert them to HSV and create a HSV mask over the original BGR image. The final bounding rectangles of the ellipses are stored in boundingRectangles, additionally I draw results on the deep copy of the original input:
# Final bounding rectangles are stored here:
boundingRectangles = []
# Loop through sampled pixels:
for p in range(len(sampledPixels)):
# Get current pixel tuple:
currentPixelTuple = sampledPixels[p]
# Prepare the HSV mask:
imageHeight, imageWidth = binaryImage.shape[:2]
hsvMask = np.zeros((imageHeight, imageWidth), np.uint8)
# Process the two sampling pixels:
for m in range(len(currentPixelTuple)):
# Get current pixel in the list:
currentPixel = currentPixelTuple[m]
# Create BGR Mat:
pixelMat = np.zeros([1, 1, 3], dtype=np.uint8)
pixelMat[0, 0] = currentPixel
# Convert the BGR pixel to HSV:
hsvPixel = cv2.cvtColor(pixelMat, cv2.COLOR_BGR2HSV)
H = hsvPixel[0][0][0]
S = hsvPixel[0][0][1]
V = hsvPixel[0][0][2]
# Create HSV range for this pixel:
rangeThreshold = 5
lowerValues = np.array([H - rangeThreshold, S - rangeThreshold, V - rangeThreshold])
upperValues = np.array([H + rangeThreshold, S + rangeThreshold, V + rangeThreshold])
# Create HSV mask:
hsvImage = cv2.cvtColor(inputImage.copy(), cv2.COLOR_BGR2HSV)
tempMask = cv2.inRange(hsvImage, lowerValues, upperValues)
hsvMask = hsvMask + tempMask
First, I create a 1 x 1 Matrix (or Numpy Array) with just a BGR pixel value - the first of two I previously sampled. In this way, I can use cv2.cvtColor to get the corresponding HSV values. Then, I create lower and upper threshold values for the HSV mask. However, the image seems synthetic, and a range-based thresholding could be reduced to a unique tuple. After that, I create the HSV mask using cv2.inRange.
This will yield the HSV mask for the ellipse. After applying the method for the circle we will end up with two HSV masks. Well, I just added the two arrays to combine both masks. At the end you will have something like this, this is the "composite" HSV mask created for the first saucer-like figure:
We can apply a little bit of morphology to join both shapes, just a little closing will do:
# Set kernel (structuring element) size:
kernelSize = 3
# Set morph operation iterations:
opIterations = 2
# Get the structuring element:
morphKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelSize, kernelSize))
# Perform closing:
hsvMask = cv2.morphologyEx(hsvMask, cv2.MORPH_CLOSE, morphKernel, None, None, opIterations,cv2.BORDER_REFLECT101)
This is the result:
Nice. Let's continue and get the bounding rectangles of all the shapes. I'm using the boundingRectangles list to store each bounding rectangle, like this:
# Process current contour:
currentContour, _ = cv2.findContours(hsvMask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
for _, c in enumerate(currentContour):
# Get the contour's bounding rectangle:
boundRect = cv2.boundingRect(c)
# Get the dimensions of the bounding rect:
rectX = boundRect[0]
rectY = boundRect[1]
rectWidth = boundRect[2]
rectHeight = boundRect[3]
# Store and set bounding rect:
boundingRectangles.append(boundRect)
color = (0, 0, 255)
cv2.rectangle(inputImageCopy, (int(rectX), int(rectY)),
(int(rectX + rectWidth), int(rectY + rectHeight)), color, 2)
cv2.imshow("Objects", inputImageCopy)
cv2.waitKey(0)
This is the image of the located rectangles once every sampled pixel is processed:
I am trying to filter the background of images presenting electric cables. I tried to do the following:
Transform from color to gray
Apply cv2.Laplacian or 2 times of cv2.Sobel for finding edges in both directions.
Apply thresholding cv2.THRESH_BINARY(_INV), cv2.THRESH_OTSU
Lastly, I tried to find edges with 'filtered' images using cv2.Canny together with cv2.HoughLinesP
Overall, the results aren't satisfying at all. I will give an example of 2 images:
And the output of my script:
I also played with the values in config, but the results weren't different much.
Here's the little script I managed to do:
import cv2
import matplotlib.pyplot as plt
import numpy as np
def img_show(images, cmap=None):
fig = plt.figure(figsize=(17, 10))
root = 3 # len(images) ** 0.5
for i, img in enumerate(images):
ax = fig.add_subplot(root, root, i + 1)
ax.imshow(img, cmap=cmap[i])
plt.show()
class Config:
scale = 0.4
min_threshold = 120
max_threshold = 200
canny_min_threshold = 100
canny_max_threshold = 200
config = Config()
def find_lines(img, rgb_img):
dst = cv2.Canny(img, config.canny_min_threshold, config.canny_max_threshold)
cdstP = np.copy(rgb_img)
lines = cv2.HoughLinesP(dst, 1, np.pi / 180, 150, None, 0, 0)
lines1 = lines[:, 0, :]
for x1, y1, x2, y2 in lines1[:]:
cv2.line(cdstP, (x1, y1), (x2, y2), (255, 0, 0), 5)
return cdstP
if __name__ == "__main__":
bgr_img = cv2.imread('DJI_0009.JPG')
bgr_img = cv2.resize(bgr_img, (0, 0), bgr_img, config.scale, config.scale)
rgb_img = cv2.cvtColor(bgr_img, cv2.COLOR_BGR2RGB)
gray_img = cv2.cvtColor(bgr_img, cv2.COLOR_BGR2GRAY)
# _, threshold = cv2.threshold(gray_img, config.min_threshold, config.max_threshold, cv2.THRESH_BINARY)
# laplacian = cv2.Laplacian(rgb_img, cv2.CV_8UC1)
sobelx = cv2.Sobel(gray_img, cv2.CV_8UC1, 1, 0)
sobely = cv2.Sobel(gray_img, cv2.CV_8UC1, 0, 1)
blended = cv2.addWeighted(src1=sobelx, alpha=0.5, src2=sobely, beta=0.5, gamma=0)
_, threshold = cv2.threshold(blended, config.min_threshold, config.max_threshold,
cv2.THRESH_BINARY | cv2.THRESH_OTSU)
p1 = find_lines(threshold, rgb_img)
p2 = find_lines(blended, rgb_img)
p3 = find_lines(gray_img, rgb_img)
plots = [rgb_img, p1, p2, p3]
cmaps = [None] + ['gray'] * (len(plots) - 1)
img_show(plots, cmaps)
I am assuming I need to do much better filtring. However, I also tried image segmentation, but the results weren't promising at all.
Any ideas on how to improve this?
Thanks
Here is one way to do that in Python/OpenCV. I threshold, then optionally clean with morphology. Then get the contours and for each contour compute its rotated rectangle. Then get the dimensions of the rotated rectangle and compute the aspect ratio (largest dimension / smallest dimension) and optionally the area. Then I threshold on the aspect ratio (and optionally the area) and keep only those contours that pass)
Input:
import cv2
import numpy as np
image = cv2.imread("DCIM-100-MEDIA-DJI-0009-JPG.jpg")
hh, ww = image.shape[:2]
# convert to grayscale
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# create a binary thresholded image
thresh = cv2.threshold(gray, 64, 255, cv2.THRESH_BINARY)[1]
# invert so line is white on black background
thresh = 255 - thresh
# apply morphology
kernel = np.ones((11,11), np.uint8)
clean = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)
# get external contours
contours = cv2.findContours(clean, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
area_thresh = ww / 2
aspect_thresh = ww / 30
print(area_thresh,aspect_thresh)
print('')
result = image.copy()
for c in contours:
# get rotated rectangle from contour
# get its dimensions
rotrect = cv2.minAreaRect(c)
(center), (dim1,dim2), angle = rotrect
maxdim = max(dim1,dim2)
mindim = min(dim1,dim2)
area = dim1 * dim2
if mindim != 0:
aspect = maxdim / mindim
#print(area, aspect)
#if area > area_thresh and aspect > aspect_thresh:
if aspect > aspect_thresh:
# draw contour on input
cv2.drawContours(result,[c],0,(0,0,255),3)
print(area, aspect)
# save result
cv2.imwrite("DCIM-100-MEDIA-DJI-0009-JPG_thresh.jpg",thresh)
cv2.imwrite("DCIM-100-MEDIA-DJI-0009-JPG_clean.jpg",clean)
cv2.imwrite("DCIM-100-MEDIA-DJI-0009-JPG_result.jpg",result)
# display result
cv2.imshow("thresh", thresh)
cv2.imshow("clean", clean)
cv2.imshow("result", result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Thresholded image:
Morphology cleaned image:
Result image:
I am facing a contour problem, so that you can read the license plate in a correct way, it is best to take out the contours, and thus perhaps apply some OCR.
For example if I want to use this photo, you can see that on the axis it has a white outline, how could I eliminate those white outlines in a generic way? for can be used on more license plates
I am thinking of applying a threshold in the 2 axis (horizontally and vertically) again to clean possible white borders, any ideas?
A little what I'm have made:
# Creating copies of the original images
output_cp = output.copy()
img_cp = straightened.copy()
# threshold
ret,thresh = cv2.threshold(output_cp, 215, 255, cv2.THRESH_BINARY_INV)
imshow(thresh)
But then when It suposed to work, doesn't work because it is a binary image,
pseudo-code, when xxx I don't know what contour apply:
mask = np.zeros(image.shape, np.uint8)
mask_cnt = cv2.drawContours(mask.copy(), [xxx], 0, (255,255,255,255), -1)
removed = cv2.subtract(mask_cnt, image)
plt.figure()
plt.imshow(removed, cmap='gray')
Any help is welcome!
To remove the white margin around the plate, you can :
find the external contours of the image with findCountours
take its rotated bounding box with minAreaRect
compute a transformation to correct the orientation with getPerspectiveTransforme
then apply it with warpPerspective
This remove the margin and also correct the orientation of the plate which should make it much easier to read by any OCR.
Here is a python implementation of my solution:
#!/usr/bin/env python3
import numpy as np
import cv2
img = cv2.imread("plate.png")
gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
# invert image and detect contours
inverted = cv2.bitwise_not(gray)
contours, hierarchy = cv2.findContours(inverted,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
# get the biggest contour
biggest_index = -1
biggest_area = -1
i = 0
for c in contours:
area = cv2.contourArea(c)
if area > biggest_area:
biggest_area = area
biggest_index = i
i = i+1
print("biggest area: " + str(biggest_area) + " index: " + str(biggest_index))
cv2.drawContours(img, contours, biggest_index, [0,0,255])
center, size, angle = cv2.minAreaRect(contours[biggest_index])
rot_mat = cv2.getRotationMatrix2D(center, angle, 1.)
#cv2.warpPerspective()
print(size)
dst = cv2.warpAffine(gray, rot_mat, (int(size[0]), int(size[1])))
mask = dst * 0
x1 = max([int(center[0] - size[0] / 2)+1, 0])
y1 = max([int(center[1] - size[1] / 2)+1, 0])
x2 = int(center[0] + size[0] / 2)-1
y2 = int(center[1] + size[1] / 2)-1
point1 = (x1, y1)
point2 = (x2, y2)
print(point1)
print(point2)
cv2.rectangle(dst, point1, point2, [0,0,0])
cv2.rectangle(mask, point1, point2, [255,255,255], cv2.FILLED)
masked = cv2.bitwise_and(dst, mask)
cv2.imshow("img", img)
cv2.imshow("dst", dst)
cv2.imshow("masked", masked)
cv2.imshow("mask", mask)
key = -1;
while key != 27:
key = cv2.waitKey(1)
And the resulting image :
This is not perfect, but a good start I think, slightly different approche than thresholding.
You might also try to apply some morphological operator to close some gap or remove dirty parts.