After have segmented my lemons successfully I would like to get his size in pixels and then convert this value to millimeters. I'm reading a thesis were this guys did that but with strawberries. The first step was crop the segmented strawberries in a rectangle:
The image (b) was called the 'minimum rectangle'. According the authors to create it, This is built depending on the extreme values of the region:
- the highest point
- the extreme left point
- the lowest point
- the extreme right point of the region of interest.
Once this is done, the width of the rectangle is measured, which will indicate the measurement of the diameter of the strawberry in pixels.
In my case this is my input image:
And this is my desired output:
I'm programming in python with opencv. I would like to crop my input image and then find the minimum rectangle to get the width of the rectangle which will show the diameter of the lemon in pixels.
According the thesis, to convert the measure in pixels to a measure of the real world as in millimeters, I should take a photography with a rectangle with a 3 cm of side with the same conditions as were take the images of the lemons. Then I should segment this rectangle and then find his minimun rectangle as the image of above and find his width in pixels as result of it with a rectangle of known measures i.g they got 176 pixels of width. Of this way they got:
1mm = 176/30 = 5.87 pixels
With this information I would like to compute the width of my lemons and get this first in pixels, the convert it to milimetters. Guys if you can do it, please suppost that I taked a photography of a know figure of 3cm of side, the same as the thesis. By the moment I can't get the minimun rectangle because I don't know how get it, is because that I asking for his help to you.
Well guys I would like to see your suggestions, any I idea I will apreciate it. Thanks so much.
Thanks you.
Once you have the thresholded image (mask) of your blob of interest (the lemon) it is very straightforward to get its (rotated) minimum area rectangle or bounding rectangle. Use the cv2.minAreaRect function to get the former or the cv2.boundingRect function to get the later. In both cases you need to compute the contours of the binary mask, get the outer and biggest contour and pass that to either function.
Let's see an example for getting both:
# image path
path = "C://opencvImages//"
fileName = "TAkY2.png"
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
# Grayscale conversion:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Thresholding:
threshValue, binaryImage = cv2.threshold(grayscaleImage, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
This is just to get the binary mask, you already have this. This is the result:
Now, get the contours. And just to draw some results, prepare a couple of deep copies of the input that we will use to check out things:
# Find the big contours/blobs on the filtered image:
contours, hierarchy = cv2.findContours(binaryImage, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
# Deep copies of the input image to draw results:
minRectImage = inputImage.copy()
polyRectImage = inputImage.copy()
Now, get the contours and filter them by a minimum area (minArea) value. You want to just keep the biggest contour - that's the lemon perimeter:
# Look for the outer bounding boxes:
for i, c in enumerate(contours):
if hierarchy[0][i][3] == -1:
# Get contour area:
contourArea = cv2.contourArea(c)
# Set minimum area threshold:
minArea = 1000
# Look for the largest contour:
if contourArea > minArea:
# Option 1: Get the minimum area bounding rectangle
# for this contour:
boundingRectangle = cv2.minAreaRect(c)
# Get the rectangle points:
rectanglePoints = cv2.boxPoints(boundingRectangle)
# Convert float array to int array:
rectanglePoints = np.intp(rectanglePoints)
# Draw the min area rectangle:
cv2.drawContours(minRectImage, [rectanglePoints], 0, (0, 0, 255), 2)
cv2.imshow("minAreaRect", minRectImage)
cv2.waitKey(0)
This portion of code gets you these results. Note that this rectangle is rotated to encompass the minimum area of the contour, just as if you were actually measuring the lemon with a caliper:
You can also get the position of the four corners of this rectangle. Still, inside the loop, we have the following bit of code:
# Draw the corner points:
for p in rectanglePoints:
cv2.circle(minRectImage, (p[0], p[1]), 3, (0, 255, 0), -1)
cv2.imshow("minAreaRect Points", minRectImage)
cv2.waitKey(0)
These are the corners of the min area rectangle:
You might or might not like this result. You might be looking for the bounding rectangle that is not rotated. In such case you can use cv2.boundingRect, but first, you need to approximate the contour to a polygon-based set of points. This is the approach, continuing from the last line of code:
# Option2: Approximate the contour to a polygon:
contoursPoly = cv2.approxPolyDP(c, 3, True)
# Convert the polygon to a bounding rectangle:
boundRect = cv2.boundingRect(contoursPoly)
# Set the rectangle dimensions:
rectangleX = boundRect[0]
rectangleY = boundRect[1]
rectangleWidth = boundRect[0] + boundRect[2]
rectangleHeight = boundRect[1] + boundRect[3]
# Draw the rectangle:
cv2.rectangle(polyRectImage, (int(rectangleX), int(rectangleY)),
(int(rectangleWidth), int(rectangleHeight)), (0, 255, 0), 2)
cv2.imshow("Poly Rectangle", polyRectImage)
cv2.imwrite(path + "polyRectImage.png", polyRectImage)
cv2.waitKey(0)
This is the result:
Edit:
This is the bit that actually crops the lemon from the last image:
# Crop the ROI:
croppedImg = inputImage[rectangleY:rectangleHeight, rectangleX:rectangleWidth]
This is the final output:
Related
I'm not very good at image processing so this question may have a very simple answer. I'm not even sure that I use the right words to describe my problem.
I'm trying to rotate images taken by different cameras. For example look at the two images below. I have absolutely no information regarding the camera used. I'm trying to rotate the images so the sharpest point of the leaf is always looking up.
I also need a way to mesure to what extend the images were correctly rotated.
I have tried using the code below but I can't seem to get a lot of meaningful information. I think it's because the images are taken with different cameras.
import cv2
import imageio
import matplotlib.pyplot as plt
imgRef = imageio.imread("Ex2/Picture 538.jpg")
imgTest = imageio.imread("Ex2/Capture586_export001_r.JPG")
imgTest_grey = cv2.cvtColor(imgTest, cv2.COLOR_BGR2GRAY)
imgRef_grey = cv2.cvtColor(imgRef, cv2.COLOR_BGR2GRAY)
height, width = imgRef_grey.shape
orb = cv2.ORB_create(1000)
keypoints1, descriptors1 = orb.detectAndCompute(imgRef_grey, None)
keypoints2, descriptors2 = orb.detectAndCompute(imgTest_grey, None)
# Match features.
matcher = cv2.DescriptorMatcher_create(cv2.DESCRIPTOR_MATCHER_BRUTEFORCE_HAMMING)
matches = matcher.match( descriptors2, descriptors1, None)
# Sort matches by score
matches.sort(key=lambda x: x.distance, reverse=False)
# Remove not so good matches
numGoodMatches = int(len(matches) * 0.3)
matches = matches[:numGoodMatches]
# Draw top matches
imMatches = cv2.drawMatches(imgTest,keypoints2, imgRef, keypoints1, matches, None)
plt.imshow(imMatches)
plt.show()
All help is highly appreciated.
I would say that based on the shape of the leaf (especially the form of the sharpest point), the detection of the correct current angle and the rotation angle would be non-deterministic.
Any way id tolerance of 5 deg is ok for you the following is my suggestion.
get original image, make it grayscale, make it blurred and find the biggest contour(the leaf)
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
blurred = cv.GaussianBlur(gray, (17,17),0)
_, thresh_image = cv.threshold(blurred, 120, 255, cv.THRESH_BINARY_INV)
contours, hierarchy = cv.findContours(thresh_image, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
c_max = max(contours, key = cv.contourArea)
find the fitted ellipse over the leaf and get the angle of surrounding ellipse. this for sure does not consider the shape of the leaf and as a result, something that from your perspective is +90 is actually -90 (the sharpest point is on the other side).
So for example if we detect the angle to be 90 and then rotate is -90 it might happen that the sharpest point is downward, so the hard part would be the detection of the sharpest point being downward and if so rotate it again 180 degrees.
# remove the background so that the rotation is easier (since the aspect ratio of images are not 1)
img_bg_removed = img.copy()
stencil = np.zeros(img_bg_removed.shape[:-1]).astype(np.uint8)
cv.drawContours(stencil, [c_max], -1, color=(255, 255, 255), thickness=cv.FILLED)
sel = stencil != 255 # select everything that is not mask_value
img_bg_removed[sel] = (0,0,0)
#rotate the image
img_rotated = rotate_image(img_bg_removed, -1 * angle)
again find the contour and fitting rectangle. cut the fitting rectangle to half and see in which half the proportion of the white point(leaf) to all pixels is less. if the lower half had a lower proportion then the sharp point is there and the leaf should be rotated 180 again.
gray = cv.cvtColor(img_rotated, cv.COLOR_BGR2GRAY)
blurred = cv.GaussianBlur(gray, (17,17),0)
_, thresh_image = cv.threshold(blurred, 50, 255, cv.THRESH_BINARY)
contours, hierarchy = cv.findContours(thresh_image, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
c_max = max(contours, key = cv.contourArea)
x_rect,y_rect,w_rect,h_rect = cv.boundingRect(c_max)
up_side = thresh_image[y_rect: y_rect+h_rect//2, :][:, x_rect: x_rect + w_rect]
down_side = thresh_image[y_rect+h_rect//2: y_rect+h_rect, :][:, x_rect: x_rect + w_rect]
up_proportion = (up_side[:,:] == 255).sum() / left_side.size
down_proportion = (down_side[:,:] == 255).sum() / right_side.size
if down_proportion < up_proportion:
img_rotated = rotate_image(img_rotated, -180)
by the way, the rotation function is borrowed from here
PS: this solution can be optimized in a few ways. and one of which is the second Thresholding/Contour-finding can be removed. and simply as we find the surrounding ellipse we find the surrounding rectangle and then as we rotate the leaf we rotate the rectangle too.
This is my first question on Stackoverflow. I'm a little excited, forgive me if I'm wrong. We have mixed ellipses with and without overlapping drawn randomly from paint. I'm sharing the image I'm working on and my code. I am not a professional in opencv module, I wrote my code as a result of research inspired by sources.
The purpose of my code is,
Detection of randomly drawn with and without overlapping ellipses using the cv2.fitEllipse method. Next, find the major axis, minor axis and areas of the detected ellipses.
The problem with my code is actually this,
In overlapping ellipses, while fitting the ellipse under normal conditions, 2 ellipses should be fit, but about 6-7 ellipses are fit and I cannot reach the values I want to be calculated.
I'm open to your help, thank you in advance.
Example image:
import cv2
import numpy as np
import random as rng
import math
img = cv2.imread('overlapping_ellipses.png', 1)
imge= cv2.cvtColor(img,cv2.COLOR_RGB2BGR)
gray = cv2.cvtColor(imge, cv2.COLOR_BGR2GRAY)
blur = cv2.blur(gray, (2,2), 3)
edged = cv2.Canny(blur, 50, 100)
kernel= np.ones((2,2))
edged1 = cv2.dilate(edged, kernel, iterations=2)
edged2 = cv2.erode(edged1, kernel, iterations=2)
def thresh_callback(val):
threshold = val
canny_output = cv2.Canny(edged2, threshold, threshold * 4)
contours, _ = cv2.findContours(canny_output, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
minRect = [None]*len(contours)
minEllipse = [None]*len(contours)
for i, c in enumerate(contours):
minRect[i] = cv2.minAreaRect(c)
if c.shape[0] > 5:
minEllipse[i] = cv2.fitEllipse(c)
(x1,y1),(d1,d2),angle = minEllipse[i]
print('\nX1: ', round(x1,4), '\nY1: ', round(y1,4), '\nD1:',round(d1,4), '\nD2',round(d2,4), '\nAngle:', round(angle,4))
long= x1-d2
small= y1-d1
major= long/2
minor= small/2
pixel= 37.795275591
major1= major/pixel
minor1= minor/pixel
print('--------------------------------')
print('Major axis is: ', abs(round(major1,4)), 'cm')
print('Minor axis is: ', abs(round(minor1,4)), 'cm')
print('--------------------------------')
drawing = np.zeros((canny_output.shape[1], canny_output.shape[1], 3), dtype=np.uint8)
for i, c in enumerate(contours):
color = (rng.randint(0,256), rng.randint(0,256), rng.randint(0,256))
cv2.drawContours(drawing, contours, i, color)
if c.shape[0] > 5:
cv2.ellipse(drawing, minEllipse[i], color, 1)
cv2.imshow('Fitting Ellips', drawing)
source_window = 'Source'
cv2.namedWindow(source_window)
cv2.imshow(source_window, img)
max_thresh = 255
thresh = 100
cv2.createTrackbar('Canny Thresh:', source_window, thresh, max_thresh, thresh_callback)
thresh_callback(thresh)
cv2.waitKey()
Step 1: Identify and separate the blobs in the input image.
Since we don't care about colour information here, we can directly load the image as grayscale.
image = cv2.imread('input.png', cv2.IMREAD_GRAYSCALE)
The input image contains black ellipses on white background.
We only need the external contours of the blobs, and cv2.findContours expects white blobs on black background.
Therefore we need to invert the image. At the same time we need a binary image. We can use cv2.threshold to accomplish both tasks.
Once we detect the blob contours, we can collect some useful information for each blob into a simple map-based data structure.
def detect_blobs(image):
_,img_binary = cv2.threshold(image, 127, 255, cv2.THRESH_BINARY_INV)
contours, _ = cv2.findContours(img_binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
blobs = []
for i, contour in enumerate(contours):
orig_x, orig_y, width, height = cv2.boundingRect(contour)
roi_image = image[orig_y:orig_y+height,orig_x:orig_x+width]
blobs.append({
"i" : i
, "contour" : contour
, "origin" : (orig_x, orig_y)
, "size" : (width, height)
, "roi_image" : roi_image
})
return blobs
Step 2: Process each blob
First we need to determine whether the blob is a single ellipse, or whether it is a pair over intersecting ellipses.
One way to do this is by looking for convexity defects.
Since the coordinates of our contour are represented by integers, even the single-ellipse scenario will exhibit some convexity defects.
However, their magnitude (the distance between the furthest point on the contour from the enclosing convex hull segment) will be very small, generally below 1 pixel.
On the other hand, the contour of a pair of intersecting ellipses will have large convexity defects, one for each of the four points where the curves intersect.
This distinction can be seen on the following two images (contour is blue, convex hull red, identified intersection points/locations of large convexity defects are orange circles):
Single ellipse
Two intersecting ellipses
We therefore filter out any small convexity defects, and note the locations of the large ones. Now we're left with 3 possible scenarios.
Scenario A: No intersection points detected
Only small convexity defects were identified, which means this is very likely a single ellipse. We simply fit an ellipse to the contour and move on.
Scenario B: Exactly 4 intersection points detected
In this case we have 2 intersecting ellipses. We use the intersection points to split the contour into 4 segments, one for each "lobe" of the blob. Each of the segments should include the two intersection points that delimit it.
In the following picture, the segments are show in green, yellow, cyan and magenta, while the intersection points are orange circles:
Now, we can combine the pairs of segments that lie opposite each other (i.e. green+cyan and yellow+magenta) to get two lists of points, one for each ellipse. Again, we simply fit an ellipse to each list of points.
Scenario C: Some other number of intersection points detected
This is considered an invalid situation.
def process_blob(blob):
MAJOR_DEFECT_THRESHOLD = 2.0
contour = blob["contour"]
blob["hull"] = cv2.convexHull(contour)
hull_idx = cv2.convexHull(contour, returnPoints=False)
defects = cv2.convexityDefects(contour, hull_idx)
intersections = []
for i,defect in enumerate(np.squeeze(defects, 1)):
_, _, far_idx, far_dist = defect
real_far_dist = far_dist / 256.0
if real_far_dist >= MAJOR_DEFECT_THRESHOLD:
intersections.append(far_idx)
if len(intersections) == 0:
print("One ellipse")
blob["ellipses"] = [cv2.fitEllipse(contour)]
elif len(intersections) == 4:
print("Two ellipses")
blob["segments"] = [
contour[intersections[0]:intersections[1]+1]
, contour[intersections[1]:intersections[2]+1]
, contour[intersections[2]:intersections[3]+1]
, np.vstack([contour[intersections[3]:],contour[:intersections[0]+1]])
]
split_contours = [
np.vstack([blob["segments"][0], blob["segments"][2]])
, np.vstack([blob["segments"][1], blob["segments"][3]])
]
blob["ellipses"] = [cv2.fitEllipse(c) for c in split_contours]
else:
print("Invalid scenario")
blob["ellipses"] = []
return blob["ellipses"]
At this point, it's trivial to calculate the parameters you need -- I'll leave this as an excercise to the reader.
As a bonus, here's some simple visualization for debugging purposes:
def visualize_blob(blob):
PADDING = 20
orig_x, orig_y = blob["origin"]
offset = (orig_x - PADDING, orig_y - PADDING)
input_img = cv2.copyMakeBorder(blob["roi_image"]
, PADDING, PADDING, PADDING, PADDING
, cv2.BORDER_CONSTANT, None, 255)
adjusted_img = cv2.add(input_img, 127) - 63
output_img_ch = cv2.cvtColor(adjusted_img, cv2.COLOR_GRAY2BGR)
output_img_seg = output_img_ch.copy()
output_img_el = output_img_ch.copy()
cv2.drawContours(output_img_ch, [blob["hull"] - offset], 0, (127,127,255), 4)
cv2.drawContours(output_img_ch, [blob["contour"] - offset], 0, (255,127,127), 2)
SEGMENT_COLORS = [(0,255,0),(0,255,255),(255,255,0),(255,0,255)]
if "segments" in blob:
for i in range(4):
cv2.polylines(output_img_seg, [blob["segments"][i] - offset], False, SEGMENT_COLORS[i], 4)
for i in range(4):
center = (blob["segments"][i] - offset)[0][0]
cv2.circle(output_img_ch, center, 4, (0,191,255), -1)
cv2.circle(output_img_seg, center, 4, (0,191,255), -1)
for ellipse in blob["ellipses"]:
offset_ellipse = ((ellipse[0][0] - offset[0], ellipse[0][1] - offset[1]), ellipse[1], ellipse[2])
cv2.ellipse(output_img_el, offset_ellipse, (0,0,255), 2)
cv2.imshow('', np.hstack([output_img_ch,output_img_seg, output_img_el]))
cv2.imwrite('output_%d_ch.png' % blob["i"], output_img_ch)
cv2.imwrite('output_%d_seg.png' % blob["i"], output_img_seg)
cv2.imwrite('output_%d_el.png' % blob["i"], output_img_el)
cv2.waitKey()
Pulling it all together:
import cv2
import numpy as np
## INSERT THE FUNCTIONS LISTED ABOVE IN THE QUESTION ##
image = cv2.imread('input.png', cv2.IMREAD_GRAYSCALE)
blobs = detect_blobs(image)
print("Found %d blob(s)." % len(blobs))
for blob in blobs:
process_blob(blob)
visualize_blob(blob)
Our team set up a vision system with a camera, a microscope and a tunable lens to look at the internal surface of a cone.
Visually speaking, the camera takes 12 image for one cone with each image covering 30 degrees.
Now we've collected many sample images and want to make sure each "fan"(as shown below) is at least 30 degree.
Is there any way in Python, with cv2 or other packages, to measure this central angle. Thanks.
Here is one way to do that in Python/OpenCV.
Read the image
Convert to gray
Threshold
Use morphology open and close to smooth and fill out the boundary
Apply Canny edge extraction
Separate the image into top edge and bottom edge by blackening the opposite side to each edge
Fit lines to the top and bottom edges
Compute the angle of each edge
Compute the difference between the two angles
Draw the lines on the input
Save the results
Input:
import cv2
import numpy as np
import math
# read image
img = cv2.imread('cone_shape.jpg')
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# threshold
thresh = cv2.threshold(gray,11,255,cv2.THRESH_BINARY)[1]
# apply open then close to smooth boundary
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (13,13))
morph = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel)
kernel = np.ones((33,33), np.uint8)
morph = cv2.morphologyEx(morph, cv2.MORPH_CLOSE, kernel)
# apply canny edge detection
edges = cv2.Canny(morph, 150, 200)
hh, ww = edges.shape
hh2 = hh // 2
# split edge image in half vertically and blacken opposite half
top_edge = edges.copy()
top_edge[hh2:hh, 0:ww] = 0
bottom_edge = edges.copy()
bottom_edge[0:hh2, 0:ww] = 0
# get coordinates of white pixels in top and bottom
# note: need to transpose y,x in numpy to x,y for opencv
top_white_pts = np.argwhere(top_edge.transpose()==255)
bottom_white_pts = np.argwhere(bottom_edge.transpose()==255)
# fit lines to white pixels
# (x,y) is point on line, (vx,vy) is unit vector along line
(vx1,vy1,x1,y1) = cv2.fitLine(top_white_pts, cv2.DIST_L2, 0, 0.01, 0.01)
(vx2,vy2,x2,y2) = cv2.fitLine(bottom_white_pts, cv2.DIST_L2, 0, 0.01, 0.01)
# compute angle for vectors vx,vy
top_angle = (180/math.pi)*math.atan(vy1/vx1)
bottom_angle = (180/math.pi)*math.atan(vy2/vx2)
print(top_angle, bottom_angle)
# cone angle is the difference
cone_angle = math.fabs(top_angle - bottom_angle)
print(cone_angle)
# draw lines on input
lines = img.copy()
p1x1 = int(x1-1000*vx1)
p1y1 = int(y1-1000*vy1)
p1x2 = int(x1+1000*vx1)
p1y2 = int(y1+1000*vy1)
cv2.line(lines, (p1x1,p1y1), (p1x2,p1y2), (0, 0, 255), 1)
p2x1 = int(x2-1000*vx2)
p2y1 = int(y2-1000*vy2)
p2x2 = int(x2+1000*vx2)
p2y2 = int(y2+1000*vy2)
cv2.line(lines, (p2x1,p2y1), (p2x2,p2y2), (0, 0, 255), 1)
# save resulting images
cv2.imwrite('cone_shape_thresh.jpg',thresh)
cv2.imwrite('cone_shape_morph.jpg',morph)
cv2.imwrite('cone_shape_edges.jpg',edges)
cv2.imwrite('cone_shape_lines.jpg',lines)
# show thresh and result
cv2.imshow("thresh", thresh)
cv2.imshow("morph", morph)
cv2.imshow("edges", edges)
cv2.imshow("top edge", top_edge)
cv2.imshow("bottom edge", bottom_edge)
cv2.imshow("lines", lines)
cv2.waitKey(0)
cv2.destroyAllWindows()
Thresholded image:
Morphology processed image:
Edge Image:
Lines on input:
Cone Angle (in degrees):
42.03975696357633
That sounds possible. You need to do some preprocessing and filtering to figure out what works and there is probably some tweaking involved.
There are three approaches that could work.
1.)
The basic idea is to somehow get two lines and measure the angle between them.
Define a threshold to define the outer black region (out of the central angle) and set all values below it to zero.
This will also set some of the blurry stripes inside the central angle to zero so we have to try to "heal" them away. This is done by using Morphological Transformations. You can read about them here and here.
You could try the operation Closing, but I don't know if it fixes stripes. Usually it fixes dots or scratches. This answer seems to indicate that it should work on lines.
Maybe at that point apply some Gaussian blurring and to the threshold thing again. Then try to use some edge or line detection.
It's basically try and error, you have to see what works.
2.)
Another thing that could work is to try to use the arc-enter code herelike scratches, maybe even strengthen them and use the Hough Circle Transform. I think it detects arcs as well.
Just try it and see what the function returns. In the best case there are several circles / arcs that you can use to estimate the central angle.
There are several approaches on arc detection here on StackOverflow or here.
I am not sure if that's the same with all your image, but the one above looks like there are some thin, green and pink arcs that seem to stretch all along the central angle. You could use that to filter for that color, then make it grey scale.
This question might be helpful.
3.)
Apply an edge filter, e.g Canny skimage.feature.canny
Try several sigmas and post the images in your question, then we can try to think on how to continue.
What could work is to calculate the convex hull around all points that are part of an edge. Then get the two lines that form the central angle from the convex hull.
I'm trying to detect CText fields on rasterized PDFs by converting them to PNG format then analyzing the images with OpenCV.
So far I've been able to produce a set of corners that are relatively accurate with the following code:
image = cv2.imread(img_filename)
# add 10px border padding (white)
image = cv2.copyMakeBorder(image, 10, 10, 10, 10, cv2.BORDER_CONSTANT, value=[255, 255, 255])
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
gray = np.float32(gray)
corners = cv2.cornerHarris(gray, 2, 3, 0.1)
cond = corners>0.0001*corners.max()
image[cond] = [0,0,255]
cv2.imshow('dst', image)
cv2.waitKey(0)
Here's my input:
And here's the output:
I want to be able to compute from this set of points, which points are part of a rectangle. This would involve checking that for each corner point, three other corner points exist:
A point in the same row (approximately), but different column
A point in the same column (approximately), but different row
A point with the column from (1) above and the row from (2) above
The goal is to produce a set of rectangles that do not contain any [significant] amount of corner points inside of them, unless those corner points belong to a rectangle themselves (in the case of the bounding rectangle of each inner rectangle, for ex.)
What is the most efficient way to do this in OpenCV?
I've read a lot about the Circular Hough transform on Stack Overflow, but I seem to be missing something. I wrote a program that is supposed to detect the circles of a "Bull's Eye" target. However, even after playing with the parameters, the algorithm is quite bad - it ignores most of the circles and one time it finds a circle but seems to "wander off". I've even tried applying an "Unsharp Mask" to no avail. I have added my code, the image I started with and the output. I hope someone can point me at the right direction.
import cv2
import cv2.cv as cv
import numpy as np
import math
# Load Image
img = cv2.imread('circles1.png',0)
# Apply Unsharp Mask
tmp = cv2.medianBlur(img,5)
img = cv2.addWeighted(img,1.5,tmp,-0.5,0)
cimg = cv2.cvtColor(img,cv2.COLOR_GRAY2BGR)
# Hough Transform
circles = cv2.HoughCircles(img,cv.CV_HOUGH_GRADIENT,1,5,
param1=100,param2=100,minRadius=0,maxRadius=0)
circles = np.uint16(np.around(circles))
# Go over circles, eliminating the ones that are not cocentric enough
height, width = img.shape
center = (width/2,height/2)
for i in circles[0,:]:
# draw the outer circle
if math.sqrt((center[0]-i[0])**2 + (center[1]-i[1])**2) < 15:
cv2.circle(cimg,(i[0],i[1]),i[2],(0,255,0),1)
# draw the center of the circle
cv2.circle(cimg,(i[0],i[1]),2,(0,0,255),3)
cv2.imshow('detected circles',cimg)
cv2.waitKey(0)
cv2.destroyAllWindows()
Quick explanation: I load the image, apply Unsharp Mask, use the Hough Transfrom to detect circles, then draw the circles that are close to the center (I found that the other circles are false circles).
I tried playing with the parameters, and this is the best I got. I feel like this is a simple enough problem which has me buffled. I appriciate any help.
My input image:
My output image:
As I mentioned in my comment, you'll need to run successive iterations of cv2.HoughCircles for different range of radii to ensure that you get all of the circles. With the way the Circular Hough Transform works, specifying a minimum and maximum radius that has quite a large range will be inaccurate and will also be slow. They don't tell you this in the documentation, but for the Circular Hough Transform to work successfully, the following two things need to be valid:
maxRadius < 3*minRadius
maxRadius - minRadius < 100
With the above, blindly making the minimum radius very small and the maximum radius very large won't give you great results. Therefore, what you could do is start at... say...radius=1, then iterate up to radius=300 in steps of 20. Between each chunk of 20, run cv2.HoughCircles and update your image with these contours.
Doing this requires very little modification to your code. BTW, I removed the unsharp masking it because I was getting poor results with it. I also changed a couple of parameters in cv2.HoughCircles slightly to get this to work as best as possible given your situation:
import cv2
import cv2.cv as cv
import numpy as np
import math
# Load Image
img = cv2.imread('circles1.png',0)
cimg = cv2.cvtColor(img,cv2.COLOR_GRAY2BGR)
# Specify different radii
radii = np.arange(0,310,10)
# For each pair of radii...
for idx in range(len(radii)-1):
# Get the minimum and maximum radius
# Note you need to add 1 to each minimum
# as the maximum of the previous pair covers this new minimum
minRadius = radii[idx]+1
maxRadius = radii[idx+1]
# Hough Transform - Change here
circles = cv2.HoughCircles(img,cv.CV_HOUGH_GRADIENT,1,5,
param1=25,param2=75,minRadius=minRadius,maxRadius=maxRadius)
# Skip if no circles are detected - Change here
if circles is None:
continue
circles = np.uint16(np.around(circles))
# Go over circles, eliminating the ones that are not cocentric enough
height, width = img.shape
center = (width/2,height/2)
for i in circles[0,:]:
# draw the outer circle
if math.sqrt((center[0]-i[0])**2 + (center[1]-i[1])**2) < 15:
cv2.circle(cimg,(i[0],i[1]),i[2],(0,255,0),1)
# draw the center of the circle
cv2.circle(cimg,(i[0],i[1]),2,(0,0,255),3)
cv2.imshow('detected circles',cimg)
cv2.waitKey(0)
cv2.destroyAllWindows()
I get this figure:
Unfortunately it isn't perfect as it doesn't detect all of the circles. You'll have to play around with the cv2.HoughCircles function until you get good results.
However, I wouldn't recommend using cv2.HoughCircles here. May I suggest using cv2.findContours instead? This finds all of the contours in the image. In this case, these will be the black circles. However, you need to reverse the image because cv2.findContours assumes non-zero pixels are object pixels, so we can subtract 255 from the image assuming a np.uint8 type:
# Make copy of original image
cimg2 = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
# Find contours
contours,_ = cv2.findContours(255 - img, cv2.RETR_LIST, cv.CV_CHAIN_APPROX_NONE)
# Draw all detected contours on image in green with a thickness of 1 pixel
cv2.drawContours(cimg2, contours, -1, color=(0,255,0), thickness=1)
# Show the image
cv2.imshow('detected circles', cimg2)
cv2.waitKey(0)
cv2.destroyAllWindows()
This is what I get: