Related
In the image above, I want to eventually be able to fill in the contours of the colored circles. Unfortunately, the yellow circle on the bottom is right at the edge of the image, so cv2.findContours() doesn't work:
_, green_contours, _ = cv2.findContours(green_seg, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
Does anyone know how I can somehow fill in the yellow circle contour even though it's at the edge of the image? Since it's at the edge of the image, the pixel value on the bottom edge of the image doesn't complete the circle, isn't valued at 255, if that makes sense.
I looked online and some people say it's possible to draw a big box around the image and then do the contours, but if I do that, then opencv only draws the big contour around the entire image and not the circle.
Any other thoughts would be greatly appreciated!
EDIT: The image I show above is just one case I'm considering. I'm trying to think how I can make this general enough such that for any contour that is at the edge of the border, I can still fill in the contour with cv2.drawContour().
If you have (more or less) convex polygons, you actually CAN use cv2.findContours. Having the contours, try to find the center of mass, e.g. using cv2.moments, and then use this as the seed point in cv2.floodFill.
Please see the following code snippet. I assume, you can identify your polygons by color. Also, instead of some advanced finding of the center of mass, I just used the center point of the bounding rectangle of each contour. Maybe, that's also sufficient for your use case!?
import cv2
import numpy as np
# Set up test image
colors = [(0, 255, 0), (0, 0, 255)]
input = np.zeros((400, 400, 3), np.uint8)
cv2.circle(input, (100, 100), 50, colors[0], 10)
cv2.circle(input, (150, 350), 75, colors[1], 10)
output = input.copy()
# Iterate all colors...
for i, c in enumerate(colors):
# Mask color
img = np.all(output == c, axis=2).astype(np.uint8) * 255
# Find contours with respect to OpenCV version
cnts = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
# Get bounding rectangles; derive seed points for flood filling; flood fill
rects = [cv2.boundingRect(c) for c in cnts]
seeds = [(np.int32(r[0] + r[2] / 2), np.int32(r[1] + r[3] / 2)) for r in rects]
[cv2.floodFill(output, mask=None, seedPoint=s, newVal=c) for s in seeds]
cv2.imshow('input', input)
cv2.imshow('output', output)
cv2.waitKey(0)
cv2.destroyAllWindows()
That's the input:
And, that's the output:
Hope that helps!
----------------------------------------
System information
----------------------------------------
Platform: Windows-10-10.0.16299-SP0
Python: 3.8.1
NumPy: 1.18.1
OpenCV: 4.1.2
----------------------------------------
A simple way, perhaps not so precise, is to compute the convex hull for each contour and draw the interior with a fill color:
import cv2
import numpy as np
import sys
# load input image from the cmd-line
img = cv2.imread('test_images/partial_contour.png')
if (img is None):
print('!!! Failed imread')
sys.exit(-1)
output_img = img.copy()
# isolate just the colored drawings
ret, thres_bgr = cv2.threshold(img, 0, 255, cv2.THRESH_BINARY)
thres_bgr[np.where((thres_bgr == [255,255,255]).all(axis=2))] = [0,0,0] # replace white pixels for black
# convert from 3-channels (BGR) to a single channel (gray)
gray_img = cv2.cvtColor(thres_bgr, cv2.COLOR_BGR2GRAY)
# this loop processes all the contours found in the image
contours, hierarchy = cv2.findContours(gray_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
for contourIdx, cnt in enumerate(contours):
# compute a convex hull
hull = cv2.convexHull(cnt)
# fill the inside with red
cv2.fillPoly(output_img, pts=[hull], color=(0, 0, 255))
cv2.imshow('output_img', output_img)
cv2.imwrite('fill_partial_cnt_output.png', output_img)
cv2.waitKey(0)
I am trying to find accurate locations for the corners on ink blotches as seen below:
My idea is to fit lines to the edges and then find where they intersect. As of now, I've tried using cv2.approxPolyDP() with various values of epsilon to approximate the edges, however this doesn't look like the way to go. My cv2.approxPolyDP code gives the following result:
Ideally, this is what I want to produce (drawn on paint):
Are there CV functions in place for this sort of problem? I've considered using Gaussian blurring before the threshold step although that method does not seem like it would be very accurate for corner finding. Additionally, I would like this to be robust to rotated images, so filtering for vertical and horizontal lines won't necessarily work without other considerations.
Code:*
import numpy as np
from PIL import ImageGrab
import cv2
def process_image4(original_image): # Douglas-peucker approximation
# Convert to black and white threshold map
gray = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY)
gray = cv2.GaussianBlur(gray, (5, 5), 0)
(thresh, bw) = cv2.threshold(gray, 128, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
# Convert bw image back to colored so that red, green and blue contour lines are visible, draw contours
modified_image = cv2.cvtColor(bw, cv2.COLOR_GRAY2BGR)
contours, hierarchy = cv2.findContours(bw, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(modified_image, contours, -1, (255, 0, 0), 3)
# Contour approximation
try: # Just to be sure it doesn't crash while testing!
for cnt in contours:
epsilon = 0.005 * cv2.arcLength(cnt, True)
approx = cv2.approxPolyDP(cnt, epsilon, True)
# cv2.drawContours(modified_image, [approx], -1, (0, 0, 255), 3)
except:
pass
return modified_image
def screen_record():
while(True):
screen = np.array(ImageGrab.grab(bbox=(100, 240, 750, 600)))
image = process_image4(screen)
cv2.imshow('window', image)
if cv2.waitKey(25) & 0xFF == ord('q'):
cv2.destroyAllWindows()
break
screen_record()
A note about my code: I'm using screen capture so that I can process these images live. I have a digital microscope that can display live feed on a screen, so the constant screen recording will allow me to sample from the video feed and locate the corners live on the other half of my screen.
Here's a potential solution using thresholding + morphological operations:
Obtain binary image. We load the image, blur with cv2.bilateralFilter(), grayscale, then Otsu's threshold
Morphological operations. We perform a series of morphological open and close to smooth the image and remove noise
Find distorted approximated mask. We find the bounding rectangle coordinates of the object with cv2.arcLength() and cv2.approxPolyDP() then draw this onto a mask
Find corners. We use the Shi-Tomasi Corner Detector already implemented as cv2.goodFeaturesToTrack() for corner detection. Take a look at this for an explanation of each parameter
Here's a visualization of each step:
Binary image -> Morphological operations -> Approximated mask -> Detected corners
Here are the corner coordinates:
(103, 550)
(1241, 536)
Here's the result for the other images
(558, 949)
(558, 347)
Finally for the rotated image
(201, 99)
(619, 168)
Code
import cv2
import numpy as np
# Load image, bilaterial blur, and Otsu's threshold
image = cv2.imread('1.png')
mask = np.zeros(image.shape, dtype=np.uint8)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.bilateralFilter(gray,9,75,75)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Perform morpholgical operations
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (10,10))
opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel, iterations=1)
close = cv2.morphologyEx(opening, cv2.MORPH_CLOSE, kernel, iterations=1)
# Find distorted rectangle contour and draw onto a mask
cnts = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
rect = cv2.minAreaRect(cnts[0])
box = cv2.boxPoints(rect)
box = np.int0(box)
cv2.drawContours(image,[box],0,(36,255,12),4)
cv2.fillPoly(mask, [box], (255,255,255))
# Find corners
mask = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
corners = cv2.goodFeaturesToTrack(mask,4,.8,100)
offset = 25
for corner in corners:
x,y = corner.ravel()
cv2.circle(image,(x,y),5,(36,255,12),-1)
x, y = int(x), int(y)
cv2.rectangle(image, (x - offset, y - offset), (x + offset, y + offset), (36,255,12), 3)
print("({}, {})".format(x,y))
cv2.imshow('image', image)
cv2.imshow('thresh', thresh)
cv2.imshow('close', close)
cv2.imshow('mask', mask)
cv2.waitKey()
Note: The idea for the distorted bounding box came from a previous answer in How to find accurate corner positions of a distorted rectangle from blurry image
After seeing the description of the corners, here is what I would recommend:
by any method, find the gross location of the corners (for instance by looking for the extreme values of (±X+±Y, ±X+±Y) or (±X, ±Y)).
consider a strip than joins two corners, with a certain width. Extract the pixels in that strip, on a portion close to the corner, rotate to make it horizontal and average the values along horizontals.
you will obtain a gray profile that tells you the accurate position of the edge, at the mean between the background and foreground intensities.
repeat on all four edges and at both ends. This will give you four accurate corners, by intersection.
I am attempting to draw a bounding box around the two rectangles that are found in the image, but not including the curvy lined 'noise'
I have tried multiple methods, including Hough Line Transform and trying to extract coordinates, but to no avail. My methods seemed too arbitrary, and I tried to find the black space between the true rectangles and the noise at the top of the frames but couldn't get a solid general algorithm that could fit that in.
This is not so simple to do, you can try to isolate the vertical lines which are quite distinguishable, dilate/erode to make the rectangle a rectangle, and find the contours of what it is left and filter them accordingly... The code would look like:
import numpy as np
import cv2
minArea = 20 * 20 # area of 20 x 20 pixels
# load image and threshold it
original = cv2.imread("a.png")
img = cv2.cvtColor(original, cv2.COLOR_BGR2GRAY)
ret, thres = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY )
# Get the vertical lines
verticalStructure = cv2.getStructuringElement(cv2.MORPH_RECT, (1, 10))
vertical = cv2.erode(thres, verticalStructure)
vertical = cv2.dilate(vertical, verticalStructure)
# close holes to make it solid rectangle
kernel = np.ones((45,45),np.uint8)
close = cv2.morphologyEx(vertical, cv2.MORPH_CLOSE, kernel)
# get contours
im2, contours, hierarchy = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# draw the contours with area bigger than a minimum and that is almost rectangular
for cnt in contours:
x,y,w,h = cv2.boundingRect(cnt)
area = cv2.contourArea(cnt)
if area > (w*h*.60) and area > minArea:
original = cv2.rectangle(original, (x,y),(x+w,y+h), (0,0,255), 3)
cv2.imshow("image", original)
cv2.waitKey(0)
cv2.destroyAllWindows()
And the result is:
If it does not work with other images, try adjusting the parameters.
Let's say I have an image like this:
For which I want to remove the border and get this:
So exactly the same image, without the border.
I found a "hacky" way to do it which finds the outer contour and draws a line over it... To be honest it's not the best way because I need to adjust the "thickness" of the line so that it is thick enough to cover up the border, but not too thick so it doesn't cover any of the circles.
The image variable is the image you see above (already grayscaled, thresholded).
cnts = cv2.findContours(image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if imutils.is_cv2() else cnts[1]
cv2.drawContours(image, cnts, -1, 0, 15) # 15 is the right thickness for this image, but might not be for other ones...
The results is the 2nd picture above. Works great, but it doesn't work for ALL images (because of different thickness). Is there a better way to do this?
This is what I meant in the comments... fill everything that is black like the top-left corner, and connected to it, with white so that the bit you want is now entirely surrounded by white all the way to the edges. Then fill everything that is white and connected to the top-left corner with black.
#!/usr/local/bin/python3
import numpy as np
import cv2
from PIL import Image
# Load image and greyscale it
im = np.array(Image.open("framedcircles.jpg").convert('L'))
# Normalize and threshold image
im = cv2.normalize(im, None, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX)
res, im = cv2.threshold(im, 64, 255, cv2.THRESH_BINARY)
# Fill everything that is the same colour (black) as top-left corner with white
cv2.floodFill(im, None, (0,0), 255)
# Fill everything that is the same colour (white) as top-left corner with black
cv2.floodFill(im, None, (0,0), 0)
# Save result
Image.fromarray(im).save("result.png")
Result:
Here I am considering you had done all the image processing and others.
The recommended threshold (cv2.THRESH_OTSU)!
mask = np.zeros(img.shape, dtype=img.dtype)
cont = cv2.findContours(img_th, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cont = cont[0] if len(cont) == 2 else cont[1]
for c in cont:
area = cv2.contourArea(c)
if area < 500:
cv2.drawContours(mask, [c], -1, (255, 255, 255), -1)
print(mask.shape)
mask = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY) # making it to 2D (removing the channel value)
removed_border = cv2.bitwise_and(img, img, mask=mask)
cv2.imshow("result", removed_border)
cv2.waitKey()
cv2.destroyAllWindows()
If you have any problem with understanding the code please let me know. I m happy to help you out and if you like the answer please thumps up!!! :)
Result:
I have a little hack for this.
If you're falling low on thickness, repeat the process.
Again, it's not the best way to do it, but it should work.
I'm trying to spot black rectangles using OpenCV in Python 2.7. I'm confused as to why my contour-finding code isn't spotting the black rectangle at the bottom of a PNG (downloadable image, pre-grayscale):
Obviously, I want to spot the large black box. Here's my code:
import cv2
import numpy as np
img = cv2.imread(f)
# grayscale
imgrey = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# threshold and invert
ret, thresh = cv2.threshold(imgrey, 127, 255, cv2.THRESH_BINARY_INV)
# find contours, and draw red highlights around them
contours, h = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
for cnt in contours:
x, y, w, h = cv2.boundingRect(cnt)
mask = np.zeros(imgrey.shape, np.uint8)
cv2.drawContours(mask, [cnt], 0, 255, -1)
cv2.rectangle(img, (x-2, y), (x+w+2, y+h), (0, 0, 255), 3)
cv2.imwrite(f.replace('.png', '-output.png'), img)
I get output that looks like this - it seems to be doing a brilliant job of spotting every possible contour except the one I'm interested in (note the lack of red line around the black box):
I can easily find ways to exclude the smaller contours (e.g. by looking at the mean colour value). But simply not spotting the contour of the black box, I'm not sure what to do about that.
What am I doing wrong?
For reference, this is what thresh looks like if I save it - the threshold seems to be working OK:
try Eroding and Dilating before findContours like below ( sorry i am not familiar with python)
erode(thresh,thresh,Mat(),Point(-1,-1),20);
dilate(thresh,thresh,Mat(),Point(-1,-1),20);
by this way you will get only one contour