I am trying to split this image by the gray color of each rectangle.
The idea is to return the coordinate of each corner so I can use it to process what is inside.
I am a OpenCV newbie so I would like to know what is the best approach to do this. Is findContours enough to get these coordinates or is there a better function for this?
Regards
It simple to solve using findContours:
Read input image as Grayscale (not as RGB).
Apply threshold, and inverse polarity (make all gray pixel white, and background black).
Find contours on threshold image.
Find bounding rectangle of each contour.
The solution draws a green rectangle around each contour, for testing.
Here is a working code sample:
import numpy as np
import cv2
# Read input image as Grayscale
img = cv2.imread('boxes.png', cv2.IMREAD_GRAYSCALE)
# Convert img to uint8 binary image with values 0 and 255
# All white pixels above 250 goes to 255, and other pixels goes to 0
ret, thresh_gray = cv2.threshold(img, 250, 255, cv2.THRESH_BINARY)
# Inverse polarity
thresh_gray = 255 - thresh_gray
# Find contours in thresh_gray.
contours = cv2.findContours(thresh_gray, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[-2] # [-2] indexing takes return value before last (due to OpenCV compatibility issues).
corners = []
# Iterate contours, find bounding rectangles, and add corners to a list
for c in contours:
# Get bounding rectangle
x, y, w, h = cv2.boundingRect(c)
# Append corner to list of corners - format is corners[i] holds a tuple: ((x0, y0), (x1, y1))
corners.append(((x, y), (x+w, y+h)))
# Convert grayscale to BGR (just for testing - for drawing rectangles in green color).
out = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
# Draw green rectangle (for testing)
for c in corners:
cv2.rectangle(out, c[0], c[1], (0, 255, 0), thickness = 2)
cv2.imwrite('out.png', out) #Save out to file (for testing).
# Show result (for testing).
cv2.imshow('out', out)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
List of corners:
corners = [((491, 153), (523, 181)),
((24, 151), (68, 178)),
((231, 123), (277, 158)),
((442, 103), (488, 131)),
((7, 99), (76, 132)),
((211, 75), (285, 110)),
((268, 57), (269, 58)),
((420, 49), (494, 84)),
((5, 47), (58, 83)),
((213, 18), (267, 59)),
((420, 0), (477, 33))]
Related
I'm having problem extracting a square shape ( the square end of the yellow container in the image below ) from a colour image.
There is no visual break between the square shape and background which also has similar shades of colour as square shape.
You can see the trees above and to the right of the container that create the problem.
Ive tried to crate a yellow HSV colour bounds, but get a result ( black and white image below ) , but it wont fit the bounding box to the container end.
Any help appreciated.
import cv2 as cv
im = cv.imread("check1.jpg")
# Convert BGR ( not RGB ) to HSV as OpenCV works in BGR, not RGB.
hsv = cv.cvtColor(im, cv.COLOR_BGR2HSV)
cv.imshow('BGR2HSV__hsv-1',hsv)
# set range of yellowish colour, in HSV
lower = (8, 0, 0)
upper = (78, 255, 255)
# Threshold the hSV range to get only yellow colours
mask1 = cv.inRange(hsv, lower, upper)
#erode the image to get rid of any background dross.
mask2 = cv.erode(mask1, kernel=None, iterations=2)
(x, y, w, h) = cv.boundingRect(mask2)
canvas = cv.cvtColor(mask2, cv.COLOR_GRAY2BGR)
final = cv.rectangle(canvas, (x,y), (x+w, y+h), color=(0,0,255), thickness=3)
cv.imshow('bounding box-final',final)
cv.waitKey(0)
enter image description here
I have foreground mask, like this
If I use findContours method I receive this result
I want to find only main contours on hand without trash. How can I do it?
thresh = 100
# get threshold image
ret, thresh_img = cv.threshold(fgMask, thresh, 255, cv.THRESH_BINARY)
# find contours
contours, hierarchy = cv.findContours(thresh_img, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
# create an empty image for contours
img_contours = np.zeros(img.shape)
# draw the contours on the empty image
cv.drawContours(img_contours, contours, -1, (0, 255, 0), 3)
cv.imwrite("img.png", fgMask)
cv.imwrite("img1.png", img_contours)
Maybe you can try blurring image before threshold to reduce the noise.
However, your foreground mask is quite bad actually for this purpose. Since I cannot see what you have done before, it is generally better to use HSV color space if you want to achieve human skin. Because
The color of human skin is created by a combination of blood (red) and
melanin (yellow, brown). Skin colors lie between these two extreme
hues and are somewhat saturated.
HSV: (Hue-Saturation-Value) defined in a way that is similar to how
humans perceive color.
img_path = "hand.jpg"
img = cv.imread(img_path)
# define the upper and lower boundaries of the HSV pixel intensities
# to be considered 'skin'
hsvim = cv.cvtColor(img, cv.COLOR_BGR2HSV)
lower = np.array([0, 48, 80], dtype="uint8")
upper = np.array([20, 255, 255], dtype="uint8")
skinMask= cv.inRange(hsvim, lower, upper)
# blur the mask to help remove noise
skinMask= cv.blur(skinMask, (2, 2))
# get threshold image
ret, thresh = cv.threshold(skinMask, 100, 255, cv.THRESH_BINARY)
cv.imshow("thresh", thresh)
# draw the contours on the empty image
contours, hierarchy = cv.findContours(thresh, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
contours = max(contours, key=lambda x: cv.contourArea(x))
cv.drawContours(img, [contours], -1, (255, 255, 0), 2)
cv.imshow("contours", img)
cv.waitKey()
should resulted better like
Code Reference
I'm attempting to remove all but the largest contours while retaining the child contours within the largest contour. Effectively I want to go from here: Input Image to: Output Image I don't know how to go about this though (I'm not a programmer by any stretch and had help putting together the below). The code I do have doesn't retain the child contours. Output of Below.
import cv2 as cv
import numpy as np
img = cv.imread('small_contour.jpg')
image_contours = np.zeros((img.shape[1], img.shape[0], 1), np.uint8)
image_binary = np.zeros((img.shape[1], img.shape[0], 1), np.uint8)
for channel in range(img.shape[2]):
ret, image_thresh = cv.threshold(img[:, :, channel], 127, 255, cv.THRESH_BINARY)
contours = cv.findContours(image_thresh, 1, 1)[0]
cv.drawContours(image_contours, contours, -1, (255,255,255), 3)
contours = cv.findContours(image_contours, cv.RETR_LIST,
cv.CHAIN_APPROX_SIMPLE)[0]
cv.drawContours(image_binary, [max(contours, key = cv.contourArea)],
-1, (255, 255, 255), -1)
cv.imwrite('fill_contour.jpg', image_binary)
cv.imshow('Small Contour', image_binary)
cv.waitKey(0)
What your code does in this line cv.drawContours(image_binary, [max(contours, key = cv.contourArea)], -1, (255, 255, 255), -1) is to set to color (255, 255, 255) the pixels of the image_binary corresponding to the boundary and inner pixels of the bigger contour found by cv.findContours.
Creating the image_binary you are also swapping width with height and this is an error.
You don't need to keep the children: you should use cv.RETR_TREE for that, then find the index of the father and look for all contours with that father doing it recursively for the nested contours (children of children ...). See https://docs.opencv.org/4.1.0/d9/d8b/tutorial_py_contours_hierarchy.html But this is not useful for your goal.
You can use the result you currently have as a mask for the original image, but the code need some fixes, starting from the linked image named VS0L9.jpg.
Note that I'm using cv2, just change to cv.
Load the image
im = cv2.imread('VS0L9.jpg', cv2.IMREAD_GRAYSCALE)
Find the contours (I used RETR_TREE)
contours, hierarchy = cv2.findContours(im, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
Find the bigger contour:
bigger = max(contours, key=lambda item: cv2.contourArea(item))
Initialize the mask (what you tried with image_binary):
the_mask = np.zeros_like(im)
Draw the fill the bigger contour on the mask image, the region of interest is set to (255, 255, 255):
cv2.drawContours(the_mask, [bigger], -1, (255, 255, 255), cv2.FILLED)
At this point the mask looks like the output of your code, but with proper width and height.
Use the mask with the original image to show only the area of interest:
res = cv2.bitwise_and(im, im, mask = the_mask)
Or, alternatively:
res = im.copy()
res[the_mask == 0] = 0
Now res is the desired result.
You can use morphological reconstruction, with erode image as marker.
import cv2
img = cv2.imread('VS0L9.jpg', cv2.IMREAD_GRAYSCALE)
thresh = cv2.threshold(img, 40, 255, cv2.THRESH_BINARY)[1]
kernel=cv2.getStructuringElement(cv2.MORPH_RECT, (17,17))
kernel2 = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
marker = cv2.erode(thresh,kernel,iterations = 5)
while True:
tmp=marker.copy()
marker=cv2.dilate(marker, kernel2)
marker=cv2.min(thresh, marker)
difference = cv2.subtract(marker, tmp)
if cv2.countNonZero(difference) == 0:
break
cv2.imwrite('out.png', marker)
cv2.imshow('result', marker )
I am trying to find accurate locations for the corners on ink blotches as seen below:
My idea is to fit lines to the edges and then find where they intersect. As of now, I've tried using cv2.approxPolyDP() with various values of epsilon to approximate the edges, however this doesn't look like the way to go. My cv2.approxPolyDP code gives the following result:
Ideally, this is what I want to produce (drawn on paint):
Are there CV functions in place for this sort of problem? I've considered using Gaussian blurring before the threshold step although that method does not seem like it would be very accurate for corner finding. Additionally, I would like this to be robust to rotated images, so filtering for vertical and horizontal lines won't necessarily work without other considerations.
Code:*
import numpy as np
from PIL import ImageGrab
import cv2
def process_image4(original_image): # Douglas-peucker approximation
# Convert to black and white threshold map
gray = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY)
gray = cv2.GaussianBlur(gray, (5, 5), 0)
(thresh, bw) = cv2.threshold(gray, 128, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
# Convert bw image back to colored so that red, green and blue contour lines are visible, draw contours
modified_image = cv2.cvtColor(bw, cv2.COLOR_GRAY2BGR)
contours, hierarchy = cv2.findContours(bw, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(modified_image, contours, -1, (255, 0, 0), 3)
# Contour approximation
try: # Just to be sure it doesn't crash while testing!
for cnt in contours:
epsilon = 0.005 * cv2.arcLength(cnt, True)
approx = cv2.approxPolyDP(cnt, epsilon, True)
# cv2.drawContours(modified_image, [approx], -1, (0, 0, 255), 3)
except:
pass
return modified_image
def screen_record():
while(True):
screen = np.array(ImageGrab.grab(bbox=(100, 240, 750, 600)))
image = process_image4(screen)
cv2.imshow('window', image)
if cv2.waitKey(25) & 0xFF == ord('q'):
cv2.destroyAllWindows()
break
screen_record()
A note about my code: I'm using screen capture so that I can process these images live. I have a digital microscope that can display live feed on a screen, so the constant screen recording will allow me to sample from the video feed and locate the corners live on the other half of my screen.
Here's a potential solution using thresholding + morphological operations:
Obtain binary image. We load the image, blur with cv2.bilateralFilter(), grayscale, then Otsu's threshold
Morphological operations. We perform a series of morphological open and close to smooth the image and remove noise
Find distorted approximated mask. We find the bounding rectangle coordinates of the object with cv2.arcLength() and cv2.approxPolyDP() then draw this onto a mask
Find corners. We use the Shi-Tomasi Corner Detector already implemented as cv2.goodFeaturesToTrack() for corner detection. Take a look at this for an explanation of each parameter
Here's a visualization of each step:
Binary image -> Morphological operations -> Approximated mask -> Detected corners
Here are the corner coordinates:
(103, 550)
(1241, 536)
Here's the result for the other images
(558, 949)
(558, 347)
Finally for the rotated image
(201, 99)
(619, 168)
Code
import cv2
import numpy as np
# Load image, bilaterial blur, and Otsu's threshold
image = cv2.imread('1.png')
mask = np.zeros(image.shape, dtype=np.uint8)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.bilateralFilter(gray,9,75,75)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Perform morpholgical operations
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (10,10))
opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel, iterations=1)
close = cv2.morphologyEx(opening, cv2.MORPH_CLOSE, kernel, iterations=1)
# Find distorted rectangle contour and draw onto a mask
cnts = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
rect = cv2.minAreaRect(cnts[0])
box = cv2.boxPoints(rect)
box = np.int0(box)
cv2.drawContours(image,[box],0,(36,255,12),4)
cv2.fillPoly(mask, [box], (255,255,255))
# Find corners
mask = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
corners = cv2.goodFeaturesToTrack(mask,4,.8,100)
offset = 25
for corner in corners:
x,y = corner.ravel()
cv2.circle(image,(x,y),5,(36,255,12),-1)
x, y = int(x), int(y)
cv2.rectangle(image, (x - offset, y - offset), (x + offset, y + offset), (36,255,12), 3)
print("({}, {})".format(x,y))
cv2.imshow('image', image)
cv2.imshow('thresh', thresh)
cv2.imshow('close', close)
cv2.imshow('mask', mask)
cv2.waitKey()
Note: The idea for the distorted bounding box came from a previous answer in How to find accurate corner positions of a distorted rectangle from blurry image
After seeing the description of the corners, here is what I would recommend:
by any method, find the gross location of the corners (for instance by looking for the extreme values of (±X+±Y, ±X+±Y) or (±X, ±Y)).
consider a strip than joins two corners, with a certain width. Extract the pixels in that strip, on a portion close to the corner, rotate to make it horizontal and average the values along horizontals.
you will obtain a gray profile that tells you the accurate position of the edge, at the mean between the background and foreground intensities.
repeat on all four edges and at both ends. This will give you four accurate corners, by intersection.
I'm trying to make an OpenCV detect a bed in the image. I am running the usual Grayscale, Blur, Canny, and I've tried Convex Hull. However, since there's quite a number of "noise" which gives extra contours and messes up the object detection. Because of this, I am unable to detect the bed properly.
Here is the input image as well as the Canny Edge Detection result:
As you can see, it's almost there. I have the outline of the bed already, albeit, that the upper right corner has a gap - which is preventing me from detecting a closed rectangle.
Here's the code I'm running:
import cv2
import numpy as np
def contoursConvexHull(contours):
print("contours length = ", len(contours))
print("contours length of first item = ", len(contours[1]))
pts = []
for i in range(0, len(contours)):
for j in range(0, len(contours[i])):
pts.append(contours[i][j])
pts = np.array(pts)
result = cv2.convexHull(pts)
print(len(result))
return result
def auto_canny(image, sigma = 0.35):
# compute the mediam of the single channel pixel intensities
v = np.median(image)
# apply automatic Canny edge detection using the computed median
lower = int(max(0, (1.0 - sigma) * v))
upper = int(min(255, (1.0 + sigma) *v))
edged = cv2.Canny(image, lower, upper)
# return edged image
return edged
# Get our image in color mode (1)
src = cv2.imread("bed_cv.jpg", 1)
# Convert the color from BGR to Gray
srcGray = cv2.cvtColor(src, cv2.COLOR_BGR2GRAY)
# Use Gaussian Blur
srcBlur = cv2.GaussianBlur(srcGray, (3, 3), 0)
# ret is the returned value, otsu is an image
##ret, otsu = cv2.threshold(srcBlur, 0, 255,
## cv2.THRESH_BINARY+cv2.THRESH_OTSU)
# Use canny
##srcCanny = cv2.Canny(srcBlur, ret, ret*2, 3)
srcCanny1 = auto_canny(srcBlur, 0.70)
# im is the output image
# contours is the contour list
# I forgot what hierarchy was
im, contours, hierarchy = cv2.findContours(srcCanny1,
cv2.RETR_TREE,
cv2.CHAIN_APPROX_SIMPLE)
##cv2.drawContours(src, contours, -1, (0, 255, 0), 3)
ConvexHullPoints = contoursConvexHull(contours)
##cv2.polylines(src, [ConvexHullPoints], True, (0, 0, 255), 3)
cv2.imshow("Source", src)
cv2.imshow("Canny1", srcCanny1)
cv2.waitKey(0)
Since the contour of the bed isn't closed, I can't fit a rectangle nor detect the contour with the largest area.
The solution I can think of is to extrapolate the largest possible rectangle using the contour points in the hopes of bridging that small gap, but I'm not too sure how to proceed since the rectangle is incomplete.
Since you haven't provided any other examples, I provide an algorithm working with this case. But bare in mind that you will have to find ways of adapting it to however the light and background changes on other samples.
Since there is a lot of noise and a relatively high dynamic range, I suggest not to use Canny and instead use Adaptive Thresholding and Find Contours on that (it doesn't need edges as an input), that helps with choosing different threshold values for different parts of the image.
My result:
Code:
import cv2
import numpy as np
def clahe(img, clip_limit=2.0, grid_size=(8,8)):
clahe = cv2.createCLAHE(clipLimit=clip_limit, tileGridSize=grid_size)
return clahe.apply(img)
src = cv2.imread("bed.png")
# HSV thresholding to get rid of as much background as possible
hsv = cv2.cvtColor(src.copy(), cv2.COLOR_BGR2HSV)
lower_blue = np.array([0, 0, 120])
upper_blue = np.array([180, 38, 255])
mask = cv2.inRange(hsv, lower_blue, upper_blue)
result = cv2.bitwise_and(src, src, mask=mask)
b, g, r = cv2.split(result)
g = clahe(g, 5, (3, 3))
# Adaptive Thresholding to isolate the bed
img_blur = cv2.blur(g, (9, 9))
img_th = cv2.adaptiveThreshold(img_blur, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
cv2.THRESH_BINARY, 51, 2)
im, contours, hierarchy = cv2.findContours(img_th,
cv2.RETR_CCOMP,
cv2.CHAIN_APPROX_SIMPLE)
# Filter the rectangle by choosing only the big ones
# and choose the brightest rectangle as the bed
max_brightness = 0
canvas = src.copy()
for cnt in contours:
rect = cv2.boundingRect(cnt)
x, y, w, h = rect
if w*h > 40000:
mask = np.zeros(src.shape, np.uint8)
mask[y:y+h, x:x+w] = src[y:y+h, x:x+w]
brightness = np.sum(mask)
if brightness > max_brightness:
brightest_rectangle = rect
max_brightness = brightness
cv2.imshow("mask", mask)
cv2.waitKey(0)
x, y, w, h = brightest_rectangle
cv2.rectangle(canvas, (x, y), (x+w, y+h), (0, 255, 0), 1)
cv2.imshow("canvas", canvas)
cv2.imwrite("result.jpg", canvas)
cv2.waitKey(0)