I am using OpenCV to put bounding boxes on handwritten math equation inputs. Currently, my code sometimes places multiple smaller bounding boxes around different parts of a singular image instead of creating one large box around the image. I'm not sure why this is happening. My current code to filter the image and find the contours to draw the bounding box is as follows:
img = cv2.imread(imgpath)
morph = img.copy()
morph = cv2.fastNlMeansDenoising(img)
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1, 1))
morph = cv2.morphologyEx(morph, cv2.MORPH_CLOSE, kernel)
morph = cv2.morphologyEx(morph, cv2.MORPH_OPEN, kernel)
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 15))
# take morphological gradient
gradient_image = cv2.morphologyEx(morph, cv2.MORPH_GRADIENT, kernel)
gray = cv2.cvtColor(gradient_image, cv2.COLOR_BGR2GRAY)
img_grey = cv2.morphologyEx(gray, cv2.MORPH_CLOSE, kernel)
blur = cv2.medianBlur(img_grey,3)
ret, thing = cv2.threshold(blur, 0.0, 255.0, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
img_dilation = cv2.dilate(thing, kernel, iterations=3)
conturs_lst = cv2.findContours(img_dilation, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[-2]
An example of the actual result is as follows:
OG Image:
You have the right idea but I think you're overusing cv2.morphologyEx to continuously erode and dilate the image. You mention your problem:
Currently, my code sometimes places multiple smaller bounding boxes around different parts of a singular image instead of creating one large box around the image.
When you use cv2.findContours, its working correctly but since your contours are actually blobs instead of one interconnected singular image, it creates multiple bounding boxes. To remedy this problem, you can dilate the image to connect the blobs together.
I've rewrote your code without the extra cv2.morphologyEx repetitions. The main idea is as follows:
Convert the image into grayscale
Blur image
Threshold image to separate background from desired object
Dilate image to connect blobs to form a singular image
Find contours and filter contours using threshold min/max area
Threshold image to isolate desired sections. Note some of the contours have broken connections. To fix this, we dilate the image to connect the blobs.
Dilate image to form singular objects. Now note we have the unwanted horizontal section at the bottom, we can find contours and then filter using area to remove that section.
Results
import numpy as np
import cv2
original_image = cv2.imread("1.jpg")
image = original_image.copy()
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (3, 3), 0)
thresh = cv2.threshold(blurred, 160, 255, cv2.THRESH_BINARY_INV)[1]
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
dilate = cv2.dilate(thresh, kernel , iterations=4)
cv2.imshow("thresh", thresh)
cv2.imshow("dilate", dilate)
# Find contours in the image
cnts = cv2.findContours(dilate.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
contours = []
threshold_min_area = 400
threshold_max_area = 3000
for c in cnts:
x,y,w,h = cv2.boundingRect(c)
area = cv2.contourArea(c)
if area > threshold_min_area and area < threshold_max_area:
# cv2.drawContours(original_image,[c], 0, (0,255,0), 3)
cv2.rectangle(original_image, (x,y), (x+w, y+h), (0,255,0),1)
contours.append(c)
cv2.imshow("detected", original_image)
print('contours detected: {}'.format(len(contours)))
cv2.waitKey(0)
Related
I want to crop the image to only extract the text sections. There are thousands of them with different sizes so I can't hardcode coordinates. I'm trying to remove the unwanted lines on the left and on the bottom. How can I do this?
Original
Expected
Determine the least spanning bounding box by finding all the non-zero points in the image. Finally, crop your image using this bounding box. Finding the contours is time-consuming and unnecessary here, especially because your text is axis-aligned. You may accomplish your goal by combining cv2.findNonZero and cv2.boundingRect.
Hope this will work ! :
import numpy as np
import cv2
img = cv2.imread(r"W430Q.png")
# Read in the image and convert to grayscale
img = img[:-20, :-20] # Perform pre-cropping
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gray = 255*(gray < 50).astype(np.uint8) # To invert the text to white
gray = cv2.morphologyEx(gray, cv2.MORPH_OPEN, np.ones(
(2, 2), dtype=np.uint8)) # Perform noise filtering
coords = cv2.findNonZero(gray) # Find all non-zero points (text)
x, y, w, h = cv2.boundingRect(coords) # Find minimum spanning bounding box
# Crop the image - note we do this on the original image
rect = img[y:y+h, x:x+w]
cv2.imshow("Cropped", rect) # Show it
cv2.waitKey(0)
cv2.destroyAllWindows()
in above code from forth line of code is where I set the threshold below 50 to make the dark text white. However, because this outputs a binary image, I convert to uint8, then scale by 255. The text is effectively inverted.
Then, using cv2.findNonZero, we discover all of the non-zero locations for this image.We then passed this to cv2.boundingRect, which returns the top-left corner of the bounding box, as well as its width and height. Finally, we can utilise this to crop the image. This is done on the original image, not the inverted version.
Here's a simple approach:
Obtain binary image. Load the image, grayscale, Gaussian blur, then Otsu's threshold to obtain a binary black/white image.
Remove horizontal lines. Since we're trying to only extract text, we remove horizontal lines to aid us in our next step so incorrect contours will not merge together.
Merge text into a single contour. The idea is that characters which are adjacent to each other are part of the wall of text. So we can dilate individual contours together to obtain a single contour to extract.
Find contours and extract ROI. We find contours, sort contours by area, then extract the largest contour ROI using Numpy slicing.
Here's the visualization of each step:
Binary image -> Removed horizontal lines in green
1
2
Dilate to combine into a single contour -> Detected ROI to extract in green
3
4
Result
Code
import cv2
import numpy as np
# Load image, grayscale, Gaussian blur, Otsu's threshold
image = cv2.imread('1.png')
original = image.copy()
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (3, 3), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Remove horizontal lines
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (25,1))
detected_lines = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=1)
cnts = cv2.findContours(detected_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(thresh, [c], -1, 0, -1)
# Dilate to merge into a single contour
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (2,30))
dilate = cv2.dilate(thresh, vertical_kernel, iterations=3)
# Find contours, sort for largest contour and extract ROI
cnts, _ = cv2.findContours(dilate, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[-2:]
cnts = sorted(cnts, key=cv2.contourArea, reverse=True)[:-1]
for c in cnts:
x,y,w,h = cv2.boundingRect(c)
cv2.rectangle(image, (x, y), (x + w, y + h), (36,255,12), 4)
ROI = original[y:y+h, x:x+w]
break
cv2.imshow('image', image)
cv2.imshow('dilate', dilate)
cv2.imshow('thresh', thresh)
cv2.imshow('ROI', ROI)
cv2.waitKey()
I am new to OpenCV.
I have a "simple" image of a stamp that I have already processed a bit, as you can see in the code below.
Now I have the problem of cropping the image to get just the stamp.
The dots and the stripes on the edges interfere with my current code recognizing the stamp.
The images can be different so it is not an option to fix the location of the image.
Code:
img = cv2.imread('./images/image.JPG')
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_blur = cv2.GaussianBlur(img_gray, (3,3), 0)
edges = cv2.Canny(image=img_blur, threshold1=100, threshold2=200)
Real Images
Here's a simple method:
Obtain binary image. We load the image, convert to grayscale, Gaussian blur, then Otsu's threshold to obtain a binary image.
Remove small artifacts and noise. Create a rectangular structuring element and morph open to remove small bits of noise. Then morph close to combine individual contours into a single contour with the assumption that a stamp is a single contour.
Detect and extract stamp ROI. Find contours, filter using contour area and shape approximation. The idea is that if a contour has four vertices then it's a square shape. We can extract the stamp ROI using Numpy slicing and save the stamp
Extracted ROI results
Here's the results with the other two input images from the comments. The assumption is that for each image, there's only one stamp, or one group of adjacent stamps. For these cases, we sort by contour area and assume the largest contour is the stamp.
Code
import cv2
# Load image, grayscale, Gaussian blur, Otsu's threshold
image = cv2.imread("1.jpg")
original = image.copy()
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (5,5), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
# Morph operations to remove small artifacts and noise
open_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, open_kernel, iterations=1)
close_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (7,7))
close = cv2.morphologyEx(opening, cv2.MORPH_CLOSE, close_kernel, iterations=2)
# Find contours, filter using contour area, and shape approximation
cnts = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
cnts = sorted(cnts, key=cv2.contourArea, reverse=True)
for c in cnts:
peri = cv2.arcLength(c, True)
area = cv2.contourArea(c)
approx = cv2.approxPolyDP(c, 0.05 * peri, True)
# Assumption is if the contour has 4 vertices then its a square shape
# 2nd assumption is that there's only one stamp, or one group of stamps
if len(approx) == 4 and area > 100:
x,y,w,h = cv2.boundingRect(approx)
ROI = original[y:y+h, x:x+w]
cv2.imshow("ROI", ROI)
cv2.imwrite("ROI.png", ROI)
break
cv2.waitKey()
I'm trying to detect and draw a rectangular contour on every painting on for example this image:
I followed some guides and did the following:
Grayscale conversion
Applied median blur
Sharpen image
Applied adaptive Threshold
Applied Morphological Gradient
Find contours
Draw contours
And got the following result:
I know it's messy but is there a way to somehow detect and draw a contour around the paintings better?
Here is the code I used:
path = '<PATH TO THE PICTURE>'
#reading in and showing original image
image = cv2.imread(path)
image = cv2.resize(image,(880,600)) # resize was nessecary because of the large images
cv2.imshow("original", image)
cv2.waitKey(0)
cv2.destroyAllWindows()
# grayscale conversion
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
cv2.imshow("painting_gray", gray)
cv2.waitKey(0)
cv2.destroyAllWindows()
# we need to find a way to detect the edges better so we implement a couple of things
# A little help was found on stackoverflow: https://stackoverflow.com/questions/55169645/square-detection-in-image
median = cv2.medianBlur(gray,5)
cv2.imshow("painting_median_blur", median) #we use median blur to smooth the image
cv2.waitKey(0)
cv2.destroyAllWindows()
# now we sharpen the image with help of following URL: https://www.analyticsvidhya.com/blog/2021/08/sharpening-an-image-using-opencv-library-in-python/
kernel = np.array([[0, -1, 0],
[-1, 5,-1],
[0, -1, 0]])
image_sharp = cv2.filter2D(src=median, ddepth=-1, kernel=kernel)
cv2.imshow('painting_sharpend', image_sharp)
cv2.waitKey(0)
cv2.destroyAllWindows()
# now we apply adapptive thresholding
# thresholding: https://opencv24-python-tutorials.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_thresholding/py_thresholding.html#adaptive-thresholding
thresh = cv2.adaptiveThreshold(src=image_sharp,maxValue=255,adaptiveMethod=cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
thresholdType=cv2.THRESH_BINARY,blockSize=61,C=20)
cv2.imshow('thresholded image', thresh)
cv2.waitKey(0)
cv2.destroyAllWindows()
# lets apply a morphological transformation
kernel = np.ones((7,7),np.uint8)
gradient = cv2.morphologyEx(thresh, cv2.MORPH_GRADIENT, kernel)
cv2.imshow('dilated image', gradient)
cv2.waitKey(0)
cv2.destroyAllWindows()
# # lets now find the contours of the image
# # find contours: https://docs.opencv.org/4.x/dd/d49/tutorial_py_contour_features.html
contours, hierarchy = cv2.findContours(gradient, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
print("contours: ", len(contours))
print("hierachy: ", len(hierarchy))
print(hierarchy)
cv2.drawContours(image, contours, -1, (0,255,0), 3)
cv2.imshow("contour image", image)
cv2.waitKey(0)
cv2.destroyAllWindows()
Tips, help or code is appreciated!
Here's a simple approach:
Obtain binary image. We load the image, grayscale, Gaussian blur, then Otsu's threshold to obtain a binary image.
Two pass dilation to merge contours. At this point, we have a binary image but individual separated contours. Since we can assume that a painting is a single large square contour, we can merge small individual adjacent contours together to form a single contour. To do this, we create a vertical and horizontal kernel using cv2.getStructuringElement then dilate to merge them together. Depending on the image, you may need to adjust the kernel sizes or number of dilation iterations.
Detect paintings. Now we find contours and filter using contour area using a minimum threshold area to filter out small contours. Finally we obtain the bounding rectangle coordinates and draw the rectangle with cv2.rectangle.
Code
import cv2
# Load image, grayscale, Gaussian blur, Otsu's threshold
image = cv2.imread('1.jpeg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (13,13), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
# Two pass dilate with horizontal and vertical kernel
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (9,5))
dilate = cv2.dilate(thresh, horizontal_kernel, iterations=2)
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5,9))
dilate = cv2.dilate(dilate, vertical_kernel, iterations=2)
# Find contours, filter using contour threshold area, and draw rectangle
cnts = cv2.findContours(dilate, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
area = cv2.contourArea(c)
if area > 20000:
x,y,w,h = cv2.boundingRect(c)
cv2.rectangle(image, (x, y), (x + w, y + h), (36, 255, 12), 3)
cv2.imshow('thresh', thresh)
cv2.imshow('dilate', dilate)
cv2.imshow('image', image)
cv2.waitKey()
So here is the actual size of the portrait frame.
So here is small code.
#!/usr/bin/python 37
#OpenCV 4.3.0, Raspberry Pi 3/B/4B-w/4/8GB RAM, Buster,v10.
#Date: 3rd, June, 2020
import cv2
# Load the image
img = cv2.imread('portrait.jpeg')
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
edged = cv2.Canny(img, 120,890)
# Apply adaptive threshold
thresh = cv2.adaptiveThreshold(edged, 255, 1, 1, 11, 2)
thresh_color = cv2.cvtColor(thresh, cv2.COLOR_GRAY2BGR)
# apply some dilation and erosion to join the gaps - change iteration to detect more or less area's
thresh = cv2.dilate(thresh,None,iterations = 50)
thresh = cv2.erode(thresh,None,iterations = 50)
# Find the contours
contours,hierarchy = cv2.findContours(thresh,
cv2.RETR_TREE,
cv2.CHAIN_APPROX_SIMPLE)
# For each contour, find the bounding rectangle and draw it
for cnt in contours:
area = cv2.contourArea(cnt)
if area > 20000:
x,y,w,h = cv2.boundingRect(cnt)
cv2.rectangle(img,
(x,y),(x+w,y+h),
(0,255,0),
2)
cv2.imshow('img',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Here is output:
Problem Summary: I have got many complex histopathology images with different dimensions. Complex means a single image having multiple images in it as shown in below input image examples. I need to separate out or crop each single image only and not the text/label/caption of it from that input complex image and further save each of them individually. For the bounding boxes I have gone through the white boundaries (separation) along the single images.
Complex Input Image Example 1:
Complex Input Image Example 2:
Complex Input Image Example 3:
Code I have tried:
import cv2
import numpy as np
# reading the input image
img = cv2.imread('cmp.jpg')
cv2.imshow("histology image", img)
# defining border color
lower = (0, 80, 110)
upper = (0, 120, 150)
# applying thresholding on border color
mask = cv2.inRange(img, lower, upper)
cv2.imshow("masked", mask)
# Using dilate threshold
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (15, 15))
mask = cv2.morphologyEx(mask, cv2.MORPH_DILATE, kernel)
# coloring border to white for other images
img[mask==255] = (255,255,255)
cv2.imshow("white_border", img)
# converting image to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# applying otsu threshold
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_OTSU )[1]
cv2.imshow("thresholded", thresh)
# applying 'Open' morphological operation
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (17,17))
morph = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel)
morph = 255 - morph
cv2.imshow("morphed", morph)
# finding contours and bounding boxes
bboxes = []
bboxes_img = img.copy()
contours = cv2.findContours(morph, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for cntr in contours:
x,y,w,h = cv2.boundingRect(cntr)
cv2.rectangle(bboxes_img, (x, y), (x+w, y+h), (0, 0, 255), 1)
bboxes.append((x,y,w,h))
cv2.imshow("boundingboxes", bboxes_img)
cv2.waitKey(0)
cv2.destroyAllWindows()
I am not getting exact bounding boxes for each of single images present in the input complex image and further I need to save each cropped image individually. Any kind of help will be much appreciated.
I want to count number of trees on this picture from above.
I know how to count elements, but till now I used images with white background, so counting is much easier. But on image like this i do not know what to do:
I converted the image to gray, and then done the threshold *(threshold value is done by hand, is there a way to find it automaticly?), my next idea is to find the 'centers' of black dots, or to 'group' them.
I also tried to change brightness and contrast but it didnt work.
What should I do?
This is the code that I wrote:
import cv2
import numpy as np
# Read image
img = cv2.imread('slika.jpg')
# Convert image to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# Show grayscale image
cv2.imshow('gray image', gray)
cv2.waitKey(0)
#BIG PROBLEM: IM FINDING VALUE OF `40` IN THE LINE BELOW MANUALLY
# Inverse binary threshold image with threshold at 40,
_, threshold_one = cv2.threshold(gray, 40 , 255, cv2.THRESH_BINARY_INV)
# Show thresholded image
cv2.imshow('threshold image', threshold_one)
cv2.waitKey(0)
# Find contours
contours, h = cv2.findContours(threshold_one, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
print('Number of trees found:', len(contours)) #GIVES WRONG RESULT
# Iterate all found contours
for cnt in contours:
# Draw contour in original/final image
cv2.drawContours(img, [cnt], 0, (0, 0, 255), 1)
# Show final image
cv2.imshow('result image', img)
cv2.waitKey(0)
This is the image with treshold, I have tried to blur it (in order to connect black dots), but the final output is the same:
This is the result image:
Here's a rough method to estimate the number of trees. The idea to to model each tree as a blob then use contour filtering with a minimum threshold area to ignore the noise. To determine automatic threshold levels, you can use Otsu's threshold by appending cv2.THRESH_OTSU or Adaptive threshold with cv2.adaptiveThreshold(). This approach has problems when the trees are very close together since they form a single blob. Possible improvements could be to find the average area of each tree then find the floor with a large blob. You would probably need to train a classifier and use deep/machine learning for better accuracy
Trees: 102
import cv2
image = cv2.imread('1.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (5,5), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3,3))
close = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel, iterations=2)
opening = cv2.morphologyEx(close, cv2.MORPH_OPEN, kernel, iterations=2)
cnts = cv2.findContours(opening, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
trees = 0
for c in cnts:
area = cv2.contourArea(c)
if area > 50:
x,y,w,h = cv2.boundingRect(c)
cv2.drawContours(image, [c], -1, (36,255,12), 2)
trees += 1
print('Trees:', trees)
cv2.imshow('image', image)
cv2.waitKey()