We Keep Coding

How to extract diagram from an image?

I've used Contour based approach but its detecting so many contours. How can I extract my ROI contour?
image = cv2.imread('ULTI.png')
original = image.copy()
cv2.imwrite("bg.png",bg)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (3, 3), 0)
canny = cv2.Canny(blurred, 120, 255, 1)
kernel = np.ones((5,5),np.uint8)
dilate = cv2.dilate(canny, kernel, iterations=1)
# Find contours
cnts = cv2.findContours(dilate, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
# Iterate thorugh contours and filter for ROI
image_number = 0
cnts = max(cnts, key = cv2.contourArea)
print("no ",len(cnts))
for c in cnts:
x,y,w,h = cv2.boundingRect(c)
cv2.rectangle(image, (x, y), (x + w, y + h), (36,255,12), 2)
ROI = original[y:y+h, x:x+w]
#cv2.imwrite("ROI_{}.png".format(image_number), ROI)
image_number += 1

You can get your ROI by specifying that you only want to use the contour that has the greatest area, that is, if your diagram will produce a contour with an area greater then the rest of the components in your image.
Here is an example:
import cv2
def preprocess(img):
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_blur = cv2.GaussianBlur(img_gray, (5, 5), 1)
img_canny = cv2.Canny(img_blur, 50, 50)
return img_canny
def get_roi(img, pad=3):
contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
max_area = 0
for cnt in contours:
peri = cv2.arcLength(cnt, True)
approx = cv2.approxPolyDP(cnt, 0.02 * peri, True)
x, y, w, h = cv2.boundingRect(approx)
rect_area = w * h
if rect_area > max_area:
max_area = rect_area
dim = x, y, w, h
if max_area:
x, y, w, h = dim
return x - pad, y - pad, w + pad * 2, h + pad * 2
img = cv2.imread("ULTI.png")
img_processed = preprocess(img)
x, y, w, h = get_roi(img_processed)
cv2.imshow("Image", img[y:y + h, x:x + w])
cv2.waitKey(0)
Output:
Explanation:
Import the necessary module(s):
import cv2
Define a function that will take in an image, and return a processed version of the image that will allow python to properly detect the necessary contours (you can tweak the values to further meet your needs):
def preprocess(img):
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_blur = cv2.GaussianBlur(img_gray, (5, 5), 1)
img_canny = cv2.Canny(img_blur, 50, 50)
return img_canny
Now, let's see how we can define a function that will retrieve the ROI of the image. First, define it so that a processed image array and a pad amount (optional) can be passed in as parameters:
def get_roi(img, pad=3):
Find the contours of the processed image, and define a variable to store the greatest area of the contours:
contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
max_area = 0
Use a for loop to loop through the contours, and find the area of the contour of each iteration of the loop:
for cnt in contours:
peri = cv2.arcLength(cnt, True)
approx = cv2.approxPolyDP(cnt, 0.02 * peri, True)
area = cv2.contourArea(approx)
Use an if statement to check if the area is greater than the defined variable that should store that greatest area. If the are of the contour of that iteration is greater than the variable, update the value of the variable to be equal to the new area. Also, save the contour of that iteration to a variable:
if area > max_area:
max_area = area
max_cnt = approx
After the for loop, if the max_area variable doesn't equal to 0, then a max_cnt has also been defined. Use the cv2.boundingRect to get the x, y, w and h properties:
if max_area:
x, y, w, h = cv2.boundingRect(max_cnt)
return x - pad, y - pad, w + pad * 2, h + pad * 2
Finally, after reading the image into a variable, you can utilize the 2 functions we defined, and display the resulting image:
img = cv2.imread("ULTI.png")
img_processed = preprocess(img)
x, y, w, h = get_roi(img_processed)
cv2.imshow("Image", img[y:y + h, x:x + w])
cv2.waitKey(0)
Note: The code likely will not work for all diagrams. But again, you can tweak the values in the preprocess function to meet your needs.

Remove Contours OpenCV

My Image
I want to get
https://ibb.co/t8hNkM2
I could only get
I was able to find the maximum contour
def img_counter_max(image_file: str):
img = cv2.imread(image_file)
# grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # меняем цветовую модель с BGR на HSV
cv2.waitKey(0)
# binarize
ret, thresh = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY_INV)
cv2.waitKey(0)
# find contours
ctrs, hier = cv2.findContours(thresh.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
# sort contours
sorted_ctrs = sorted(ctrs, key=lambda ctr: cv2.boundingRect(ctr)[0])
# sorted_ctrs sorted(ctrs, key=cv2.contourArea, reverse=True)[0]
contour_sizes = [(cv2.contourArea(contour), contour) for contour in sorted_ctrs]
biggest_contour = max(contour_sizes, key=lambda x: x[0])[1]
x, y, w, h = cv2.boundingRect(biggest_contour)
roi = img[y:y + h, x:x + w]
cv2.imwrite("C:\\Users\\dennn\\PycharmProjects\\untitled2\\imag\\roi1.jpg",
roi)
cv2.rectangle(img, (x, y), (x + w, y + h), (90, 255, 0), 2)
from tensorflow.python import Size
resize_img = cv2.resize(img, (512,512))
# cv2.resize(img, Size(512,512), interpolation=cv2.INTER_AREA)
cv2.namedWindow("Display frame", cv2.WINDOW_AUTOSIZE);
cv2.imshow('Display frame', resize_img)
cv2.waitKey(0)
How do I get the image I need?

I found that sorting by contourArea() gives wrong results. Probably it calculates all points inside contour but not rectangle area which it uses - and this rectangle can be bigger.
I use boundingRect() to get rectangle used by contour and later calculate size using w*h and then it sorts contours in correct way.
I use for-loop to display image with different rectangles and see which contour gives expected region. And this way I see that third contour gives expected region so I can use [2] to get it and save it.
Eventually I would use size to select region which has w*h is in some range
expecte_region_size - range < w*h < expecte_region_size + range
Eventually I would use for-loop which display image with different rectangles to select manually which rectangle to use to save in file.
import cv2
img = cv2.imread('image.jpg')
# grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # меняем цветовую модель с BGR на HSV
# binarize
ret, thresh = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY_INV)
# find contours
ctrs, hier = cv2.findContours(thresh.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
# find rect and area - and create items [contour, rect, area] - but sorting by area gives wrong results
#items = [[ctr, cv2.boundingRect(ctr), cv2.contourArea(ctr)] for ctr in ctrs]
# find rect - and create items [contour, rect]
items = [[ctr, cv2.boundingRect(ctr)] for ctr in ctrs]
# find rect's size and create items [contour, rect, size]
items = [[ctr, rect, rect[2]*rect[3]] for ctr, rect in items]
# sort by size
items = sorted(items, key=lambda x: x[2], reverse=True)
for index, item in enumerate(items[:5]):
contour = item[0]
x, y, w, h = item[1]
size = item[2]
print(index, '->', size, '(', x, y, w, h, ')')
img_copy = img.copy()
cv2.rectangle(img_copy, (x, y), (x + w, y + h), (0, 0, 255), 15)
resize_img = cv2.resize(img_copy, (512,512))
cv2.imshow('frame', resize_img)
cv2.waitKey(0)
cv2.destroyAllWindows()
# --- save image ---
item = items[2]
contour = item[0]
x, y, w, h = item[1]
size = item[2]
img = img[y:y+h, x:x+w]
cv2.imwrite('output.jpg', img)
Preview:
Output:

The code finds characters well,but outputs them out of order
I found a piece of code that should solve this problem, but I can't -
after finding the contours using contours=cv2.findContours(),use -
boundary=[]
for c,cnt in enumerate(contours):
x,y,w,h = cv2.boundingRect(cnt)
boundary.append((x,y,w,h))
count=np.asarray(boundary)
max_width = np.sum(count[::, (0, 2)], axis=1).max()
max_height = np.max(count[::, 3])
nearest = max_height * 1.4
ind_list=np.lexsort((count[:,0],count[:,1]))
c=count[ind_list]
Find symbols
img = "C:\\Users\\dennn\\PycharmProjects\\untitled2\\output.jpg" dir = os.curdir
path = os.path.join(dir,img)
raw_image = cv2.imread(path,0)
cv2.imshow("original",raw_image)
plt.subplot(2,3,1)
plt.title("Original")
plt.imshow(raw_image,'gray')
plt.xticks([]),plt.yticks([]);
sm_image = cv2.blur(raw_image,(8,8))
cv2.imshow("smoothed",sm_image)
plt.subplot(2,3,2)
plt.title("Smoothed")
plt.imshow(sm_image,'gray')
plt.xticks([]),plt.yticks([]);
#cv2.imshow("smoothed",sm_image)
ret,bw_image = cv2.threshold(sm_image,160,255,cv2.THRESH_BINARY_INV)
cv2.imshow("thresholded",bw_image)
plt.subplot(2,3,3)
plt.title("Thresholded")
plt.imshow(bw_image,'gray')
plt.xticks([]),plt.yticks([]);
kernel = np.ones((4,4),np.uint8)
er_image = cv2.erode(bw_image,kernel)
cv2.imshow("eroded",er_image)
plt.subplot(2,3,4)
plt.title("Eroded")
plt.imshow(er_image,'gray')
plt.xticks([]),plt.yticks([]);
kernel = np.ones((2,2),np.uint8)
di_image = cv2.dilate(er_image,kernel)
cv2.imshow("dilated",di_image)
plt.title("Dilated")
plt.subplot(2,3,5)
plt.imshow(di_image,'gray')
plt.xticks([]),plt.yticks([]);
mo_image = di_image.copy()
contour0 =
cv2.findContours(mo_image.copy(),cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
contours = [cv2.approxPolyDP(cnt,3,True) for cnt in contour0[0]]
maxArea = 0
rect = []
for ctr in contours:
maxArea = max(maxArea, cv2.contourArea(ctr))
if img == "C:\\Users\\dennn\\PycharmProjects\\untitled2\\output.jpg":
areaRatio = 0.05
for ctr in contours:
if cv2.contourArea(ctr) > maxArea * areaRatio:
rect.append(cv2.boundingRect(cv2.approxPolyDP(ctr, 1, True)))
symbols = []
for i in rect:
x = i[0]
y = i[1]
w = i[2]
h = i[3]
p1 = (x, y)
p2 = (x + w, y + h)
cv2.rectangle(mo_image, p1, p2, 255, 2)
image = cv2.resize(mo_image[y:y + h, x:x + w], (32, 32))
symbols.append(image.reshape(1024, ).astype("uint8"))
testset_data = np.array(symbols)
cv2.imshow("segmented", mo_image)
plt.subplot(2, 3, 6)
plt.title("Segmented")
plt.imshow(mo_image, 'gray')
plt.xticks([]), plt.yticks([]);
# plt.show()
# garbage collection
cv2.destroyAllWindows()
plt.close()
# show glyphs
for i in range(len(symbols)):
image = np.zeros(shape=(64,64))
image[15:47,15:47] = symbols[i].reshape((32,32))
cv2.imshow("sym",image)
cv2.waitKey(0)
cv2.destroyAllWindows()
plt.close()

How to use Python and openCV to merge the contours of the same object?

I have used the function named cv2.findContours() to draw the outline, however some places could not be identified which made the contours discontinuous. Here is my code and the result.
Thanks a lot if someone could teach me how to merge the contours of the same object?
import numpy as np
import os
import cv2
image = cv2.imread("/home/rafael/Desktop/2.jpg")
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
lower = np.array([50, 10, 10])
upper = np.array([120, 255, 255])
mask = cv2.inRange(hsv, lower, upper)
res = cv2.bitwise_and(image, image, mask = mask)
kernel = np.ones((5, 5), np.uint8)
d_im = cv2.dilate(mask, kernel, iterations = 1)
e_im = cv2.erode(d_im, kernel, iterations = 1)
src, contours, hierarchy = cv2.findContours(e_im, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
temp = []
tmp = []
num = 0
for i in range(len(contours)):
if len(contours[i]) < 35:
temp.append(i)
for i in temp:
del contours[i - num]
num = num + 1
cv2.drawContours(image, contours, -1, (0, 0, 255), 2)
cv2.imwrite('/home/rafael/Desktop/13.jpg', image)
Result
The full picture
The original Picture

I want to remove the boundary using opencv

I have this original image:
then I have applied the following code to
Converted the Original image to HSV image
Then using cv2.findContours() I have made a list containing all the contours.
Then i have removed all the contours of area less than 30.
Then I got the following image:
What I want is to remove the boundary from the resulting image it is of no use (outer boundary of leaf).I only need the inner patches of the leaf.
This is the code i used.
import cv2
import numpy as np
img = cv2.imread('Apple___Blackrot30.JPG')
hsv = cv2.cvtColor(img,cv2.COLOR_BGR2HSV)
lower_gr = np.array([25,0,0])
upper_gr = np.array([90,255,255])
mask = cv2.inRange(hsv,lower_gr,upper_gr)
mask=~mask
res = cv2.bitwise_and(img,img,mask = mask)
blur = cv2.bilateralFilter(res,9,75,75)
im2,cont,_ = cv2.findContours(mask,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
areas = [cv2.contourArea(each_conts) for each_conts in cont]
cont_counter = 0
for each_conts in areas:
if each_conts < 30:
cv2.fillPoly(im2, pts =[cont[cont_counter]], color=(0,0,0))
if each_conts > 1024:
cv2.drawContours(mask, cont[cont_counter], 0, (255,255,255), -1)
cont_counter+=1
cv2.imshow('cn',im2)

You can use the concept of hierarchy of contours to solve this problem. But there is a caveat, all your images must be the same as the one in the question.
I just added some additional stuff to your code.
Code:
img2 = img.copy()
im2, cont, hierarchy = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
l = []
for e, h in enumerate(hierarchy[0]):
#print (e, h[3])
if h[3] == -1:
l.append(e)
for i in l:
if cv2.contourArea(cont[i]) < 1000:
cv2.drawContours(img2, [cont[i]], -1, (0, 255, 255), 2)
cv2.imshow('img2', img2)
Result:
hierarchy returns an array expressing the parent-child relationship of contours. As per the documentation link,
it as an array of four values : [Next, Previous, First_Child, Parent].
In the hierarchy array I scanned the Parent column (4th column) to see whether it has no parent contours (-1) and drew them

I assume you only need the inner spots inside a leaf.
Segment using the otsu algorithm
apply the flood-fill operation to ensure you capture all leaf pixels
Extract only the inner contour
All can simply be done using opencv below are the codes:
import cv2
import numpy as np
def flood_fill_binary(binary):
hh = binary.shape[0]
ww = binary.shape[1]
xx = 10
yy = 10
black = [0,0,0]
binary = cv2.copyMakeBorder(binary,10,10,10,10,cv2.BORDER_CONSTANT,value=black)
im_floodfill = binary.copy()
h, w = binary.shape[:2]
mask = np.zeros((h+2, w+2), np.uint8)
cv2.floodFill(im_floodfill, mask, (0,0), 255)
im_floodfill_inv = cv2.bitwise_not(im_floodfill)
im_out = binary | im_floodfill_inv
crop_og = im_out[yy:yy+hh,xx:xx+ww]
return crop_og
def leaf_spots_detector(image):
image = image.astype('uint8')
hh = image.shape[0]
ww = image.shape[1]
xx = 10
yy = 10
#kernel = np.ones((3,3),np.uint8)
grayed_image = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
_, segmented = cv2.threshold(grayed_image,0, 255,
cv2.THRESH_BINARY+cv2.THRESH_OTSU)
segmented = flood_fill_binary(segmented)
segmented = cv2.copyMakeBorder(segmented,xx,xx,yy,yy,cv2.BORDER_CONSTANT,value=
[0,0,0])
major = cv2.__version__.split('.')[0]
if major == '3':
ret, contours, hierarchy = cv2.findContours(segmented, cv2.RETR_TREE,
cv2.CHAIN_APPROX_SIMPLE)
else:
contours, hierarchy = cv2.findContours(segmented, cv2.RETR_TREE,
cv2.CHAIN_APPROX_SIMPLE)
print(hierarchy.shape)
image_external = np.zeros(segmented.shape, segmented.dtype)
for i in range(1,len(contours)):
#if hierarchy[0][i][3] == -1:
cv2.drawContours(image_external, contours, i,(225,255,255), -1)
image_external = image_external[yy:yy+hh,xx:xx+ww]
#image_external = cv2.dilate(image_external,kernel,iterations = 1)
return image_external
image = cv2.imread('image/path.png')
leaf_spots = leaf_spots_detector(image)
cv2.imshow("detected spots", leaf_spots)
cv2.waitKey(0)
cv2.destroyAllWindow()

How to find the coordinates of the outside corners of these 4 squares? (morphological closing/opening does not conserve squares if image is rotated)

One of the first processing steps in a tool I'm coding is to find the coordinates of the outside corners of 4 big black squares. They will then be used to do a homographic transform, in order to deskew / unrotate the image (a.k.a perspective transform), to finally get a rectangular image. Here is an example of - rotated and noisy - input (download link here):
To keep the big squares only, I'm using morphological transformations like closing/opening:
import cv2, numpy as np
img = cv2.imread('rotatednoisy-cropped.png', cv2.IMREAD_GRAYSCALE)
kernel = np.ones((30, 30), np.uint8)
img = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel)
cv2.imwrite('output.png', img)
Input file (download link):
Output, after morphological transform:
Problem: the output squares are not square anymore, and therefore the coordinates of the top left corner of the square will be not precise at all!
I could reduce the kernel size, but then it would keep more unwanted small elements.
Question: how to get a better detection of the corners of the squares?
Note:
As a morphological closing is just a dilatation + an erosion, I found the culprit:
import cv2, numpy as np
img = cv2.imread('rotatednoisy-cropped.png', cv2.IMREAD_GRAYSCALE)
kernel = np.ones((30, 30), np.uint8)
img = cv2.dilate(img, kernel, iterations = 1)
After this step, it's still ok:
Then
img = cv2.erode(img, kernel, iterations = 1)
gives
and it's not ok anymore!

See this link for detailed explanation on how to de-skew an image.
import cv2
import numpy as np
def corners(box):
cx,cy,w,h,angle = box[0][0],box[0][1],box[1][0],box[1][1],box[2]
CV_PI = 22./7.
_angle = angle*CV_PI/180.;
b = np.cos(_angle)*0.5;
a = np.sin(_angle)*0.5;
pt = []
pt.append((int(cx - a*h - b*w),int(cy + b*h - a*w)));
pt.append((int(cx + a*h - b*w),int(cy - b*h - a*w)));
pt.append((int(2*cx - pt[0][0]),int(2*cy - pt[0][1])));
pt.append((int(2*cx - pt[1][0]),int(2*cy - pt[1][1])));
return pt
if __name__ == '__main__':
image = cv2.imread('image.jpg',cv2.IMREAD_UNCHANGED)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
n = 3
sigma = 0.3 * (n/2 - 1) + 0.8
gray = cv2.GaussianBlur(gray, ksize=(n,n), sigmaX=sigma)
ret,binary = cv2.threshold(gray, 0, 255, cv2.THRESH_OTSU+cv2.THRESH_BINARY)
_,contours,_ = cv2.findContours(binary, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
contours.sort(key=lambda x: len(x), reverse=True)
points = []
for i in range(0,4):
shape = cv2.approxPolyDP(contours[i], 0.05*cv2.arcLength(contours[i],True), True)
if len(shape) == 4:
points.append(shape)
points = np.array(points,dtype=np.int32)
points = np.reshape(points, (-1,2))
box = cv2.minAreaRect(points)
pt = corners(box)
for i in range(0,4):
image = cv2.line(image, (pt[i][0],pt[i][1]), (pt[(i+1)%4][0],pt[(i+1)%4][1]), (0,0,255))
(h,w) = image.shape[:2]
(center) = (w//2,h//2)
angle = box[2]
if angle < -45:
angle = (angle+90)
else:
angle = -angle
M = cv2.getRotationMatrix2D(center, angle, 1.0)
rotated = cv2.warpAffine(image, M, (w,h), flags=cv2.INTER_CUBIC, borderMode=cv2.BORDER_CONSTANT)
cv2.imshow('image', image)
cv2.imshow('rotated', rotated)
cv2.waitKey(0)
cv2.destroyAllWindows()

You could try by searching and filtering out your specific contours (black rectangles) and sorting them with a key. Then select the extreme point for each contour (left, right, top, bottom) and you will get the points. Note that this approach is ok for this picture only and if the picture was roteted in other direction, you would have to change the code accordingly. I am not an expert but I hope this helps a bit.
import numpy as np
import cv2
img = cv2.imread("rotate.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, threshold = cv2.threshold(gray,150,255,cv2.THRESH_BINARY)
im, contours, hierarchy = cv2.findContours(threshold,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
contours.sort(key=lambda c: np.min(c[:,:,1]))
j = 1
if len(contours) > 0:
for i in range(0, len(contours)):
size = cv2.contourArea(contours[i])
if 90 < size < 140:
if j == 1:
c1 = contours[i]
j += 1
elif j == 2:
c2 = contours[i]
j += 1
elif j == 3:
c3 = contours[i]
j += 1
elif j == 4:
c4 = contours[i]
break
Top = tuple(c1[c1[:, :, 1].argmin()][0])
Right = tuple(c2[c2[:, :, 0].argmax()][0])
Left = tuple(c3[c3[:, :, 0].argmin()][0])
Bottom = tuple(c4[c4[:, :, 1].argmax()][0])
cv2.circle(img, Top, 2, (0, 255, 0), -1)
cv2.circle(img, Right, 2, (0, 255, 0), -1)
cv2.circle(img, Left, 2, (0, 255, 0), -1)
cv2.circle(img, Bottom, 2, (0, 255, 0), -1)
cv2.imshow("Image", img)
cv2.waitKey(0)
Result:

You can extract the squares as single blobs after binarization with a suitable threshold, and select the appropriate ones based on size. You can also first denoise with a median filter if you want.
Then a tight rotated bounding rectangle will give you the corners (you can obtain it by running Rotating Calipers on the Convex Hull).