We Keep Coding

How to separate each rectangle tiles and crop it from below image of video conferencing?

The image has 4 streams, how do we crop each stream, Disclaimer: The number of streams could be 4, 8, 16, 32
I have tried to develop an algorithm with the below steps.
Get horizontal and vertical lines
Draw on the line.
Find rectangle tiles using the find contours method from OpenCV
Crop each stream and save it into a different file.
But this approach does not hold true for a random number of streams.
Let me know your inputs on this.
import cv2
# Load image, convert to grayscale, Otsu's threshold
image = cv2.imread(r'C:\oldLaptopData\PF_2021\iotG_work\test_black\MSBlack.jpg')
#image = cv2.imread(r"C:\oldLaptopData\PF_2021\iotG_data\video_samples\split_frames\out1.jpg")
result = image.copy()
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Detect horizontal lines
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (200, 1))
detect_horizontal = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
cnts = cv2.findContours(detect_horizontal, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
print(len(cnts))
for c in cnts:
cv2.drawContours(result, [c], -1, (36,255,12), 10)
# Detect vertical lines
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1, 200))
detect_vertical = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, vertical_kernel, iterations=2)
cnts = cv2.findContours(detect_vertical, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(result, [c], -1, (36,255,12), 10)
cv2.imwrite('result.jpg', result)

How to find the contour of a completed form scanned image?

I would like to detect the contour of the completed form in this scan.
Ideally I would want to find the corners of the table painted with red.
My final goal is to detect that the whole document was scanned and that the four corners are within the boundaries of the scan.
I used OpenCV from python - but it was not able to find the contour of the big container.
Any ideas?

With the observation that the form can be identified using the table grid, here's a simple approach:
Obtain binary image. Load the image, grayscale, Gaussian blur, then Otsu's threshold to get a binary image
Find horizontal sections. We create a horizontal shaped kernel and find horizontal table lines and draw onto a mask
Find vertical sections. We create a vertical shaped kernel and find vertical table lines and draw onto a mask
Fill text document body and morph open. We perform morph operations to close the table then find contours and fill the mask to obtain a contour of the shape. This step fulfills your needs since you can just find contours on the mask but we can go further and extract only the desired sections.
Perform four-point perspective transform. We find contours, sort for the largest contour, sort using contour approximation then perform a four-point perspective transform to obtain a birds eye view of the image.
Here's the results:
Input image
Detected contour to extract highlighted in green
Output after 4-point perspective transform
Code
import cv2
import numpy as np
from imutils.perspective import four_point_transform
# Load image, create mask, grayscale, and Otsu's threshold
image = cv2.imread('1.jpg')
mask = np.zeros(image.shape, dtype=np.uint8)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (3,3), 0)
thresh = cv2.adaptiveThreshold(blur,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV,11,3)
# Find horizontal sections and draw on mask
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (80,1))
detect_horizontal = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
cnts = cv2.findContours(detect_horizontal, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(mask, [c], -1, (255,255,255), -1)
# Find vertical sections and draw on mask
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,50))
detect_vertical = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, vertical_kernel, iterations=2)
cnts = cv2.findContours(detect_vertical, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(mask, [c], -1, (255,255,255), -1)
# Fill text document body
mask = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
close_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (9,9))
close = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, close_kernel, iterations=3)
cnts = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(mask, [c], -1, 255, -1)
# Perform morph operations to remove noise
# Find contours and sort for largest contour
opening = cv2.morphologyEx(mask, cv2.MORPH_OPEN, close_kernel, iterations=5)
cnts = cv2.findContours(opening, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
cnts = sorted(cnts, key=cv2.contourArea, reverse=True)
displayCnt = None
for c in cnts:
# Perform contour approximation
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.02 * peri, True)
if len(approx) == 4:
displayCnt = approx
break
# Obtain birds' eye view of image
warped = four_point_transform(image, displayCnt.reshape(4, 2))
cv2.imwrite('mask.png', mask)
cv2.imwrite('thresh.png', thresh)
cv2.imwrite('warped.png', warped)
cv2.imwrite('opening.png', opening)

What about using the Hough transform with a narrow direction range, to find the verticals and horizontals ? If you are lucky, those that you need will be the longest, and after selecting them you can reconstruct the rectangle.

Extract handwritten characters from a boxed form field image

I am trying to extract handwritten characters from field boxes
My desired output would be the character segments with the boxes removed. So far, I've tried defining contours and filtering by area but that hasn't yielded any good results.
# Reading image and binarization
im = cv2.imread('test.png')
char_gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
char_bw = cv2.adaptiveThreshold(char_gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 75, 10)
# Applying erosion and dilation
kernel = np.ones((5,5), np.uint8)
img_erosion = cv2.erode(char_bw, kernel, iterations=1)
img_dilation = cv2.dilate(img_erosion, kernel, iterations=1)
# Find Canny edges
edged = cv2.Canny(img_dilation, 100, 200)
# Finding Contours
edged_copy = edged.copy()
im2, cnts, hierarchy = cv2.findContours(edged_copy, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
print("Number of Contours found = " + str(len(cnts)))
# Draw all contours
cv2.drawContours(im, cnts, -1, (0, 255, 0), 3)
# Filter using area and save
for no, c in enumerate(cnts):
area = cv2.contourArea(c)
if area > 100:
contour = c
(x, y, w, h) = cv2.boundingRect(contour)
img = im[y:y+h, x:x+w]
cv2.imwrite(f'./cnts/cnt-{no}.png', img_dilation)

Here's a simple approach:
Obtain binary image. We load the image, enlarge using imutils.resize(), convert to grayscale, and perform Otsu's thresholding to obtain a binary image
Remove horizontal lines. We create a horizontal kernel then perform morphological opening and remove the horizontal lines using cv2.drawContours
Remove vertical lines. We create a vertical kernel then perform morphological opening and remove the vertical lines using cv2.drawContours
Here's a visualization of each step:
Binary image
Detected lines/boxes to remove highlighted in green
Result
Code
import cv2
import numpy as np
import imutils
# Load image, enlarge, convert to grayscale, Otsu's threshold
image = cv2.imread('1.png')
image = imutils.resize(image, width=500)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Remove horizontal
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (25,1))
detect_horizontal = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
cnts = cv2.findContours(detect_horizontal, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(image, [c], -1, (255,255,255), 5)
# Remove vertical
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,25))
detect_vertical = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, vertical_kernel, iterations=2)
cnts = cv2.findContours(detect_vertical, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(image, [c], -1, (255,255,255), 5)
cv2.imshow('thresh', thresh)
cv2.imshow('image', image)
cv2.waitKey()

Obtain only external contours in image

I have this code, that draws contours in my image, but I need only the external contours:
import cv2
import numpy as np
camino= "C:/Users/Usuario/Documents/Deteccion de Objetos/123.jpg"
img = cv2.imread("C:/Users/Usuario/Documents/Deteccion de Objetos/123.jpg")
grises= cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
bordes= cv2.Canny(grises, 100, 250)
ctns = cv2.findContours(bordes, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
ctns = ctns[0] if len(ctns)==2 else ctns[1]
for c in ctns:
cv2.drawContours(img,[c], -1,(0,0,255),2)
print ('Numero de contornos es ', len(ctns))
texto= 'Contornos encontrados ' + str(len(ctns))
cv2.putText(img, texto, (10, 20), cv2.FONT_HERSHEY_SIMPLEX, 0.7,
(255, 0, 0), 1)
cv2.imshow('Bordes', bordes)
cv2.imshow('Imagen', img)
cv2.waitKey(0)
cv2.destroyAllWindows().
This is my original image:
This is the obtained image with the contours:
In this case I just only need to detect 10 contours 1 for each entity, but it detects 450 contours.

Here's an approach using thresholding + morphological operations + contour filtering
First we convert to grayscale then Otsu's threshold for a binary image (left) then remove dotted lines using contour area filtering (right)
From here we perform morph close to remove the text then invert the image (left). We find contours and fill all contours smaller than a threshold to black (right)
Next we invert again and perform morph open with a large rectangle kernel to remove the small edges and spikes
Finally we find contours to get our result
import cv2
image = cv2.imread('1.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (5,5), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Remove dotted lines
cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
area = cv2.contourArea(c)
if area < 5000:
cv2.drawContours(thresh, [c], -1, (0,0,0), -1)
# Fill contours
close_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5,5))
close = 255 - cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, close_kernel, iterations=6)
cnts = cv2.findContours(close, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
area = cv2.contourArea(c)
if area < 15000:
cv2.drawContours(close, [c], -1, (0,0,0), -1)
# Smooth contours
close = 255 - close
open_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (20,20))
opening = cv2.morphologyEx(close, cv2.MORPH_OPEN, open_kernel, iterations=3)
# Find contours and draw result
cnts = cv2.findContours(opening, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(image, [c], -1, (36,255,12), 3)
cv2.imshow('thresh', thresh)
cv2.imshow('opening', opening)
cv2.imshow('image', image)
cv2.waitKey()

You can try flood fill combined with some morph operators.

Remove small vertical lines in between the character from an image

I want to remove all horizontal and vertical lines. I am able to remove horizontal lines, but while removing small vertical lines the original text is also getting impacted. This is the code I'm using:
image = cv2.imread('opt/doc/uploads/img1.png')
result = image.copy()
blur = image.copy()
gray = cv2.cvtColor(blur,cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0,255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
rows,cols = thresh.shape
horizontalsize = int(cols // 30)
verticalsize = int(rows // 30)
# Remove horizontal lines
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (horizontalsize,1))
remove_horizontal = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel)
cnts = cv2.findContours(remove_horizontal, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(result, [c], -1, (255,255,255), 3)
# Remove vertical lines
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,20))
remove_vertical = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, vertical_kernel)
cnts = cv2.findContours(remove_vertical, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(result, [c], -1, (255,255,255),3)
cv2.imwrite('result.png', result)
PFB 2 the input image:
PFB output Image of above 2 images respectively:

Instead of trying to detect horizontal/vertical lines, another approach is to filter using contour area to "ignore" the lines and just take the desired text characters. One limitation is it will not detect text that is connected to the horizontal/vertical lines
import cv2
import numpy as np
image = cv2.imread('1.png')
mask = np.ones(image.shape, dtype=np.uint8) * 255
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
area = cv2.contourArea(c)
if area < 1000:
x,y,w,h = cv2.boundingRect(c)
mask[y:y+h, x:x+w] = image[y:y+h, x:x+w]
cv2.imshow('thresh', thresh)
cv2.imshow('mask', mask)
cv2.waitKey()