We Keep Coding

Using OpenCV to find a rotated rectangle of a certain aspect ratio?

I am not having difficulty transforming a found box, it is the fact that I am not able to detect the box in the first place when it is at an angle.
Here is a sample image I want the largest ~1230:123 rectangle in the image the problem is the rectangle can be rotated.
Here is a picture of a rotated barcode that I am unable to detect:
The function I have been using to process uses contour area just looks for the largest rectangle.
What methods should I use to look for a rotated rectangle so that even when rotated I can detect it?
#PYTHON 3.6 Snippet for Image Processing
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# compute the Scharr gradient magnitude representation of the images
# in both the x and y direction using OpenCV 2.4
ddepth = cv2.cv.CV_32F if imutils.is_cv2() else cv2.CV_32F
gradX = cv2.Sobel(gray, ddepth=ddepth, dx=1, dy=0, ksize=-1)
gradY = cv2.Sobel(gray, ddepth=ddepth, dx=0, dy=1, ksize=-1)
# subtract the y-gradient from the x-gradient
gradient = cv2.subtract(gradX, gradY)
gradient = cv2.convertScaleAbs(gradient)
# blur and threshold the image
blurred = cv2.blur(gradient, (8, 8))
(_, thresh) = cv2.threshold(blurred, 225, 255, cv2.THRESH_BINARY)
# construct a closing kernel and apply it to the thresholded image
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (21, 7))
closed = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)
# perform a series of erosions and dilations
closed = cv2.erode(closed, None, iterations = 4)
closed = cv2.dilate(closed, None, iterations = 4)
# find the contours in the thresholded image, then sort the contours
# by their area, keeping only the largest one
cnts = cv2.findContours(closed.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
c = sorted(cnts, key = cv2.contourArea, reverse = True)[0]
# compute the rotated bounding box of the largest contour
rect = cv2.minAreaRect(c)

You don't need all the preprocessing (like Sobel, erode, dilate) for finding before executing findContours.
findContours works better when contours are full (filled with white color) instead of having just the edges.
I suppose you can keep the code from cv2.findContours to the end, and get the result you are looking for.
You may use the following stages:
Apply binary threshold using Otsu's thresholding (just in case image is not a binary image).
Execute cv2.findContours, and Find the contour with the maximum area.
Use cv2.minAreaRect for finding the minimum area bounding rectangle.
Here is a code sample:
import numpy as np
import cv2
img = cv2.imread('img.png', cv2.IMREAD_GRAYSCALE) # Read input image as gray-scale
ret, img = cv2.threshold(img, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU) # Apply threshold using Otsu's thresholding (just in case image is not a binary image).
# Find contours in img.
cnts = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[-2] # [-2] indexing takes return value before last (due to OpenCV compatibility issues).
# Find the contour with the maximum area.
c = max(cnts, key=cv2.contourArea)
# Find the minimum area bounding rectangle
# https://stackoverflow.com/questions/18207181/opencv-python-draw-minarearect-rotatedrect-not-implemented
rect = cv2.minAreaRect(c)
box = cv2.boxPoints(rect)
box = np.int0(box)
# Convert image to BGR (just for drawing a green rectangle on it).
bgr_img = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
cv2.drawContours(bgr_img, [box], 0, (0, 255, 0), 2)
# Show images for debugging
cv2.imshow('bgr_img', bgr_img)
cv2.waitKey()
cv2.destroyAllWindows()
Result:
Note: The largest contour seems to be a parallelogram and not a perfect rectangle.

How to enclose the irregular figure contour and fill it with with 5px dots in opencv using python?

This is my
I want to get this
but the problem is I am not able to enclose the contour and how should I add these dots?
Does Open cv have any such function to handle this?
So basically,
The first problem is how to enclose this image
Second, how to add Dots.
Thank you

Here is one way to do that in Python/OpenCV. However, I cannot close your dotted outline without connecting separate regions. But it will give you some idea how to proceed with most of what you want to do.
If you manually add a few more dots to your input image where there are large gaps, then the morphology kernel can be made smaller such that it can connected the regions without merging separate parts that should remain isolated.
Read the input
Convert to grayscale
Threshold to binary
Apply morphology close to try to close the dotted outline. Unfortunately it connected separate regions.
Get the external contours
Draw white filled contours on a black background as a mask
Draw a single black circle on a white background
Tile out the circle image to the size of the input
Mask the tiled circle image with the filled contour image
Save results
Input:
import cv2
import numpy as np
import math
# read input image
img = cv2.imread('island.png')
hh, ww = img.shape[:2]
# convert img to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# threshold
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY)[1]
# use morphology to close figure
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (35,35))
morph = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel, )
# find contours and bounding boxes
mask = np.zeros_like(thresh)
contours = cv2.findContours(morph, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for cntr in contours:
cv2.drawContours(mask, [cntr], 0, 255, -1)
# create a single tile as black circle on white background
circle = np.full((11,11), 255, dtype=np.uint8)
circle = cv2.circle(circle, (7,7), 3, 0, -1)
# tile out the tile pattern to the size of the input
numht = math.ceil(hh / 11)
numwd = math.ceil(ww / 11)
tiled_circle = np.tile(circle, (numht,numwd))
tiled_circle = tiled_circle[0:hh, 0:ww]
# composite tiled_circle with mask
result = cv2.bitwise_and(tiled_circle, tiled_circle, mask=mask)
# save result
cv2.imwrite("island_morph.jpg", morph)
cv2.imwrite("island_mask.jpg", mask)
cv2.imwrite("tiled_circle.jpg", tiled_circle)
cv2.imwrite("island_result.jpg", result)
# show images
cv2.imshow("morph", morph)
cv2.imshow("mask", mask)
cv2.imshow("tiled_circle", tiled_circle)
cv2.imshow("result", result)
cv2.waitKey(0)
Morphology connected image:
Contour Mask image:
Tiled circles:
Result:

How to approximate jagged edges as lines using Python OpenCV?

I am trying to find accurate locations for the corners on ink blotches as seen below:
My idea is to fit lines to the edges and then find where they intersect. As of now, I've tried using cv2.approxPolyDP() with various values of epsilon to approximate the edges, however this doesn't look like the way to go. My cv2.approxPolyDP code gives the following result:
Ideally, this is what I want to produce (drawn on paint):
Are there CV functions in place for this sort of problem? I've considered using Gaussian blurring before the threshold step although that method does not seem like it would be very accurate for corner finding. Additionally, I would like this to be robust to rotated images, so filtering for vertical and horizontal lines won't necessarily work without other considerations.
Code:*
import numpy as np
from PIL import ImageGrab
import cv2
def process_image4(original_image): # Douglas-peucker approximation
# Convert to black and white threshold map
gray = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY)
gray = cv2.GaussianBlur(gray, (5, 5), 0)
(thresh, bw) = cv2.threshold(gray, 128, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
# Convert bw image back to colored so that red, green and blue contour lines are visible, draw contours
modified_image = cv2.cvtColor(bw, cv2.COLOR_GRAY2BGR)
contours, hierarchy = cv2.findContours(bw, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(modified_image, contours, -1, (255, 0, 0), 3)
# Contour approximation
try: # Just to be sure it doesn't crash while testing!
for cnt in contours:
epsilon = 0.005 * cv2.arcLength(cnt, True)
approx = cv2.approxPolyDP(cnt, epsilon, True)
# cv2.drawContours(modified_image, [approx], -1, (0, 0, 255), 3)
except:
pass
return modified_image
def screen_record():
while(True):
screen = np.array(ImageGrab.grab(bbox=(100, 240, 750, 600)))
image = process_image4(screen)
cv2.imshow('window', image)
if cv2.waitKey(25) & 0xFF == ord('q'):
cv2.destroyAllWindows()
break
screen_record()
A note about my code: I'm using screen capture so that I can process these images live. I have a digital microscope that can display live feed on a screen, so the constant screen recording will allow me to sample from the video feed and locate the corners live on the other half of my screen.

Here's a potential solution using thresholding + morphological operations:
Obtain binary image. We load the image, blur with cv2.bilateralFilter(), grayscale, then Otsu's threshold
Morphological operations. We perform a series of morphological open and close to smooth the image and remove noise
Find distorted approximated mask. We find the bounding rectangle coordinates of the object with cv2.arcLength() and cv2.approxPolyDP() then draw this onto a mask
Find corners. We use the Shi-Tomasi Corner Detector already implemented as cv2.goodFeaturesToTrack() for corner detection. Take a look at this for an explanation of each parameter
Here's a visualization of each step:
Binary image -> Morphological operations -> Approximated mask -> Detected corners
Here are the corner coordinates:
(103, 550)
(1241, 536)
Here's the result for the other images
(558, 949)
(558, 347)
Finally for the rotated image
(201, 99)
(619, 168)
Code
import cv2
import numpy as np
# Load image, bilaterial blur, and Otsu's threshold
image = cv2.imread('1.png')
mask = np.zeros(image.shape, dtype=np.uint8)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.bilateralFilter(gray,9,75,75)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Perform morpholgical operations
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (10,10))
opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel, iterations=1)
close = cv2.morphologyEx(opening, cv2.MORPH_CLOSE, kernel, iterations=1)
# Find distorted rectangle contour and draw onto a mask
cnts = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
rect = cv2.minAreaRect(cnts[0])
box = cv2.boxPoints(rect)
box = np.int0(box)
cv2.drawContours(image,[box],0,(36,255,12),4)
cv2.fillPoly(mask, [box], (255,255,255))
# Find corners
mask = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
corners = cv2.goodFeaturesToTrack(mask,4,.8,100)
offset = 25
for corner in corners:
x,y = corner.ravel()
cv2.circle(image,(x,y),5,(36,255,12),-1)
x, y = int(x), int(y)
cv2.rectangle(image, (x - offset, y - offset), (x + offset, y + offset), (36,255,12), 3)
print("({}, {})".format(x,y))
cv2.imshow('image', image)
cv2.imshow('thresh', thresh)
cv2.imshow('close', close)
cv2.imshow('mask', mask)
cv2.waitKey()
Note: The idea for the distorted bounding box came from a previous answer in How to find accurate corner positions of a distorted rectangle from blurry image

After seeing the description of the corners, here is what I would recommend:
by any method, find the gross location of the corners (for instance by looking for the extreme values of (±X+±Y, ±X+±Y) or (±X, ±Y)).
consider a strip than joins two corners, with a certain width. Extract the pixels in that strip, on a portion close to the corner, rotate to make it horizontal and average the values along horizontals.
you will obtain a gray profile that tells you the accurate position of the edge, at the mean between the background and foreground intensities.
repeat on all four edges and at both ends. This will give you four accurate corners, by intersection.

Extract building edges from map image using Python

I got a map image here.
I need to extract the edges of buildings for further process, the result would be like step 2 for the post here.
Since I am not familiar with this field, can this be done by libraries such as OpenCV?

Seems you want to select individual buildings, so I used color separation. The walls are darker, which makes for good separation in the HSV colorspace. Note that the final result can be improved by zooming in more and/or by using an imagetype with less compression, such as PNG.
Select walls
First I determined good values for separation. For that I used this script. I found that the best result would be to separate the yellow and the gray separately and then combine the resulting masks. Not all walls closed perfectly, so I improved the result by closing the mask a bit. The result is a mask that displays all walls:
Left to right: Yellow mask, Gray mask, Combined and solidified mask
Find buildings
Next I used findCountours to separate out buildings. Since the wall contours will probably not be very useful (as walls are interconnected), I used the hierarchy to find the 'lowest' contours (that have no other contours inside of them). These are the buildings.
Result of findContours: the outline of all contours in green, the outline of individual buildings in red
Note that buildings on the edge are not detected. This is because using this technique they are not a separate contour, but part of the exterior of the image. This can be solve this by drawing a rectangle in gray on the border of the image. You may not want this in your final application, but I included it in case you do.
Code:
import cv2
import numpy as np
#load image and convert to hsv
img = cv2.imread("fLzI9.jpg")
# draw gray box around image to detect edge buildings
h,w = img.shape[:2]
cv2.rectangle(img,(0,0),(w-1,h-1), (50,50,50),1)
# convert image to HSV
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# define color ranges
low_yellow = (0,28,0)
high_yellow = (27,255,255)
low_gray = (0,0,0)
high_gray = (179,255,233)
# create masks
yellow_mask = cv2.inRange(hsv, low_yellow, high_yellow )
gray_mask = cv2.inRange(hsv, low_gray, high_gray)
# combine masks
combined_mask = cv2.bitwise_or(yellow_mask, gray_mask)
kernel = np.ones((3,3), dtype=np.uint8)
combined_mask = cv2.morphologyEx(combined_mask, cv2.MORPH_DILATE,kernel)
# findcontours
contours, hier = cv2.findContours(combined_mask,cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
# find and draw buildings
for x in range(len(contours)):
# if a contour has not contours inside of it, draw the shape filled
c = hier[0][x][2]
if c == -1:
cv2.drawContours(img,[contours[x]],0,(0,0,255),-1)
# draw the outline of all contours
for cnt in contours:
cv2.drawContours(img,[cnt],0,(0,255,0),2)
# display result
cv2.imshow("Result", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
With buildings drawn solid red and all contours as green overlay

Here's a simple approach
Convert image to grayscale and Gaussian blur to smooth edges
Threshold image
Perform Canny edge detection
Find contours and draw contours
Threshold image using cv2.threshold()
Perform Canny edge detection with cv2.Canny()
Find contours using cv2.findContours() and cv2.drawContours()
import cv2
image = cv2.imread('1.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (3, 3), 0)
thresh = cv2.threshold(blurred, 240 ,255, cv2.THRESH_BINARY_INV)[1]
canny = cv2.Canny(thresh, 50, 255, 1)
cnts = cv2.findContours(canny, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(image,[c], 0, (36,255,12), 2)
cv2.imshow('thresh', thresh)
cv2.imshow('canny', canny)
cv2.imshow('image', image)
cv2.imwrite('thresh.png', thresh)
cv2.imwrite('canny.png', canny)
cv2.imwrite('image.png', image)
cv2.waitKey(0)

Find contour of rectangle in object

I would like to find the contour of the rectangular photograph inside of the object. I've tried using the corner detection feature of OpenCV, but to no avail. I also tried to find all the contours using findContours, and filter out the contours with more (or less) than 4 edges, but this also didn't lead anywhere.
I have a sample scan here.

I have a solution for you, but it involves a lot of steps. Also, it may not generalize that well. It does work pretty good for your image though.
First a grayscale and threshold is made and findContours is used to create a mask of the paper area. That mask is inverted and combined with the original image, which makes the black edges white. A new grayscale and threshold is made on the resulting image, which is then inverted so findContours can find the dark pixels of the photo. A rotated box around the largest contours is selected, which is the area you seek.
I added a little extra, which you may not need, but could be convenient: perspectivewarp is applied to the box, so the area you want is made into a straight rectangle.
There is quite a lot happening, so I advise you to take some time a look at the intermediate steps, to understand what happens.
Result:
Code:
import numpy as np
import cv2
# load image
image = cv2.imread('photo.jpg')
# resize to easily view on screen, remove for final processing
image = cv2.resize(image,None,fx=0.2, fy=0.2, interpolation = cv2.INTER_CUBIC)
### remove outer black edge
# create grayscale
gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# perform threshold
retr , mask = cv2.threshold(gray_image, 190, 255, cv2.THRESH_BINARY)
# remove noise
kernel = np.ones((5,5),np.uint8)
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
# create emtpy mask
mask_2 = np.zeros(image.shape[:3], dtype=image.dtype)
# find contours
ret, contours, hier = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# draw the found shapes (white, filled in ) on the empty mask
for cnt in contours:
cv2.drawContours(mask_2, [cnt], 0, (255,255,255), -1)
# invert mask and combine with original image - this makes the black outer edge white
mask_inv_2 = cv2.bitwise_not(mask_2)
tmp = cv2.bitwise_or(image, mask_inv_2)
### Select photo - not inner edge
# create grayscale
gray_image2 = cv2.cvtColor(tmp, cv2.COLOR_BGR2GRAY)
# perform threshold
retr, mask3 = cv2.threshold(gray_image2, 190, 255, cv2.THRESH_BINARY)
# remove noise
maskX = cv2.morphologyEx(mask3, cv2.MORPH_CLOSE, kernel)
# invert mask, so photo area can be found with findcontours
maskX = cv2.bitwise_not(maskX)
# findcontours
ret, contours2, hier = cv2.findContours(maskX, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# select the largest contour
largest_area = 0
for cnt in contours2:
if cv2.contourArea(cnt) > largest_area:
cont = cnt
largest_area = cv2.contourArea(cnt)
# find the rectangle (and the cornerpoints of that rectangle) that surrounds the contours / photo
rect = cv2.minAreaRect(cont)
box = cv2.boxPoints(rect)
box = np.int0(box)
print(rect)
#### Warp image to square
# assign cornerpoints of the region of interest
pts1 = np.float32([box[1],box[0],box[2],box[3]])
# provide new coordinates of cornerpoints
pts2 = np.float32([[0,0],[0,450],[630,0],[630,450]])
# determine and apply transformationmatrix
M = cv2.getPerspectiveTransform(pts1,pts2)
result = cv2.warpPerspective(image,M,(630,450))
#draw rectangle on original image
cv2.drawContours(image, [box], 0, (255,0,0), 2)
#show image
cv2.imshow("Result", result)
cv2.imshow("Image", image)
cv2.waitKey(0)
cv2.destroyAllWindows()