We Keep Coding

Rounded rectangle perspective transform

I have an image where I'm trying to export and do transform as rectangle (no matter in base photo). I have done with finding largest contour in image using mask (trained in pytorch). In variable pr_mask I have stored mask from network. After that from that mask I can get extract contours of my image. So from this image:
I can extract contours like this:
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray,(5,5),5)
canny = cv2.Canny(blur, 30, 80, 3)
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (9,9))
flag, thresh = cv2.threshold(canny, 80, 255, cv2.THRESH_BINARY)
kernel = np.ones((10,10),np.uint8)
dilation = cv2.dilate(canny,kernel,iterations = 1)
ret,thresh = cv2.threshold(dilation, 200, 255, 0)
contours, hierarchy = cv2.findContours(thresh, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
After that I fetch largest contour and then trying to extract contours:
approx = np.ones((10,10),np.uint8)
d = 0
while (len(approx)>4):
d=d+1
approx = cv2.approxPolyDP(largestcontour,d, True)
cv2.drawContours(imagebase, [largestcontour], -1, (0,255,0), 3)
hull = []
hull.append(cv2.convexHull(largestcontour, False))
cv2.drawContours(imagebase, hull, -1, (0,0,255), 3)
After that I get tthis result:
Is it possible to extract square (rectangle) which will transform my image to exact square without rounded corners and to fit whole image?
I'm trying to find a way how to solve this "perspective" issue and align element to specific position. My image is taken by cameras, so it can be in different positions, where boundingRectangle will not work ... any suggestions?

How to find the junction points or segments in a skeletonized image Python OpenCV?

I am trying to convert the result of a skeletonization into a set of line segments, where the vertices correspond to the junction points of the skeleton. The shape is not a closed polygon and it may be somewhat noisy (the segments are not as straight as they should be).
Here is an example input image:
And here are the points I want to retrieve:
I have tried using the harris corner detector, but it has trouble in some areas even after trying to tweak the algorithm's parameters (such as the angled section on the bottom of the image). Here are the results:
Do you know of any method capable of doing this? I am using python with mostly OpenCV and Numpy but I am not bound to any library. Thanks in advance.
Edit: I've gotten some good responses regarding the junction points, I am really grateful. I would also appreciate any solutions regarding extracting line segments from the junction points. I think #nathancy's answer could be used to extract line segments by subtracting the masks with the intersection mask, but I am not sure.

My approach is based on my previous answer here. It involves convolving the image with a special kernel. This convolution identifies the end-points of the lines, as well as the intersections. This will result in a points mask containing the pixel that matches the points you are looking for. After that, apply a little bit of morphology to join possible duplicated points. The method is sensible to the corners produced by the skeleton.
This is the code:
import cv2
import numpy as np
# image path
path = "D://opencvImages//"
fileName = "Repn3.png"
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
inputImageCopy = inputImage.copy()
# Convert to grayscale:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Compute the skeleton:
skeleton = cv2.ximgproc.thinning(grayscaleImage, None, 1)
# Threshold the image so that white pixels get a value of 10 and
# black pixels a value of 0:
_, binaryImage = cv2.threshold(skeleton, 128, 10, cv2.THRESH_BINARY)
# Set the convolution kernel:
h = np.array([[1, 1, 1],
[1, 10, 1],
[1, 1, 1]])
# Convolve the image with the kernel:
imgFiltered = cv2.filter2D(binaryImage, -1, h)
So far I convolved the skeleton image with my special kernel. You can inspect the image produced and search for the numerical values at the corners and intersections.
This is the output so far:
Next, identify a corner or an intersection. This bit is tricky, because the threshold value depends directly on the skeleton image, which sometimes doesn't produce good (close to straight) corners:
# Create list of thresholds:
thresh = [130, 110, 40]
# Prepare the final mask of points:
(height, width) = binaryImage.shape
pointsMask = np.zeros((height, width, 1), np.uint8)
# Perform convolution and create points mask:
for t in range(len(thresh)):
# Get current threshold:
currentThresh = thresh[t]
# Locate the threshold in the filtered image:
tempMat = np.where(imgFiltered == currentThresh, 255, 0)
# Convert and shape the image to a uint8 height x width x channels
# numpy array:
tempMat = tempMat.astype(np.uint8)
tempMat = tempMat.reshape(height,width,1)
# Accumulate mask:
pointsMask = cv2.bitwise_or(pointsMask, tempMat)
This is the binary mask:
Let's dilate to join close points:
# Set kernel (structuring element) size:
kernelSize = 3
# Set operation iterations:
opIterations = 4
# Get the structuring element:
morphKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelSize, kernelSize))
# Perform Dilate:
pointsMask = cv2.morphologyEx(pointsMask, cv2.MORPH_DILATE, morphKernel, None, None, opIterations, cv2.BORDER_REFLECT101)
This is the output:
Now simple extract external contours. Get their bounding boxes and calculate their centroid:
# Look for the outer contours (no children):
contours, _ = cv2.findContours(pointsMask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# Store the points here:
pointsList = []
# Loop through the contours:
for i, c in enumerate(contours):
# Get the contours bounding rectangle:
boundRect = cv2.boundingRect(c)
# Get the centroid of the rectangle:
cx = int(boundRect[0] + 0.5 * boundRect[2])
cy = int(boundRect[1] + 0.5 * boundRect[3])
# Store centroid into list:
pointsList.append( (cx,cy) )
# Set centroid circle and text:
color = (0, 0, 255)
cv2.circle(inputImageCopy, (cx, cy), 3, color, -1)
font = cv2.FONT_HERSHEY_COMPLEX
cv2.putText(inputImageCopy, str(i), (cx, cy), font, 0.5, (0, 255, 0), 1)
# Show image:
cv2.imshow("Circles", inputImageCopy)
cv2.waitKey(0)
This is the result. Some corners are missed, you might one to improve the solution before computing the skeleton.

Here's a simple approach, the idea is:
Obtain binary image. Load image, convert to grayscale, Gaussian blur, then Otsu's threshold.
Obtain horizontal and vertical line masks. Create horizontal and vertical structuring elements with cv2.getStructuringElement then perform cv2.morphologyEx to isolate the lines.
Find joints. We cv2.bitwise_and the two masks together to get the joints. The idea is that the intersection points on the two masks are the joints.
Find centroid on joint mask. We find contours then calculate the centroid.
Find leftover endpoints. Endpoints do not correspond to an intersection so to find those, we can use the Shi-Tomasi Corner Detector
Horizontal and vertical line masks
Results (joints in green and endpoints in blue)
Code
import cv2
import numpy as np
# Load image, grayscale, Gaussian blur, Otsus threshold
image = cv2.imread('1.png')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (3,3), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
# Find horizonal lines
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5,1))
horizontal = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=1)
# Find vertical lines
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,5))
vertical = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, vertical_kernel, iterations=1)
# Find joint intersections then the centroid of each joint
joints = cv2.bitwise_and(horizontal, vertical)
cnts = cv2.findContours(joints, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
# Find centroid and draw center point
x,y,w,h = cv2.boundingRect(c)
centroid, coord, area = cv2.minAreaRect(c)
cx, cy = int(centroid[0]), int(centroid[1])
cv2.circle(image, (cx, cy), 5, (36,255,12), -1)
# Find endpoints
corners = cv2.goodFeaturesToTrack(thresh, 5, 0.5, 10)
corners = np.int0(corners)
for corner in corners:
x, y = corner.ravel()
cv2.circle(image, (x, y), 5, (255,100,0), -1)
cv2.imshow('thresh', thresh)
cv2.imshow('joints', joints)
cv2.imshow('horizontal', horizontal)
cv2.imshow('vertical', vertical)
cv2.imshow('image', image)
cv2.waitKey()

OpenCV Python remove small contours, save child contours

I'm attempting to remove all but the largest contours while retaining the child contours within the largest contour. Effectively I want to go from here: Input Image to: Output Image I don't know how to go about this though (I'm not a programmer by any stretch and had help putting together the below). The code I do have doesn't retain the child contours. Output of Below.
import cv2 as cv
import numpy as np
img = cv.imread('small_contour.jpg')
image_contours = np.zeros((img.shape[1], img.shape[0], 1), np.uint8)
image_binary = np.zeros((img.shape[1], img.shape[0], 1), np.uint8)
for channel in range(img.shape[2]):
ret, image_thresh = cv.threshold(img[:, :, channel], 127, 255, cv.THRESH_BINARY)
contours = cv.findContours(image_thresh, 1, 1)[0]
cv.drawContours(image_contours, contours, -1, (255,255,255), 3)
contours = cv.findContours(image_contours, cv.RETR_LIST,
cv.CHAIN_APPROX_SIMPLE)[0]
cv.drawContours(image_binary, [max(contours, key = cv.contourArea)],
-1, (255, 255, 255), -1)
cv.imwrite('fill_contour.jpg', image_binary)
cv.imshow('Small Contour', image_binary)
cv.waitKey(0)

What your code does in this line cv.drawContours(image_binary, [max(contours, key = cv.contourArea)], -1, (255, 255, 255), -1) is to set to color (255, 255, 255) the pixels of the image_binary corresponding to the boundary and inner pixels of the bigger contour found by cv.findContours.
Creating the image_binary you are also swapping width with height and this is an error.
You don't need to keep the children: you should use cv.RETR_TREE for that, then find the index of the father and look for all contours with that father doing it recursively for the nested contours (children of children ...). See https://docs.opencv.org/4.1.0/d9/d8b/tutorial_py_contours_hierarchy.html But this is not useful for your goal.
You can use the result you currently have as a mask for the original image, but the code need some fixes, starting from the linked image named VS0L9.jpg.
Note that I'm using cv2, just change to cv.
Load the image
im = cv2.imread('VS0L9.jpg', cv2.IMREAD_GRAYSCALE)
Find the contours (I used RETR_TREE)
contours, hierarchy = cv2.findContours(im, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
Find the bigger contour:
bigger = max(contours, key=lambda item: cv2.contourArea(item))
Initialize the mask (what you tried with image_binary):
the_mask = np.zeros_like(im)
Draw the fill the bigger contour on the mask image, the region of interest is set to (255, 255, 255):
cv2.drawContours(the_mask, [bigger], -1, (255, 255, 255), cv2.FILLED)
At this point the mask looks like the output of your code, but with proper width and height.
Use the mask with the original image to show only the area of interest:
res = cv2.bitwise_and(im, im, mask = the_mask)
Or, alternatively:
res = im.copy()
res[the_mask == 0] = 0
Now res is the desired result.

You can use morphological reconstruction, with erode image as marker.
import cv2
img = cv2.imread('VS0L9.jpg', cv2.IMREAD_GRAYSCALE)
thresh = cv2.threshold(img, 40, 255, cv2.THRESH_BINARY)[1]
kernel=cv2.getStructuringElement(cv2.MORPH_RECT, (17,17))
kernel2 = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
marker = cv2.erode(thresh,kernel,iterations = 5)
while True:
tmp=marker.copy()
marker=cv2.dilate(marker, kernel2)
marker=cv2.min(thresh, marker)
difference = cv2.subtract(marker, tmp)
if cv2.countNonZero(difference) == 0:
break
cv2.imwrite('out.png', marker)
cv2.imshow('result', marker )

Find contour of rectangle in object

I would like to find the contour of the rectangular photograph inside of the object. I've tried using the corner detection feature of OpenCV, but to no avail. I also tried to find all the contours using findContours, and filter out the contours with more (or less) than 4 edges, but this also didn't lead anywhere.
I have a sample scan here.

I have a solution for you, but it involves a lot of steps. Also, it may not generalize that well. It does work pretty good for your image though.
First a grayscale and threshold is made and findContours is used to create a mask of the paper area. That mask is inverted and combined with the original image, which makes the black edges white. A new grayscale and threshold is made on the resulting image, which is then inverted so findContours can find the dark pixels of the photo. A rotated box around the largest contours is selected, which is the area you seek.
I added a little extra, which you may not need, but could be convenient: perspectivewarp is applied to the box, so the area you want is made into a straight rectangle.
There is quite a lot happening, so I advise you to take some time a look at the intermediate steps, to understand what happens.
Result:
Code:
import numpy as np
import cv2
# load image
image = cv2.imread('photo.jpg')
# resize to easily view on screen, remove for final processing
image = cv2.resize(image,None,fx=0.2, fy=0.2, interpolation = cv2.INTER_CUBIC)
### remove outer black edge
# create grayscale
gray_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# perform threshold
retr , mask = cv2.threshold(gray_image, 190, 255, cv2.THRESH_BINARY)
# remove noise
kernel = np.ones((5,5),np.uint8)
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
# create emtpy mask
mask_2 = np.zeros(image.shape[:3], dtype=image.dtype)
# find contours
ret, contours, hier = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# draw the found shapes (white, filled in ) on the empty mask
for cnt in contours:
cv2.drawContours(mask_2, [cnt], 0, (255,255,255), -1)
# invert mask and combine with original image - this makes the black outer edge white
mask_inv_2 = cv2.bitwise_not(mask_2)
tmp = cv2.bitwise_or(image, mask_inv_2)
### Select photo - not inner edge
# create grayscale
gray_image2 = cv2.cvtColor(tmp, cv2.COLOR_BGR2GRAY)
# perform threshold
retr, mask3 = cv2.threshold(gray_image2, 190, 255, cv2.THRESH_BINARY)
# remove noise
maskX = cv2.morphologyEx(mask3, cv2.MORPH_CLOSE, kernel)
# invert mask, so photo area can be found with findcontours
maskX = cv2.bitwise_not(maskX)
# findcontours
ret, contours2, hier = cv2.findContours(maskX, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# select the largest contour
largest_area = 0
for cnt in contours2:
if cv2.contourArea(cnt) > largest_area:
cont = cnt
largest_area = cv2.contourArea(cnt)
# find the rectangle (and the cornerpoints of that rectangle) that surrounds the contours / photo
rect = cv2.minAreaRect(cont)
box = cv2.boxPoints(rect)
box = np.int0(box)
print(rect)
#### Warp image to square
# assign cornerpoints of the region of interest
pts1 = np.float32([box[1],box[0],box[2],box[3]])
# provide new coordinates of cornerpoints
pts2 = np.float32([[0,0],[0,450],[630,0],[630,450]])
# determine and apply transformationmatrix
M = cv2.getPerspectiveTransform(pts1,pts2)
result = cv2.warpPerspective(image,M,(630,450))
#draw rectangle on original image
cv2.drawContours(image, [box], 0, (255,0,0), 2)
#show image
cv2.imshow("Result", result)
cv2.imshow("Image", image)
cv2.waitKey(0)
cv2.destroyAllWindows()

How to find area of a curve in python where coordinates are not known?

So I am currently working on a project where I need to find the area in terms of cm of a particular curve.The problem is the curve has more than one colors each representing a different values
Something Like This
Can someone help me do it? There are more than one such curves in the image. How to simultaneously calculate all of them in Python.

You can use the following code to print area in terms of pixels. To get the area in cm^2, you need to know the relationship between pixels and actual length.
The following code prints the area of largest blob in the image.
To get the area of all the blobs in the image, just replace [c] with contours
import cv2
import numpy as np
img = cv2.imread("image.png", 0)
blank = np.zeros_like(img)
ret, thresh = cv2.threshold(img, 0 ,255, cv2.THRESH_BINARY)
im2, contours, hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
if( len(contours) != 0 ):
c = max(contours, key = cv2.contourArea)
cv2.drawContours(blank, [c], -1, 255, -1)
print cv2.countNonZero(blank)
cv2.imshow("img", blank)
cv2.waitKey(0)
cv2.destroyAllWindows()
Edit:
import cv2
import numpy as np
img = cv2.imread("images.png", 0)
blank = np.zeros_like(img)
ret, thresh = cv2.threshold(img, 0 ,255, cv2.THRESH_BINARY)
im2, contours, hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
for i in range(len(contours)):
cv2.drawContours(blank, contours[i], -1, 255, -1)
print "area of contour " + str(i)+" = " + str(cv2.contourArea(contours[i]))
cv2.imshow("img", blank)
cv2.waitKey(0)
cv2.destroyAllWindows()

1, separate out the different colored blobs. They look like they are generated by some other mapping software so presumably the colors are fixed and known. Make a new image for each color
2, For an image that contains only blobs of a fixed color and a black background you can make contours of the outlines (see findContours). Opencv will give you a separate contour for each blob.
3, Calculate the area of each contour -there is an opencv function to do this.