We Keep Coding

How do I use OpenCV to detect exclusively almost straight edges?

I'm trying to detect straight edges in a basketball card and what I have so far does a good job of detecting all edges. I would like for this piece of code however, to detect exclusively straight edges (the outline of the card).
import cv2
import numpy as np
import imutils
img = cv2.imread('edgedetection/cardgiannis.jpeg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
kernel_size = 5
blur_gray = cv2.GaussianBlur(gray,(kernel_size, kernel_size),0)
low_threshold = 50
high_threshold = 150
edges = cv2.Canny(blur_gray, low_threshold, high_threshold)
rho = 1 # distance resolution in pixels of the Hough grid
theta = np.pi / 180 # angular resolution in radians of the Hough grid
threshold = 15 # minimum number of votes (intersections in Hough grid cell)
min_line_length = 50 # minimum number of pixels making up a line
max_line_gap = 20 # maximum gap in pixels between connectable line segments
line_image = np.copy(img) * 0 # creating a blank to draw lines on
# Run Hough on edge detected image
# Output "lines" is an array containing endpoints of detected line segments
lines = cv2.HoughLinesP(edges, rho, theta, threshold, np.array([]),
min_line_length, max_line_gap)
for line in lines:
for x1,y1,x2,y2 in line:
cv2.line(line_image,(x1,y1),(x2,y2),(255,0,0),5)
# Draw the lines on the image
lines_edges = cv2.addWeighted(img, 0.8, line_image, 1, 0)
cv2.imshow('lines', lines_edges)
cv2.waitKey()

Here is what I think.
If you can somehow get the highest & lowest valued coordinates in the pixels that are forming the lines, you can use those pixels to form a rectangle that consist of the straight edges.
Using the pixels to (roughly) form a rectangle can be the solution!
To find the pixels in a line, you can take a look in here, here and here.

you may use cv.approxPolyDP to extract dominant points from edges and stick to the edges that have 2 dominant points.
ps: you apply approxPolyDP on edges after collecting them by cv.findcontours

Find contours of rectangular objects touching each other

Here is grayscale uint8 image I'm working with: source grayscale image.
This image is a result of stitching 6 different colorized depth images into one. There are 3 rectangular objects in the image, and my goal is to find edges of these objects. Obviously, I have no problem to find external edges of objects. But, separating objects from each other is a big pain.
Desired rectangles in image:
Input image as numpy array: https://drive.google.com/file/d/1uN9R4MgVQBzjJuMhcqWMUAhWDJCatHSf/view?usp=sharing
First of all I was trying to threshold binarize the image, following with some
erosion + dilation processing to distinguish all three objects from
each other. Then contours + minAreaRect would give me necessary
result. This option isn't robust enough, because objects in the scene
can be so close to each other, that edge between them has the same
depth as roughness of the object surfaces. So important edges can be
"blended" with object surfaces deviations. Consequently, sometimes,
I'm getting two objects united in one object.
Using canny edge detection with automatically calculated coefficients
(from picture median) catches all unnecessary brightness changes together with edges. Canny with manually adjusted coefficients works better, but it doesn't give closed edge result + it is not reliable (must be manually tweaked each time).
Another thing I tried - adjusting brightness of image nonlinearly (power-law transformation) - to increase brightness of objects surfaces leaving dark edge cavities unchanged.
p = 0.2; c = (input_image.max()) / (input_image.max()**(p)); output_image = (c*blur_gray.astype(np.float)**(p)).astype(np.uint8)
Here is a result: brightness adjusted image.
Threshold binarizing of this image give better results in terms of edges. I tried canny and Laplacian edge detection, but obtained results give disconnected parts of contour with some noise in object surface areas: binarized result of Laplacian filtering. Next step, in my mind, must be some kind of edge estimation/restoration algorithm. I tried Hough transform to get edge lines, but it didn't give any intelligible result.
It seems to me that I just go around in circles without achieving any intelligible result. So I request help. Probably my approach is fundamentally wrong, or I am missing something due to the fact that I do not have sufficient knowledge. Any ideas or suggestions?
P.S. After posting this, I'll continue, and will try to implement wateshed segmentation algorithm, may be it would work.

I tried to come up with a method to emphasize the vertical and horizontal lines separating the shapes.
I started by thresholding the original image (from numpy) and just used a [0, 10] range that seemed reasonable.
I ran a vertical and horizontal line kernel over the image to generate two masks
Vertical Kernel
Horizontal Kernel
I combined the two masks so that we'd have both of the lines separating the boxes
Now we can use findContours to find the boxes. I filtered out small contours to get just the 3 rectangles and used a 4-sided approximation to try and get just their sides.
import cv2
import numpy as np
import random
# approx n-sided shape
def approxSides(contour, numSides, step_size):
# approx until numSides points
num_points = 999999;
percent = step_size;
while num_points >= numSides:
# get number of points
epsilon = percent * cv2.arcLength(contour, True);
approx = cv2.approxPolyDP(contour, epsilon, True);
num_points = len(approx);
# increment
percent += step_size;
# step back and get the points
# there could be more than numSides points if our step size misses it
percent -= step_size * 2;
epsilon = percent * cv2.arcLength(contour, True);
approx = cv2.approxPolyDP(contour, epsilon, True);
return approx;
# convolve
def conv(mask, kernel, size, half):
# get res
h,w = mask.shape[:2];
# loop
nmask = np.zeros_like(mask);
for y in range(half, h - half):
print("Y: " + str(y) + " || " + str(h));
for x in range(half, w - half):
total = np.sum(np.multiply(mask[y-half:y+half+1, x-half:x+half+1], kernel));
total /= 255;
if total > half:
nmask[y][x] = 255;
else:
nmask[y][x] = 0;
return nmask;
# load numpy array
img = np.load("output_data.npy");
mask = cv2.inRange(img, 0, 10);
# resize
h,w = mask.shape[:2];
scale = 0.25;
h = int(h*scale);
w = int(w*scale);
mask = cv2.resize(mask, (w,h));
# use a line filter
size = 31; # size / 2 is max bridge size
half = int(size/2);
vKernel = np.zeros((size,size), np.float32);
for a in range(size):
vKernel[a][half] = 1/size;
hKernel = np.zeros((size,size), np.float32);
for a in range(size):
hKernel[half][a] = 1/size;
# run filters
vmask = cv2.filter2D(mask, -1, vKernel);
vmask = cv2.inRange(vmask, (half * 255 / size), 255);
hmask = cv2.filter2D(mask, -1, hKernel);
hmask = cv2.inRange(hmask, (half * 255 / size), 255);
combined = cv2.bitwise_or(vmask, hmask);
# contours OpenCV3.4, if you're using OpenCV 2 or 4, it returns (contours, _)
combined = cv2.bitwise_not(combined);
_, contours, _ = cv2.findContours(combined, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE);
# filter out small contours
cutoff_size = 1000;
big_cons = [];
for con in contours:
area = cv2.contourArea(con);
if area > cutoff_size:
big_cons.append(con);
# do approx for 4-sided shape
colored = cv2.cvtColor(combined, cv2.COLOR_GRAY2BGR);
four_sides = [];
for con in big_cons:
approx = approxSides(con, 4, 0.01);
color = [random.randint(0,255) for a in range(3)];
cv2.drawContours(colored, [approx], -1, color, 2);
four_sides.append(approx); # not used for anything
# show
cv2.imshow("Image", img);
cv2.imshow("mask", mask);
cv2.imshow("vmask", vmask);
cv2.imshow("hmask", hmask);
cv2.imshow("combined", combined);
cv2.imshow("Color", colored);
cv2.waitKey(0);

Calculate the distance between two pixels in an image

I'm trying to calculate the distance between two pixels, but in a specific way. I need to know the thickness of the red line in the image, so my idea was to go through the image by columns, find the coordinates of the two edge points and calculate the distance between them. Do this for the two lines, both top and bottom. Do this for each column and then calculate the average.
I should also do a conversion from pixels to real scale.
This is my code for now:
# Make numpy array from image
npimage = np.array(image)
# Describe what a single red pixel looks like
red = np.array([255, 0, 0], dtype=np.uint8)
firs_point = 0
first_find = False
for i in range(image.width):
column = npimage[:,i]
for row in column:
comparison = row == red
equal_arrays = comparison.all()
if equal_arrays == True and first_find == False:
first_x_coord = i
first_find = True
I can't get the coordinates. Can someone help me please? Of course, if there are more optimal ways to calculate it, I will be happy to accept proposals. I am very new!
Thank you very much!

After properly masking all red pixels, you can calculate the cumulative sum per each column in that mask:
Below each red line, you have a large area with a constant value: Below the first red line, it's the thickness of that red line. Below the second red line, it's the cumulative thickness of both red lines, and so on (if there would be even more red lines).
So, now, for each column, calculate the histogram from the cumulative sum, and filter out these peaks; leaving out the 0 in the histogram, that'd be the large black area at the top. Per column, you get the above mentioned (cumulative) thickness values for all red lines. The remainder is to extract the actual, single thickness values, and calculate the mean over all those.
Here's my code:
import cv2
import numpy as np
# Read image
img = cv2.imread('Dc4zq.png')
# Mask RGB pure red
mask = (img == [0, 0, 255]).all(axis=2)
# We check for two lines
n = 2
# Cumulative sum for each column
cs = np.cumsum(mask, axis=0)
# Thickness values for each column
tvs = np.zeros((n, img.shape[1]))
for c in range(img.shape[1]):
# Calculate histogram of cumulative sum for a column
hist = np.histogram(cs[:, c], bins=np.arange(img.shape[1]+1))
# Get n highest histogram values
# These are the single thickness values for a column
tv = np.sort(np.argsort(hist[0][1:])[::-1][0:n]+1)
tv[1:] -= tv[:-1]
tvs[:, c] = tv
# Get mean thickness value
mtv = np.mean(tvs.flatten())
print('Mean thickness value:', mtv)
The final result is:
Mean thickness value: 18.92982456140351
----------------------------------------
System information
----------------------------------------
Platform: Windows-10-10.0.16299-SP0
Python: 3.9.1
NumPy: 1.20.1
OpenCV: 4.5.1
----------------------------------------
EDIT: I'll provide some more details on the "NumPy magic" involved.
# Calculate the histogram of the cumulative sum for a single column
hist = np.histogram(cs[:, c], bins=np.arange(img.shape[1] + 1))
Here, bins represent the intervals for the histogram, i.e. [0, 1], [1, 2], and so on. To also get the last interval [569, 570], you need to use img.shape[1] + 1 in the np.arange call, because the right limit is not included in np.arange.
# Get the actual histogram starting from bin 1
hist = hist[0][1:]
In general, np.histogram returns a tuple, where the first element is the actual histogram. We extract that, and only look at all bins larger 0 (remember, the large black area).
Now, let's disassemble this code:
tv = np.sort(np.argsort(hist[0][1:])[::-1][0:n]+1)
This line can be rewritten as:
# Get the actual histogram starting from bin 1
hist = hist[0][1:]
# Get indices of sorted histogram; these are the actual bins
hist_idx = np.argsort(hist)
# Reverse the found indices, since we want those bins with the highest counts
hist_idx = hist_idx[::-1]
# From that indices, we only want the first n elements (assuming there are n red lines)
hist_idx = hist_idx[:n]
# Add 1, because we cut the 0 bin
hist_idx = hist_idx + 1
# As a preparation: Sort the (cumulative) thickness values
tv = np.sort(hist_idx)
By now, we have the (cumulative) thickness values for each column. To reconstruct the actual, single thickness values, we need the "inverse" of the cumulative sum. There's this nice Q&A on that topic.
# The "inverse" of the cumulative sum to reconstruct the actual thickness values
tv[1:] -= tv[:-1]
# Save thickness values in "global" array
tvs[:, c] = tv

using opencv:
img = cv2.imread(image_path)
average_line_width = np.average(np.count_nonzero((img[:,:,:]==np.array([0,0,255])).all(2),axis=0))/2
print(average_line_width)
using pil
img = np.asarray(Image.open(image_path))
average_line_width = np.average(np.count_nonzero((img[:,:,:]==np.array([255,0,0])).all(2),axis=0))/2
print(average_line_width)
output in both cases:
18.430701754385964

I'm not sure I got it but I used the answer of joostblack to calculcate both average thickness in pixel of both lines. Here is my code with comments:
import numpy as np
## Read the image
img = cv2.imread('img.png')
## Create a mask on the red part (I don't use hsv here)
lower_val = np.array([0,0,0])
upper_val = np.array([150,150,255])
mask = cv2.inRange(img, lower_val, upper_val)
## Apply the mask on the image
only_red = cv2.bitwise_and(img,img, mask= mask)
gray = cv2.cvtColor(only_red, cv2.COLOR_BGR2GRAY)
## Find Canny edges
edged = cv2.Canny(gray, 30, 200)
## Find contours
img, contours, hier = cv2.findContours(edged, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
## Select contours using a bonding box
coords=[]
for c in contours:
x,y,w,h = cv2.boundingRect(c)
if w>10:
## Get coordinates of the bounding box is width is sufficient (to avoid noise because you have a huge red line on the left of your image)
coords.append([x,y,w,h])
## Use the previous coordinates to cut the image and compute the average thickness for one red line using the answer proposed by joostblack
for x,y,w,h in coords:
average_line_width = np.average(np.count_nonzero(only_red[y:y+h,x:x+w],axis=0))
print(average_line_width)
## Show you the selected result
cv2.imshow('image',only_red[y:y+h,x:x+w])
cv2.waitKey(0)
The first one is average 6.34 pixels when the 2nd is 5.94 pixels (in the y axis). If you want something more precise you'll need to change this formula!

One way of doing this is to calculate the medial axis (centreline) of the red pixels. And then, as that line is 1px wide, the number of centreline pixels gives the length of the red lines. If you also calculate the number of red pixels, you can easily determine the average line thickness using:
average thickness = number of red pixels / length of red lines
The code looks like this:
#!/usr/bin/env python3
import cv2
import numpy as np
from skimage.morphology import medial_axis
# Load image
im=cv2.imread("Dc4zq.png")
# Make mask of all red pixels and count them
mask = np.alltrue(im==[0,0,255], axis=2)
nRed = np.count_nonzero(mask)
# Get medial axis of red lines and line length
skeleton = (medial_axis(mask*255)).astype(np.uint8)
lenRed = np.count_nonzero(skeleton)
cv2.imwrite('DEBUG-skeleton.png',(skeleton*255).astype(np.uint8))
# We now know the length of the red lines and the total number of red pixels
aveThickness = nRed/lenRed
print(f'Average thickness: {aveThickness}, red line length={lenRed}, num red pixels={nRed}')
That gives the skeleton as follows:
Sample Output
Average thickness: 16.662172878667725, red line length=1261, num red pixels=21011

Finding distinct center of 2 objects

I am trying to find centers of 2 squares in the same image which looks as follows:
I am able to detect the lines that make up the square. My output looks as follows:
As documented here to find the center of a polygon, I used moments to find center. Here is what I did.
import cv2
import numpy as np
img = cv2.imread('images/sq.png', 0)
gray = img
kernel_size = 5
blur_gray = cv2.GaussianBlur(gray,(kernel_size, kernel_size),0)
ret,thresh = cv2.threshold(blur_gray,100,255,0)
low_threshold = 50
high_threshold = 150
edges = cv2.Canny(thresh, low_threshold, high_threshold)
rho = 1 # distance resolution in pixels of the Hough grid
theta = np.pi / 180 # angular resolution in radians of the Hough grid
threshold = 3 # minimum number of votes (intersections in Hough grid cell)
min_line_length = 50 # minimum number of pixels making up a line
max_line_gap = 20 # maximum gap in pixels between connectable line segments
line_image = np.copy(img) * 0 # creating a blank to draw lines on
# Run Hough on edge detected image
# Output "lines" is an array containing endpoints of detected line segments
lines = cv2.HoughLinesP(edges, rho, theta, threshold, np.array([]),
min_line_length, max_line_gap)
for line in lines:
for x1,y1,x2,y2 in line:
cv2.line(line_image,(x1,y1),(x2,y2),(255,0,0),2)
print("x1 {} y1 {} x2 {} y2 {}".format(x1,y1,x2,y2))
lines_edges = cv2.addWeighted(img, 0.5, line_image, 1, 0)
line_image_gray = cv2.cvtColor(line_image, cv2.COLOR_RGB2GRAY)
M = cv2.moments(line_image_gray)
cx = int(M['m10']/M['m00'])
cy = int(M['m01']/M['m00'])
cv2.circle(lines_edges, (cx, cy), 5, (0, 0, 255), 1)
cv2.imshow("res", lines_edges)
cv2.imshow("line_image", line_image)
cv2.waitKey(0)
cv2.destroyAllWindows()
But this finds the center between 2 detected squares. How could I find the centers of each square while only using Hough methods?

Given that you have a requirement to use the Hough transform, I suggest you prepare the image better for it. The Canny edge detector will detect the inner and outer edges of the black line here, leading to two pairs of lines detected by Hough.
Instead, follow a procedure like this:
Find all black (or nearly-black) pixels. For example pixels where all three RGB components are below 50. This will return the squares by themselves.
Apply a morphological thinning (or a skeleton) to turn this into a 1-pixel thick outline of the squares.
Apply the Hough transform on the result, and detect line segments.
Proper pre-processing makes the Hough transform easier to set up, as there will be a larger range of parameters that yields the correct results.
Next, find segments that start or end at the same pixel, with a little bit of tolerance (i.e. start or end points are within a few pixels of each other), to determine which of the lines belong together in the same shape.

You could use this method combined with the following code to find which lines are part of the same square:
How can I check if two segments intersect?
Where 'lines' is a list of the recognized lines, and intersects(line1, line2) is a function using the process in the above link
squares = [[lines(1)]]
for line1 in lines:
for square in squares:
for line2 in square:
if line1 != line2:
if intersects(line1, line2):
square.append(line1)
else:
squares.append([line1])
This gives you 'squares' that contain the lines that are a part of it. You could then use the moment function on each individually.

Non-maximum suppression in corner detection

I am writing a Harris Corner Detection algorithm in Python, and am up to performing non-max suppression in order to detect the corner points.
I have found the corner response function R which appears to be accurate when I print it out, however I do not know where to go from here. I roughly understand the concept of non-max suppression, i.e. taking the pixel with the highest intensity within a window, setting that as the corner point and the rest to 0. Though I am not sure how to go about this in regard to implementation.
After calculating it, would I then use the map it creates to set those pixels within the original image to a particular color (to indicate which are corners)?
My code so far is as follows:
import matplotlib.pyplot as plt
import numpy as np
import cv2
# Load image
img = cv2.imread('mountains.jpg')
# Make a copy of the image
img_copy = np.copy(img)
# Convert image from BGR to RGB
img_copy = cv2.cvtColor(img_copy, cv2.COLOR_BGR2RGB)
# Convert to grayscale for filtering
gray = cv2.cvtColor(img_copy, cv2.COLOR_RGB2GRAY)
# Copy grayscale and convert to float32 type
gray_1 = np.copy(gray)
gray_1 = np.float32(gray_1)
img_1 = np.copy(img)
# Compute derivatives in both x and y directions
sobelx = cv2.Sobel(gray_1, cv2.CV_64F, 1, 0, ksize=5)
sobely = cv2.Sobel(gray_1, cv2.CV_64F, 0, 1, ksize=5)
# Determine M = [A C ; C B] performing element-wise multiplication
A = np.square(sobelx)
B = np.square(sobely)
C = np.multiply(sobelx, sobely)
# Apply gaussian filter to matrix components
gauss = np.array([[1, 2, 1],
[2, 4, 2],
[1, 2, 1]])/16
A_fil = cv2.filter2D(A, cv2.CV_64F, gauss)
B_fil = cv2.filter2D(B, cv2.CV_64F, gauss)
C_fil = cv2.filter2D(C, cv2.CV_64F, gauss)
# Calculate determinant
det = A_fil * B_fil - (C_fil ** 2)
# Calculate trace (alpha = 0.04 to 0.06)
alpha = 0.04
trace = alpha * (A_fil + B_fil) ** 2
# Using determinant and trace, calculate corner response function
R = det - trace
# Display corner response function
f, ax1 = plt.subplots(1, 1, figsize=(20,10))
ax1.set_title('Corner response fuction')
ax1.imshow(R, cmap="gray")
(Note: Stack overflow images were not working properly)
Output:
Using OpenCV's Harris Corner Detection:

It's been a while so you likely already have your answer but since no one has responded yet, I'll leave this for posterity.
Here is a simple algorithm for non-max suppression:
partition your image into tiles (e.g. 64 x 48 pixels)
choose a minimum separation space (e.g. >= 10 pixels between features)
choose a max number of features per tile (such as 5)
Sort your features in each window tile by R (highest to lowest) and then only accept (at most) 5 largest features that are at least 10 pixels away from other features. You won't be able to guarantee a minimum 10px distance across window tiles but this is a reasonable start.