I'm trying to detect straight edges in a basketball card and what I have so far does a good job of detecting all edges. I would like for this piece of code however, to detect exclusively straight edges (the outline of the card).
import cv2
import numpy as np
import imutils
img = cv2.imread('edgedetection/cardgiannis.jpeg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
kernel_size = 5
blur_gray = cv2.GaussianBlur(gray,(kernel_size, kernel_size),0)
low_threshold = 50
high_threshold = 150
edges = cv2.Canny(blur_gray, low_threshold, high_threshold)
rho = 1 # distance resolution in pixels of the Hough grid
theta = np.pi / 180 # angular resolution in radians of the Hough grid
threshold = 15 # minimum number of votes (intersections in Hough grid cell)
min_line_length = 50 # minimum number of pixels making up a line
max_line_gap = 20 # maximum gap in pixels between connectable line segments
line_image = np.copy(img) * 0 # creating a blank to draw lines on
# Run Hough on edge detected image
# Output "lines" is an array containing endpoints of detected line segments
lines = cv2.HoughLinesP(edges, rho, theta, threshold, np.array([]),
min_line_length, max_line_gap)
for line in lines:
for x1,y1,x2,y2 in line:
cv2.line(line_image,(x1,y1),(x2,y2),(255,0,0),5)
# Draw the lines on the image
lines_edges = cv2.addWeighted(img, 0.8, line_image, 1, 0)
cv2.imshow('lines', lines_edges)
cv2.waitKey()
Here is what I think.
If you can somehow get the highest & lowest valued coordinates in the pixels that are forming the lines, you can use those pixels to form a rectangle that consist of the straight edges.
Using the pixels to (roughly) form a rectangle can be the solution!
To find the pixels in a line, you can take a look in here, here and here.
you may use cv.approxPolyDP to extract dominant points from edges and stick to the edges that have 2 dominant points.
ps: you apply approxPolyDP on edges after collecting them by cv.findcontours
Related
I have a 512x512 image of a street grid:
I'd like to extract polylines for each of the streets in this image (large blue dots = intersections, small blue dots = points along polylines):
I've tried a few techniques! One idea was to start with skeletonize to compress the streets down to 1px wide lines:
from skimage import morphology
morphology.skeletonize(streets_data))
Unfortunately this has some gaps that break the connectivity of the street network; I'm not entirely sure why, but my guess is that this is because some of the streets are 1px narrower in some places and 1px wider in others. (update: the gaps aren't real; they're entirely artifacts of how I was displaying the skeleton. See this comment for the sad tale. The skeleton is well-connected.)
I can patch these using a binary_dilation, at the cost of making the streets somewhat variable width again:
out = morphology.skeletonize(streets_data)
out = morphology.binary_dilation(out, morphology.selem.disk(1))
With a re-connected grid, I can run the Hough transform to find line segments:
import cv2
rho = 1 # distance resolution in pixels of the Hough grid
theta = np.pi / 180 # angular resolution in radians of the Hough grid
threshold = 8 # minimum number of votes (intersections in Hough grid cell)
min_line_length = 10 # minimum number of pixels making up a line
max_line_gap = 2 # maximum gap in pixels between connectable line segments
# Run Hough on edge detected image
# Output "lines" is an array containing endpoints of detected line segments
lines = cv2.HoughLinesP(
out, rho, theta, threshold, np.array([]),
min_line_length, max_line_gap
)
line_image = streets_data.copy()
for i, line in enumerate(lines):
for x1,y1,x2,y2 in line:
cv2.line(line_image,(x1,y1),(x2,y2), 2, 1)
This produces a whole jumble of overlapping line segments, along with some gaps (look at the T intersection on the right side):
At this point I could try to de-dupe overlapping line segments, but it's not really clear to me that this is a path towards a solution, especially given that gap.
Are there more direct methods available to get at the network of polylines I'm looking for? In particular, what are some methods for:
Finding the intersections (both four-way and T intersections).
Shrinking the streets to all be 1px wide, allowing that there may be some variable width.
Finding the polylines between intersections.
If you want to improve your "skeletonization", you could try the following algorithm to obtain the "1-px wide streets":
import imageio
import numpy as np
from matplotlib import pyplot as plt
from scipy.ndimage import distance_transform_edt
from skimage.segmentation import watershed
# read image
image_rgb = imageio.imread('1mYBD.png')
# convert to binary
image_bin = np.max(image_rgb, axis=2) > 0
# compute the distance transform (only > 0)
distance = distance_transform_edt(image_bin)
# segment the image into "cells" (i.e. the reciprocal of the network)
cells = watershed(distance)
# compute the image gradients
grad_v = np.pad(cells[1:, :] - cells[:-1, :], ((0, 1), (0, 0)))
grad_h = np.pad(cells[:, 1:] - cells[:, :-1], ((0, 0), (0, 1)))
# given that the cells have a constant value,
# only the edges will have non-zero gradient
edges = (abs(grad_v) > 0) + (abs(grad_h) > 0)
# extract points into (x, y) coordinate pairs
pos_v, pos_h = np.nonzero(edges)
# display points on top of image
plt.imshow(image_bin, cmap='gray_r')
plt.scatter(pos_h, pos_v, 1, np.arange(pos_h.size), cmap='Spectral')
The algorithm works on the "blocks" rather than the "streets", take a look into the cells image:
I was on the right track with skeleton; it does produce a connected, 1px wide version of the street grid. It's just that there was a bug in my display code (see this comment). Here's what the skeleton actually looks like:
from skimage import morphology
morphology.skeletonize(streets_data))
From here the reference to NEFI2 in RJ Adriaansen's comment was extremely helpful. I wasn't able to get an extraction pipeline to run on my image using their GUI, but I was able to cobble together something using their code and approach.
Here's the general procedure I wound up with:
Find candidate nodes. These are pixels in the skeleton with either one neighbor (end of a line) or 3+ neighbors (an intersection). NEFI2 calls this the Zhang-Suen algorithm. For four-way intersections, it produces multiple nodes so we need to merge them.
Repeat until no two nodes are too close together:
Use breadth-first search (flood fill) to connect nodes.
If two connected nodes are within D of each other, merge them.
Run shapely's simplify on the paths between nodes to get polylines.
I put this all together in a repo here: extract-raster-network.
This works pretty well on my test images! Here are some sample images:
Our team set up a vision system with a camera, a microscope and a tunable lens to look at the internal surface of a cone.
Visually speaking, the camera takes 12 image for one cone with each image covering 30 degrees.
Now we've collected many sample images and want to make sure each "fan"(as shown below) is at least 30 degree.
Is there any way in Python, with cv2 or other packages, to measure this central angle. Thanks.
Here is one way to do that in Python/OpenCV.
Read the image
Convert to gray
Threshold
Use morphology open and close to smooth and fill out the boundary
Apply Canny edge extraction
Separate the image into top edge and bottom edge by blackening the opposite side to each edge
Fit lines to the top and bottom edges
Compute the angle of each edge
Compute the difference between the two angles
Draw the lines on the input
Save the results
Input:
import cv2
import numpy as np
import math
# read image
img = cv2.imread('cone_shape.jpg')
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# threshold
thresh = cv2.threshold(gray,11,255,cv2.THRESH_BINARY)[1]
# apply open then close to smooth boundary
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (13,13))
morph = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel)
kernel = np.ones((33,33), np.uint8)
morph = cv2.morphologyEx(morph, cv2.MORPH_CLOSE, kernel)
# apply canny edge detection
edges = cv2.Canny(morph, 150, 200)
hh, ww = edges.shape
hh2 = hh // 2
# split edge image in half vertically and blacken opposite half
top_edge = edges.copy()
top_edge[hh2:hh, 0:ww] = 0
bottom_edge = edges.copy()
bottom_edge[0:hh2, 0:ww] = 0
# get coordinates of white pixels in top and bottom
# note: need to transpose y,x in numpy to x,y for opencv
top_white_pts = np.argwhere(top_edge.transpose()==255)
bottom_white_pts = np.argwhere(bottom_edge.transpose()==255)
# fit lines to white pixels
# (x,y) is point on line, (vx,vy) is unit vector along line
(vx1,vy1,x1,y1) = cv2.fitLine(top_white_pts, cv2.DIST_L2, 0, 0.01, 0.01)
(vx2,vy2,x2,y2) = cv2.fitLine(bottom_white_pts, cv2.DIST_L2, 0, 0.01, 0.01)
# compute angle for vectors vx,vy
top_angle = (180/math.pi)*math.atan(vy1/vx1)
bottom_angle = (180/math.pi)*math.atan(vy2/vx2)
print(top_angle, bottom_angle)
# cone angle is the difference
cone_angle = math.fabs(top_angle - bottom_angle)
print(cone_angle)
# draw lines on input
lines = img.copy()
p1x1 = int(x1-1000*vx1)
p1y1 = int(y1-1000*vy1)
p1x2 = int(x1+1000*vx1)
p1y2 = int(y1+1000*vy1)
cv2.line(lines, (p1x1,p1y1), (p1x2,p1y2), (0, 0, 255), 1)
p2x1 = int(x2-1000*vx2)
p2y1 = int(y2-1000*vy2)
p2x2 = int(x2+1000*vx2)
p2y2 = int(y2+1000*vy2)
cv2.line(lines, (p2x1,p2y1), (p2x2,p2y2), (0, 0, 255), 1)
# save resulting images
cv2.imwrite('cone_shape_thresh.jpg',thresh)
cv2.imwrite('cone_shape_morph.jpg',morph)
cv2.imwrite('cone_shape_edges.jpg',edges)
cv2.imwrite('cone_shape_lines.jpg',lines)
# show thresh and result
cv2.imshow("thresh", thresh)
cv2.imshow("morph", morph)
cv2.imshow("edges", edges)
cv2.imshow("top edge", top_edge)
cv2.imshow("bottom edge", bottom_edge)
cv2.imshow("lines", lines)
cv2.waitKey(0)
cv2.destroyAllWindows()
Thresholded image:
Morphology processed image:
Edge Image:
Lines on input:
Cone Angle (in degrees):
42.03975696357633
That sounds possible. You need to do some preprocessing and filtering to figure out what works and there is probably some tweaking involved.
There are three approaches that could work.
1.)
The basic idea is to somehow get two lines and measure the angle between them.
Define a threshold to define the outer black region (out of the central angle) and set all values below it to zero.
This will also set some of the blurry stripes inside the central angle to zero so we have to try to "heal" them away. This is done by using Morphological Transformations. You can read about them here and here.
You could try the operation Closing, but I don't know if it fixes stripes. Usually it fixes dots or scratches. This answer seems to indicate that it should work on lines.
Maybe at that point apply some Gaussian blurring and to the threshold thing again. Then try to use some edge or line detection.
It's basically try and error, you have to see what works.
2.)
Another thing that could work is to try to use the arc-enter code herelike scratches, maybe even strengthen them and use the Hough Circle Transform. I think it detects arcs as well.
Just try it and see what the function returns. In the best case there are several circles / arcs that you can use to estimate the central angle.
There are several approaches on arc detection here on StackOverflow or here.
I am not sure if that's the same with all your image, but the one above looks like there are some thin, green and pink arcs that seem to stretch all along the central angle. You could use that to filter for that color, then make it grey scale.
This question might be helpful.
3.)
Apply an edge filter, e.g Canny skimage.feature.canny
Try several sigmas and post the images in your question, then we can try to think on how to continue.
What could work is to calculate the convex hull around all points that are part of an edge. Then get the two lines that form the central angle from the convex hull.
I have a dataset with two classes of images: Cityscape and Landscape. What I want to do is calculate the gradient(orientation) of the edges of each image and show that images of cityscapes have more vertical/horizontal edges than landscape images.
What I've done is calculated vertical, horizontal, 45 degree and 135 degree edges. I've applied a Canny filter to the images, calculated the x,y gradients and also applied a threshold to the images show it shows edges above that threshold. The result of this thresholding is seen here:
This is my code for this image manipulation as well as calculating the gradients:
def gradient(image):
# Step 1
img = image
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# Step 2
bi = cv2.bilateralFilter(gray, 15, 75, 75)
# Step 3
dst = cv2.Canny(bi, 100, 200)
#print(np.count_nonzero(dst)) #--> make sure it's not all zeroes
# Step 4
#--- create a black image to see where those edges occur ---
mask = np.zeros_like(gray)
#--- applying a threshold and turning those pixels above the threshold to white ---
mask[dst > 0.1 * dst.max()] = 255
# Step 5
img[dst > 0.1 * dst.max()] = [255, 0, 0] #--- [255, 0, 0] --> Red ---
Gx = cv2.Sobel(mask,cv2.CV_64F,1,0,ksize=5)
Gy = cv2.Sobel(mask,cv2.CV_64F,0,1,ksize=5)
#orientation of the edges
theta = np.arctan2(Gy, Gx)
#magnitude
M = np.sqrt(Gx*Gx + Gy*Gy)
#Vertical edges:
v = abs(Gy)
#Horizontal edges:
h = abs(Gx)
#45 Degree edges:
deg45 = M*abs(np.cos(theta - np.pi/4))
#135 Degree edges:
deg135 = M*abs(np.cos(theta - 3*np.pi/4))
print('Vertical:')
#print(v)
print(np.count_nonzero(v))
print('Horizontal:')
#print(h)
print(np.count_nonzero(h))
What I want is to calculate the v,h,deg45,deg135 for the edges shown as red in the image above (Step 5). If that is not possible, then do that for the image with the white edges (Step 4). Can anyone help?
EDIT: So as to avoid confusion, what I want to do is to get the amount of vertical, horizontal etc edges in a given image, so that I can compare those numbers for cityscapes vs landscape images.
If what you want is the total number of pixels comprising horizontal vs vertical edges, I would suggest defining some threshold for horizontal vs vertical (say 15 degrees). So you can count the number of elements of theta for which
abs(theta) < pi/12 (horizontal)
or abs(theta) > pi-pi/12 (horizontal)
or pi/2 - pi/12 < abs(theta) < pi/2+pi/12 (vertical)
What you're storing in v and h are the vertical and horizontal components of the gradient at each point and what you need is to compare the values of v and h to determine for each point if the gradient vector should count as horizontal or vertical. Comparing theta is probably the most intuitive way to do this.
In order to get the number of elements of theta that satisfy a particular condition, I would suggest using a generator expression:
sum(1 for i in theta if (abs(i)<pi/12) or (abs(i)>pi-pi/12))
would give you the number of horizontal edge pixels for example.
I think Hough transform fits more to your needs if you want to count or control how many linear features you have in your image, either you could count how many linear feature for each specific orientation (in the hough space). As soon you are using Python, this and this link might be helpful!
I am trying to find centers of 2 squares in the same image which looks as follows:
I am able to detect the lines that make up the square. My output looks as follows:
As documented here to find the center of a polygon, I used moments to find center. Here is what I did.
import cv2
import numpy as np
img = cv2.imread('images/sq.png', 0)
gray = img
kernel_size = 5
blur_gray = cv2.GaussianBlur(gray,(kernel_size, kernel_size),0)
ret,thresh = cv2.threshold(blur_gray,100,255,0)
low_threshold = 50
high_threshold = 150
edges = cv2.Canny(thresh, low_threshold, high_threshold)
rho = 1 # distance resolution in pixels of the Hough grid
theta = np.pi / 180 # angular resolution in radians of the Hough grid
threshold = 3 # minimum number of votes (intersections in Hough grid cell)
min_line_length = 50 # minimum number of pixels making up a line
max_line_gap = 20 # maximum gap in pixels between connectable line segments
line_image = np.copy(img) * 0 # creating a blank to draw lines on
# Run Hough on edge detected image
# Output "lines" is an array containing endpoints of detected line segments
lines = cv2.HoughLinesP(edges, rho, theta, threshold, np.array([]),
min_line_length, max_line_gap)
for line in lines:
for x1,y1,x2,y2 in line:
cv2.line(line_image,(x1,y1),(x2,y2),(255,0,0),2)
print("x1 {} y1 {} x2 {} y2 {}".format(x1,y1,x2,y2))
lines_edges = cv2.addWeighted(img, 0.5, line_image, 1, 0)
line_image_gray = cv2.cvtColor(line_image, cv2.COLOR_RGB2GRAY)
M = cv2.moments(line_image_gray)
cx = int(M['m10']/M['m00'])
cy = int(M['m01']/M['m00'])
cv2.circle(lines_edges, (cx, cy), 5, (0, 0, 255), 1)
cv2.imshow("res", lines_edges)
cv2.imshow("line_image", line_image)
cv2.waitKey(0)
cv2.destroyAllWindows()
But this finds the center between 2 detected squares. How could I find the centers of each square while only using Hough methods?
Given that you have a requirement to use the Hough transform, I suggest you prepare the image better for it. The Canny edge detector will detect the inner and outer edges of the black line here, leading to two pairs of lines detected by Hough.
Instead, follow a procedure like this:
Find all black (or nearly-black) pixels. For example pixels where all three RGB components are below 50. This will return the squares by themselves.
Apply a morphological thinning (or a skeleton) to turn this into a 1-pixel thick outline of the squares.
Apply the Hough transform on the result, and detect line segments.
Proper pre-processing makes the Hough transform easier to set up, as there will be a larger range of parameters that yields the correct results.
Next, find segments that start or end at the same pixel, with a little bit of tolerance (i.e. start or end points are within a few pixels of each other), to determine which of the lines belong together in the same shape.
You could use this method combined with the following code to find which lines are part of the same square:
How can I check if two segments intersect?
Where 'lines' is a list of the recognized lines, and intersects(line1, line2) is a function using the process in the above link
squares = [[lines(1)]]
for line1 in lines:
for square in squares:
for line2 in square:
if line1 != line2:
if intersects(line1, line2):
square.append(line1)
else:
squares.append([line1])
This gives you 'squares' that contain the lines that are a part of it. You could then use the moment function on each individually.
I'm trying to count dendritic spines (the tiny protuberances) in mouse dendrites obtained by fluorescent microscopy, using Python and OpenCV.
Here is the original image, from which I'm starting:
Raw picture:
After some preprocessing (code below) I've obtained these contours:
Raw picture with contours (White):
What I need to do is to recognize all protuberances, obtaining something like this:
Raw picture with contours in White and expected counts in red:
What I intended to do, after preprocessing the image (binarizing, thresholding and reducing its noise), was drawing the contours and try to find convex defects in them. The problem arose as some of the "spines" (the technical name of those protuberances) are not recognized as they en up bulged together in the same convexity defect, underestimating the result. Is there any way to be more "precise" when marking convexity defects?
Raw image with contour marked in White. Red dots mark spines that were identified with my code. Green dots mark spines I still can't recognize:
My Python code:
import cv2
import numpy as np
from matplotlib import pyplot as plt
#Image loading and preprocessing:
img = cv2.imread('Prueba.jpg')
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = cv2.pyrMeanShiftFiltering(img,5,11)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret,thresh1 = cv2.threshold(gray,5,255,0)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))
img1 = cv2.morphologyEx(thresh1, cv2.MORPH_OPEN, kernel)
img1 = cv2.morphologyEx(img1, cv2.MORPH_OPEN, kernel)
img1 = cv2.dilate(img1,kernel,iterations = 5)
#Drawing of contours. Some spines were dettached of the main shaft due to
#image bad quality. The main idea of the code below is to identify the shaft
#as the biggest contour, and count any smaller as a spine too.
_, contours,_ = cv2.findContours(img1,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
print("Number of contours detected: "+str(len(contours)))
cv2.drawContours(img,contours,-1,(255,255,255),6)
plt.imshow(img)
plt.show()
lengths = [len(i) for i in contours]
cnt = lengths.index(max(lengths))
#The contour of the main shaft is stored in cnt
cnt = contours.pop(cnt)
#Finding convexity points with hull:
hull = cv2.convexHull(cnt)
#The next lines are just for visualization. All centroids of smaller contours
#are marked as spines.
for i in contours:
M = cv2.moments(i)
centroid_x = int(M['m10']/M['m00'])
centroid_y = int(M['m01']/M['m00'])
centroid = np.array([[[centroid_x, centroid_y]]])
print(centroid)
cv2.drawContours(img,centroid,-1,(0,255,0),25)
cv2.drawContours(img,centroid,-1,(255,0,0),10)
cv2.drawContours(img,hull,-1,(0,255,0),25)
cv2.drawContours(img,hull,-1,(255,0,0),10)
plt.imshow(img)
plt.show()
#Finally, the number of spines is computed as the sum between smaller contours
#and protuberances in the main shaft.
spines = len(contours)+len(hull)
print("Number of identified spines: " + str(spines))
I know my code has many smaller problems to solve yet, but I think the biggest one is the one presented here.
Thanks for your help! and have a good one
I would approximate the contour to a polygon as Silencer suggests (don't use the convex hull). Maybe you should simplify the contour just a little bit to keep most of the detail of the shape.
This way, you will have many vertices that you have to filter: looking at the angle of each vertex you can tell if it is concave or convex. Each spine is one or more convex vertices between concave vertices (if you have several consecutive convex vertices, you keep only the sharper one).
EDIT: in order to compute the angle you can do the following: let's say that a, b and c are three consecutive vertices
angle1 = arctan((by-ay)/(bx-ax))
angle2 = arctan((cy-by)/(cx-bx))
angleDiff=angle2-angle1
if(angleDiff<-PI) angleDiff=angleDiff+2PI
if(angleDiff>0) concave
Else convex
Or vice versa, depending if your contour is clockwise or counterclockwise, black or white. If you sum all angleDiff of any polygon, the result should be 2PI. If it is -2PI, then the last "if" should be swapped.