Our team set up a vision system with a camera, a microscope and a tunable lens to look at the internal surface of a cone.
Visually speaking, the camera takes 12 image for one cone with each image covering 30 degrees.
Now we've collected many sample images and want to make sure each "fan"(as shown below) is at least 30 degree.
Is there any way in Python, with cv2 or other packages, to measure this central angle. Thanks.
Here is one way to do that in Python/OpenCV.
Read the image
Convert to gray
Threshold
Use morphology open and close to smooth and fill out the boundary
Apply Canny edge extraction
Separate the image into top edge and bottom edge by blackening the opposite side to each edge
Fit lines to the top and bottom edges
Compute the angle of each edge
Compute the difference between the two angles
Draw the lines on the input
Save the results
Input:
import cv2
import numpy as np
import math
# read image
img = cv2.imread('cone_shape.jpg')
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# threshold
thresh = cv2.threshold(gray,11,255,cv2.THRESH_BINARY)[1]
# apply open then close to smooth boundary
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (13,13))
morph = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel)
kernel = np.ones((33,33), np.uint8)
morph = cv2.morphologyEx(morph, cv2.MORPH_CLOSE, kernel)
# apply canny edge detection
edges = cv2.Canny(morph, 150, 200)
hh, ww = edges.shape
hh2 = hh // 2
# split edge image in half vertically and blacken opposite half
top_edge = edges.copy()
top_edge[hh2:hh, 0:ww] = 0
bottom_edge = edges.copy()
bottom_edge[0:hh2, 0:ww] = 0
# get coordinates of white pixels in top and bottom
# note: need to transpose y,x in numpy to x,y for opencv
top_white_pts = np.argwhere(top_edge.transpose()==255)
bottom_white_pts = np.argwhere(bottom_edge.transpose()==255)
# fit lines to white pixels
# (x,y) is point on line, (vx,vy) is unit vector along line
(vx1,vy1,x1,y1) = cv2.fitLine(top_white_pts, cv2.DIST_L2, 0, 0.01, 0.01)
(vx2,vy2,x2,y2) = cv2.fitLine(bottom_white_pts, cv2.DIST_L2, 0, 0.01, 0.01)
# compute angle for vectors vx,vy
top_angle = (180/math.pi)*math.atan(vy1/vx1)
bottom_angle = (180/math.pi)*math.atan(vy2/vx2)
print(top_angle, bottom_angle)
# cone angle is the difference
cone_angle = math.fabs(top_angle - bottom_angle)
print(cone_angle)
# draw lines on input
lines = img.copy()
p1x1 = int(x1-1000*vx1)
p1y1 = int(y1-1000*vy1)
p1x2 = int(x1+1000*vx1)
p1y2 = int(y1+1000*vy1)
cv2.line(lines, (p1x1,p1y1), (p1x2,p1y2), (0, 0, 255), 1)
p2x1 = int(x2-1000*vx2)
p2y1 = int(y2-1000*vy2)
p2x2 = int(x2+1000*vx2)
p2y2 = int(y2+1000*vy2)
cv2.line(lines, (p2x1,p2y1), (p2x2,p2y2), (0, 0, 255), 1)
# save resulting images
cv2.imwrite('cone_shape_thresh.jpg',thresh)
cv2.imwrite('cone_shape_morph.jpg',morph)
cv2.imwrite('cone_shape_edges.jpg',edges)
cv2.imwrite('cone_shape_lines.jpg',lines)
# show thresh and result
cv2.imshow("thresh", thresh)
cv2.imshow("morph", morph)
cv2.imshow("edges", edges)
cv2.imshow("top edge", top_edge)
cv2.imshow("bottom edge", bottom_edge)
cv2.imshow("lines", lines)
cv2.waitKey(0)
cv2.destroyAllWindows()
Thresholded image:
Morphology processed image:
Edge Image:
Lines on input:
Cone Angle (in degrees):
42.03975696357633
That sounds possible. You need to do some preprocessing and filtering to figure out what works and there is probably some tweaking involved.
There are three approaches that could work.
1.)
The basic idea is to somehow get two lines and measure the angle between them.
Define a threshold to define the outer black region (out of the central angle) and set all values below it to zero.
This will also set some of the blurry stripes inside the central angle to zero so we have to try to "heal" them away. This is done by using Morphological Transformations. You can read about them here and here.
You could try the operation Closing, but I don't know if it fixes stripes. Usually it fixes dots or scratches. This answer seems to indicate that it should work on lines.
Maybe at that point apply some Gaussian blurring and to the threshold thing again. Then try to use some edge or line detection.
It's basically try and error, you have to see what works.
2.)
Another thing that could work is to try to use the arc-enter code herelike scratches, maybe even strengthen them and use the Hough Circle Transform. I think it detects arcs as well.
Just try it and see what the function returns. In the best case there are several circles / arcs that you can use to estimate the central angle.
There are several approaches on arc detection here on StackOverflow or here.
I am not sure if that's the same with all your image, but the one above looks like there are some thin, green and pink arcs that seem to stretch all along the central angle. You could use that to filter for that color, then make it grey scale.
This question might be helpful.
3.)
Apply an edge filter, e.g Canny skimage.feature.canny
Try several sigmas and post the images in your question, then we can try to think on how to continue.
What could work is to calculate the convex hull around all points that are part of an edge. Then get the two lines that form the central angle from the convex hull.
Related
I've been researching and trying a couple functions to get what I want and I feel like I might be overthinking it.
One version of my code is below. The sample image is here.
My end goal is to find the angle (yellow) of the approximated line with respect to the frame (green line) Final
I haven't even got to the angle portion of the program yet.
The results I was obtaining from the below code were as follows. Canny Closed Small Removed
Anybody have a better way of creating the difference and establishing the estimated line?
Any help is appreciated.
import cv2
import numpy as np
pX = int(512)
pY = int(768)
img = cv2.imread('IMAGE LOCATION', cv2.IMREAD_COLOR)
imgS = cv2.resize(img, (pX, pY))
aimg = cv2.imread('IMAGE LOCATION', cv2.IMREAD_GRAYSCALE)
# Blur image to reduce noise and resize for viewing
blur = cv2.medianBlur(aimg, 5)
rblur = cv2.resize(blur, (384, 512))
canny = cv2.Canny(rblur, 120, 255, 1)
cv2.imshow('canny', canny)
kernel = np.ones((2, 2), np.uint8)
#fringeMesh = cv2.dilate(canny, kernel, iterations=2)
#fringeMesh2 = cv2.dilate(fringeMesh, None, iterations=1)
#cv2.imshow('fringeMesh', fringeMesh2)
closing = cv2.morphologyEx(canny, cv2.MORPH_CLOSE, kernel)
cv2.imshow('Closed', closing)
nb_components, output, stats, centroids = cv2.connectedComponentsWithStats(closing, connectivity=8)
#connectedComponentswithStats yields every separated component with information on each of them, such as size
sizes = stats[1:, -1]; nb_components = nb_components - 1
min_size = 200 #num_pixels
fringeMesh3 = np.zeros((output.shape))
for i in range(0, nb_components):
if sizes[i] >= min_size:
fringeMesh3[output == i + 1] = 255
#contours, _ = cv2.findContours(fringeMesh3, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
#cv2.drawContours(fringeMesh3, contours, -1, (0, 255, 0), 1)
cv2.imshow('final', fringeMesh3)
#cv2.imshow("Natural", imgS)
#cv2.imshow("img", img)
cv2.imshow("aimg", aimg)
cv2.imshow("Blur", rblur)
cv2.waitKey()
cv2.destroyAllWindows()
You can fit a straight line to the first white pixel you encounter in each column, starting from the bottom.
I had to trim your image because you shared a screen grab of it with a window decoration, title and frame rather than your actual image:
import cv2
import math
import numpy as np
# Load image as greyscale
im = cv2.imread('trimmed.jpg', cv2.IMREAD_GRAYSCALE)
# Get index of first white pixel in each column, starting at the bottom
yvals = (im[::-1,:]>200).argmax(axis=0)
# Make the x values 0, 1, 2, 3...
xvals = np.arange(0,im.shape[1])
# Fit a line of the form y = mx + c
z = np.polyfit(xvals, yvals, 1)
# Convert the slope to an angle
angle = np.arctan(z[0]) * 180/math.pi
Note 1: The value of z (the result of fitting) is:
array([ -0.74002694, 428.01463745])
which means the equation of the line you are looking for is:
y = -0.74002694 * x + 428.01463745
i.e. the y-intercept is at row 428 from the bottom of the image.
Note 2: Try to avoid JPEG format as an intermediate format in image processing - it is lossy and changes your pixel values - so where you have thresholded and done your morphology you are expecting values of 255 and 0, JPEG will lossily alter those values and you end up testing for a range or thresholding again.
Your 'Closed' image seems to quite clearly segment the two regions, so I'd suggest you focus on turning that boundary into a line that you can do something with. Connected components analysis and contour detection don't really provide any useful information here, so aren't necessary.
One quite simple approach to finding the line angle is to find the first white pixel in each row. To get only the rows that are part of your diagonal, don't include rows where that pixel is too close to either side (e.g. within 5%). That gives you a set of points (pixel locations) on the boundary of your two types of grass.
From there you can either do a linear regression to get an equation for the straight line, or you can get two points by averaging the x values for the top and bottom half of the rows, and then calculate the gradient angle from that.
An alternative approach would be doing another morphological close with a very large kernel, to end up with just a solid white region and a solid black region, which you could turn into a line with canny or findContours. From there you could either get some points by averaging, use the endpoints, or given a smooth enough result from a large enough kernel you could detect the line with hough lines.
I have used:
result = cv2.matchTemplate(frame, template, cv2.TM_CCORR_NORMED)
to generate this output:
I need a list of (x, y) tuples at each of the local maxima (bright spots) in the result. Simply finding all points above a threshold doesn't work, since there are many such points around each maximum.
I can guarantee the minimum distance between any two maxima, which ought to help speed things up.
Is there an efficient technique for doing this?
(P.S.: this is cross-posted from https://forum.opencv.org/t/locating-local-maximums/1534)
update
Based on an excellent suggestion by Michael Lee, I've added skeletonizing to the thresholded image. It's close, but the skeletonizing still has many "worms" rather than single points. My processing flow is as follows:
# read the image
im = cv.imread("image.png", cv.IMREAD_GRAYSCALE)
# apply thresholding
ret, im2 = cv.threshold(im, args.threshold, 255, cv.THRESH_BINARY)
# dilate the thresholded image to eliminate "pinholes"
im3 = cv.dilate(im2, None, iterations=2)
# skeletonize the result
im4 = cv.ximgproc.thinning(im3, None, cv.ximgproc.THINNING_ZHANGSUEN)
# print the number of points found
x, y = np.nonzero(im5)
print(x.shape)
# => 1208
This is a step in the right direction, but there should be more like 220 points, not 1208.
Here are the intermediate results. As you can see in the last picture (skeletonized), there are still lots of little "worms" rather than single point. Is there a better approach?
Thresholded:
Dilated:
Skeletonized:
Update 2/14: Seems like skeletonization only took you part of the way there. Here's a better solution which I believe should get you the rest of the way. Here's how you would do it in scikit-image - maybe you can find the analog in OpenCV - seems like cv2.findContours would be a good start.
# mask is the thresholded image (before or after dilation should work, no skeletonization.
from skimage.measure import label, regionprops
labeled_image = label(mask)
output_points = [region.centroid for region in regionprops(labeled_image)]
Explanation: Label will convert your binary image into a labeled image, where each mask has a different integer value. Then, regionprops uses these labels in order to separate each mask, from which we can use the centroid property to compute the middle point from each - this is guaranteed to be a single point.
Simply finding all points above a threshold doesn't work, since there
are many such points around each maximum.
Actually, this does work - as long as you apply one more processing step. After thresholding, then we want to skeletonize. Scikit-image has a good function to achieve that here, which should give you a binary mask with single points.
Afterwards, you're probably going to want to run something like:
indices = zip(*np.where(skeleton))
to get your final points!
Based on Michael Lee's answer, here's the solution that worked for me (using all openCV rather than skimage):
# read in color image and create a grayscale copy
im = cv.imread("image.png")
img = cv.cvtColor(im, cv.COLOR_BGR2GRAY)
# apply thresholding
ret, im2 = cv.threshold(img, args.threshold, 255, cv.THRESH_BINARY)
# dilate the thresholded peaks to eliminate "pinholes"
im3 = cv.dilate(im2, None, iterations=2)
contours, hier = cv.findContours(im3, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
print('found', len(contours), 'contours')
# draw a bounding box around each contour
for contour in contours:
x,y,w,h = cv.boundingRect(contour)
cv.rectangle(im, (x,y), (x+w,y+h), (255,0,0), 2)
cv.imshow('Contours', im)
cv.waitKey()
which results in just what we're looking for:
I am trying to write an algorithm to systematically determine how many different "curves" are in an image. Example Image. I'm specifically interested in the white lines here, so I've used a color threshold to mask the rest of the image and only get the white pixels. These lines represent a path run by a player (wide receivers in the NFL), so I'm interested in the x and y coordinates that the path represents - and each "curve" represents a different path that the player took (or "route"). All curves should start on or behind the blue line.
However, while I can get just the white pixels, I can't figure out how to systematically identify the separate curves. In this example image, there are 8 white curves (or routes) present. I've identified those curves in this image. I tried edge detection, and then using scipy ndimage to get the number of connected components, but because the curves overlap it counts them as connected and only gives me 3 labeled components for this image as opposed to eight. Here's what the edge detection output looks like. Is there a better way to go about this? Here is my sample code.
import cv2
from skimage.morphology import skeletonize
import numpy as np
from scipy import ndimage
#Read in image
image = cv2.imread('example_image.jpeg')
#Color boundary to get white pixels
lower_white = np.array([230, 230, 230])
upper_white = np.array([255, 255, 255])
#mask image for white pixels
mask = cv2.inRange(image, lower_white, upper_white)
c_pixels = cv2.bitwise_and(image, image, mask=mask)
#make pixels from 0 to 1 form to use in skeletonize
c_pixels = c_pixels.clip(0,1)
ske_c = skeletonize(c_pixels[:,:,1]).astype(np.uint8)
#Edge Detection
inputImage =ske_c*255
edges = cv2.Canny(inputImage,100,200,apertureSize = 7)
#Show edges
cv2.imshow('edges', edges)
cv2.waitKey(0)
cv2.destroyAllWindows()
#Find number of components
# smooth the image (to remove small objects); set the threshold
edgesf = ndimage.gaussian_filter(edges, 1)
T = 50 # set threshold by hand to avoid installing `mahotas` or
# `scipy.stsci.image` dependencies that have threshold() functions
# find connected components
labeled, nr_objects = ndimage.label(edgesf > T) # `dna[:,:,0]>T` for red-dot case
print("Number of objects is %d " % nr_objects)
I'm trying to count dendritic spines (the tiny protuberances) in mouse dendrites obtained by fluorescent microscopy, using Python and OpenCV.
Here is the original image, from which I'm starting:
Raw picture:
After some preprocessing (code below) I've obtained these contours:
Raw picture with contours (White):
What I need to do is to recognize all protuberances, obtaining something like this:
Raw picture with contours in White and expected counts in red:
What I intended to do, after preprocessing the image (binarizing, thresholding and reducing its noise), was drawing the contours and try to find convex defects in them. The problem arose as some of the "spines" (the technical name of those protuberances) are not recognized as they en up bulged together in the same convexity defect, underestimating the result. Is there any way to be more "precise" when marking convexity defects?
Raw image with contour marked in White. Red dots mark spines that were identified with my code. Green dots mark spines I still can't recognize:
My Python code:
import cv2
import numpy as np
from matplotlib import pyplot as plt
#Image loading and preprocessing:
img = cv2.imread('Prueba.jpg')
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = cv2.pyrMeanShiftFiltering(img,5,11)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret,thresh1 = cv2.threshold(gray,5,255,0)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))
img1 = cv2.morphologyEx(thresh1, cv2.MORPH_OPEN, kernel)
img1 = cv2.morphologyEx(img1, cv2.MORPH_OPEN, kernel)
img1 = cv2.dilate(img1,kernel,iterations = 5)
#Drawing of contours. Some spines were dettached of the main shaft due to
#image bad quality. The main idea of the code below is to identify the shaft
#as the biggest contour, and count any smaller as a spine too.
_, contours,_ = cv2.findContours(img1,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
print("Number of contours detected: "+str(len(contours)))
cv2.drawContours(img,contours,-1,(255,255,255),6)
plt.imshow(img)
plt.show()
lengths = [len(i) for i in contours]
cnt = lengths.index(max(lengths))
#The contour of the main shaft is stored in cnt
cnt = contours.pop(cnt)
#Finding convexity points with hull:
hull = cv2.convexHull(cnt)
#The next lines are just for visualization. All centroids of smaller contours
#are marked as spines.
for i in contours:
M = cv2.moments(i)
centroid_x = int(M['m10']/M['m00'])
centroid_y = int(M['m01']/M['m00'])
centroid = np.array([[[centroid_x, centroid_y]]])
print(centroid)
cv2.drawContours(img,centroid,-1,(0,255,0),25)
cv2.drawContours(img,centroid,-1,(255,0,0),10)
cv2.drawContours(img,hull,-1,(0,255,0),25)
cv2.drawContours(img,hull,-1,(255,0,0),10)
plt.imshow(img)
plt.show()
#Finally, the number of spines is computed as the sum between smaller contours
#and protuberances in the main shaft.
spines = len(contours)+len(hull)
print("Number of identified spines: " + str(spines))
I know my code has many smaller problems to solve yet, but I think the biggest one is the one presented here.
Thanks for your help! and have a good one
I would approximate the contour to a polygon as Silencer suggests (don't use the convex hull). Maybe you should simplify the contour just a little bit to keep most of the detail of the shape.
This way, you will have many vertices that you have to filter: looking at the angle of each vertex you can tell if it is concave or convex. Each spine is one or more convex vertices between concave vertices (if you have several consecutive convex vertices, you keep only the sharper one).
EDIT: in order to compute the angle you can do the following: let's say that a, b and c are three consecutive vertices
angle1 = arctan((by-ay)/(bx-ax))
angle2 = arctan((cy-by)/(cx-bx))
angleDiff=angle2-angle1
if(angleDiff<-PI) angleDiff=angleDiff+2PI
if(angleDiff>0) concave
Else convex
Or vice versa, depending if your contour is clockwise or counterclockwise, black or white. If you sum all angleDiff of any polygon, the result should be 2PI. If it is -2PI, then the last "if" should be swapped.
I've read a lot about the Circular Hough transform on Stack Overflow, but I seem to be missing something. I wrote a program that is supposed to detect the circles of a "Bull's Eye" target. However, even after playing with the parameters, the algorithm is quite bad - it ignores most of the circles and one time it finds a circle but seems to "wander off". I've even tried applying an "Unsharp Mask" to no avail. I have added my code, the image I started with and the output. I hope someone can point me at the right direction.
import cv2
import cv2.cv as cv
import numpy as np
import math
# Load Image
img = cv2.imread('circles1.png',0)
# Apply Unsharp Mask
tmp = cv2.medianBlur(img,5)
img = cv2.addWeighted(img,1.5,tmp,-0.5,0)
cimg = cv2.cvtColor(img,cv2.COLOR_GRAY2BGR)
# Hough Transform
circles = cv2.HoughCircles(img,cv.CV_HOUGH_GRADIENT,1,5,
param1=100,param2=100,minRadius=0,maxRadius=0)
circles = np.uint16(np.around(circles))
# Go over circles, eliminating the ones that are not cocentric enough
height, width = img.shape
center = (width/2,height/2)
for i in circles[0,:]:
# draw the outer circle
if math.sqrt((center[0]-i[0])**2 + (center[1]-i[1])**2) < 15:
cv2.circle(cimg,(i[0],i[1]),i[2],(0,255,0),1)
# draw the center of the circle
cv2.circle(cimg,(i[0],i[1]),2,(0,0,255),3)
cv2.imshow('detected circles',cimg)
cv2.waitKey(0)
cv2.destroyAllWindows()
Quick explanation: I load the image, apply Unsharp Mask, use the Hough Transfrom to detect circles, then draw the circles that are close to the center (I found that the other circles are false circles).
I tried playing with the parameters, and this is the best I got. I feel like this is a simple enough problem which has me buffled. I appriciate any help.
My input image:
My output image:
As I mentioned in my comment, you'll need to run successive iterations of cv2.HoughCircles for different range of radii to ensure that you get all of the circles. With the way the Circular Hough Transform works, specifying a minimum and maximum radius that has quite a large range will be inaccurate and will also be slow. They don't tell you this in the documentation, but for the Circular Hough Transform to work successfully, the following two things need to be valid:
maxRadius < 3*minRadius
maxRadius - minRadius < 100
With the above, blindly making the minimum radius very small and the maximum radius very large won't give you great results. Therefore, what you could do is start at... say...radius=1, then iterate up to radius=300 in steps of 20. Between each chunk of 20, run cv2.HoughCircles and update your image with these contours.
Doing this requires very little modification to your code. BTW, I removed the unsharp masking it because I was getting poor results with it. I also changed a couple of parameters in cv2.HoughCircles slightly to get this to work as best as possible given your situation:
import cv2
import cv2.cv as cv
import numpy as np
import math
# Load Image
img = cv2.imread('circles1.png',0)
cimg = cv2.cvtColor(img,cv2.COLOR_GRAY2BGR)
# Specify different radii
radii = np.arange(0,310,10)
# For each pair of radii...
for idx in range(len(radii)-1):
# Get the minimum and maximum radius
# Note you need to add 1 to each minimum
# as the maximum of the previous pair covers this new minimum
minRadius = radii[idx]+1
maxRadius = radii[idx+1]
# Hough Transform - Change here
circles = cv2.HoughCircles(img,cv.CV_HOUGH_GRADIENT,1,5,
param1=25,param2=75,minRadius=minRadius,maxRadius=maxRadius)
# Skip if no circles are detected - Change here
if circles is None:
continue
circles = np.uint16(np.around(circles))
# Go over circles, eliminating the ones that are not cocentric enough
height, width = img.shape
center = (width/2,height/2)
for i in circles[0,:]:
# draw the outer circle
if math.sqrt((center[0]-i[0])**2 + (center[1]-i[1])**2) < 15:
cv2.circle(cimg,(i[0],i[1]),i[2],(0,255,0),1)
# draw the center of the circle
cv2.circle(cimg,(i[0],i[1]),2,(0,0,255),3)
cv2.imshow('detected circles',cimg)
cv2.waitKey(0)
cv2.destroyAllWindows()
I get this figure:
Unfortunately it isn't perfect as it doesn't detect all of the circles. You'll have to play around with the cv2.HoughCircles function until you get good results.
However, I wouldn't recommend using cv2.HoughCircles here. May I suggest using cv2.findContours instead? This finds all of the contours in the image. In this case, these will be the black circles. However, you need to reverse the image because cv2.findContours assumes non-zero pixels are object pixels, so we can subtract 255 from the image assuming a np.uint8 type:
# Make copy of original image
cimg2 = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
# Find contours
contours,_ = cv2.findContours(255 - img, cv2.RETR_LIST, cv.CV_CHAIN_APPROX_NONE)
# Draw all detected contours on image in green with a thickness of 1 pixel
cv2.drawContours(cimg2, contours, -1, color=(0,255,0), thickness=1)
# Show the image
cv2.imshow('detected circles', cimg2)
cv2.waitKey(0)
cv2.destroyAllWindows()
This is what I get: