I am trying to implement Canny Algorithm using python from scratch.
I am following the steps
Bilateral Filtering the image
Gradient calculation using First Derivative of Gaussian oriented in 4 different directions
def deroGauss(w=5,s=1,angle=0):
wlim = (w-1)/2
y,x = np.meshgrid(np.arange(-wlim,wlim+1),np.arange(-wlim,wlim+1))
G = np.exp(-np.sum((np.square(x),np.square(y)),axis=0)/(2*np.float64(s)**2))
G = G/np.sum(G)
dGdx = -np.multiply(x,G)/np.float64(s)**2
dGdy = -np.multiply(y,G)/np.float64(s)**2
angle = angle*math.pi/180 #converting to radians
dog = math.cos(angle)*dGdx + math.sin(angle)*dGdy
return dog
Non max suppression in all the 4 gradient image
def nonmaxsup(I,gradang):
dim = I.shape
Inms = np.zeros(dim)
weak = np.zeros(dim)
strong = np.zeros(dim)
final = np.zeros(dim)
xshift = int(np.round(math.cos(gradang*np.pi/180)))
yshift = int(np.round(math.sin(gradang*np.pi/180)))
Ipad = np.pad(I,(1,),'constant',constant_values = (0,0))
for r in xrange(1,dim[0]+1):
for c in xrange(1,dim[1]+1):
maggrad = [Ipad[r-xshift,c-yshift],Ipad[r,c],Ipad[r+xshift,c+yshift]]
if Ipad[r,c] == np.max(maggrad):
Inms[r-1,c-1] = Ipad[r,c]
return Inms
Double Thresholding and Hysteresis: Now here the real problem comes.
I am using Otsu's method toe calculate the thresholds.
Should I use the grayscale image or the Gradient images to calculate the threshold?
Because in the gradient Image the pixel intensity values are getting reduced to a very low value after bilateral filtering and then after convolving with Derivative of Gaussian it is reduced further. For example :: 28, 15
Threshold calculated using the grayscale is much above the threshold calculated using the gradient image.
Also If I use the grayscale or even the gradient images to calculate the thresholds the resultant image is not good enough and does not contain all the edges.
So practically, I have nothing left to apply Hysteresis on.
I have tried
img_edge = img_edge*255/np.max(img_edge)
to scale up the values but the result remains the same
But if I use the same thresholds with cv2.Canny the result is very good.
What actually can be wrong?
Applying the Otsu threshold from the original image doesn't make sense, it is completely unrelated to the gradient intensities.
Otsu from the gradient intensities is not perfect because the statistical distributions of noise and edges are skewed and overlap a lot.
You can try some small multiple of Otsu or some small multiple of the average. But in no case will you get perfect results by simple or hysteresis thresholding. Edge detection is an ill-posed problem.
Related
I am working on a lane detection project and I want to feed in the path robot can take between crop rows. I initially converted the image to birds eye view for better processing and tried Hough transform, but Hough transform is not giving me good results.
Bird's eye view of the image
Are there any other approaches I am missing out?
Before applying the Hough lines algorithm you could do the following :
1)Color shifting
Apply color shifting where you split the colors of the image into blue, green and red channels. Since the crop rows are green you can amplify the color green to stand out more from the rest of the channels
b,g,r = cv2.split(img)
gscale = 2*g-r-b
2)Canny Edge Detection
Fiddle with the min and max arguments in the cv2.Canny() function until satisfactory.
gscale = cv2.Canny(gscale,minVal,maxValue)
3)Skeletonization
Skeletonization is the process of thinning the regions of interest to their binary constituents. This makes it easier for to perform pattern recognition.
size = np.size(gscale) #returns the product of the array dimensions
skel = np.zeros(gscale.shape,np.uint8) #array of zeros
ret,gscale = cv2.threshold(gscale,128,255,0) #thresholding the image
element = cv2.getStructuringElement(cv2.MORPH_CROSS,(3,3))
done = False
while( not done):
eroded = cv2.erode(gscale,element)
temp = cv2.dilate(eroded,element)
temp = cv2.subtract(gscale,temp)
skel = cv2.bitwise_or(skel,temp)
gscale = eroded.copy()
zeros = size - cv2.countNonZero(gscale)
if zeros==size:
done = True
You should get better performance in the Hough lines algorithm after applying all these in their respective order.
We need to detect whether the images produced by our tunable lens are blurred or not.
We want to find a proxy measure for blurriness.
My current thinking is to first apply Sobel along the x direction because the jumps or the stripes are mostly along this direction. Then computing the x direction marginal means and finally compute the standard deviation of these marginal means.
We expect this Std is bigger for a clear image and smaller for a blurred one because clear images shall have a large intensity or more bigger jumps of pixel values.
But we get the opposite results. How could we improve this blurriness measure?
def sobel_image_central_std(PATH):
# use the blue channel
img = cv2.imread(PATH)[:,:,0]
# extract the central part of the image
hh, ww = img.shape
hh2 = hh // 2
ww2 = ww// 2
hh4 = hh // 4
ww4 = hh //4
img_center = img[hh4:(hh2+hh4), ww4:(ww2+ww4)]
# Sobel operator
sobelx = cv2.Sobel(img_center, cv2.CV_64F, 1, 0, ksize=3)
x_marginal = sobelx.mean(axis = 0)
plt.plot(x_marginal)
return(x_marginal.std())
Blur #1
Blur #2
Clear #1
Clear #2
In general:
Is there a way to detect if an image is blurry?
You can combine calculation this with your other question where you are searching for the central angle.
Once you have the angle (and the center, maybe outside of the image) you can make an axis transformation to remove the circular component of the cone. Instead you get x (radius) and y (angle) where y would run along the circular arcs.
Maybe you can get the center of the image from the camera set-up.
Then you don't need to calculate it using the intersection of the edges from the central angle. Or just do it manually once if it is fixed for all images.
Look at polar coordinate systems.
Due to the shape of the cone the image will be more dense at the peak but this should be a fixed factor. But this will probably bias the result when calculation the blurriness along the transformed image.
So what you could to correct this is create a synthetic cone image with circular lines and do the transformation on it. Again, requires some try-and-error.
But it should deliver some mask that you could use to correct the "blurriness bias".
I want to apply Otsu thresholding to image gradients (to remove noise). After that, I want to compute the gradients orientation. Unfortunately, when I do so, I only get gradient orientations between 0 and 90 degrees. Without Otsu thresholding, the values are between 0 and 360.
See my code in Python
import numpy as np
import cv2
img = cv2.imread('Ob.png',cv2.IMREAD_GRAYSCALE)
img = img.astype('float32')
img2 =
dst1 = cv2.Sobel(img,cv2.CV_64F,1,0,ksize=5)
dst2 = cv2.Sobel(img,cv2.CV_64F,0,1,ksize=5)
ret1,th1 = cv2.threshold(dst1.astype(np.uint8),0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
ret2,th2 = cv2.threshold(dst2.astype(np.uint8),0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
mag, ang = cv2.cartToPolar(dst1.astype(np.float32),dst2.astype(np.float32))
np.rad2deg(ang)
What is happening in your code is quite simple to explain:
dst1 and dst2, the output of the two Sobel filters, are the x and y components of the gradient vector. For one given pixel, the gradient vector is given by (dst1[i,j], dst2[i,j]). This vector can have any values, for example (5.8,-2.1), leading to an angle of about 340 degrees.
Next, you threshold these two images. Otsu thresholding will find a value for which the image is nicely separated into pixels of low intensity and pixels of high intensity. These are assigned values of 0 and 255, respectively. But first, you convert the floating-point images to uint8, setting all negative values to 0. So, our vector (5.8,-2.1) is first converted to (5,0), and then thresholded, after which it becomes either (255,0) or (0,0) depending on what side of the threshold the 5 falls.
Thus, we have converted the vector with an angle of 340 degrees to one with an angle of 0 degrees or no computable angle (though atan2(0,0) typically yields 0 also).
In fact, all vectors have become either (0,0), (0,255), (255,0) or (255,255), meaning that you will only find angles of 0, 45 and 90 degrees.
What you should do instead is compute the magnitude, and threshold that (I don't know if Otsu is the ideal method for such an image). Next, use only the angle for those pixels where the magnitude is above the threshold.
Another common alternative is to use Gaussian gradients instead of Sobel. There, you can set a smoothing (regularization) parameter, which allows you to remove more or less noise. I often see this implemented as a Gaussian blur followed by the Sobel filters, though it makes more sense to me to directly use Gaussian derivative filters.
If I may why the first thing you do is to convert the data to float32 ?
I think it would be more efficient to just let it does during the Sobel processing.
That is just my point of view.
The thing you named "noise" as result of the gradient filter is actually called non maxima.
Oftenly algorithm such as Canny does consist to threshold it after the Sobel filtering.
The inconvenient with this approach is to find the appropriate thresholds.
Personally I use the non maxima suppression of another algorithm.
Your code would become:
import numpy as np
import cv2
img = cv2.imread('Ob.png',cv2.IMREAD_GRAYSCALE)
dx,dy = cv2.spatialGradient(img,ksize=5)
mag = cv2.magnitude(dx.astype(np.float32),dy.astype(np.float32))
se = cv2.ximgproc_StructuredEdgeDetection()
ori = se.computeOrientation(mag)
edges_without_nms = se.edgesNms(mag,ori)
I hope it helps you.
I am trying to threshold images with challenging noise.
.
The numbers on the side are the dimensions. I have tried various standard methods:
ret,thresh1 = cv2.threshold(img,95,255,0)
#cv2.THRESH_BINARY
thresh2 = cv2.adaptiveThreshold(img,255,cv2.ADAPTIVE_THRESH_MEAN_C,cv2.THRESH_BINARY,7,0.5)
thresh3 = cv2.adaptiveThreshold(img,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C,cv2.THRESH_BINARY,3,1.5)
# Otsu's thresholding after Gaussian filtering
blur = cv2.GaussianBlur(img,(3,3),0)
ret3,th3 = cv2.threshold(blur,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
I want to segment the "lighter" portion inside the darker grey zone (or vice versa). I have played with various kernel sizes, and constant values but nothing is giving me a good segmentation. Any ideas what else i can try or how to improve the results? Some sample results i get using the code is
I am trying to segment the blood vessels in retinal images using Python and OpenCV. Here is the original image:
Ideally I want all the blood vessels to be very visible like this (different image):
Here is what I have tried so far. I took the green color channel of the image.
img = cv2.imread('images/HealthyEyeFundus.jpg')
b,g,r = cv2.split(img)
Then I tried to create a matched filter by following this article and this is what the output image is:
Then I tried doing max entropy thresholding:
def max_entropy(data):
# calculate CDF (cumulative density function)
cdf = data.astype(np.float).cumsum()
# find histogram's nonzero area
valid_idx = np.nonzero(data)[0]
first_bin = valid_idx[0]
last_bin = valid_idx[-1]
# initialize search for maximum
max_ent, threshold = 0, 0
for it in range(first_bin, last_bin + 1):
# Background (dark)
hist_range = data[:it + 1]
hist_range = hist_range[hist_range != 0] / cdf[it] # normalize within selected range & remove all 0 elements
tot_ent = -np.sum(hist_range * np.log(hist_range)) # background entropy
# Foreground/Object (bright)
hist_range = data[it + 1:]
# normalize within selected range & remove all 0 elements
hist_range = hist_range[hist_range != 0] / (cdf[last_bin] - cdf[it])
tot_ent -= np.sum(hist_range * np.log(hist_range)) # accumulate object entropy
# find max
if tot_ent > max_ent:
max_ent, threshold = tot_ent, it
return threshold
img = skimage.io.imread('image.jpg')
# obtain histogram
hist = np.histogram(img, bins=256, range=(0, 256))[0]
# get threshold
th = max_entropy.max_entropy(hist)
print th
ret,th1 = cv2.threshold(img,th,255,cv2.THRESH_BINARY)
This is the result I'm getting, which is obviously not showing all the blood vessels:
I've also tried taking the matched filter version of the image and taking the magnitude of its sobel values.
img0 = cv2.imread('image.jpg',0)
sobelx = cv2.Sobel(img0,cv2.CV_64F,1,0,ksize=5) # x
sobely = cv2.Sobel(img0,cv2.CV_64F,0,1,ksize=5) # y
magnitude = np.sqrt(sobelx**2+sobely**2)
This makes the vessels pop out more:
Then I tried Otsu thresholding on it:
img0 = cv2.imread('image.jpg',0)
# # Otsu's thresholding
ret2,th2 = cv2.threshold(img0,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
# Otsu's thresholding after Gaussian filtering
blur = cv2.GaussianBlur(img0,(9,9),5)
ret3,th3 = cv2.threshold(blur,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
one = Image.fromarray(th2).show()
one = Image.fromarray(th3).show()
Otsu doesn't give adequate results. It ends up including noise in the results:
Any help is appreciated on how I can segment the blood vessels successfully.
I worked on retina vessel detection for a bit few years ago, and there are different ways to do it:
If you don't need a top result but something fast, you can use oriented openings, see here and here.
Then you have an other version using mathematical morphology version here.
For better results, here are some ideas:
Personally, I used combination of Gabor filters, and results where pretty good. See the segmentation result here on the first image of drive.
And Gabor can be combined with learning for a good result, or here.
Few years ago, they claimed to have the best algorithm, but I've never had the opportunity to test it. I was sceptic about the performance gap and the way they thresholded the line detector results, it was kind of obscure.
But I know that nowadays, many people try to tackle the problem using CNN, but I've not heard about significant improvements.