I want to apply Otsu thresholding to image gradients (to remove noise). After that, I want to compute the gradients orientation. Unfortunately, when I do so, I only get gradient orientations between 0 and 90 degrees. Without Otsu thresholding, the values are between 0 and 360.
See my code in Python
import numpy as np
import cv2
img = cv2.imread('Ob.png',cv2.IMREAD_GRAYSCALE)
img = img.astype('float32')
img2 =
dst1 = cv2.Sobel(img,cv2.CV_64F,1,0,ksize=5)
dst2 = cv2.Sobel(img,cv2.CV_64F,0,1,ksize=5)
ret1,th1 = cv2.threshold(dst1.astype(np.uint8),0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
ret2,th2 = cv2.threshold(dst2.astype(np.uint8),0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
mag, ang = cv2.cartToPolar(dst1.astype(np.float32),dst2.astype(np.float32))
np.rad2deg(ang)
What is happening in your code is quite simple to explain:
dst1 and dst2, the output of the two Sobel filters, are the x and y components of the gradient vector. For one given pixel, the gradient vector is given by (dst1[i,j], dst2[i,j]). This vector can have any values, for example (5.8,-2.1), leading to an angle of about 340 degrees.
Next, you threshold these two images. Otsu thresholding will find a value for which the image is nicely separated into pixels of low intensity and pixels of high intensity. These are assigned values of 0 and 255, respectively. But first, you convert the floating-point images to uint8, setting all negative values to 0. So, our vector (5.8,-2.1) is first converted to (5,0), and then thresholded, after which it becomes either (255,0) or (0,0) depending on what side of the threshold the 5 falls.
Thus, we have converted the vector with an angle of 340 degrees to one with an angle of 0 degrees or no computable angle (though atan2(0,0) typically yields 0 also).
In fact, all vectors have become either (0,0), (0,255), (255,0) or (255,255), meaning that you will only find angles of 0, 45 and 90 degrees.
What you should do instead is compute the magnitude, and threshold that (I don't know if Otsu is the ideal method for such an image). Next, use only the angle for those pixels where the magnitude is above the threshold.
Another common alternative is to use Gaussian gradients instead of Sobel. There, you can set a smoothing (regularization) parameter, which allows you to remove more or less noise. I often see this implemented as a Gaussian blur followed by the Sobel filters, though it makes more sense to me to directly use Gaussian derivative filters.
If I may why the first thing you do is to convert the data to float32 ?
I think it would be more efficient to just let it does during the Sobel processing.
That is just my point of view.
The thing you named "noise" as result of the gradient filter is actually called non maxima.
Oftenly algorithm such as Canny does consist to threshold it after the Sobel filtering.
The inconvenient with this approach is to find the appropriate thresholds.
Personally I use the non maxima suppression of another algorithm.
Your code would become:
import numpy as np
import cv2
img = cv2.imread('Ob.png',cv2.IMREAD_GRAYSCALE)
dx,dy = cv2.spatialGradient(img,ksize=5)
mag = cv2.magnitude(dx.astype(np.float32),dy.astype(np.float32))
se = cv2.ximgproc_StructuredEdgeDetection()
ori = se.computeOrientation(mag)
edges_without_nms = se.edgesNms(mag,ori)
I hope it helps you.
Related
Suppose that I have an input x of size [H,W] and also a mu_x and mu_y (which may be fractional)representing the pixels in x and y direction to shift. Is there any efficient way in pytorch without using c++ to shift the tensor x for mu_x and mu_y units with bilinear interpolation.
To be more precise, let's say we have an image. mu_x = 5 and mu_y = 3, we may want to shift the image so that the image moves rightward 5 pixels and downward 3 pixels, with the pixels out of boundary of [H,W] removed and new pixels introduced at the other end of the boundary to be 0. However, with fractional mu_x and mu_y, we need to use bilinear interpolation to estimate the resulting image.
Is it possible to be implemented with pure pytorch tensor operations? Or do I need to use c++.
I believe you can achieve this by applying grid sampling on your original input and using a grid to guide the sampling process. If you take a coordinate grid of your image and sample using that the resulting image will be equal to the original image. However you can apply a shift on this grid and therefore sample with the given shift. Grid sampling works with floating-point grids of course, which means you can apply an arbitrary non-round shift to your image and choose a sampling mode (bilinear is the default).
This can be implemented out of the box with F.grid_sampling. Given an image tensor img, we first construct a pixel grid of that image using torch.meshgrid. Keep in mind the grid used by the sampler must be normalized to [-1, -1]. Therefore pixel x=0,y=0 should be mapped to (-1,-1), pixel x=w,y=h mapped to (1,1), and the center pixel will end up at around (0,0).
Use two torch.arange with a [0,1]-normalization followed by a remapping to [-1,1]:
>>> c,h,w = img.shape
>>> x, y = torch.arange(h)/(h-1), torch.arange(w)/(w-1)
>>> grid = torch.dstack(torch.meshgrid(x, y))*2-1
So the resulting grid has a shape of (c, h, w) which will be the dimensions of the output image produced by the sampling process.
Since we are not working with batched elements, we need to unsqueeze singleton dimensions on both img and grid. Then we can apply F.grid_sample:
>>> sampled = F.grid_sample(img[None], grid[None])
Following this you can apply your arbitrary mu_x, mu_y shift and even easily use this to batches of images and shifts. The way you would define your sampling is by defining a shifted grid:
>>> x_s, y_s = (torch.arange(h)+mu_y)/(h-1), (torch.arange(w)+mu_x)/(w-1)
Where mu_x and mu_y are the values in pixels (floating point) with wish which the image is shifted on the horizontal and vertical axes respectively. To acquire the sampled image, apply F.grid_sampling on a grid made up of x_s and y_s:
>>> grid_shifted = torch.dstack(torch.meshgrid(x_s, y_s))*2-1
>>> sampled = F.grid_sample(img[None], grid_shifted[None])
I am trying to find all the circular particles in the image attached. This is the only image I am have (along with its inverse).
I have read this post and yet I can't use hsv values for thresholding. I have tried using Hough Transform.
circles = cv2.HoughCircles(img, cv2.HOUGH_GRADIENT, dp=0.01, minDist=0.1, param1=10, param2=5, minRadius=3,maxRadius=6)
and using the following code to plot
names =[circles]
for nums in names:
color_img = cv2.imread(path)
blue = (211,211,211)
for x, y, r in nums[0]:
cv2.circle(color_img, (x,y), r, blue, 1)
plt.figure(figsize=(15,15))
plt.title("Hough")
plt.imshow(color_img, cmap='gray')
The following code was to plot the mask:
for masks in names:
black = np.zeros(img_gray.shape)
for x, y, r in masks[0]:
cv2.circle(black, (x,y), int(r), 255, -1) # -1 to draw filled circles
plt.imshow(black, gray)
Yet I am only able to get the following mask which if fairly poor.
This is an image of what is considered a particle and what is not.
One simple approach involves slightly eroding the image, to separate touching circular objects, then doing a connected component analysis and discarding all objects larger than some chosen threshold, and finally dilating the image back so the circular objects are approximately of the original size again. We can do this dilation on the labelled image, such that you retain the separated objects.
I'm using DIPlib because I'm most familiar with it (I'm an author).
import diplib as dip
a = dip.ImageRead('6O0Oe.png')
a = a(0) > 127 # the PNG is a color image, but OP's image is binary,
# so we binarize here to simulate OP's condition.
separation = 7 # tweak these two parameters as necessary
size_threshold = 500
b = dip.Erosion(a, dip.SE(separation))
b = dip.Label(b, maxSize=size_threshold)
b = dip.Dilation(b, dip.SE(separation))
Do note that the image we use here seems to be a zoomed-in screen grab rather than the original image OP is dealing with. If so, the parameters must be made smaller to identify the smaller objects in the smaller image.
My approach is based on a simple observation that most of the particles in your image have approximately same perimeter and the "not particles" have greater perimeter than them.
First, have a look at the RANSAC algorithm and how does it find inliers and outliers. It basically is for 2D data but we will have to transform it to 1D data in our case.
In your case, I am calling inliers to the correct particles and Outliers to incorrect particles.
Our data on which we have to work on will be the perimeter of these particles. To get the perimeter, find contours in this image and get the perimeter of each contour. Refer this for information about Contours.
Now we have the data, knowledge about RANSAC algo and our simple observation mentioned above. Now in this data, we have to find the most dense and compact cluster which will contain all the inliers and others will be outliers.
Now let's assume the inliers are in the range of 40-60 and the outliers are beyond 60. Let's define a threshold value T = 0. We say that for each point in the data, inliers for that point are in the range of (value of that point - T, value of that point + T).
Now first iterate over all the points in the data and count number of inliers to that point for a T and store this information. Find the maximum number of inliers possible for a value of T. Now increment the value of T by 1 and again find the maximum number of inliers possible for that T. Repeat these steps by incrementing value of T one by one.
There will be a range of values of T for which Maximum number of inliers are the same. These inliers are the particles in your image and the particles having perimeter greater than these inliers are the outliers thus the "not particles" in your image.
I have tried this algorithm in my test cases which are similar to your and it works. I am always able to determine the outliers. I hope it works for you too.
One last thing, I see that boundary of your particles are irregular and not smooth, try to make them smooth and use this algorithm if this doesn't work for you in this image.
We need to detect whether the images produced by our tunable lens are blurred or not.
We want to find a proxy measure for blurriness.
My current thinking is to first apply Sobel along the x direction because the jumps or the stripes are mostly along this direction. Then computing the x direction marginal means and finally compute the standard deviation of these marginal means.
We expect this Std is bigger for a clear image and smaller for a blurred one because clear images shall have a large intensity or more bigger jumps of pixel values.
But we get the opposite results. How could we improve this blurriness measure?
def sobel_image_central_std(PATH):
# use the blue channel
img = cv2.imread(PATH)[:,:,0]
# extract the central part of the image
hh, ww = img.shape
hh2 = hh // 2
ww2 = ww// 2
hh4 = hh // 4
ww4 = hh //4
img_center = img[hh4:(hh2+hh4), ww4:(ww2+ww4)]
# Sobel operator
sobelx = cv2.Sobel(img_center, cv2.CV_64F, 1, 0, ksize=3)
x_marginal = sobelx.mean(axis = 0)
plt.plot(x_marginal)
return(x_marginal.std())
Blur #1
Blur #2
Clear #1
Clear #2
In general:
Is there a way to detect if an image is blurry?
You can combine calculation this with your other question where you are searching for the central angle.
Once you have the angle (and the center, maybe outside of the image) you can make an axis transformation to remove the circular component of the cone. Instead you get x (radius) and y (angle) where y would run along the circular arcs.
Maybe you can get the center of the image from the camera set-up.
Then you don't need to calculate it using the intersection of the edges from the central angle. Or just do it manually once if it is fixed for all images.
Look at polar coordinate systems.
Due to the shape of the cone the image will be more dense at the peak but this should be a fixed factor. But this will probably bias the result when calculation the blurriness along the transformed image.
So what you could to correct this is create a synthetic cone image with circular lines and do the transformation on it. Again, requires some try-and-error.
But it should deliver some mask that you could use to correct the "blurriness bias".
I have 10 greyscale brain MRI scans from BrainWeb. They are stored as a 4d numpy array, brains, with shape (10, 181, 217, 181). Each of the 10 brains is made up of 181 slices along the z-plane (going through the top of the head to the neck) where each slice is 181 pixels by 217 pixels in the x (ear to ear) and y (eyes to back of head) planes respectively.
All of the brains are type dtype('float64'). The maximum pixel intensity across all brains is ~1328 and the minimum is ~0. For example, for the first brain, I calculate this by brains[0].max() giving 1328.338086605072 and brains[0].min() giving 0.0003886114541273855. Below is a plot of a slice of a brain[0]:
I want to binarize all these brain images by rescaling the pixel intensities from [0, 1328] to {0, 1}. Is my method correct?
I do this by first normalising the pixel intensities to [0, 1]:
normalized_brains = brains/1328
And then by using the binomial distribution to binarize each pixel:
binarized_brains = np.random.binomial(1, (normalized_brains))
The plotted result looks correct:
A 0 pixel intensity represents black (background) and 1 pixel intensity represents white (brain).
I experimented by implementing another method to normalise an image from this post but it gave me just a black image. This is because np.finfo(np.float64) is 1.7976931348623157e+308, so the normalization step
normalized_brains = brains/1.7976931348623157e+308
just returned an array of zeros which in the binarizition step also led to an array of zeros.
Am I binarising my images using a correct method?
Your method of converting the image to a binary image basically amounts to random dithering, which is a poor method of creating the illusion of grey values on a binary medium. Old-fashioned print is a binary medium, they have fine-tuned the methods to represent grey-value photographs in print over centuries. This process is called halftoning, and is shaped in part by properties of ink on paper, that we do not have to deal with in binary images.
So what methods have people come up with outside of print? Ordered dithering (mostly Bayer matrix), and error diffusion dithering. Read more about dithering on Wikipedia. I wrote a blog post showing how to implement all of these methods in MATLAB some years ago.
I would recommend you use error diffusion dithering for your particular application. Here is some code in MATLAB (taken from my blog post liked above) for the Floyd-Steinberg algorithm, I hope that you can translate this to Python:
img = imread('https://i.stack.imgur.com/d5E9i.png');
img = img(:,:,1);
out = double(img);
sz = size(out);
for ii=1:sz(1)
for jj=1:sz(2)
old = out(ii,jj);
%new = 255*(old >= 128); % Original Floyd-Steinberg
new = 255*(old >= 128+(rand-0.5)*100); % Simple improvement
out(ii,jj) = new;
err = new-old;
if jj<sz(2)
% right
out(ii ,jj+1) = out(ii ,jj+1)-err*(7/16);
end
if ii<sz(1)
if jj<sz(2)
% right-down
out(ii+1,jj+1) = out(ii+1,jj+1)-err*(1/16);
end
% down
out(ii+1,jj ) = out(ii+1,jj )-err*(5/16);
if jj>1
% left-down
out(ii+1,jj-1) = out(ii+1,jj-1)-err*(3/16);
end
end
end
end
imshow(out)
Resampling the image before applying the dithering greatly improves the results:
img = imresize(img,4);
% (repeat code above)
imshow(out)
NOTE that the above process expects the input to be in the range [0,255]. It is easy to adapt to a different range, say [0,1328] or [0,1], but it is also easy to scale your images to the [0,255] range.
Have you tried a threshold on the image?
This is a common way to binarize images, rather than trying to apply a random binomial distribution. You could try something like:
binarized_brains = (brains > threshold_value).astype(int)
which returns an array of 0s and 1s according to whether the image value was less than or greater than your chosen threshold value.
You will have to experiment with the threshold value to find the best one for your images, but it does not need to be normalized first.
If this doesn't work well, you can also experiment with the thresholding options available in the skimage filters package.
IT is easy in OpenCV. as mentioned a very common way is defining a threshold, But your result looks like you are allocating random values to your intensities instead of thresholding it.
import cv2
im = cv2.imread('brain.png', cv2.CV_LOAD_IMAGE_GRAYSCALE)
(th, brain_bw) = cv2.threshold(imy, 128, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)
th = (DEFINE HERE)
im_bin = cv2.threshold(im, th, 255, cv
cv2.imwrite('binBrain.png', brain_bw)
brain
binBrain
I am trying to implement Canny Algorithm using python from scratch.
I am following the steps
Bilateral Filtering the image
Gradient calculation using First Derivative of Gaussian oriented in 4 different directions
def deroGauss(w=5,s=1,angle=0):
wlim = (w-1)/2
y,x = np.meshgrid(np.arange(-wlim,wlim+1),np.arange(-wlim,wlim+1))
G = np.exp(-np.sum((np.square(x),np.square(y)),axis=0)/(2*np.float64(s)**2))
G = G/np.sum(G)
dGdx = -np.multiply(x,G)/np.float64(s)**2
dGdy = -np.multiply(y,G)/np.float64(s)**2
angle = angle*math.pi/180 #converting to radians
dog = math.cos(angle)*dGdx + math.sin(angle)*dGdy
return dog
Non max suppression in all the 4 gradient image
def nonmaxsup(I,gradang):
dim = I.shape
Inms = np.zeros(dim)
weak = np.zeros(dim)
strong = np.zeros(dim)
final = np.zeros(dim)
xshift = int(np.round(math.cos(gradang*np.pi/180)))
yshift = int(np.round(math.sin(gradang*np.pi/180)))
Ipad = np.pad(I,(1,),'constant',constant_values = (0,0))
for r in xrange(1,dim[0]+1):
for c in xrange(1,dim[1]+1):
maggrad = [Ipad[r-xshift,c-yshift],Ipad[r,c],Ipad[r+xshift,c+yshift]]
if Ipad[r,c] == np.max(maggrad):
Inms[r-1,c-1] = Ipad[r,c]
return Inms
Double Thresholding and Hysteresis: Now here the real problem comes.
I am using Otsu's method toe calculate the thresholds.
Should I use the grayscale image or the Gradient images to calculate the threshold?
Because in the gradient Image the pixel intensity values are getting reduced to a very low value after bilateral filtering and then after convolving with Derivative of Gaussian it is reduced further. For example :: 28, 15
Threshold calculated using the grayscale is much above the threshold calculated using the gradient image.
Also If I use the grayscale or even the gradient images to calculate the thresholds the resultant image is not good enough and does not contain all the edges.
So practically, I have nothing left to apply Hysteresis on.
I have tried
img_edge = img_edge*255/np.max(img_edge)
to scale up the values but the result remains the same
But if I use the same thresholds with cv2.Canny the result is very good.
What actually can be wrong?
Applying the Otsu threshold from the original image doesn't make sense, it is completely unrelated to the gradient intensities.
Otsu from the gradient intensities is not perfect because the statistical distributions of noise and edges are skewed and overlap a lot.
You can try some small multiple of Otsu or some small multiple of the average. But in no case will you get perfect results by simple or hysteresis thresholding. Edge detection is an ill-posed problem.