Using the threshold functions in open CV on an image to get a binary image, with Otsu's thresholding I get a image that has white spots due to different lighting conditions in parts of the image
or with adaptive threshold to fix the lighting conditions, it fails to accurately represent the pencil-filled bubbles that Otsu actually can represent.
How can I get both the filled bubbles represented and a fixed lighting conditions without patches?
Here's the original image
Here is my code
#binary image conversion
thresh2 = cv2.adaptiveThreshold(papergray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY_INV, 21, 13)
thresh = cv2.threshold(papergray, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
cv2.imshow("Binary", thresh) #Otsu's
cv2.imshow("Adpative",thresh2)
An alternative approach would be to apply a morphological closing, which would remove all the drawing, yielding an estimate of the illumination level. Dividing the image by the illumination level gives you an image of the sheet corrected for illumination:
In this image we can easily apply a global threshold:
I used the following code:
import diplib as dip
img = dip.ImageRead('tlCw6.jpg')(1)
corrected = img / dip.Closing(img, dip.SE(40, 'rectangular'))
out = dip.Threshold(corrected, method='triangle')[0]
This can be done with cv2.ADAPTIVE_THRESH_MEAN_C:
import cv2
img = cv2.imread("omr.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
thresh = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY_INV, 31, 10)
cv2.imshow("Mean Adaptive Thresholding", thresh)
cv2.waitKey(0)
The output is:
Problems with your approach:
The methods you have tried out:
Otsu threshold is decided based on all the pixel values in the entire image (global technique). If you look at the bottom-left of your image, there is a gray shade which can have an adverse effect in deciding the threshold value.
Adaptive threshold: here is a recent answer on why it isn't helpful. In short, it acts like an edge detector for smaller kernel sizes
What you can try:
OpenCV's ximgproc module has specialized binary image generation methods. One such method is the popular Niblack threshold technique.
This is a local threshold technique that depends on statistical measures. It divides the image into blocks (sub-images) of size predefined by the user. A threshold is set based on the mean minus k times standard deviation of pixel values for each block. The k is decided by the user.
Code:
img =cv2.imread('omr_sheet.jpg')
blur = cv2.GaussianBlur(img, (3, 3), 0)
gray = cv2.cvtColor(blur, cv2.COLOR_BGR2GRAY)
niblack = cv2.ximgproc.niBlackThreshold(gray, 255, cv2.THRESH_BINARY, 41, -0.1, binarizationMethod=cv2.ximgproc.BINARIZATION_NICK)
Result:
Links:
To know more about cv2.ximgproc.niBlackThreshold
There are other binarization techniques available that you may want to explore. It also contains links to research papers that explain each of these techniques on detail.
Edit:
Adaptive threshold actually works if you know what you are working with. You can decide the kernel size beforehand.
See Prashant's answer.
Related
Take a look at this image. I want to turn this card blue.
I use Python and OpenCV to perform image processing.
Here's how I do it now:
import cv2
import numpy as np
# Load the image
image = cv2.imread("input.jpg")
# Convert the image to the HSV color space
hsv_image = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
# Threshold the HSV image to get only the red colors
# Bitwise OR unites Hue value 170-179 and 0-10.
mask = cv2.bitwise_or(
cv2.inRange(hsv_image, np.array([0, 120, 100]), np.array([10, 255, 255])),
cv2.inRange(hsv_image, np.array([170, 120, 100]), np.array([180, 255, 255]))
)
# Perform median blurring
mask = cv2.medianBlur(mask, 5)
# Define a kernel for the morphological operations
kernel = np.ones((5, 5), np.uint8)
# Perform an opening operation to remove small objects
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel, iterations=2)
# Perform a closing operation to fill small holes
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel, iterations=2)
# Perform a gradient operation to extract object boundaries
gradient_mask = cv2.morphologyEx(mask, cv2.MORPH_GRADIENT, kernel)
# Modify hue value of every masked pixel
hsv_image[:, :, 0][mask != 0] = (hsv_image[:, :, 0][mask != 0].astype(int, copy=False) + 120) % 180
# Convert the HSV image back to BGR color space
result = cv2.cvtColor(hsv_image, cv2.COLOR_HSV2BGR)
# Display the result
cv2.namedWindow("output", cv2.WINDOW_NORMAL)
cv2.imshow("output", result)
# Save images of the mask, result and gradient
cv2.imwrite('mask.jpg', mask)
cv2.imwrite('result.jpg', result)
cv2.imwrite('gradient.jpg', gradient_mask)
# Wait for the window to close
while cv2.getWindowProperty('output', 0) >= 0:
cv2.waitKey(50)
cv2.destroyAllWindows()
It works well. The result, The filtering mask
But if you take a closer look you'll see the problem: link, link
Some red pixels is still here, and here's why. They do not fall in the filtered range of the red color: 170 <= Hue <= 10, Saturation >= 120, Value >= 100. Their HSV color is near to (178, 32, 60). So, the saturation and value fall out of the filter range.
Why I can't lower the range of saturation and value? That's because in this case there would be too much noise on another background that has more colors. The noise in this case is hard to avoid even using multiple iterations of opening morphological operation.
I don't have much experience in image processing and using OpenCV, so my ideas may be far from the best solution. It's okay if you propose another approach.
Possible solution. Would it be possible to perform the dilate morphological operation on the filtering mask (expand the filtering mask) but only to those pixels that fall in another, broader range of red (with saturation and value range equal to 10, hue range stays the same). So that all the red pixels that fall in the broader range of the red color AND that are adjacent to the pixels of the existing mask (so no pixels from the background is added creating noise).
If that is a good idea, how can I implement it, especially the part of dilating only to pixels that fall in the broader range of the red color? Maybe it's already implemented in OpenCV and I just don't know about that?
Also I would be glad to hear any suggestions or recommendations. I am a student, I want to learn more.
Thanks in advance!
I found myself how to do it, but it's still not the best mask.
I create a broader mask as follows:
broader_mask = cv2.bitwise_or(
cv2.inRange(hsv_image, np.array([0, 30, 30]), np.array([20, 255, 255])),
cv2.inRange(hsv_image, np.array([160, 30, 30]), np.array([180, 255, 255]))
)
And apply bitwise AND on the dilated primary mask and the broader mask to get the resulting mask:
mask = cv2.bitwise_and(cv2.dilate(mask, kernel, iterations=2), broader_mask)
It works much better, but now the mask might be bigger than I want in some cases. For example:
It all depends on the kernel and the number of iterations of the dilation operation. I don't think it's the best solution because the kernel and the number of iterations might be different for different image sizes, and I'm still looking for a better solution.
I'm working in a script using different OpenCV operations for processing an image with solar panels in a house roof. My original image is the following:
After processing the image, I get the edges of the panels as follows:
It can be seen how some rectangles are broken due to reflection of the Sun in the picture.
I would like to know if it's possible to fix those broken rectangles, maybe by using the pattern of those which are not broken.
My code is the following:
# Load image
color_image = cv2.imread("google6.jpg")
cv2.imshow("Original", color_image)
# Convert to gray
img = cv2.cvtColor(color_image, cv2.COLOR_BGR2GRAY)
# Apply various filters
img = cv2.GaussianBlur(img, (5, 5), 0)
img = cv2.medianBlur(img, 5)
img = img & 0x88 # 0x88
img = cv2.fastNlMeansDenoising(img, h=10)
# Invert to binary
ret, thresh = cv2.threshold(img, 127, 255, 1)
# Perform morphological erosion
kernel = np.ones((5, 5),np.uint8)
erosion = cv2.morphologyEx(thresh, cv2.MORPH_ERODE, kernel, iterations=2)
# Invert image and blur it
ret, thresh1 = cv2.threshold(erosion, 127, 255, 1)
blur = cv2.blur(thresh1, (10, 10))
# Perform another threshold on blurred image to get the central portion of the edge
ret, thresh2 = cv2.threshold(blur, 145, 255, 0)
# Perform morphological erosion to thin the edge by ellipse structuring element
kernel1 = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))
contour = cv2.morphologyEx(thresh2, cv2.MORPH_ERODE, kernel1, iterations=2)
# Get edges
final = cv2.Canny(contour, 249, 250)
cv2.imshow("final", final)
I have tried to modify all the filters I'm using in order to reduce as much as possible the effect of the Sun in the original picture, but that is as far as I have been able to go.
I'm in general happy with the result of all those filters (although any advice is welcome), so I'd like to work on the black/white imaged I showed, which is already smooth enough for the post-processing I need to do.
Thansk!
The pattern is not broken in the original image, so it being broken in your binarized result must mean your binarization is not optimal.
You apply threshold() to binarize the image, and then Canny() to the binary image. The problems here are:
Thresholding removes a lot of information, this should always be the last step of any processing pipeline. Anything you lose here, you've lost for good.
Canny() should be applied to a gray-scale image, not a binary image.
The Canny edge detector is an edge detector, but you want to detect lines, not edges. See here for the difference.
So, I suggest starting from scratch.
The Laplacian of Gaussian is a very simple line detector. I took these steps:
Read in image, convert to grayscale.
Apply Laplacian of Gaussian with sigma = 2.
Invert (negate) the result and then set negative values to 0.
This is the output:
From here, it should be relatively straight-forward to identify the grid pattern.
I don't post code because I used MATLAB for this, but you can accomplish the same result in Python with OpenCV, here is a demo for applying the Laplacian of Gaussian in OpenCV.
This is Python + OpenCV code to replicate the above:
import cv2
color_image = cv2.imread("/Users/cris/Downloads/L3RVh.jpg")
img = cv2.cvtColor(color_image, cv2.COLOR_BGR2GRAY)
out = cv2.GaussianBlur(img, (0, 0), 2) # Note! Specify size of Gaussian by the sigma, not the kernel size
out = cv2.Laplacian(out, cv2.CV_32F)
_, out = cv2.threshold(-out, 0, 1e9, cv2.THRESH_TOZERO)
However, it looks like OpenCV doesn't linearize (apply gamma correction) when converting from BGR to gray, as the conversion function does that I used when creating the image above. I think this gamma correction might have improved the results a bit by reducing the response to the roof tiles.
I am trying to OCR the picture of documents and my current approach is
Read an image as a grayscale
Binarize it thresholding
Wrap perspective along the contours obtained from cv2.findContours()
The above works well if image is not shadowed. Now I want to get contours of shadowed pictures. My first attempt is to use cv2.adaptiveThreshold for step 2. The adaptive threshold successfully weakened the shadow but the resulted image lost the contrast between the paper and the background. That made cv2 impossible to find contours of the paper. So I need to use other method to remove the shadow.
Is there any way to remove shadow maintaining the background colour?
For reference here is the sample picture I am processing with various approaches. From left, I did
grayscale
thresholding
adaptive thresholdin
normalization
My goal is to obtain the second picture without shadow.
Please note that I actually have a temporary solution specifically to the picture which is to process the part of the picture with shadow separately. Yet, it is not the general solution to shadowed picture as its performance depends on the size, shape and position of a shadow so please use other methods.
This is the original picture.
Here is one way in Python/OpenCV using division normalization, optionally followed by sharpening and/or thresholding.
Input:
import cv2
import numpy as np
import skimage.filters as filters
# read the image
img = cv2.imread('receipt.jpg')
# convert to gray
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# blur
smooth = cv2.GaussianBlur(gray, (95,95), 0)
# divide gray by morphology image
division = cv2.divide(gray, smooth, scale=255)
# sharpen using unsharp masking
sharp = filters.unsharp_mask(division, radius=1.5, amount=1.5, multichannel=False, preserve_range=False)
sharp = (255*sharp).clip(0,255).astype(np.uint8)
# threshold
thresh = cv2.threshold(sharp, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
# save results
cv2.imwrite('receipt_division.png',division)
cv2.imwrite('receipt_division_sharp.png',sharp)
cv2.imwrite('receipt_division_thresh.png',thresh)
# show results
cv2.imshow('smooth', smooth)
cv2.imshow('division', division)
cv2.imshow('sharp', sharp)
cv2.imshow('thresh', thresh)
cv2.waitKey(0)
cv2.destroyAllWindows()
Division:
Sharpened:
Thresholded:
I am trying to do characters detection, have to draw a box around them, then crop and then feed to a neural network for recognition. Everything is working but before I was using sets of characters on a single color background image and segmentation was easily done.
However with real photos I have different lighting conditions and really struggle to find the contours.
After applying some adaptive thresholding I managed to get the folowing results, but starting from that I really can't figure how to properly proceed and detect each character. I can detect half of the characters easily, but not all of them. Probably because they are surrounded by lots of small irrelevant contours.
I have a feeling there is one step left but I can't figure which one.
Find Countours is capable of finding only about half of the characters.
For now, in short, im doing:
im_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
im_gray = cv2.GaussianBlur(im_gray, (5, 5), 0)
_, th1 = cv2.threshold(im_gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
cim, ctrs, hier = cv2.findContours(th1.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
and
th2 = cv2.adaptiveThreshold(im_gray,255,cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY,11,2)
Images below - original image and some variations of intermediate results.
Original picture:
After some thresholding:
After some thresholding:
Inverse thresholding:
So the question is - what is the step/steps after to segment the characters?
You can perform difference of gaussians. The idea is to blur the image with two different kernels and subtract their respective results:
Code:
im = cv2.imread(img, 0)
#--- it is better to take bigger kernel sizes to remove smaller edges ---
kernel1 = 15
kernel2 = 31
blur1 = cv2.GaussianBlur(im,(kernel1, kernel1), 0)
blur2 = cv2.GaussianBlur(im,(kernel2, kernel2), 0)
cv2.imshow('Difference of Gaussians',blur2 - blur1)
Result:
I am determining adaptive threshold using OTSU and then using determined threshold to convert image to black & white. On this processed image I want to carry out further steps of determining density of each circle, but my black & white image is over corrected and resulting in data loss. Any suggestions on how to tweak adaptive threshold.
im_gray = cv2.imread(img, cv2.CV_LOAD_IMAGE_GRAYSCALE)
(thresh, im_bw) = cv2.threshold(im_gray, 128, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)
img_bw = cv2.threshold(im_gray, thresh, 255, cv2.THRESH_BINARY)[1]
Try adaptiveThreshold() from openCV, It will calculate the threshold based on the intensities in the window. It seems that the OTSU method doesn’t work as expected in your case since adaptiveThreshold uses just an average (minus a constant) and works better (see image below) than OTSU that uses a more optimal criterion.
It is also not clear what is a spatial extent of the OTSU. If it is a whole image, then it should fail since the right side of the image is more blurred than the left and thus dark is washed out on the right. The adaptive threshold makes calculations in a window so it is locally adaptive. Note that the last two parameters in the function below are the size of the window and the value to subtract from the average when forming a threshold.
adaptiveThreshold(I, dst, 255, ADAPTIVE_THRESH_GAUSSIAN_C, THRESH_BINARY, 21, 15);
You may get better results when putting together OTSU and locally adaptive properties. However, the white color is typically oversampled and this causes a bias in the estimate. It is better to sample on the both sides of the gradient to get equal samples of white and dark. It is even better to take into account connectivity and color when thresholding, see grab cut;
Finally, the loss of information is always inevitable during thresholding.