So I have images of a bunch of straight lines forming various shapes. However these straight line have a tendency to not be quite straight because the underlying source often has these lines go between pixels. In that case the underlying source produces pixels that are next to a line. In that case I would like for these extra pixels to be removed.
Here we have a source image:
And here we have the same image denoised:
.
I'm messing around with houghline transforms or de-noising algorithms but none of these are working well and it feels like there should be a good way of fixing this that uses the specific fact that these are lines rather than just normal pepper and salt noise.
I'm working with python right now but other language answers are acceptable.
I thought of the same solution as commented by Cris:
Convert the image to binary image.
Apply morphological opening with np.ones((3, 15)) kernel - keeping only horizontal lines.
Apply morphological opening with np.ones((15, 3)) kernel - keeping only vertical lines.
Apply bitwise or between the above matrices to form a mask.
Apply mask on image - zero pixels in the mask gets zero values.
Here is a code sample:
import numpy as np
import cv2
img = cv2.imread('input.png') # Read input image (BGR color format)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Convert to grayscale
thresh = cv2.threshold(gray, 1, 255, cv2.THRESH_BINARY)[1] # Convert to binary (only zeros and 255)
horz = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, np.ones((3, 15))) # Keep only horizontal lines
vert = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, np.ones((15, 3))) # Keep only vertical lines
mask = cv2.bitwise_or(horz, vert) # Unite horizontal and vertical lines to form a mask
res = cv2.bitwise_and(img, img, mask=mask) # Place zeros where mask is zero
# Show the result
cv2.imshow('mask', mask)
cv2.imshow('res', res)
cv2.waitKey()
cv2.destroyAllWindows()
Result:
mask:
res:
The result is not perfect, and not generalized.
You may get better result using for loops.
Example: Delete pixels that has many horizontal pixels below them, but only few next to them (repeat for left, right, and bottom).
Related
Take a look at this image. I want to turn this card blue.
I use Python and OpenCV to perform image processing.
Here's how I do it now:
import cv2
import numpy as np
# Load the image
image = cv2.imread("input.jpg")
# Convert the image to the HSV color space
hsv_image = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
# Threshold the HSV image to get only the red colors
# Bitwise OR unites Hue value 170-179 and 0-10.
mask = cv2.bitwise_or(
cv2.inRange(hsv_image, np.array([0, 120, 100]), np.array([10, 255, 255])),
cv2.inRange(hsv_image, np.array([170, 120, 100]), np.array([180, 255, 255]))
)
# Perform median blurring
mask = cv2.medianBlur(mask, 5)
# Define a kernel for the morphological operations
kernel = np.ones((5, 5), np.uint8)
# Perform an opening operation to remove small objects
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel, iterations=2)
# Perform a closing operation to fill small holes
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel, iterations=2)
# Perform a gradient operation to extract object boundaries
gradient_mask = cv2.morphologyEx(mask, cv2.MORPH_GRADIENT, kernel)
# Modify hue value of every masked pixel
hsv_image[:, :, 0][mask != 0] = (hsv_image[:, :, 0][mask != 0].astype(int, copy=False) + 120) % 180
# Convert the HSV image back to BGR color space
result = cv2.cvtColor(hsv_image, cv2.COLOR_HSV2BGR)
# Display the result
cv2.namedWindow("output", cv2.WINDOW_NORMAL)
cv2.imshow("output", result)
# Save images of the mask, result and gradient
cv2.imwrite('mask.jpg', mask)
cv2.imwrite('result.jpg', result)
cv2.imwrite('gradient.jpg', gradient_mask)
# Wait for the window to close
while cv2.getWindowProperty('output', 0) >= 0:
cv2.waitKey(50)
cv2.destroyAllWindows()
It works well. The result, The filtering mask
But if you take a closer look you'll see the problem: link, link
Some red pixels is still here, and here's why. They do not fall in the filtered range of the red color: 170 <= Hue <= 10, Saturation >= 120, Value >= 100. Their HSV color is near to (178, 32, 60). So, the saturation and value fall out of the filter range.
Why I can't lower the range of saturation and value? That's because in this case there would be too much noise on another background that has more colors. The noise in this case is hard to avoid even using multiple iterations of opening morphological operation.
I don't have much experience in image processing and using OpenCV, so my ideas may be far from the best solution. It's okay if you propose another approach.
Possible solution. Would it be possible to perform the dilate morphological operation on the filtering mask (expand the filtering mask) but only to those pixels that fall in another, broader range of red (with saturation and value range equal to 10, hue range stays the same). So that all the red pixels that fall in the broader range of the red color AND that are adjacent to the pixels of the existing mask (so no pixels from the background is added creating noise).
If that is a good idea, how can I implement it, especially the part of dilating only to pixels that fall in the broader range of the red color? Maybe it's already implemented in OpenCV and I just don't know about that?
Also I would be glad to hear any suggestions or recommendations. I am a student, I want to learn more.
Thanks in advance!
I found myself how to do it, but it's still not the best mask.
I create a broader mask as follows:
broader_mask = cv2.bitwise_or(
cv2.inRange(hsv_image, np.array([0, 30, 30]), np.array([20, 255, 255])),
cv2.inRange(hsv_image, np.array([160, 30, 30]), np.array([180, 255, 255]))
)
And apply bitwise AND on the dilated primary mask and the broader mask to get the resulting mask:
mask = cv2.bitwise_and(cv2.dilate(mask, kernel, iterations=2), broader_mask)
It works much better, but now the mask might be bigger than I want in some cases. For example:
It all depends on the kernel and the number of iterations of the dilation operation. I don't think it's the best solution because the kernel and the number of iterations might be different for different image sizes, and I'm still looking for a better solution.
I have these images
enter image description here
enter image description here
I want to remove noise from these images so I can convert them into text using pytesseract. The noise is only in blue colour so I tried to remove blue from the image. Still not good results.
This is what I did
import cv2
import pytesseract
# Extract the blue channel
blue = img[:, :, 0]
# Apply thresholding to the blue channel
thresh = cv2.threshold(blue, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
# Perform morphological operations to remove noise
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,1))
opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel, iterations=7)
# Apply blur to smooth out the image
blur = opening#cv2.medianBlur(opening, 1)
cv2.imwrite("/Users/arjunmalik/Desktop/blur.png",blur)
display("/Users/arjunmalik/Desktop/blur.png")
The result was
enter image description here
The OCR results were FL1S4y.
As stated by Sembei, You need to use a closing operator which's a must for a situation like this because you want to close black points on the object to improve the image quality.
Solution:
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (4,4))
closing = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel, iterations=1)
You can modify your code to this one to achieve the following output for the second image.
Output:
Result
You might need to change the size of the kernel for different input images.
Thoughts:
I think it'd be better if you do the character segmentation first before applying the closing operator in order to achieve the finest results.
I am planning to split the questions from this PDF document. The challenge is that the questions are not orderly spaced. For example the first question occupies an entire page, second also the same while the third and fourth together make up one page. If I have to manually slice it, it will be ages. So, I thought to split it up into images and work on them. Is there a possibility to take image as this
and split into individual components like this?
This is a classic situation for dilate. The idea is that adjacent text corresponds with the same question while text that is farther away is part of another question. Whenever you want to connect multiple items together, you can dilate them to join adjacent contours into a single contour. Here's a simple approach:
Obtain binary image. Load the image, convert to grayscale, Gaussian blur, then Otsu's threshold to obtain a binary image.
Remove small noise and artifacts. We create a rectangular kernel and morph open to remove small noise and artifacts in the image.
Connect adjacent words together. We create a larger rectangular kernel and dilate to merge individual contours together.
Detect questions. From here we find contours, sort contours from top-to-bottom using imutils.sort_contours(), filter with a minimum contour area, obtain the rectangular bounding rectangle coordinates and highlight the rectangular contours. We then crop each question using Numpy slicing and save the ROI image.
Otsu's threshold to obtain a binary image
Here's where the interesting section happens. We assume that adjacent text/characters are part of the same question so we merge individual words into a single contour. A question is a section of words that are close together so we dilate to connect them all together.
Individual questions highlighted in green
Top question
Bottom question
Saved ROI questions (assumption is from top-to-bottom)
Code
import cv2
from imutils import contours
# Load image, grayscale, Gaussian blur, Otsu's threshold
image = cv2.imread('1.png')
original = image.copy()
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (7,7), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Remove small artifacts and noise with morph open
open_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5,5))
opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, open_kernel, iterations=1)
# Create rectangular structuring element and dilate
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (9,9))
dilate = cv2.dilate(opening, kernel, iterations=4)
# Find contours, sort from top to bottom, and extract each question
cnts = cv2.findContours(dilate, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
(cnts, _) = contours.sort_contours(cnts, method="top-to-bottom")
# Get bounding box of each question, crop ROI, and save
question_number = 0
for c in cnts:
# Filter by area to ensure its not noise
area = cv2.contourArea(c)
if area > 150:
x,y,w,h = cv2.boundingRect(c)
cv2.rectangle(image, (x, y), (x + w, y + h), (36,255,12), 2)
question = original[y:y+h, x:x+w]
cv2.imwrite('question_{}.png'.format(question_number), question)
question_number += 1
cv2.imshow('thresh', thresh)
cv2.imshow('dilate', dilate)
cv2.imshow('image', image)
cv2.waitKey()
We may solve it using (mostly) morphological operations:
Read the input image as grayscale.
Apply thresholding with inversion.
Automatic thresholding using cv2.THRESH_OTSU is working well.
Apply opening morphological operation for removing small artifacts (using the kernel np.ones(1, 3))
Dilate horizontally with very long horizontal kernel - make horizontal lines out of the text lines.
Apply closing vertically - create two large clusters.
The size of the vertical kernel should be tuned according to the typical gap.
Finding connected components with statistics.
Iterate the connected components and crop the relevant area in the vertical direction.
Complete code sample:
import cv2
import numpy as np
img = cv2.imread('scanned_image.png', cv2.IMREAD_GRAYSCALE) # Read image as grayscale
thesh = cv2.threshold(img, 0, 255, cv2.THRESH_OTSU + cv2.THRESH_BINARY_INV)[1] # Apply automatic thresholding with inversion.
thesh = cv2.morphologyEx(thesh, cv2.MORPH_OPEN, np.ones((1, 3), np.uint8)) # Apply opening morphological operation for removing small artifacts.
thesh = cv2.dilate(thesh, np.ones((1, img.shape[1]), np.uint8)) # Dilate horizontally - make horizontally lines out of the text.
thesh = cv2.morphologyEx(thesh, cv2.MORPH_CLOSE, np.ones((50, 1), np.uint8)) # Apply closing vertically - create two large clusters
nlabel, labels, stats, centroids = cv2.connectedComponentsWithStats(thesh, 4) # Finding connected components with statistics
parts_list = []
# Iterate connected components:
for i in range(1, nlabel):
top = int(stats[i, cv2.CC_STAT_TOP]) # Get most top y coordinate of the connected component
height = int(stats[i, cv2.CC_STAT_HEIGHT]) # Get the height of the connected component
roi = img[top-5:top+height+5, :] # Crop the relevant part of the image (add 5 extra rows from top and bottom).
parts_list.append(roi.copy()) # Add the cropped area to a list
cv2.imwrite(f'part{i}.png', roi) # Save the image part for testing
cv2.imshow(f'part{i}', roi) # Show part for testing
# Show image and thesh testing
cv2.imshow('img', img)
cv2.imshow('thesh', thesh)
cv2.waitKey()
cv2.destroyAllWindows()
Results:
Stage 1:
Stage 2:
Stage 3:
Stage 4:
Top area:
Bottom area:
I'm very new to OpenCV and recently, I'm trying to compare two images of rails, one with a train and one without. After the comparison, I apply a threshold, and there are some 'holes' in the white regions which I do not want. Currently, I am using dilation with 4 iterations and kernel set to "None", which defaults to a 3x3 by my understanding.
How do I decide what sort of kernel to use so that the dilation does a better job at making the white region continuous? Would also be nice if I could remove the small white blobs in the background. Here is the code:
resized = imutils.resize(img2, width=1050)
resized2 = imutils.resize(img3, width=1050)
grayA = cv2.cvtColor(resized, cv2.COLOR_BGR2GRAY)
grayB = cv2.cvtColor(resized2, cv2.COLOR_BGR2GRAY)
grayA = cv2.GaussianBlur(grayA,(7,7),0)
grayB = cv2.GaussianBlur(grayB,(7,7),0)
frameDelta = cv2.absdiff(grayA, grayB)
thresh = cv2.threshold(frameDelta, 20, 255, cv2.THRESH_BINARY)[1]
thresh = cv2.dilate(thresh, None, iterations=4)
Complete beginner in this, so even general tips/advice to improve these comparisons would be vastly appreciated!
Perhaps this will give you some idea about morphology in Python/OpenCV. First I use a square "open" kernel about the size of the small white spots to remove them. Then I use a horizontal rectangle "close" kernel about the size of the black gap to fill it. "Open" removes white regions (or fills black gaps) and close removes black regions (or fills white gaps)
Input:
import cv2
import numpy as np
# read image as grayscale
img = cv2.imread('blob3.png', cv2.IMREAD_GRAYSCALE)
# threshold to binary
thresh = cv2.threshold(img, 0, 255, cv2.THRESH_BINARY)[1]
# apply morphology open with square kernel to remove small white spots
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (19,19))
morph1 = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel)
# apply morphology close with horizontal rectangle kernel to fill horizontal gap
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (101,1))
morph2 = cv2.morphologyEx(morph1, cv2.MORPH_CLOSE, kernel)
# write results
cv2.imwrite("blob3_morph1.png", morph1)
cv2.imwrite("blob3_morph2.png", morph2)
# show results
cv2.imshow("thresh", thresh)
cv2.imshow("morph1", morph1)
cv2.imshow("morph2", morph2)
cv2.waitKey(0)
Morphology Square Open:
Morphology Rectangle Close:
Alternate Morphology Square Close:
I'm working in a script using different OpenCV operations for processing an image with solar panels in a house roof. My original image is the following:
After processing the image, I get the edges of the panels as follows:
It can be seen how some rectangles are broken due to reflection of the Sun in the picture.
I would like to know if it's possible to fix those broken rectangles, maybe by using the pattern of those which are not broken.
My code is the following:
# Load image
color_image = cv2.imread("google6.jpg")
cv2.imshow("Original", color_image)
# Convert to gray
img = cv2.cvtColor(color_image, cv2.COLOR_BGR2GRAY)
# Apply various filters
img = cv2.GaussianBlur(img, (5, 5), 0)
img = cv2.medianBlur(img, 5)
img = img & 0x88 # 0x88
img = cv2.fastNlMeansDenoising(img, h=10)
# Invert to binary
ret, thresh = cv2.threshold(img, 127, 255, 1)
# Perform morphological erosion
kernel = np.ones((5, 5),np.uint8)
erosion = cv2.morphologyEx(thresh, cv2.MORPH_ERODE, kernel, iterations=2)
# Invert image and blur it
ret, thresh1 = cv2.threshold(erosion, 127, 255, 1)
blur = cv2.blur(thresh1, (10, 10))
# Perform another threshold on blurred image to get the central portion of the edge
ret, thresh2 = cv2.threshold(blur, 145, 255, 0)
# Perform morphological erosion to thin the edge by ellipse structuring element
kernel1 = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))
contour = cv2.morphologyEx(thresh2, cv2.MORPH_ERODE, kernel1, iterations=2)
# Get edges
final = cv2.Canny(contour, 249, 250)
cv2.imshow("final", final)
I have tried to modify all the filters I'm using in order to reduce as much as possible the effect of the Sun in the original picture, but that is as far as I have been able to go.
I'm in general happy with the result of all those filters (although any advice is welcome), so I'd like to work on the black/white imaged I showed, which is already smooth enough for the post-processing I need to do.
Thansk!
The pattern is not broken in the original image, so it being broken in your binarized result must mean your binarization is not optimal.
You apply threshold() to binarize the image, and then Canny() to the binary image. The problems here are:
Thresholding removes a lot of information, this should always be the last step of any processing pipeline. Anything you lose here, you've lost for good.
Canny() should be applied to a gray-scale image, not a binary image.
The Canny edge detector is an edge detector, but you want to detect lines, not edges. See here for the difference.
So, I suggest starting from scratch.
The Laplacian of Gaussian is a very simple line detector. I took these steps:
Read in image, convert to grayscale.
Apply Laplacian of Gaussian with sigma = 2.
Invert (negate) the result and then set negative values to 0.
This is the output:
From here, it should be relatively straight-forward to identify the grid pattern.
I don't post code because I used MATLAB for this, but you can accomplish the same result in Python with OpenCV, here is a demo for applying the Laplacian of Gaussian in OpenCV.
This is Python + OpenCV code to replicate the above:
import cv2
color_image = cv2.imread("/Users/cris/Downloads/L3RVh.jpg")
img = cv2.cvtColor(color_image, cv2.COLOR_BGR2GRAY)
out = cv2.GaussianBlur(img, (0, 0), 2) # Note! Specify size of Gaussian by the sigma, not the kernel size
out = cv2.Laplacian(out, cv2.CV_32F)
_, out = cv2.threshold(-out, 0, 1e9, cv2.THRESH_TOZERO)
However, it looks like OpenCV doesn't linearize (apply gamma correction) when converting from BGR to gray, as the conversion function does that I used when creating the image above. I think this gamma correction might have improved the results a bit by reducing the response to the roof tiles.