I have implemented segmentation for given images, but images may vary based on different color. How can I separate background from the foreground where foreground contains hollow circle/filled circle only. My goal is to find threshold value automatically based on color of image.
[Sample images][1]
import numpy as np
import cv2
import os
image =cv2.imread("cropped2/pnr6.jpg")
img = image.copy()
MARKER_LOWER_BOUND = ( 0, 0, 0)
MARKER_UPPER_BOUND = (255, 255, 25)
marker_seg_mask = cv2.inRange(img, MARKER_LOWER_BOUND, MARKER_UPPER_BOUND)
cv2.imshow("thresold.jpg",marker_seg_mask)
Look at the cv2.adaptiveThreshold() function, which should do what you need.
From the docs:
Adaptive Thresholding
In the previous section, we used a global value as threshold value.
But it may not be good in all the conditions where image has different
lighting conditions in different areas. In that case, we go for
adaptive thresholding. In this, the algorithm calculate the threshold
for a small regions of the image. So we get different thresholds for
different regions of the same image and it gives us better results for
images with varying illumination.
Related
I'm having trouble preprocessing an image so that it's read correctly
The multicolor is messing up the OCR, not sure which preprocessing steps to take so that it interprets it correctly. I need it to work for multiple colors too, as I have more tables that have accents of blue, green, brown, etc...
Here is my code so far in jupyter notebook:
import pytesseract as tess
import cv2
import matplotlib.pyplot as plt
img = cv2.imread("table_img.png")
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img = cv2.threshold(img, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
plt.imshow(img, interpolation='nearest')
plt.show()
print(tess.image_to_string(img))
How can I preprocess the image so that it's read correctly?
That's not a simple question. There are some whole PhD dissertations about this.
But, well, in your case, if all your images are of this kind, an adaptative threshold (behaving differently when background is "greyish" and when background is purple)
OTSU is not at all convenient here. OTSU's postulate is that your image pixels are distributed in 2 colors, and that any variations around those 2 colors is just noise. And it tries to find the binarisation that both maximize the inter-class deviation and minimize the intra class deviation. In other words, find the threshold that separate the most the histogram of all pixel's value in 2 different peaks.
But that postulate is precisely what is not in your image. In your image, you have mainly 3 colors. Black, grayish, and purple. There is no way to claim that purple is just a form of gray with noise. Noise is not what explain the background difference.
So, what you need is adaptative background. That set the threshold locally.
Like this
img = cv2.adaptiveThreshold(img,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY,11,5)
Note, 11 is the dimension of the area in which average background color is computed for threshold computation. So, black pixels are the one that are lower than the mean value in that 11x11 area (it has to be odd, since it is centered around the pixel). And 5 is what is removed from mean to compute the threshold. It should be non-zero. Because if 0, then, in "white" area, threshold is the mean, and everything is exactly at threshold, not over, and appears black, or, with noise randomly black or white. In area with text, since mean is in between background and text, text is under threshold, and background over. That is why, on your image, with 0, you get essentialy black image, with some white zones containing black text.
So, that number should be non-zero. It should be big enough so that noise in white areas do not create too much black pixels (but your images don't seem noisy). And small enough so that what ever the background color is, difference between background and text is always way bigger that this 5.
I am new to deep learning but have succeeded in semantic segmentation of the image I am trying to get the pixel count of each class in the label. As an example in the image I want to get the pixel count of the carpet, or the chandelier or the light stand. How do I go about? Thanks any suggestions will help.
Edit: In what format the regions are returned? Do you have only the final image or the regions are given as contours? If you have them as contours (list of coordinates), you can apply findContourArea directly on that structure.
If you can receive/sample the regions one by one in an image (but do not have the contour), you can sequentially paint each of the colors/classes in a clear image, either convert it to grayscale or directly paint it in grayscale or binary, or binarize with threshold; then numberPixels = len(cv2.findNonZero(bwImage)). cv2.findContour and cv2.contourArea should do the same.
Instead of rendering each class in a separate image, if your program receives only the final segmentation and not per-class contours, you can filter/mask the regions by color ranges on that image. I built that and it seemed to do the job, 14861 pixels for the pink carpet:
import cv2
import numpy as np
# rgb 229, 0, 178 # the purple carpet in RGB (sampled with IrfanView)
# b,g,r = 178, 0, 229 # cv2 uses BGR
class_color = [178, 0, 229]
multiclassImage = cv2.imread("segmented.png")
cv2.imshow("MULTI", multiclassImage)
filteredImage = multiclassImage.copy()
low = np.array(class_color);
mask = cv2.inRange(filteredImage, low, low)
filteredImage[mask == 0] = [0, 0, 0]
filteredImage[mask != 0] = [255,255,255]
cv2.imshow("FILTER", filteredImage)
# numberPixelsFancier = len(cv2.findNonZero(filteredImage[...,0]))
# That also works and returns 14861 - without conversion, taking one color channel
bwImage = cv2.cvtColor(filteredImage, cv2.COLOR_BGR2GRAY)
cv2.imshow("BW", bwImage)
numberPixels = len(cv2.findNonZero(bwImage))
print(numberPixels)
cv2.waitKey(0)
If you don't have the values of the colors given or/and can't control them, you can use numpy.unique(): https://numpy.org/doc/stable/reference/generated/numpy.unique.html and it will return the unique colors, then they could be applied in the algorithm above.
Edit 2: BTW, another way to compute or verify such counts is by calculating histograms. That's with IrfanView on the black-white image:
So I am trying to make a neural network that categorizes resistor strength by recognizing the color bands. Before I get to that step I want to use OpenCV to threshold all the colors except the resistor bands so that it is easier for the neural network to categorize. However I do not know what threshold type is best suited for this.
I tried several ranges of HLS, RGB, and HSV, but they all do not get rid of the background of the resistor.
Note: I have already used contours to get rid of the background, so now all that is left is the resistor with the colored lines on it.
HLS in my case got rid of the colors, but kept the resistor background, as shown in the code below
frame_HLS = cv2.cvtColor(masked_data, cv2.COLOR_BGR2HLS)
frame_threshold = cv2.inRange(frame_HLS, (50, 0, 0), (139, 149, 255))
Here is an image of the original image, and the HLS output
So overall, I am just wondering if anyone knows if the other color modes like LUV work well for this, or whether or not I will just have to use contours or other methods to separate them.
You're on the right track and color thresholding is a great approach to segmenting the resistor. Currently, the thresholding is performing correctly, you just need to do a few simple steps to remove the background.
I tried several ranges of HLS, RGB, and HSV, but they all do not get rid of the background of the resistor.
To remove the background we can make use of the binary mask that cv2.inRange() generated. We simply use cv2.bitwise_and() and convert all black pixels on the mask to white with these two lines
result = cv2.bitwise_and(original, original, mask=frame_threshold)
result[frame_threshold==0] = (255,255,255)
Here's the masked image of what you currently have (left) and after removing the background (right)
import cv2
image = cv2.imread('1.png')
original = image.copy()
frame_HLS = cv2.cvtColor(image, cv2.COLOR_BGR2HLS)
frame_threshold = cv2.inRange(frame_HLS, (50, 0, 0), (139, 149, 255))
result = cv2.bitwise_and(original, original, mask=frame_threshold)
result[frame_threshold==0] = (255,255,255)
cv2.imshow('result', result)
cv2.waitKey()
However I do not know what threshold type is best suited for this.
Right now you're using color thresholding, you could continue using this method and experiment with other ranges in the HLS, RGB, or HSV color space. In all of these cases, you can remove the background by converting in all black pixels on the mask to white. If you decide to pivot to another thresholding method, take a look at Otsu's threshold or Adaptive thresholding which automatically calculates the threshold value.
I am trying to detect edges in images of a video, but edge detection methods such as canny does not work very well might be due to in similarity between boxes's color and floor color or brightness so I want to find a way to make all red and blue boxes look as white as possible, or may be the best way to detect edges as perfect as possible for every frame since that is the ultimate goal.
I recommend you using color tracking then.
Convert to HSV
cv2.bgr2hsv
Why hsv? eventhough the brightness change, u can still detect that color
Filtering
You can use cv2.inrange
Noise cancelling
Use cv2.Gaussianblur
Contouring
use cv2.findContours
Find the edge
use ur method
Repeat this step for every color of your box
Hope this help
Just to complete my comment in your question. One can use HSV/HLS colorspaces and use inRanges with the Hue channel. For example:
import numpy as np
import cv2
# load image and threshold it
original = cv2.imread("a.jpg")
hsvframe = cv2.cvtColor(original, cv2.COLOR_BGR2HLS)
mask = cv2.inRange(hsvframe, (160,40,40), (180, 255, 255))
mask = mask + cv2.inRange(hsvframe, (0,40,40), (12, 255, 255)) # color red is at the beginning and end of the hue wheel
original[mask==255] = (0,255,0)
cv2.imshow("image", original)
cv2.waitKey(0)
cv2.destroyAllWindows()
Things to remember, Hue goes from 0-180 in np.uint8. This means if you need hue 300-360 the limits will be 150-180. The other two values are 0-255 where 255 = 100%.
The result of this small code is:
It is not perfect, but one can refine it using the methods suggested by the other answer. I hope this helps.
I am working on an image processing feature extraction. I have a photo of a bird in which I have to extract bird area and tell what color the bird has. I used canny feature extraction method to get the edges of a bird.
How to extract only bird area and make the background to blue color?
openCv solution should also be fine.
import skimage
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
import os
filename = os.path.join(os.getcwd(),'image\image_bird.jpeg')
from skimage import io
bird =io.imread(filename,as_grey=True)
plt.imshow(bird)
from skimage import feature
edges = feature.canny(bird,sigma=1)
plt.imshow(edges )
Actual bird image can be taken from bird link
Identify the edges of your image
Binarize the image via automatic thresholding
Use contour detection to identify black regions which are inside a white region and merge them with the white region. (Mockup, image may slightly vary)
Use the created image as mask to color the background and color it
This can be done by simply setting each background pixel (black) to its respective color.
As you can see, the approach is far from perfect, but should give you a general idea about how to accomplish your task. The final image quality might be improved by slightly eroding the map to tighten it to the contours of the bird. You then also use the mask to calculate your color histogram by only taking foreground pixels into account.
Edit: Look here:
Eroded mask
Final image
According to this article https://www.pyimagesearch.com/2016/04/11/finding-extreme-points-in-contours-with-opencv/
and this question CV - Extract differences between two images
I wrote some python code as below. As my predecessor said it is also far from perfect. The main disadvantages of this code are constants value to set manually: minThres (50), maxThres(100), dilate iteration count and erode iteration count.
import cv2
import numpy as np
windowName = "Edges"
pictureRaw = cv2.imread("bird.jpg")
## set to gray
pictureGray = cv2.cvtColor(pictureRaw, cv2.COLOR_BGR2GRAY)
## blur
pictureGaussian = cv2.GaussianBlur(pictureGray, (7,7), 0)
## canny edge detector - you must specify threshold values
pictureCanny = cv2.Canny(pictureGaussian, 50, 100)
## perform a series of erosions + dilations to remove any small regions of noise
pictureDilate = cv2.dilate(pictureCanny, None, iterations=20)
pictureErode = cv2.erode(pictureDilate, None, iterations=5)
## find the nozero regions in the erode
imask2 = pictureErode>0
## create a Mat like pictureRaw
canvas = np.full_like(pictureRaw, np.array([255,0,0]), dtype=np.uint8)
## set mask
canvas[imask2] = pictureRaw[imask2]
cv2.imwrite("result.png", canvas)