We Keep Coding

Python OpenCV, check if thresholded image is mostly white or black

I have a few images, I converted them to grayscale and thresholded them by the usual steps:
image = cv2.imread('img_path')
image_gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
ret, frame_thresh = cv2.threshold(image_gray,128,255,cv2.THRESH_BINARY)
How can I check if the resulting image is mostly black or white? I tried using an histogram:
hist = cv2.calcHist([frame_thresh], [0], None, [2], [0, 1])
hist_ok /= hist_ok.sum()
but I tested it on two images, one of kind (one is mostly black, the other mostly white)
and the resulting hist is always the same
print(hist)
>> [[1.]
[0.]]

You're looking for the average pixel value of the image, to do that you can use numpy:
import numpy as np
avg_color_per_row = np.average(frame_thresh, axis=0)
avg_color = np.average(avg_color_per_row, axis=0)
it outputs a value that ranges from 0 to 255.

After thresholding your image contains only black (0) and white (255) pixels. You can use countNonZero function to calculate the number of non-zero pixels (not black) and compare it with image size.
Alternatively you can just calculate mean value of an image. If its lower than 127 (half of the range) image is mostly black.

How do I crop an image using a binary mask image of the same picture to remove the background in python?

I have tried to get the edge of the mask image with the following code:
import numpy as np
from matplotlib import pyplot as plt
img = cv2.imread('ISIC_0000000_segmentation.png',0)
edges = cv2.Canny(img,0,255)
plt.subplot(121), plt.imshow(img, cmap='gray')
plt.title('Original Image'), plt.xticks([]), plt.yticks([])
plt.subplot(122), plt.imshow(edges, cmap='gray')
plt.title('Edge Image'), plt.xticks([]), plt.yticks([])
plt.show
What I get is this:
But the edge isn't smooth for some reason.
My plan was to use the edge image to crop the following picture:
Does anyone know how I could make the edge image better and how I could use this to crop the normal image?
EDIT: #Mark Setchell made a good point: If i could use the mask image directly to crop the image that would be great.
Also: It is maybe possible to lay the normal image precisely on the mask image so that the black area on the mask would cover the blue-ish area on the normal picture.
EDIT: #Mark Setchell introduced the idea of multiplying the normale image with the mask image so what the background would result in 0(black) and the rest would keep its color. Would it be a problem when my mask image is .png and my normal picture is .jpg when multiplying?
EDIT:
I have written the following code to try to multiply two pictures:
# Importing Image and ImageChops module from PIL package
from PIL import Image, ImageChops
# creating a image1 object
im1 = Image.open("ISIC_0000000.jpg")
# creating a image2 object
im2 = Image.open("ISIC_0000000_segmentation.png")
# applying multiply method
im3 = ImageChops.multiply(im1, im2)
im3.show()
But I get the error:
ValueError: images do not match
Does anyone know how I could solve this?

If I understand correctly, you want to extract the object and remove the background. To do this, you can just do a simple cv2.bitwise_and() with the mask and the original input image.
Does anyone know how I could make the edge image better and how I could use this to crop the normal image?
To extract the background from the image, you don't need an edge image, the thresholded image can be used to remove only the desired parts of the image. You can use the mask image to directly drop the image and remove the background. Other approaches of obtaining a binary mask include using a fixed threshold value, adaptive threshold, or Canny edge detection. Here's a simple example using Otsu's threshold to obtain a binary mask followed by a bitwise-and operation.
Here's the result with the removed background
You can also turn all pixels on the mask to white if you wanted the removed background to be white
Note: Depending on how "smooth" you want the result, you can apply any blur to the image before thresholding to smooth out the edges. This can include averaging, Gaussian, median, or bilaterial filtering.
Code
import cv2
# Load image, grayscale, Otsu's threshold
image = cv2.imread('1.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Remove background using bitwise-and operation
result = cv2.bitwise_and(image, image, mask=thresh)
result[thresh==0] = [255,255,255] # Turn background white
cv2.imshow('thresh', thresh)
cv2.imshow('result', result)
cv2.waitKey()

The detected edge isn't smooth because the actual edge in the image isn't smooth. You can try filtering the original image first with low-pass filters.
If you can use contours, the following will work:
import numpy as np
import cv2
from matplotlib import pyplot as plt
# Read in image
imgRaw = cv2.imread('./Misc/edgesImg.jpg',0)
# Blur image
blurSize = 25
blurredImg = cv2.blur(imgRaw,(blurSize,blurSize))
# Convert to Binary
thrImgRaw, binImgRaw = cv2.threshold(imgRaw, 0, 255, cv2.THRESH_OTSU)
thrImgBlur, binImgBlur = cv2.threshold(blurredImg, 0, 255, cv2.THRESH_OTSU)
# Detect the contours in the image
contoursRaw = cv2.findContours(binImgRaw,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
contoursBlur = cv2.findContours(binImgBlur,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
# Draw all the contours
contourImgOverRaw = cv2.drawContours(imgRaw, contoursRaw[0], -1, (0,255,0),5)
contourImgOverBlur = cv2.drawContours(blurredImg, contoursBlur[0], -1, (0,255,0),5)
# Plotting
plt.figure()
plt.subplot(121)
plt.imshow(contourImgOverRaw)
plt.title('Raw Edges'), plt.xticks([]), plt.yticks([])
plt.subplot(122)
plt.imshow(contourImgOverBlur)
plt.title('Edges with {}px Blur'.format(blurSize)), plt.xticks([]), plt.yticks([])
plt.show()
EDIT: Here's more info on getting a mask of an image from contours.

You can your morphological operations to get the edge.
Sorry for using MATLAB:
I = imbinarize(rgb2gray(imread('I.png'))); %Load input image, and convert to binary image.
%Erode the image with mask 3x3
J = imerode(I, ones(3));
%Pefrom XOR operation (1 xor 1 = 0, 0 xor 0 = 0, 0 xor 1 = 1, 1 xor 0 = 1)
J = xor(I, J);
%Use "skeleton" operation to make sure eage thikness is 1 pixel.
K = bwskel(J);
Result:
As Mark mentioned, you don't need the edges for cropping (unless your are using special cropping method that I am not aware of).

Numpy where() creates spots of different color than the ones defined

I'm writing a script that creates a mask for an image. My input image looks like this:
The original image is only 40x40px, here it is for reference:
I want to create a mask of the purple area in the center of the image. This is what I do:
# read the 40x40 image and convert it to RGB
input_image = cv2.cvtColor(cv2.imread('image.png'), cv2.COLOR_BGR2RGB)
# get the value of the color in the center of the image
center_color = input_image[20, 20]
# create the mask: pixels with same color = 255 (white), other pixels = 0 (black)
mask_bw = np.where(input_image == center_color, 255, 0)
# show the image
plt.imshow(mask_bw)
Most of the time this works perfectly fine, but for some images (like the one I attached to this question) I consistently get some blue areas in my mask like on the image below. This is reproducible and the areas are always the same for the same input images.
This is already weird enough, but if I try to remove the blue areas, this doesn't work either.
mask_bw[mask_bw != (255, 255, 255)] = 0 # this doesn't change anything..
Why is this happening and how do I fix this?
Additional info
tried with numpy version 1.17.3 and 1.17.4
Reproduced in my local environment and in a google colab notebook

The main problem is that you're trying to compare three channels but only setting the value for one channel. This is most likely causing the blue areas on the mask. When you use np.where() to set the other pixels to black, you are only setting this on the 1st channel instead of all three channels. You can visualize this by splitting each channel and printing the before/after arrays which will show you that the resulting array values are RGB(0,0,255). So to fix this problem, we need to compare each individual channel then set the desired area in white while setting any black areas on the mask to black for all three channels. Here is one way to do it:
import numpy as np
import cv2
image = cv2.imread('1.png')
center_color = image[20, 20]
b, g, r = cv2.split(image)
mask = (b == center_color[0]) & (g == center_color[1]) & (r == center_color[2])
image[mask] = 255
image[mask==0] = 0
cv2.imshow('image', image)
cv2.waitKey()
A hotfix to remove the blue areas using your current code would be to convert the image to grayscale (1-channel) then change all non-white pixels to black.
import numpy as np
import cv2
# Load image, find color, create mask
image = cv2.imread('1.png')
center_color = image[20, 20]
mask = np.where(image == center_color, 255, 0)
mask = np.array(mask, dtype=np.uint8)
# Convert image to grayscale, convert all non-white pixels to black
mask = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
mask[mask != 255] = 0
cv2.imshow('mask', mask)
cv2.waitKey()
Here are two alternative methods to obtain a mask of the purple area
Method #1: Work in grayscale space
import numpy as np
import cv2
image = cv2.imread('1.png')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
center_color = gray[20, 20]
mask = np.array(np.where(gray == center_color, 255, 0), dtype=np.uint8)
cv2.imshow('mask', mask)
cv2.waitKey()
Method #2: Color thresholding
The idea is to convert the image to HSV color space then use a lower and upper color range to segment the image to create a binary mask
import numpy as np
import cv2
image = cv2.imread('1.png')
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
lower = np.array([0, 124, 0])
upper = np.array([179, 255, 255])
mask = cv2.inRange(hsv, lower, upper)
cv2.imshow('mask', mask)
cv2.waitKey()
Both methods should yield the same result

If you have a 3-channel image (i.e. RGB or BGR or somesuch) and you want to generate a single channel mask (i.e. you want 0/1 or True/False) for each pixel, then you effectively need to group the 3 values into a single using np.all() like this:
import cv2
import numpy as np
# Load image and get centre colour
image = cv2.imread('40x40.png')
cc = im[20, 20]
print(image.shape)
(40, 40, 3)
# Generate list of unique colours present in image so we know what we are dealing with
print(np.unique(im.reshape(-1,3), axis=0))
array([[140, 109, 142],
[151, 106, 140],
[160, 101, 137],
[165, 134, 157],
[175, 149, 171],
[206, 87, 109],
[206, 185, 193]], dtype=uint8)
# Generate mask of pixels matching centre colour
mask_bw = np.where(np.all(im==cc,axis=2), 255, 0)
# Check shape of mask - no 3rd dimension !!!
print(mask_bw.shape)
(40, 40)
# Check unique colours in mask
print(np.unique(mask_bw.reshape(-1,1), axis=0))
array([[ 0],
[255]])

Apply colored overlay over a colored rgb image with some transparency in Skimage

So basically I have a colored RGB image and I want to add a colored overlay over the RGB image without converting it to gray level.
For example if I have a colored image(RGB). And I want to add a transparent blue color over the index like this
img[200:350, 200:350] = [0, 0, 1] # Blue block
This question is a sibling question to this one:
Applying a coloured overlay to an image in either PIL or Imagemagik
Difference is the color space. The above question is for gray level images rather colored (RGB).
from skimage import io, data
import numpy as np
img = data.astronaut()
Please use the above code to answer.

Here is the code in OpenCV:
import cv2
# load the image
image = cv2.imread("2.jpg")
# I resized the images because they were to big
image = cv2.resize(image, (0,0), fx=0.75, fy=0.75)
overlay = image.copy()
output = image.copy()
#select the region that has to be overlaid
cv2.rectangle(overlay, (420, 205), (595, 385),(0, 255, 255), -1)
#Adding the transparency parameter
alpha = 1
#Performing image overlay
cv2.addWeighted(overlay, alpha, output, 1 - alpha,0, output)
#Save the overlaid image
cv2.imwrite('Output'+str(alpha) +'.jpg', output)
cv2.waitKey(0)
cv2.destroyAllWindows()
Some results:
when alpha = 0.1
when alpha = 0.5
when alpha = 0.8
when alpha = 1.0 (the overlay is no longer transparent but opaque)

Python + OpenCV color segmentation using Kmeans

I am trying to apply the kmeans from opencv in order to segment the image in HSV color space.
def leftOffset(src, p_countours):
height, width, size = src.shape
p_width = width/p_countours
o_left = src[0:height, 0:p_width]
HSV_img = cv2.cvtColor(o_left, cv2.COLOR_BGR2HSV)
hue = HSV_img[0]
hue = np.float32(HSV_img)
# Define criteria = ( type, max_iter = 10 , epsilon = 1.0 )
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0)
# Set flags (Just to avoid line break in the code)
flags = cv2.KMEANS_RANDOM_CENTERS
# Apply KMeans
compactness,labels,centers = cv2.kmeans(hue,2,criteria,10,flags)
centers = np.uint8(centers)
res = centers[labels.flatten()]
res2 = res.reshape((hue.shape))
cv2.imshow("o_left", hue)
cv2.waitKey(0)
I am now able to apply the kmeans algorithm to the HSVImage[0] with K=2, and how can I get a image like threshold according to the result?
Thanks
To clarify the question:
I have color-based captchas, and I want to segment each digits.
The image is like
I am going to use k-means method to find out the dominant color and segment the digits inside.

1) If all you need is to find the dominant color why not find the histograms of each color channel? Find the dominant channel then segment only that channel using otsu? For example if I threshold only the hue I can get nice results. K-means could be an overkill for this task:
import cv2
import numpy as np
import matplotlib.pylab as plt
## Simple Otsu over hue
six = cv2.imread('7zovC.jpg')
##convert to hsv
hsv = cv2.cvtColor(six, cv2.COLOR_BGR2HSV)
hue = hsv[:, :, 0]
binary_img = cv2.threshold(hue, 128, 255, cv2.THRESH_OTSU)
plt.figure()
plt.imshow(binary_img*255)
plt.show()
2) Why not use all channels for clustering instead of just hue? What you need is clustering -> color quantization this link should be useful. This is for opencv version > 3.0.0
Note for python 2.4.11, cv2.kmeans has a slightly difference interface and you could use this instead:
def color_quantize(img, K):
Z = img.reshape((-1, 3))
# convert to np.float32
Z = np.float32(Z)
# define criteria, number of clusters(K) and apply kmeans()
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0)
ret, label, center = cv2.kmeans(Z, 2, criteria, 10, cv2.KMEANS_RANDOM_CENTERS)
# Now convert back into uint8, and make original image
center = np.uint8(center)
res = center[label.flatten()]
quantized_img = res.reshape((img.shape))
label_img = label.reshape((img.shape[:2]))
return label_img, quantized_img
six = cv2.imread('7zovC.jpg')
##convert to hsv
hsv = cv2.cvtColor(six, cv2.COLOR_BGR2HSV)
K = 2
label_img, six_q = color_quantize(hsv, K)
plt.figure()
plt.imshow(label_img)
plt.show()
My results for color quantization were not impressive.

May I suggest a conventional alternative? I you get rid of the very dark and bright regions first, you may be able to simply rely on the most frequent value of the hue component calculated from the histogram.
Mind that the borders of the numbers will never be absolutely exact, since colours are similar in the surrounding.
Furthrmore, you could select the maximum blob only (according to size) to suppress remaining small blobs outside.
Results:
Code:
import cv2
import numpy as np
from matplotlib import pyplot as plt
img = cv2.imread('image1.jpg')
#get rid of very bright and very dark regions
delta=30
lower_gray = np.array([delta, delta,delta])
upper_gray = np.array([255-delta,255-delta,255-delta])
# Threshold the image to get only selected
mask = cv2.inRange(img, lower_gray, upper_gray)
# Bitwise-AND mask and original image
res = cv2.bitwise_and(img,img, mask= mask)
#Convert to HSV space
HSV_img = cv2.cvtColor(res, cv2.COLOR_BGR2HSV)
hue = HSV_img[:, :, 0]
#select maximum value of H component from histogram
hist = cv2.calcHist([hue],[0],None,[256],[0,256])
hist= hist[1:, :] #suppress black value
elem = np.argmax(hist)
print np.max(hist), np.argmax(hist)
tolerance=10
lower_gray = np.array([elem-tolerance, 0,0])
upper_gray = np.array([elem+tolerance,255,255])
# Threshold the image to get only selected
mask = cv2.inRange(HSV_img, lower_gray, upper_gray)
# Bitwise-AND mask and original image
res2 = cv2.bitwise_and(img,img, mask= mask)
titles = ['Original Image', 'Selected Gray Values', 'Hue', 'Result']
images = [img, res, hue, res2]
for i in xrange(4):
plt.subplot(2,2,i+1),plt.imshow(images[i],'gray')
plt.title(titles[i])
plt.xticks([]),plt.yticks([])
plt.show()