We Keep Coding

How To Get The Pixel Count Of A Segmented Area in an Image I used Vgg16 for Segmentation

I am new to deep learning but have succeeded in semantic segmentation of the image I am trying to get the pixel count of each class in the label. As an example in the image I want to get the pixel count of the carpet, or the chandelier or the light stand. How do I go about? Thanks any suggestions will help.

Edit: In what format the regions are returned? Do you have only the final image or the regions are given as contours? If you have them as contours (list of coordinates), you can apply findContourArea directly on that structure.
If you can receive/sample the regions one by one in an image (but do not have the contour), you can sequentially paint each of the colors/classes in a clear image, either convert it to grayscale or directly paint it in grayscale or binary, or binarize with threshold; then numberPixels = len(cv2.findNonZero(bwImage)). cv2.findContour and cv2.contourArea should do the same.
Instead of rendering each class in a separate image, if your program receives only the final segmentation and not per-class contours, you can filter/mask the regions by color ranges on that image. I built that and it seemed to do the job, 14861 pixels for the pink carpet:
import cv2
import numpy as np
# rgb 229, 0, 178 # the purple carpet in RGB (sampled with IrfanView)
# b,g,r = 178, 0, 229 # cv2 uses BGR
class_color = [178, 0, 229]
multiclassImage = cv2.imread("segmented.png")
cv2.imshow("MULTI", multiclassImage)
filteredImage = multiclassImage.copy()
low = np.array(class_color);
mask = cv2.inRange(filteredImage, low, low)
filteredImage[mask == 0] = [0, 0, 0]
filteredImage[mask != 0] = [255,255,255]
cv2.imshow("FILTER", filteredImage)
# numberPixelsFancier = len(cv2.findNonZero(filteredImage[...,0]))
# That also works and returns 14861 - without conversion, taking one color channel
bwImage = cv2.cvtColor(filteredImage, cv2.COLOR_BGR2GRAY)
cv2.imshow("BW", bwImage)
numberPixels = len(cv2.findNonZero(bwImage))
print(numberPixels)
cv2.waitKey(0)
If you don't have the values of the colors given or/and can't control them, you can use numpy.unique(): https://numpy.org/doc/stable/reference/generated/numpy.unique.html and it will return the unique colors, then they could be applied in the algorithm above.
Edit 2: BTW, another way to compute or verify such counts is by calculating histograms. That's with IrfanView on the black-white image:

Replace Color Values in Image with Random Noise

I found something reasonably close to what I want to do here:
Python: PIL replace a single RGBA color
However, in my scenario I have images that were originally grayscale with color annotations added to the image (an x-ray with notes in color). I would like to replace any pixel that is not grayscale with random noise. My main problem is replacing values with noise and not a single color.
Edit: I figured out the random noise part, now just trying to figure out how to separate the color pixels from the pixels that were originally in grayscale.
from PIL import Image
import numpy as np
im = Image.open('test.jpg')
data = np.array(im) # "data" is a height x width x 3 numpy array
red, green, blue = data.T # Temporarily unpack the bands for readability
# Replace white with random noise...
white_areas = (red == 255) & (blue == 255) & (green == 255)
Z = random.random(data[...][white_areas.T].shape)
data[...][white_areas.T] = Z
im2 = Image.fromarray(data)
im2.show()

You could try
col_areas = np.logical_or(np.not_equal(red, blue), np.not_equal(red, green))

You could use this Pixel Editing python module
from PixelMenu import ChangePixels as cp
im = Image.open('test.jpg')
grayscalergb=(128, 128, 128) #RGB value of gray in your image
noise=(100,30,5) #You can adjust the noise based upon your requirements
outputimg=cp(im, col=grayscalergb, col2=noise, save=False,tolerance=100) #Adjust the tolerance until you get the right amount of noise in your image
Also:
I'd suggest you to use png images instead of jpg images because JPEG is designed with compression, everytime you load the image the RGB values change making it hard for your code to function perfectly everytime

Caxis auto equivalent in OpenCV Python or Pillow?

MATLAB has the function caxis auto which comes in handy for viewing images if they have a low contrast and are looking almost black.
In Matlab it works like this:
figure;imshow(I);caxis auto
I know how to adjust the contrast in OpenCV Python, like:
img = cv2.imread('someimage.jpg',0)
equ = cv2.equalizeHist(img)
#Showing image
cv2.imshow('image',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
However, this does is not the same implementation as caxis auto which changes the illuminuation adaptively.

OpenCV does not use colormaps for displaying single channel images. In Matlab, caxis sets the colormap limits. In other words, if your image has values between 0.5 and 1.0, you can set the displayed color limits so that 0.5 is rendered as if it were 0.0, and scale that way. OpenCV just displays the image values as they are.
What that means is that if you want a different scaling for the values, you can just change them in the array that gets displayed.
With OpenCV, you can easily accomplish this scaling with cv2.normalize() with the normType=cv2.NORM_MINMAX.
img = np.random.rand(500, 500, 3)*0.2
cv2.imshow('Low contrast image', img)
cv2.waitKey()
img_scaled = cv2.normalize(img, None, alpha=0.0, beta=1.0, norm_type=cv2.NORM_MINMAX)
cv2.imshow('Scaled colors for display', img_scaled)
cv2.waitKey()
Note that this works for both grayscale and color images.
However, if you just have a grayscale image, another option is to display them with Matplotlib. MPL will automatically scale single channel images to the min/max display values.
from matplotlib import pyplot as plt
img = np.random.rand(500, 500)*0.2
plt.imshow(img, cmap='gray')
plt.show()
Note here I included the cmap='gray' argument because the default colormap for a single channel image is Viridis, and not grayscale. You can read more about Matplotlib colormaps here. Generally, if you're looking for similar syntax to Matlab, Matplotlib has you covered---it was initially geared towards people coming to Python from Matlab, so it has similar functionality and syntax.
One last option is to just scale manually in numpy. If your image is an unsigned int type, you can do this simply by shifting the image to have it's lowest value as 0, and then multiply so that it's max value is the maximum of your datatype. Note that when you multiply your image will be converted to a float, so you'll need to recast it. For e.g., if you have a uint16 image, the values range from 0 to 65535, so you could do:
img_scaled = img.copy() - np.min(img)
img_scaled = np.uint16(img_scaled*(65535/np.max(img)))
Or, if you'd like to do this automatically, you can grab the datatype information from the image (including the min and max values possible in a datatype) with np.iinfo() to automatically scale and convert as necessary. Here's a function that will normalize any (signed or unsigned) integer or floating point image:
def normalize_minmax(img):
"""Scales an image into an interval [min, max] with the same datatype.
If img.dtype is a float type, img is scaled into [0.0, 1.0].
If img.dtype is an integer type, img is scaled into [dtype.min, dtype.max].
If img.dtype is some other type, the original image is returned.
"""
dtype = img.dtype
if np.issubdtype(dtype, np.integer): # scale to [dtype.min, dtype.max]
dtype_info = np.iinfo(img.dtype)
img_scaled = img - np.min(img)
scale = (dtype_info.max - dtype_info.min) / np.max(img_scaled)
img_scaled = img_scaled * scale + dtype_info.min
return np.round(img_scaled).astype(dtype)
elif np.issubdtype(dtype, np.float): # scale to [0, 1]
img_scaled = img - np.min(img)
img_scaled = img_scaled / np.max(img_scaled)
return img_scaled.astype(dtype)
return img
How it works is easy to describe. For the float image case, you simply shift it so that the lowest value is zero, and then divide by the max to get it between 0 and 1. Then just simply cast it back to the original type. For the integer datatypes, it takes a little more effort if you care to include signed integers. In this case the range extends to negative numbers as well. You start off the same by shifting the lowest value to zero. But then you need to scale so that the values span the entire range of the datatype (e.g. (datatype_max - datatype_min) / max(img)). So now your values will span from [0, datatype_max - datatype_min], so you need to add back the datatype_min to get to [datatype_min, datatype_max]. I also rounded the values instead of truncating before converting back to the original datatype.

Remove background of the image using opencv Python

I have two images, one with only background and the other with background + detectable object (in my case its a car). Below are the images
I am trying to remove the background such that I only have car in the resulting image. Following is the code that with which I am trying to get the desired results
import numpy as np
import cv2
original_image = cv2.imread('IMG1.jpg', cv2.IMREAD_COLOR)
gray_original = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY)
background_image = cv2.imread('IMG2.jpg', cv2.IMREAD_COLOR)
gray_background = cv2.cvtColor(background_image, cv2.COLOR_BGR2GRAY)
foreground = np.absolute(gray_original - gray_background)
foreground[foreground > 0] = 255
cv2.imshow('Original Image', foreground)
cv2.waitKey(0)
The resulting image by subtracting the two images is
Here is the problem. The expected resulting image should be a car only.
Also, If you take a deep look in the two images, you'll see that they are not exactly same that is, the camera moved a little so background had been disturbed a little. My question is that with these two images how can I subtract the background. I do not want to use grabCut or backgroundSubtractorMOG algorithm right now because I do not know right now whats going on inside those algorithms.
What I am trying to do is to get the following resulting image
Also if possible, please guide me with a general way of doing this not only in this specific case that is, I have a background in one image and background+object in the second image. What could be the best possible way of doing this. Sorry for such a long question.

I solved your problem using the OpenCV's watershed algorithm. You can find the theory and examples of watershed here.
First I selected several points (markers) to dictate where is the object I want to keep, and where is the background. This step is manual, and can vary a lot from image to image. Also, it requires some repetition until you get the desired result. I suggest using a tool to get the pixel coordinates.
Then I created an empty integer array of zeros, with the size of the car image. And then I assigned some values (1:background, [255,192,128,64]:car_parts) to pixels at marker positions.
NOTE: When I downloaded your image I had to crop it to get the one with the car. After cropping, the image has size of 400x601. This may not be what the size of the image you have, so the markers will be off.
Afterwards I used the watershed algorithm. The 1st input is your image and 2nd input is the marker image (zero everywhere except at marker positions). The result is shown in the image below.
I set all pixels with value greater than 1 to 255 (the car), and the rest (background) to zero. Then I dilated the obtained image with a 3x3 kernel to avoid losing information on the outline of the car. Finally, I used the dilated image as a mask for the original image, using the cv2.bitwise_and() function, and the result lies in the following image:
Here is my code:
import cv2
import numpy as np
import matplotlib.pyplot as plt
# Load the image
img = cv2.imread("/path/to/image.png", 3)
# Create a blank image of zeros (same dimension as img)
# It should be grayscale (1 color channel)
marker = np.zeros_like(img[:,:,0]).astype(np.int32)
# This step is manual. The goal is to find the points
# which create the result we want. I suggest using a
# tool to get the pixel coordinates.
# Dictate the background and set the markers to 1
marker[204][95] = 1
marker[240][137] = 1
marker[245][444] = 1
marker[260][427] = 1
marker[257][378] = 1
marker[217][466] = 1
# Dictate the area of interest
# I used different values for each part of the car (for visibility)
marker[235][370] = 255 # car body
marker[135][294] = 64 # rooftop
marker[190][454] = 64 # rear light
marker[167][458] = 64 # rear wing
marker[205][103] = 128 # front bumper
# rear bumper
marker[225][456] = 128
marker[224][461] = 128
marker[216][461] = 128
# front wheel
marker[225][189] = 192
marker[240][147] = 192
# rear wheel
marker[258][409] = 192
marker[257][391] = 192
marker[254][421] = 192
# Now we have set the markers, we use the watershed
# algorithm to generate a marked image
marked = cv2.watershed(img, marker)
# Plot this one. If it does what we want, proceed;
# otherwise edit your markers and repeat
plt.imshow(marked, cmap='gray')
plt.show()
# Make the background black, and what we want to keep white
marked[marked == 1] = 0
marked[marked > 1] = 255
# Use a kernel to dilate the image, to not lose any detail on the outline
# I used a kernel of 3x3 pixels
kernel = np.ones((3,3),np.uint8)
dilation = cv2.dilate(marked.astype(np.float32), kernel, iterations = 1)
# Plot again to check whether the dilation is according to our needs
# If not, repeat by using a smaller/bigger kernel, or more/less iterations
plt.imshow(dilation, cmap='gray')
plt.show()
# Now apply the mask we created on the initial image
final_img = cv2.bitwise_and(img, img, mask=dilation.astype(np.uint8))
# cv2.imread reads the image as BGR, but matplotlib uses RGB
# BGR to RGB so we can plot the image with accurate colors
b, g, r = cv2.split(final_img)
final_img = cv2.merge([r, g, b])
# Plot the final result
plt.imshow(final_img)
plt.show()
If you have a lot of images you will probably need to create a tool to annotate the markers graphically, or even an algorithm to find markers automatically.

The problem is that you're subtracting arrays of unsigned 8 bit integers. This operation can overflow.
To demonstrate
>>> import numpy as np
>>> a = np.array([[10,10]],dtype=np.uint8)
>>> b = np.array([[11,11]],dtype=np.uint8)
>>> a - b
array([[255, 255]], dtype=uint8)
Since you're using OpenCV, the simplest way to achieve your goal is to use cv2.absdiff().
>>> cv2.absdiff(a,b)
array([[1, 1]], dtype=uint8)

I recommend using OpenCV's grabcut algorithm. You first draw a few lines on the foreground and background, and keep doing this until your foreground is sufficiently separated from the background. It is covered here: https://docs.opencv.org/trunk/d8/d83/tutorial_py_grabcut.html
as well as in this video: https://www.youtube.com/watch?v=kAwxLTDDAwU

reduce image to N colors in OpenCV Python

I can only ever find examples in C/C++ and they never seem to map well to the OpenCV API. I'm loading video frames (both from files and from a webcam) and want to reduce them to 16 color, but mapped to a 24-bit RGB color-space (this is what my output requires - a giant LED display).
I read the data like this:
ret, frame = self._vid.read()
image = cv2.cvtColor(frame, cv2.COLOR_RGB2BGRA)
I did find the below python example, but cannot figure out how to map that to the type of output data I need:
import numpy as np
import cv2
img = cv2.imread('home.jpg')
Z = img.reshape((-1,3))
# convert to np.float32
Z = np.float32(Z)
# define criteria, number of clusters(K) and apply kmeans()
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0)
K = 8
ret,label,center=cv2.kmeans(Z,K,None,criteria,10,cv2.KMEANS_RANDOM_CENTERS)
# Now convert back into uint8, and make original image
center = np.uint8(center)
res = center[label.flatten()]
res2 = res.reshape((img.shape))
cv2.imshow('res2',res2)
cv2.waitKey(0)
cv2.destroyAllWindows()
That obviously works for the OpenCV image viewer but trying to do the same errors on my output code since I need an RGB or RGBA format. My output works like this:
for y in range(self.height):
for x in range(self.width):
self._led.set(x,y,tuple(image[y,x][0:3]))
Each color is represented as an (r,g,b) tuple.
Any thoughts on how to make this work?

I think the following could be faster than kmeans, specially with a k = 16.
Convert the color image to gray
Contrast stretch this gray image to so that resulting image gray levels are between 0 and 255 (use normalize with NORM_MINMAX)
Calculate the histogram of this stretched gray image using 16 as the number of bins (calcHist)
Now you can modify these 16 values of the histogram. For example you can sort and assign ranks (say 0 to 15), or assign 16 uniformly distributed values between 0 and 255 (I think these could give you a consistent output for a video)
Backproject this histogram onto the stretched gray image (calcBackProject)
Apply a color-map to this backprojected image (you might want to scale the backprojected image befor applying a colormap using applyColorMap)
Tip for kmeans:
If you are using kmeans for video, you can use the cluster centers from the previous frame as the initial positions in kmeans for the current frame. That way, it'll take less time to converge, so kmeans in the subsequent frames will most probably run faster.

You can speed up your processing by applying the k-means on a downscaled version of your image. This will give you the cluster centroids. You can then quantify each pixel of the original image by picking the closest centroid.