We Keep Coding

[cv2.filter2D() on Python]: why does it return these specific values?

The following Python script computes the 2D convolution of the blue color channel of a .jpg image:
It reads a 6x6 BGR image:
It extracts the channel 0. In cv2 this corresponds to color channel blue.
I print the values of the channel
Input data type is uint8. Therefore, we making cv2.filter2D() and setting ddepth=-1, the output will have data type uint8 too and hence values >255 cannot be represented. Hence, I decided to convert the image from uint8 to, for example, short to have a wider numeric range and be able to represent the values at the output of the filter.
I define a kernel of size 3x3 (see the values of the kernel in the code below).
I filter the blue channel with the kernel and I obtain a filtered image of the same size due to the padding
The filtered values given by the function filter2D() don't correspond to what I would expect. For example, for the top left value the functions returns 449, however I would have expected 425 instead since 71*0+60*0+65*1+69*1+58*3+61*1+89*0+66*0+56*1=425.
Does anyone have any idea about how the filtered image is being calculated by filter2D() function? Is there anything wrong with my proposed calculation?
import cv2
import numpy as np
image = cv2.imread('image.jpg')
# Read image 6x6x3 (BGR)
blue_channel=image[:,:,0]
# Obtain blue channel
print(blue_channel)
# Result is:
#[[71 60 65 71 67 67]
# [69 58 61 69 69 67]
# [89 66 56 55 45 37]
# [65 37 27 32 31 30]
# [46 23 22 38 43 45]
# [55 36 44 60 60 47]]
blue_channel=np.short(blue_channel)
# Convert image from uint8 to short. Otherwise, output of the filter will have the same data type as the
# input when using ddepth=-1 and hence filtered values >255 won't be able to be represented
print(blue_channel)
# Result is (same as before, ok...):
# 71 60 65 71 67 67
# 69 58 61 69 69 67
# 89 66 56 55 45 37
# 65 37 27 32 31 30
# 46 23 22 38 43 45
# 55 36 44 60 60 47
kernel=np.array([ [0, 0, 1], [1, 3, 1], [0, 0, 1] ])
# Kernel is of size 3x3
# [0 0 1]
# [1 3 1]
# [0 0 1]
filtered_image = cv2.filter2D(blue_channel, -1, kernel)
# Blue channel is filtered with the kernel and the result gives:
# 449 438 464 483 473 473
# 449 425 436 449 447 451
# 494 431 390 366 324 301
# 358 281 243 242 237 240
# 257 208 219 270 289 312
# 283 251 304 370 377 347
print(filtered_image)
# Why top left filtered value is 449?
# I would expect this:
# 71*0+60*0+65*1+69*1+58*3+61*1+89*0+66*0+56*1=425
# In short, I would expect 425 instead of 449, how is that 449 computed?

Your calculation is not wrong, but you have actually written convolution for value at [2,2], which match with your result 425.
To calculate value e.g. [1,1] you need values outside of the image, you have to handle surrounding edges. And by default in function filter2D they are handled as reflect 101 , in wiki its shifted mirror edge handling by +1.
To understand diffrence between mirror(reflect) and reflect 101:
Mirror (reflect)
left edge | image | right edge
| |
b a | a b c | c b
Reflect 101
left edge | image | right edge
| |
c b | a b c | b a
So calculation for [1,1] with default edge handling in filder2D would be:
0*58 + 0*69 + 1*58 + 1*60 + 3*71 + 1*60 + 0*58 + 0*69 + 1*58 = 449

How to obtain a matrix of size n*3(where n is the total number of pixels of an image) having R,G and B components of the image

I want to obtain a matrix consisting of only the R,G and B components of every pixel of an image. The cv2.imread function returns a 3 channel array. I wish to convert the 3d array into an array of size n*3 where 'n' must be the total number of pixels of the image and the 3 columns must contain the red, green and blue values at each pixel.

You can reshape the array:
import numpy as np
import cv2
# Load an color image in color (its the default as well)
img = cv2.imread(r'somepath\2015-08-05 10.58.36.jpg',cv2.IMREAD_COLOR )
print (img.shape)
c = img.reshape((img.shape[0]*img.shape[1],3))
print(c.shape)
Output:
(2048, 1536, 3) # my original image
(3145728, 3) # reshaped
for i in range(9):
print(img[0][i])
print(c[i])
Output:
[191 125 84] # from img
[191 125 84] # from c
[191 125 84] # etc.
[191 125 84]
[185 119 78]
[185 119 78]
[184 118 77]
[184 118 77]
[188 122 81]
[188 122 81]
[189 123 82]
[189 123 82]
[188 122 81]
[188 122 81]
[190 124 83]
[190 124 83]
[190 126 85]
[190 126 85]
BGR
For each color, #0 is Blue, #1 is Green, #2 is Red.

I believe this is what you are after (note that cv2.imread reads the image as BGR, so np.flip(j,1) converts the pixel order to RGB):
import cv2
import numpy as np
img = cv2.imread(r'Path to Image')
pixels = [np.flip(j, 1) for j in [i for i in img]]

subtracting RGB values from an Image in Python

I'm working in a project where I need to subtract the RGB values from an Image. In example I want to subtract the BLUE channel from RED, so RED gets the difference value of the subtraction.
I have the next properties of the image:
Dimension:1456x2592,
bpp:3
The image I'm using gives me the following arrays:
[[[ 63 58 60]
[ 63 58 60]
[ 64 59 61]
...,
[155 155 161]
[155 155 161]
[155 155 161]]
[[ 58 53 55]
[ 60 55 57]
[ 62 57 59]
...,
[157 157 163]
[157 157 163]
[158 158 164]]
I know those are the values(RGB) from the image, so now I move on to do the code (I based on this code)
import cv2
import numpy as np
from PIL import Image
# read image into matrix.
m = cv2.imread("ITESO.jpeg")
# get image properties.
h,w,bpp = np.shape(m)
# iterate over the entire image.
# BLUE = 0, GREEN = 1, RED = 2.
for py in range(0,h):
for px in range(0,w):
#m[py][px][2] = 2
n = m[py][px][2] //n takes the value of RED
Y = [n, 0, 0] //I create an array with [RED, 0, 0]
m, Y = np.array(m), np.array(Y)
m = np.absolute(m - Y) //Get the matriz with the substraction
y = 1
x = 1
print (m)
print (m[x][y])
#display image
#cv2.imshow('matrix', m)
#cv2.waitKey(0)
cv2.imwrite('new.jpeg',m)
img = Image.open('new.jpeg')
img.show()
img = Image.open('new.jpeg').convert('L')
img.save('new_gray_scale.jpg')
img.show()
When I print the J matrix it gives the following arrays:
B,G,R
Blue = BLUE - RED
[[[ 3 58 60]
[ 3 58 60]
[ 4 59 61]
...,
[ 95 155 161]
[ 95 155 161]
[ 95 155 161]]
[[ 2 53 55]
[ 0 55 57]
[ 2 57 59]
...,
[ 97 157 163]
[ 97 157 163]
[ 98 158 164]]
But I'm not able to open the new image and if I set one RGB channel to one value it shows me the image. I use the next lines for that:
import cv2
import numpy as np
# read image into matrix.
m = cv2.imread("python.png")
# get image properties.
h,w,bpp = np.shape(m)
# iterate over the entire image.
for py in range(0,h):
for px in range(0,w):
m[py][px][0] = 0 //setting channel Blue to values of 0
# display image
cv2.imshow('matrix', m)
cv2.waitKey(0)
How can I subtract the RGB channels from each other?
PS: In MatLab it works like a charm, but I'm not able to do it in python.

Pay attention that this operation is changing the dtype of the matrix (image) from uint8 to int32, and this can cause other problems. A better way (and more efficient) to do this, IMO, is this:
import cv2
import numpy as np
img = cv2.imread('image.png').astype(np.float) # BGR, float
img[:, :, 2] = np.absolute(img[:, :, 2] - img[:, :, 0]) # R = |R - B|
img = img.astype(np.uint8) # convert back to uint8
cv2.imwrite('new-image.png', img) # save the image
cv2.imshow('img', img)
cv2.waitKey()

Code manipulating RGB negative values to zero...
m = cv2.imread("img.jpg")
# get image properties.
h,w,bpp = np.shape(m)
# iterate over the entire image.
# BLUE = 0, GREEN = 1, RED = 2.
for py in range(0,h):
for px in range(0,w):
n = m[py][px][1]
Y = [0, 0, n]
m, Y = np.array(m), np.array(Y)
a = (m - Y)
if (a[py][px][0] <=0): #if Blue is negative or equal 0
a[py][px][0] = 0 #Blue set to 0
cv2.imwrite('img_R-G.jpg',a)
img = Image.open('img_R-G.jpg').convert('L')
img.save('img_R-G_GS.jpg')

Contrast stretching in Python/ OpenCV

Searching Google for Histogram Equalization Python or Contrast Stretching Python I am directed to the same links from python documentation in OpenCv which are actually both related to equalization and not stretching (IMO).
http://docs.opencv.org/2.4/doc/tutorials/imgproc/histograms/histogram_equalization/histogram_equalization.html
http://docs.opencv.org/3.2.0/d5/daf/tutorial_py_histogram_equalization.html
Read the documentation, it seems to be a confusion with the wording, as it describes equalization as a stretching operation:
What Histogram Equalization does is to stretch out this range.
AND
So you need to stretch this histogram to either ends (as given in below image, from wikipedia) and that is what Histogram Equalization does (in simple words)
I feel that is wrong because nowhere on Wikipedia it says that histogram equalization means stretching, and reading other sources they clearly distinguish the two operations.
http://homepages.inf.ed.ac.uk/rbf/HIPR2/histeq.htm
http://homepages.inf.ed.ac.uk/rbf/HIPR2/stretch.htm
My questions:
does the OpenCV documentation actually implements Histogram Equalization, while badly explaining it?
Is there any implementation for contrast stretching in Python? (OpenCV, etc?)

OpenCV doesn't have any function for contrast stretching and google yields the same result because histogram equalization does stretch the histogram horizontally but its just the difference of the transformation function. (Both methods increase the contrast of the images.Transformation function transfers the pixel intensity levels from the given range to required range.)
Histogram equalization derives the transformation function(TF) automatically from probability density function (PDF) of the given image where as in contrast stretching you specify your own TF based on the applications' requirement.
One simple TF through which you can do contrast stretching is min-max contrast stretching -
((pixel – min) / (max – min))*255.
You do this for each pixel value. min and max being the minimum and maximum intensities.

You can also use cv2.LUT for contrast stretching by creating a custom table using np.interp. Links to their documentation are this and this respectively. Below an example is shown.
import cv2
import numpy as np
img = cv2.imread('messi.jpg')
original = img.copy()
xp = [0, 64, 128, 192, 255]
fp = [0, 16, 128, 240, 255]
x = np.arange(256)
table = np.interp(x, xp, fp).astype('uint8')
img = cv2.LUT(img, table)
cv2.imshow("original", original)
cv2.imshow("Output", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
The table created
[ 0 0 0 0 1 1 1 1 2 2 2 2 3 3 3 3 4 4
4 4 5 5 5 5 6 6 6 6 7 7 7 7 8 8 8 8
9 9 9 9 10 10 10 10 11 11 11 11 12 12 12 12 13 13
13 13 14 14 14 14 15 15 15 15 16 17 19 21 23 24 26 28
30 31 33 35 37 38 40 42 44 45 47 49 51 52 54 56 58 59
61 63 65 66 68 70 72 73 75 77 79 80 82 84 86 87 89 91
93 94 96 98 100 101 103 105 107 108 110 112 114 115 117 119 121 122
124 126 128 129 131 133 135 136 138 140 142 143 145 147 149 150 152 154
156 157 159 161 163 164 166 168 170 171 173 175 177 178 180 182 184 185
187 189 191 192 194 196 198 199 201 203 205 206 208 210 212 213 215 217
219 220 222 224 226 227 229 231 233 234 236 238 240 240 240 240 240 241
241 241 241 242 242 242 242 243 243 243 243 244 244 244 244 245 245 245
245 245 246 246 246 246 247 247 247 247 248 248 248 248 249 249 249 249
250 250 250 250 250 251 251 251 251 252 252 252 252 253 253 253 253 254
254 254 254 255]
Now cv2.LUT will replace the values of the original image with the values in the table. For example, all the pixels having values 1 will be replaced by 0 and all pixels having values 4 will be replaced by 1.
Original Image
Contrast Stretched Image
The values of xp and fp can be varied to create custom tables as required and it will stretch the contrast even if min and max pixels are 0 and 255 unlike the answer provided by hashcode55.

Python/OpenCV can do contrast stretching via the cv2.normalize() method using min_max normalization. For example:
Input:
#!/bin/python3.7
import cv2
import numpy as np
# read image
img = cv2.imread("zelda3_bm20_cm20.jpg", cv2.IMREAD_COLOR)
# normalize float versions
norm_img1 = cv2.normalize(img, None, alpha=0, beta=1, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_32F)
norm_img2 = cv2.normalize(img, None, alpha=0, beta=1.2, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_32F)
# scale to uint8
norm_img1 = (255*norm_img1).astype(np.uint8)
norm_img2 = np.clip(norm_img2, 0, 1)
norm_img2 = (255*norm_img2).astype(np.uint8)
# write normalized output images
cv2.imwrite("zelda1_bm20_cm20_normalize1.jpg",norm_img1)
cv2.imwrite("zelda1_bm20_cm20_normalize2.jpg",norm_img2)
# display input and both output images
cv2.imshow('original',img)
cv2.imshow('normalized1',norm_img1)
cv2.imshow('normalized2',norm_img2)
cv2.waitKey(0)
cv2.destroyAllWindows()
Normalize1:
Normalize2:
You can also do your own stretching by using a simple linear equation with 2 pair of input/ouput values using the form y=A*x+B and solving the two simultaneous equations. See concept for stretching shown in How can I make the gradient appearance of one image equal to the other?

Ok, so I wrote this function that does Standard Deviation Contrast Stretching, on each band of an image.
For normal distributions, 68% of the observations lie within – 1 standard deviation of the mean, 95.4% of all observations lie within – 2 standard deviations, and 99.73% within – 3 standard deviations.
this is basically a min-max stretch but the max is mean+sigma*std and min is mean-sigma*std
def stretch(img,sigma =3,plot_hist=False):
stretched = np.zeros(img.shape)
for i in range(img.shape[2]): #looping through the bands
band = img[:,:,i] # copiying each band into the variable `band`
if np.min(band)<0: # if the min is less that zero, first we add min to all pixels so min becomes 0
band = band + np.abs(np.min(band))
band = band / np.max(band)
band = band * 255 # convertaning values to 0-255 range
if plot_hist:
plt.hist(band.ravel(), bins=256) #calculating histogram
plt.show()
# plt.imshow(band)
# plt.show()
std = np.std(band)
mean = np.mean(band)
max = mean+(sigma*std)
min = mean-(sigma*std)
band = (band-min)/(max-min)
band = band * 255
# this streching cuases the values less than `mean-simga*std` to become negative
# and values greater than `mean+simga*std` to become more than 255
# so we clip the values ls 0 and gt 255
band[band>255]=255
band[band<0]=0
print('band',i,np.min(band),np.mean(band),np.std(band),np.max(band))
if plot_hist:
plt.hist(band.ravel(), bins=256) #calculating histogram
plt.show()
stretched[:,:,i] = band
stretched = stretched.astype('int')
return stretched
in the case above, I didn't need the band ratios to stay the same, but the best practice for an RGB image would be like this:
https://docs.opencv.org/4.x/d5/daf/tutorial_py_histogram_equalization.html
Unfortunately, this CLAHE stretching does not work on multi-band images so should be applied to each band separately - which gives wrong results since the contrast between each band will be lost and the images tend to be gray. what we need to do is:
we need to transform the image into HSV color space and stretch the V (value - intensity) and leave the rest. this is how we get a good stretch(pun intended).
The thing about cv.COLOR_HSV2RGB is that it actually returns BGR instead of RGB so after the HSV2RGB we need to reverse the bands.
here's the function I wrote:
def stack_3_channel(r,g,b , clipLimit = 20 , tileGridSize=(16,16) ):
img = np.stack([r,g,b], axis=2)
img = cv.normalize(img, None, 0, 255, cv.NORM_MINMAX, dtype=cv.CV_8U)
hsv_img = cv.cvtColor(img, cv.COLOR_BGR2HSV)
h, s, v = hsv_img[:,:,0], hsv_img[:,:,1], hsv_img[:,:,2]
clahe = cv.createCLAHE(clipLimit, tileGridSize)
v = clahe.apply(v) #stretched histogram for showing the image with better contrast - its not ok to use it for scientific calculations
hsv_img = np.dstack((h,s,v))
# NOTE: HSV2RGB returns BGR instead of RGB
bgr_stretched = cv.cvtColor(hsv_img, cv.COLOR_HSV2RGB)
#reversing the bands back to RGB
rgb_stretched = np.zeros(bgr_stretched.shape)
rgb_stretched[:,:,0] = bgr_stretched[:,:,2]
rgb_stretched[:,:,1] = bgr_stretched[:,:,1]
rgb_stretched[:,:,2] = bgr_stretched[:,:,0]
# if the valuse are float, plt will have problem showing them
rgb_stretched = rgb_stretched.astype('uint8')
return img , rgb_stretched

Inverting numpy array image which might be uint8, uint16,

Is there a prettier way to do this? Specifically, are these max values available through the numpy API? I haven't been able to find them in the API, although they are easily found here in the docs.
MAX_VALUES = {np.uint8: 255, np.uint16: 65535, np.uint32: 4294967295, \
np.uint64: 18446744073709551615}
try:
image = MAX_VALUES[image.dtype] - image
except KeyError:
raise ValueError, "Image must be array of unsigned integers."
Packages like PIL and cv2 provide convenient tools for inverting an image, but at this point in the code I have a numpy array -- more sophisticated analysis follows -- and I'd like to stick with numpy.

Try doing
image ^= MAX_VALUES[image.dtype]

By the way, you do not need to define MAX_VALUES yourself. NumPy has them built-in:
import numpy as np
h, w = 100, 100
image = np.arange(h*w).reshape((h,w)).astype(np.uint8)
max_val = np.iinfo(image.dtype).max
print(max_val)
# 255
image ^= max_val
print(image)
# [[255 254 253 ..., 158 157 156]
# [155 154 153 ..., 58 57 56]
# [ 55 54 53 ..., 214 213 212]
# ...,
# [ 27 26 25 ..., 186 185 184]
# [183 182 181 ..., 86 85 84]
# [ 83 82 81 ..., 242 241 240]]