I'm trying to paint part of an image as black and white using OpenCV2 and Python3. This is the code I'm trying:
(x, y, w, h) = cv2.boundingRect(c)
cv2.rectangle(frame, (x,y), (x+w,y+h),0,0)
sub_face = frame[y:y+h, x:x+w]
# apply a gaussian blur on this new recangle image
# sub_face = cv2.GaussianBlur(sub_face,(9, 9), 30, borderType = 0)
sub_face = cv2.cvtColor(sub_face, cv2.COLOR_BGR2GRAY)
# merge this blurry rectangle to our final image
result_frame[y:y+sub_face.shape[0], x:x+sub_face.shape[1]] = sub_face
When I apply the GaussianBlur method, it works properly, but when I try the cvtColor method it fails with a message (on the last line): could not broadcast input array from shape (268,182) into shape (268,182,3). What am I doing wrong?
The c variable in first line is a contour (from motion detection).
I'm new into Python and OpenCV.
Thanks!
You are trying to assign a single channel that results from your cv2.cvtColor call to three channels at once as result_frame is a RGB / three channel image. You are probably wanting to assign the single channel to all three channels. One way to do this cleanly is to exploit NumPy broadcasting by creating a singleton channel in the third dimension, then broadcasting the result over all channels. Since you are using the cv2 interface to OpenCV, the native datatype used for manipulating images is a NumPy array:
# merge this blurry rectangle to our final image
result_frame[y:y+sub_face.shape[0], x:x+sub_face.shape[1]] = sub_face[:,:,None]
The : operation in this context accesses all values in a particular dimension. In this case, we want the first and second dimensions. Therefore, sub_face[:,:,None] will make your single channel image 3D with the third dimension being a singleton (i.e. 1). Using NumPy broadcasting will then broadcast this single channel image to all channels simultaneously.
Note that I didn't have to explicitly access the third dimension when assigning to result_frame. That is because result_frame[y:y+sub_face.shape[0], x:x+sub_face.shape[1]] and result_frame[y:y+sub_face.shape[0], x:x+sub_face.shape[1],:] are the same thing as dropping indexing after the last dimension you specify implicitly assumes :.
You converted sub_face to a single channel image, but result_frame is a 3 channel image.
In the last line you are trying to assign a single channel array to a 3 channel slice.
You could do this:
result_frame[y:y+sub_face.shape[0], x:x+sub_face.shape[1], 0] = sub_face
result_frame[y:y+sub_face.shape[0], x:x+sub_face.shape[1], 1] = sub_face
result_frame[y:y+sub_face.shape[0], x:x+sub_face.shape[1], 2] = sub_face
Related
With a uint8, 3-channel image and uint8 binary mask, I have done the following in opencv and python in order to change an object on a black background into an object on a transparent background:
# Separate image into its 3 channels
b, g, r = cv2.split(img)
# Merge channels back with mask (resulting in a 4-channel image)
imgBGRA = cv2.merge((b, g, r, mask))
However, when I try doing this with a uint16, 3-channel image and uint16 binary mask, the saved result is 4-channel, but the background is still black. (I saved it as a .tiff file and viewed it in Photoshop.)
How can I make the background transparent, keeping the output image uint16?
UPDATE
Seeing #Shamshirsaz.Navid and #fmw42 comments, I tried
imgBGRA=cv2.cvtColor(imgBGR, cv2.COLOR_BGR2BGRA). Then used Numpy to add the alpha channel from the mask: imgBGRA[:,:,3]=mask. (I hadn't tried this, as I thought that cvtColor operations required an 8-bit image.) Nonetheless, my results are the same.
I think the problem is my mask. When I run numpy.amin(mask), I get 0, and for numpy.amax(mask), I get 1. What should they be? I tried multiplying the mask by 255 prior to using the split/merge technique, but the background was still black. Then I tried mask*65535, but again the background was black.
I had tried to keep the scope of my initial post narrow. But it seems that my problem does lie somewhere in the larger scope of what I'm doing and how this uint16 mask gets created.
I'm using connectedComponentsWithStats (CC) to cut out the components on a uint16 image. CC requires an 8-bit mask, which I am using as input to CC. But the cutout results need to be from my uint16 original. This has required some alterations to the way I learned to use CC on uint8 images. Note that the per-component mask (which I eventually use to try to make the background transparent) is created as uint16. Here is the whittled down version:
# img is original image, dtype=uint16
# bin is binary mask, dtype=uint8
cc = cv2.connectedComponentsWithStats(bin, connectivity, cv2.CV_32S)
num_labels = cc[0]
labels = cc[1]
for i in range(1, num_labels):
maskg = (labels == i).astype(np.uint16) # with uint8: maskg = (labels == i).astype(np.uint8) * 255
# NOTE: I don't understand why removing the `* 255` works; but after hours of experimenting, it's the only way I could get the original to appear correctly when saving 'glyph'; for all other methods I tried the colors were off in some significant way -- either grayish blue whereas the object in my original is variations of brown, or else a pixelated rainbow of colors)
glyph = img * maskg[..., np.newaxis] # with uint8: glyph = cv2.bitwise_and(img, img, mask=maskg)
b, g, r = cv2.split(glyph)
glyphBGRA = cv2.merge((b, g, r, maskg))
example (my real original image is huge and, also, I am not able share it; so I put together this example)
img (original uint16 image)
bin (input uint8 mask)
maskg (uint16 component mask created within loop)
(this is a screenshot -- it shows up all black when uploaded directly)
glyph (img with maskg applied)
glyphBGRA (result of split and merge method trying to add transparency)
(this is also a screenshot -- this one showed up all white/blank when added directly)
I hope this added info provides sufficient context for my problem.
I checked your last comment. I think an example might be better. Your code is correct; The question is, how did you use it? I attached a picture and a mask to test on them.
import sys,cv2
main = cv2.imread(sys.path[0]+'/main.png')
mask = cv2.imread(sys.path[0]+'/mask.png', cv2.IMREAD_GRAYSCALE)
mask = cv2.threshold(mask, 127, 255, cv2.THRESH_BINARY)[1]
b, g, r = cv2.split(main)
bgra = cv2.merge((b, g, r, mask))
cv2.imwrite(sys.path[0]+'/out_split_merge.png',bgra)
Main:
Mask:
Output:
If you open the final output with an image editing software, you will notice that part of it is transparent.
Diagnosis: Opencv is not able to save tiff with an alpha channel.
The following is from the opencv docs' entry for imwrite():
The function imwrite saves the image to the specified file. The image
format is chosen based on the filename extension (see cv::imread for
the list of extensions). In general, only 8-bit single-channel or
3-channel (with 'BGR' channel order) images can be saved using this
function, with these exceptions:
16-bit unsigned (CV_16U) images can be saved in the case of PNG, JPEG 2000, and TIFF formats
32-bit float (CV_32F) images can be saved in PFM, TIFF, OpenEXR, and Radiance HDR formats; 3-channel (CV_32FC3) TIFF images will be
saved using the LogLuv high dynamic range encoding (4 bytes per
pixel)
PNG images with an alpha channel can be saved using this function. To do this, create 8-bit (or 16-bit) 4-channel image BGRA, where
the alpha channel goes last. Fully transparent pixels should have
alpha set to 0, fully opaque pixels should have alpha set to
255/65535 (see the code sample below).
How I got to this point:
I manually removed the background in Photoshop and saved as png file and as tiff file. (They both look like this:)
Then I ran:
import cv2
import numpy as np
png16 = cv2.imread('c:/users/scott/desktop/python2/teststack/png16.png', cv2.IMREAD_UNCHANGED)
tif16 = cv2.imread('c:/users/scott/desktop/python2/teststack/tif16.tiff', cv2.IMREAD_UNCHANGED)
print('png16:', png16.dtype, png16.shape)
b, g, r, a = cv2.split(png16)
mmin = np.amin(a)
mmax = np.amax(a)
print('png16-a channel:', a.dtype, a.shape, mmin, mmax)
pixvals = np.unique(a.flatten()) # get all unique pixel values in a
print('png16-a channel pixel values:', pixvals)
print('tif16:', tif16.dtype, tif16.shape)
b, g, r, a = cv2.split(tif16)
mmin = np.amin(a)
mmax = np.amax(a)
print('tif16-a channel:', a.dtype, a.shape, mmin, mmax)
pixvals = np.unique(a.flatten()) # get all unique pixel values in a
print('tif16-a channel pixel values:', pixvals)
png16copy = png16.copy()
tif16copy = tif16.copy()
cv2.imwrite('c:/users/scott/desktop/python2/teststack/png16copy.png', png16copy)
cv2.imwrite('c:/users/scott/desktop/python2/teststack/tif16copy.tiff', tif16copy)
The output is all as one should expect:
png16: uint16 (312, 494, 4)
png16-a channel: uint16 (312, 494) 0 65535
png16-a channel pixel values: [ 0 65535]
tif16: uint16 (312, 494, 4)
tif16-a channel: uint16 (312, 494) 0 65535
tif16-a channel pixel values: [ 0 65535]
Back in Photoshop, the png file looked like it did before:
But the tiff file did not.
Without alpha channel visible:
With alpha channel visible:
So I knew at this point that the problem was in the saving. I reread the opencv docs for imwrite and picked up on the logic: if it's not 8-bit single-channel or 3-channel, and if it's not spelled out explicitly in the exceptions, it won't work.
I did some more searching and found something that does work. I installed tifffile and ran:
from tifffile import imsave
tif16copy2 = cv2.cvtColor(tif16copy, cv2.COLOR_BGRA2RGBA)
imsave('c:/users/scott/desktop/python2/teststack/tif16copy2.tiff', tif16copy2)
Here is the result in Photoshop:
I'm currently trying to apply an activation heatmap to a photo.
Currently, I have the original photo, as well as a mask of probabilities. I multiply the probabilities by 255 and then round down to the nearest integer. I'm then using cv2.applyColorMap with COLORMAP.JET to apply the colormap to the image with an opacity of 25%.
img_cv2 = cv2.cvtColor(np_img, cv2.COLOR_RGB2BGR)
heatmapshow = np.uint8(np.floor(mask * 255))
colormap = cv2.COLORMAP_JET
heatmapshow = cv2.applyColorMap(np.uint8(heatmapshow - 255), colormap)
heatmap_opacity = 0.25
image_opacity = 1.0 - heatmap_opacity
heatmap_arr = cv2.addWeighted(heatmapshow, heatmap_opacity, img_cv2, image_opacity, 0)
This current code successfully produces a heatmap. However, I'd like to be able to make two changes.
Keep the opacity at 25% For all values above a certain threshold (Likely > 0, but I'd prefer more flexibility), but then when the mask is below that threshold, reduce the opacity to 0% for those cells. In other words, if there is very little activation, I want to preserve the color of the original image.
If possible I'd also like to be able to specify a custom colormap, since the native ones are pretty limited, though I might be able to get away without this if I can do the custom opacity thing.
I read on Stackoverflow that you can possibly trick cv2 into not overlaying any color with NaN values, but also read that only works for floats and not ints, which complicates things since I'm using int8. I'm also concerned that this functionality could change in the future as I don't believe this is intentional design purposefully built into cv2.
Does anyone have a good way of accomplishing these goals? Thanks!
With regard to your second question:
Here is how to create a simple custom two color gradient color map in Python/OpenCV.
Input:
import cv2
import numpy as np
# load image as grayscale
img = cv2.imread('lena_gray.png', cv2.IMREAD_GRAYSCALE)
# convert to 3 equal channels
img = cv2.merge((img, img, img))
# create 1 pixel red image
red = np.full((1, 1, 3), (0,0,255), np.uint8)
# create 1 pixel blue image
blue = np.full((1, 1, 3), (255,0,0), np.uint8)
# append the two images
lut = np.concatenate((red, blue), axis=0)
# resize lut to 256 values
lut = cv2.resize(lut, (1,256), interpolation=cv2.INTER_LINEAR)
# apply lut
result = cv2.LUT(img, lut)
# save result
cv2.imwrite('lena_red_blue_lut_mapped.png', result)
# display result
cv2.imshow('RESULT', result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result of colormap applied to image:
With regard to your first question:
You are blending the heat map image with the original image using a constant "opacity" value. You can replace the single opacity value with an image. You just have to do the addWeighted manually as heatmap * opacity_img + original * (1-opacity_img) where your opacity image is float in the range 0 to 1. Then clip and convert back to uint8. If your opacity image is binary, then you can use cv2.bitWiseAnd() in place of multiply.
I am trying to use DeepLab v3 to detect object and mask where the actual object is.
DeepLab model produces a resized_im(3D) and a mask seg_map (2D) of 0 and non-zero values, 0 means it's the background.
Currently, it is only possible to plot an image with an overlay mask on the object. I want to crop the object out of the resized_im with transparent background. Is there any advice for the work?
You can play around with the notebook here:
https://colab.research.google.com/drive/138dTpcYfne40hqrb13n_36okSGYhrJnz?usp=sharing&hl=en#scrollTo=p47cYGGOQE1W&forceEdit=true&sandboxMode=true
I also tried here: How to crop image based on binary mask but none seems to work on my case
You just need to convert your segmentation mask to boolean numpy array, then multiply image by it. Don't forget that your image has 3 channels while mask has only 1. It may look something like that:
# seg_map - segmentation mask from network, resized_im - your input image
mask = np.greater(seg_map, 0) # get only non-zero positive pixels/labels
mask = np.expand_dims(mask, axis=-1) # (H, W) -> (H, W, 1)
mask = np.concatenate((mask, mask, mask), axis=-1) # (H, W, 1) -> (H, W, 3), (don't like it, so if you know how to do it better, please let me know)
crops = resized_im * mask # apply mask on image
You can use different logical numpy function if you want to choose certain labels, for example:
mask = np.equal(seg_map, 5) # to get only objects with label 5
I am trying to extract blue colour of an input image. For that I create a blue HSV colour boundary and threshold HSV image by using the command
mask_img = cv2.inRange(hsv, lower_blue, upper_blue)
After that I used a bitwise_and on the input image and the threshold image by using
res = cv2.bitwise_and(img, img, mask = mask_img)
Where img is the input image. I got this code from opencv. But I didn't understand why are three arguments used in bitwise_and and what actually each arguments mean? Why the same image is used at src1 and src2 ?
And also what is the use of mask keyword here? Please help me to find out the answer
The basic concept behind this is the value of color black ,it's value is 0 in OPEN_CV.So black + anycolor= anycolor because value of black is 0.
Now suppose we have two images one is named img1 and other is img2.
img2 contains a logo which we want to place on the img1. We create threshold and then the mask and mask_inv of img2,and also create roi of img1.
Now we have to do two things to add the logo of img2 on img1.
We create background of roi as img1_bg with help of : mask_inv,mask_inv will have two region one black and one white, in the white region we will put img1 part and leave black as it is-
img1_bg = cv2.bitwise_and(roi,roi,mask = mask_inv)
In your question you have used directly the mask of the img created
res = cv2.bitwise_and(img,img,mask = mask_img)
and in img2 we need to create the logo as foreground of roi ,
img2_fg = cv2.bitwise_and(img2,img2,mask = mask)
here we have used mask layer , the logo part of img2 gets filled in the white part of mask
Now when we add both we get a perfect combined roi
For full description and understanding visit:
OPEN CV CODE FILES AND FULL DESCRIPTION
The operation of "And" will be performed only if mask[i] doesn't equal zero, else the the result of and operation will be zero. The mask should be either white or black image with single channel. you can see this link
http://docs.opencv.org/2.4.13.2/modules/core/doc/operations_on_arrays.html?highlight=bitwise#bitwise-and
what is actually each arguments mean?
res = cv2.bitwise_and(img,img,mask = mask_img)
src1: the first image (the first object for merging)
src2: the second image (the second object for merging)
mask: understood as rules to merge. If region of image (which is gray-scaled, and then masked) has black color (valued as 0), then it is not combined (merging region of the first image with that of the second one), vice versa, it will be carried out. In your code, referenced image is "mask_img".
In my case, my code is correct, when it makes white + anycolor = anycolor
import cv2
import numpy as np
# Load two images
img1 = cv2.imread('bongSung.jpg')
img2 = cv2.imread('opencv.jpg')
# I want to put logo on top-left corner, so I create a ROI
rows, cols, channels = img2.shape
roi = img1[0:rows, 0:cols]
# NOw we need to create a mask of the logo, mask is conversion to grayscale of an image
img2gray = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 220, 255, cv2.THRESH_BINARY_INV)
cv2.imshow('mask', mask)
mask_inv = cv2.bitwise_not(mask)
#cv2.imshow("mask_inv", mask_inv)
#When using bitwise_and() in opencv with python then white + anycolor = anycolor; black + anycolor = black
img1_bg = cv2.bitwise_and(roi,roi,mask = mask_inv)
#cv2.imshow("img1_bg", img1_bg)
cv2.imshow("img2", img2)
img2_fg = cv2.bitwise_and(img2,img2,mask = mask)
cv2.imshow('img2_fg', img2_fg)
dst = cv2.add(img1_bg,img2_fg)
img1[0:rows, 0:cols] = dst
#cv2.imshow("Image", img1)
cv2.waitKey(0)
cv2.destroyAllWindows()
From above answers we may know the definitions of the parameters of bitwise_and(), but they all do not answer the other question
Why the same image is used at src1 and src2 ?
This question should be caused by the too simplified function definition in the document of OpenCV, it may be ambiguous to some people, in the document the bitwise_and() is defined as
dst(I)=sur1(I) ^ sur2(I), if mask(I) != 0, where ^ represents the 'and' operator
from this definition at first sight I cannot get the picture about how to process the dst(I) when the mask(I) is 0.
From the test result, I think that it should give a more clear function definition as
dst(I)=sur1(I) ^sur2(I), if mask(I) != 0,
otherwise the dst(I) keep its original value and the default value of all elements of the dst array is 0.
Now we may know that using the same image for sur1 and sur2, it will only keep the original image parts in the area of mask(I) !=0 and the other area shows the part of the dst image (as the mask shape)
Additionally for other bitwise operations the definitions should be the same as above, they also need to add the otherwise condition and the default value description of the dst array
The below link explains clearly the bitwise operation and also the significance of each parameters.
http://opencvexamples.blogspot.com/2013/10/bitwise-and-or-xor-and-not.html
void bitwise_and(InputArray src1, InputArray src2, OutputArray dst, InputArray mask=noArray())
Calculates the per-element bit-wise conjunction of two arrays or an array and a scalar.
Parameters:
src1 – first input array or a scalar.
src2 – second input array or a scalar.
src – single input array.
value – scalar value.
dst – output array that has the same size and type as the input arrays.
mask – optional operation mask, 8-bit single channel array, that specifies elements of the output array to be changed
Regarding using img twice, my guess is that we don't really care what img[i] and img[i] is, as it's just img[i] for binary. What matters is that, as mentioned by Mohammed Awney, when the mask is 0, we make img[i] be 0, otherwise we leave the pixel alone. This is a way to make certain pixels in img black, according to our mask.
bitwise_and ( InputArray src1,
InputArray src2,
OutputArray dst,
InputArray mask = noArray()
)
src1 first input array or a scalar.
src2 second input array or a scalar.
dst output array that has the same size and type as the input arrays.
mask optional operation mask, 8-bit single channel array, that specifies elements of the output array to be changed.
dst(I)=src1(I)∧src2(I)if mask(I)≠0
and mask operate on dst
computes bitwise conjunction of the two arrays (dst = src1 & src2) Calculates the per-element bit-wise conjunction of two arrays or an array and a scalar.
I am using opencv (cv2 module) in python to recognize objects in a video. In each frame, I want to extract a particular region aka, the contour. After learning from opencv docs, I have the following code snippet:
# np is numpy module, contours are expected results,
# frame is each frame of the video
# Iterate through the contours.
for contour in contours:
# Compute the bounding box for the contour, draw
# it on the frame, and update the text.
x, y, w, h = cv2.boundingRect(contour)
# Find the mask and build a histogram for the object.
mask = np.zeros(frame.shape[:2], np.uint8)
mask[y:h, x:w] = 255
masked_img = cv2.bitwise_and(frame, frame, mask = mask)
obj_hist = cv2.calcHist([masked_img], [0], None, [256], [0, 256])
However, when I use matplotlib to show the masked_img, it returns a dark image. The obj_hist has only one element with number greater than 0, which is the first one. What is wrong?
The problem is the way you are setting the values in your mask. Specifically this line:
mask[y:h, x:w] = 255
You are trying to slice into each dimension of the image by using y:h and x:w to set up the mask. The left of the colon is the starting row or column, and the right of the colon denotes the end row or column. Given that you start at y, you need to offset by h using the same reference y... the same goes for x and w.
Doing the slicing where the right value of the colon is less than the left will not modify the array in any way, and that's most why you aren't getting any output as you are not modifying the mask when it's initially all zeroes.
You probably meant to do:
mask[y:y+h, x:x+w] = 255
This will properly set the proper region given by cv2.boundingRect to white (255).