I have two images and a mask, all of same dimensions, as Numpy arrays:
Desired output
I would like to merge them in such a way that the output will be like this:
Current code
def merge(lena, rocket, mask):
'''Mask init and cropping'''
mask = np.zeros(lena.shape[:2], dtype='uint8')
cv2.fillConvexPoly(mask, circle, 255) # might be polygon
'''Bitwise operations'''
lena = cv2.bitwise_or(lena, lena, mask=mask)
mask_inv = cv2.bitwise_not(mask) # mask inverting
rocket = cv2.bitwise_or(rocket, rocket, mask=mask_inv)
output = cv2.bitwise_or(rocket, lena)
return output
Current result
This code gives me this result:
Applying cv2.GaussianBlur(mask, (51,51), 0) distorts colors of overlayed image in different ways.
Other SO questions relate to similar problems but not solving exactly this type of blurred compositing.
Update: this gives same result as a current one
mask = np.zeros(lena.shape[:2], dtype='uint8')
mask = cv2.GaussianBlur(mask, (51,51), 0)
mask = mask[..., np.newaxis]
cv2.fillConvexPoly(mask, circle, 1)
output = mask * lena + (1 - mask) * rocket
Temporal solution
Possibly this is not optimal due to many conversions, please advise
mask = np.zeros(generated.shape[:2])
polygon = np.array(polygon, np.int32) # 2d array of x,y coords
cv2.fillConvexPoly(mask, polygon, 1)
mask = cv2.GaussianBlur(mask, (51, 51), 0)
mask = mask.astype('float32')
mask = cv2.cvtColor(mask, cv2.COLOR_GRAY2BGR)
foreground = cv2.multiply(lena, mask, dtype=cv2.CV_8U)
background = cv2.multiply(rocket, (1 - mask), dtype=cv2.CV_8U)
output = cv2.add(foreground, background)
Please advise how can I blur a mask, properly merge it with foreground and then overlay on background image?
You need to renormalize the mask before blending:
def blend_merge(lena, rocket, mask):
mask = cv2.GaussianBlur(mask, (51, 51), 0)
mask = cv2.cvtColor(mask, cv2.COLOR_GRAY2BGR)
mask = mask.astype('float32') / 255
foreground = cv2.multiply(lena, mask, dtype=cv2.CV_8U)
background = cv2.multiply(rocket, (1 - mask), dtype=cv2.CV_8U)
output = cv2.add(foreground, background)
return output
A full working example is here.
Here is how to do that in Python/OpenCV. Your second method is close.
Read the 3 input images
Apply linear (or Gaussian) blur to the circle
Stretch the circle to full dynamic range (0 to 255) as a mask
Convert the mask to float in the range 0 to 1 as 3 channels
Apply mask to image1 and inverted mask to image2 via multiplication and add the products together
Convert the result to 8-bit range (0 to 255) clipping to be sure no overflow or wrap around
Save the results
Input images:
import cv2
import numpy as np
# Read images
image1 = cv2.imread('lena_wide.jpg')
image2 = cv2.imread('rocket.jpg')
circle = cv2.imread('white_circle.jpg', cv2.IMREAD_GRAYSCALE)
# linear blur mask
mask = cv2.blur(circle, (30,30))
# alternate using Gaussian blur
#mask = cv2.GaussianBlur(circle, (0,0), sigmaX=10, sigmaY=10)
# stretch mask to full dynamic range
mask = cv2.normalize(mask, None, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_8U)
# convert mask to float in range 0 to 1
maskf = (mask/255).astype(np.float64)
maskf = cv2.merge([maskf,maskf,maskf])
# apply mask to image1 and inverted mask to image2
result = maskf*image1 + (1-maskf)*image2
result = result.clip(0,255).astype(np.uint8)
# save results
cv2.imwrite('white_circle_ramped.jpg', mask)
cv2.imwrite('lena_wide_rocked_composited.png', result)
# show results
cv2.imshow("mask", mask)
cv2.imshow("result", result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Ramped mask image:
Result:
ADDITION:
Here is an alternate approach using mostly Numpy.
import cv2
import numpy as np
# Read images
image1 = cv2.imread('lena_wide.jpg')
image2 = cv2.imread('rocket.jpg')
circle = cv2.imread('white_circle2.jpg', cv2.IMREAD_GRAYSCALE)
# linear blur mask
mask = cv2.blur(circle, (30,30))
# alternate using Gaussian blur
#mask = cv2.GaussianBlur(circle, (0,0), sigmaX=10, sigmaY=10)
# stretch mask to full dynamic range
mask = cv2.normalize(mask, None, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_8U)
# convert mask to 3 channels
mask = cv2.merge([mask,mask,mask])
# apply mask to image1 and inverted mask to image2
image1_masked = np.multiply(image1, mask/255).clip(0,255).astype(np.uint8)
image2_masked = np.multiply(image2, 1-mask/255).clip(0,255).astype(np.uint8)
# add the two masked images together
result = np.add(image1_masked, image2_masked)
# save results
cv2.imwrite('white_circle_ramped2.jpg', mask)
cv2.imwrite('lena_wide_rocked_composited2.png', result)
# show results
cv2.imshow("mask", mask)
cv2.imshow("image1_masked", image1_masked)
cv2.imshow("image2_masked", image2_masked)
cv2.imshow("result", result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Related
Here is the background image;
Here is the foreground image;
Here is the Mask image;
The code below;
background = cv2.imread('back.jpg')
image = cv2.imread('img.jpg')
mask = cv2.imread('mask.jpg')
crop_background = cv2.resize(background, (image.shape[1], image.shape[0]), interpolation = cv2.INTER_NEAREST)
mask = np.where(mask == 0, mask, image)
crop_background1 = np.where(mask == 0, crop_background, mask)
cv2.imshow(f'{i}', crop_background1)
cv2.waitKey(0)
output image After running the code;
Carefully look at the data types you are handling. All three images are BGR, they have three intensity values per pixel. In the where function you are only checking and setting one intensity value, so you only change the blue channel, instead of all three channels.
Now, you also have to be careful while composing these images. Be aware of pixel values possible “wrapping around” (unsigned integer overflow) and masking the correct pixels. One possible solution is to compose the image using a linear relationship between pixels values. Nothing fancy, just assigning the pixel values according to the mask matrix:
newPixel = (originalPixel * maskPixel + backgroundPixel * (1-maskPixel))
Note what happens when maskPixel is 0 or 1 (You'll need to use float types). You can see that the "ramp" correctly sets the newPixel value. The operation is also vectorizable. Let's check out the new code:
import cv2
import numpy as np
# Image path
path = "D://opencvImages//"
# Reading the images in default mode:
background = cv2.imread(path + "jkU1C.jpg")
image = cv2.imread(path + "QNuXL.jpg")
mask = cv2.imread(path + "6yyNG.png")
# Resize background:
crop_background = cv2.resize(background, (image.shape[1], image.shape[0]), interpolation = cv2.INTER_NEAREST)
# BGR to Gray mask:
mask = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
# Threshold mask:
_, mask = cv2.threshold(mask, 0, 255, cv2.THRESH_BINARY)
# Back to BGR:
mask = cv2.cvtColor(mask, cv2.COLOR_GRAY2BGR)
# Convert mask to float in range 0 to 1
mask = mask.astype(np.float64) / 255
# Get new pixel according to the mask matrix:
result = (image * mask + crop_background * (1 - mask))
# Clip values and convert back to uint8:
result = result.clip(0, 255).astype(np.uint8)
# Show result:
cv2.imshow("result", result)
cv2.waitKey(0)
The result:
I'm working on automating changing image colors using python. The image I'm using is below, i'd love to move it from red to another range of colors, say green, keeping the detail and shading if possible. I've been able to convert some of the image to a solid color, losing all detail.
The code I'm currently using is below, I can't quite figure out the correct range of red to make it work correctly, and also it only converts to a single color, again losing all detail and shade.
Any help is appreciated, thank you.
import cv2
import numpy as np
import skimage.exposure
# load image and get dimensions
img = cv2.imread("test5.jpg")
# convert to hsv
hsv = cv2.cvtColor(img,cv2.COLOR_BGR2HSV)
## mask of upper red (170,50,50) ~ (180,255,255)
## mask of lower red (0,50,50) ~ (10,255,255)
# threshold using inRange
range1 = (0,50,50)
range2 = (1,255,255)
mask = cv2.inRange(hsv,range1,range2)
mask = 255 - mask
# apply morphology opening to mask
kernel = np.ones((3,3), np.uint8)
mask = cv2.morphologyEx(mask, cv2.MORPH_ERODE, kernel)
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
# antialias mask
mask = cv2.GaussianBlur(mask, (0,0), sigmaX=3, sigmaY=3, borderType = cv2.BORDER_DEFAULT)
mask = skimage.exposure.rescale_intensity(mask, in_range=(127.5,255), out_range=(0,255))
result = img.copy()
result[mask==0] = (255,255,255)
# write result to disk
cv2.imwrite("test6.jpg", result)
This is one way to approach the problem in Python/OpenCV. But for red, it is very hard to do because red spans 0 hue, which also is the hue for gray and white and black, which you have in your image. The other issue you have is that skin tones has red shades, so you cannot pick too large of ranges for your colors. Also when dealing with red ranges, you need two sets, one for hues up to 180 and another for hues above 0.
Input:
import cv2
import numpy as np
# load image
img = cv2.imread('red_clothes.jpg')
# convert to HSV
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
h,s,v = cv2.split(hsv)
blue_hue = 120
red_hue = 0
# diff hue (blue_hue - red_hue)
diff_hue = blue_hue - red_hue
# create mask for red color in hsv
lower1 = (150,150,150)
upper1 = (180,255,255)
mask1 = cv2.inRange(hsv, lower1, upper1)
lower2 = (0,150,150)
upper2 = (30,255,255)
mask2 = cv2.inRange(hsv, lower2, upper2)
mask = cv2.add(mask1,mask2)
mask = cv2.merge([mask,mask,mask])
# apply morphology to clean mask
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (9,9))
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
# modify hue channel by adding difference and modulo 180
hnew = np.mod(h + diff_hue, 180).astype(np.uint8)
# recombine channels
hsv_new = cv2.merge([hnew,s,v])
# convert back to bgr
bgr_new = cv2.cvtColor(hsv_new, cv2.COLOR_HSV2BGR)
# blend with original using mask
result = np.where(mask==(255, 255, 255), bgr_new, img)
# save output
cv2.imwrite('red_clothes_mask.png', mask)
cv2.imwrite('red_clothes_hue_shift.png', bgr_new)
cv2.imwrite('red_clothes_red2blue.png', result)
# Display various images to see the steps
cv2.imshow('mask1',mask1)
cv2.imshow('mask2',mask2)
cv2.imshow('mask',mask)
cv2.imshow('bgr_new',bgr_new)
cv2.imshow('result',result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Mask:
Hue Shifted Image:
Blend between Input and Hue Shifted Image using Mask to blend:
So the result is speckled because of the black mixed with the red and from limited ranges due to skin color.
You can start with red, but the trick is to invert the image so red is now at hue 90 in OpenCV range and for example blue is at hue 30. So in Python/OpenCV, you can do the following:
Input:
import cv2
import numpy as np
# load image
img = cv2.imread('red_clothes.jpg')
# invert image
imginv = 255 - img
# convert to HSV
hsv = cv2.cvtColor(imginv, cv2.COLOR_BGR2HSV)
h,s,v = cv2.split(hsv)
blueinv_hue = 30 #(=120+180/2=210-180=30)
redinv_hue = 90 #(=0+180/2=90)
# diff hue (blue_hue - red_hue)
diff_hue = blueinv_hue - redinv_hue
# create mask for redinv color in hsv
lower = (80,150,150)
upper = (100,255,255)
mask = cv2.inRange(hsv, lower, upper)
mask = cv2.merge([mask,mask,mask])
# apply morphology to clean mask
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (9,9))
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
# modify hue channel by adding difference and modulo 180
hnew = np.mod(h + diff_hue, 180).astype(np.uint8)
# recombine channels
hsv_new = cv2.merge([hnew,s,v])
# convert back to bgr
bgrinv_new = cv2.cvtColor(hsv_new, cv2.COLOR_HSV2BGR)
# invert
bgr_new = 255 -bgrinv_new
# blend with original using mask
result = np.where(mask==(255, 255, 255), bgr_new, img)
# save output
cv2.imwrite('red_clothes_mask.png', mask)
cv2.imwrite('red_clothes_hue_shift.png', bgr_new)
cv2.imwrite('red_clothes_red2blue.png', result)
# Display various images to see the steps
cv2.imshow('mask',mask)
cv2.imshow('bgr_new',bgr_new)
cv2.imshow('result',result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Mask:
Red to Blue before masking:
Red to Blue after masking:
However, one is still limited by the fact that red is close to skin tones, so the range for red is limited.
Starting with a blue image rather than red allows one to use an expanded range for inRange() and do a better job in Python/OpenCV. Here is a change from blue to red.
Input:
import cv2
import numpy as np
# load image
img = cv2.imread('blue_clothes.jpg')
# convert to HSV
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
h,s,v = cv2.split(hsv)
red_hue = 0
blue_hue = 120
# diff hue (red_hue - blue_hue)
diff_hue = red_hue - blue_hue
# create mask for blue color in hsv
lower = (100,90,90)
upper = (140,255,255)
mask = cv2.inRange(hsv, lower, upper)
mask = cv2.merge([mask,mask,mask])
# apply morphology to clean mask
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (9,9))
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
# modify hue channel by adding difference and modulo 180
hnew = np.mod(h + diff_hue, 180).astype(np.uint8)
# recombine channels
hsv_new = cv2.merge([hnew,s,v])
# convert back to bgr
bgr_new = cv2.cvtColor(hsv_new, cv2.COLOR_HSV2BGR)
# blend with original using mask
result = np.where(mask==(255, 255, 255), bgr_new, img)
# save output
cv2.imwrite('blue_clothes_mask.png', mask)
cv2.imwrite('blue_clothes_hue_shift.png', bgr_new)
cv2.imwrite('blue_clothes_blue2red.png', result)
# Display various images to see the steps
cv2.imshow('mask',mask)
cv2.imshow('bgr_new',bgr_new)
cv2.imshow('result',result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Mask:
Blue to Red before masking:
Blue to Red after masking:
I am trying to segment lung CT images using Kmeans by using code below:
def process_mask(mask):
convex_mask = np.copy(mask)
for i_layer in range(convex_mask.shape[0]):
mask1 = np.ascontiguousarray(mask[i_layer])
if np.sum(mask1)>0:
mask2 = convex_hull_image(mask1)
if np.sum(mask2)>2*np.sum(mask1):
mask2 = mask1
else:
mask2 = mask1
convex_mask[i_layer] = mask2
struct = generate_binary_structure(3,1)
dilatedMask = binary_dilation(convex_mask,structure=struct,iterations=10)
return dilatedMask
def lumTrans(img):
lungwin = np.array([-1200.,600.])
newimg = (img-lungwin[0])/(lungwin[1]-lungwin[0])
newimg[newimg<0]=0
newimg[newimg>1]=1
newimg = (newimg*255).astype('uint8')
return newimg
def lungSeg(imgs_to_process,output,name):
if os.path.exists(output+'/'+name+'_clean.npy') : return
imgs_to_process = Image.open(imgs_to_process)
img_to_save = imgs_to_process.copy()
img_to_save = np.asarray(img_to_save).astype('uint8')
imgs_to_process = lumTrans(imgs_to_process)
imgs_to_process = np.expand_dims(imgs_to_process, axis=0)
x,y,z = imgs_to_process.shape
img_array = imgs_to_process.copy()
A1 = int(y/(512./100))
A2 = int(y/(512./400))
A3 = int(y/(512./475))
A4 = int(y/(512./40))
A5 = int(y/(512./470))
for i in range(len(imgs_to_process)):
img = imgs_to_process[i]
print(img.shape)
x,y = img.shape
#Standardize the pixel values
allmean = np.mean(img)
allstd = np.std(img)
img = img-allmean
img = img/allstd
# Find the average pixel value near the lungs
# to renormalize washed out images
middle = img[A1:A2,A1:A2]
mean = np.mean(middle)
max = np.max(img)
min = np.min(img)
kmeans = KMeans(n_clusters=2).fit(np.reshape(middle,[np.prod(middle.shape),1]))
centers = sorted(kmeans.cluster_centers_.flatten())
threshold = np.mean(centers)
thresh_img = np.where(img<threshold,1.0,0.0) # threshold the image
eroded = morphology.erosion(thresh_img,np.ones([4,4]))
dilation = morphology.dilation(eroded,np.ones([10,10]))
labels = measure.label(dilation)
label_vals = np.unique(labels)
regions = measure.regionprops(labels)
good_labels = []
for prop in regions:
B = prop.bbox
if B[2]-B[0]<A3 and B[3]-B[1]<A3 and B[0]>A4 and B[2]<A5:
good_labels.append(prop.label)
mask = np.ndarray([x,y],dtype=np.int8)
mask[:] = 0
for N in good_labels:
mask = mask + np.where(labels==N,1,0)
mask = morphology.dilation(mask,np.ones([10,10])) # one last dilation
imgs_to_process[i] = mask
m1 = imgs_to_process
convex_mask = m1
dm1 = process_mask(m1)
dilatedMask = dm1
Mask = m1
extramask = dilatedMask ^ Mask
bone_thresh = 180
pad_value = 0
img_array[np.isnan(img_array)]=-2000
sliceim = img_array
sliceim = sliceim*dilatedMask+pad_value*(1-dilatedMask).astype('uint8')
bones = sliceim*extramask>bone_thresh
sliceim[bones] = pad_value
x,y,z = sliceim.shape
if not os.path.exists(output):
os.makedirs(output)
img_to_save[sliceim.squeeze()==0] = 0
im = Image.fromarray(img_to_save)
im.save(output + name + '.png', 'PNG')
The problem is the segmented lung still contains white borderers like this:
Segmented lung (output):
Unsegmented lung (input):
The full code can be found in Google Colab Notebook. code.
And sample of the dataset is here.
For this problem, I don't recommend using Kmeans color quantization since this technique is usually reserved for a situation where there are various colors and you want to segment them into dominant color blocks. Take a look at this previous answer for a typical use case. Since your CT scan images are grayscale, Kmeans would not perform very well. Here's a potential solution using simple image processing with OpenCV:
Obtain binary image. Load input image, convert to grayscale, Otsu's threshold, and find contours.
Create a blank mask to extract desired objects. We can use np.zeros() to create a empty mask with the same size as the input image.
Filter contours using contour area and aspect ratio. We search for the lung objects by ensuring that contours are within a specified area threshold as well as aspect ratio. We use cv2.contourArea(), cv2.arcLength(), and cv2.approxPolyDP() for contour perimeter and contour shape approximation. If we have have found our lung object, we utilize cv2.drawContours() to fill in our mask with white to represent the objects that we want to extract.
Bitwise-and mask with original image. Finally we convert the mask to grayscale and bitwise-and with cv2.bitwise_and() to obtain our result.
Here is our image processing pipeline visualized step-by-step:
Grayscale -> Otsu's threshold
Detected objects to extract highlighted in green -> Filled mask
Bitwise-and to get our result -> Optional result with white background instead
Code
import cv2
import numpy as np
image = cv2.imread('1.png')
highlight = image.copy()
original = image.copy()
# Convert image to grayscale, Otsu's threshold, and find contours
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
contours = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
# Create black mask to extract desired objects
mask = np.zeros(image.shape, dtype=np.uint8)
# Search for objects by filtering using contour area and aspect ratio
for c in contours:
# Contour area
area = cv2.contourArea(c)
# Contour perimeter
peri = cv2.arcLength(c, True)
# Contour approximation
approx = cv2.approxPolyDP(c, 0.035 * peri, True)
(x, y, w, h) = cv2.boundingRect(approx)
aspect_ratio = w / float(h)
# Draw filled contour onto mask if passes filter
# These are arbitary values, may need to change depending on input image
if aspect_ratio <= 1.2 or area < 5000:
cv2.drawContours(highlight, [c], 0, (0,255,0), -1)
cv2.drawContours(mask, [c], 0, (255,255,255), -1)
# Convert 3-channel mask to grayscale then bitwise-and with original image for result
mask = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
result = cv2.bitwise_and(original, original, mask=mask)
# Uncomment if you want background to be white instead of black
# result[mask==0] = (255,255,255)
# Display
cv2.imshow('gray', gray)
cv2.imshow('thresh', thresh)
cv2.imshow('highlight', highlight)
cv2.imshow('mask', mask)
cv2.imshow('result', result)
# Save images
# cv2.imwrite('gray.png', gray)
# cv2.imwrite('thresh.png', thresh)
# cv2.imwrite('highlight.png', highlight)
# cv2.imwrite('mask.png', mask)
# cv2.imwrite('result.png', result)
cv2.waitKey(0)
A simpler approach to solve this problem is using morphological erosion. Its just that than you will have to tune in threshold values
How can I apply mask to a color image in latest python binding (cv2)? In previous python binding the simplest way was to use cv.Copy e.g.
cv.Copy(dst, src, mask)
But this function is not available in cv2 binding. Is there any workaround without using boilerplate code?
Here, you could use cv2.bitwise_and function if you already have the mask image.
For check the below code:
img = cv2.imread('lena.jpg')
mask = cv2.imread('mask.png',0)
res = cv2.bitwise_and(img,img,mask = mask)
The output will be as follows for a lena image, and for rectangular mask.
Well, here is a solution if you want the background to be other than a solid black color. We only need to invert the mask and apply it in a background image of the same size and then combine both background and foreground. A pro of this solution is that the background could be anything (even other image).
This example is modified from Hough Circle Transform. First image is the OpenCV logo, second the original mask, third the background + foreground combined.
# http://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_houghcircles/py_houghcircles.html
import cv2
import numpy as np
# load the image
img = cv2.imread('E:\\FOTOS\\opencv\\opencv_logo.png')
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
# detect circles
gray = cv2.medianBlur(cv2.cvtColor(img, cv2.COLOR_RGB2GRAY), 5)
circles = cv2.HoughCircles(gray, cv2.HOUGH_GRADIENT, 1, 20, param1=50, param2=50, minRadius=0, maxRadius=0)
circles = np.uint16(np.around(circles))
# draw mask
mask = np.full((img.shape[0], img.shape[1]), 0, dtype=np.uint8) # mask is only
for i in circles[0, :]:
cv2.circle(mask, (i[0], i[1]), i[2], (255, 255, 255), -1)
# get first masked value (foreground)
fg = cv2.bitwise_or(img, img, mask=mask)
# get second masked value (background) mask must be inverted
mask = cv2.bitwise_not(mask)
background = np.full(img.shape, 255, dtype=np.uint8)
bk = cv2.bitwise_or(background, background, mask=mask)
# combine foreground+background
final = cv2.bitwise_or(fg, bk)
Note: It is better to use the opencv methods because they are optimized.
import cv2 as cv
im_color = cv.imread("lena.png", cv.IMREAD_COLOR)
im_gray = cv.cvtColor(im_color, cv.COLOR_BGR2GRAY)
At this point you have a color and a gray image. We are dealing with 8-bit, uint8 images here. That means the images can have pixel values in the range of [0, 255] and the values have to be integers.
Let's do a binary thresholding operation. It creates a black and white masked image. The black regions have value 0 and the white regions 255
_, mask = cv.threshold(im_gray, thresh=180, maxval=255, type=cv.THRESH_BINARY)
im_thresh_gray = cv.bitwise_and(im_gray, mask)
The mask can be seen below on the left. The image on its right is the result of applying bitwise_and operation between the gray image and the mask. What happened is, the spatial locations where the mask had a pixel value zero (black), became pixel value zero in the result image. The locations where the mask had pixel value 255 (white), the resulting image retained its original gray value.
To apply this mask to our original color image, we need to convert the mask into a 3 channel image as the original color image is a 3 channel image.
mask3 = cv.cvtColor(mask, cv.COLOR_GRAY2BGR) # 3 channel mask
Then, we can apply this 3 channel mask to our color image using the same bitwise_and function.
im_thresh_color = cv.bitwise_and(im_color, mask3)
mask3 from the code is the image below on the left, and im_thresh_color is on its right.
You can plot the results and see for yourself.
cv.imshow("original image", im_color)
cv.imshow("binary mask", mask)
cv.imshow("3 channel mask", mask3)
cv.imshow("im_thresh_gray", im_thresh_gray)
cv.imshow("im_thresh_color", im_thresh_color)
cv.waitKey(0)
The original image is lenacolor.png that I found here.
Answer given by Abid Rahman K is not completely correct. I also tried it and found very helpful but got stuck.
This is how I copy image with a given mask.
x, y = np.where(mask!=0)
pts = zip(x, y)
# Assuming dst and src are of same sizes
for pt in pts:
dst[pt] = src[pt]
This is a bit slow but gives correct results.
EDIT:
Pythonic way.
idx = (mask!=0)
dst[idx] = src[idx]
The other methods described assume a binary mask. If you want to use a real-valued single-channel grayscale image as a mask (e.g. from an alpha channel), you can expand it to three channels and then use it for interpolation:
assert len(mask.shape) == 2 and issubclass(mask.dtype.type, np.floating)
assert len(foreground_rgb.shape) == 3
assert len(background_rgb.shape) == 3
alpha3 = np.stack([mask]*3, axis=2)
blended = alpha3 * foreground_rgb + (1. - alpha3) * background_rgb
Note that mask needs to be in range 0..1 for the operation to succeed. It is also assumed that 1.0 encodes keeping the foreground only, while 0.0 means keeping only the background.
If the mask may have the shape (h, w, 1), this helps:
alpha3 = np.squeeze(np.stack([np.atleast_3d(mask)]*3, axis=2))
Here np.atleast_3d(mask) makes the mask (h, w, 1) if it is (h, w) and np.squeeze(...) reshapes the result from (h, w, 3, 1) to (h, w, 3).
I want to remove the background by using the mask image. Now, I have already get the mask image.I try to let the value of the original image's background become 0 where the value of mask is 0. But the result is very bad. How can I solve this problem. Thank you
from skimage import io
import numpy as np
img = io.imread("GT06.jpg")
mask = io.imread("GT03.png")
mask2 = np.where((mask==0),0,1).astype('uint8')
img = img*mask2[:,:,np.newaxis]
io.imshow(img)
io.show()
GT06.jpg
GT03.png
This results in:
I want to get the foreground like this:
The problem is that your mask isn't pure black and white, i.e. all 0 or 255 changing you mask two generation to:
mask2 = np.where((mask<200),0,1).astype('uint8')
results in:
You could either play with the mask or the threshold number - I used 200.
In Python you could use OpenCV. Here are instructions to install OpenCV in Python if you don't have it in your system. I think you could do the same with other libraries, the procedure will be the same, the trick is to invert the mask and apply it to some background, you will have your masked image and a masked background, then you combine both.
The image1 is your image masked with the original mask, image2 is the background image masked with the inverted mask, and image3 is the combined image. Important. image1, image2 and image3 must be of the same size and type. The mask must be grayscale.
import cv2
import numpy as np
# opencv loads the image in BGR, convert it to RGB
img = cv2.cvtColor(cv2.imread('E:\\FOTOS\\opencv\\iT5q1.png'),
cv2.COLOR_BGR2RGB)
# load mask and make sure is black&white
_, mask = cv2.threshold(cv2.imread('E:\\FOTOS\\opencv\\SH9jL.png', 0),
0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
# load background (could be an image too)
bk = np.full(img.shape, 255, dtype=np.uint8) # white bk, same size and type of image
bk = cv2.rectangle(bk, (0, 0), (int(img.shape[1] / 2), int(img.shape[0] / 2)), 0, -1) # rectangles
bk = cv2.rectangle(bk, (int(img.shape[1] / 2), int(img.shape[0] / 2)), (img.shape[1], img.shape[0]), 0, -1)
# get masked foreground
fg_masked = cv2.bitwise_and(img, img, mask=mask)
# get masked background, mask must be inverted
mask = cv2.bitwise_not(mask)
bk_masked = cv2.bitwise_and(bk, bk, mask=mask)
# combine masked foreground and masked background
final = cv2.bitwise_or(fg_masked, bk_masked)
mask = cv2.bitwise_not(mask) # revert mask to original