I wanted to work on a small project to challenge my computer vision and image processing skills. I came across a project where I want to remove the hidden marks from the image. Hidden here refers to the watermarks that are not easily visible in rgb space but when you convert into hsv or some other space the marks become visible.
Here's one example:
BGR SPACE:
HSV SPACE:
I've tried different ways but was able to implement a solution that would remove those watermarks from the image. I am posting this question here to get different ideas to tackle this problem.
What I have tried:
I have tried various approaches but none of them worked, sharing the code might not help. It is not necessary to provide code for it, a pseudo code, idea or any lead would be appreciated.
I noticed that the hidden marks are all the colors similar to RGB(90,94,105). And when I showed R, G, and B separately I noticed that the watermarks were only visible in B channel. I thought that if adjust/remove the marks in B channel and merge the image again, may be I could get better results.
Code:
b,g,r = cv2.split(img)
b = b//2;
r = cv2.merge((r,g,b))
cv2.imshow("image",r)
Problems: This doesn't does solve the problem, it did make the colors little dimmer but the image colors were also disturbed.
I tried playing around with B channel to see if could accomplish something.
I also noticed that if we convert the image to LUV space then the marks are visible in V space.
This might be a possible approach. The underlying idea is that there are edges visible in the HSV channel that are not present in the original image. Here are the H, S and V channels side-by-side:
So if we find the edges in the original image and the edges in the HSV image and difference them, the watermarking should show up. That can then be used as a mask to do in-painting in the original image with OpenCV inpaint.
I am just using ImageMagick here in Terminal, but it could all be done equally with OpenCV, PIL or scikit-image:
# Detect edges visible in original image and auto-level
convert watermarked.png -colorspace gray -auto-level -canny 0x1+1%+3% -auto-level RGB-edges.png
# Find visible edges in H, S and V colourspace, generate mean across all three and auto-level
convert watermarked.png -colorspace hsv -separate -canny 0x1+1%+3% -evaluate-sequence mean -auto-level HSV-edges.png
# Find changemask between the two sets of edges
convert RGB-edges.png HSV-edges.png -compose changemask -composite result.png
The idea is that the watermarking is now identified in black, so use the black areas (maybe morphologically closed) as a mask in OpenCV to inpaint - see link above.
I didn't find any answer that completely solved the question. I appreciate everyone's effort though (Thank you).
I did something on my own and would like to share. It results in little quality loss (a little bluish blurriness) but successfully removes the watermarks. The solution is very simple but took time to analyze the image.
I WOULD BE VERY GLAD IF SOMEONE CAN EXTEND THIS APPROACH AND COME UP WITH SOMETHING EVEN BETTER
I observed that the watermarks were only visible in B space (out of RGB) and there were no traces of watermarks in R and G space.
B space:
I also red somewhere that blue light contributes little to the overall image compared to R and G channel so here's what I decided to do.
Blur the B channel by a large enough amount to remove traces of those patterns. Here's how the B channel would appear afterwards:
Finally, merge the image with the new B channel, previous R and previous G channel. Here's how the RGB channel would appear afterwards:
The advantage of using approach is that the traces are gone.
The only disadvantage is that the bluish and purplish colors appear at the black edges and the image is a little bluish in general.
My Code:
import cv2
from matplotlib import pyplot as plt
import numpy as np
img = cv2.imread("img.png")
b, g, r = cv2.split(img) # split into B,G,R spaces
b = cv2.GaussianBlur(b, None, 8)
plt.imshow(cv2.merge((r,g,b)), cmap='gray')
Here is a slight variation and extension of your processing in Python/OpenCV.
The main difference is that I use the median rather than a blurring and that I try to extract the black lines and impose them on the median before recombining.
Input:
import cv2
import numpy as np
# read image
img = cv2.imread("cartoon_hidden_marks.png")
# separate channels
b,g,r = cv2.split(img)
# median filter blue
median = cv2.medianBlur(b, 21)
# threshold blue image to extract black lines
thresh = cv2.threshold(b, 20, 255, cv2.THRESH_BINARY)[1]
# apply thresh to median
b_new = cv2.bitwise_and(median, thresh)
# combine b_new, g, b
img_new = cv2.merge([b_new,g,r])
# write results to disk
cv2.imwrite("cartoon_hidden_marks_median.jpg", median)
cv2.imwrite("cartoon_hidden_marks_thresh.jpg", thresh)
cv2.imwrite("cartoon_hidden_marks_new_blue.jpg", b_new)
cv2.imwrite("cartoon_hidden_marks_result.png", img_new)
# display it
cv2.imshow("median", median)
cv2.imshow("thresh", thresh)
cv2.imshow("b_new", b_new)
cv2.imshow("img_new", img_new)
cv2.waitKey(0)
Blue channel median:
Blue channel threshold (for black lines):
New blue channel:
Result:
Many of the erroneous blue lines are now black, but not all. Increasing the threshold would have gotten more black lines, but then the hidden marks would have appeared again in part.
If you have managed to isolate the watermarks in any channel, you should be able to threshold it and create a binary mask. Then you could use inpainting to fill the gaps with something like:
clean_image = cv2.inpaint(marked_image, mask_of_marks, 3, cv2.INPAINT_TELEA)
Another trivial solution in Python/OpenCV is simply to replace the green channel for the blue channel, since most of the green channel is about the same intensity distribution as that of the blue channel.
Input:
import cv2
import numpy as np
# read image
img = cv2.imread("cartoon_hidden_marks.png")
# separate channels
b,g,r = cv2.split(img)
# combine replacing b with g
img_new = cv2.merge([g,g,r])
# write results to disk
cv2.imwrite("cartoon_hidden_marks_result2.png", img_new)
# display it
cv2.imshow("result", img_new)
cv2.waitKey(0)
Result:
The issue is that the coat and the green tree are slightly different color and texture.
One might try modifying a copy of the green channel image to have the mean and standard-deviation as the blue channel to fix the coat issue. For the green tree, it is outside the region of the watermark, so one could mask that using inRange for the green tree color and then replace the blue channel image's tree in the copy of the green channel. Then recombine the modified green channel in place of the blue channel.
Related
I am trying to detect edges in images of a video, but edge detection methods such as canny does not work very well might be due to in similarity between boxes's color and floor color or brightness so I want to find a way to make all red and blue boxes look as white as possible, or may be the best way to detect edges as perfect as possible for every frame since that is the ultimate goal.
I recommend you using color tracking then.
Convert to HSV
cv2.bgr2hsv
Why hsv? eventhough the brightness change, u can still detect that color
Filtering
You can use cv2.inrange
Noise cancelling
Use cv2.Gaussianblur
Contouring
use cv2.findContours
Find the edge
use ur method
Repeat this step for every color of your box
Hope this help
Just to complete my comment in your question. One can use HSV/HLS colorspaces and use inRanges with the Hue channel. For example:
import numpy as np
import cv2
# load image and threshold it
original = cv2.imread("a.jpg")
hsvframe = cv2.cvtColor(original, cv2.COLOR_BGR2HLS)
mask = cv2.inRange(hsvframe, (160,40,40), (180, 255, 255))
mask = mask + cv2.inRange(hsvframe, (0,40,40), (12, 255, 255)) # color red is at the beginning and end of the hue wheel
original[mask==255] = (0,255,0)
cv2.imshow("image", original)
cv2.waitKey(0)
cv2.destroyAllWindows()
Things to remember, Hue goes from 0-180 in np.uint8. This means if you need hue 300-360 the limits will be 150-180. The other two values are 0-255 where 255 = 100%.
The result of this small code is:
It is not perfect, but one can refine it using the methods suggested by the other answer. I hope this helps.
I am really new to opencv and a beginner to python.
I have this image:
I want to somehow apply proper thresholding to keep nothing but the 6 digits.
The bigger picture is that I intend to try to perform manual OCR to the image for each digit separately, using the k-nearest neighbours algorithm on a per digit level (kNearest.findNearest)
The problem is that I cannot clean up the digits sufficiently, especially the '7' digit which has this blue-ish watermark passing through it.
The steps I have tried so far are the following:
I am reading the image from disk
# IMREAD_UNCHANGED is -1
image = cv2.imread(sys.argv[1], cv2.IMREAD_UNCHANGED)
Then I'm keeping only the blue channel to get rid of the blue watermark around digit '7', effectively converting it to a single channel image
image = image[:,:,0]
# openned with -1 which means as is,
# so the blue channel is the first in BGR
Then I'm multiplying it a bit to increase contrast between the digits and the background:
image = cv2.multiply(image, 1.5)
Finally I perform Binary+Otsu thresholding:
_,thressed1 = cv2.threshold(image,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
As you can see the end result is pretty good except for the digit '7' which has kept a lot of noise.
How to improve the end result? Please supply the image example result where possible, it is better to understand than just code snippets alone.
You can try to medianBlur the gray(blur) image with different kernels(such as 3, 51), divide the blured results, and threshold it. Something like this：
#!/usr/bin/python3
# 2018/09/23 17:29 (CST)
# (中秋节快乐)
# (Happy Mid-Autumn Festival)
import cv2
import numpy as np
fname = "color.png"
bgray = cv2.imread(fname)[...,0]
blured1 = cv2.medianBlur(bgray,3)
blured2 = cv2.medianBlur(bgray,51)
divided = np.ma.divide(blured1, blured2).data
normed = np.uint8(255*divided/divided.max())
th, threshed = cv2.threshold(normed, 100, 255, cv2.THRESH_OTSU)
dst = np.vstack((bgray, blured1, blured2, normed, threshed))
cv2.imwrite("dst.png", dst)
The result：
Why not just keep values in the image that are above a certain threshold?
Like this:
import cv2
import numpy as np
img = cv2.imread("./a.png")[:,:,0] # the last readable image
new_img = []
for line in img:
new_img.append(np.array(list(map(lambda x: 0 if x < 100 else 255, line))))
new_img = np.array(list(map(lambda x: np.array(x), new_img)))
cv2.imwrite("./b.png", new_img)
Looks great:
You could probably play with the threshold even more and get better results.
It doesn't seem easy to completely remove the annoying stamp.
What you can do is flattening the background intensity by
computing a lowpass image (Gaussian filter, morphological closing); the filter size should be a little larger than the character size;
dividing the original image by the lowpass image.
Then you can use Otsu.
As you see, the result isn't perfect.
I tried a slightly different approach then Yves on the blue channel:
Apply median filter (r=2):
Use Edge detection (e.g. Sobel operator):
Automatic thresholding (Otsu)
Closing of the image
This approach seems to make the output a little less noisy. However, one has to address the holes in the numbers. This can be done by detecting black contours which are completely surrounded by white pixels and simply filling them with white.
I have two images, one with only background and the other with background + detectable object (in my case its a car). Below are the images
I am trying to remove the background such that I only have car in the resulting image. Following is the code that with which I am trying to get the desired results
import numpy as np
import cv2
original_image = cv2.imread('IMG1.jpg', cv2.IMREAD_COLOR)
gray_original = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY)
background_image = cv2.imread('IMG2.jpg', cv2.IMREAD_COLOR)
gray_background = cv2.cvtColor(background_image, cv2.COLOR_BGR2GRAY)
foreground = np.absolute(gray_original - gray_background)
foreground[foreground > 0] = 255
cv2.imshow('Original Image', foreground)
cv2.waitKey(0)
The resulting image by subtracting the two images is
Here is the problem. The expected resulting image should be a car only.
Also, If you take a deep look in the two images, you'll see that they are not exactly same that is, the camera moved a little so background had been disturbed a little. My question is that with these two images how can I subtract the background. I do not want to use grabCut or backgroundSubtractorMOG algorithm right now because I do not know right now whats going on inside those algorithms.
What I am trying to do is to get the following resulting image
Also if possible, please guide me with a general way of doing this not only in this specific case that is, I have a background in one image and background+object in the second image. What could be the best possible way of doing this. Sorry for such a long question.
I solved your problem using the OpenCV's watershed algorithm. You can find the theory and examples of watershed here.
First I selected several points (markers) to dictate where is the object I want to keep, and where is the background. This step is manual, and can vary a lot from image to image. Also, it requires some repetition until you get the desired result. I suggest using a tool to get the pixel coordinates.
Then I created an empty integer array of zeros, with the size of the car image. And then I assigned some values (1:background, [255,192,128,64]:car_parts) to pixels at marker positions.
NOTE: When I downloaded your image I had to crop it to get the one with the car. After cropping, the image has size of 400x601. This may not be what the size of the image you have, so the markers will be off.
Afterwards I used the watershed algorithm. The 1st input is your image and 2nd input is the marker image (zero everywhere except at marker positions). The result is shown in the image below.
I set all pixels with value greater than 1 to 255 (the car), and the rest (background) to zero. Then I dilated the obtained image with a 3x3 kernel to avoid losing information on the outline of the car. Finally, I used the dilated image as a mask for the original image, using the cv2.bitwise_and() function, and the result lies in the following image:
Here is my code:
import cv2
import numpy as np
import matplotlib.pyplot as plt
# Load the image
img = cv2.imread("/path/to/image.png", 3)
# Create a blank image of zeros (same dimension as img)
# It should be grayscale (1 color channel)
marker = np.zeros_like(img[:,:,0]).astype(np.int32)
# This step is manual. The goal is to find the points
# which create the result we want. I suggest using a
# tool to get the pixel coordinates.
# Dictate the background and set the markers to 1
marker[204][95] = 1
marker[240][137] = 1
marker[245][444] = 1
marker[260][427] = 1
marker[257][378] = 1
marker[217][466] = 1
# Dictate the area of interest
# I used different values for each part of the car (for visibility)
marker[235][370] = 255 # car body
marker[135][294] = 64 # rooftop
marker[190][454] = 64 # rear light
marker[167][458] = 64 # rear wing
marker[205][103] = 128 # front bumper
# rear bumper
marker[225][456] = 128
marker[224][461] = 128
marker[216][461] = 128
# front wheel
marker[225][189] = 192
marker[240][147] = 192
# rear wheel
marker[258][409] = 192
marker[257][391] = 192
marker[254][421] = 192
# Now we have set the markers, we use the watershed
# algorithm to generate a marked image
marked = cv2.watershed(img, marker)
# Plot this one. If it does what we want, proceed;
# otherwise edit your markers and repeat
plt.imshow(marked, cmap='gray')
plt.show()
# Make the background black, and what we want to keep white
marked[marked == 1] = 0
marked[marked > 1] = 255
# Use a kernel to dilate the image, to not lose any detail on the outline
# I used a kernel of 3x3 pixels
kernel = np.ones((3,3),np.uint8)
dilation = cv2.dilate(marked.astype(np.float32), kernel, iterations = 1)
# Plot again to check whether the dilation is according to our needs
# If not, repeat by using a smaller/bigger kernel, or more/less iterations
plt.imshow(dilation, cmap='gray')
plt.show()
# Now apply the mask we created on the initial image
final_img = cv2.bitwise_and(img, img, mask=dilation.astype(np.uint8))
# cv2.imread reads the image as BGR, but matplotlib uses RGB
# BGR to RGB so we can plot the image with accurate colors
b, g, r = cv2.split(final_img)
final_img = cv2.merge([r, g, b])
# Plot the final result
plt.imshow(final_img)
plt.show()
If you have a lot of images you will probably need to create a tool to annotate the markers graphically, or even an algorithm to find markers automatically.
The problem is that you're subtracting arrays of unsigned 8 bit integers. This operation can overflow.
To demonstrate
>>> import numpy as np
>>> a = np.array([[10,10]],dtype=np.uint8)
>>> b = np.array([[11,11]],dtype=np.uint8)
>>> a - b
array([[255, 255]], dtype=uint8)
Since you're using OpenCV, the simplest way to achieve your goal is to use cv2.absdiff().
>>> cv2.absdiff(a,b)
array([[1, 1]], dtype=uint8)
I recommend using OpenCV's grabcut algorithm. You first draw a few lines on the foreground and background, and keep doing this until your foreground is sufficiently separated from the background. It is covered here: https://docs.opencv.org/trunk/d8/d83/tutorial_py_grabcut.html
as well as in this video: https://www.youtube.com/watch?v=kAwxLTDDAwU
I was doing some research about how can i crop the dress in this image (see image1) using python and some other libraries, so i need to do this for different images with many models on the photo, they will have different sizes and shapes so i need to do something generic that could take the image, analize it and remove all but the dress,
image1
I have a code that takes this image and do some mask around the model's shape and put the alpha channel so i get this (image2):
image2
As you can see this is the result of my code, but is not what i need, i really need to remove all the colors around the model, if possible all the colors around the dress, and need to be generic.. i.e. should work with different models that have different shapes and sizes
this is the code i have written on python using PIL and numpy libraries, i was using python 3.4
import numpy
from numpy import array
from PIL import Image
#import cv2
# read image as RGB and add alpha (transparency)
im = Image.open("one.jpg").convert("RGBA")
# convert to numpy (for convenience)
imArray = numpy.asarray(im)
# create mask (zeros + circle with ones)
center = (100,100)
radius = 100
mask = numpy.zeros((imArray.shape[0],imArray.shape[1]))
for i in range(imArray.shape[0]):
for j in range(imArray.shape[1]):
#if (i-center[0])**2 + (j-center[0])**2 < radius**2:
# mask[i,j] = 1
if ((j > 110 and j<240 and i>65 ) or (j > 440 and j<580 and i>83 )):
mask[i, j] = 1
"""
lower = numpy.array([0,0,0])
upper = numpy.array([15, 15, 15])
shapeMask = cv2.inRange(imArray, lower, upper)
"""
# assemble new image (uint8: 0-255)
newImArray = numpy.empty(imArray.shape,dtype='uint8')
# colors (three first columns, RGB)
newImArray[:,:,:3] = imArray[:,:,:3]
# transparency (4th column)
newImArray[:,:,3] = mask*255
# back to Image from numpy
newIm = Image.fromarray(newImArray, "RGBA")
newIm.save("one2.png")
The result should be a PNG image with all transparent except the model, or the dress if possible
As you can see im only making a static mask that always will be in the same place, and it is rectangular, not adjusted to the model, let me know if you need more explanation of what i need
Thanks a lot!
cesar
This is a very hard problem, especially when you do not know what the background is going to be and when the background has shadows.
The netting of the dress is also going to be lost in part or whole as might the areas between the body and the arms.
Here is an attempt using ImageMagick. But OpenCV has similar commands.
Input:
First, blur the image slightly and then extract the Hue channel from HCL colorspace.
Second I change all white colors within a tolerance of 30% to black.
Third I perform Otsu thresholding using one of my scripts.
Fourth I do a small amount of morphology close.
Fifth I use connected components processing to remove all regions smaller than 150 pixels in area. In OpenCV, that would be blob detection (SimpleBlobDetection) and invert (negate) the result as a mask.
Last, I put the mask into the alpha channel of the input to make the background transparent (which will show up white here).
convert image.jpg -blur 0x1 -colorspace HCL -channel r -separate hue.png
convert hue.png -fuzz 30% -fill black -opaque white filled.png
otsuthresh -g save filled.png thresh.png
convert thresh.png -morphology open disk:1 morph.png
convert morph.png -type bilevel \
-define connected-components:mean-color=true \
-define connected-components:area-threshold=150 \
-connected-components 4 \
-negate \
mask.png
convert image.jpg mask.png -alpha off -compose copy_opacity -composite result.png
Here are the image for the steps:
Hue Image:
Filled Image after converting white to black:
Otsu Thresholded Image:
Mask:
Result:
As you can see, the result is not very good at keeping to the outline of the woman and the dress, especially in the hair and the netting of the dress.
You might investigate OpenCV GrabCut Foreground Extaction at https://docs.opencv.org/3.4/d8/d83/tutorial_py_grabcut.html
If you can assume the background is fairly simple, (uniform in color, or only nearly horizontal lines) you could do edge detection, and the remove all pixels that's outside the first occuring edge.
Any edge detection filter should be sufficient, But I would probably go for a simple high pass filter, that enhances vertical edges only.
You'r merely trying to figure out where the models silhouette is!
Then remove all the pixels from the frame, going inwards, till the first edge is encountered. (cleans up background outside model).
To remove holes between arms and dress etc.. Median the color value of the removed pixels, to get the background color for this row, then remove pixels with a color value close to the found mean on the remainder of the row.
removals should be done via building a mask image, and then subtract it from the image, as the mask can be used for an opacity / alpha channel afterwards.
risks:
if dress or model is too close in colour to the background, holes will appear in the model/dress.
patterns in background disturbs algorithm and leaves rows untouched.
noise in the background can cause the removal or colour value to be set from pixels close to the frame only.
some of those problems can be minimized by opening and closing the deletion mask.
others by a spacial median filter prior to edge detection.
First step is to calculate the background color(s). Get a block of 50*50 find the variance, shift 10-20 pixels to right and get another block, calculate its variance as well and many more. Store the variances in an array. (and their means as well).
The ones with lowest variance are background colors, you will see bunch of those. After finding the background color, choose 5*5 blocks and if the variance is very small and its mean is equal to one of the backgrounds (i.e similar characteristic), then make it white or do whatever you want.
That is just my intuition, I'm not professional about image processing.
You can give this a try in order to extract dress from image of a model.
The link is github repo of image-conditional image generation model called PixelDTGAN. This model will perform a challenging task of generating a piece of clothing from an input image of a dressed person
This model transfers an input domain to a target domain in semantic level, and generates the target image in pixel level.
To generate realistic target images, the real/fake-discriminator is used as in Generative Adversarial Nets, a domain-discriminator is used to make the generated image relevant to the input image.
I'm loading and saving out images with PIL just fine but I can't seem to change the "overall" hue of a given image ~ google and here revealed an answer, sort of, with the numpy module, but thats not an option for me
It should be quite simple, given a gray image with alpha, I'd like to make it's hue red
I think you want a mono-hue image. Is this true?
It's not clear what you want done with the existing bands (alpha and greyscale/level). Do you want alpha to remain alpha and the greyscale to become red saturation? Do you want the alpha to become your red saturation? Do you want greyscale to be the image lightness and the alpha to become the saturation?
Edit:
I've changed the output based on your comment. You wanted the darkest shade of the greyscale band to represent fully saturated red and the lightest grey to represent white (in other words full-saturated with all colors). You also indicated that you wanted alpha to be preserved as alpha in the output. I've made that change too.
This is possible with some band swapping:
import Image
# get an image that is greyscale with alpha
i = Image.open('hsvwheel.png').convert('LA')
# get the two bands
L,A = i.split()
# a fully saturated band
S, = Image.new('L', i.size, 255).split()
# re-combine the bands
# this keeps tha alpha channel in the new image
i2 = Image.merge('RGBA', (S,L,L,A))
# save
i2.save('test.png')