After reading the docs and searching all over the internet I still do not understand how to interpret the output of the matchTemplate function from openCV.
What I understand:
result = cv2.matchTemplate(img, template, cv2.TM_CCOEFF_NORMED)
I understand that I get kind of a matrix with a matching value for every part in the picture. Each element in this matrix determines how much similarity it shows to the template.
e.g. I can filter all locations that have a matching value below 0.7 with
numpy.where(result >= 0.7)
What I do not understand is how this information is stored in the output I get from the matchTemplate function and how the position of the match can be extracted from the output.
Basically what I wanna do is match several templates to one image and then determine which template matches best to which location (has the max. matchingValue of all applied templates for a location).
My idea is to extract the matching value into a matrix for every template and then compare the matrices (their elements) to one another to find the best match.
Thanks for helping and please correct me where I'm wrong,
Greetings Don
What I do not understand is how this information is stored in the output I get from the matchTemplate function and how the position of the match can be extracted from the output.
This result will return the possibility of each pixel in the image like to top corner pixel of template
When you do this
loc =numpy.where(result >= 0.7)
We will filter out the possibility with the above method. We will get the x coordinate, y coordinate for the pixel of the image which the possibility of like to top corner of template greater than 0.7
#(array([202, 203, 203, 203, 204]), array([259, 258, 259, 260, 259]))
Now we get the image locations which are like to top left corner template with the possibility of higher than or equal to 0.7
In our example output, you can see that we can get plenty of matching points based on our threshold. we need to loop them to find each location.
since we know that the loc variable is a tuple with two NumPy array(Y coordinate NumPy array and X coordinate NumPy array), We need to unpack the tuple and reverse the order of arrays to get the actual places of the templates as follow.
Basically what I wanna do is match several templates to one image and then determine which template matches best to which location (has the max. matchingValue of all applied templates for a location).
The problem is, each result is not in the same shape since it depends on the template height and width.
What we can easily do is make all templates to the same shape( which can approximate for all templates) using cv2.resize method
'''
my original template shapes
img (1256, 1300)
tempate (215, 223)
tempate (217, 204)
tempate (207, 203)
width =220
height = 225
temp = cv2.resize(temp,(width,height),cv2.INTER_CUBIC)
This one makes our result in the same shape as below
res = cv2.matchTemplate(gray, temp,cv2.TM_CCOEFF_NORMED)
print(res.shape) #output (1032, 1081)
**My Approach**
1. I set the threshold to 0.7 (your choice) less than this threshold values i will set them into zero
```
res[res
I found the maximum possibility array with all results list. All res are added to results list
maximum_values_array = np.maximum(*results)
I found which res has the maximum value for that particular location
maximum_value_contains_array =np.array(results).argmax(axis=0)
I iterate each possibility value greater than zero(threshold>0.7). select the colour based on which array has that maximum value.
for i in range(len(maximum_values_array)):
for j in range(len(maximum_values_array[i])):
if maximum_values_array[i][j]>0:
colour = colours[maximum_value_contains_array[i][j]]
top_lect = (j,i)
bottom_right = (j+width, i+height)
cv2.rectangle(img,top_lect,bottom_right, colour, 2)
Full python code
import cv2
import numpy as np
img = cv2.imread('test.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
templates = [cv2.imread('blue_temp.jpg', 0), cv2.imread('yellow_temp.jpg', 0), cv2.imread('red_temp.jpg', 0)]
colours =[(255,0,0), (0,255,0),(0,0,255)]
results =[]
for temp in templates:
print(temp.shape)
width =205
height = 205
# temp = cv2.resize(temp,(width,height),cv2.INTER_CUBIC)
res = cv2.matchTemplate(gray, temp,cv2.TM_CCOEFF_NORMED)
res[res<0.9] =0
results.append(res)
maximum_values_array = np.maximum(*results)
maximum_value_contains_array =np.array(results).argmax(axis=0)
print(maximum_values_array.shape)
print(maximum_value_contains_array.shape)
for i in range(len(maximum_values_array)):
for j in range(len(maximum_values_array[i])):
if maximum_values_array[i][j]>0:
print(maximum_values_array[i][j])
colour = colours[maximum_value_contains_array[i][j]]
top_lect = (j,i)
bottom_right = (j+width, i+height)
cv2.rectangle(img,top_lect,bottom_right, colour, 2)
cv2_imshow(img)
cv2.imwrite('output.png', img)
You can use the following code:
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)
When using cv2.TM_CCOEFF_NORMED max_loc will be the location of the template in your img. And max_val will be the correlation of the match
It simply slides the template image over the input image (as in 2D convolution) and compares the template and patch of input image under the template image.
If input image is of size (WxH) and template image is of size (wxh), output image will have a size of (W-w+1, H-h+1).
Related
Problem Statement: after successfully getting the boundary box around the object in yolo, i wanted to separate the background from the object itself.
My Solution: i have an RGB-D camera that returns a depth map as well as the image (image is given to yolo obv) , using the depth map , i made a simple function to get the depths (rounded) and how many pixels have that same value
def GetAllDepthsSortedMeters(depth_image_ocv):
_depth = depth_image_ocv[np.isfinite(depth_image_ocv)]
_depth= -np.sort(-depth_image_ocv)[:int(len(_depth)/2)]
_depth= np.round(_depth,1)
unique, counts = np.unique(_depth, return_counts=True)
return dict(zip(counts, unique))
and plotting them, i noticed that there are dominant peaks and the rest lay around them, after some filtering i was able to successfully get those peaks each time.
#get the values of depths and their number of occurences
counts,values = GetKeysAndValues(_depths)
#find the peaks of depths in those values
peaks = find_peaks_cwt(counts, widths=np.ones(counts.shape)*2)-1
using those peaks, i was able to segment the required object from the background by checking what peaks is this value close to, and make a mask for each peak(and pixels around it).
def GetAcceptedMasks(h,w,depth_map,depths_of_accepted_peaks,accepted_masks):
prev=None
prev_index=None
for pos in product(range(h), range(w)):
pixel = depth_map.item(pos)
if ( (prev is not None) and (round(prev,1) == round(pixel,1)) ):
accepted_masks[prev_index][pos[0],pos[1]]= 255
else:
_temp_array = abs(depths_of_accepted_peaks-pixel)
_min = np.amin(_temp_array)
_ind = np.where( _temp_array == _min )[0][0]
accepted_masks[_ind][pos[0],pos[1]]= 255
prev_index = _ind
prev = pixel
return accepted_masks
after passing the image through YOLOv3 and applying the filtering and depth segmentation, it takes 0.8s which is far from optimal,
it's mostly result of above funcution, any help would be amazing. thank you
this is masks i get at the end
Mask1-Of-Closest-Depth
Mask2-Of-2nd-Closest-Depth
Mask3-Of-3rd-Closest-Depth
Edit:
Example of distance:
[0.60000002 1.29999995 1.89999998]
Example of DepthMap when show with imshow:
Example of Depth Map
Here's a way to do it.
Make an array of floats the same height and width as your image, and with the final dimension equal to the number of unique depths you want to identify
At each pixel location, calculate the distance to each of the three desired depths and store in the final dimension
Use np.argmin(..., axis=2) to select the nearest depth of the three
I am not at a computer to test, and your image is not your actual image but rather a picture of it with window decorations and title bar and different values, but something like this:
import cv2
# Load the image as greyscale float - so we can store positive and negative distances
im = cv2.imread('depth.png', cv2.IMREAD_GRAYSCALE).astype(np.float)
# Make list of the desired depths
depths = [255, 181, 125]
# Make array with distance to each depth
d2each = np.zeros(((im.shape[0],im.shape[1],len(depths)), dtype=np.float)
for i in range(len(depths)):
d2each[...,i] = np.abs(im - depths[i])
# Now let Numpy choose nearest of three distances
mask = np.argmin(d2each, axis=2)
Another way, is to range test the distances. Load the image as above:
# Make mask of pixels matching first distance
d0 = np.logical_and(im>100, im<150)
# Make mask of pixels matching second distance
d1 = np.logical_and(im>180, im<210)
# Make mask of pixels matching third distance
d2 = im >= 210
Those masks will be logical (i.e. True/False), but if you want to make them black and white, just multiply them by 255 and cast with mask0 = d0.astype(np.uint8)
Another approach could be to use K-means clustering.
I have one template and several images. So the problem is to find out is this template in the image or not. I wrote some loop, but I think it doesn't return Boolean value...
for i in images:
res = cv2.matchTemplate(i,templateDealer,cv2.TM_CCOEFF_NORMED)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)
top_left = max_loc
bottom_right = (top_left[0] + w, top_left[1] + h)
cv2.rectangle(i, top_left, bottom_right, (0, 255, 0), 2)
result.append(res)
Please help me to make it...
matchTemplate returns a 2D array with the match values for each pixel location - how well the template matches that location.
cv2.minMaxLoc(res) returns the value and location of the best match in the image. It may not be an actual match when you compare it visually, but it is the highest value the algorithm returned. You can compare this highest value over multiple images, the highest overall is the one you seek.
Note: you should not use a normalizing algorithm for this, use cv2.TM_CCOEFF instead of cv2.TM_CCOEFF_NORMED
What you need to do is append a tuple that holds the highest value and it's location:
result.append((max_val, max_loc)).
After you have processed all images, find the highest max_val, and draw a rectangle using it's max_loc
minMaxLoc will always give you something.
It totally depends on ur task. There are a few possibility I can think of
(1) You are looking for only 1 image of the most likely case among the N image.
In this case, concatenated all Image across one direction. Then run standard opencv tempalte matching and minmaxloc to find the most likely loation
vis = np.concatenate((img1, img2), axis=0) # do it for N image if necessary
(2) You are just want to check the similarity of the template to all N image.
Then you need to declare a threshold after minmax to see if there is a point in it that is above the threshold, if have return 1 if no return 0
def getBestMatch():
images = [
cv2.imread('tmp/1.png'),
cv2.imread('tmp/2.png'),
cv2.imread('tmp/3.png'),
cv2.imread('tmp/4.png'),
cv2.imread('tmp/5.png'),
cv2.imread('tmp/6.png')
]
template = cv2.imread('template.png')
result = []
for i in images:
match = cv2.matchTemplate(i, template, cv2.TM_CCOEFF_NORMED)
_, confidence, _, _ = cv2.minMaxLoc(match)
result.append(confidence)
posNum = result.index(max(result))
return posNum
I have an image, using steganography I want to save the data in border pixels only.
In other words, I want to save data only in the least significant bits(LSB) of border pixels of an image.
Is there any way to get border pixels to store data( max 15 characters text) in the border pixels?
Plz, help me out...
OBTAINING BORDER PIXELS:
Masking operations are one of many ways to obtain the border pixels of an image. The code would be as follows:
a= cv2.imread('cal1.jpg')
bw = 20 //width of border required
mask = np.ones(a.shape[:2], dtype = "uint8")
cv2.rectangle(mask, (bw,bw),(a.shape[1]-bw,a.shape[0]-bw), 0, -1)
output = cv2.bitwise_and(a, a, mask = mask)
cv2.imshow('out', output)
cv2.waitKey(5000)
After I get an array of ones with the same dimension as the input image, I use cv2.rectangle function to draw a rectangle of zeros. The first argument is the image you want to draw on, second argument is start (x,y) point and the third argument is the end (x,y) point. Fourth argument is the color and '-1' represents the thickness of rectangle drawn (-1 fills the rectangle). You can find the documentation for the function here.
Now that we have our mask, you can use 'cv2.bitwise_and' (documentation) function to perform AND operation on the pixels. Basically what happens is, the pixels that are AND with '1' pixels in the mask, retain their pixel values. Pixels that are AND with '0' pixels in the mask are made 0. This way you will have the output as follows:
.
The input image was :
You have the border pixels now!
Using LSB planes to store your info is not a good idea. It makes sense when you think about it. A simple lossy compression would affect most of your hidden data. Saving your image as JPEG would result in loss of info or severe affected info. If you want to still try LSB, look into bit-plane slicing. Through bit-plane slicing, you basically obtain bit planes (from MSB to LSB) of the image. (image from researchgate.net)
I have done it in Matlab and not quite sure about doing it in python. In Matlab,
the function, 'bitget(image, 1)', returns the LSB of the image. I found a question on bit-plane slicing using python here. Though unanswered, you might want to look into the posted code.
To access border pixel and enter data into it.
A shape of an image is accessed by t= img.shape. It returns a tuple of the number of rows, columns, and channels.A component is RGB which 1,2,3 respectively.int(r[0]) is variable in which a value is stored.
import cv2
img = cv2.imread('xyz.png')
t = img.shape
print(t)
component = 2
img.itemset((0,0,component),int(r[0]))
img.itemset((0,t[1]-1,component),int(r[1]))
img.itemset((t[0]-1,0,component),int(r[2]))
img.itemset((t[0]-1,t[1]-1,component),int(r[3]))
print(img.item(0,0,component))
print(img.item(0,t[1]-1,component))
print(img.item(t[0]-1,0,component))
print(img.item(t[0]-1,t[1]-1,component))
cv2.imwrite('output.png',img)
I'm having a bit of trouble with opencv and template matching, so I was hoping someone here could help a lost soul out.
So as part of the code I'm using, I've got the following 2 lines which I don't quite understand as well as I should.
result = cv2.matchTemplate(edged, template, cv2.TM_CCOEFF)
(_, maxVal, _, maxLoc) = cv2.minMaxLoc(result)
From my understanding, the first line stores a correlation coefficient in the variable "result". This in turn is passed into cv2.minMaxLoc(...) which in turn generates a 4 element array composing of (minVal, maxVal, minLoc, maxLoc) of which we are only interested in maxVal and maxLoc.
Upon printing the value of maxVal, I seem to be getting values in the between 2,000,000 to 7,000,000 depending on the template, lighting conditions etc.
My questions are as follows:
What does maxVal mean?
What is the range of maxVal?
What physical characteristics affect the values of maxVal?
Thank you in advance for all your help and guidance!
Ideally, cv2.matchTemplate returns a correlation map, essentially a grayscale image, where each pixel denotes how much does the neighborhood of that pixel match with the template.
You suggest that we are interested only in maxLoc and maxVal and that isn't true, it is subject to the type of correlation you are considering when matching a template.
Now, to your questions, the minMaxLoc function returns the max and min intensity values in a Mat or an array along with the location of these intensities.
MaxLoc means where is the highest intensity in the image returned by matchTemplate which would correspond to the best match in your image w.r.t. to your template ( for specific correlation methods only, for TM_SQDIFF or TM_SQDIFF_NORMED the best match would be the minVal).
Since the image returned by matchTemplate is gray-scale, the range should be dependent on the original image, so 2000000 to 7000000 seem a bit out of order to me.
The only "physical characteristics" that affect the maxVal should be the degree of correlation the template has with the image and nothing else.
Hope it helps!
As the other answers already explain, you are matching based on cross-correlation. So maxVal is the maximum of your cross-correlation. It is difficult to make a generic guess about the range. But you can always limit the range to [0, 1] by
normalize(result, result, 0, 1, NORM_MINMAX, -1, Mat());
If you crop the region of the image that best matches the template, then the value of the peak of the cross-correlation function is
np.sum(cropped * template)
This value will get bigger when the image is brighter, when the template is brighter, and when the template is bigger.
Here is the link I find
How do I find an image contained within an image
And I get the code in the link above.
Here is my code
import re
import os
import time
import cv2
from cv2 import cv
METHOD = cv.CV_TM_SQDIFF_NORMED
ROOTDIR = 'GeneralIMG/'
# Load The Deck Img
LARGE_IMAGE = cv2.imread('deck.bmp')
for subdir, dirs, files in os.walk(ROOTDIR):
for myfile in files:
filepath = os.path.join(subdir, myfile)
# Read the images from the file
# small_image = cv2.imread(myfile)
# small_image = cv2.imread('GeneralIMG/cfrB.bmp')
Ffile = subdir + myfile
small_image = cv2.imread(Ffile)
result = cv2.matchTemplate(small_image, LARGE_IMAGE, METHOD)
# We want the minimum squared difference
mn,_,mnLoc,_ = cv2.minMaxLoc(result)
MPx,MPy = mnLoc
if MPx == 0 and MPy == 0:
print 'Match failure!'
else:
print MPx, MPy
print re.sub(r"B|D", "", os.path.splitext(myfile)[0])
# Draw the rectangle:
# Extract the coordinates of our best match
MPx,MPy = mnLoc
# Step 2: Get the size of the template. This is the same size as the match.
trows,tcols = small_image.shape[:2]
# Step 3: Draw the rectangle on LARGE_IMAGE
cv2.rectangle(LARGE_IMAGE, (MPx,MPy),(MPx+tcols,MPy+trows),(0,0,255),2)
# Display the original image with the rectangle around the match.
cv2.imshow('output',LARGE_IMAGE)
# The image is only displayed if we call this
cv2.waitKey(0)
# time.sleep(5)
This is the small pic which I want to find in the large image but the large image don't contain the small image. To my surprise the matchTemplate still get the result.
12~07_00-03-03.bmp http://7xpvdr.com1.z0.glb.clouddn.com/12~07_00-03-03.bmp
But the match result I get is not accurate enough like the pic below. The red rectangle is the match result.
12~07_00-01-58.bmp http://7xpvdr.com1.z0.glb.clouddn.com/12~07_00-01-58.bmp
Here I want to find exact the same pic in the large one. If not contain it, just return a false will be fine. I don't want a fuzzy search what should I do?
--- update ---
this is the small img:
small img
This is the original larger bmp image
large image
When the image is uploaded the format will be changed. The small img is png file, and the larger is bmp.
Thing is I wonder if the matchtemplate use the fuzzy search here. So the issue here why I get the wrong search result seems reasonable. I want to know how to make the search exactly. Thx~
update:
I check the doc of the [matchtemplate]
And I find this which means it is a best match search not perfect match.
So How can I make it perfect match ?
What is template matching?
Template matching is a technique for finding areas of an image that match (are similar) to a template image (patch).
Have you tried applying a threshold on the results given by minMaxLoc?
I never used matchTemplate and minMaxLoc so I don't really know how it works, but from what I understand, if what you want is a perfect match, then the value you are looking for could possibly be 0 (or 1, depending on your METHOD parameter). So if you discard every other value, you shouldn't get positive results unless you have a perfect match.
You can get perfect matching using cv.matchTemplate:
use method cv.TM_SQDIFF and threshold the absolute values of the result at <= 1/16
Mathematically all patches with perfect matching should have a squared diff of 0, not [-0.0625,0.0625]. However, the optimization in the actual implementation of cv.matchTemplate result in slight differences (at least the function ippiSqrDistanceNorm used at OpenCV 4.5.1 did for uint8 input and 3x3 uint8 masks)
Note that the matching compares starting from the left/top corner, not the center of the template, so if you want to get all pixels with the exact surrounding as your template, you have to add an offset of half the template's width and height.