As part of a program which contains a series of images to be processed, I first need to detect a green-coloured rectangle. I'm trying to write a program that doesn't use colour masking, since the lighting and glare on the images will make it difficult to find the appropriate HSV ranges.
(p.s. I already have two questions based on this program, but this one is unrelated to those. It's not a follow up, I want to address a separate issue.)
I used the standard rectangle detection technique, making use of findContours() and approxPolyDp() methods. I added some constraints that got rid of unnecessary rectangles (like aspectRatio>2.5, since my desired rectangle is clearly the "widest" and area>1500, to discard random small rectangles) .
import numpy as np
import cv2 as cv
img = cv.imread("t19.jpeg")
width=0
height=0
start_x=0
start_y=0
end_x=0
end_y=0
output = img.copy()
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
#threshold
th = cv.adaptiveThreshold(gray,255,cv.ADAPTIVE_THRESH_GAUSSIAN_C,cv.THRESH_BINARY,9,2)
cv.imshow("th",th)
#rectangle detection
contours, _ = cv.findContours(th, cv.RETR_TREE, cv.CHAIN_APPROX_NONE)
for contour in contours:
approx = cv.approxPolyDP(contour, 0.01* cv.arcLength(contour, True), True)
cv.drawContours(img, [approx], 0, (0, 0, 0), 5)
x = approx.ravel()[0]
y = approx.ravel()[1]
x1 ,y1, w, h = cv.boundingRect(approx)
a=w*h
if len(approx) == 4 and x>15 :
aspectRatio = float(w)/h
if aspectRatio >= 2.5 and a>1500:
print(x1,y1,w,h)
width=w
height=h
start_x=x1
start_y=y1
end_x=start_x+width
end_y=start_y+height
cv.rectangle(output, (start_x,start_y), (end_x,end_y), (0,0,255),3)
cv.putText(output, "rectangle "+str(x1)+" , " +str(y1-5), (x1, y1-5), cv.FONT_HERSHEY_COMPLEX, 0.5, (0, 0, 0))
cv.imshow("op",output)
print("start",start_x,start_y)
print("end", end_x,end_y)
print("width",width)
print("height",height)
It is working flawlessly for all the images, except one:
I used adaptive thresholding to create the threshold, which was used by the findContours() method.
I tried displaying the threshold and the output , and it looks like this:
The thresholds for the other images also looked similar...so I can't pinpoint what exactly has gone wrong in the rectangle detection procedure.
Some tweaks I have tried:
Changing the last two parameters in the adaptive parameters method.
I tried 11,1 , 9,1, and for both of them, the rectangle in the
threshold looked more prominent : but in this case the output
detected no rectangles at all.
I have already disregarded otsu thresholding, as it is not working
for about 4 of my test images.
What exactly can I tweak in the rectangle detection procedure for it to detect this rectangle?
I also request , if possible, only slight modifications to this method and not some entirely new method. As I have mentioned, this method is working perfectly for all of my other test images, and if the new suggested method works for this image but fails for the others, then I'll find myself back here asking why it failed.
Edit: The method that abss suggested worked for this image, however failed for:
image 4
image 1, far off
Other test images:
image 1, normal
image 2
image 3
image 9, part 1
image 9, part 2
You can easily do it by adding this line of code after your threshold
kernel = cv.getStructuringElement(cv.MORPH_RECT,(3,3))
th = cv.morphologyEx(th,cv.MORPH_OPEN,kernel)
This will remove noise within the image. you can see this link for more understanding about morphologyEx https://docs.opencv.org/master/d9/d61/tutorial_py_morphological_ops.html
The results I got is shown below
I have made a few modifications to your code so that it works with all of your test images. There are a few false positives that you may have to filter based on HSV color range for green (since your target is always a shade of green). Alternately you can take into account the fact that the one of the child hierarchy of your ROI contour is going to be > 0.4 or so times than the outer contour. Here are the modifications:
Used DoG for thresholding useful contours
Changed arcLength multiplier to 0.5 instead of 0.1 as square corners are not smooth
cv2.RETR_CCOMP to get 2 level hierarchy
Moved ApproxPolyDP inside to make it more efficient
Contour filter area changed to 600 to filter ROI for all test images
Removed a little bit of unnecessary code
Check with all the other test images that you may have and modify the parameters accordingly.
img = cv2.imread("/path/to/your_image")
width=0
height=0
start_x=0
start_y=0
end_x=0
end_y=0
output = img.copy()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gw, gs, gw1, gs1, gw2, gs2 = (3,1.0,7,3.0, 3, 2.0)
img_blur = cv2.GaussianBlur(gray, (gw, gw), gs)
g1 = cv2.GaussianBlur(img_blur, (gw1, gw1), gs1)
g2 = cv2.GaussianBlur(img_blur, (gw2, gw2), gs2)
ret, thg = cv2.threshold(g2-g1, 127, 255, cv2.THRESH_BINARY)
contours, hier = cv2.findContours(thg, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_NONE)
img_cpy = img.copy()
width=0
height=0
start_x=0
start_y=0
end_x=0
end_y=0
for i in range(len(contours)):
if hier[0][i][2] == -1:
continue
x ,y, w, h = cv2.boundingRect(contours[i])
a=w*h
aspectRatio = float(w)/h
if aspectRatio >= 2.5 and a>600:
approx = cv2.approxPolyDP(contours[i], 0.05* cv2.arcLength(contours[i], True), True)
if len(approx) == 4 and x>15 :
width=w
height=h
start_x=x
start_y=y
end_x=start_x+width
end_y=start_y+height
cv2.rectangle(img_cpy, (start_x,start_y), (end_x,end_y), (0,0,255),3)
cv2.putText(img_cpy, "rectangle "+str(x)+" , " +str(y-5), (x, y-5), cv2.FONT_HERSHEY_COMPLEX, 0.5, (0, 0, 0))
plt.imshow(img_cpy)
print("start",start_x,start_y)
print("end", end_x,end_y)
Related
I am in the initial stages of writing a Rubik's cube solver and am stuck at the following challenge:
Using the following image-processing code gives me the following image:
import cv2 as cv
import glob
import numpy as np
for img in glob.glob("captured_images/*.jpg"):
image = cv.imread(img)
copy = image.copy()
grey = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
decrease_noise = cv.fastNlMeansDenoising(grey, 10, 15, 7, 21)
blurred = cv.GaussianBlur(decrease_noise, (3, 3), 0)
canny = cv.Canny(blurred, 20, 40)
thresh = cv.threshold(canny, 0, 255, cv.THRESH_OTSU + cv.THRESH_BINARY)[1]
contours = cv.findContours(thresh, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for c in contours:
# obtain the bounding rectangle coordinates for each square
x, y, w, h = cv.boundingRect(c)
# With the bounding rectangle coordinates we draw the green bounding boxes
cv.rectangle(copy, (x, y), (x + w, y + h), (36, 255, 12), 2)
cv.imshow('copy', copy)
cv.waitKey(0)
cv.destroyAllWindows()
There are numerous bound rectangles highlighted. Trying to filter out only the squares using this code:
contour_list = []
for contour in contours:
approx = cv.approxPolyDP(contour, 0.01 * cv.arcLength(contour, True), True)
area = cv.contourArea(contour)
if len(approx) == 4:
(x, y, w, h) = cv.boundingRect(approx)
if (float(w)/h) == 1:
cv.rectangle(copy, (x, y), (x + w, y + h), (36, 255, 12), 2)
contour_list.append(contour)
doesn't work as the squares aren't precise enough to fit the definition of "all sides of square are equal".
I though retaking the images against a white background might help to more easily find the relevant squares, however modifying the original image to a cube with a white background and using the original code causes only the larger cube to be recognised as a square:
My question is three-fold:
1a) How can I modify my original code for the original image to accurately measure only the relevant squares by using the following criteria for finding squares:
There must be four corners
All four lines must be roughly the same length
All four corners must be roughly 90 degrees
1b) In the second image with the white background, how can I select everything outside the bound rectangle and convert that white background to black, which helps greatly in correctly detecting the appropriate squares?
1c) In general, why is a black background so much more beneficial than a white background in using the cv2.rectangle() function?
Any help in gaining some clearer understanding is much appreciated! :)
How can I modify my original code for the original image to accurately measure only the relevant squares by using the following criteria for finding squares:
Your code only accepts contours that are exactly square. You need to have a "squaredness" factor and then determine some acceptable threshold.
The "squaredness" factor is h/w if w > h else w/h. The closer that value to one, the more square the rectangle is. Then you can accept only rectangles with a factor of .9 or higher (or whatever works best).
In general, why is a black background so much more beneficial than a white background in using the cv2.rectangle() function?
The contour finding algorithm that OpenCV uses is actually:
Suzuki, S. and Abe, K., Topological Structural Analysis of Digitized Binary Images by Border Following. CVGIP 30 1, pp 32-46 (1985)
In your case, the algorithm might just have picked up the contours just fine, but you have set the RETR_EXTERNAL flag, which will cause OpenCV to only report the outermost contours. Try changing it to RETR_LIST.
Find the OpenCV docs with regards to contour finding here: https://docs.opencv.org/master/d9/d8b/tutorial_py_contours_hierarchy.html
This is my first question on Stackoverflow. I'm a little excited, forgive me if I'm wrong. We have mixed ellipses with and without overlapping drawn randomly from paint. I'm sharing the image I'm working on and my code. I am not a professional in opencv module, I wrote my code as a result of research inspired by sources.
The purpose of my code is,
Detection of randomly drawn with and without overlapping ellipses using the cv2.fitEllipse method. Next, find the major axis, minor axis and areas of the detected ellipses.
The problem with my code is actually this,
In overlapping ellipses, while fitting the ellipse under normal conditions, 2 ellipses should be fit, but about 6-7 ellipses are fit and I cannot reach the values I want to be calculated.
I'm open to your help, thank you in advance.
Example image:
import cv2
import numpy as np
import random as rng
import math
img = cv2.imread('overlapping_ellipses.png', 1)
imge= cv2.cvtColor(img,cv2.COLOR_RGB2BGR)
gray = cv2.cvtColor(imge, cv2.COLOR_BGR2GRAY)
blur = cv2.blur(gray, (2,2), 3)
edged = cv2.Canny(blur, 50, 100)
kernel= np.ones((2,2))
edged1 = cv2.dilate(edged, kernel, iterations=2)
edged2 = cv2.erode(edged1, kernel, iterations=2)
def thresh_callback(val):
threshold = val
canny_output = cv2.Canny(edged2, threshold, threshold * 4)
contours, _ = cv2.findContours(canny_output, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
minRect = [None]*len(contours)
minEllipse = [None]*len(contours)
for i, c in enumerate(contours):
minRect[i] = cv2.minAreaRect(c)
if c.shape[0] > 5:
minEllipse[i] = cv2.fitEllipse(c)
(x1,y1),(d1,d2),angle = minEllipse[i]
print('\nX1: ', round(x1,4), '\nY1: ', round(y1,4), '\nD1:',round(d1,4), '\nD2',round(d2,4), '\nAngle:', round(angle,4))
long= x1-d2
small= y1-d1
major= long/2
minor= small/2
pixel= 37.795275591
major1= major/pixel
minor1= minor/pixel
print('--------------------------------')
print('Major axis is: ', abs(round(major1,4)), 'cm')
print('Minor axis is: ', abs(round(minor1,4)), 'cm')
print('--------------------------------')
drawing = np.zeros((canny_output.shape[1], canny_output.shape[1], 3), dtype=np.uint8)
for i, c in enumerate(contours):
color = (rng.randint(0,256), rng.randint(0,256), rng.randint(0,256))
cv2.drawContours(drawing, contours, i, color)
if c.shape[0] > 5:
cv2.ellipse(drawing, minEllipse[i], color, 1)
cv2.imshow('Fitting Ellips', drawing)
source_window = 'Source'
cv2.namedWindow(source_window)
cv2.imshow(source_window, img)
max_thresh = 255
thresh = 100
cv2.createTrackbar('Canny Thresh:', source_window, thresh, max_thresh, thresh_callback)
thresh_callback(thresh)
cv2.waitKey()
Step 1: Identify and separate the blobs in the input image.
Since we don't care about colour information here, we can directly load the image as grayscale.
image = cv2.imread('input.png', cv2.IMREAD_GRAYSCALE)
The input image contains black ellipses on white background.
We only need the external contours of the blobs, and cv2.findContours expects white blobs on black background.
Therefore we need to invert the image. At the same time we need a binary image. We can use cv2.threshold to accomplish both tasks.
Once we detect the blob contours, we can collect some useful information for each blob into a simple map-based data structure.
def detect_blobs(image):
_,img_binary = cv2.threshold(image, 127, 255, cv2.THRESH_BINARY_INV)
contours, _ = cv2.findContours(img_binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
blobs = []
for i, contour in enumerate(contours):
orig_x, orig_y, width, height = cv2.boundingRect(contour)
roi_image = image[orig_y:orig_y+height,orig_x:orig_x+width]
blobs.append({
"i" : i
, "contour" : contour
, "origin" : (orig_x, orig_y)
, "size" : (width, height)
, "roi_image" : roi_image
})
return blobs
Step 2: Process each blob
First we need to determine whether the blob is a single ellipse, or whether it is a pair over intersecting ellipses.
One way to do this is by looking for convexity defects.
Since the coordinates of our contour are represented by integers, even the single-ellipse scenario will exhibit some convexity defects.
However, their magnitude (the distance between the furthest point on the contour from the enclosing convex hull segment) will be very small, generally below 1 pixel.
On the other hand, the contour of a pair of intersecting ellipses will have large convexity defects, one for each of the four points where the curves intersect.
This distinction can be seen on the following two images (contour is blue, convex hull red, identified intersection points/locations of large convexity defects are orange circles):
Single ellipse
Two intersecting ellipses
We therefore filter out any small convexity defects, and note the locations of the large ones. Now we're left with 3 possible scenarios.
Scenario A: No intersection points detected
Only small convexity defects were identified, which means this is very likely a single ellipse. We simply fit an ellipse to the contour and move on.
Scenario B: Exactly 4 intersection points detected
In this case we have 2 intersecting ellipses. We use the intersection points to split the contour into 4 segments, one for each "lobe" of the blob. Each of the segments should include the two intersection points that delimit it.
In the following picture, the segments are show in green, yellow, cyan and magenta, while the intersection points are orange circles:
Now, we can combine the pairs of segments that lie opposite each other (i.e. green+cyan and yellow+magenta) to get two lists of points, one for each ellipse. Again, we simply fit an ellipse to each list of points.
Scenario C: Some other number of intersection points detected
This is considered an invalid situation.
def process_blob(blob):
MAJOR_DEFECT_THRESHOLD = 2.0
contour = blob["contour"]
blob["hull"] = cv2.convexHull(contour)
hull_idx = cv2.convexHull(contour, returnPoints=False)
defects = cv2.convexityDefects(contour, hull_idx)
intersections = []
for i,defect in enumerate(np.squeeze(defects, 1)):
_, _, far_idx, far_dist = defect
real_far_dist = far_dist / 256.0
if real_far_dist >= MAJOR_DEFECT_THRESHOLD:
intersections.append(far_idx)
if len(intersections) == 0:
print("One ellipse")
blob["ellipses"] = [cv2.fitEllipse(contour)]
elif len(intersections) == 4:
print("Two ellipses")
blob["segments"] = [
contour[intersections[0]:intersections[1]+1]
, contour[intersections[1]:intersections[2]+1]
, contour[intersections[2]:intersections[3]+1]
, np.vstack([contour[intersections[3]:],contour[:intersections[0]+1]])
]
split_contours = [
np.vstack([blob["segments"][0], blob["segments"][2]])
, np.vstack([blob["segments"][1], blob["segments"][3]])
]
blob["ellipses"] = [cv2.fitEllipse(c) for c in split_contours]
else:
print("Invalid scenario")
blob["ellipses"] = []
return blob["ellipses"]
At this point, it's trivial to calculate the parameters you need -- I'll leave this as an excercise to the reader.
As a bonus, here's some simple visualization for debugging purposes:
def visualize_blob(blob):
PADDING = 20
orig_x, orig_y = blob["origin"]
offset = (orig_x - PADDING, orig_y - PADDING)
input_img = cv2.copyMakeBorder(blob["roi_image"]
, PADDING, PADDING, PADDING, PADDING
, cv2.BORDER_CONSTANT, None, 255)
adjusted_img = cv2.add(input_img, 127) - 63
output_img_ch = cv2.cvtColor(adjusted_img, cv2.COLOR_GRAY2BGR)
output_img_seg = output_img_ch.copy()
output_img_el = output_img_ch.copy()
cv2.drawContours(output_img_ch, [blob["hull"] - offset], 0, (127,127,255), 4)
cv2.drawContours(output_img_ch, [blob["contour"] - offset], 0, (255,127,127), 2)
SEGMENT_COLORS = [(0,255,0),(0,255,255),(255,255,0),(255,0,255)]
if "segments" in blob:
for i in range(4):
cv2.polylines(output_img_seg, [blob["segments"][i] - offset], False, SEGMENT_COLORS[i], 4)
for i in range(4):
center = (blob["segments"][i] - offset)[0][0]
cv2.circle(output_img_ch, center, 4, (0,191,255), -1)
cv2.circle(output_img_seg, center, 4, (0,191,255), -1)
for ellipse in blob["ellipses"]:
offset_ellipse = ((ellipse[0][0] - offset[0], ellipse[0][1] - offset[1]), ellipse[1], ellipse[2])
cv2.ellipse(output_img_el, offset_ellipse, (0,0,255), 2)
cv2.imshow('', np.hstack([output_img_ch,output_img_seg, output_img_el]))
cv2.imwrite('output_%d_ch.png' % blob["i"], output_img_ch)
cv2.imwrite('output_%d_seg.png' % blob["i"], output_img_seg)
cv2.imwrite('output_%d_el.png' % blob["i"], output_img_el)
cv2.waitKey()
Pulling it all together:
import cv2
import numpy as np
## INSERT THE FUNCTIONS LISTED ABOVE IN THE QUESTION ##
image = cv2.imread('input.png', cv2.IMREAD_GRAYSCALE)
blobs = detect_blobs(image)
print("Found %d blob(s)." % len(blobs))
for blob in blobs:
process_blob(blob)
visualize_blob(blob)
I've been researching and trying a couple functions to get what I want and I feel like I might be overthinking it.
One version of my code is below. The sample image is here.
My end goal is to find the angle (yellow) of the approximated line with respect to the frame (green line) Final
I haven't even got to the angle portion of the program yet.
The results I was obtaining from the below code were as follows. Canny Closed Small Removed
Anybody have a better way of creating the difference and establishing the estimated line?
Any help is appreciated.
import cv2
import numpy as np
pX = int(512)
pY = int(768)
img = cv2.imread('IMAGE LOCATION', cv2.IMREAD_COLOR)
imgS = cv2.resize(img, (pX, pY))
aimg = cv2.imread('IMAGE LOCATION', cv2.IMREAD_GRAYSCALE)
# Blur image to reduce noise and resize for viewing
blur = cv2.medianBlur(aimg, 5)
rblur = cv2.resize(blur, (384, 512))
canny = cv2.Canny(rblur, 120, 255, 1)
cv2.imshow('canny', canny)
kernel = np.ones((2, 2), np.uint8)
#fringeMesh = cv2.dilate(canny, kernel, iterations=2)
#fringeMesh2 = cv2.dilate(fringeMesh, None, iterations=1)
#cv2.imshow('fringeMesh', fringeMesh2)
closing = cv2.morphologyEx(canny, cv2.MORPH_CLOSE, kernel)
cv2.imshow('Closed', closing)
nb_components, output, stats, centroids = cv2.connectedComponentsWithStats(closing, connectivity=8)
#connectedComponentswithStats yields every separated component with information on each of them, such as size
sizes = stats[1:, -1]; nb_components = nb_components - 1
min_size = 200 #num_pixels
fringeMesh3 = np.zeros((output.shape))
for i in range(0, nb_components):
if sizes[i] >= min_size:
fringeMesh3[output == i + 1] = 255
#contours, _ = cv2.findContours(fringeMesh3, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
#cv2.drawContours(fringeMesh3, contours, -1, (0, 255, 0), 1)
cv2.imshow('final', fringeMesh3)
#cv2.imshow("Natural", imgS)
#cv2.imshow("img", img)
cv2.imshow("aimg", aimg)
cv2.imshow("Blur", rblur)
cv2.waitKey()
cv2.destroyAllWindows()
You can fit a straight line to the first white pixel you encounter in each column, starting from the bottom.
I had to trim your image because you shared a screen grab of it with a window decoration, title and frame rather than your actual image:
import cv2
import math
import numpy as np
# Load image as greyscale
im = cv2.imread('trimmed.jpg', cv2.IMREAD_GRAYSCALE)
# Get index of first white pixel in each column, starting at the bottom
yvals = (im[::-1,:]>200).argmax(axis=0)
# Make the x values 0, 1, 2, 3...
xvals = np.arange(0,im.shape[1])
# Fit a line of the form y = mx + c
z = np.polyfit(xvals, yvals, 1)
# Convert the slope to an angle
angle = np.arctan(z[0]) * 180/math.pi
Note 1: The value of z (the result of fitting) is:
array([ -0.74002694, 428.01463745])
which means the equation of the line you are looking for is:
y = -0.74002694 * x + 428.01463745
i.e. the y-intercept is at row 428 from the bottom of the image.
Note 2: Try to avoid JPEG format as an intermediate format in image processing - it is lossy and changes your pixel values - so where you have thresholded and done your morphology you are expecting values of 255 and 0, JPEG will lossily alter those values and you end up testing for a range or thresholding again.
Your 'Closed' image seems to quite clearly segment the two regions, so I'd suggest you focus on turning that boundary into a line that you can do something with. Connected components analysis and contour detection don't really provide any useful information here, so aren't necessary.
One quite simple approach to finding the line angle is to find the first white pixel in each row. To get only the rows that are part of your diagonal, don't include rows where that pixel is too close to either side (e.g. within 5%). That gives you a set of points (pixel locations) on the boundary of your two types of grass.
From there you can either do a linear regression to get an equation for the straight line, or you can get two points by averaging the x values for the top and bottom half of the rows, and then calculate the gradient angle from that.
An alternative approach would be doing another morphological close with a very large kernel, to end up with just a solid white region and a solid black region, which you could turn into a line with canny or findContours. From there you could either get some points by averaging, use the endpoints, or given a smooth enough result from a large enough kernel you could detect the line with hough lines.
I would like to know if a big image contains a small image. The small image can be semi-transparent (similar to watermark, so it's not a fully filled photo). I've tried following different SO answers on this topic, but they're all matching the EXACT photo, but what I am looking for is whether the photo exists with 80% accuracy as the photo will be a lossy rendered version of the original one.
This is a procedure of how the images I am searching in will be generated:
Use any photo, put a semi-transparent "watermark" on it within Photoshop and save it. Then I want to check if the "watermark" exists within created photo with certain percent of accuracy (80% is good enough).
I've tried using the original template matching example provided on their docs page but I'm getting barely any match at all.
This is the code I'm using:
import cv2
import numpy as np
img_rgb = cv2.imread('photo2.jpeg')
img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
template = cv2.imread('small-image.png', 0)
w, h = template.shape[::-1]
res = cv2.matchTemplate(img_gray,template,cv2.TM_CCOEFF_NORMED)
threshold = 0.7
loc = np.where( res >= threshold)
for pt in zip(*loc[::-1]):
cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0,0,255), 2)
cv2.imshow('output', img_rgb)
cv2.waitKey(0)
Here are the photos I've been using for the test, as this is something similar I am trying to make a match on.
small-image.png
photo2.jpeg
I am assuming the whole watermark will have the same RGB values and the text will have a little different RGB values otherwise this technique will not work. Based on this we can obtain the RGB values of a pixel of the small image and treated it as a mask by using cv2.inRange to find those pixel values in the large image. Similarly a mask is also created for the small image using those pixel values.
small = cv2.imread('small_find.png')
large = cv2.imread('large_find.jpg')
pixel = np.reshape(small[3,3], (1,3))
lower =[pixel[0,0]-10,pixel[0,1]-10,pixel[0,2]-10]
lower = np.array(lower, dtype = 'uint8')
upper =[pixel[0,0]+10,pixel[0,1]+10,pixel[0,2]+10]
upper = np.array(upper, dtype = 'uint8')
mask = cv2.inRange(large,lower, upper)
mask2 = cv2.inRange(small, lower, upper)
I had to take a buffer value of 20 because the values were not clearly matching in the large image otherwise only 1 is enough in either upper or lower. Then we find contours in mask and find values of its bounding rectangle which is cut out and reshaped to the size of mask2.
im, contours, hierarchy = cv2.findContours(mask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
#cv2.drawContours(large, contours, -1, (0,0,255), 1)
cnt = max(contours, key = cv2.contourArea)
x,y,w,h = cv2.boundingRect(cnt)
wanted_part = mask[y:y+h, x:x+w]
wanted_part = cv2.resize(wanted_part, (mask2.shape[1], mask2.shape[0]), interpolation = cv2.INTER_LINEAR)
The two masks side by side (inverted them otherwise they were not visible).
For comparing they you can use any parameter and check whether it satisfies your condition or not. I used mean square error and got error of only 6.20 which is very low.
def MSE(img1, img2):
squared_diff = img1 - img2
summed = np.sum(squared_diff)
num_pix = img1.shape[0] * img1.shape[1] #img1 and 2 should have same shape
err = summed / num_pix
return err
I do have some old bank statements as scan and would like to use google´s thesseract engine to extract the text. Works pretty well unless the image is slightly rotated. I thought of detecting the dashed lines in order to estimate the slope and afterwards the angle of rotation. However, it is tricky to get the parameters right.
If I could get rid of the large line artefact, I might use the minimum rotated bounding box (cv2.minAreaRect) on the text characters.
Maybe another strategy is suited better ? Any ideas ?
An example image (deleted some characters for data protection):
EIDT: I have found a solution which seems to work. However, I am stil wondering if there might be a faster solution (takes about 1.5 seconds per Image)
I do use template matching from skimage with following template:
template = plt.imread('template_long.png')
template = rgb2gray(template)
template = template > threshold_mean(template)
for i in range(1):
# read in image
img = cv2.imread('conversion/umsatz_{}.png'.format(i))
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
gray = cv2.bitwise_not(gray)
# threshold the image, setting all foreground pixels to
# 255 and all background pixels to 0
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]
# edge detection
#edges = cv2.Canny(thresh,2,100, apertureSize = 3)
# fill the holes from detected edges
#kernel = np.ones((2,2),np.uint8)
#dilate = cv2.dilate(thresh, kernel, iterations=1)
result = match_template(thresh, template)
mask = result < 0.5
r = result.copy()
r[mask] = 0
r[~mask] = 1
plt.imshow(r)