I have read through dozens of questions on this topic here. (see eg: 1, 2, 3). There are a lot of helpful explanations of how to play around with parameters etc, watershedding, etc. Yet no matter what I have tried to put together I am still not managing a halfway-passable count of the cells in my image
Here are two examples of the kind of images I need to process.
Initially I was trying to count all the cells, but because of the difference in focus at the edges (where it gets blurrier) I thought it might be easier to count cells within a rectangle the user selects.
I was hopeful this would improve the results, but as you can see, HoughCircles is both selecting as circles empty spaces with nothing in them, and missing many cells:
Other algorithms I have tried have fared worse.
My code:
cap = cv2.VideoCapture(video_file)
frames = []
while True:
frame_exists, curr_frame = cap.read()
if frame_exists:
frames.append(curr_frame)
else:
break
frames = [cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) for frame in frames]
for img in frames:
circles = cv2.HoughCircles(img,
cv2.HOUGH_GRADIENT,
minDist=10,
dp=1.1,
param1=4, #the lower the number the more circles found
param2=13,
minRadius=4,
maxRadius=10)
if circles is not None:
circles = np.round(circles[0, :]).astype("int")
for (x, y, r) in circles:
cv2.circle(img, (x, y), r, (0, 255, 0), 1)
cv2.imshow("result", img)
Editing to add in my not helpful preprocessing code:
denoise = cv2.fastNlMeansDenoising(img, h=4.0, templateWindowSize=15, searchWindowSize=21)
thresh=cv2.adaptiveThreshold(denoise,255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY,41,2)
(and then I passed thresh to HoughCircles instead of img)
It just didn't seem to make any difference...
I believe that these are not circular enough for Hough to work well, you would have to lower param2 too much to account for the lack of uniformity. I would recommend looking into the cv2.findContours method instead and use your 'thresh' image.
https://docs.opencv.org/4.x/dd/d49/tutorial_py_contour_features.html
Related
I have used:
result = cv2.matchTemplate(frame, template, cv2.TM_CCORR_NORMED)
to generate this output:
I need a list of (x, y) tuples at each of the local maxima (bright spots) in the result. Simply finding all points above a threshold doesn't work, since there are many such points around each maximum.
I can guarantee the minimum distance between any two maxima, which ought to help speed things up.
Is there an efficient technique for doing this?
(P.S.: this is cross-posted from https://forum.opencv.org/t/locating-local-maximums/1534)
update
Based on an excellent suggestion by Michael Lee, I've added skeletonizing to the thresholded image. It's close, but the skeletonizing still has many "worms" rather than single points. My processing flow is as follows:
# read the image
im = cv.imread("image.png", cv.IMREAD_GRAYSCALE)
# apply thresholding
ret, im2 = cv.threshold(im, args.threshold, 255, cv.THRESH_BINARY)
# dilate the thresholded image to eliminate "pinholes"
im3 = cv.dilate(im2, None, iterations=2)
# skeletonize the result
im4 = cv.ximgproc.thinning(im3, None, cv.ximgproc.THINNING_ZHANGSUEN)
# print the number of points found
x, y = np.nonzero(im5)
print(x.shape)
# => 1208
This is a step in the right direction, but there should be more like 220 points, not 1208.
Here are the intermediate results. As you can see in the last picture (skeletonized), there are still lots of little "worms" rather than single point. Is there a better approach?
Thresholded:
Dilated:
Skeletonized:
Update 2/14: Seems like skeletonization only took you part of the way there. Here's a better solution which I believe should get you the rest of the way. Here's how you would do it in scikit-image - maybe you can find the analog in OpenCV - seems like cv2.findContours would be a good start.
# mask is the thresholded image (before or after dilation should work, no skeletonization.
from skimage.measure import label, regionprops
labeled_image = label(mask)
output_points = [region.centroid for region in regionprops(labeled_image)]
Explanation: Label will convert your binary image into a labeled image, where each mask has a different integer value. Then, regionprops uses these labels in order to separate each mask, from which we can use the centroid property to compute the middle point from each - this is guaranteed to be a single point.
Simply finding all points above a threshold doesn't work, since there
are many such points around each maximum.
Actually, this does work - as long as you apply one more processing step. After thresholding, then we want to skeletonize. Scikit-image has a good function to achieve that here, which should give you a binary mask with single points.
Afterwards, you're probably going to want to run something like:
indices = zip(*np.where(skeleton))
to get your final points!
Based on Michael Lee's answer, here's the solution that worked for me (using all openCV rather than skimage):
# read in color image and create a grayscale copy
im = cv.imread("image.png")
img = cv.cvtColor(im, cv.COLOR_BGR2GRAY)
# apply thresholding
ret, im2 = cv.threshold(img, args.threshold, 255, cv.THRESH_BINARY)
# dilate the thresholded peaks to eliminate "pinholes"
im3 = cv.dilate(im2, None, iterations=2)
contours, hier = cv.findContours(im3, cv.RETR_TREE, cv.CHAIN_APPROX_SIMPLE)
print('found', len(contours), 'contours')
# draw a bounding box around each contour
for contour in contours:
x,y,w,h = cv.boundingRect(contour)
cv.rectangle(im, (x,y), (x+w,y+h), (255,0,0), 2)
cv.imshow('Contours', im)
cv.waitKey()
which results in just what we're looking for:
I'm making a document scanner for a college project, my code work quite well for any of the uniform lighted images. However I came across issues detecting images with even a little amount of light reflections (or too much light) on the background surface.
I first tried different simple codes I found online, then using different morphological operation, with the result that now my code is a little messy and inaccurate.
Here's the code:
def scanner(img):
clahe = cv2.createCLAHE(clipLimit=3.0, tileGridSize=(8,8))
image = cv2.imread(img)
original = image.copy()
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
contrast = clahe.apply(gray)
blurred = cv2.medianBlur(contrast, 21)
canny = cv2.Canny(blurred, 0, 70)
dialated = cv2.dilate(canny, cv2.getStructuringElement(cv2.MORPH_RECT,(5,5)), iterations = 3)
closing = cv2.morphologyEx(dialated, cv2.MORPH_CLOSE, np.ones((5,5),np.uint8),iterations = 10)
contimage, contours, hierarchy = cv2.findContours(closing, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
contours = sorted(contours, key = cv2.contourArea, reverse = True)[:5]
target = None
for c in contours:
p = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.09 * p, True)
if len(approx) == 4:
target = approx
cv2.drawContours(image, [target], -1, (0, 255, 0), 2)
break
plt.figure(figsize = (20,20))
plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
plt.title("final")
plt.show()
Here's an example of the code working:
input1, output1, input2, output2
And not working:
input1, output1, input2, output2
This image shows a successful segmentation:
This image shows a failed segmentation:
It looks like the main issue here is that in the failing images, there is not a sufficient contrast between the paper and the table it's placed on. the ones that work it's basically white paper on dark background, so it's relatively easy to segment out the paper, however when you place the paper on a light colored surface, there isn't sufficient contrast between the paper and the background to tell what is what. Unfortunately in image processing, there is only so much you can do when your input image is bad so there is no easy automated fix for this but I can think of a few workarounds, but they're all going to require extra work.
One would be instead of having your program automatically detect where the paper is, just have a static box that the user has to place the document inside of and simply capture the contents that way. Probably the most simple to implement however it seems like you WANT to detect it automatically so this probably isn't what you're looking for.
Two would be to have some intermediate step allowing the user to select a specific threshold value to apply to the image. Basically you would take the picture, then have the user set a threshold value such that the paper ends up being white, and the background is dark, then you could use that as a template to create the boundary of the paper which you can then segment from the original image. This is probably the most work but closest to what you're looking for.
Three would be similar to number one but instead of having a set area you place the documents within, you could take the picture then have the user manually select where the corners are and segment it that way, more work than #1, less work than #2, but probably still not what you're looking for.
Finally you could just leave it as is and use it knowing that you need a sufficiently dark background for it to work correctly. There are probably some other work arounds but with a lot of image processing stuff you can be very constrained by the quality of your image and there isn't always a software solution for exactly what you're looking to do.
i am working on a puzzle, my final task here is to identify edge type of the puzzle piece.
as shown in the above image i have mange to rotate and crop out every edge of the piece in same angle. my next step is to separate the edge line into a separate image like as shown in the image bellow
then to fill up one side of the line with with a color and try to process it to decide what type of edge it is.
i dont see a proper way to separate the edge line from the image for now.
my approach::
one way to do is scan pixel by pixel and find the black pixels where there is a nun black pixel next to it. this is a code that i can implement. but it feels like a primitive and a time consuming approach.
so if there you can offer any help or ideas, or any completely different way to detect the hollows and humps.
thanks in advance..
First convert your color image to grayscale. Then apply a threshold, say zero to obtain a binary image. You may have to use morphological operations to further process the binary image if there are holes. Then find the contours of this image and draw them to a new image.
A simple code is given below, using opencv 4.0.1 in python 2.7.
bgr = cv2.imread('puzzle.png')
gray = cv2.cvtColor(bgr, cv2.COLOR_BGR2GRAY)
_, roi = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY)
cv2.imwrite('/home/dhanushka/stack/roi.png', roi)
cont = cv2.findContours(roi, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
output = np.zeros(gray.shape, dtype=np.uint8)
cv2.drawContours(output, cont[0], -1, (255, 255, 255))
# removing boundary
boundary = 255*np.ones(gray.shape, dtype=np.uint8)
boundary[1:boundary.shape[0]-1, 1:boundary.shape[1]-1] = 0
toremove = output & boundary
output = output ^ toremove
I created one task, where I have white background and black digits.
I need to take the largest by thickness digit. I have made my picture bw, recognized all symbols, but I don't understand, how to scale thickness. I have tried arcLength(contours), but it gave me the largest by size. I have tried morphological operations, but as I undestood, it helps to remove noises and another mistakes in picture, right? And I had a thought to check the distance between neighbour points of contours, but then I thought that it would be hard because of not exact and clear form of symbols(I draw tnem on paint). So, that's all Ideas, that I had. Can you help me in this question by telling names of themes in Comp. vision and OpenCV, that could help me to solve this task? I don't need exact algorithm of solution, only themes. And if that's not OpenCV task, so which is? What library? Should I learn some pack of themes and basics before the solution of my task?
One possible solution that I can think of is to alternate erosion and find contours till you have only one contour left (that should be the thicker). This could work if the difference in thickness is enough, but I can also foresee many particular cases that can prevent a correct identification, so it depends very much on how is your original image.
Have you thought about drawing a line from a certain point of the contour and look for points where the line intersects your contour? I mean if you get the coordinates from two points you can measure the distance. I have made a sample to demonstrate what I mean. Note that this script is meant just for the demonstration of solution and it will not work with other pictures except my sample one. I would give a better one but I have only encountered with programming a few months back.
So the first thing is to extract the contours which you said you have already done (mind that cv2.findContours finds white values). then you can get referential coordinates with cv2.boundingRect() - it returns x,y coordinate, width and height of an bounding rectangle for your contour (you can of course do something similar by extracting a little fracture of your contour on a mask and work from there). In my example I defined the center of the box and moved the line slightly downwards then made a line to the left (I have done it by appending to lists and converting it to arrays and there are probably a million better solutions). Then you look for points that are in your contour and in your line (those points are the points of intersection). I have calculated simply by difference of two x coordinates because it works for this demonstration but better approach would be sqrt(x2-x1)^2+(y2-y1)^2. Maybe it will give you an idea. Cheers!
Sample code:
import cv2
import numpy as np
import numpy
img = cv2.imread('Thickness2.png')
gray_image = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(gray_image,10,255,cv2.THRESH_BINARY_INV)
im2, cnts, hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
font = cv2.FONT_HERSHEY_TRIPLEX
for c in cnts:
two_points = []
coord_x = []
coord_y = []
area = cv2.contourArea(c)
perimeter = cv2.arcLength(c, False)
if area > 1 and perimeter > 1:
x,y,w,h = cv2.boundingRect(c)
cx = int((x+(w/2))) -5
cy = int((y+(h/2))) +15
cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
for a in range(cx, cx+70):
coord_x.append(a)
coord_y.append(cy)
coord = list(zip(coord_x, coord_y))
arrayxy = np.array(coord)
arraycnt = np.array(c)
for a in arraycnt:
for b in arrayxy:
if a[:,0] == b[0] and a[:,1] == b[1]:
cv2.circle(img,(b[0],b[1]), 2, (0,255,255), -1)
two_points.append(b)
pointsarray = np.array(two_points)
thickness = int(pointsarray[1,0]) - int(pointsarray[0,0])
print(thickness)
cv2.line(img, (cx, cy), (cx+50, cy), (0,0,255), 1)
cv2.putText(img, 'Thickness : '+str(thickness),(x-20, y-10), font, 0.4,(0,0,0),1,cv2.LINE_AA)
cv2.imshow('img', img)
Output:
I read this blog post where he uses a Laser and a Webcam to estimated the distance of the cardboard from the Webcam.
I had another idea about that. I don't want to calculate the distance from the webcam.
I want to check if an object is approaching the webcam. The algorithm, according to me, will be something like:
Detect the object in the webcam feed.
If the object is approaching the webcam it'll grow larger and larger in the video feed.
Use this data for further calculations.
Since I want to detect random objects, I am using the findContours() method to find the contours in the video feed. Using that, I will at least have the outlines of the objects in the video feed. The source code is:
import numpy as np
import cv2
vid=cv2.VideoCapture(0)
ans, instant=vid.read()
average=np.float32(instant)
cv2.accumulateWeighted(instant, average, 0.01)
background=cv2.convertScaleAbs(average)
while(1):
_,f=vid.read()
imgray=cv2.cvtColor(f, cv2.COLOR_BGR2GRAY)
ret, thresh=cv2.threshold(imgray,127,255,0)
diff=cv2.absdiff(f, background)
cv2.imshow("input", f)
cv2.imshow("Difference", diff)
if cv2.waitKey(5)==27:
break
cv2.destroyAllWindows()
The output is:
I am stuck here. I have the contours stored in an array. What do I do with it when the size increases? How do I proceed?
One trouble here is recognising and differentiating the moving objects from other stuff in the video feed. An approach might be to let the camera 'learn' what the background looks like with no object. Then you can constantly compare its input against this background. One way to get the background is to use a running average.
Any difference greater than a small threshold means there is a moving object. If you constantly display this difference, you basically have a motion tracker. The size of the objects is simply the sum of all the non-zero (thresholded) pixels, or their bounding rectangles. You can track this size and use it to guess whether the object is moving closer or further. Morphological operations can help group the contours into one cohesive object.
Since it will be tracking ANY movement, if there are two objects, they will be counted together. Here is where you can use the contours to find and track individual objects, e.g. using the contour bounds or centroids. You could also possibly separate them by colour.
Here are some results using this strategy (the grey blob is my hand):
It actually did a fairly good job of guessing which way my hand was moving.
Code:
import cv2
import numpy as np
AVERAGE_ALPHA = 0.2 # 0-1 where 0 never adapts, and 1 instantly adapts
MOVEMENT_THRESHOLD = 30 # Lower values pick up more movement
REDUCED_SIZE = (400, 600)
MORPH_KERNEL = np.ones((10, 10), np.uint8)
def reduce_image(input_image):
"""Make the image easier to deal with."""
reduced = cv2.resize(input_image, REDUCED_SIZE)
reduced = cv2.cvtColor(reduced, cv2.COLOR_BGR2GRAY)
return reduced
# Initialise
vid = cv2.VideoCapture(0)
average = None
old_sizes = np.zeros(20)
size_update_index = 0
while (True):
got_frame, frame = vid.read()
if got_frame:
# Reduce image
reduced = reduce_image(frame)
if average is None: average = np.float32(reduced)
# Get background
cv2.accumulateWeighted(reduced, average, AVERAGE_ALPHA)
background = cv2.convertScaleAbs(average)
# Get thresholded difference image
movement = cv2.absdiff(reduced, background)
_, threshold = cv2.threshold(movement, MOVEMENT_THRESHOLD, 255, cv2.THRESH_BINARY)
# Apply morphology to help find object
dilated = cv2.dilate(threshold, MORPH_KERNEL, iterations=10)
closed = cv2.morphologyEx(dilated, cv2.MORPH_CLOSE, MORPH_KERNEL)
# Get contours
contours, _ = cv2.findContours(closed, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(closed, contours, -1, (150, 150, 150), -1)
# Find biggest bounding rectangle
areas = [cv2.contourArea(c) for c in contours]
if (areas != list()):
max_index = np.argmax(areas)
max_cont = contours[max_index]
x, y, w, h = cv2.boundingRect(max_cont)
cv2.rectangle(closed, (x, y), (x+w, y+h), (255, 255, 255), 5)
# Guess movement direction
size = w*h
if size > old_sizes.mean():
print "Towards"
else:
print "Away"
# Update object size
old_sizes[size_update_index] = size
size_update_index += 1
if (size_update_index) >= len(old_sizes): size_update_index = 0
# Display image
cv2.imshow('RaptorVision', closed)
Obviously this needs more work in terms of identifying, selecting and tracking the objects etc (at the moment it does horribly if there is something else moving in the background). There are also many parameters to vary and tweak (the ones set are what worked well for my system). I'll leave that up to you though.
Some links:
background extraction
motion tracking
If you want to get a bit more high-tech with the background removal, have a look here:
wallflower
Detect the object in the webcam feed.
If the object is approaching the webcam it'll grow larger and larger in the video feed.
Use this data for further calculations.
Good idea.
If you want to use the contour detection approach, you could do it the following way:
You have a series of Images I1, I2, ... In
Do a contour detection on each one. C1, C2, ..., Cn (Contour is a set of points in OpenCV)
Take a large enough sample on your Image i and i+1: S_i \leq C_i, i \in 1...n
Check for all points in your sample for the nearest point on i+1. Then you trajectorys for all your points.
Check if this trajectorys point mostly outwards (tricky part ;)
If they appear outwards for a suffiecent number of frames your contour got bigger.
Alternative you could try to prune the points that are not part of the correct contour and work with a covering rectangle. It's very easy to check the size that way, but i don't knwo how easy it will be to choose the "correct" points.