I have a very simple task which is detecting some very obvious blobs. I am very new to image processing, I am probably making a very simple mistake. I prepared a blob detection function by using this example and I am using this function on an artificial image. My original task is to detect blob in a much complex image but I thought I can start small. Originally I have a gray image with small whitish(less gray) dirts over it. Anyways my problem is I cannot detect any blobs. I think I am failing on adjusting the parameters. Although I tried many different combinations, I couldn't get any result. Here is my simpler artificial image:
Here is my code
import matplotlib.pyplot as plt
import numpy as np
import cv2
def normalizeImage(image):
print("Normalizing image..")
min = np.min(image)
image = image-min #to have only positive values
max=np.max(image)
div=max/255 #calculate the normalize divisor
image_8u = np.uint8(np.round(image / div))
return image_8u
def detectBlobs(dustImage):
print("Detecting blobs...")
pp = cv2.SimpleBlobDetector_Params()
pp.filterByArea = True# Set Area filtering parameters
#minDiameter = 1
#maxDiameter = 5
pp.minArea = 100 #3.14159 * minDiameter * minDiameter
#pp.maxArea = 3.14159 * maxDiameter * maxDiameter
pp.filterByCircularity = True # Set Circularity filtering parameters
pp.minCircularity = 0.9
pp.filterByConvexity = True # Set Convexity filtering parameters
pp.minConvexity = 0.2
pp.filterByInertia = True # Set inertia filtering parameters
pp.minInertiaRatio = 0.1
detector = cv2.SimpleBlobDetector_create(pp) # Create a detector with the parameters
keypoints = detector.detect(dustImage) # Detect blobs
numberOfBlobs = len(keypoints)
if(numberOfBlobs > 0):
print(numberOfBlobs + "blobs detected!")
blank = np.zeros((1, 1))# Draw blobs on our image as red circles
blobs = cv2.drawKeypoints(dustImage, keypoints, blank, (0, 0, 255), cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
text = "Number of Circular Blobs: " + str(len(keypoints))
cv2.putText(blobs, text, (20, 550), cv2.FONT_HERSHEY_SIMPLEX, 2, (0, 100, 255), 3)
# Show blobs
cv2.namedWindow('img', cv2.WINDOW_NORMAL)
cv2.resizeWindow("img", 900 ,900)
cv2.imshow("img", blobs)
cv2.waitKey(0)
cv2.destroyAllWindows()
else:
print('No blobs detected')
return numberOfBlobs
if __name__ == '__main__':
img0 = plt.imread('artificial.png')
#plt.imshow(img0)
#plt.show()
img1 = normalizeImage(img0)
img2 = cv2.cvtColor(img1, cv2.CV_8UC1)
detectBlobs(img2)
exit()
As part of a program which contains a series of images to be processed, I first need to detect a green-coloured rectangle. I'm trying to write a program that doesn't use colour masking, since the lighting and glare on the images will make it difficult to find the appropriate HSV ranges.
(p.s. I already have two questions based on this program, but this one is unrelated to those. It's not a follow up, I want to address a separate issue.)
I used the standard rectangle detection technique, making use of findContours() and approxPolyDp() methods. I added some constraints that got rid of unnecessary rectangles (like aspectRatio>2.5, since my desired rectangle is clearly the "widest" and area>1500, to discard random small rectangles) .
import numpy as np
import cv2 as cv
img = cv.imread("t19.jpeg")
width=0
height=0
start_x=0
start_y=0
end_x=0
end_y=0
output = img.copy()
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
#threshold
th = cv.adaptiveThreshold(gray,255,cv.ADAPTIVE_THRESH_GAUSSIAN_C,cv.THRESH_BINARY,9,2)
cv.imshow("th",th)
#rectangle detection
contours, _ = cv.findContours(th, cv.RETR_TREE, cv.CHAIN_APPROX_NONE)
for contour in contours:
approx = cv.approxPolyDP(contour, 0.01* cv.arcLength(contour, True), True)
cv.drawContours(img, [approx], 0, (0, 0, 0), 5)
x = approx.ravel()[0]
y = approx.ravel()[1]
x1 ,y1, w, h = cv.boundingRect(approx)
a=w*h
if len(approx) == 4 and x>15 :
aspectRatio = float(w)/h
if aspectRatio >= 2.5 and a>1500:
print(x1,y1,w,h)
width=w
height=h
start_x=x1
start_y=y1
end_x=start_x+width
end_y=start_y+height
cv.rectangle(output, (start_x,start_y), (end_x,end_y), (0,0,255),3)
cv.putText(output, "rectangle "+str(x1)+" , " +str(y1-5), (x1, y1-5), cv.FONT_HERSHEY_COMPLEX, 0.5, (0, 0, 0))
cv.imshow("op",output)
print("start",start_x,start_y)
print("end", end_x,end_y)
print("width",width)
print("height",height)
It is working flawlessly for all the images, except one:
I used adaptive thresholding to create the threshold, which was used by the findContours() method.
I tried displaying the threshold and the output , and it looks like this:
The thresholds for the other images also looked similar...so I can't pinpoint what exactly has gone wrong in the rectangle detection procedure.
Some tweaks I have tried:
Changing the last two parameters in the adaptive parameters method.
I tried 11,1 , 9,1, and for both of them, the rectangle in the
threshold looked more prominent : but in this case the output
detected no rectangles at all.
I have already disregarded otsu thresholding, as it is not working
for about 4 of my test images.
What exactly can I tweak in the rectangle detection procedure for it to detect this rectangle?
I also request , if possible, only slight modifications to this method and not some entirely new method. As I have mentioned, this method is working perfectly for all of my other test images, and if the new suggested method works for this image but fails for the others, then I'll find myself back here asking why it failed.
Edit: The method that abss suggested worked for this image, however failed for:
image 4
image 1, far off
Other test images:
image 1, normal
image 2
image 3
image 9, part 1
image 9, part 2
You can easily do it by adding this line of code after your threshold
kernel = cv.getStructuringElement(cv.MORPH_RECT,(3,3))
th = cv.morphologyEx(th,cv.MORPH_OPEN,kernel)
This will remove noise within the image. you can see this link for more understanding about morphologyEx https://docs.opencv.org/master/d9/d61/tutorial_py_morphological_ops.html
The results I got is shown below
I have made a few modifications to your code so that it works with all of your test images. There are a few false positives that you may have to filter based on HSV color range for green (since your target is always a shade of green). Alternately you can take into account the fact that the one of the child hierarchy of your ROI contour is going to be > 0.4 or so times than the outer contour. Here are the modifications:
Used DoG for thresholding useful contours
Changed arcLength multiplier to 0.5 instead of 0.1 as square corners are not smooth
cv2.RETR_CCOMP to get 2 level hierarchy
Moved ApproxPolyDP inside to make it more efficient
Contour filter area changed to 600 to filter ROI for all test images
Removed a little bit of unnecessary code
Check with all the other test images that you may have and modify the parameters accordingly.
img = cv2.imread("/path/to/your_image")
width=0
height=0
start_x=0
start_y=0
end_x=0
end_y=0
output = img.copy()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gw, gs, gw1, gs1, gw2, gs2 = (3,1.0,7,3.0, 3, 2.0)
img_blur = cv2.GaussianBlur(gray, (gw, gw), gs)
g1 = cv2.GaussianBlur(img_blur, (gw1, gw1), gs1)
g2 = cv2.GaussianBlur(img_blur, (gw2, gw2), gs2)
ret, thg = cv2.threshold(g2-g1, 127, 255, cv2.THRESH_BINARY)
contours, hier = cv2.findContours(thg, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_NONE)
img_cpy = img.copy()
width=0
height=0
start_x=0
start_y=0
end_x=0
end_y=0
for i in range(len(contours)):
if hier[0][i][2] == -1:
continue
x ,y, w, h = cv2.boundingRect(contours[i])
a=w*h
aspectRatio = float(w)/h
if aspectRatio >= 2.5 and a>600:
approx = cv2.approxPolyDP(contours[i], 0.05* cv2.arcLength(contours[i], True), True)
if len(approx) == 4 and x>15 :
width=w
height=h
start_x=x
start_y=y
end_x=start_x+width
end_y=start_y+height
cv2.rectangle(img_cpy, (start_x,start_y), (end_x,end_y), (0,0,255),3)
cv2.putText(img_cpy, "rectangle "+str(x)+" , " +str(y-5), (x, y-5), cv2.FONT_HERSHEY_COMPLEX, 0.5, (0, 0, 0))
plt.imshow(img_cpy)
print("start",start_x,start_y)
print("end", end_x,end_y)
I'm trying to make a general object counting algorithm using python and openCV (open to try other methods) however I can't seem to get a good count on a variety of objects and don't know how to accomodate for that
https://imgur.com/a/yAkRxWH are some example test images.
This is for to speed up inventory counting of smaller objects.
**EDIT
This is my current code (simple blob detector)
# Standard imports
import cv2
import numpy as np;
# Read image
im = cv2.imread("./images/screw_simple.jpg", cv2.IMREAD_GRAYSCALE)
im = cv2.resize(im, (1440, 880))
# Setup SimpleBlobDetector parameters.
params = cv2.SimpleBlobDetector_Params()
# Change thresholds
params.minThreshold = 10 #10
params.maxThreshold = 200 #200
# Filter by Area.
params.filterByArea = True # True
params.minArea = 500 #1500
# Filter by Circularity
params.filterByCircularity = True #True
params.minCircularity = 0.1 #0.1
# Filter by Convexity
params.filterByConvexity = True #True
params.minConvexity = 0.0 #0.87
# Filter by Inertia
params.filterByInertia = True #True
params.minInertiaRatio = 0.0 #0.01
# Create a detector with the parameters
ver = (cv2.__version__).split('.')
if int(ver[0]) < 3:
detector = cv2.SimpleBlobDetector(params)
else:
detector = cv2.SimpleBlobDetector_create(params)
# Detect blobs.
keypoints = detector.detect(im)
# Draw detected blobs as red circles.
# cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS ensures
# the size of the circle corresponds to the size of blob
total_count = 0
for i in keypoints:
total_count = total_count + 1
im_with_keypoints = cv2.drawKeypoints(im, keypoints, np.array([]), (0, 0, 255), cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
# Show blobs
cv2.imshow("Keypoints", im_with_keypoints)
cv2.waitKey(0)
print(total_count)
Here are the results I'm getting: https://imgur.com/a/id6OlIA
How can I improve this algorithm to get better detection for a general use case of objects without having to modify the parameters each time for each object?
You can try with an OpenCV approach, you could use a
SimpleBlobDetector
Obviously this is a test image and the result I got is also not perfect, since there are a lot of hyperparameters to set. The hyperparameters make it pretty flexible, so it is a decent place to start from.
This is what the Detector does (see details here):
Thresholding: Convert the source images to several binary images by thresholding the source image with thresholds starting at minThreshold. These thresholds are incremented by thresholdStep until maxThreshold. So the first threshold is minThreshold, the second is minThreshold + thresholdStep, the third is minThreshold + 2 x thresholdStep, and so on.
Grouping: In each binary image, connected white pixels are grouped together. Let’s call these binary blobs.
Merging: The centers of the binary blobs in the binary images are computed, and blobs located closer than minDistBetweenBlobs are merged.
Center & Radius Calculation: The centers and radii of the new merged blobs are computed and returned.
Find the code bellow the image.
# Standard imports
import cv2
import numpy as np
# Read image
im = cv2.imread("petri.png", cv2.IMREAD_COLOR)
# Setup SimpleBlobDetector parameters.
params = cv2.SimpleBlobDetector_Params()
# Change thresholds
params.minThreshold = 0
params.maxThreshold = 255
# Set edge gradient
params.thresholdStep = 5
# Filter by Area.
params.filterByArea = True
params.minArea = 10
# Set up the detector with default parameters.
detector = cv2.SimpleBlobDetector_create(params)
# Detect blobs.
keypoints = detector.detect(im)
# Draw detected blobs as red circles.
# cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS ensures the size of the circle corresponds to the size of blob
im_with_keypoints = cv2.drawKeypoints(im, keypoints, np.array([]), (0, 0, 255),
cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
# Show keypoints
cv2.imshow("Keypoints", im_with_keypoints)
cv2.waitKey(0)
For better readability I rather put this in a second answer: You could
use a segmentation approach e.g. watershed algorithm
Any grayscale image can be viewed as a topographic surface where high intensity denotes peaks and hills while low intensity denotes valleys. You start filling every isolated valleys (local minima) with different colored water (labels). As the water rises, depending on the peaks (gradients) nearby, water from different valleys, obviously with different colors will start to merge. To avoid that, you build barriers in the locations where water merges. You continue the work of filling water and building barriers until all the peaks are under water. Then the barriers you created gives you the segmentation result. This is the "philosophy" behind the watershed.
I have some colored rectangles in my image that I successfully recognize by HSV thresholding. The result looks like this:
Now I want to detect the big blob as one point. I tried it with cv2.SimpleBlobDetector() and custom parameters:
import cv2
import numpy as np
mask = cv2.imread('mask.png')
original = cv2.imread('original.png')
params = cv2.SimpleBlobDetector_Params()
# thresholds
params.minThreshold = 10
params.maxThreshold = 200
#params.thresholdStep = 20
# filter by area
params.filterByArea = True
params.minArea = 1
params.maxArea = 10000
# filter by circularity
params.filterByCircularity = False
# filter by convexity
params.filterByConvexity = False
# filter by inertia
params.filterByInertia = False
detector = cv2.SimpleBlobDetector(params)
keypoints = detector.detect(mask)
img_keypoints = cv2.drawKeypoints(original, keypoints, np.array([]), (0,0,255), cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
cv2.imwrite('keypoints.png', img_keypoints)
This is how the result AND the original picture looks like:
I would expect a red point sitting in the center of the green point.
How can I fix this? Help is very appreciated.
EDIT: I forgot to mention: Is it really necessary to generate multiple binary images in cv2.SimpleBlobDetector() since I already have a binary image as input? Is it OK to change the values to the following:
params.minThreshold = 127
params.maxThreshold = 127
to reduce unnecessary CPU usage by generating binary images?
EDIT2 : Please note that I'm using OpenCV 2, not 3
Thank you.
When using cv2.SimpleBlobDetector(), it looks for blobs that are of a darker shade. In your case, the rectangle in mask is in white while the rest of the image is dark. As a result it is unable to find any blobs for the custom parameters set.
I just made a few changes to the existing code:
Read the mask as a grayscale image not a color image:
mask = cv2.imread('mask.png', 0)
Convert the mask to a binary image with the rectangle highlighted in dark:
ret, mask = cv2.threshold(mask, 127, 255, cv2.THRESH_BINARY_INV)
Proceeding from here using you code gave the following result as you expected.
Result:
So I am trying to plan the approach I want to take to count vehicles and pedestrians in a video. Here are my basic steps for the approach I want to take.
Use background subtraction to distinguish between moving objects.
Use cv2.SimpleBlobDetector to detect blobs from the mask generated in the BGS step and return the keypoints.
Perform tracking of all blobs ( Not yet implemented in the example ) with the given keypoints.
The question: Can this approach be applied to both pedestrian and vehicles and if so, I am not clear on how can one distinguish the different blobs?
I am wondering if may be the size of the blob can be a used to distinguish between pedestrians ( small blobs ) and vehicles ( larger blobs ). However, I am not sure how to handle the case of a vehicle being further away from the source and hence appearing to be small.
import numpy as np
import cv2
cap = cv2.VideoCapture('video.avi')
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3))
fgbg = cv2.BackgroundSubtractorMOG(500, 6, 0.9, 1)
# Setup SimpleBlobDetector parameters.
params = cv2.SimpleBlobDetector_Params()
# Change thresholds
params.minThreshold = 10;
params.maxThreshold = 200;
# Filter by Area.
params.filterByArea = True
params.minArea = 400
# Filter by Circularity
params.filterByCircularity = True
params.minCircularity = 0.1
# Filter by Convexity
params.filterByConvexity = True
params.minConvexity = 0.87
# Filter by Inertia
params.filterByInertia = True
params.minInertiaRatio = 0.01
# Create a detector with the parameters
ver = (cv2.__version__).split('.')
if int(ver[0]) < 3 :
detector = cv2.SimpleBlobDetector(params)
else :
detector = cv2.SimpleBlobDetector_create(params)
while(1):
ret, frame = cap.read()
fgmask = fgbg.apply(frame)
fgmask = cv2.morphologyEx(fgmask, cv2.MORPH_OPEN, kernel)
#fgmask = frame;
# Detect blobs.
keypoints = detector.detect(fgmask)
# Draw detected blobs as red circles.
# cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS ensures the size of the circle corresponds to the size of blob
im_with_keypoints = cv2.drawKeypoints(frame, keypoints, np.array([]), (0,0,255), cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
print keypoints
cv2.imshow('frame',im_with_keypoints)
k = cv2.waitKey(30) & 0xff
if k == 27:
break
cap.release()
cv2.destroyAllWindows()
I would suggest not to take blob area approach as a way to distinguish pedestrians from vehicles. An obvious drawback has been already explained by you - more distant cars would be for sure taken as a pedestrians.
There is a necessity to involve more complicated logic between steps 2 and 3 e.g.:
person detector based on HOG - see http://lear.inrialpes.fr/people/triggs/pubs/Dalal-cvpr05.pd. Already implemented in OpenCV, empirecally to have good accuracy in person detection.
car detector / car features detector (wheels, plate number etc) based on Haar classifier - need to prepare your own with the tools opencv provided. Having good classifier for detecting some car feature, you will increase accuracy of overall detection by increasing number of car detected positives.
Having at least one of those bullets in your final solution would be a must-have for me to have a good accuracy solution. Having them both will even have accuracy better.