I am working on the blur detection of images. I have used the variance of the Laplacian method in OpenCV.
img = cv2.imread(imgPath)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
value = cv2.Laplacian(gray, cv2.CV_64F).var()
The function failed in some cases like pixelated blurriness. It shows a higher value for those blur images than the actual clear images. Is there any better approach that detects Pixelated as well as motion blurriness?
Sample images:
This image is much clearer but showing value of 266.79
Where as this image showing the value of 446.51 .
Also this image seems to be much clearer but showing value only 38.96
I need to classify 1st and 3rd one as not blur whereas the second one as a blur.
I may be late to answer this one, but here is one potential approach.
The blur_detector library in pypi can be used to identify regions in an image which are sharp vs blurry. Here is the paper on which the library is created: https://arxiv.org/pdf/1703.07478.pdf
The way this library operates is that it looks at every pixel in the image at multiple scales and performs the discrete cosine transform at each scale. These DCT coefficients are then filtered such that we only use the high frequency coefficients. All the high frequency DCT coefficients at multiple scales are then fused together and sorted to form the multiscale-fused and sorted high-frequency transform coefficients
A subset of these sorted coefficients is selected. This is a tunable parameter and user can experiment with it based on the application. The output of the selected DCT coefficients is then sent through a max pooling to retain the maximum activation at multiple scales. This makes the algorithm quite robust to detect blurry areas in an image.
Here are the results that I see on the images that you have provided in the question:
Note: I have used a face detector from the default cascade_detectors in opencv to select a region of interest. the output of these two approaches (spatial blur detection + face detection) can be used to get the sharpness map in the image.
Here we can see that in the sharp images, the intensity of the pixels in the eyes region is very high, whereas for the blurry image, it is low.
You can threshold this to identify which images are sharp and which images are blurry.
Here is the code snippet which generated the above results:
pip install blur_detector
import blur_detector
import cv2
if __name__ == '__main__':
face_cascade = cv2.CascadeClassifier('cv2/data/haarcascade_frontalface_default.xml')
img = cv2.imread('1.png', 0)
blur_map1 = blur_detector.detectBlur(img, downsampling_factor=1, num_scales=3, scale_start=1)
faces = face_cascade.detectMultiScale(img, 1.1, 4)
for (x, y, w, h) in faces:
cv2.rectangle(blur_map1, (x, y), (x + w, y + h), (255, 0, 0), 2)
img = cv2.imread('2.png', 0)
blur_map2 = blur_detector.detectBlur(img, downsampling_factor=1, num_scales=3, scale_start=1)
faces = face_cascade.detectMultiScale(img, 1.1, 4)
for (x, y, w, h) in faces:
cv2.rectangle(blur_map2, (x, y), (x + w, y + h), (255, 0, 0), 2)
img = cv2.imread('3.png', 0)
blur_map3 = blur_detector.detectBlur(img, downsampling_factor=1, num_scales=3, scale_start=1)
faces = face_cascade.detectMultiScale(img, 1.1, 4)
for (x, y, w, h) in faces:
cv2.rectangle(blur_map3, (x, y), (x + w, y + h), (255, 0, 0), 2)
cv2.imshow('a', blur_map1)
cv2.imshow('b', blur_map2)
cv2.imshow('c', blur_map3)
cv2.waitKey(0)
To understand the details about the algorithm regarding the blur detector, please take a look at this github page: https://github.com/Utkarsh-Deshmukh/Spatially-Varying-Blur-Detection-python
You can try to define a threshold as float, so for every result falling under the threshold == blurry. But if the pixel images shows very high every time, even if not blurry, you could check for another value that is very high. Another way might be to detect focus of the picture.
Related
I am in the initial stages of writing a Rubik's cube solver and am stuck at the following challenge:
Using the following image-processing code gives me the following image:
import cv2 as cv
import glob
import numpy as np
for img in glob.glob("captured_images/*.jpg"):
image = cv.imread(img)
copy = image.copy()
grey = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
decrease_noise = cv.fastNlMeansDenoising(grey, 10, 15, 7, 21)
blurred = cv.GaussianBlur(decrease_noise, (3, 3), 0)
canny = cv.Canny(blurred, 20, 40)
thresh = cv.threshold(canny, 0, 255, cv.THRESH_OTSU + cv.THRESH_BINARY)[1]
contours = cv.findContours(thresh, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for c in contours:
# obtain the bounding rectangle coordinates for each square
x, y, w, h = cv.boundingRect(c)
# With the bounding rectangle coordinates we draw the green bounding boxes
cv.rectangle(copy, (x, y), (x + w, y + h), (36, 255, 12), 2)
cv.imshow('copy', copy)
cv.waitKey(0)
cv.destroyAllWindows()
There are numerous bound rectangles highlighted. Trying to filter out only the squares using this code:
contour_list = []
for contour in contours:
approx = cv.approxPolyDP(contour, 0.01 * cv.arcLength(contour, True), True)
area = cv.contourArea(contour)
if len(approx) == 4:
(x, y, w, h) = cv.boundingRect(approx)
if (float(w)/h) == 1:
cv.rectangle(copy, (x, y), (x + w, y + h), (36, 255, 12), 2)
contour_list.append(contour)
doesn't work as the squares aren't precise enough to fit the definition of "all sides of square are equal".
I though retaking the images against a white background might help to more easily find the relevant squares, however modifying the original image to a cube with a white background and using the original code causes only the larger cube to be recognised as a square:
My question is three-fold:
1a) How can I modify my original code for the original image to accurately measure only the relevant squares by using the following criteria for finding squares:
There must be four corners
All four lines must be roughly the same length
All four corners must be roughly 90 degrees
1b) In the second image with the white background, how can I select everything outside the bound rectangle and convert that white background to black, which helps greatly in correctly detecting the appropriate squares?
1c) In general, why is a black background so much more beneficial than a white background in using the cv2.rectangle() function?
Any help in gaining some clearer understanding is much appreciated! :)
How can I modify my original code for the original image to accurately measure only the relevant squares by using the following criteria for finding squares:
Your code only accepts contours that are exactly square. You need to have a "squaredness" factor and then determine some acceptable threshold.
The "squaredness" factor is h/w if w > h else w/h. The closer that value to one, the more square the rectangle is. Then you can accept only rectangles with a factor of .9 or higher (or whatever works best).
In general, why is a black background so much more beneficial than a white background in using the cv2.rectangle() function?
The contour finding algorithm that OpenCV uses is actually:
Suzuki, S. and Abe, K., Topological Structural Analysis of Digitized Binary Images by Border Following. CVGIP 30 1, pp 32-46 (1985)
In your case, the algorithm might just have picked up the contours just fine, but you have set the RETR_EXTERNAL flag, which will cause OpenCV to only report the outermost contours. Try changing it to RETR_LIST.
Find the OpenCV docs with regards to contour finding here: https://docs.opencv.org/master/d9/d8b/tutorial_py_contours_hierarchy.html
As part of a program which contains a series of images to be processed, I first need to detect a green-coloured rectangle. I'm trying to write a program that doesn't use colour masking, since the lighting and glare on the images will make it difficult to find the appropriate HSV ranges.
(p.s. I already have two questions based on this program, but this one is unrelated to those. It's not a follow up, I want to address a separate issue.)
I used the standard rectangle detection technique, making use of findContours() and approxPolyDp() methods. I added some constraints that got rid of unnecessary rectangles (like aspectRatio>2.5, since my desired rectangle is clearly the "widest" and area>1500, to discard random small rectangles) .
import numpy as np
import cv2 as cv
img = cv.imread("t19.jpeg")
width=0
height=0
start_x=0
start_y=0
end_x=0
end_y=0
output = img.copy()
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
#threshold
th = cv.adaptiveThreshold(gray,255,cv.ADAPTIVE_THRESH_GAUSSIAN_C,cv.THRESH_BINARY,9,2)
cv.imshow("th",th)
#rectangle detection
contours, _ = cv.findContours(th, cv.RETR_TREE, cv.CHAIN_APPROX_NONE)
for contour in contours:
approx = cv.approxPolyDP(contour, 0.01* cv.arcLength(contour, True), True)
cv.drawContours(img, [approx], 0, (0, 0, 0), 5)
x = approx.ravel()[0]
y = approx.ravel()[1]
x1 ,y1, w, h = cv.boundingRect(approx)
a=w*h
if len(approx) == 4 and x>15 :
aspectRatio = float(w)/h
if aspectRatio >= 2.5 and a>1500:
print(x1,y1,w,h)
width=w
height=h
start_x=x1
start_y=y1
end_x=start_x+width
end_y=start_y+height
cv.rectangle(output, (start_x,start_y), (end_x,end_y), (0,0,255),3)
cv.putText(output, "rectangle "+str(x1)+" , " +str(y1-5), (x1, y1-5), cv.FONT_HERSHEY_COMPLEX, 0.5, (0, 0, 0))
cv.imshow("op",output)
print("start",start_x,start_y)
print("end", end_x,end_y)
print("width",width)
print("height",height)
It is working flawlessly for all the images, except one:
I used adaptive thresholding to create the threshold, which was used by the findContours() method.
I tried displaying the threshold and the output , and it looks like this:
The thresholds for the other images also looked similar...so I can't pinpoint what exactly has gone wrong in the rectangle detection procedure.
Some tweaks I have tried:
Changing the last two parameters in the adaptive parameters method.
I tried 11,1 , 9,1, and for both of them, the rectangle in the
threshold looked more prominent : but in this case the output
detected no rectangles at all.
I have already disregarded otsu thresholding, as it is not working
for about 4 of my test images.
What exactly can I tweak in the rectangle detection procedure for it to detect this rectangle?
I also request , if possible, only slight modifications to this method and not some entirely new method. As I have mentioned, this method is working perfectly for all of my other test images, and if the new suggested method works for this image but fails for the others, then I'll find myself back here asking why it failed.
Edit: The method that abss suggested worked for this image, however failed for:
image 4
image 1, far off
Other test images:
image 1, normal
image 2
image 3
image 9, part 1
image 9, part 2
You can easily do it by adding this line of code after your threshold
kernel = cv.getStructuringElement(cv.MORPH_RECT,(3,3))
th = cv.morphologyEx(th,cv.MORPH_OPEN,kernel)
This will remove noise within the image. you can see this link for more understanding about morphologyEx https://docs.opencv.org/master/d9/d61/tutorial_py_morphological_ops.html
The results I got is shown below
I have made a few modifications to your code so that it works with all of your test images. There are a few false positives that you may have to filter based on HSV color range for green (since your target is always a shade of green). Alternately you can take into account the fact that the one of the child hierarchy of your ROI contour is going to be > 0.4 or so times than the outer contour. Here are the modifications:
Used DoG for thresholding useful contours
Changed arcLength multiplier to 0.5 instead of 0.1 as square corners are not smooth
cv2.RETR_CCOMP to get 2 level hierarchy
Moved ApproxPolyDP inside to make it more efficient
Contour filter area changed to 600 to filter ROI for all test images
Removed a little bit of unnecessary code
Check with all the other test images that you may have and modify the parameters accordingly.
img = cv2.imread("/path/to/your_image")
width=0
height=0
start_x=0
start_y=0
end_x=0
end_y=0
output = img.copy()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gw, gs, gw1, gs1, gw2, gs2 = (3,1.0,7,3.0, 3, 2.0)
img_blur = cv2.GaussianBlur(gray, (gw, gw), gs)
g1 = cv2.GaussianBlur(img_blur, (gw1, gw1), gs1)
g2 = cv2.GaussianBlur(img_blur, (gw2, gw2), gs2)
ret, thg = cv2.threshold(g2-g1, 127, 255, cv2.THRESH_BINARY)
contours, hier = cv2.findContours(thg, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_NONE)
img_cpy = img.copy()
width=0
height=0
start_x=0
start_y=0
end_x=0
end_y=0
for i in range(len(contours)):
if hier[0][i][2] == -1:
continue
x ,y, w, h = cv2.boundingRect(contours[i])
a=w*h
aspectRatio = float(w)/h
if aspectRatio >= 2.5 and a>600:
approx = cv2.approxPolyDP(contours[i], 0.05* cv2.arcLength(contours[i], True), True)
if len(approx) == 4 and x>15 :
width=w
height=h
start_x=x
start_y=y
end_x=start_x+width
end_y=start_y+height
cv2.rectangle(img_cpy, (start_x,start_y), (end_x,end_y), (0,0,255),3)
cv2.putText(img_cpy, "rectangle "+str(x)+" , " +str(y-5), (x, y-5), cv2.FONT_HERSHEY_COMPLEX, 0.5, (0, 0, 0))
plt.imshow(img_cpy)
print("start",start_x,start_y)
print("end", end_x,end_y)
I am playing around with the SVM-HOG pipeline for pedestrian detection using OpenCV in python.
I have a lot of images of different sizes, some are small(e.g. 21×32), and some are larger(e.g. 127×264).
I first defined my HOG descriptor as hog = cv2.HOGDescriptor()
When I compute the HOG features by calling h = hog.compute(image) I found that when the image is smaller than 64*128, the descriptor won't be able to compute the features, instead, it just terminates the program.
The original paper used 64*128 sized images, but I think it also stated it is ok to use images with the same aspect ratio which is 1:2 (correct me if I'm wrong).
My question is that is 64*128 the smallest size that a HOG descriptor can compute the feature on? because I tried to compute HOG features on images resized to 32*64, and it won't work.
Thanks.
for entries in os.listdir(dir_path):
if entries.endswith(".jpg"):
img = cv2.imread(os.path.join(test_path, entries))
rects, scores = hog.detectMultiScale(img, winStride=(4,4), padding=(8,8), scale=1.05)
sc = [score[0] for score in scores]
for i in range(len(rects)):
r = rects[i]
rects[i][2] = r[0]+r[2]
rects[i][3] = r[1]+r[3]
pick =[]
pick = non_max_suppression(rects, probs = sc, overlapThresh = 0.3)
for (x,y,w,h) in pick:
cv2.rectangle(img, (x, y), (w, h), (0, 255, 0), 1)
cv2.imshow("A",img)
I really don't know if "UV's" is the right word as i'm from the world of Unity and am trying to write some stuff in python. What i'm trying to do is to take a picture of a human (from webcam) take the placement of their landmarks/key features and alter a second image (of a different person) to make their key features in the same place whilst morphing / warping the parts of their skin that are within the face to fit the position of the first input image (webcam)'s landmarks. After i do that I need to put the face back on the non-webcam input. (i'm sorry for how much that made me sound like a serial killer, stretching and cutting faces) I know that probably didn't make any sense but I want it to look like this.
I have the face landmark and cutting done with DLIB and OpenCV but i need a way to find a way to take these "cut" face chunks and stretch them "dynamically". What I mean by dynamically is that you don't just put a mask on by linearly re-sizing it on 1 or 2 axises. You can select a point of the mask and change that, I wanna do that but my mask is my cut chunk and the point is a section of that chunk that needs to change for the chunk to comply with the position of the generated landmarks. I know this is a very hard topic to think about and if you guys need any clarification just ask. My code:
import cv2
import numpy as np
import dlib
cap = cv2.VideoCapture(0)
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")
while True:
_, frame = cap.read()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = detector(gray)
for face in faces:
x1 = face.left()
y1 = face.top()
x2 = face.right()
y2 = face.bottom()
cv2.rectangle(frame, (x1, y1), (x2, y2), (0, 255, 0), 3)
landmarks = predictor(gray, face)
for n in range(0, 68):
x = landmarks.part(n).x
y = landmarks.part(n).y
cv2.circle(frame, (x, y), 4, (255, 0, 0), -1)
cv2.imshow("Frame", frame)
key = cv2.waitKey(1)
if key == 27:
break
EDIT: No i'm not a serial killer
If you need to deform source image like a rubber sheet using 2 sets of keypoints, you need to use thin plate spline (TPS), or, better, piecewice affine transformation like here. The last one is more similar to texture rasterization methods (triangle to triangle texture transform).
I am trying to detect Elon Musks facial features. Right now it is sort of able to but I am detecting too many eye, nose, and mouth features. I'm not sure how I can fix this to only find one set of each.
I am using the haar files provided by opencv's github and for nose and mouth I found some haar files somewhere online.
haarcascade_frontalface_default.xml
haarcascade_eye.xml
haar files
class Filterize(object):
def __init__(self, filterpath):
self.filterpath = filterpath
self.haarpath = os.path.join(os.getcwd(), 'haar')
self.face_cascade = cv2.CascadeClassifier(os.path.join(self.haarpath, 'face.xml'))
self.eye_cascade = cv2.CascadeClassifier(os.path.join(self.haarpath, 'eye.xml'))
self.nose_cascade = cv2.CascadeClassifier(os.path.join(self.haarpath, 'nose.xml'))
self.mouth_cascade = cv2.CascadeClassifier(os.path.join(self.haarpath, 'mouth.xml'))
def get_filter_facial_features(self):
filter = cv2.imread(self.filterpath)
grayscale_filter = cv2.cvtColor(filter, cv2.COLOR_BGR2GRAY)
face = self.face_cascade.detectMultiScale(grayscale_filter, 1.3, 5)
for x, y, w, h in face:
cv2.rectangle(filter, (x, y), (x + w, y + h), (255, 0, 0), 2)
roi_gray = grayscale_filter[y:y + h, x:x + w]
roi_color = filter[y:y + h, x:x + w]
eyes = self.eye_cascade.detectMultiScale(roi_gray, 1.3, 5)
nose = self.nose_cascade.detectMultiScale(roi_gray, 1.3, 5)
mouth = self.mouth_cascade.detectMultiScale(roi_gray, 1.3, 5)
for eye_x, eye_y, eye_w, eye_h in eyes:
cv2.rectangle(roi_color, (eye_x, eye_y), (eye_x + eye_w, eye_y + eye_h), (0, 255, 0), 2)
for nose_x, nose_y, nose_w, nose_h in nose:
cv2.rectangle(roi_color, (nose_x, nose_y), (nose_x + nose_w, nose_y + nose_h), (0, 255, 0), 2)
for mouth_x, mouth_y, mouth_w, mouth_h in mouth:
cv2.rectangle(roi_color, (mouth_x, mouth_y), (mouth_x + mouth_w, mouth_y + mouth_h), (0, 255, 0), 2)
cv2.imwrite('face.png', filter)
To create the photo:
a = Filterize(filterpath)
a.get_filter_facial_features()
In this line here:
face = self.face_cascade.detectMultiScale(grayscale_filter, 1.3, 5)
You pass in the following parameters available (taken from the docs):
Parameters:
cascade – Haar classifier cascade (OpenCV 1.x API only). It can be loaded from XML or YAML file using Load(). When the cascade is not
needed anymore, release it using
cvReleaseHaarClassifierCascade(&cascade).
image – Matrix of the type CV_8U containing an image where objects are detected. objects – Vector of rectangles where each
rectangle contains the detected object.
scaleFactor – Parameter specifying how much the image size is reduced at each image scale.
minNeighbors – Parameter specifying how many neighbors each candidate rectangle should have to retain it.
flags – Parameter with the same meaning for an old cascade as in the function cvHaarDetectObjects. It is not used for a new cascade.
minSize – Minimum possible object size. Objects smaller than that are ignored.
maxSize – Maximum possible object size. Objects larger than that are ignored.
What this function does is detect all the features it can with defined boundaries. I would suggest you need to play around with these values until the amount of rectangles comes to an acceptable amount.
In fact maxSize looks like a good start, as you have smaller rectangles for each detection