We Keep Coding

Mtcnn face-extractor for head extraction

I'm using Mtcnn network (https://towardsdatascience.com/face-detection-using-mtcnn-a-guide-for-face-extraction-with-a-focus-on-speed-c6d59f82d49) to detect faces and heads. For this I'm using the classical lines code for face detection :I get the coordinate of the top-left corner of the bouding-box of the face (x,y) + the height and width of the box (h,w), then I expand the box to get the head in my crop :
import mtcnn
img = cv2.imread('images/'+path_res)
faces = detector.detect_faces(img)# result
for result in faces:
x, y, w, h = result['box']
x1, y1 = x + w, y + h
x, y, w, h = result['box']
x1, y1 = x + w, y + h
if x-100>=0:
a=x-100
else:
a=0
if y-150 >=0:
b=y-150
else:
b=0
if x1+100 >= w:
c=x1+100
else:
c=w
if y1+60 >= h:
d=y1+60
else:
d=h
crop=img[b:d,a:c] #<--- final crop of the head
the problem is this solution works for some images, but for many anothers, in my crop, I get the shoulders and the neck of the target person. I think, it's because, the pixels/inch in each image (i.e. +150pixels in one image isn't the same in another image). Hence, what can I do to extract the head properly ?
Many thanks

You can use relative instead of absolute sizes for the margins around the detected faces. For example, 50% on top, bottom, left and right:
import mtcnn
img = cv2.imread('images/'+path_res)
faces = []
for result in detector.detect_faces(img):
x, y, w, h = result['box']
b = max(0, y - (h//2))
d = min(img.shape[0], (y+h) + (h//2))
a = max(0, x - (w//2):(x+w))
c = min(img.shape[1], (x+w) + (w//2))
face = img[b:d, a:c, :]
faces.append(face)

How to compare 2 list for object detection

I have made a function that detects a object on screen using opencv matchTemplate and returns its center locations as (x, y).
I want to compare the results of running the same function on 2 different objects to detect the location of one object in reference of the other. In my case the two objects are a player avatar and some bushes I want to know is the player currently standing near a bush or not.There is only one user avatar on screen therefore it only returns single (x, y) value for it but there are multiple bushes therefore multiple (x, y) values. I want a way to compare those two matrices. Sorry about the code format.
def center(base_img, needle_img, th, color):
result = cv.matchTemplate(BASE_IMG, needle_img, cv.TM_CCOEFF_NORMED)
threshold = th
yloc, xloc = np.where(result >= threshold)
w = needle_img.shape[1]
h = needle_img.shape[0]
rectangles = []
for (x, y) in zip(xloc, yloc):
rectangles.append([int(x), int(y), int(w), int(h)])
rectangles.append([int(x), int(y), int(w), int(h)])
rectangles, weights = cv.groupRectangles(rectangles, 1, 0.4)
points = []
for (x, y, w, h) in rectangles:
certre_x = x + int(w / 2)
certre_y = y + int(h / 2)
cv.drawMarker(base_img, (certre_x, certre_y), color, cv.MARKER_DIAMOND)
points.append((certre_x, certre_y))
# cv.imshow("result", base_img)
# print(points)
# return points

You can loop through the center of the bushes and get the distance to each bush sqrt(( x2-x1)**2 + ( y2-y1)**2) or you could use nearest neighbor and numpy.
Post a code snippet if it worked.

get_boudingbox not defined error (opencv2,python)

I am trying to extract faces from images by creating bounding boxes, but I am getting an error shown below.
First code block:
def get_predicition(image):
"""Expects the image input, this image further cropped to face
and the cropped face image will be sent for evalution funtion
finally
returns the annotated reusult with bounding box around the face.
"""
height, width = image.shape[:2]
try: # If in case face is not detected at any frame
face = face_detector(image, 1)[0] # Face detection
x, y, size = get_boundingbox(face=face, width=width, height=height) # Calling to get bound box around the face
except IndexError:
pass
cropped_face = image[y:y+size, x:x+size] # cropping the face
output,label = evaluate(cropped_face) # Sending the cropped face to get classifier result
font_face = cv2.FONT_HERSHEY_SIMPLEX # font settings
thickness = 2
font_scale = 1
if label=='Real':
color = (0,255, 0)
else:
color = (0, 0, 255)
x = face.left() # Setting the bounding box on uncropped image
y = face.top()
w = face.right() - x
h = face.bottom() - y
cv2.putText(image, label+'_'+str('%.2f'%output)+'%', (x, y+h+30),
font_face, font_scale,
color, thickness, 2) # Putting the label and confidence values
return cv2.rectangle(image, (x, y), (x + w, y + h), color, 2)# draw box over face
Second code block:
cropped_face =[]
for image in images:
faces = face_detector(image[0], 1)
height, width = image[0].shape[:2]
try: # If in case face is not detected at any frame
x, y, size = get_boundingbox(face=faces[0], width=width, height=height)
except IndexError:
continue
cropped_face.append([image[0][y:y+size, x:x+size],image[1]])
The first code block runs without any errors but when I try to run the second code block. I am getting the error shown below:
NameError Traceback (most recent call last)
<ipython-input-48-52e274eb31f6> in <module>
5 try: # If in case face is not detected at any frame
6
----> 7 x, y, size = get_boundingbox(face=faces[0], width=width, height=height)
8 except IndexError:
9 continue
NameError: name 'get_boundingbox' is not defined

image aling with cv2 instead of HOG

Hi I am working on facial recognition.
To increase performance I want to use facial alignment.
When I use the HOG face identifier, described e.g., by Adrian I get an aligned image out.
from imutils.face_utils import rect_to_bb
from dlib import get_frontal_face_detector
detector = dlib.get_frontal_face_detector()
shape_predictor = dlib.shape_predictor('/home/base/Documents/facial_landmarks/shape_predictor_5_face_landmarks.dat')
fa = face_utils.facealigner.FaceAligner(shape_predictor, desiredFaceWidth=112, desiredLeftEye=(0.3, 0.3))
img=cv2.imread(pathtoimage)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
rects = detector(gray, 2)
for rect in rects:
(x, y, w, h) = rect_to_bb(rect)
faceAligned = fa.align(img, gray, rect)
However, I have to work on an embedded hardware and the HOG facial recognition is not fast enough. The best working is the cv2 lbpcascader.
With cv2 I also get the box of the found face, but using that works not.
faces_detected = face_cascade.detectMultiScale(img, scaleFactor=1.1, minNeighbors=4)
In other examples using the HOG, the coordinates are extracted from the HOG-rect with:
(x, y, w, h) = rect_to_bb(rect)
and then used with
aligned_face = fa.align(img, gray, dlib.rectangle(left = x, top=y, right=w, bottom=h))
The idea would be to exchange the x,y,w,h with the cv2 values. Unfortunately, that does not work as the two lines above result in a complete false alignment. In the first code example, the rect_to_bb function is included but not used.
I checked the values and they are somehow off:
224x224 the image
156 70 219 219 the cv2 values (slightly different of course)
165 101 193 193 the rect values with rect_to_bb
[(165, 101) (358, 294)] the rect values
I checked the rect_to_bb function, but this seems straight forward:
def rect_to_bb(rect):
# take a bounding predicted by dlib and convert it
# to the format (x, y, w, h) as we would normally do
# with OpenCV
x = rect.left()
y = rect.top()
w = rect.right() - x
h = rect.bottom() - y
# return a tuple of (x, y, w, h)
return (x, y, w, h)

While typing I got the answer... classic
the alignment function needs the bounding box marks slightly different.
It can be seen in the rect_to_bb() function.
def rect_to_bb(rect):
# take a bounding predicted by dlib and convert it
# to the format (x, y, w, h) as we would normally do
# with OpenCV
x = rect.left()
y = rect.top()
w = rect.right() - x
h = rect.bottom() - y
# return a tuple of (x, y, w, h)
return (x, y, w, h)
There the rect.right (w in cv2) and the rect.bottom (h in cv2) are subtracted with x and y. So in the alignment function you have to add the values, otherwise the image fed to the alignment function is much to small and out of shape. And this can also be the values from the cv2 detection.
aligned_face = fa.align(img, gray, dlib.rectangle(left = x, top=y, right=w+x, bottom=h+y))
Keep healthy

variable increment in python beginner error

I am trying to increment the counter each time it detects an object.
count=0
def detect_and_draw(img, cascade):
# allocate temporary images
gray = cv.CreateImage((img.width,img.height), 8, 1)
small_img = cv.CreateImage((cv.Round(img.width / image_scale),
cv.Round (img.height / image_scale)), 8, 1)
# convert color input image to grayscale
cv.CvtColor(img, gray, cv.CV_BGR2GRAY)
# scale input image for faster processing
cv.Resize(gray, small_img, cv.CV_INTER_LINEAR)
cv.EqualizeHist(small_img, small_img)
if(cascade):
t = cv.GetTickCount()
faces = cv.HaarDetectObjects(small_img, cascade, cv.CreateMemStorage(0),
haar_scale, min_neighbors, haar_flags, min_size)
t = cv.GetTickCount() - t
print "time taken for detection = %gms" % (t/(cv.GetTickFrequency()*1000.))
if faces:
for ((x, y, w, h), n) in faces:
# the input to cv.HaarDetectObjects was resized, so scale the
# bounding box of each face and convert it to two CvPoints
pt1 = (int(x * image_scale), int(y * image_scale))
pt2 = (int((x + w) * image_scale), int((y + h) * image_scale))
cv.Rectangle(img, pt1, pt2, cv.RGB(255, 0, 0), 3, 8, 0)
count += 1
print count
But I can't get why it has this error .I tried to do the increment just after if but still same error. It's the first time to use python so I can't get the scoping rules. what should I modify?
File "detect.py", line 42, in detect_and_draw
count +=1
UnboundLocalError: local variable 'count' referenced before assignment

You have a global count. The function doesn't know it.
def detect_and_draw(img, cascade):
global count