I am trying to use mediapipe to generate facial landmarks using this code
faceModule = mediapipe.solutions.face_mesh
with faceModule.FaceMesh(static_image_mode=True) as faces:
# loading the image
image = cv2.imread(file_name)
# processing the face to extract the landmark points (468 point) for each x,y,z
results = faces.process(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
if results.multi_face_landmarks != None:
print("there is a face")
else:
frame = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
plt.imshow(frame)
plt.show()
The vast majority of the images that result in results.multi_face_landmarks = None are faces from the side.
Is there a way to fix that?
I tried to search to find if anyone faced that issue before and i didn't find any.
Are facial landmarks not for faces from the side?
Related
With the below sample code, i am using basic dlib face detection.
I was initially drawing a bounding box around the detected face but I now wanted to display only within what is detected(AKA the face): img[top:bottom,left:right,:]
import sys
import dlib
import cv2
detector = dlib.get_frontal_face_detector()
cam = cv2.VideoCapture(1)
color_green = (0,255,0)
line_width = 3
while True:
ret_val, img = cam.read()
rgb_image = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
dets = detector(rgb_image)
#for det in dets:
#cv2.rectangle(img,(det.left(), det.top()), (det.right(), det.bottom()), color_green, line_width)
new_img = img[top:bottom,left:right,:]
cv2.imshow('my webcam', new_img)
if cv2.waitKey(1) == 27:
break # esc to quit
cv2.destroyAllWindows()
The issue that I am facing, is that it is successfully showing me what is within the x,y,w,h but the image kept resizing depending on how close I am to the camera.
What I did is the following steps:
I got the coordinates of the detection: img[top:bottom,left:right,:]
I then resized the image to a 480 to 480 size focus_face = cv2.resize(img, (480, 480))
And then passed the image to show.
So the issue Im having is if I resize the array(img) it does not seem to be following the detected face but focusing at the centre of the screen, especially the more I move back.. So if im at the centre of the screen then it shows my whole face, if im at the sides, it will only show a part of my face.
I did my best to explain this, but If you have any questions please let me know.
Best.
I am attempting to create a model that can recognize green circles or green rectangular arrows as shown in the "image".
I figured I would need to make my own classifier and I am using Cascade Trainer to accomplish this. (See repo for the classification images and .xml file.)
Complete code and supporting files here:
https://github.com/ThePieMonster/Computer-Vision
# Analyze Screenshots
print("Analyze Screenshots")
img = cv2.imread('Data/image.png')
classifier_path = 'Data/train/classifier/cascade.xml'
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
print("- load classifier")
cascade = cv2.CascadeClassifier(classifier_path)
print("- detect objects")
objects = cascade.detectMultiScale(image=img_rgb, scaleFactor=1.10, minNeighbors=3)
print("- draw rectangle around objects")
for(x,y,w,h) in objects:
# cv2.rectangle(<image>, (x,y), (x+w,y+h), <rectangle rgb color>, <rectangle thickness>)
img = cv2.rectangle(img, (x,y), (x+w,y+h), (0,255,0), 2)
cv2.imshow('img', img)
cv2.waitKey(0)
print()
As you can see via the below image, I can't seem to get the classifier to recognize the green dots. There are about 41 green dots in the image and I am hoping to spot them all, currently I have a few red dots spotted and some random map squares. Any help is much appreciated!
I am trying to detect faces (specifically, faces with opened eyes) using OpenCV haar cascade classifier. However, I had a problem detecting the faces that do not have eyebrows and/or jaw, as shown in the following image. I had tried many haar cascade for face detection such as haarcascade_frontalface_default.xml, haarcascade_frontalface_alt_tree.xml, etc. But all of these did not work.
Here is my code:
import cv2
import os
import glob
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier('haarcascade_eye_tree_eyeglasses.xml')
count = 0
path = "./test/*.png"
for index, filename in enumerate(glob.glob(path)):
img = cv2.imread(filename)
basename = os.path.splitext(os.path.basename(filename))[0]
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(gray, 1.3, 5)
for (x,y,w,h) in faces:
# cv2.rectangle(img,(x,y),(x+w, y+h),(255,0,0), 2)
roi_gray = gray[y:y+h, x:x+w]
roi_color = img[y:y+h, x:x+w]
eyes = eye_cascade.detectMultiScale(roi_gray)
if len(eyes) >= 2:
count = count + 1
output_dir = './test/output'
cv2.imwrite(f'{output_dir}/{basename}.png', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Thank you in advance!
Use facial landmarks with dlib, this method may work for you, see these two links:
facial-landmarks-dlib-opencv-python
opencv-face-recognition
Also, see this link:
dlib-and-deep-learning
If you have tensorflow installed you can use a neural net to detect faces which give much better accuracy than the simple haar classifier.
Here's an example using the MTCNN detector which uses tensorflow as the backend.
from mtcnn.mtcnn import MTCNN
from PIL import Image
import numpy as np
img = Image.open('6qNFu.png') # load the image
img = np.asarray(img, dtype='uint8') # convert to numpy array
img = img[:,:,0:3] # drop the alpha channel
detector = MTCNN() # initialize MTCNN detector
print(detector.detect_faces(img)) # use MTCNN detector to return bounding box and face metrics
Using the bounding box you can extract the face from the image. Note: if the face is truncated like in the instances above, it might return a negative coordinate which is an extrapolation of where it thinks the face might be.
Here is a the documentation on MTCNN library: https://pypi.org/project/mtcnn/
It also tells you how to install it.
I have build my custom object detection model using SSD-Mobilenet and Tensorflow. Now, I have to crop license plate number from the video file and display it along with the model name on top of the boundary box.
i5 processor
NVIDIA GeForce MX150 with 2GB VRAM
import numpy as np
import cv2
import pytesseract
import matplotlib.pyplot as plt
img = cv2.imread('/home/dora/Desktop/image1.jpg')
#convert my image to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
#perform adaptive threshold so that I can extract proper contours from the image
#need this to extract the name plate from the image.
thresh = cv2.adaptiveThreshold(gray,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY,11,2)
contours,h = cv2.findContours(thresh,1,2)
#once I have the contours list, i need to find the contours which form rectangles.
#the contours can be approximated to minimum polygons, polygons of size 4 are probably rectangles
largest_rectangle = [0,0]
for cnt in contours:
approx = cv2.approxPolyDP(cnt,0.01*cv2.arcLength(cnt,True),True)
if len(approx)==4: #polygons with 4 points is what I need.
area = cv2.contourArea(cnt)
if area > largest_rectangle[0]:
#find the polygon which has the largest size.
largest_rectangle = [cv2.contourArea(cnt), cnt, approx]
x,y,w,h = cv2.boundingRect(largest_rectangle[1])
#crop the rectangle to get the number plate.
roi=img[y:y+h,x:x+w]
#cv2.drawContours(img,[largest_rectangle[1]],0,(0,0,255),-1)
plt.imshow(roi, cmap = 'gray')
plt.show()
The above code crops license plate from images.
I want to crop from a video file.
Looks like you want to read a video. In this case I would use cv2's VideoCapture class.
If you put all your processing in a method, you can just call it on every frame.
.
.
.
video = cv2.VideoCapture("<filename>")
ret = True
while ret: # run while there are more frames
frame, ret = video.read() # get next frame
process(frame) # do your processing
video.release()
I'm using a guide provided online with opencv2.4 that shows you how to detect faces with opencv2 and python. I followed the guide and understand what it says. However I can't seem to find the issue with my program because the video shows but now face is detected and the video is very clear. There are no errors. I ran in debug mode and the value faces remains a blank tuple so I'm assuming that means its not finding the face. What I don't understand is why and I think it has something to do with the hash table.
By hash table I mean the cascade xml file. I understand cascades are basically the guidelines for detecting the facial artifacts correct?
Links to the guides. The hash table i.e the xml file is on the github linked.
https://github.com/shantnu/FaceDetect/blob/master/haarcascade_frontalface_default.xml
https://realpython.com/blog/python/face-detection-in-python-using-a-webcam/
import cv2
import sys
import os
#cascPath = sys.argv[1]
cascPath = os.getcwd()+'facehash.xml'
faceCascade = cv2.CascadeClassifier(cascPath)
print faceCascade
video_capture = cv2.VideoCapture(0)
while True:
# Capture frame-by-frame
ret, frame = video_capture.read()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = faceCascade.detectMultiScale(
gray,
scaleFactor=1.1,
minNeighbors=5,
minSize=(30, 30),
flags=cv2.cv.CV_HAAR_SCALE_IMAGE
)
cv2.cv
# Draw a rectangle around the faces
for (x, y, w, h) in faces:
cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
# Display the resulting frame
cv2.imshow('Video', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# When everything is done, release the capture
video_capture.release()
cv2.destroyAllWindows()
You have a wrong path to your xml classifier. (I guess you've changed the name to get a shorter form).
Instead of your cascPath:
cascPath = os.getcwd()+'facehash.xml'
Try this:
cascPath = "{base_path}/folder_with_your_xml/haarcascade_frontalface_default.xml".format(
base_path=os.path.abspath(os.path.dirname(__file__)))
And now it should work as well.