With the below sample code, i am using basic dlib face detection.
I was initially drawing a bounding box around the detected face but I now wanted to display only within what is detected(AKA the face): img[top:bottom,left:right,:]
import sys
import dlib
import cv2
detector = dlib.get_frontal_face_detector()
cam = cv2.VideoCapture(1)
color_green = (0,255,0)
line_width = 3
while True:
ret_val, img = cam.read()
rgb_image = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
dets = detector(rgb_image)
#for det in dets:
#cv2.rectangle(img,(det.left(), det.top()), (det.right(), det.bottom()), color_green, line_width)
new_img = img[top:bottom,left:right,:]
cv2.imshow('my webcam', new_img)
if cv2.waitKey(1) == 27:
break # esc to quit
cv2.destroyAllWindows()
The issue that I am facing, is that it is successfully showing me what is within the x,y,w,h but the image kept resizing depending on how close I am to the camera.
What I did is the following steps:
I got the coordinates of the detection: img[top:bottom,left:right,:]
I then resized the image to a 480 to 480 size focus_face = cv2.resize(img, (480, 480))
And then passed the image to show.
So the issue Im having is if I resize the array(img) it does not seem to be following the detected face but focusing at the centre of the screen, especially the more I move back.. So if im at the centre of the screen then it shows my whole face, if im at the sides, it will only show a part of my face.
I did my best to explain this, but If you have any questions please let me know.
Best.
Related
I am trying to build a GUI using pygame but when I try to stream a camera using opencv
and show it on the screen, pygame rotates the picture 90 degrees to the right.
How do I prevent pygame from rotating the picture?
image = self.cam.read() # getting the picture
if image is not None:
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) # transforming the picture from BGR to RGB
image = pygame.surfarray.make_surface(image) # making the picture a pygame surface
self.snapshot = image
# blit it to the display surface. simple!
self.display.blit(self.snapshot, (0, 0)) # displaying the picture
pygame.display.update() # updating the screen
Use numpy.transpose to permute the x and y axis of the image. This causes a rotation by 90°:
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
image = image.transpose(1, 0, 2)
so I'm making a bot that detects icons on the screen and moves the mouse to the detected icon the user chose. this is what the code looks like:
import numpy as np
import pyautogui
import cv2
from PIL import ImageGrab
fourcc = cv2.VideoWriter_fourcc(*'XVID')
face_csc = cv2.CascadeClassifier('improved_cascade.xml')
out = cv2.VideoWriter("output.avi", fourcc, 5.0, (1366, 768))
while True:
img = ImageGrab.grab(bbox=None)
# convert image to numpy array
img_np = np.array(img)
# convert color space from BGR to RGB
frame = cv2.cvtColor(img_np, cv2.COLOR_BGR2RGB)
# show image on OpenCV frame
faces = face_csc.detectMultiScale(frame, 1.1 , 15)
for (x,y,w,h) in faces:
detected_icon = cv2.rectangle(frame,(x,y),(x+w,y+h), (255,0,0), 2)
roi_gray = frame[y:y+h, x:x+w]
roi_color = img_np[y:y+h,x:x+w]
cv2.putText(frame,'icon',(x,y),cv2.FONT_HERSHEY_TRIPLEX,0.8,(0,0,255),1)
cv2.imshow("stream", frame)
# write frame to video writer
out.write(frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
out.release()
cv2.destroyAllWindows()
but I'm having trouble making my mouse click on an icon opencv detected. for example: lets say that I set my program so that when it detects chrome on the screen, it hovers the mouse automatically to the icon and click on it. how would I be able to do that? thanks
I don't have a handy Windows box to run ImageGrab on, but assuming it produces a screenshot with the same width and height (in pixels) as the actual screen, given that both Pyautogui and OpenCV put the origin in the top left, the translation to Pyautogui should be straightforward:
for (x,y,w,h) in faces:
center_x = x + 0.5 * w
center_y = y + 0.5 * h
pyautogui.click(center_x, center_y)
That teleports the mouse pointer to the center of the object rectangle and clicks it. In case you want to simulate more human-like mouse movement and clicking, Pyautogui has a number of tools for that purpose.
I have been searching through OpenCV Docs, but I was unable to find the answer to my problem
I want to remove all OpenCv prebuilt drawings from the image
Here is an example:
with the following generic code:
import numpy as np
import cv2
# Create a black image
img = np.zeros((512,512,3), np.uint8)
# Draw a diagonal blue line with thickness of 5 px
img = cv2.line(img,(0,0),(511,511),(255,0,0),5)
while True:
cv2.imshow('generic frame', img)
if cv2.waitKey(1) & 0xFF is ord('q'):
break
I want the output to be like this following image
The background needs to stay the same, without the OpenCv drawings
this is the result that I have from my code :
enter image description here
I've made this mask on my face using contour , as I show bellow in the code .
the final result of the project is to delete the face and show the background that I am not defined it yet .
my question is : is there any way to make a mask with this counter so i could use something like this cv2.imshow('My Image',cmb(foreground,background,mask)) to just show the foreground under the mask on the background ? ( the problem here is that i must have the mask as a video in this form but i want it to be real time)
or perhaps the other way , could i somehow delete the pixels of frame in (or under) my counter ?
this is my code :
from imutils.video import VideoStream
from imutils import face_utils
import datetime
import argparse
import imutils
import time
import dlib
import cv2
import numpy as np
# path to facial landmark predictor
ap = argparse.ArgumentParser()
ap.add_argument("-p", "--shape-predictor", required=True)
print("[INFO] loading facial landmark predictor...")
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(args["shape_predictor"])
# grab the indexes of the facial landmarks
(lebStart, lebEnd) = face_utils.FACIAL_LANDMARKS_IDXS["left_eyebrow"]
(rebStart, rebEnd) = face_utils.FACIAL_LANDMARKS_IDXS["right_eyebrow"]
(jawStart, jawEnd) = face_utils.FACIAL_LANDMARKS_IDXS["jaw"]
# initialize the video stream and allow the cammera sensor to warmup
print("[INFO] camera sensor warming up...")
vs = VideoStream(usePiCamera=args["picamera"] > 0).start()
time.sleep(2.0)
# loop over the frames from the video stream
while True:
# grab the frame from the threaded video stream, resize it to
# have a maximum width of 400 pixels, and convert it to
# grayscale
frame = vs.read()
frame = imutils.resize(frame, width=400)
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
# detect faces in the grayscale frame
rects = detector(gray, 0)
# loop over the face detections
for rect in rects:
shape = predictor(gray, rect)
shape = face_utils.shape_to_np(shape)
# extract the face coordinates, then use the
faceline = shape[jawStart:lebEnd]
# compute the convex hull for face, then
# visualize each of the face
facelineHull = cv2.convexHull(faceline)
mask = np.zeros(frame.shape,dtype='uint8')
cv2.drawContours(frame, [facelineHull], -1, (0, 0, 0),thickness=cv2.FILLED)
cv2.drawContours(frame, [facelineHull], -1, (0, 255, 0))
# show the frame
cv2.imshow("Frame", frame)
# cv2.imshow("Frame", mask)
key = cv2.waitKey(1) & 0xFF
# if the `q` key was pressed, break from the loop
if key == ord("q"):
break
# do a bit of cleanup
cv2.destroyAllWindows()
vs.stop()
here is my solution for deleting face from frame( it is faster but again thanks to #meetaig for help)
mask = np.zeros(frame.shape,dtype='uint8')
mask = cv2.drawContours(mask, [facelineHull], -1, (255 , 255 , 255),thickness=cv2.FILLED)
mask = cv2.bitwise_not(mask)
img2gray = cv2.cvtColor(mask,cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 10, 255, cv2.THRESH_BINARY)
result= cv2.bitwise_and(frame,frame,mask=mask)
and if i show result now it will work.
cv2.imshow("Frame", result)
Assuming your mask is a binary mask you could do the following:
def cmb(foreground,background,mask):
result = background.copy()
result[mask] = foreground[mask]
return result
I did not test this code, but I hope the idea comes across. You invert the mask for the background and leave the mask alone for the foreground. You apply this to every frame and voilà, you have your masked images.
edit:
adjusted code according to comment. Of course that solution is much clearer than what I originally wrote. The functionality stays the same, though.
I want to detect only the left eye in a video streaming, but I can't. When I run this code, it detects two eyes (right and left) and sometimes it also detects the mouth or the nose like it is an eye.
Follow the code below:
import cv2
import numpy as np
faceCascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
#eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')
eye_cascade = cv2.CascadeClassifier('haarcascade_lefteye_2splits.xml')
video_capture = cv2.VideoCapture(0)
while True:
ret, img = video_capture.read()
gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = faceCascade.detectMultiScale(
gray_img,
scaleFactor=1.98,
minNeighbors=8,
minSize=(80, 80),
flags=cv2.cv.CV_HAAR_SCALE_IMAGE
)
#print 'faces: ', faces
for (p,q,r,s) in faces:
#cv2.rectangle(img,(p,q),(p+r,q+s),(255,0,0),3)
face_gray = gray_img[q:q+s, p:p+r]
face_color = img[q:q+s, p:p+r]
eyes = eye_cascade.detectMultiScale(face_gray)
for (ep,eq,er,es) in eyes:
cv2.rectangle(face_color,(ep,eq),(ep+er,eq+es), (0,255,0),3)
rimg = cv2.flip(img, 1) # invert the object img
cv2.imshow("Video", rimg)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
video_capture.release()
cv2.destroyAllWindows()
So how should I do to detect only the left eye?
PS: I'm using Python 2.7.13, NumPy 1.10.0 and OpenCV 2.4.13.3 in this program.
You can use a simple for loop to iterate through the eyes returned by eye-cascade and find the index of the eye with minimum x-coordinate. That will give you the index of the left-most eye.
def getleftmosteye(eyes):
leftmost=9999999
leftmostindex=-1
for i in range(0,2):
if eyes[i][0]<leftmost:
leftmost=eyes[i][0]
leftmostindex=i
return eyes[leftmostindex]
Now you can get coordinates of the left eye once get its index.