python, opencv: duration of time an object is detected on webcam - python

i am able to track an object in each frame returned by my webcam. i want to note when the object was detected for the first time and the duration till it was continuously detected thereafter. the webcam is on for an indefinite period i.e. till it is closed by user input.
since the set of code for detecting the object is within a while loop that is needed to read the next frame from cv2.VideoCapture() i am not able to come up with an efficient, pythonic way for doing what i want.
right now i am appending a list with the tuple (timestamp,flag) for each frame. timestamp is the value from python's time.time() and flag is a boolean to indicate if the object is detected. i then sum up all values of timestamp where flag is 'Yes'. but this doesn't quite give me what i want. can you suggest a more appropriate way?
*i wish there was a generic function in opencv like cv2.detectionDuration() :P
--EDIT--
Here is a code for tracking frontal face:
import cv2
import time
faceCascade = cv2.CascadeClassifier('haarcascade_frontalface_alt.xml')
capture = cv2.VideoCapture(0)
keyPressed = -1
faceFound = []
print 'press esc to quit'
while(keyPressed != 27):
ret, camImage = capture.read()
cv2.imshow('camImage', camImage)
try:
faceRegion = faceCascade.detectMultiScale(camImage)
timestamp = time.time()
flag = 1
faceFound.append((timestamp,flag))
except TypeError:
timestamp = time.time()
flag = 0
faceFound.append((timestamp,flag))
print 'check if front face is visible to camera'
pass
keyPressed = cv2.waitKey(1)
cv2.destroyAllWindows()
timeDelta = 0
for tup in faceFound:
if tup[1] == 1:
timeDelta += tup[0]
print timeDelta
also, could you help me obtain a better format for timeDelta so that it can be displayed as day:hour:min:sec:microsec. is there a better alternative to time.time() for my current requirements?

[time_first, time_last, got_first]
if got_first is false and face is detected, you assign time_first to now() and got_first to true.
when the face is not detected, you assign time_last to now() and got_first to false.
import cv2
import time
faceCascade = cv2.CascadeClassifier('haarcascade_frontalface_alt.xml')
capture = cv2.VideoCapture(0)
keyPressed = -1
faceFound = []
ts = [0,0,False]
print 'press esc to quit'
while(keyPressed != 27):
ret, camImage = capture.read()
cv2.imshow('camImage', camImage)
try:
faceRegion = faceCascade.detectMultiScale(camImage)
if ts[2] == False:
ts[0] = time.time()
ts[2] = True
except TypeError:
if ts[2] == True:
ts[1] = time.time()
ts[2] = False
faceFound.append([ts[0], ts[1]])
print 'check if front face is visible to camera'
pass
keyPressed = cv2.waitKey(1)
cv2.destroyAllWindows()
for list in faceFound:
print list[1] - list[0]
although I think there's a problem with your code, cause no face gets detected.
you can print faceRegion and see that, it's an empty tuple.

What you are trying to do is known as dwell time in which we calculate the time of the detected object inside the frame. In order to achieve this you need have some kind of tracking inside you inferencing. Basically the tracker will assign an object id to your detected object and then that object id remains same till the object is detected. Based on that object id you can start a timer and keep on counting it until the object is detected. In opencv you can go with CentroidTracking algorithm
Have a look at this video, this might give you basic idea: https://www.youtube.com/watch?v=JbNeFMKXybw&list=PLWw98q-Xe7iH8UHARl8RGk8MRj1raY4Eh&index=7

Related

I am trying to build Motion Detector using openCV and python but display window is not responding when I terminate program

Here is the code for the same, have a look at it. In this, below code I am creating a Motion Detector and with this I will be recording the timings of when the various objects appeared and disappeared for which I am using dataframe.
The issue with this is that the program executes but when the output is displayed in the form of a Window, it freezes when I try to terminate.
import cv2, pandas
from datetime import datetime
first_frame = None
status_list = [None,None]
times = []
df = pandas.DataFrame(columns=["Start", "End"]) #Dataframe to store the time values during which object detection and movement appears.
video = cv2.VideoCapture(0)#VideoCapture object is used to record video using web cam
while True:
#check is a bool data type, returns true if VideoCapture object is read
check,frame = video.read()
status = 0
gray = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY) # For converting the frame color to gray scale
gray = cv2.GaussianBlur(gray,(21,21),0) # For converting the gray scale frame to GaussianBlur
if first_frame is None:
first_frame = gray # used to store the first image/frame of the video
continue
delta_frame = cv2.absdiff(first_frame,gray)#calculates the difference between first and other frames
thresh_delta = cv2.threshold(delta_frame,30,255,cv2.THRESH_BINARY)[1]
thresh_delta = cv2.dilate(thresh_delta,None,iterations=0) #Provides threshold value, so if the difference is <30 it will turn to black otherwise if >30 pixels will turn to white
(_,cnts,_) = cv2.findContours(thresh_delta.copy(),cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE) #Define the contour area,i.e. adding borders
#Removing noises and shadows, any part which is greater than 1000 pixels will be converted to white
for contour in cnts:
if cv2.contourArea(contour) < 1000:
continue
status = 1 #change in status when the object is detected
#Creating a rectangular box around the object in frame
(x,y,w,h) = cv2.boundingRect(contour)
cv2.rectangle(frame,(x,y),(x+w,y+h),(0,255,0),3)
#list of status for every frame
status_list.append(status)
status_list=status_list[-2:]
#Record datetime in a list when change occurs
if status_list[-1]==1 and status_list[-2]==0:
times.append(datetime.now())
if status_list[-1]==0 and status_list[-2]==1:
times.append(datetime.now())
cv2.imshow('frame',frame)
#cv2.imshow('Capturing',gray)
#cv2.imshow('delta',delta_frame)
#cv2.imshow('thresh',thresh_delta)
key = cv2.waitKey(1) #changing the frame after 1 millisecond
#Used for terminating the loop once 'q' is pressed
if key == ord('q'):
break
print(status_list)
print(times)
for i in range(0,len(times),2):
df = df.append({"Start":times[i],"End":times[i+1]},ignore_index=True)
df.to_csv("Times.csv")
video.release()
cv2.destroyAllWindows #will be closing all the windows
Try this:
if cv2.waitKey(1) & 0xFF == ord('q'):
break
For a brief explanation about what "& 0xFF" is, see: What's 0xFF for in cv2.waitKey(1)?
It is basically a bit mask that takes the portion of 'waitKey' value(32 bit) that can be compared to ASCII (8 bit).
You can try this.
k = cv2.waitKey(1) & 0xFF
if k == 27 :
break
where k can be any ASCII code on the keyboard ASCII code help
In this case 27 is 'Esc' button.
However, the problem you're having where it is freezing, have you tried continuously pressing any button besides q? I had a similar problem when I was testing out OpenCV's tutorial code here
but i figured it out by playing around and then realizing i needed to hold down any button besides the 'exit' button.
I had faced the same issue bcz of this piece of code from geeksforgeeks
Please chck the last line of the code:
cv2.destroyAllWindows #will be closing all the windows
Add parenthesis, it shud b:
cv2.destroyAllWindows()
Please try this, rest looks fine
Well you can also pass a particular frame that you want to close like:
cv2.destroyAllWindows("frame")

time.sleep for only a part of the code inside infinite while loop

I'm working with a code that analyzes frames from a live stream with OpenCV and if a condition is met saves the current frame to file.
The infinite loop to analyze the video frame by frame is something like this:
while True:
ret,frame = stream.read()
if conditionisMet :
pil_image = Image.fromarray(frame)
pil_image.save("/path/to/folder/image.jpg")
cv2.imshow("LiveStream", frame)
What I want to add is that if the condition is met again too soon (20-30 sec) the image does not have to be saved and the while loop has to grab another frame and continue its work. I've tried with time.sleep(30.0) inside the if statement but it blocks the while loop waiting for the 30 sec to pass. Is there a way to use time.sleep in this case, or another method suitable for my needs?
Thanks in advance
you could do something like this:
last_grab=time.time()-30 # this to get things started
while True:
if condition and time.time()-last_grab > 30:
last_grab=time.time()
# Do things here
else:
continue
Just add a variable to keep track of your last saving time:
last_save_time = time.time()
while True:
ret,frame = stream.read()
if conditionisMet and time.time() - last_save_time() > 20:
pil_image = Image.fromarray(frame)
pil_image.save("/path/to/folder/image.jpg")
# update last save time
last_save_time = time.time()
cv2.imshow("LiveStream", frame)
Why not just capture the amount of time running then save image if greater than the given amount of time....
a = dt.now()
b = dt.now()
c = b - a
if c < x:
do something

Optimise Performance - OpenCV and Threads with Python

I am having a problem related to performance using OpenCV3.2 with Python. I have just integrated another subsystem to my main program and it slowed down a lot.
This is my initial code without integrating the new subsystem, I am using cv2.getTickCount to measure time, as suggested by OpenCv3.2 Official Website.
# Infinite loop
while True:
# getting tick count
e1 = cv2.getTickCount()
# storing frame
_, img = cap.read()
# define red colours in the screen
findRedColours(img, board)
# getting tick count after the functions
e2 = cv2.getTickCount()
# calculating time
t = (e2 - e1) / cv2.getTickFrequency()
# print time
print(t)
# check if img is none
if img is not None:
# omitted code
k = cv2.waitKey(20) & 0xFF
# start the game, hide info
if (k == ord('s') or k == ord('S')) and start is False:
# create new thread to play game
t = Thread(target=playGame)
t.start()
Basically, I am calling a function in the infinite loop to find red colours and by pressing start I create the Thread and the game starts.
This is the time needed before I press 'S' to create the Thread:
0.019336862
0.016924178
0.022487864
This is the time needed after I press 'S' to create the Thread:
0.091731532
0.125760734
0.098221829
Here everything works fine, there is a light change in the time, but nothing too important. I start to have problems when I add my new subsystem. Here the following code with the integration of the new system, it is the same of the previous one, it is just a function call that changes:
# Infinite loop
while True:
# getting tick count
e1 = cv2.getTickCount()
# storing frame
_, img = cap.read()
# extract grid
gridExtractor.extractGrid(img)
# define red colours in the screen
findRedColours(img, board)
# getting tick count after the functions
e2 = cv2.getTickCount()
# calculating time
t = (e2 - e1) / cv2.getTickFrequency()
# print time
print(t)
# check if img is none
if img is not None:
# omitted code
k = cv2.waitKey(20) & 0xFF
# start the game, hide info
if (k == ord('s') or k == ord('S')) and start is False:
# create new thread to play game
t = Thread(target=playGame)
t.start()
and this is the time before I create the Thread:
0.045629524
0.023788123
0.10517206
It is slightly higher than the one without the integration, but still ok. Here is the time after I create Thread:
1.061517957
0.568310864
0.691701059
There is an enormous difference between this one and the previous one, it reaches even a whole second. It is noticeable even from the camera output, really slow.
My questions are, am I creating my Thread in the wrong way? There is a better and more efficient way to use rather the Threads? Or is there actually a way to optimise performance in this case without having to modify these functions findRedColours(img, board), t = Thread(target=playGame), gridExtractor.extractGrid(img)?
I am new using OpenCV and Python and still having troubles around. Hope someone can address me to the right way. Thanks.
Thanks to user 'deets' who help commenting above it has been possible to optimise performance.
In this case is enough to substitute Thread with Process from multiprocessing module in Python.
from multiprocessing import Process
#omitted code
while True:
# getting tick count
e1 = cv2.getTickCount()
# storing frame
_, img = cap.read()
# extract grid - first subsystem
gridExtractor.extractGrid(img)
# define red colours in the screen - second subsystem
findRedColours(img, board)
# getting tick count after the functions
e2 = cv2.getTickCount()
# calculating time
t = (e2 - e1) / cv2.getTickFrequency()
# print time
print(t)
# check if img is none
if img is not None:
# omitted code
k = cv2.waitKey(20) & 0xFF
# start the game, hide info
if (k == ord('s') or k == ord('S')) and start is False:
# create new thread to play game
p = Process(target=playGame)
p.start()
And the relative time needed is:
0.022570883
0.11354852
0.119643379
Compared to Thread the use of Process is way more efficient in terms of performance.

Inadvertently Stacking While Loops

I've got a program right now that is meant to display an image, and allow for various functions on keypress (e.g., load a new frame, crop a region of interest, etc). It's also set up so that if too much time passes with no action, it'll automatically attempt to load a new image.
def triggeredbytimer():
global f, x1i, y1i, x2i, y2i, sid, keepgoing
keepgoing = False
print "Inactivity has lead to an automatic refresh. Loading newest image."
f = 0; t = 0
incoming = imagerequest(sid, f, t, x1i, y1i, x2i, y2i)
imagecallback(incoming)
def imagecallback(data):
print "----------------------- imagecallback"
global sid, f, x1i, y1i, x2i, y2i, img, refPt, cropping, keepgoing
keepgoing = True
########### \\// change this to change wait for autorefresh
tmr = Timer(10.0, triggeredbytimer)
tmr.start()
##################################
b = data.b; t = data.tr; sid = data.sid; error = int(data.error)
t = float(t); sid = int(sid)
expandedt = time.strftime("%m-%d-%Y %H:%M:%S", time.localtime(t))
print str(t) + " ----> " + expandedt
print "----------------------- image incoming"
timestr = time.strftime("%H:%M:%S", time.gmtime(t))
print "Loaded image from %s."%timestr
imagedata = bridge.imgmsg_to_cv2(b, "bgr8")
npimg = cv2.imencode('.jpg', imagedata)
cv2.imwrite('temp_image_np.jpg', imagedata)
img = cv2.imread('temp_image_np.jpg')
clone = img.copy()
cv2.namedWindow('image', cv2.WINDOW_NORMAL)
cv2.setMouseCallback("image", mouseclickcb)
cv2.imshow("image", img)
while keepgoing == True:
key = cv2.waitKey(0) & 0xFF
## (here there are a lot of keypress functions, for brevity I included only one)
elif key == ord("t"):
print "Looking for newest frame..."
tmr.cancel()
t = 0; f = 0
incoming = imagerequest(sid, f, t, x1i, y1i, x2i, y2i)
imagecallback(incoming)
(NOTE: The images are fetched from a ROS program via services, and that is where the "imagerequest" function comes from. All of that works fine: the problem is specifically as prescribed.)
To summarize here- since there's a bit of irrelevant clippings in this snippet - an image is loaded by imagecallback. When that function is called, a timer starts (time limit is arbitrary, set low here for testing). When the timer expires, triggeredbytime() requests a new image. It acquires the image via a ROS service, then takes the return from that and uses it as an input to imagecallback(data), thus loading the image and allowing all the keypress functions to still be valid.
So the problem is that whenever the program refreshes with a new frame, the while keepgoing == True loop does not stop, leading to stacked loops that go on and get worse with every refresh. this leads to a single keypress executing multiple times, which quickly overwhelms the program it's pulling images from. (Originally, this was just a while True: loop, but I added the "keepgoing" variable to try to remedy the situation - unsuccessfully, as it were.) I'm not sure how to resolve this issue, and attempting to turn the loop "off" long enough to kill the first loop but allowing the next one to execute has not worked out so far. Is there any way to exit this while keepgoing == True loop cleanly to prevent things from stacking as such, or is there a better way in general to address the problem of loading a new image after a certain amount of time?
Thanks!
EDIT: to be clear, the problem is with ending the 'while' loop when the timer hits 0. setting keepgoing = False doesn't seem to work (probably because I can't 'return' it), so I need something similar to that if it's possible...

Start video read from a particular frame

I have obtained the frame_count of a saved video.
self.frame_count = self.capture.get(cv.CV_CAP_PROP_FRAME_COUNT) - 1
Now, I want to start a frame read from a particular frame_count. How do I do this?
Reason: I need to track an object and I have found the location of the object I want to track using HSV image segmentation. Now to track it, I intend to start the video from that particular frame and set the track window to the objects' coordinates.
Want: It should not be redundant and computationally intensive.
Use the below code to accomplish your task.
The Opencv version you are using is old. Use Opencv2.
Link : http://docs.opencv.org/trunk/doc/py_tutorials/py_tutorials.html
import numpy as np
import cv2
import os
cam = cv2.VideoCapture('full_path/bird.avi')
i = 1
initialize_first_frame = cam.read()[1]
start_from_frame = 5
dir = './testImages/'
os.makedirs(dir)
while True:
if(i>=start_from_frame):
cv2.imshow('Frame Conversion',cam.read()[1])
cv2.imwrite(dir + "/image-"+str(i).zfill(5)+".png",cam.read()[1])
i = i + 1
key = cv2.waitKey(10)
if key == 27:
cv2.destroyWindow('Frame Conversion')
break
print "End"
I hope this is the code you want.
Try the following:
f = # put here the frame from which you want to start
self.capture.set(CV_CAP_PROP_POS_FRAMES, f)
while f < self.frame_count:
retval, image = self.capture.read()
f += 1
For any version 3 and 4 of OpenCV Python, follow this -
import cv2
# suppose you want to start reading from frame no 500
frame_set_no = 500
cap = cv2.VideoCapture("full_path/cow.mp4")
# capture is set to the desired frame
cap.set(cv2.CAP_PROP_POS_FRAMES, frame_set_no)
while True:
ret, frame = cap.read()
cv2.imshow("Video", frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cv2.destroyAllWindows()

Categories