OpenCV VideoCapture only updates after 5 read()s - python

I have a very strange error that has been plaguing my research for a few years now. I'm using OpenCV2 with Python to read image data from a webcam. However, the image is lagged by 5 frames. In other words, each call to read() is 5 frame behind real-time.
One bandage fix I have been using is to grab() 4 frames and then read the 5th every time I need an updated image, but that absolutely murders my performance.
Here's the code that I am using to display the images from the webcam
frame = self.getGoodFrame()
if self.DEBUG:
window = cv2.namedWindow("Angles")
imgHSV = cv2.cvtColor(frame, cv2.cv.CV_BGR2HSV)
... Reseach specific code I shouldn't be giving out here ...
... It finds the center of a few bright colors in an image
if self.DEBUG:
debugImage = numpy.zeros((self.CAMERA_HEIGHT, self.CAMERA_WIDTH), numpy.uint8) #blank image
... Then we draw some stuff to the image to be displayed ...
cv2.imshow("Angles", debugImage)
cv2.waitKey(1)
raw_input()
and getGoodFrame()
def getGoodFrame(self):
MIN_READS_FOR_GOOD_FRAME = 4
for i in xrange(MIN_READS_FOR_GOOD_FRAME):
successful_read = self.capture.grab()
successful_read, frame = self.capture.read()
if not successful_read:
print "Unable to read from webcam, exiting."
return frame
You'll notice that I have a raw_input() call. That makes it so I can see how many reads need to happen by pressing enter a few times in console. This shows that there is exactly 5 frames of lag.
I don't think it's a hardware issue, I've had this happen with multiple webcams and multiple USB cords. I haven't tried reproducing the error on another machine though.

So the problem was something to do with the way that my hardware was buffering the frames. I never quite got to the bottom of it but the solution I found was very simple paralellization. I modified my code to open up a thread and constantly update a variable holding the current frame. Then, when I need the current frame I just ask for whatever that variable is at the moment.
Note that this is still 5 read() calls behind because of the ever-present buffering, however because I am doing read() calls continuously and it's all on its own thread, it is very up to date. This is because I can call many read() calls per second. The image I get is not noticeably behind real-time at all.
The Python class I made to do this is below. It is not very nice code but it works. To do this properly, I would (and will) add graceful ways to exit the infinite loop. It will work for me though and it has improved the speed of my image detection code by well over 100 fold, which is extremely awesome and exciting for me :)
class webcamImageGetter:
def __init__(self):
self.currentFrame = None
self.CAMERA_WIDTH = #webcam width
self.CAMERA_HEIGHT = #webcam height
self.CAMERA_NUM = 0
self.capture = cv2.VideoCapture(0) #Put in correct capture number here
#OpenCV by default gets a half resolution image so we manually set the correct resolution
self.capture.set(cv2.cv.CV_CAP_PROP_FRAME_WIDTH,self.CAMERA_WIDTH)
self.capture.set(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT,self.CAMERA_HEIGHT)
#Starts updating the images in a thread
def start(self):
Thread(target=self.updateFrame, args=()).start()
#Continually updates the frame
def updateFrame(self):
while(True):
ret, self.currentFrame = self.capture.read()
while (self.currentFrame == None): #Continually grab frames until we get a good one
ret, frame = self.capture.read()
def getFrame(self):
return self.currentFrame
To use this, you initialize it then called start on the instance. This will make it so when you later called getFrame(), it will have the most up to date frame from the webcam. Woohoo!

Default buffer is set to 4.0 instead of 1.0.
To Fix it:
cap.set(38,1)
reference: https://docs.opencv.org/4.x/d4/d15/group__videoio__flags__base.html#ga41c5cfa7859ae542b71b1d33bbd4d2b4

Related

Python cv2 VideoCapture() in daemon thread turns black randomly

I have a webcam that captures something I want to be able to monitor live. Every 30 seconds an operation needs to be done on the most recent frame of the video. To do this, I've set up a cv2.NamedWindow() in a daemon thread that allows me to see the webcam feed. Then using a scheduler, I can do the operation I need every 30 seconds. The problem is, the camera feed will randomly go black, and save an all black image. Sometimes this happens after 15 minutes of operation, sometimes after 4 hours of operation., so it's very inconsistent and there seems to be no pattern. Here is the code:
import time
import gc
import cv2
import threading
import schedule
import numpy
class operator():
def __init__(self):
print("Start")
def start_cam(self):
is_blurry = True
while is_blurry == True:
print("Start of start_cam")
self.camera = cv2.VideoCapture(0, cv2.CAP_DSHOW) # Make camera object with correct format, size, and autofocus. All of these appear to be necessary to get 4K resolution in OpenCV
self.camera.set(cv2.CAP_PROP_FRAME_WIDTH, 3840) #3840
self.camera.set(cv2.CAP_PROP_FRAME_HEIGHT, 2160) #2160
self.camera.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter.fourcc('m','j','p','g'))
self.camera.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter.fourcc('M','J','P','G'))
self.camera.set(cv2.CAP_PROP_AUTOFOCUS, 1)
self.camera.set(cv2.CAP_PROP_FPS, 60)
i=10
while i>0: # Takes 10 pictures and waits 1 sec in between each. This is done to make sure it has time to autofocus
i=i-1
cv2.waitKey(1000) #1000
ret, self.frame = self.camera.read()
if ret != True:
print("image error") # Check to make sure it actually took a picture and didn't fail
if ret != True and i == 1:
restart_loop = True
print("Image error, restarting loop") #
continue
laplace = cv2.Laplacian(self.frame, cv2.CV_64F).var() # Checks if image is blurry. If it is, it restarts the camera.
print(laplace)
if laplace < 20: # Blurry
print("Blurry image, reinitializing")
self.camera.release()
del(self.camera)
cv2.destroyAllWindows()
gc.collect()
time.sleep(2)
else:
print("image not blurry")
break
def live_loop(self):
loop_bool = True
cv2.namedWindow("Live Feed", cv2.WINDOW_NORMAL) # Creates a normal window so that the 4k image is scaled correctly for the live feed (I have a relatively small monitor)
while loop_bool == True:
ret, self.frame = self.camera.read()
if ret != True:
print("image error")
break
self.frame_rotate = cv2.rotate(self.frame, cv2.ROTATE_180)
cv2.imshow("Live Feed", self.frame_rotate)
k = cv2.waitKey(10)
gc.collect()
def data_operation(self):
print("Start data operation")
imgpath = "path where image is saved"
cv2.imwrite(imgpath, self.frame)
t = time.localtime() # If the image can't be read as a number, saves the image with a timestamp so that it can be examined later
timestamp = time.strftime('%b-%d-%Y_%H;%M', t)
print(timestamp)
if __name__== "__main__":
op = operator()
op.start_cam()
x = threading.Thread(target=op.live_loop, daemon = True)
x.start()
schedule.every(30).seconds.do(op.data_operation)
try:
while True:
schedule.run_pending()
time.sleep(1)
except KeyboardInterrupt:
print("Ended Program")
gc.collect()
This is the smallest amount of code I can reproduce the issue with. There is more going on just this, but this code block has been run and the issue persists. I have found that when I remove the line that saves the image in the data_operation function, which is the cv2.imwrite(imgpath, self.frame), the program seems to work and the camera never goes black and the issue is gone. So I think that is the problem, but I don't know how else to save the frame as an image every 30 seconds while also keeping the live feed up and running. As far as I know, the rest of the code works perfectly except for this, and I do need to be able to save and access the image.
I am not a programmer by trade so I've just been figuring this out as I go. If something is glaringly stupid let me know. If there is a much easier/safer way to display a live webcam feed while also automatically grabbing a picture every 30 seconds and doing stuff with that picture while keeping the live loop going, I would be happy to hear that too.
Any help would be appreciated, and let me know if you have further questions.
EDIT: I have updated the code to simplify and continue shrinking the smallest reproduceable sample. I realized the start_cam function was unnecessary, as all of the cv2 initialization can be done at the start of the live_loop function. The issue persists. I have tried adding a mutex, however that seems to not help (though it's possible I am implementing it wrong). Any info or working examples would be greatly appreciated.
EDIT 2: Another thing to note is that the exact same program running on my laptop (using the built-in webcam rather than the 4k Logitech Brio) doe not seem to have the same issue. Is the frame rate or resolution causing the issue perhaps? Testing is difficult since I need to keep a working program running while working on this one, but I will do more tests when I am able and keep this post updated if anything develops. Still, feel free to toss any ideas you have to me.

OpenCV real time streaming video capture is slow. How to drop frames or get synced with real time?

Goal and problem
I'd like to set up an opencv system to process either HLS streams or RMTP streams, however, I am running into a strange issue regarding a reduced frame-rate and an accumulating lag. It's as if the video gets further and further behind from where it is supposed to be in the stream.
I'm looking for a way to keep up to date with a live source even if it means dropping frames.
Current approach
import cv2
cap = cv2.VideoCapture()
cap.open('https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w1421640637.m3u8')
while (True):
_, frame = cap.read()
cv2.imshow("camCapture", frame)
cv2.waitKey(1)
I've validated the quality of the stream on VLC and it seems to work fine there.
cv2 speed
.
realistic/expected speed
Questions:
What am I doing wrong here?
Why is it so slow?
How do I sync it to real-time speeds?
My hypothesis is that the jitter is most likely due to network limitations and occurs when a frame packet is dropped. When a frame is dropped, this causes the program to display the last "good" frame which results in the display freezing. This is probably a hardware or bandwidth issue but we can alleviate some of this with software. Here are some possible changes:
1. Set maximum buffer size
We set the cv2.videoCapture() object to have a limited buffer size with the cv2.CAP_PROP_BUFFERSIZE parameter. The idea is that by limiting the buffer, we will always have the latest frame. This can also help to alleviate the problem of frames randomly jumping ahead.
2. Set frame retrieval delay
Currently, I believe the read() is reading too fast even though it is in its own dedicated thread. This may be one reason why all the frames appear to pool up and suddenly burst in the next frame. For instance, say in a one second time interval, it may produce 15 new frames but in the next one second interval, only 3 frames are returned. This may be due to the network packet frame loss so to ensure that we obtain constant frame rates, we simply add a delay in the frame retrieval thread. A delay to obtain roughly ~30 FPS does a good job to "normalize" the frame rate and smooth the transition between frames incase there is packet loss.
Note: We should try to match the frame rate of the stream but I'm not sure what the FPS of the webcam is so I just guessed 30 FPS. Also, there is usually a "direct" stream link instead of going through a intermediate webserver which can greatly improve performance.
If you try using a saved .mp4 video file, you will notice that there is no jitter. This confirms my suspicion that the problem is most likely due to network latency.
from threading import Thread
import cv2, time
class ThreadedCamera(object):
def __init__(self, src=0):
self.capture = cv2.VideoCapture(src)
self.capture.set(cv2.CAP_PROP_BUFFERSIZE, 2)
# FPS = 1/X
# X = desired FPS
self.FPS = 1/30
self.FPS_MS = int(self.FPS * 1000)
# Start frame retrieval thread
self.thread = Thread(target=self.update, args=())
self.thread.daemon = True
self.thread.start()
def update(self):
while True:
if self.capture.isOpened():
(self.status, self.frame) = self.capture.read()
time.sleep(self.FPS)
def show_frame(self):
cv2.imshow('frame', self.frame)
cv2.waitKey(self.FPS_MS)
if __name__ == '__main__':
src = 'https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w1421640637.m3u8'
threaded_camera = ThreadedCamera(src)
while True:
try:
threaded_camera.show_frame()
except AttributeError:
pass
Related camera/IP/RTSP/streaming, FPS, video, threading, and multiprocessing posts
Python OpenCV streaming from camera - multithreading, timestamps
Video Streaming from IP Camera in Python Using OpenCV cv2.VideoCapture
How to capture multiple camera streams with OpenCV?
OpenCV real time streaming video capture is slow. How to drop frames or get synced with real time?
Storing RTSP stream as video file with OpenCV VideoWriter
OpenCV video saving
Python OpenCV multiprocessing cv2.VideoCapture mp4
Attempt at threading
I've attempted this solution from nathancy with minor success.
It involves:
creating a separate thread for image capture from the source
using the main thread exclusively for display.
Code:
import cv2
from threading import Thread
class ThreadedCamera(object):
def __init__(self, source = 0):
self.capture = cv2.VideoCapture(source)
self.thread = Thread(target = self.update, args = ())
self.thread.daemon = True
self.thread.start()
self.status = False
self.frame = None
def update(self):
while True:
if self.capture.isOpened():
(self.status, self.frame) = self.capture.read()
def grab_frame(self):
if self.status:
return self.frame
return None
if __name__ == '__main__':
stream_link = "https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w1421640637.m3u8"
streamer = ThreadedCamera(stream_link)
while True:
frame = streamer.grab_frame()
if frame is not None:
cv2.imshow("Context", frame)
cv2.waitKey(1)
Jittery, but real-time results
.
The streaming works. It maintains real-time. However, it is as if all the frames pool up and suddenly burst into the video. I would like somebody to explain that.
Room for improvement
The real-time stream can be found here.
https://www.earthcam.com/usa/newyork/timessquare/?cam=tsstreet
This site is scraped for the m3u8 using python's streamlink stream scraper.
import streamlink
streams = streamlink.streams("https://www.earthcam.com/usa/newyork/timessquare/?cam=tsstreet")
print(streams)
which yeilds:
OrderedDict([
('720p',<HLSStream('https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w202109066.m3u8')>),
('live', <RTMPStream({'rtmp': 'rtmp://videos3.earthcam.com/fecnetwork/', 'playpath': '9974.flv', 'pageUrl': 'https://www.earthcam.com/usa/newyork/timessquare/?cam=tsstreet','swfUrl': 'http://static.earthcam.com/swf/streaming/stream_viewer_v3.swf', 'live': 'true'}, redirect=False>),
('worst', <HLSStream('https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w202109066.m3u8')>),
('best', <RTMPStream({'rtmp': 'rtmp://videos3.earthcam.com/fecnetwork/', 'playpath': '9974.flv', 'pageUrl': 'https://www.earthcam.com/usa/newyork/timessquare/?cam=tsstreet', 'swfUrl': 'http://static.earthcam.com/swf/streaming/stream_viewer_v3.swf', 'live': 'true'}, redirect=False>)
])
The possibility that the streams are being read wrong.
I would suggest double checking the compatible video stream codecs with the hardware. I ran into the same issue, frame rate dropped to 5 fps only during streaming, because it was defaulting to a format that is not being streamed so it would convert it then display very lagged (~1s) with lower fps as well.
use Self.capture.set(cv2.CAP_PROP_FOURCC ,cv2.VideoWriter_fourcc('M', 'J', 'P', 'G') ) with the proper codec in place of MJPG and with your cv2.VideoCapture and see if that helps.

Increase Matplotlib graphing and OpenCV video processing performance [duplicate]

Goal and problem
I'd like to set up an opencv system to process either HLS streams or RMTP streams, however, I am running into a strange issue regarding a reduced frame-rate and an accumulating lag. It's as if the video gets further and further behind from where it is supposed to be in the stream.
I'm looking for a way to keep up to date with a live source even if it means dropping frames.
Current approach
import cv2
cap = cv2.VideoCapture()
cap.open('https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w1421640637.m3u8')
while (True):
_, frame = cap.read()
cv2.imshow("camCapture", frame)
cv2.waitKey(1)
I've validated the quality of the stream on VLC and it seems to work fine there.
cv2 speed
.
realistic/expected speed
Questions:
What am I doing wrong here?
Why is it so slow?
How do I sync it to real-time speeds?
My hypothesis is that the jitter is most likely due to network limitations and occurs when a frame packet is dropped. When a frame is dropped, this causes the program to display the last "good" frame which results in the display freezing. This is probably a hardware or bandwidth issue but we can alleviate some of this with software. Here are some possible changes:
1. Set maximum buffer size
We set the cv2.videoCapture() object to have a limited buffer size with the cv2.CAP_PROP_BUFFERSIZE parameter. The idea is that by limiting the buffer, we will always have the latest frame. This can also help to alleviate the problem of frames randomly jumping ahead.
2. Set frame retrieval delay
Currently, I believe the read() is reading too fast even though it is in its own dedicated thread. This may be one reason why all the frames appear to pool up and suddenly burst in the next frame. For instance, say in a one second time interval, it may produce 15 new frames but in the next one second interval, only 3 frames are returned. This may be due to the network packet frame loss so to ensure that we obtain constant frame rates, we simply add a delay in the frame retrieval thread. A delay to obtain roughly ~30 FPS does a good job to "normalize" the frame rate and smooth the transition between frames incase there is packet loss.
Note: We should try to match the frame rate of the stream but I'm not sure what the FPS of the webcam is so I just guessed 30 FPS. Also, there is usually a "direct" stream link instead of going through a intermediate webserver which can greatly improve performance.
If you try using a saved .mp4 video file, you will notice that there is no jitter. This confirms my suspicion that the problem is most likely due to network latency.
from threading import Thread
import cv2, time
class ThreadedCamera(object):
def __init__(self, src=0):
self.capture = cv2.VideoCapture(src)
self.capture.set(cv2.CAP_PROP_BUFFERSIZE, 2)
# FPS = 1/X
# X = desired FPS
self.FPS = 1/30
self.FPS_MS = int(self.FPS * 1000)
# Start frame retrieval thread
self.thread = Thread(target=self.update, args=())
self.thread.daemon = True
self.thread.start()
def update(self):
while True:
if self.capture.isOpened():
(self.status, self.frame) = self.capture.read()
time.sleep(self.FPS)
def show_frame(self):
cv2.imshow('frame', self.frame)
cv2.waitKey(self.FPS_MS)
if __name__ == '__main__':
src = 'https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w1421640637.m3u8'
threaded_camera = ThreadedCamera(src)
while True:
try:
threaded_camera.show_frame()
except AttributeError:
pass
Related camera/IP/RTSP/streaming, FPS, video, threading, and multiprocessing posts
Python OpenCV streaming from camera - multithreading, timestamps
Video Streaming from IP Camera in Python Using OpenCV cv2.VideoCapture
How to capture multiple camera streams with OpenCV?
OpenCV real time streaming video capture is slow. How to drop frames or get synced with real time?
Storing RTSP stream as video file with OpenCV VideoWriter
OpenCV video saving
Python OpenCV multiprocessing cv2.VideoCapture mp4
Attempt at threading
I've attempted this solution from nathancy with minor success.
It involves:
creating a separate thread for image capture from the source
using the main thread exclusively for display.
Code:
import cv2
from threading import Thread
class ThreadedCamera(object):
def __init__(self, source = 0):
self.capture = cv2.VideoCapture(source)
self.thread = Thread(target = self.update, args = ())
self.thread.daemon = True
self.thread.start()
self.status = False
self.frame = None
def update(self):
while True:
if self.capture.isOpened():
(self.status, self.frame) = self.capture.read()
def grab_frame(self):
if self.status:
return self.frame
return None
if __name__ == '__main__':
stream_link = "https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w1421640637.m3u8"
streamer = ThreadedCamera(stream_link)
while True:
frame = streamer.grab_frame()
if frame is not None:
cv2.imshow("Context", frame)
cv2.waitKey(1)
Jittery, but real-time results
.
The streaming works. It maintains real-time. However, it is as if all the frames pool up and suddenly burst into the video. I would like somebody to explain that.
Room for improvement
The real-time stream can be found here.
https://www.earthcam.com/usa/newyork/timessquare/?cam=tsstreet
This site is scraped for the m3u8 using python's streamlink stream scraper.
import streamlink
streams = streamlink.streams("https://www.earthcam.com/usa/newyork/timessquare/?cam=tsstreet")
print(streams)
which yeilds:
OrderedDict([
('720p',<HLSStream('https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w202109066.m3u8')>),
('live', <RTMPStream({'rtmp': 'rtmp://videos3.earthcam.com/fecnetwork/', 'playpath': '9974.flv', 'pageUrl': 'https://www.earthcam.com/usa/newyork/timessquare/?cam=tsstreet','swfUrl': 'http://static.earthcam.com/swf/streaming/stream_viewer_v3.swf', 'live': 'true'}, redirect=False>),
('worst', <HLSStream('https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w202109066.m3u8')>),
('best', <RTMPStream({'rtmp': 'rtmp://videos3.earthcam.com/fecnetwork/', 'playpath': '9974.flv', 'pageUrl': 'https://www.earthcam.com/usa/newyork/timessquare/?cam=tsstreet', 'swfUrl': 'http://static.earthcam.com/swf/streaming/stream_viewer_v3.swf', 'live': 'true'}, redirect=False>)
])
The possibility that the streams are being read wrong.
I would suggest double checking the compatible video stream codecs with the hardware. I ran into the same issue, frame rate dropped to 5 fps only during streaming, because it was defaulting to a format that is not being streamed so it would convert it then display very lagged (~1s) with lower fps as well.
use Self.capture.set(cv2.CAP_PROP_FOURCC ,cv2.VideoWriter_fourcc('M', 'J', 'P', 'G') ) with the proper codec in place of MJPG and with your cv2.VideoCapture and see if that helps.

OpenCV Python Trackbar Callback

I'm currently writing a Python (2.7) edge detection script with opencv (3.0) which basically works fine so far.
Now I want to switch between my Laptop camera and a second webcam while the program is running.
So i implemented a trackbar as a switch but i have no idea how to get the information that the trackbar has changed.
The normal getTrackbarPos() isn't enough, i need something like:
if TrackbarHasChanged() -> restart program-> cv2.VideoCapture(changed camera) -> while(true) loop
Thanks in advance
You are in luck. Actually that behavior already exists in OpenCV trackbar. If you read the documentation of createTrackbar you will see that for python you have:
cv2.createTrackbar(trackbarName, windowName, value, count, onChange) → None
The onChange argument is:
onChange – Pointer to the function to be called every time the slider changes position. This function should be prototyped as void Foo(int,void*); , where the first parameter is the trackbar position and the second parameter is the user data (see the next parameter). If the callback is the NULL pointer, no callbacks are called, but only value is updated.
Which basically means what you want to do. Instead of checking the pos every loop, if it has a change do the change.
For the restart the program part it is a little bit tricky. As far as I know (I may be wrong) this runs in another thread, and may get some race condition problem....
Here is some small code (which I cannot test fully, since I don't have a webcam) that creates a trackbar, creates the callback function, changes the camera, and avoid thread problems (I think, you may need to actually use Lock when using cameraToUse and cameraChange, to really be thread-safe). Without camera it runs, however it will always print error in connection. With cameras it may actually work :)
I added a lot of comments, but if you don't get a part feel free to ask in a comment
import cv2
import numpy as np
# global variables
amountOfCameras = 3 # how many cameras you want to use
cameraToUse = 0 #initial camera
cameraChange = True #starts true to connect at start up
camera = cv2.VideoCapture() # empty placeholder
# callback function for the tracker, x is the position value
# you may put whatever name in here
def trackerCallback(x):
global cameraToUse
global cameraChange
if cameraToUse != x:
print "I change to this camera", x
cameraToUse = x
cameraChange = True
# function to connect to a camera and replace the videoCapture variable
def connectToCamera():
global cameraChange
global camera
print "Connecting to camera", cameraToUse
camera = cv2.VideoCapture(cameraToUse)
# basic check for connection error
if camera.isOpened():
print "Successfully connected"
else:
print "Error connecting to camera", cameraToUse
cameraChange = False
#initial image with the tracker
img = np.zeros((200,600,3), np.uint8)
cv2.namedWindow('image')
cv2.createTrackbar('Camera','image',0,amountOfCameras-1,trackerCallback)
while(1):
#check if it has to connect to something else
if cameraChange:
connectToCamera()
# if no problems with the current camera, grab a frame
if camera.isOpened():
ret, frame = camera.read()
if ret:
img = frame
# displays the frame, in case of none, displays the previous one
cv2.imshow('image',img)
# if esc button exit
k = cv2.waitKey(1) & 0xFF
if k == 27:
break
cv2.destroyAllWindows()

Capture Raspicam image via PiCamera module - How to avoid old images?

I am working on a project where we are using the Raspicam attached to a Raspberry Pi to capture (and process) images with python using the PiCamera module.
With our current implementation I am experiencing an unexpected behaviour.
The camera is configured to capture images with 15 frames per second. To simulate processing time the program waits 5 seconds.
Minimal example:
#!/usr/bin/env python
import cv2
from picamera import PiCamera
from picamera.array import PiRGBArray
class RaspiCamera:
def __init__(self, width, height, framerate):
self.camera = PiCamera()
self.camera.resolution = (width, height)
self.camera.framerate = framerate
self.rawCapture = PiRGBArray(self.camera, size=self.camera.resolution)
self.capture_continuous = self.camera.capture_continuous(self.rawCapture, format="bgr", use_video_port=True)
def capture(self):
frame = self.capture_continuous.next()
image = self.rawCapture.array
self.rawCapture.truncate(0)
return image
if __name__ == "__main__":
camera = RaspiCamera(640, 480, 15)
while True:
frame = camera.capture()
cv2.imshow("Image", frame)
if cv2.waitKey(5000) & 0xFF == ord('q'):
break
When capture() is called for the first time, self.capture_continuous.next() returns an up to date image. When calling capture() consecutively, it often happens that self.capture_continuous.next() does not return the latest image but one that is already a few seconds old (verified by pointing the camera at a clock). From time to time, it's even older than 10 seconds. On the other hand, sometimes self.capture_continuous.next() actually returns the latest image.
Since capture_continuous is an object of the type generator, my assumption is that it keeps generating camera images in the background that accumulate in a queue while the program waits and on the next call of self.capture_continuous.next() the next element in the queue is returned.
Anyway, I am only interested in the latest, most up to date image the camera has captured.
Some first attempts to get hold of the latest images failed. I tried to call self.capture_continuous.next() repeatedly in a while loop to get to the latest image.
Since a generator is apparently also an iterator I tried some methods mentioned in this post: Cleanest way to get last item from Python iterator.
Simply using the capture() function of the PiCamera class itself is not an option since it takes approx. 0.3 seconds till the image is captured what is too much for our use case.
Does anyone have a clue what might cause the delay described above and how it could be avoided?

Categories