We Keep Coding

How to stop FastAPI/Uvicorn from hanging when there's an image decoding error?

I'm building a web service where I display the stream from an IP camera. I'm using rtsp and opencv-python to read from the camera. I'm using WebSockets to send the frames from the backend to the frontend. For my backend server, I'm using FastAPI and Uvicorn. Here is what my current WebSocket endpoint looks like.
#app.websocket("/ws/video_feed")
async def websocket_video_feed(websocket: WebSocket):
await websocket.accept()
camera = cv2.VideoCapture(camera_source)
if not camera.isOpened():
raise RuntimeError("Could not start camera.")
try:
while True:
ret, img = camera.read()
if ret:
try:
_, frame = cv2.imencode(".jpg", img)
await websocket.send_json(
{
"type": "frame",
"payload": str(base64.b64encode(frame).decode("utf-8")),
}
)
except:
continue
else:
camera = cv2.VideoCapture(camera_source)
if not camera.isOpened():
raise RuntimeError("Could not start camera.")
continue
except WebSocketDisconnect:
print("WS Client disconnected.")
return
The issue is that when I switch pages on the front end and the WebSocket connection is closed, the backend will sometimes freeze and not respond to any other API requests, particularly after encountering some h264 decoding issues. I found a stack overflow thread talking about splitting up the image reading and writing into separate threads and using a simple first-in-first-out queue but I still encounter the same issue.
q = queue.Queue()
def Receive():
camera = cv2.VideoCapture(camera_source)
if not camera.isOpened():
raise RuntimeError("Could not start camera.")
while True:
if camera.isOpened():
ret, img = camera.read()
if ret:
q.put(img)
else:
camera.release()
camera = cv2.VideoCapture(camera_source)
if not camera.isOpened():
raise RuntimeError("Could not start camera.")
continue
else:
camera.release()
camera = cv2.VideoCapture(camera_source)
if not camera.isOpened():
raise RuntimeError("Could not start camera.")
continue
#app.websocket("/ws/video_feed")
async def websocket_video_feed(websocket: WebSocket):
await websocket.accept()
try:
while True:
try:
if q.empty() != True:
img = q.get()
_, frame = cv2.imencode(".jpg", img)
await websocket.send_json(
{
"type": "frame",
"payload": str(base64.b64encode(frame).decode("utf-8")),
}
)
except:
continue
ack = await websocket.receive_json()
if ack["type"] == "close":
return
except WebSocketDisconnect:
print("WS Client disconnected.")
return
if __name__ == '__main__':
p1 = threading.Thread(target=Receive)
p1.start()
uvicorn.run(app, host="0.0.0.0", port=5000)
What is the proper way to do this?

'select timeout' error while running python opencv script

I'm trying to capture video/image feed from a USB 3.0 webcam. When ever I try to run this code,the code runs and captures feed. But, after a while I get 'select timeout'.
The error is not grabbed by exception.
Can someone help me with this issue?
select timeout
OpenCV Error: Assertion failed (total() == 0 || data != __null) in Mat, file /home/opencv-3.4.0/modules/core/include/opencv2/core/mat.inl.hpp, line 500
try:
while True:
ret, frame = cap.read()
if ret != True:
break
if stat == None:
stat = ret
print('[INFO] ---- Reading video feed')
continue
frame = cv2.resize(frame , (int(320/2),int(240/2)))
video.write(frame)
if stat1 == None:
stat1 = ret
print('[INFO] ---- Video recording on process')
continue
c+=1
print(c)
key = cv2.waitKey(30) & 0xFF
if key == 27:
break
elif c == 1200:
break
except Exception as e:
print(e)
sys.exit()
I have also tried editing the 'uvcvideo' driver parameters
$ modprobe uvcvideo nodrop=1 timeout=6000
And also,
Enabled module traces,
$ sudo echo 0xffff > /sys/module/uvcvideo/parameters/trace
Observed these errors on 'dmesg' IMAGE
Link which I followed:
Select Timeout error in Ubuntu - Opencv

Python: Automatically reconnect IP camera

My IP camera seems to be a little unstable and disconnects randomly. I'd like my script to be able to determine when its disconnected and attempt to reconnect a few times, probably waiting 5-10 seconds between attempts. I've tried a few things, but nothing is working.
This is my basic script, when ret is false the script ends:
#!/usr/local/bin/python3
import cv2
import time
import datetime
print("start time: " + datetime.datetime.now().strftime("%A %d %B %Y %I:%M:%S%p"))
cap = cv2.VideoCapture('rtsp://<ip><port>/live0.264')
while(True):
# Capture frame-by-frame
ret, frame = cap.read()
# Confirm we have a valid image returned
if not ret:
print("disconnected!")
break
# Our operations on the frame come here
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2BGRA)
# Display the resulting frame
cv2.imshow('frame', gray)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
print("end time: " + time.strftime("%X"))
# When everything is done, release the capture
cap.release()
cv2.destroyAllWindows()
Edit: I would also like the script to try to reconnect to the camera in the event that my network goes down temporarily or anything like that as well.

I was finally able to solve this myself. Hopefully this is useful for anyone else looking to do the same thing.
This is actually a shell of a more complex script that has logic for motion detection and video recording when motion is detected. Everything is working very well with this basic logic (and my crappy IP camera) although I am still doing testing.
#!/usr/local/bin/python3
import cv2
import datetime
import time
def reset_attempts():
return 50
def process_video(attempts):
while(True):
(grabbed, frame) = camera.read()
if not grabbed:
print("disconnected!")
camera.release()
if attempts > 0:
time.sleep(5)
return True
else:
return False
recall = True
attempts = reset_attempts()
while(recall):
camera = cv2.VideoCapture("rtsp://<ip><port>/live0.264")
if camera.isOpened():
print("[INFO] Camera connected at " +
datetime.datetime.now().strftime("%m-%d-%Y %I:%M:%S%p"))
attempts = reset_attempts()
recall = process_video(attempts)
else:
print("Camera not opened " +
datetime.datetime.now().strftime("%m-%d-%Y %I:%M:%S%p"))
camera.release()
attempts -= 1
print("attempts: " + str(attempts))
# give the camera some time to recover
time.sleep(5)
continue

More detailed description:
https://github.com/Combinacijus/various-code-samples/tree/master/Python/OpenCV/ip_cam_reconnecting
Wrote a class to deal with IP camera disconnecting randomly. Main idea is to check if cap.read() returns a frame and if it doesn't it tries to reconnect to the camera.
import cv2
import requests # NOTE: Only used for forceful reconnection
import time # NOTE: Only used for throttling down printing when connection is lost
class IPVideoCapture:
def __init__(self, cam_address, cam_force_address=None, blocking=False):
"""
:param cam_address: ip address of the camera feed
:param cam_force_address: ip address to disconnect other clients (forcefully take over)
:param blocking: if true read() and reconnect_camera() methods blocks until ip camera is reconnected
"""
self.cam_address = cam_address
self.cam_force_address = cam_force_address
self.blocking = blocking
self.capture = None
self.RECONNECTION_PERIOD = 0.5 # NOTE: Can be changed. Used to throttle down printing
self.reconnect_camera()
def reconnect_camera(self):
while True:
try:
if self.cam_force_address is not None:
requests.get(self.cam_force_address)
self.capture = cv2.VideoCapture(self.cam_address)
if not self.capture.isOpened():
raise Exception("Could not connect to a camera: {0}".format(self.cam_address))
print("Connected to a camera: {}".format(self.cam_address))
break
except Exception as e:
print(e)
if self.blocking is False:
break
time.sleep(self.RECONNECTION_PERIOD)
def read(self):
"""
Reads frame and if frame is not received tries to reconnect the camera
:return: ret - bool witch specifies if frame was read successfully
frame - opencv image from the camera
"""
ret, frame = self.capture.read()
if ret is False:
self.reconnect_camera()
return ret, frame
if __name__ == "__main__":
CAM_ADDRESS = "http://192.168.8.102:4747/video" # NOTE: Change
CAM_FORCE_ADDRESS = "http://192.168.8.102:4747/override" # NOTE: Change or omit
cap = IPVideoCapture(CAM_ADDRESS, CAM_FORCE_ADDRESS, blocking=True)
# cap = IPVideoCapture(CAM_ADDRESS) # Minimal init example
while True:
ret, frame = cap.read()
if ret is True:
cv2.imshow(CAM_ADDRESS, frame)
if cv2.waitKey(1) == ord("q"):
break

cv2.VideoCapture.read() gets old frame after time.sleep()

I tried to capture (stereo) images with Python's opencv and two cameras so therefore every 5 seconds an image should be saved. But the problem here is that an old frame is saved.
The minified code is as follows:
cap = cv2.VideoCapture(0)
for i in range(20):
time.sleep(5)
print "Taking image %d:" % i
ret, frame = cap.read()
cv2.imwrite("image %d" % i, frame)
print " image done." if ret else " Error while taking image..."
cap.release()
cv2.destroyAllWindows()
To check this, I changed the position of the camera after each taken image. But nevertheless an image from an old position is saved (actually not the same, but I assume some frames after the last saved image). After 5 (or more) images finally the captured location in the image does also change.
So, is there any problem with time.sleep? I guess that I'm not getting the actual frame, but a buffered one. If this is the case, how could I fix it and capture the actual frame?

VideoCapture is buffering.
If you always want the actual frame, do this:
while True:
cap = cv2.VideoCapture(0)
ret, frame = cap.read()
cap.release()
cv2.imshow(" ", frame)
if cv2.waitKey(2000) != -1:
break

you need to count the elapsed time, but not stop read frames. like this:
import cv2
import time
cap = cv2.VideoCapture(0)
preframe_tm = time.time()
i = 0
while True:
ret, frame = cap.read()
elapsed_time = time.time() - preframe_tm
if elapsed_time < 5:
continue
preframe_tm = time.time()
i += 1
print("Taking image %d:" % i)
cv2.imwrite("image_%d.jpg" % i, frame)
if i >= 20:
break
cap.release()
cv2.destroyAllWindows()

How to solve OverflowError: Python int too large to convert to C long with opencv?

__author__ = 'Kyle'
HELP_MESSAGE = """Script: frame_grabber.py
Usage: python frame_grabber.py path/to/video
Requires: OpenCV 2.4.8+
Purpose: Select and save frames from a given video.
Commands:
Key Function
a previous frame
d next frame
q exit
SHIFT + a skip 10 frames forward
SHIFT + d skip 10 frames backwards
s saves current frame
dbl click saves current frame
Controls:
Slider Navigate through the video
"""
# Check if the user has provided a path to a file
# otherwise display the HELP_MESSAGE
import sys
import time as t
# Check if OpenCV module is present
# otherwise stop the application
try:
import cv2
except ImportError as e:
print "Fatal Error: Could not import OpenCV, ", e
exit(-1)
else:
print "Using OpenCV ", cv2.__version__
# these flags may depend on your opencv version:
# in opencv 3.0.0 these flags are implemented as
# cv2.CAP_PROP_POS_FRAMES and
# cv2.CAP_PROP_FRAME_COUNT
CURRENT_FRAME_FLAG = cv2.cv.CV_CAP_PROP_POS_FRAMES
TOTAL_FRAMES_FLAG = cv2.cv.CV_CAP_PROP_FRAME_COUNT
WIN_NAME = "Frame Grabber"
POS_TRACKBAR = "pos_trackbar"
#VIDEO_PATH = 'camera13.h264'
#try:
# VIDEO_PATH = sys.argv[1]
# except IndexError as e:
# print HELP_MESSAGE
# exit(-1)
cap = cv2.VideoCapture("camera13.h264")
if not cap.isOpened():
print "Fatal Error: Could not open the specified file."
exit(-1)
ret, frame = cap.read()
if not ret:
print "Fatal Error: Could not read/decode frames from specified file."
exit(-1)
def dummy():
pass
def save_image():
filename = "image_%0.5f.png" % t.time()
cv2.imwrite(filename, frame)
def seek_callback(x):
global frame
i = cv2.getTrackbarPos(POS_TRACKBAR, WIN_NAME)
cap.set(CURRENT_FRAME_FLAG, i-1)
_, frame = cap.read()
cv2.imshow(WIN_NAME, frame)
def mouse_callback(event,x,y,flags,param):
if event == cv2.EVENT_LBUTTONDBLCLK:
save_image()
def skip_frame_generator(df):
def skip_frame():
global frame
cf = cap.get(CURRENT_FRAME_FLAG) - 1
cap.set(CURRENT_FRAME_FLAG, cf+df)
cv2.setTrackbarPos(POS_TRACKBAR, WIN_NAME, int(cap.get(CURRENT_FRAME_FLAG)))
_, frame = cap.read()
return skip_frame
cv2.namedWindow(WIN_NAME)
cv2.createTrackbar(POS_TRACKBAR, WIN_NAME, 0, int(cap.get(TOTAL_FRAMES_FLAG)), seek_callback)
cv2.setMouseCallback(WIN_NAME, mouse_callback)
actions = dict()
actions[ord("D")] = skip_frame_generator(10)
actions[ord("d")] = skip_frame_generator(1)
actions[ord("a")] = skip_frame_generator(-1)
actions[ord("A")] = skip_frame_generator(-10)
actions[ord("q")] = lambda: exit(0)
actions[ord("s")] = save_image
while True:
cv2.imshow(WIN_NAME, frame)
key = cv2.waitKey(0) & 0xFF
actions.get(key, dummy)()``
When the code is executed, I receive the error - OverflowError: Python int too large to convert to C long. How may I go about solving this please? This code is meant to create a frame grabber that allows me to pick a frame that I want and save it.

Well, the error is telling you that you are passing a number in this line:
cv2.createTrackbar(POS_TRACKBAR, WIN_NAME, 0,int(cap.get(TOTAL_FRAMES_FLAG)), seek_callback)
That is too large to be cast to ctypes.c_long; The maximum number that can be represented by this type is (2**31)-1 = 2147483647.
This is presumably your TOTAL_FRAMES_FLAG. Now that's a lot of frames (approx. 500 days #50fps), so it can't be right.
I notice that you are getting this value before you open the video file, so it's likely to be undefined at that point. Try doing it after you've opened your file, and see if that fixes things. You could print the value, as well, to see if it is indeed too large (before you try the suggested change).