We Keep Coding

Python multiprocessing queue using a lot of resources with opencv

I am using multiprocessing to get frames of a video using Opencv in python.
My class looks like this :-
import cv2
from multiprocessing import Process, Queue
class StreamVideos:
def __init__(self):
self.image_data = Queue()
def start_proces(self):
p = Process(target=self.echo)
p.start()
def echo(self):
cap = cv2.VideoCapture('videoplayback.mp4')
while cap.isOpened():
ret,frame = cap.read()
self.image_data.put(frame)
# print("frame")
I start the process "echo" using :-
p = Process(target=self.echo)
p.start()
the echo function looks like this :-
def echo(self):
cap = cv2.VideoCapture('videoplayback.mp4')
while cap.isOpened():
ret,frame = cap.read()
self.image_data.put(frame)
in which i am using queue where i put these frames
self.image_data.put(frame)
and then in another process I start reviving these frames
self.obj = StreamVideos()
def start_process(self):
self.obj.start_proces()
p = Process(target=self.stream_videos)
p.start()
def stream_videos(self):
while True:
self.img = self.obj.image_data.get()
print(self.img)
but as soon as I start putting frames to queue, the ram gets filled very quickly and the system gets stuck. The video I am using is just 25 fps and 39mb in size, so it does not make any sense.
One thing I noticed is that the "echo" process is putting a lot of frames in the queue before the "stream_videos" process retrives it.
What could be the root of this problem?
Thanks in advance.
Expectations: -
Able to retrieve the frames continuosly.
Tried :-
Not putting frames in queue, in which case the ram is not filled.

The following is a general purpose single producer/multiple consumer implementation. The producer (class StreamVideos) creates a shared memory array whose size is the number of bytes in the video frame. One or more consumers (you specify the number of consumers to StreamVideos) can then call StreamVideos.get_next_frame() to retrieve the next frame. This method converts the shared array back into a numpy array for subsequent processing. The producer will only read the next frame into the shared array after all consumers have called get_next_frame:
#!/usr/bin/env python3
import multiprocessing
import numpy as np
import ctypes
import cv2
class StreamVideos:
def __init__(self, path, n_consumers):
"""
path is the path to the video:
n_consumers is the number of tasks to which we will be sreaming this.
"""
self._path = path
self._event = multiprocessing.Event()
self._barrier = multiprocessing.Barrier(n_consumers + 1, self._reset_event)
# Discover how large a framesize is by getting the first frame
cap = cv2.VideoCapture(self._path)
ret, frame = cap.read()
if ret:
self._shape = frame.shape
frame_size = self._shape[0] * self._shape[1] * self._shape[2]
self._arr = multiprocessing.RawArray(ctypes.c_ubyte, frame_size)
else:
self._arr = None
cap.release()
def _reset_event(self):
self._event.clear()
def start_streaming(self):
cap = cv2.VideoCapture(self._path)
while True:
self._barrier.wait()
ret, frame = cap.read()
if not ret:
# No more readable frames:
break
# Store frame into shared array:
temp = np.frombuffer(self._arr, dtype=frame.dtype)
temp[:] = frame.flatten(order='C')
self._event.set()
cap.release()
self._arr = None
self._event.set()
def get_next_frame(self):
# Tell producer that this consumer is through with the previous frame:
self._barrier.wait()
# Wait for next frame to be read by the producer:
self._event.wait()
if self._arr is None:
return None
# Return shared array as a numpy array:
return np.ctypeslib.as_array(self._arr).reshape(self._shape)
def consumer(producer, id):
frame_name = f'Frame - {id}'
while True:
frame = producer.get_next_frame()
if frame is None:
break
cv2.imshow(frame_name, frame)
cv2.waitKey(1)
cv2.destroyAllWindows()
def main():
producer = StreamVideos('videoplayback.mp4', 2)
consumer1 = multiprocessing.Process(target=consumer, args=(producer, 1))
consumer1.start()
consumer2 = multiprocessing.Process(target=consumer, args=(producer, 2))
consumer2.start()
"""
# Run as a child process:
producer_process = multiprocessing.Process(target=producer.start_streaming)
producer_process.start()
producer_process.join()
"""
# Run in main process:
producer.start_streaming()
consumer1.join()
consumer2.join()
if __name__ == '__main__':
main()

How to measure cpu usage (as a percentage) of a single code line in Python?

I need to know how much this particular line of code loads my cpu when whole program is executed:
cap.set(cv.CAP_PROP_POS_FRAMES,random_frame)
This line is part of a certain program. But I'm only interested in how this particular line loads the cpu. I don't know exactly how to measure it.
To be more precised, this is my full code:
import cv2 as cv
import random
cap = cv.VideoCapture('file_name.avi')
random_frame = random.randint(1,99999)
cap.set(cv.CAP_PROP_POS_FRAMES,random_frame)
ret, frame = cap.read()
cv.imshow("random_frame",frame)
while cap.isOpened():
if cv.waitKey(1) == ord('q'):
break
cap.release()
cv.destroyAllWindows()
Windows 7 operating system

import multiprocessing as mp
import psutil
import random
import cv2 as cv
import random
cap = cv.VideoCapture ('file_name.avi')
random_frame = random.randint (1, 99999)
def monitorFunction():
cap.set (cv.CAP_PROP_POS_FRAMES,random_frame)
def monitor (target):
worker_process = mp.Process(target=target)
worker_process.start()
p = psutil.Process(worker_process.pid)
# log cpu usage of `worker_process` every 10 ms
cpu_percents = []
while worker_process.is_alive():
cpu_percents.append(p.cpu_percent())
time.sleep(0.01)
worker_process.join()
return cpu_percents
cpu_percents = monitor(target=monitorFunction)
print (cpu_percents)
This should work.

Constant camera grabbing with OpenCV & Python multiprocessing - Follow up

This is a followup on this where #Aaron helped quite a lot.
Context:
OpenCV, Python, a USB Webcam and multiprocessing - Grabbing all the time from the camera on a subprocess and returning a single frame upon request to the main process.
#Aaron did most of the heavylifting, of not all :). What I am missing is how to turn this into a process that gets a single frame upon request.
Help Please?
import numpy as np
import cv2
from multiprocessing import Process, Queue
from multiprocessing.shared_memory import SharedMemory
def produce_frames(q):
#get the first frame to calculate size of buffer
cap = cv2.VideoCapture(0)
success, frame = cap.read()
shm = SharedMemory(create=True, size=frame.nbytes)
framebuffer = np.ndarray(frame.shape, frame.dtype, buffer=shm.buf) #could also maybe use array.array instead of numpy, but I'm familiar with numpy
framebuffer[:] = frame #in case you need to send the first frame to the main process
q.put(shm) #send the buffer back to main
q.put(frame.shape) #send the array details
q.put(frame.dtype)
try:
while True:
cap.read(framebuffer)
except KeyboardInterrupt:
pass
finally:
shm.close() #call this in all processes where the shm exists
shm.unlink() #call this in at least one process
def consume_frames(q):
shm = q.get() #get the shared buffer
shape = q.get()
dtype = q.get()
framebuffer = np.ndarray(shape, dtype, buffer=shm.buf) #reconstruct the array
try:
while True:
cv2.imshow("window title", framebuffer)
cv2.waitKey(100)
except KeyboardInterrupt:
pass
finally:
shm.close()
if __name__ == "__main__":
q = Queue()
producer = Process(target=produce_frames, args=(q,))
producer.start()
consume_frames(q)

how to convert pytorch model( ) output to something cv2.imshow fit

Recently, I want to show tello stream with image detection. My first thought is to save model’s output to my local with save() method, and show it by cv2.imshow() method. It works but the stream with objects detection will have a delay about 4~5 second.
My code:
from threading import Thread
from djitellopy import Tello
import cv2, math, time
import torch
import os
import numpy as np
import asyncio
import imutils
from PIL import Image
path = r'C:\yolov5-master'
model = torch.hub.load(path, 'yolov5s',source='local', pretrained=True)
tello = Tello()
tello.connect()
tello.streamon()
frame_read = tello.get_frame_read()
class VideoStreamWidget(object):
def __init__(self, src=0):
# Start the thread to read frames from the video stream
self.thread = Thread(target=self.update, args=())
self.thread.daemon = True
self.thread.start()
def update(self):
# Read the next frame from the stream
global frame
while True:
self.frame = cv2.cvtColor(frame_read.frame,cv2.COLOR_RGB2BGR)
time.sleep(.01)
def show_frame(self):
# Display frames in main program
wee = model(self.frame)
arr = wee.datah().cpu().numpy()
img = Image.fromarray.fromarray(arr, 'RGB')
result = cv2.cvtColor(img,cv2.COLOR_RGB2BGR)
cv2.imshow('frame', result)
key = cv2.waitKey(1)
if __name__ == '__main__':
video_stream_widget = VideoStreamWidget()
time.sleep(1)
while True:
try:
video_stream_widget.show_frame()
except AttributeError:
pass
I'm wondering what data type is the output of model( ).
And I tried:
wee = model(self.frame)
print( type( wee ) )
output:
<class 'models.common.Detections'>
How can I convert this kind of data to the thing fit cv2.imshow( ) method? Or is there any way to show a real-time stream with object detection without delay?
Appreciate.

Subscribing to Image Topic but doesn't output images

I have the following code that I've been struggling to get images from. There's a topic "CompressedImage" that has jpeg images from a camera in my robot. Below is my attempt. Terminal just gives me the versions from print and exits out.
#!/usr/bin/env python
import cv2
import numpy as np
from timeit import default_timer as timer
from std_msgs.msg import Float64
from sensor_msgs.msg import Image, CompressedImage
from cv_bridge import CvBridge, CvBridgeError
import rospy
import sys
print(sys.version)
print(cv2.__version__)
height = 480
width = 640
global_frame = np.zeros((height,width,3), np.uint8)
def camera_callback(data):
bridge = CvBridge()
try:
global_frame = bridge.compressed_imgmsg_to_cv2(data)
except CvBridgeError as e:
print(e)
height, width, channels = global_frame.shape
# print(height)
cv2.imshow("Original", global_frame)
def lane_pose_publisher():
# Set the node name
rospy.init_node('lane_pose_publisher', anonymous=True)
rospy.Subscriber('/camera/image_raw/compressed', CompressedImage, camera_callback, queue_size = 1)
# set rate
rate = rospy.Rate(1000) # 1000hz
if __name__ == '__main__':
try:
lane_pose_publisher()
except rospy.ROSInterruptException:
pass

This is because you are not spinning ROS. Check out the corresponding ROS tutorial to create a subscriber where
# spin() simply keeps python from exiting until this node is stopped
rospy.spin()
is called to ensure ROS can do his job (calling callbacks, ...).
So you should replace your lines
# set rate
rate = rospy.Rate(1000) # 1000hz
with the spinning ones.
Also you should add
cv2.waitKey(1)
after calling
cv2.imshow("Original", global_frame)
to ensure the image gets displayed.