my goal is to re-stream local video content / desktop screencasting, to an UDP flow that I need to process on a Python script.
Here is the FFMPEG script that I'm using:
ffmpeg -re -i C:\Users\test\Downloads\out.ts -strict -2 -c:v copy -an -preset slower -tune stillimage -b 11200k -f rawvideo udp://127.0.0.1:5000
And here is the simple Python script supposed to read the stream flow:
import cv2
cap = cv2.VideoCapture('udp://127.0.0.1:5000',cv2.CAP_FFMPEG)
if not cap.isOpened():
print('VideoCapture not opened')
exit(-1)
width = cap.get(cv2.CAP_PROP_FRAME_WIDTH) # float
height = cap.get(cv2.CAP_PROP_FRAME_HEIGHT) # float
print(str(width))
print(str(height))
while True:
ret, frame = cap.read()
imgray = frame[int(round((height/100)*70,0)):int(round((width/100)*42,0)), int(round((height/100)*74,0)):int(round((width/100)*54,0))]
if not ret:
print('frame empty')
break
cv2.imshow('image', imgray)
if cv2.waitKey(1)&0XFF == ord('q'):
break
cap.release()
I'm able to visualize portion of the stream video as expect, but I'm facing lot of issue in video quality degradation, specially video artifact probably due missing packet processing:
Also these are error log I'm geting from script:
[h264 # 0000026eb272f280] error while decoding MB 105 66, bytestream -21
[h264 # 0000026eb2fcb740] error while decoding MB 100 53, bytestream -11
[h264 # 0000026eb272f280] error while decoding MB 32 22, bytestream -11
[h264 # 0000026ead9ee300] error while decoding MB 60 20, bytestream -25
[h264 # 0000026eb27f00c0] error while decoding MB 9 62, bytestream -5
[h264 # 0000026ead9ee780] error while decoding MB 85 44, bytestream -5
[h264 # 0000026eb27f0800] error while decoding MB 64 25, bytestream -15
[h264 # 0000026eb272f280] error while decoding MB 112 23, bytestream -17
[h264 # 0000026eb2735200] error while decoding MB 30 21, bytestream -7
Actually I don't care about video fluidity,I can also reduce the FPS, important thing is the video quality. Not sure if I'm doing wrong on the scripting python part or if I'm using wrong FFMPEG command.
Many Thanks
For video quality you will have to use multi-threading. It'll fix it up. I will give you an example that you can work on.
import threading
import queue
import cv2
q = queue.Queue()
def receive():
cap = cv2.VideoCapture('udp://#127.0.0.1:5000?buffer_size=65535&pkt_size=65535&fifo_size=65535')
ret, frame = cap.read()
q.put(frame)
while ret:
ret, frame = cap.read()
q.put(frame)
def display():
while True:
if q.empty() != True:
frame = q.get()
cv2.imshow('Video', frame)
k = cv2.waitKey(1) & 0xff
if k == 27: # press 'ESC' to quit
break
tr = threading.Thread(target=receive, daemon=True)
td = threading.Thread(target=display)
tr.start()
td.start()
td.join()
Related
MWE
import cv2
FPS = 30
KEY_ESC = 27
OUTPUT_FILE = "vid.mp4"
cam = cv2.VideoCapture(0)
codec = cv2.VideoWriter.fourcc(*"mp4v") # MPEG-4 http://mp4ra.org/#/codecs
frame_size = cam.read()[1].shape[:2]
video_writer = cv2.VideoWriter(OUTPUT_FILE, codec, FPS, frame_size)
# record until user exits with ESC
while True:
success, image = cam.read()
cv2.imshow("window", image)
video_writer.write(image)
if cv2.waitKey(5) == KEY_ESC:
break
cam.release()
video_writer.release()
Problem
Video does not play.
Firefox reports "No video with supported format and MIME type found.".
VLC reports "cannot find any /moov/trak" "No steams found".
The problem is that np.ndarray.shape, although not properly documented, returns (rows, columns) for a 2d array, which corresponds to (height, width). VideoWriter(frameSize) although not properly documented, seems to expect (width, height).
You can correct this with:
frame_size = tuple(reversed(cam.read()[1].shape[:2]))
But, I recommend creating the VideoWriter using the VideoCapture properties as follows:
output_file = "vid.mp4"
codec = cv2.VideoWriter.fourcc(*"mp4v")
fps = cam.get(cv2.CAP_PROP_FPS)
frame_width = cam.get(cv2.CAP_PROP_FRAME_WIDTH)
frame_height = cam.get(cv2.CAP_PROP_FRAME_HEIGHT)
frame_size = (int(frame_width), int(frame_height))
video_writer = cv2.VideoWriter(output_file, codec, fps, frame_size)
Sources:
VideoWriter()
VideoCapture.get()
https://answers.opencv.org/question/66545/problems-with-the-video-writer-in-opencv-300/
Related:
OpenCV - Video doesn't play after saving in Python
OpenCV VideoWriter: Issues with playing video
Trying to run object detection algorithms in jetson nano (ubuntu 18.4) and Gstreamer cannot open the video files because of this warning in the Gstreamer. I have Gstreamer 1.14.5 version. (I tried .avi, .mp4 and .MOV).
It also appeared these warnings:
"Gstreamer warning:unable to start pipeline" and
"Gstreamer warning: Embedded video playback halted; module avidemux0 reported: Internal data stream error."
The code:
if __name__ == '__main__':
args = arg_parse()
confidence = float(args.confidence)
start = 0
CUDA = torch.cuda.is_available()
num_classes = 80
CUDA = torch.cuda.is_available()
videofile = args.video
# start the file video stream thread and allow the buffer to
# start to fill
print("[INFO] starting video file thread...")
fvs = FileVideoStream(videofile).start()
time.sleep(1.0)
# start the FPS timer
fps = FPS().start()
# loop over frames from the video file stream
while fvs.more():
try:
# grab the frame from the threaded video file stream, resize
# it, and convert it to grayscale (while still retaining 3
# channels)
frame = fvs.read()
frame = imutils.resize(frame, width=800)
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
frame = np.dstack([frame, frame, frame])
fps.update()
# stop the timer and display FPS information
fps.stop()
print()
print("[INFO] elasped time: {:.2f}".format(fps.elapsed()))
print("[INFO] approx. FPS: {:.2f}".format(fps.fps()))
print("[INFO] Accuracy: {:.2f}%".format(np.clip((accuracy/(frames+1))*100,1,100)))
PIL_image = Image.fromarray(np.uint8(frame)).convert('RGB')
original_image = PIL_image
# original_image = original_image.convert('RGB')
ret = detect(original_image, min_score=confidence, max_overlap=0.1, top_k=200)
ret = np.asarray(ret)
cv2.imshow("frame", ret)
frames+=1
key = cv2.waitKey(1)
if key & 0xFF == ord('q'):
break
continue
except:
pass
How will i fix it?
Thanks.
I want to read an rtsp video source, add overlay text and push it to the RTMP endpoint.I am using Videocapture to read the video source and python subprocess to write the frames back to RTMP endpoint. I referred this FFmpeg stream video to rtmp from frames OpenCV python
import sys
import subprocess
import cv2
import ffmpeg
rtmp_url = "rtmp://127.0.0.1:1935/live/test"
path = 0
cap = cv2.VideoCapture("rtsp://10.0.1.7/media.sdp")
# gather video info to ffmpeg
fps = int(cap.get(cv2.CAP_PROP_FPS))
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
command = ['ffmpeg', '-i', '-', "-c", "copy", '-f', 'flv', rtmp_url]
p = subprocess.Popen(command, stdin=subprocess.PIPE, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
font = cv2.FONT_HERSHEY_SIMPLEX
while cap.isOpened():
ret, frame = cap.read()
cv2.putText(frame, 'TEXT ON VIDEO', (50, 50), font, 1, (0, 255, 255), 2, cv2.LINE_4)
cv2.imshow('video', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
if not ret:
print("frame read failed")
break
try:
p.stdin.write(frame.tobytes())
except Exception as e:
print (e)
cap.release()
p.stdin.close()
p.stderr.close()
p.wait()
The python script returns "[Errno 32] Broken pipe". Running the ffmpeg command in the terminal works fine.
ffmpeg -i rtsp://10.0.1.7/media.sdp -c copy -f flv
rtmp://127.0.0.1:1935/live/test
The above command works fine, and I can push the input stream to RTMP endpoint. But I can't write processed frame to subprocess which has ffmpeg running.
Please let me know, if I miss anything.
You can't use "-c", "copy" when writing raw frames to stdin PIPE.
The frame returned by ret, frame = cap.read() is a uint8 NumPy array in BGR color format (cap.read() decodes the video and converts the color format).
In FFmpeg terminology, the frame format is "rawvideo".
The command should tell FFmpeg to expect raw video as input, with specific size and pixel format:
command = ['ffmpeg', '-f', 'rawvideo', '-s', f'{width}x{height}', '-pixel_format', 'bgr24', ...
Because the input is raw video, we have to re-encode it.
We may specify the encoded pixel format and video codec:
'-pix_fmt', 'yuv420p', '-c:v', 'libx264' ....
Remarks:
Decoding and re-encoding the video looses some quality (but there is no choice).
The suggested solution looses the audio (there are solutions that preserves the audio, but OpenCV lack audio support).
The posted solution reuses some code from the following post.
Few FFmpeg arguments are taken with no explanation (like '-bufsize', '64M').
Executing a listener application:
The RTMP streaming, is not going to work, without a "listener" that receives the video.
The listener should be started before starting the RTMP streaming (due to TCP usage).
We may use FFplay sub-process as "listener" application:
ffplay_process = sp.Popen(['ffplay', '-listen', '1', '-i', rtmp_url])
Streaming synthetic video frames:
Start with a simpler code sample, that streams synthetic frames (without capturing RTSP video).
The following "self contained" code sample writes yellow text on gray background, and passes the frames to FFmpeg for RTMP streaming:
import cv2
import numpy as np
import subprocess as sp
width = 320
height = 240
fps = 5
rtmp_url = "rtmp://127.0.0.1:1935/live/test"
# Start the TCP server first, before the sending client.
ffplay_process = sp.Popen(['ffplay', '-listen', '1', '-i', rtmp_url]) # Use FFplay sub-process for receiving the RTMP video.
command = ['ffmpeg',
'-re',
'-f', 'rawvideo', # Apply raw video as input
'-s', f'{width}x{height}',
'-pixel_format', 'bgr24',
'-r', f'{fps}',
'-i', '-',
'-pix_fmt', 'yuv420p',
'-c:v', 'libx264',
'-bufsize', '64M',
'-maxrate', '4M',
'-f', 'flv',
rtmp_url]
process = sp.Popen(command, stdin=sp.PIPE) # Execute FFmpeg sub-process for RTSP streaming
frame_counter = 0;
while True:
# Build sythetic frame in BGR color format (3D NumPy array).
frame = np.full((height, width, 3), 60, np.uint8)
cv2.putText(frame, 'TEXT ON VIDEO ' + str(frame_counter), (20, 50), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 255), 2, cv2.LINE_4) # Put a frame counter for showing progress.
process.stdin.write(frame.tobytes()) # Write raw frame to stdin pipe.
cv2.imshow('frame', frame) # Show frame for testing
key = cv2.waitKey(int(round(1000/fps))) # We need to call cv2.waitKey after cv2.imshow
if key == ord('q'): # Press 'q' for exit
break
frame_counter += 1
process.stdin.close() # Close stdin pipe
process.wait() # Wait for FFmpeg sub-process to finish
ffplay_process.kill() # Forcefully close FFplay sub-process
cv2.destroyAllWindows() # Close OpenCV window
Output sample:
Capturing video frames from RTSP stream.
The following code sample captures video frames from a public RTSP stream, writes text, and passes the frames to FFmpeg for RTMP streaming:
import cv2
import numpy as np
import subprocess as sp
# Use public RTSP Streaming for testing.
rtsp_stream = "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov"
rtmp_url = "rtmp://127.0.0.1:1935/live/test"
cap = cv2.VideoCapture(rtsp_stream)
# gather video info to ffmpeg
fps = int(cap.get(cv2.CAP_PROP_FPS))
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
# Start the TCP server first, before the sending client.
ffplay_process = sp.Popen(['ffplay', '-listen', '1', '-i', rtmp_url]) # Use FFplay sub-process for receiving the RTMP video.
command = ['ffmpeg',
'-re',
'-f', 'rawvideo', # Apply raw video as input
'-s', f'{width}x{height}',
'-pixel_format', 'bgr24',
'-r', f'{fps}',
'-i', '-',
'-pix_fmt', 'yuv420p',
'-c:v', 'libx264',
'-bufsize', '64M',
'-maxrate', '4M',
'-f', 'flv',
rtmp_url]
process = sp.Popen(command, stdin=sp.PIPE) # Execute FFmpeg sub-process for RTSP streaming
frame_counter = 0;
while cap.isOpened():
# Read frame from RTSP stream.
ret, frame = cap.read()
if not ret:
print("frame read failed")
break
cv2.putText(frame, 'TEXT ON VIDEO', (0, 50), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 255), 2, cv2.LINE_4)
cv2.putText(frame, str(frame_counter), (100, 100), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 255), 2, cv2.LINE_4)
process.stdin.write(frame.tobytes()) # Write raw frame to stdin pipe.
cv2.imshow('frame', frame) # Show frame for testing
key = cv2.waitKey(1) # We need to call cv2.waitKey after cv2.imshow
if key == ord('q'): # Press 'q' for exit
break
frame_counter += 1
cap.release()
process.stdin.close() # Close stdin pipe
process.wait() # Wait for FFmpeg sub-process to finish
ffplay_process.kill() # Forcefully close FFplay sub-process
cv2.destroyAllWindows() # Close OpenCV window
Output sample:
I am trying to save a video in OpenCV but i keep getting the error "could not demultiplex stream". I then checked size and found out that it was in kB. I primarily want to save grayscale videos how do i make it possible?
Is there any specific codec i need to use?
mplayer gives the following output
MPlayer 1.1-4.8 (C) 2000-2012 MPlayer Team
mplayer: could not connect to socket
mplayer: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.
Playing output.avi.
libavformat version 54.20.4 (external)
Mismatching header version 54.20.3
AVI file format detected.
[aviheader] Video stream found, -vid 0
AVI: Missing video stream!? Contact the author, it may be a bug :(
libavformat file format detected.
[lavf] stream 0: video (mpeg4), -vid 0
VIDEO: [MP4V] 1280x720 24bpp -nan fps 0.0 kbps ( 0.0 kbyte/s)
Clip info:
encoder: Lavf54.20.4
Load subtitles in ./
Failed to open VDPAU backend libvdpau_nouveau.so: cannot open shared object file: No such file or directory
[vdpau] Error when calling vdp_device_create_x11: 1
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
libavcodec version 54.35.1 (external)
Mismatching header version 54.35.0
Unsupported AVPixelFormat 53
Selected video codec: [ffodivx] vfm: ffmpeg (FFmpeg MPEG-4)
==========================================================================
Audio: no sound
Starting playback...
V: 0.0 0/ 0 ??% ??% ??,?% 0 0
Exiting... (End of file)
Right now i tried with multiple codec formats
import imutils
import cv2
import numpy as np
interval = 30
outfilename = 'output.avi'
threshold=100.
fps = 10
cap = cv2.VideoCapture("video.mp4")
ret, frame = cap.read()
height, width, nchannels = frame.shape
fourcc = cv2.cv.CV_FOURCC(*'DIVX')
out = cv2.VideoWriter( outfilename,fourcc, fps, (width,height))
ret, frame = cap.read()
frame = imutils.resize(frame, width=500)
frame = cv2.cvtColor(frame, cv2.COLOR_RGB2GRAY)
while(True):
frame0 = frame
ret, frame = cap.read()
frame = imutils.resize(frame, width=500)
frame = cv2.cvtColor(frame, cv2.COLOR_RGB2GRAY)
if not ret:
deletedcount +=1
break
if np.sum( np.absolute(frame-frame0) )/np.size(frame) > threshold:
out.write(frame)
else:
print "Deleted"
cv2.imshow('Feed - Press "q" to exit',frame)
key = cv2.waitKey(interval) & 0xFF
if key == ord('q'):
print('received key q' )
break
cap.release()
out.release()
print('Successfully completed')
import numpy as np
import cv2
cap = cv2.VideoCapture(0)
out = cv2.VideoWriter('output.avi',-1, 20.0, (640,480))
while(cap.isOpened()):
ret, frame = cap.read()
if ret:
gray = cv2.cvtColor(src=frame, code=cv2.COLOR_BGR2GRAY)
out.write(gray)
cv2.imshow('frame', gray)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
else:
break
cap.release()
out.release()
cv2.destroyAllWindows()
Try this one
Select Intel iyuv codec.
The out.avi is non working file.
The output.avi is new working file.
If video is not getting saved, possibly the reason may be its capture size which is hardcoded as (640,480).
You can try the below code:
cap = cv2.VideoCapture(0)
fourcc_codec = cv2.VideoWriter_fourcc(*'XVID')
fps = 20.0
capture_size = (int(cap.get(3)), int(cap.get(4)))
out = cv2.VideoWriter("output.avi", fourcc_codec, fps, capture_size)
You can also check if you are passing the correct shape, do it like this:
h, w, _ = frame.shape
size = (w, h)
out = cv2.VideoWriter('video.avi', cv2.VideoWriter_fourcc(*'XVID'), 30, size)
I've written a small python code, which captures video stream from the webcam, and writes it into an output file.
I've put a sleep of 50 ms, and specified a fps of 20.0 in the VideoWriter as following:
#!/usr/bin/python
import cv2
from PIL import Image
import threading
from http.server import BaseHTTPRequestHandler,HTTPServer
from socketserver import ThreadingMixIn
from io import StringIO,BytesIO
import time
import datetime
capture=None
out=None
class CamHandler(BaseHTTPRequestHandler):
def do_GET(self):
if self.path.endswith('.mjpg'):
self.send_response(200)
self.send_header('Content-type','multipart/x-mixed-replace; boundary=--jpgboundary')
self.end_headers()
while True:
try:
rc,img = capture.read()
if not rc:
continue
#Get the timestamp on the frame
timestamp = datetime.datetime.now()
ts = timestamp.strftime("%A %d %B %Y %I:%M:%S%p")
cv2.putText(img, ts, (10, img.shape[0] - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 0, 255), 1)
#Store the frame into the output file
out.write(img)
#Some processing before sending the frame to webserver
imgRGB=cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
jpg = Image.fromarray(imgRGB)
tmpFile = BytesIO()
jpg.save(tmpFile,'JPEG')
self.wfile.write("--jpgboundary".encode())
self.send_header('Content-type','image/jpeg')
self.send_header('Content-length',str(tmpFile.getbuffer().nbytes))
self.end_headers()
jpg.save(self.wfile,'JPEG')
time.sleep(0.05)
except KeyboardInterrupt:
break
return
if self.path.endswith('.html'):
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
self.wfile.write('<html><head></head><body>'.encode())
self.wfile.write('<img src="http://127.0.0.1:8080/cam.mjpg"/>'.encode())
self.wfile.write('</body></html>'.encode())
return
class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):
"""Handle requests in a separate thread."""
def main():
global capture
global out
capture = cv2.VideoCapture(0)
# Define the codec and create VideoWriter object
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter('output.avi',fourcc, 20.0, (640,480))
global img
try:
server = ThreadedHTTPServer(('0.0.0.0', 8080), CamHandler)
print( "server started")
server.serve_forever()
except KeyboardInterrupt:
capture.release()
server.socket.close()
out.release()
cv2.destroyAllWindows()
if __name__ == '__main__':
main()
It works fine, and I'm able to get the video saved. However, the fps as seen in property of the video is 600.0 (30X what I set!!)
$ mediainfo output.avi
General
Complete name : output.avi
Format : AVI
Format/Info : Audio Video Interleave
File size : 4.95 MiB
Duration : 17s 252ms
Overall bit rate : 2 408 Kbps
Writing application : Lavf58.3.100
Video
ID : 0
Format : MPEG-4 Visual
Format profile : Simple#L1
Format settings, BVOP : No
Format settings, QPel : No
Format settings, GMC : No warppoints
Format settings, Matrix : Default (H.263)
Codec ID : XVID
Codec ID/Hint : XviD
Duration : 17s 252ms
Bit rate : 2 290 Kbps
Width : 640 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate : 600.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.012
Stream size : 4.71 MiB (95%)
Writing library : Lavc58.6.103
I'm pretty sure my code looks alright, do let me know if there are any obvious mistakes. In case it matters, I'm using an ubuntu OS, Asus X553M laptop with inbuilt webcam to run the above.
EDIT 1:
I'm using python3, if that matters
EDIT 2:
Using MJPG codec did solve the problem, (thanks #api55) so can someone tell me why XVID would give the incorrect fps?
Is it possible that XVID codec incorrectly writes the fps property, while the video is actually correctly coded in 20 fps?
You getting frames every 50 ms + processing time, but writing them to video with delay 50ms, so they will be played with higher speed with ratio=(50+processing time)/ 50ms.
I have this recipe, it's pretty similar to what Andrey answered. The fps gets pretty close to what's expected.
cap = cv2.VideoCapture(capture) # 0 | filepath
fps = cap.get(cv2.CAP_PROP_FPS)
period = 1000 / fps
while cap.isOpened():
start = time.time()
success, frame = cap.read()
if not success:
if capture:
break # video
continue # cam
# processing steps
cv2.imshow('video (ESC to quit)', frame)
processing_time = (time.time() - start) * 1000
wait_ms = period - processing_time if period > processing_time else period
if cv2.waitKey(int(wait_ms)) & 0xFF == 27:
break
end = (time.time() - start)
print(f'FPS: {1.0 / end:.2f} (expected {fps:.2f})\r', end='')