After reading this thread:
streaming m3u8 file with opencv
I've implemented this code to stream a live camera:
import m3u8
import cv2
import subprocess as sp
import numpy as np
VIDEO_URL = r'https://5e0da72d486c5.streamlock.net:8443/ayalon/HaHalacha.stream/playlist.m3u8'
FFMPEG_BIN = r"C:\ffmpeg-n4.4-latest-win64-lgpl-4.4\bin\ffmpeg.exe"
# cv2.namedWindow("GoPro", cv2.CV_WINDOW_AUTOSIZE)
pipe = sp.Popen([FFMPEG_BIN, "-i", VIDEO_URL,
"-loglevel", "quiet", # no text output
"-an", # disable audio
"-f", "image2pipe",
"-pix_fmt", "bgr24",
"-vcodec", "rawvideo", "-"],
stdin=sp.PIPE, stdout=sp.PIPE)
while True:
raw_image = pipe.stdout.read(432 * 240 * 3) # read 432*240*3 bytes (= 1 frame)
image = np.frombuffer(raw_image) # np.fromstring(raw_image, dtype='uint8').reshape((240, 432, 3))
cv2.imshow("Stream", image)
if cv2.waitKey(5) == 27:
break
cv2.destroyAllWindows()
But the image i see is just white noise.
What am i missing?
Thx
Just so you know, when you go on this website to download the lib :
https://github.com/BtbN/FFmpeg-Builds/releases
The ones who have shared in their names means they're already built
The ones without mean that you have to build them yourselves
try using prebuilt one
Related
I have a HLS stream and wanted to extract frame as it appears for computer vision using opencv in python. I have tried exploring ffmpeg but it seems to read .mp4 easily but not the hls stream (m3u8). Is there any other option or other API of ffmpeg-python to extract frame from HLS stream.
Here is the sample code which I thought of giving a try but is not working with VIDEO_URL
from imutils.video import VideoStream
import ffmpeg
import cv2
import numpy as np
import subprocess as sp
VIDEO_URL = "https://bitdash-a.akamaihd.net/content/sintel/hls/playlist.m3u8"
VIDEO_FILE = "sampleStream.mp4"
process = (
ffmpeg
.input(VIDEO_FILE)
#.input(VIDEO_URL)
.output('pipe:', format='rawvideo', pix_fmt='rgb24')
.run_async(pipe_stdout=True)
)
tar = 200
val = 0
while True:
val = val + 1
in_bytes = process1.stdout.read(100 * 200 * 3)
if not in_bytes:
print('Breaking - No bytes found.')
break
in_frame = (
np
.frombuffer(in_bytes, np.uint8)
.reshape([100, 200, 3])
)
if val == tar:
print('Writing image...')
cv2.imwrite("sample.jpg", in_frame)
break
You can use cv2.VideoCapture() to read the HLS stream directly. Why suffer?
import cv2
import sys
VIDEO_URL = "http://bitdash-a.akamaihd.net/content/sintel/hls/playlist.m3u8"
cap = cv2.VideoCapture(VIDEO_URL)
if (cap.isOpened() == False):
print('!!! Unable to open URL')
sys.exit(-1)
# retrieve FPS and calculate how long to wait between each frame to be display
fps = cap.get(cv2.CAP_PROP_FPS)
wait_ms = int(1000/fps)
print('FPS:', fps)
while(True):
# read one frame
ret, frame = cap.read()
# TODO: perform frame processing here
# display frame
cv2.imshow('frame',frame)
if cv2.waitKey(wait_ms) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
Output:
#karlphillip solution works but it's important to note that you first need to install a build of cv2 that includes ffmpeg.
You can check if ffmpeg is setup correctly for cv2 by putting the following in your code:
print(cv2.getBuildInformation())
It should say "YES" after ffmpeg.
Installation for Conda users (uninstall opencv first if it was already installed):
conda install -c conda-forge ffmpeg
conda install -c conda-forge opencv
Installation for None Conda users:
Apparently it's a bit of a process to install cv2 with ffmpeg manually so I opted to use a docker image. I found this one and it works well https://github.com/Borda/docker_python-opencv-ffmpeg
My Dockerfile extends the above base image like this:
FROM borda/docker_python-opencv-ffmpeg:cpu-py3.7-cv4.5.1
# More docker file stuff...
Given its link, I'd like to capture an online video (say from YouTube) for further processing without downloading it on the disk. What I mean by this is that I'd like to load it directly to memory whenever possible. According to these links:
http://answers.opencv.org/question/24012/reading-video-stream-from-ip-camera-in-opencv-java/#24013
http://answers.opencv.org/question/24154/how-to-using-opencv-api-get-web-video-stream/#24156
http://answers.opencv.org/question/133/how-do-i-access-an-ip-camera/
https://pypi.org/project/pafy/
it should be doable. My attempt looks like this:
import cv2
import pafy
vid = pafy.new("https://www.youtube.com/watch?v=QuELiw8tbx8")
vid_cap = cv2.VideoCapture()
vid_cap.open(vid.getbest(preftype="webm").url)
However it fails with an error
(python:12925): GLib-GObject-CRITICAL **: 14:48:56.168: g_object_set: assertion 'G_IS_OBJECT (object)' failed
False
How can I achieve my goal using python?
You can achieve this by using youtube-dl and ffmpeg:
Install the latest version of youtube-dl.
Then do sudo pip install --upgrade youtube_dl
Build ffmpeg with HTTPS support. You can do this by turning on the --enable-gnutls option.
Once the installations are complete, it's time to test the youtube-dl in terminal. We'll be using this youtube video for testing.
First we get the list of formats available for this video:
youtube-dl --list-formats https://www.youtube.com/watch?v=HECa3bAFAYk
Select a format code of your choice. I want the 144p resolution so I select 160.
Next we get the video url for our format of choice by:
youtube-dl --format 160 --get-url https://www.youtube.com/watch?v=HECa3bAFAYk
https://r3---sn-4g5e6nz7.googlevideo.com/videoplayback?clen=184077&aitags=133%2C134%2C160%2C242%2C243%2C278&fvip=3&requiressl=yes&signature=5D21FFD906226C7680B26ACEF996B78B6A31F7C9.31B1115DB13F096AA5968DB2838E22A0D6A2EDCB&source=youtube&mn=sn-4g5e6nz7%2Csn-h0jeen7y&xtags=tx%3D9486108&itag=160&mime=video%2Fmp4&mt=1529091799&ms=au%2Conr&ei=XxckW-73GNCogQfqrryQAg&expire=1529113535&mm=31%2C26&c=WEB&keepalive=yes&id=o-AJExEG49WtIUkrF7OikaaGBCfKntDl75xCoO5_9cL-eP&ip=95.91.202.147&sparams=aitags%2Cclen%2Cdur%2Cei%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cxtags%2Cexpire&key=yt6&lmt=1526699176943888&dur=25.375&pl=22&gir=yes&mv=m&initcwndbps=1155000&ipbits=0&ratebypass=yes
Finally we can play this video url in either ffplay or vlc. But instead of copying and pasting, we can do this in one command:
ffplay -i $(youtube-dl --format 160 --get-url https://www.youtube.com/watch?v=HECa3bAFAYk)
Now that we have confirmed that youtube-dl and ffmpeg works, we can write a Python script to process the frames in OpenCV. See this link for more Python options.
import cv2
import numpy as np
import youtube_dl
if __name__ == '__main__':
video_url = 'https://www.youtube.com/watch?v=HECa3bAFAYkq'
ydl_opts = {}
# create youtube-dl object
ydl = youtube_dl.YoutubeDL(ydl_opts)
# set video url, extract video information
info_dict = ydl.extract_info(video_url, download=False)
# get video formats available
formats = info_dict.get('formats',None)
for f in formats:
# I want the lowest resolution, so I set resolution as 144p
if f.get('format_note',None) == '144p':
#get the video url
url = f.get('url',None)
# open url with opencv
cap = cv2.VideoCapture(url)
# check if url was opened
if not cap.isOpened():
print('video not opened')
exit(-1)
while True:
# read frame
ret, frame = cap.read()
# check if frame is empty
if not ret:
break
# display frame
cv2.imshow('frame', frame)
if cv2.waitKey(30)&0xFF == ord('q'):
break
# release VideoCapture
cap.release()
cv2.destroyAllWindows()
First of all Update youtube-dl using the command pip install -U youtube-dl
Then use my VidGear Python Library, then automates the pipelining of YouTube Video using its URL address only. Here's a complete python example:
For VidGear v0.1.9 below:
# import libraries
from vidgear.gears import CamGear
import cv2
stream = CamGear(source='https://youtu.be/dQw4w9WgXcQ', y_tube = True, logging=True).start() # YouTube Video URL as input
# infinite loop
while True:
frame = stream.read()
# read frames
# check if frame is None
if frame is None:
#if True break the infinite loop
break
# do something with frame here
cv2.imshow("Output Frame", frame)
# Show output window
key = cv2.waitKey(1) & 0xFF
# check for 'q' key-press
if key == ord("q"):
#if 'q' key-pressed break out
break
cv2.destroyAllWindows()
# close output window
# safely close video stream.
stream.stop()
For VidGear v0.2.0 and above: (y_tube changed to stream_mode)
# import libraries
from vidgear.gears import CamGear
import cv2
stream = CamGear(source='https://youtu.be/dQw4w9WgXcQ', stream_mode = True, logging=True).start() # YouTube Video URL as input
# infinite loop
while True:
frame = stream.read()
# read frames
# check if frame is None
if frame is None:
#if True break the infinite loop
break
# do something with frame here
cv2.imshow("Output Frame", frame)
# Show output window
key = cv2.waitKey(1) & 0xFF
# check for 'q' key-press
if key == ord("q"):
#if 'q' key-pressed break out
break
cv2.destroyAllWindows()
# close output window
# safely close video stream.
stream.stop()
Code Source
If still get some error, raise an issue here in its GitHub repo.
Using pafy you can have a more elegant solution:
import cv2
import pafy
url = "https://www.youtube.com/watch?v=NKpuX_yzdYs"
video = pafy.new(url)
best = video.getbest(preftype="mp4")
capture = cv2.VideoCapture()
capture.open(best.url)
success,image = capture.read()
while success:
cv2.imshow('frame', image)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
success,image = capture.read()
cv2.destroyAllWindows()
capture.release()
I want to highlight the issue I faced while running was a open-cv version problem, I was using OpenCV 3.4.x and the video feed was exiting before being read into the while loop, so, i upgraded my open cv to "opencv-contrib-python== 4.2.0.34".
Given its link, I'd like to capture an online video (say from YouTube) for further processing without downloading it on the disk. What I mean by this is that I'd like to load it directly to memory whenever possible. According to these links:
http://answers.opencv.org/question/24012/reading-video-stream-from-ip-camera-in-opencv-java/#24013
http://answers.opencv.org/question/24154/how-to-using-opencv-api-get-web-video-stream/#24156
http://answers.opencv.org/question/133/how-do-i-access-an-ip-camera/
https://pypi.org/project/pafy/
it should be doable. My attempt looks like this:
import cv2
import pafy
vid = pafy.new("https://www.youtube.com/watch?v=QuELiw8tbx8")
vid_cap = cv2.VideoCapture()
vid_cap.open(vid.getbest(preftype="webm").url)
However it fails with an error
(python:12925): GLib-GObject-CRITICAL **: 14:48:56.168: g_object_set: assertion 'G_IS_OBJECT (object)' failed
False
How can I achieve my goal using python?
You can achieve this by using youtube-dl and ffmpeg:
Install the latest version of youtube-dl.
Then do sudo pip install --upgrade youtube_dl
Build ffmpeg with HTTPS support. You can do this by turning on the --enable-gnutls option.
Once the installations are complete, it's time to test the youtube-dl in terminal. We'll be using this youtube video for testing.
First we get the list of formats available for this video:
youtube-dl --list-formats https://www.youtube.com/watch?v=HECa3bAFAYk
Select a format code of your choice. I want the 144p resolution so I select 160.
Next we get the video url for our format of choice by:
youtube-dl --format 160 --get-url https://www.youtube.com/watch?v=HECa3bAFAYk
https://r3---sn-4g5e6nz7.googlevideo.com/videoplayback?clen=184077&aitags=133%2C134%2C160%2C242%2C243%2C278&fvip=3&requiressl=yes&signature=5D21FFD906226C7680B26ACEF996B78B6A31F7C9.31B1115DB13F096AA5968DB2838E22A0D6A2EDCB&source=youtube&mn=sn-4g5e6nz7%2Csn-h0jeen7y&xtags=tx%3D9486108&itag=160&mime=video%2Fmp4&mt=1529091799&ms=au%2Conr&ei=XxckW-73GNCogQfqrryQAg&expire=1529113535&mm=31%2C26&c=WEB&keepalive=yes&id=o-AJExEG49WtIUkrF7OikaaGBCfKntDl75xCoO5_9cL-eP&ip=95.91.202.147&sparams=aitags%2Cclen%2Cdur%2Cei%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cxtags%2Cexpire&key=yt6&lmt=1526699176943888&dur=25.375&pl=22&gir=yes&mv=m&initcwndbps=1155000&ipbits=0&ratebypass=yes
Finally we can play this video url in either ffplay or vlc. But instead of copying and pasting, we can do this in one command:
ffplay -i $(youtube-dl --format 160 --get-url https://www.youtube.com/watch?v=HECa3bAFAYk)
Now that we have confirmed that youtube-dl and ffmpeg works, we can write a Python script to process the frames in OpenCV. See this link for more Python options.
import cv2
import numpy as np
import youtube_dl
if __name__ == '__main__':
video_url = 'https://www.youtube.com/watch?v=HECa3bAFAYkq'
ydl_opts = {}
# create youtube-dl object
ydl = youtube_dl.YoutubeDL(ydl_opts)
# set video url, extract video information
info_dict = ydl.extract_info(video_url, download=False)
# get video formats available
formats = info_dict.get('formats',None)
for f in formats:
# I want the lowest resolution, so I set resolution as 144p
if f.get('format_note',None) == '144p':
#get the video url
url = f.get('url',None)
# open url with opencv
cap = cv2.VideoCapture(url)
# check if url was opened
if not cap.isOpened():
print('video not opened')
exit(-1)
while True:
# read frame
ret, frame = cap.read()
# check if frame is empty
if not ret:
break
# display frame
cv2.imshow('frame', frame)
if cv2.waitKey(30)&0xFF == ord('q'):
break
# release VideoCapture
cap.release()
cv2.destroyAllWindows()
First of all Update youtube-dl using the command pip install -U youtube-dl
Then use my VidGear Python Library, then automates the pipelining of YouTube Video using its URL address only. Here's a complete python example:
For VidGear v0.1.9 below:
# import libraries
from vidgear.gears import CamGear
import cv2
stream = CamGear(source='https://youtu.be/dQw4w9WgXcQ', y_tube = True, logging=True).start() # YouTube Video URL as input
# infinite loop
while True:
frame = stream.read()
# read frames
# check if frame is None
if frame is None:
#if True break the infinite loop
break
# do something with frame here
cv2.imshow("Output Frame", frame)
# Show output window
key = cv2.waitKey(1) & 0xFF
# check for 'q' key-press
if key == ord("q"):
#if 'q' key-pressed break out
break
cv2.destroyAllWindows()
# close output window
# safely close video stream.
stream.stop()
For VidGear v0.2.0 and above: (y_tube changed to stream_mode)
# import libraries
from vidgear.gears import CamGear
import cv2
stream = CamGear(source='https://youtu.be/dQw4w9WgXcQ', stream_mode = True, logging=True).start() # YouTube Video URL as input
# infinite loop
while True:
frame = stream.read()
# read frames
# check if frame is None
if frame is None:
#if True break the infinite loop
break
# do something with frame here
cv2.imshow("Output Frame", frame)
# Show output window
key = cv2.waitKey(1) & 0xFF
# check for 'q' key-press
if key == ord("q"):
#if 'q' key-pressed break out
break
cv2.destroyAllWindows()
# close output window
# safely close video stream.
stream.stop()
Code Source
If still get some error, raise an issue here in its GitHub repo.
Using pafy you can have a more elegant solution:
import cv2
import pafy
url = "https://www.youtube.com/watch?v=NKpuX_yzdYs"
video = pafy.new(url)
best = video.getbest(preftype="mp4")
capture = cv2.VideoCapture()
capture.open(best.url)
success,image = capture.read()
while success:
cv2.imshow('frame', image)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
success,image = capture.read()
cv2.destroyAllWindows()
capture.release()
I want to highlight the issue I faced while running was a open-cv version problem, I was using OpenCV 3.4.x and the video feed was exiting before being read into the while loop, so, i upgraded my open cv to "opencv-contrib-python== 4.2.0.34".
I have a working python script that uses the video writer from opencv.
source https://gist.github.com/stanchiang/b4e4890160a054a9c1d65f9152172600
If i take in a file, and regardless of whether I simply pass the video frame through to the writer (effectively duplicating the file) or if i try to edit the frame, the file is always larger. I would like for it to be no larger than the original (since if you read my script i'm blurring a lot of stuff).
After checking their metadata, with ffprobe -v quiet -print_format json -show_format -show_streams inputFile.mp4 I notice that the bitrate of the new file is over 5.5x higher than before.
source https://www.diffchecker.com/8r2syeln
since bitrate is a big determinant of file size, I'm wondering if
i can hardcode the desired bitrate of the new file through the video writer
whether for some reason the heavily increased bit rate is needed
basically this answer https://stackoverflow.com/a/13298538/1079379
# import packages
from PIL import Image
from subprocess import Popen, PIPE
from imutils.video import VideoStream
from imutils.object_detection import non_max_suppression
from imutils import paths
import cv2
import numpy as np
import imutils
# ffmpeg setup
p = Popen(['ffmpeg', '-y', '-f', 'image2pipe', '-vcodec', 'mjpeg', '-r', '24', '-i', '-', '-vcodec', 'h264', '-qscale', '5', '-r', '24', 'video.mp4'], stdin=PIPE)
video = cv2.VideoCapture('videos.mp4')
while True:
ret, frame = video.read()
if ret:
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
im = Image.fromarray(frame)
im.save(p.stdin, 'JPEG')
else:
break
p.stdin.close()
p.wait()
video.release()
cv2.destroyAllWindows()
My VidGear Python Library's WriteGear API automates the process of pipelining OpenCV frames into FFmpeg on any platform and at the same time provides same opencv-python syntax. Here's a basic python example:
# import libraries
from vidgear.gears import WriteGear
import cv2
output_params = {"-vcodec":"libx264", "-crf": 0, "-preset": "fast"} #define (Codec,CRF,preset) FFmpeg tweak parameters for writer
stream = cv2.VideoCapture(0) #Open live webcam video stream on first index(i.e. 0) device
writer = WriteGear(output_filename = 'Output.mp4', compression_mode = True, logging = True, **output_params) #Define writer with output filename 'Output.mp4'
# infinite loop
while True:
(grabbed, frame) = stream.read()
# read frames
# check if frame empty
if not is grabbed:
#if True break the infinite loop
break
# {do something with frame here}
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
# write a modified frame to writer
writer.write(gray)
# Show output window
cv2.imshow("Output Frame", frame)
key = cv2.waitKey(1) & 0xFF
# check for 'q' key-press
if key == ord("q"):
#if 'q' key-pressed break out
break
cv2.destroyAllWindows()
# close output window
stream.release()
# safely close video stream
writer.close()
# safely close writer
Source:https://abhitronix.github.io/vidgear/latest/gears/writegear/compression/usage/#using-compression-mode-with-opencv
You can check out VidGear Docs for more advanced applications and features.
I have a Windows 7 SP1 64 Bit machine with open cv2.4.9 installed and python 2.7.6 installed.
I use pre compiled version of opencv
The following code works perfectly for me
import cv2.cv as cv
import time
cv.NamedWindow("camera", 0)
capture = cv.CaptureFromCAM(0)
while True:
img = cv.QueryFrame(capture)
cv.ShowImage("camera", img)
if cv.WaitKey(10) == 27:
break
cv.DestroyAllWindows()
Now when I try to use this code
import cv2
import numpy as np
cam = cv2.VideoCapture(0)
s, img = cam.read()
winName = "Movement Indicator"
cv2.namedWindow(winName, cv2.CV_WINDOW_AUTOSIZE)
while s:
cv2.imshow( winName,img )
s, img = cam.read()
key = cv2.waitKey(10)
if key == 27:
cv2.destroyWindow(winName)
break
print "Goodbye"
The window is opened , the camera is initialized (as camera lights are on) , but nothing is displayed and the window closes and the program exits.
WHERE am I going wrong??
QUESTION 2
Can any one also suggest me how to capture live video stream from my Linux machine 192.168.1.3 . The stream is being generated by ffmpeg.
The video stream can be opened in web browser. But I want to capture it with opencv and python.