We Keep Coding

Google Nest Hello Streaming URL Not Working

I am using the following API by Google, https://developers.google.com/nest/device-access/traits/device/camera-live-stream
I have successfully been able to see a list of my devices and relevant information. I also am able to make a successful GenerateRtspStream request. I receive the following response as documented on their API
{
"results" : {
"streamUrls" : {
"rtsp_url" : "rtsps://someurl.com/CjY5Y3VKaTZwR3o4Y19YbTVfMF...?auth=g.0.streamingToken"
},
"streamExtensionToken" : "CjY5Y3VKaTZwR3o4Y19YbTVfMF...",
"streamToken" : "g.0.streamingToken",
"expiresAt" : "2018-01-04T18:30:00.000Z"
}
}
The problem however is I am unable to access the video feed. I have tried using things like VLC player and Pot Player to view the live feed, but they say that URL does not exist. I have also tried using OpenCV in python to try and access the live-feed as well and it also does not work ( I have tested opencv on local files and they work just fine ).
Am I doing something wrong with rtsps urls? How do I access the live-feed, either in python or some third-party application like VLC Player
Here is some examples of what I have already tried doing:
import cv2 as cv
x = cv.VideoCapture(STREAM_URL)
# ret is False --- it works on local files as it returns True and I am able to view the media
ret, img = x.read()
Here is the attempt using Pot Player/VLC
My goal is to do processing on this video-feed/image in python, so ideally my solution would be using opencv or something along those lines. I was mainly using VLC and other players to debug the issue with this url first.
UPDATE
I have tested using the following public link
rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov
:
MYURL = "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov"
MYURL = STREAM_URL
import cv2 as cv
x = cv.VideoCapture(MYURL)
while(True):
ret, img = x.read()
if not ret:
print('URL not working')
break
cv.imshow('frame', img)
cv.waitKey(1)
And it works perfectly with opencv as well as Pot Player. So maybe the issue is with the Google Devices Access API? The URL they provide may not be correct? Or am I missing something here?
Maybe it has to do with the rtsps URL vs rtsp? How can I fix that?

Both ffmpeg and ffplay worked fine for me, no rebuild necessary. On MacOS I just did:
brew install ffmpeg
ffplay -rtsp_transport tcp "rtsps://..."
Fill in the huge stream URL. Note the quotes, there was something about the URL without quotes that zsh didn't like. Alternatively to save the stream to a file
ffmpeg -y -loglevel fatal -rtsp_transport tcp -i "rtsps://..." -acodec copy -vcodec copy /path/to/out.mp4
You could use different options with ffmpeg to transform the stream to something other than rtsps for consumption by some other application.
Interestingly, despite the API telling me this:
"maxVideoResolution": {
"width": 640,
"height": 480
},
this is the info from ffplay:
Metadata:
title : SDM
Duration: N/A, start: -0.110000, bitrate: N/A
Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp
Stream #0:1: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1600x1200 [SAR 1:1 DAR 4:3], 15 fps, 15 tbr, 90k tbn, 30 tbc
Indicating 1600x1200, not sure why maxVideoResolution isn't actually the max resolution?

I'd suggest trying with ffmpeg, however you may need to build it from source.
If you're having trouble with ffmpeg, you can modify the ffmpeg source to increase control_uri (in libavformat/rtsp.h) size from 1024 to 2048, and recompile. Then ffmpeg should be able to play the RTSPS streams.

Read Frames from RTSP Stream in Python

I have recently set up a Raspberry Pi camera and am streaming the frames over RTSP. While it may not be completely necessary, here is the command I am using the broadcast the video:
raspivid -o - -t 0 -w 1280 -h 800 |cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:8554/output.h264}' :demux=h264
This streams the video perfectly.
What I would now like to do is parse this stream with Python and read each frame individually. I would like to do some motion detection for surveillance purposes.
I am completely lost on where to start on this task. Can anyone point me to a good tutorial? If this is not achievable via Python, what tools/languages can I use to accomplish this?

Using the same method listed by "depu" worked perfectly for me.
I just replaced "video file" with "RTSP URL" of actual camera.
Example below worked on AXIS IP Camera.
(This was not working for a while in previous versions of OpenCV)
Works on OpenCV 3.4.1 Windows 10)
import cv2
cap = cv2.VideoCapture("rtsp://root:pass#192.168.0.91:554/axis-media/media.amp")
while(cap.isOpened()):
ret, frame = cap.read()
cv2.imshow('frame', frame)
if cv2.waitKey(20) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()

Bit of a hacky solution, but you can use the VLC python bindings (you can install it with pip install python-vlc) and play the stream:
import vlc
player=vlc.MediaPlayer('rtsp://:8554/output.h264')
player.play()
Then take a snapshot every second or so:
while 1:
time.sleep(1)
player.video_take_snapshot(0, '.snapshot.tmp.png', 0, 0)
And then you can use SimpleCV or something for processing (just load the image file '.snapshot.tmp.png' into your processing library).

use opencv
video=cv2.VideoCapture("rtsp url")
and then you can capture framse. read openCV documentation visit: https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_gui/py_video_display/py_video_display.html

Depending on the stream type, you can probably take a look at this project for some ideas.
https://code.google.com/p/python-mjpeg-over-rtsp-client/
If you want to be mega-pro, you could use something like http://opencv.org/ (Python modules available I believe) for handling the motion detection.

Here is yet one more option.
It's much more complicated than the other answers.
But this way, with just one connection to the camera, you could "fork" the same stream simultaneously to several multiprocesses, to the screen, recast it into multicast, write it to disk, etc.
Of course, just in the case you would need something like that (otherwise you'd prefer the earlier answers)
Let's create two independent python programs:
Server program (rtsp connection, decoding) server.py
Client program (reads frames from shared memory) client.py
Server must be started before the client, i.e.
python3 server.py
And then in another terminal:
python3 client.py
Here is the code:
(1) server.py
import time
from valkka.core import *
# YUV => RGB interpolation to the small size is done each 1000 milliseconds and passed on to the shmem ringbuffer
image_interval=1000
# define rgb image dimensions
width =1920//4
height =1080//4
# posix shared memory: identification tag and size of the ring buffer
shmem_name ="cam_example"
shmem_buffers =10
shmem_filter =RGBShmemFrameFilter(shmem_name, shmem_buffers, width, height)
sws_filter =SwScaleFrameFilter("sws_filter", width, height, shmem_filter)
interval_filter =TimeIntervalFrameFilter("interval_filter", image_interval, sws_filter)
avthread =AVThread("avthread",interval_filter)
av_in_filter =avthread.getFrameFilter()
livethread =LiveThread("livethread")
ctx =LiveConnectionContext(LiveConnectionType_rtsp, "rtsp://user:password#192.168.x.x", 1, av_in_filter)
avthread.startCall()
livethread.startCall()
avthread.decodingOnCall()
livethread.registerStreamCall(ctx)
livethread.playStreamCall(ctx)
# all those threads are written in cpp and they are running in the
# background. Sleep for 20 seconds - or do something else while
# the cpp threads are running and streaming video
time.sleep(20)
# stop threads
livethread.stopCall()
avthread.stopCall()
print("bye")
(2) client.py
import cv2
from valkka.api2 import ShmemRGBClient
width =1920//4
height =1080//4
# This identifies posix shared memory - must be same as in the server side
shmem_name ="cam_example"
# Size of the shmem ringbuffer - must be same as in the server side
shmem_buffers =10
client=ShmemRGBClient(
name =shmem_name,
n_ringbuffer =shmem_buffers,
width =width,
height =height,
mstimeout =1000, # client timeouts if nothing has been received in 1000 milliseconds
verbose =False
)
while True:
index, isize = client.pull()
if (index==None):
print("timeout")
else:
data =client.shmem_list[index][0:isize]
img =data.reshape((height,width,3))
img =cv2.GaussianBlur(img, (21, 21), 0)
cv2.imshow("valkka_opencv_demo",img)
cv2.waitKey(1)
If you got interested, check out some more in https://elsampsa.github.io/valkka-examples/

Hi reading frames from video can be achieved using python and OpenCV . Below is the sample code. Works fine with python and opencv2 version.
import cv2
import os
#Below code will capture the video frames and will sve it a folder (in current working directory)
dirname = 'myfolder'
#video path
cap = cv2.VideoCapture("your rtsp url")
count = 0
while(cap.isOpened()):
ret, frame = cap.read()
if not ret:
break
else:
cv2.imshow('frame', frame)
#The received "frame" will be saved. Or you can manipulate "frame" as per your needs.
name = "rec_frame"+str(count)+".jpg"
cv2.imwrite(os.path.join(dirname,name), frame)
count += 1
if cv2.waitKey(20) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()

Use in this
cv2.VideoCapture("rtsp://username:password#IPAddress:PortNO(rest of the link after the IPAdress)").

Making a video with opencv and ffmpeg. How to find the right color format?

I have a webcam video recorder program built with python, opencv and ffmpeg
It works ok except that the color of the video is more blue than the reality. The problem seems to come from color format of images.
It seems that OpenCv is giving BGR images and ffmpeg+libx264 is expecting YUV420p. I've read that YUV420p correspond to YCbCr.
opencv has no conversion from BGR to YCbCr. It only has a conversion to YCrCb.
I have made some searchs and tried different alternatives to try converting opencv image to something that could be ok for ffmpeg+libx264. None is working. At this point, I am a bit lost and I would appreciate any pointer that could help me to fix this color issue.

You are right, the default pixel format of OpenCV is BGR.
The equivalent format on the ffmpeg side would be BGR24, so you don't need to convert it to YUV420p if you don't want to.
This post shows how to use a python application to capture frames from the webcam and write the frames to stdout. The purpose is to invoke this app on the cmd-line and pipe the result directly to the ffmpeg application, which stores the frames on the disk. Quite clever indeed!
capture.py:
import cv, sys
cap = cv.CaptureFromCAM(0)
if not cap:
sys.stdout.write("failed CaptureFromCAM")
while True :
if not cv.GrabFrame(cap) :
break
frame = cv.RetrieveFrame(cap)
sys.stdout.write( frame.tostring() )
And the command to be executed on the shell is:
python capture.py | ffmpeg -f rawvideo -pix_fmt bgr24 -s 640x480 -r 30 -i - -an -f avi -r 30 foo.avi
Where:
-r gives the frame rate coming off the camera
-an says "don't encode audio"
I tested this solution on my Mac OS X with OpenCV 2.4.2.
EDIT:
In case you haven't tried to record from the camera and use OpenCV to write the video to an mp4 file on the disk, here we go:
import cv, sys
cap = cv.CaptureFromCAM(0) # 0 is for /dev/video0
if not cap:
sys.stdout.write("!!! Failed CaptureFromCAM")
sys.exit(1)
frame = cv.RetrieveFrame(cap)
if not frame:
sys.stdout.write("!!! Failed to retrieve first frame")
sys.exit(1)
# Unfortunately, the following instruction returns 0
#fps = cv.GetCaptureProperty(cap, cv.CV_CAP_PROP_FPS)
fps = 25.0 # so we need to hardcode the FPS
print "Recording at: ", fps, " fps"
frame_size = cv.GetSize(frame)
print "Video size: ", frame_size
writer = cv.CreateVideoWriter("out.mp4", cv.CV_FOURCC('F', 'M', 'P', '4'), fps, frame_size, True)
if not writer:
sys.stdout.write("!!! Error in creating video writer")
sys.exit(1)
while True :
if not cv.GrabFrame(cap) :
break
frame = cv.RetrieveFrame(cap)
cv.WriteFrame(writer, frame)
cv.ReleaseVideoWriter(writer)
cv.ReleaseCapture(cap)
I've tested this with Python 2.7 on Mac OS X and OpenCV 2.4.2.

Have you tried switching the Cb/Cr channels in OpenCV using split and merge ?

Checked the conversion formulas present in: http://en.wikipedia.org/wiki/YCbCr?

The libx264 codec is able to process BGR images. No need to use any conversion to YCbCr. NO need to give a spcific pix_ftm to ffmpeg. I was using RGB and it was causing the blueish effect on the video.
The solution was simply to use the original image retuned by the camera without any conversion. :)
I tried this in my previous investigation and it was crashing the app. The solution is to copy the frame returned by the camera.
frame = opencv.QueryFrame(camera)
if not frame:
return None, None
# RGB : use this one for displaying on the screen
im_rgb = opencv.CreateImage(self.size, opencv.IPL_DEPTH_8U, 3)
opencv.CvtColor(frame, im_rgb, opencv.CV_BGR2RGB)
# BGR : Use this one for the video
im_bgr = opencv.CreateImage(self.size, opencv.IPL_DEPTH_8U, 3)
opencv.Copy(frame, im_bgr)
return im_rgb, im_bgr

I've already answered this here. But my VidGear Python Library automates the whole process of pipelining OpenCV frames into FFmpeg and also robustly handles the format conversion. Here's a basic python example:
# import libraries
from vidgear.gears import WriteGear
import cv2
output_params = {"-vcodec":"libx264", "-crf": 0, "-preset": "fast"} #define (Codec,CRF,preset) FFmpeg tweak parameters for writer
stream = cv2.VideoCapture(0) #Open live webcam video stream on first index(i.e. 0) device
writer = WriteGear(output_filename = 'Output.mp4', compression_mode = True, logging = True, **output_params) #Define writer with output filename 'Output.mp4'
# infinite loop
while True:
(grabbed, frame) = stream.read()
# read frames
# check if frame empty
if not is grabbed:
#if True break the infinite loop
break
# {do something with frame here}
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
# write a modified frame to writer
writer.write(gray)
# Show output window
cv2.imshow("Output Frame", frame)
key = cv2.waitKey(1) & 0xFF
# check for 'q' key-press
if key == ord("q"):
#if 'q' key-pressed break out
break
cv2.destroyAllWindows()
# close output window
stream.release()
# safely close video stream
writer.close()
# safely close writer
Source: https://github.com/abhiTronix/vidgear/wiki/Compression-Mode:-FFmpeg#2-writegear-classcompression-mode-with-opencv-directly
You can check out full VidGear Docs for more advanced applications and exciting features.
Hope that helps!

Problems using webcam in python + openCV

I am using the following code to access my webcam using openCV + python...
import cv
cv.NamedWindow('webcam_feed', cv.CV_WINDOW_AUTOSIZE)
cam = cv.CaptureFromCAM(-1)
I am then getting the following error in the console...
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
I was originally using,
cv.CaptureFromCAM(0)
to access the same and had the same issue and used -1 because it is suppose to pick up any webcam.
I also tested to see if Ubuntu recognizes the webcam and it does. I am using v4l2 for the webcam.
{EDIT}
I am using the following code to display a video feed, it seems to only be showing just one image the web cam captured instead of a continuous video feed...
import cv
cv.NamedWindow('webcam_feed', cv.CV_WINDOW_AUTOSIZE)
cam = cv.CaptureFromCAM(-1)
feed = cv.QueryFrame(cam)
cv.ShowImage("webcam_feed", feed)
cv.WaitKey(-1)

WOW, answered my own question in 15 after me posting this. I did some research and the reason for the web cam only grabbing one image is because of the...
cv.WaitKey(-1)
This doesn't allow the contents of the window to refresh. I set the number to 10...
cv.WaitKey(10)
and it worked beautifully. I also tried 100, but saw no difference. I only saw a difference when the number was 1000. I use 1 because seems that it runs the smoothest.
Here is the full code to display a web cam feed
import cv
cv.NamedWindow("webcam", 1)
cam = cv.CaptureFromCAM(-1)
While True:
feed = cv.QueryFrame(cam)
cv.ShowImage("webcam", feed)
cv.WaitKey(1)

I believe you need to put
frame = cv.QueryFrame(cam)
cv.ShowImage("Webcam Feed", frame)
in a loop to continuously update the image shown in the window. That is, the frame from cv.QueryFrame is a static image, not a continuous video.
If you want to be able to exit with a key press, test cv.WaitKey with a small timeout in the loop too.

For me, the command in root
xhost +
save my time, Note to close and open new terminal.
See you.

OpenCV + python -- grab frames from a video file

I can't seem to capture frames from a file using OpenCV -- I've compiled from source on Ubuntu with all the necessary prereqs according to: http://opencv.willowgarage.com/wiki/InstallGuide%20%3A%20Debian
#!/usr/bin/env python
import cv
import sys
files = sys.argv[1:]
for f in files:
capture = cv.CaptureFromFile(f)
print capture
print cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_WIDTH)
print cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_HEIGHT)
for i in xrange(10000):
frame = cv.QueryFrame(capture)
if frame:
print frame
Output:
ubuntu#local:~/opencv$ ./test.py bbb.avi
<Capture 0xa37b130>
0.0
0.0
The frames are always None...
I've transcoded a video file to i420 format using:
mencoder $1 -nosound -ovc raw -vf format=i420 -o $2
Any ideas?

You don't have the gstreamer-ffmpeg or gsteamer-python or gsteamer-python-devel packages installed. I installed all three of them. and the exact same problem was resolved.

I'm using OpenCV 2.2.0, compiled on Ubuntu from source. I can confirm that the source code you provided works as expected. So the problem is somewhere else.
I couldn't reproduce your problem using mencoder (installing it is a bit of a problem on my machine) so I used ffmpeg to wrap a raw video in the AVI container:
ffmpeg -s cif -i ~/local/sample-video/foreman.yuv -vcodec copy foreman.avi
(foreman.yuv is a standard CIF image sequence you can find on the net if you look around).
Running the AVI from ffmpeg through your source gives this:
misha#misha-desktop:~/Desktop/stackoverflow$ python ocv_video.py foreman.avi
<Capture 0xa71120>
352.0
288.0
<iplimage(nChannels=3 width=352 height=288 widthStep=1056 )>
<iplimage(nChannels=3 width=352 height=288 widthStep=1056 )>
...
So things work as expected. What you should check:
Do you get any errors on standard output/standard error? OpenCV uses ffmpeg libraries to read video files, so be on the lookout for informative messages. Here's what happens if you try to play a RAW video file without a container (sounds similar to your problem):
error:
misha#misha-desktop:~/Desktop/stackoverflow$ python ocv_video.py foreman.yuv
[IMGUTILS # 0x7fff37c8d040] Picture size 0x0 is invalid
[IMGUTILS # 0x7fff37c8cf20] Picture size 0x0 is invalid
[rawvideo # 0x19e65c0] Could not find codec parameters (Video: rawvideo, yuv420p)
[rawvideo # 0x19e65c0] Estimating duration from bitrate, this may be inaccurate
GStreamer Plugin: Embedded video playback halted; module decodebin20 reported: Your GStreamer installation is missing a plug-in.
<Capture 0x19e3130>
0.0
0.0
Make sure your AVI file actually contains the required information to play back the video. At a minimum, this should be the frame dimensions. RAW video typically doesn't contain any information besides the actual pixel data, so knowing the frame dimensions and FPS is required. You can wrong-guess the FPS and still get a viewable video, but if you get the dimensions wrong, the video will be unviewable.
Make sure the AVI file you're trying to open is actually playable. Try ffplay file.avi -- if that fails, then the problem is likely to be with the file. Try using ffmpeg to transcode instead of mencoder.
Make sure you can play other videos, using the same method as above. If you can't, then it's likely that your ffmpeg install is broken.