We Keep Coding

Google Nest Hello Streaming URL Not Working

I am using the following API by Google, https://developers.google.com/nest/device-access/traits/device/camera-live-stream
I have successfully been able to see a list of my devices and relevant information. I also am able to make a successful GenerateRtspStream request. I receive the following response as documented on their API
{
"results" : {
"streamUrls" : {
"rtsp_url" : "rtsps://someurl.com/CjY5Y3VKaTZwR3o4Y19YbTVfMF...?auth=g.0.streamingToken"
},
"streamExtensionToken" : "CjY5Y3VKaTZwR3o4Y19YbTVfMF...",
"streamToken" : "g.0.streamingToken",
"expiresAt" : "2018-01-04T18:30:00.000Z"
}
}
The problem however is I am unable to access the video feed. I have tried using things like VLC player and Pot Player to view the live feed, but they say that URL does not exist. I have also tried using OpenCV in python to try and access the live-feed as well and it also does not work ( I have tested opencv on local files and they work just fine ).
Am I doing something wrong with rtsps urls? How do I access the live-feed, either in python or some third-party application like VLC Player
Here is some examples of what I have already tried doing:
import cv2 as cv
x = cv.VideoCapture(STREAM_URL)
# ret is False --- it works on local files as it returns True and I am able to view the media
ret, img = x.read()
Here is the attempt using Pot Player/VLC
My goal is to do processing on this video-feed/image in python, so ideally my solution would be using opencv or something along those lines. I was mainly using VLC and other players to debug the issue with this url first.
UPDATE
I have tested using the following public link
rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov
:
MYURL = "rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov"
MYURL = STREAM_URL
import cv2 as cv
x = cv.VideoCapture(MYURL)
while(True):
ret, img = x.read()
if not ret:
print('URL not working')
break
cv.imshow('frame', img)
cv.waitKey(1)
And it works perfectly with opencv as well as Pot Player. So maybe the issue is with the Google Devices Access API? The URL they provide may not be correct? Or am I missing something here?
Maybe it has to do with the rtsps URL vs rtsp? How can I fix that?

Both ffmpeg and ffplay worked fine for me, no rebuild necessary. On MacOS I just did:
brew install ffmpeg
ffplay -rtsp_transport tcp "rtsps://..."
Fill in the huge stream URL. Note the quotes, there was something about the URL without quotes that zsh didn't like. Alternatively to save the stream to a file
ffmpeg -y -loglevel fatal -rtsp_transport tcp -i "rtsps://..." -acodec copy -vcodec copy /path/to/out.mp4
You could use different options with ffmpeg to transform the stream to something other than rtsps for consumption by some other application.
Interestingly, despite the API telling me this:
"maxVideoResolution": {
"width": 640,
"height": 480
},
this is the info from ffplay:
Metadata:
title : SDM
Duration: N/A, start: -0.110000, bitrate: N/A
Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp
Stream #0:1: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1600x1200 [SAR 1:1 DAR 4:3], 15 fps, 15 tbr, 90k tbn, 30 tbc
Indicating 1600x1200, not sure why maxVideoResolution isn't actually the max resolution?

I'd suggest trying with ffmpeg, however you may need to build it from source.
If you're having trouble with ffmpeg, you can modify the ffmpeg source to increase control_uri (in libavformat/rtsp.h) size from 1024 to 2048, and recompile. Then ffmpeg should be able to play the RTSPS streams.

How to write mp4 video file with H264 codec?

On OSX I can record from my webcam and write a video file with the following simple script:
import cv2
camera = cv2.VideoCapture(0)
# Define the codec and create VideoWriter object to save the video
fourcc = cv2.VideoWriter_fourcc(*'XVID')
video_writer = cv2.VideoWriter('output.avi', fourcc, 25.0, (640, 480))
while True:
try:
(grabbed, frame) = camera.read() # grab the current frame
frame = cv2.resize(frame, (640, 480)) # resize the frame
video_writer.write(frame) # Write the video to the file system
except KeyboardInterrupt:
camera.release()
break
The resulting avi file is quite big though. I want a smaller file, preferably an mp4. So I changed the filename to output.mp4 and the fourcc codec to H264. That writes a video file which works, but gives me the following error:
$ python write_video_file.py
OpenCV: FFMPEG: tag 0x34363248/'H264' is not supported with codec id 28 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x00000021/'!???'
Since I thought I'm missing the H264 codec in ffmpeg I decided to uninstall ffmpeg and opencv and reinstall them again with H264 support. For this I used the following commands:
# First ffmpeg
brew install ffmpeg --with-fdk-aac --with-libvidstab --with-openh264 \
--with-openjpeg --with-openssl --with-tools --with-webp --with-x265 --with-zeromq
# then opencv3
brew tap homebrew/science
brew install opencv3 --with-contrib --with-ffmpeg --with-tbb
After this I ran the script again, using the following combinations:
output.mp4 with H264
output.mp4 with X264
Unfortunately I still get the OpenCV warnings/errors. The file is readable, but it still annoys me that I get these errors. Does anybody have any idea how I can make OpenCV write mp4 video file with the H264 codec?
All tips are welcome!

I spent ages trying to find a list of video codecs on macOS, and finding which codecs work with which containers, and then if QuickTime can actually read the resulting files.
I can summarise my findings as follows:
.mov container and fourcc('m','p','4','v') work and QuickTime can read it
.mov container and fourcc('a','v','c','1') work and QuickTime can read it
.avi container and fourcc('F','M','P','4') works but QuickTime cannot read it without conversion
I did manage to write h264 video in an mp4 container, as you wanted, just not using OpenCV's VideoWriter module. Instead I changed the code (mine happens to be C++) to just output raw bgr24 format data - which is how OpenCV likes to store pixels anyway:
So the output of a frame of video stored in a Mat called Frame becomes:
cout.write(reinterpret_cast<const char *>(Frame.data),width*height*3);
and then you pipe the output of your program into ffmpeg like this:
./yourProgram | ffmpeg -y -f rawvideo -pixel_format bgr24 -video_size 1024x768 -i - -c:v h264 -pix_fmt yuv420p video.mp4
Yes, I know I have made some assumptions:
that the data are CV8_UC3,
that the sizes match in OpenCV and ffmpeg, and
that there is no padding or mad stride in the OpenCV data,
but the basic technique works and you can adapt it for other sizes and situations.

FFMPEG reads fps of input h264 file wrong, resulting in wrong duration of output file

I am trying to convert an .h264 file created with python from an incoming stream to xvid format with ffmpeg.
The file is 30min long and 12fps. However, the converted file automatically creates a file that is 25fps and thus 14.4min long. If I set fps like ffmpeg -i test.h264 -r 12 test.avi it creates a video of 14.4min long with a fps of 12.
How can I set it to see the incoming video as 12fps? I tried recording immediately in xvid coded in python using FOURCC but on mac OS X the only codec that seems to work is mp4v. I also tried using MP4Box, which creates the right video duration and fps but for which I cannot set it to the xvid coded (which I need).

The options are the same for input and output. If they are set before the -i, they are applied to input file. After -i, they are applied to output.
Everything is explained in the doc
ffmpeg -r 12 -i inuputAt12fps.h264 -r 25 outputAt25Fps.avi

Unable to write video using python and opencv2 on mac os x

I am using mac os x 1.7.5 with python 2.7.5_1 and opencv 2.4.4_0 installed via macports. I seem to have all the latest dependent ports.
In my code, the cv2.Videowriter() is successfully created and opened which produces a 6kb .avi file but videoFile.write(img0) doesn't write anything into that file. I really am not able to figure out why the video stream isn't written to the file. Any insights?
My code is as follows:
import cv2
import cv
cv2.namedWindow("Original")
cap0 = cv2.VideoCapture(0)
codec = cv.CV_FOURCC('D','I','V','X')
print codec
videoFile = cv2.VideoWriter();
videoFile.open('video.avi', codec, 25, (640, 480),1)
key = -1
while(key < 0):
success0, img0 = cap0.read()
cv2.imshow("Original", img0)
videoFile.write(img0)
key = cv2.waitKey(1)
cv2.destroyAllWindows()
I've tried these codecs and none of them worked: I420, AVC1, YUV1, PIM1, MJPG, MP42, MP4V, DIV3, DIVX, XVID, IUYV,FFV1, FLV1, U263, H264, ZLIB
I've also gone through all the quick time codecs mentioned here
Using ZLIB codec I get the error:
[zlib # 0x7fb0d130a000] Specified pixel format yuv420p is invalid or not supported
Using H264 codec I get an error:
[libx264 # 0x7fe423869600] broken ffmpeg default settings detected
[libx264 # 0x7fe423869600] use an encoding preset (e.g. -vpre medium)
[libx264 # 0x7fe423869600] preset usage: -vpre <speed> -vpre <profile>
[libx264 # 0x7fe423869600] speed presets are listed in x264 --help
[libx264 # 0x7fe423869600] profile is optional; x264 defaults to high
I didn't understand what the above errors meant. I tried reinstalling ffmpeg to the latest version (1.2.2_0+gpl2) but my script still doesn't work. All the other codecs did not give any error.
I've even tried the file extensions of .mpg and .mkv with the above codecs. Sometimes I would get an error saying that the codec was not suitable for the file extension but when I didn't get error I would simply get the unreadable video file of a minuscule size.
Any help is much appreciated.
ps: I've already gone through the following SO questions which didn't solve my issue:
using opencv2 write streaming video in python
Writing video with OpenCV + Python + Mac
Create an avi video with opencv and python on a mac
Python OpenCV, can't write a video (.avi) to file
Creating AVI files in OpenCV
read/write avi video on MAC using openCV
Python OpenCV 2.4 writes half-complete PNG video frames

The problem seems to have been with the image size in the function videoFile.open('video.avi', codec, 25, (640, 480),1)
So I updated my script to include
size = (int(cap0.get(cv2.cv.CV_CAP_PROP_FRAME_WIDTH)),
int(cap0.get(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT)))
And then accordingly changed videoFile.open to
videoFile.open('video.avi', codec, 25, size,1)
Then my script started working.
I've tried the codecs with fourcc: IYUV, I420, PIM1, MJPG, FFV1 and DIVX with file extension .avi
Each of the above codec worked with frame rates 16, 20, 25 and 30 except for PIM1 which seems to work only on 20fps and above.
Also,
Codec with fourcc THEO worked with file extension .ogv
XVID worked properly with file extension .mkv and though the .mkv container should work with any encoding I got a variety of weird results with other codecs.
FLV1 with file extension .flv didn't work. It gave the error:
[flv # 0x7f8414006000] Tag FLV1/0x31564c46 incompatible with output codec id '22' ([2][0][0][0])
FLV4 with file extension .flv didn't give an error but opencv's videowrite outputted an error "Could not update video file"
Of the codecs that worked with .avi file container, DIVX produced the smallest video file (~4Mb for 4 sec video) and IYUV produced the largest file (~160Mb for 4 sec video)
Note:
fps = videoCapture.get(cv2.cv.CV_CAP_PROP_FPS) always returns 0.0 when capturing from webcam. It is a bug in OpenCV2.4.3 and OpenCV2.4.4
I also found out that ffmpeg cannot grab images from Mac's iSight and FacetimeHD webcams the way it can from Windows and Ubuntu because designers at Apple have prohibited easy access to the mac's camera... what a shame!
References:
http://en.wikipedia.org/wiki/Comparison_of_container_formats

Initially I was getting a 0kb video file. I changed the Codec from MJPG to iYUV. It worked for me. Python 2.7 and openCV 2.4.5.
cap = cv2.VideoCapture(0)
fourCC = cv2.cv.CV_FOURCC('i','Y','U', 'V'); # Important to notice cv2.cv.CV_FOURCC
size = (int(cap.get(cv2.cv.CV_CAP_PROP_FRAME_WIDTH)), int(cap.get(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT)))
out = cv2.VideoWriter("Test.avi", fourCC, 15.0, size, True)

To get h264 working, install ffmpeg from the following link. Its quite a cumbersome installation but it ll get the encoder up and running.
https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu

OpenCV + python -- grab frames from a video file

I can't seem to capture frames from a file using OpenCV -- I've compiled from source on Ubuntu with all the necessary prereqs according to: http://opencv.willowgarage.com/wiki/InstallGuide%20%3A%20Debian
#!/usr/bin/env python
import cv
import sys
files = sys.argv[1:]
for f in files:
capture = cv.CaptureFromFile(f)
print capture
print cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_WIDTH)
print cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_HEIGHT)
for i in xrange(10000):
frame = cv.QueryFrame(capture)
if frame:
print frame
Output:
ubuntu#local:~/opencv$ ./test.py bbb.avi
<Capture 0xa37b130>
0.0
0.0
The frames are always None...
I've transcoded a video file to i420 format using:
mencoder $1 -nosound -ovc raw -vf format=i420 -o $2
Any ideas?

You don't have the gstreamer-ffmpeg or gsteamer-python or gsteamer-python-devel packages installed. I installed all three of them. and the exact same problem was resolved.

I'm using OpenCV 2.2.0, compiled on Ubuntu from source. I can confirm that the source code you provided works as expected. So the problem is somewhere else.
I couldn't reproduce your problem using mencoder (installing it is a bit of a problem on my machine) so I used ffmpeg to wrap a raw video in the AVI container:
ffmpeg -s cif -i ~/local/sample-video/foreman.yuv -vcodec copy foreman.avi
(foreman.yuv is a standard CIF image sequence you can find on the net if you look around).
Running the AVI from ffmpeg through your source gives this:
misha#misha-desktop:~/Desktop/stackoverflow$ python ocv_video.py foreman.avi
<Capture 0xa71120>
352.0
288.0
<iplimage(nChannels=3 width=352 height=288 widthStep=1056 )>
<iplimage(nChannels=3 width=352 height=288 widthStep=1056 )>
...
So things work as expected. What you should check:
Do you get any errors on standard output/standard error? OpenCV uses ffmpeg libraries to read video files, so be on the lookout for informative messages. Here's what happens if you try to play a RAW video file without a container (sounds similar to your problem):
error:
misha#misha-desktop:~/Desktop/stackoverflow$ python ocv_video.py foreman.yuv
[IMGUTILS # 0x7fff37c8d040] Picture size 0x0 is invalid
[IMGUTILS # 0x7fff37c8cf20] Picture size 0x0 is invalid
[rawvideo # 0x19e65c0] Could not find codec parameters (Video: rawvideo, yuv420p)
[rawvideo # 0x19e65c0] Estimating duration from bitrate, this may be inaccurate
GStreamer Plugin: Embedded video playback halted; module decodebin20 reported: Your GStreamer installation is missing a plug-in.
<Capture 0x19e3130>
0.0
0.0
Make sure your AVI file actually contains the required information to play back the video. At a minimum, this should be the frame dimensions. RAW video typically doesn't contain any information besides the actual pixel data, so knowing the frame dimensions and FPS is required. You can wrong-guess the FPS and still get a viewable video, but if you get the dimensions wrong, the video will be unviewable.
Make sure the AVI file you're trying to open is actually playable. Try ffplay file.avi -- if that fails, then the problem is likely to be with the file. Try using ffmpeg to transcode instead of mencoder.
Make sure you can play other videos, using the same method as above. If you can't, then it's likely that your ffmpeg install is broken.