I have recently set up a Raspberry Pi camera and am streaming the frames over RTSP. While it may not be completely necessary, here is the command I am using the broadcast the video:
raspivid -o - -t 0 -w 1280 -h 800 |cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:8554/output.h264}' :demux=h264
This streams the video perfectly.
What I would now like to do is parse this stream with Python and read each frame individually. I would like to do some motion detection for surveillance purposes.
I am completely lost on where to start on this task. Can anyone point me to a good tutorial? If this is not achievable via Python, what tools/languages can I use to accomplish this?
Using the same method listed by "depu" worked perfectly for me.
I just replaced "video file" with "RTSP URL" of actual camera.
Example below worked on AXIS IP Camera.
(This was not working for a while in previous versions of OpenCV)
Works on OpenCV 3.4.1 Windows 10)
import cv2
cap = cv2.VideoCapture("rtsp://root:pass#192.168.0.91:554/axis-media/media.amp")
while(cap.isOpened()):
ret, frame = cap.read()
cv2.imshow('frame', frame)
if cv2.waitKey(20) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
Bit of a hacky solution, but you can use the VLC python bindings (you can install it with pip install python-vlc) and play the stream:
import vlc
player=vlc.MediaPlayer('rtsp://:8554/output.h264')
player.play()
Then take a snapshot every second or so:
while 1:
time.sleep(1)
player.video_take_snapshot(0, '.snapshot.tmp.png', 0, 0)
And then you can use SimpleCV or something for processing (just load the image file '.snapshot.tmp.png' into your processing library).
use opencv
video=cv2.VideoCapture("rtsp url")
and then you can capture framse. read openCV documentation visit: https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_gui/py_video_display/py_video_display.html
Depending on the stream type, you can probably take a look at this project for some ideas.
https://code.google.com/p/python-mjpeg-over-rtsp-client/
If you want to be mega-pro, you could use something like http://opencv.org/ (Python modules available I believe) for handling the motion detection.
Here is yet one more option.
It's much more complicated than the other answers.
But this way, with just one connection to the camera, you could "fork" the same stream simultaneously to several multiprocesses, to the screen, recast it into multicast, write it to disk, etc.
Of course, just in the case you would need something like that (otherwise you'd prefer the earlier answers)
Let's create two independent python programs:
Server program (rtsp connection, decoding) server.py
Client program (reads frames from shared memory) client.py
Server must be started before the client, i.e.
python3 server.py
And then in another terminal:
python3 client.py
Here is the code:
(1) server.py
import time
from valkka.core import *
# YUV => RGB interpolation to the small size is done each 1000 milliseconds and passed on to the shmem ringbuffer
image_interval=1000
# define rgb image dimensions
width =1920//4
height =1080//4
# posix shared memory: identification tag and size of the ring buffer
shmem_name ="cam_example"
shmem_buffers =10
shmem_filter =RGBShmemFrameFilter(shmem_name, shmem_buffers, width, height)
sws_filter =SwScaleFrameFilter("sws_filter", width, height, shmem_filter)
interval_filter =TimeIntervalFrameFilter("interval_filter", image_interval, sws_filter)
avthread =AVThread("avthread",interval_filter)
av_in_filter =avthread.getFrameFilter()
livethread =LiveThread("livethread")
ctx =LiveConnectionContext(LiveConnectionType_rtsp, "rtsp://user:password#192.168.x.x", 1, av_in_filter)
avthread.startCall()
livethread.startCall()
avthread.decodingOnCall()
livethread.registerStreamCall(ctx)
livethread.playStreamCall(ctx)
# all those threads are written in cpp and they are running in the
# background. Sleep for 20 seconds - or do something else while
# the cpp threads are running and streaming video
time.sleep(20)
# stop threads
livethread.stopCall()
avthread.stopCall()
print("bye")
(2) client.py
import cv2
from valkka.api2 import ShmemRGBClient
width =1920//4
height =1080//4
# This identifies posix shared memory - must be same as in the server side
shmem_name ="cam_example"
# Size of the shmem ringbuffer - must be same as in the server side
shmem_buffers =10
client=ShmemRGBClient(
name =shmem_name,
n_ringbuffer =shmem_buffers,
width =width,
height =height,
mstimeout =1000, # client timeouts if nothing has been received in 1000 milliseconds
verbose =False
)
while True:
index, isize = client.pull()
if (index==None):
print("timeout")
else:
data =client.shmem_list[index][0:isize]
img =data.reshape((height,width,3))
img =cv2.GaussianBlur(img, (21, 21), 0)
cv2.imshow("valkka_opencv_demo",img)
cv2.waitKey(1)
If you got interested, check out some more in https://elsampsa.github.io/valkka-examples/
Hi reading frames from video can be achieved using python and OpenCV . Below is the sample code. Works fine with python and opencv2 version.
import cv2
import os
#Below code will capture the video frames and will sve it a folder (in current working directory)
dirname = 'myfolder'
#video path
cap = cv2.VideoCapture("your rtsp url")
count = 0
while(cap.isOpened()):
ret, frame = cap.read()
if not ret:
break
else:
cv2.imshow('frame', frame)
#The received "frame" will be saved. Or you can manipulate "frame" as per your needs.
name = "rec_frame"+str(count)+".jpg"
cv2.imwrite(os.path.join(dirname,name), frame)
count += 1
if cv2.waitKey(20) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
Use in this
cv2.VideoCapture("rtsp://username:password#IPAddress:PortNO(rest of the link after the IPAdress)").
Related
I'm fighting with Chinese IP camera modules, one of them has chipset IMX322 and has 2MPX. I managed to connect to this camera (there was no doc. how to do it ) but url above seems to work :
rtsp://<local_ip>/user=admin&password=&channel=1&stream=0.sdp
I use standard python loop to get data, but there is some problem and I'm really not sure where, it can be problem with camera (wrong header) or library under Nvidia Jetson TX2 and Ubuntu 16.4.
#!/usr/bin/env python
import sys
from time import sleep
import numpy as np
import cv2
import time
cap = cv2.VideoCapture('rtsp://192.168.1.11/user=admin&password=&channel=0&stream=0.sdp?tpc')
while(True):
ret, frame = cap.read()
if ret:
cv2.imshow("Image", cv2.resize(frame,(800,600)))
else:
print('no video')
#cap.set(cv2.CAP_PROP_POS_FRAMES, 0)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
The problem is that I'm getting unexpected behavior of VideoCapture(), it takes about 2 seconds to generate output frame in imgShow, but before that I'm receiving errors in command line level 2,3 times.
Invalid UE golomb code
Invalid UE golomb code
During extraction of correct frames from camera errors are still showing with frequency 1,2 per second. I tried to find any useful information in google but it looks like 90% of post are related with C++ or FFMPEG.
We're doing a project in school where we need to do basic image processing. Our goal is to use every video frame for the Raspberry Pi and do real time image processing.
We've tried to include raspistill in our python-program but so far nothing has worked. The goal of our project is to design a RC-car which follows a blue/red/whatever coloured line with help from image processing.
We thought it would be a good idea to make a python-program which does all image processing necessary, but we currently struggle with the idea of bringing recorded images into the python program. Is there a way to do this with picamera or should we try a different way?
For anyone curious, this is how our program currently looks
while True:
#camera = picamera.PiCamera()
#camera.capture('image1.jpg')
img = cv2.imread('image1.jpg')
width = img.shape[1]
height = img.shape[0]
height=height-1
for x in range (0,width):
if x>=0 and x<(width//2):
blue = img.item(height,x,0)
green = img.item(height,x,1)
red = img.item(height,x,2)
if red>green and red>blue:
OpenCV already contains functions to process live camera data.
This OpenCV documentation provides a simple example:
import numpy as np
import cv2
cap = cv2.VideoCapture(0)
while(True):
# Capture frame-by-frame
ret, frame = cap.read()
# Our operations on the frame come here
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
# Display the resulting frame
cv2.imshow('frame',gray)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
Of course, you do not want to show the image but all your processing can be done there.
Remember to sleep a few hundred milliseconds so the pi does not overheat that much.
Edit:
"how exactly would I go about it though. I used "img = cv2.imread('image1.jpg')" all the time. What do I need to use instead to get the "img" variable right here? What do I use? And what is ret, for? :)"
ret indicates whether the read was successful. Exit program if not.
The read frame is nothing other than your img = cv2.imread('image1.jpg') so your detection code should work exactly the same.
The only difference is that your image does not need to be saved and reopened. Also for debugging purposes you can save the recorded image, like:
import cv2, time
cap = cv2.VideoCapture(0)
ret, frame = cap.read()
if ret:
cv2.imwrite(time.strftime("%Y%m%d-%H%M%S"), frame)
cap.release()
You can use picamera to acquire images.
To make it "real time", you can acquire data each X milliseconds. You need to set X depending on the power of your hardware (and the complexity of the openCV algorithm).
Here's an example (from http://picamera.readthedocs.io/en/release-1.10/api_camera.html#picamera.camera.PiCamera.capture_continuous) how to acquire 60 images per second using picamera:
import time
import picamera
with picamera.PiCamera() as camera:
camera.start_preview()
try:
for i, filename in enumerate(camera.capture_continuous('image{counter:02d}.jpg')):
print(filename)
time.sleep(1)
if i == 59:
break
finally:
camera.stop_preview()
This has been keeping me busy for a good part of the afternoon and I haven't been able to get it to work but I feel like I'm really close.
I've got openCV set up which takes the videofeed from a webcam. To be able to access this video feed (with openCV overlay) I want to pipe the output of the openCV python script to a VLC stream. I managed to get the stream up and running and can connect to it. VLC resizes to the correct aspect ratio and resolution so it gets some correct data but the image I get is just Jitter;
python opencv.py | cvlc --demux=rawvideo --rawvid-fps=30 --rawvid-width=320 --rawvid-height=240 --rawvid-chroma=RV24 - --sout "#transcode{vcodec=h264,vb=200,fps=30,width=320,height=240}:std{access=http{mime=video/x-flv},mux=ffmpeg{mux=flv},dst=:8081/stream.flv}" &
The output of the script is a constant video feed sent to stdout as follows
from imutils.video import WebcamVideoStream
vs = WebcamVideoStream(src=0)
while True:
frame = vs.read()
sys.stdout.write(frame.tostring())
Above example is a dumbed down version of the script I'm using; Also as seen I'm making use of the imutils library; https://github.com/jrosebr1/imutils
If anyone could give me a nudge in the right direction I would appreciate it greatly. My guess is the stdout.write(frame.tostring()) is not what vlc expects but I haven't been able to figure it out myself.
The following works for me under Python 3
import numpy as np
import sys
import cv2
cap = cv2.VideoCapture(0)
while(cap.isOpened()):
ret, frame = cap.read()
if ret==True:
sys.stdout.buffer.write(frame.tobytes())
else:
break
cap.release()
And the command line (my webcam has a different resolution, and I only display the result, but you did not have problems with that)
python opencv.py | vlc --demux=rawvideo --rawvid-fps=25 --rawvid-width=640 --rawvid-height=480 --rawvid-chroma=RV24 - --sout "#display"
Of course this requires a conversion from BGR to RGB as the former is default in OpenCV.
This worked for me, though I am sending to RTSP stream and not using imutils library:
import numpy as np
import sys
import cv2
input_rtsp = "rtsp://10.10.10.9:8080"
cap = cv2.VideoCapture(input_rtsp)
while(cap.isOpened()):
ret, frame = cap.read()
if ret==True:
sys.stdout.write(frame.tostring())
else:
break
cap.release()
Then in command line:
python opencv.py | cvlc --demux=rawvideo --rawvid-fps=25 --rawvid-width=1280 --rawvid-height=720 --rawvid-chroma=RV24 - --sout "#transcode{vcodec=h264,vb=200,fps=25,width=1280,height=720}:rtp{dst=10.10.10.10,port=8081,sdp=rtsp://10.10.10.10:8081/test.sdp}"
Note that you do not need to convert opencv BGR to RGB.
Well, it seems like my question had been asked many times before and unfortunately, no one replied. I hope someone will help.
I have an Easycap device that converts the analog images from my analog camera to digital signals through a USB port.
The device is identified to the system in the Device Manager under "Sound, Video and game controllers" category as "SMI Grabber Device".
I use a simple Python code to display the video from this device. I also have an embedded webcam in my laptop.
import numpy as np
import cv2
cap = cv2.VideoCapture(0)
while(True):
# Capture frame-by-frame
ret, frame = cap.read()
# Display the resulting frame
cv2.imshow('frame',frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
if cv2.waitKey(1) & 0xFF == ord('s'):
cv2.imwrite('screenshot.jpg',frame)
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
First, when I unplug the Easycap, CaptureVideo(0) returns the embedded webcam video stream. However, when I plug the Easycap, an error appears:
"Traceback (most recent call last):
File "C:\Users\DELL\Desktop\code\cam.py", line 10, in
cv2.imshow('frame',frame)
error: ......\src\opencv\modules\highgui\src\window.cpp:261: error: (-215) size.width>0 && size.height>0"
Notice that, any number except 0 makes the program display the webcam image. So if I tried cap = cv2.CaptureVideo(1), it will show the webcam, cap = cv2.CaptureVideo(20) is the same.
I also tried to enter "SMI Grabber Device" instead of 0 or 1 in the VideoCaptureconstructor function, but it didn't make any difference.
I'm using Windows 8, and I've installed the accompanying driver for Easycap. The software that comes with the driver (called ULead) works fine and display the CCTV camera video. I tried to display the images while I'm closing that program, and without, the result is the same.
I used before a C# program with Aforge library which had getCamList method or something which allowed me to choose the specific device I want to display from a comboBox. I can't find a similar function is opencv.
I'm using OpenCV 2.4.6. I didn't try the code on prior versions.
I really need to understand why this code doesn't work, knowing that I'm just a very beginner of opencv and image processing.
I hope someone can help.
I am using EasyCAP too.
You must check that ret is True.
I am use below code
while True:
ret, frame = vc.read()
if ret:
break
cv2.waitKey(10)
h, w = frame.shape[:2]
print h, w
while True:
ret, frame = vc.read()
if ret:
cv2.imshow(WID, frame)
if cv2.waitKey(1) == 27:
break
Let there be light!
On serious note, I struggled with the same problem and I hope this helps!
the original thread + answer
I have a webcam video recorder program built with python, opencv and ffmpeg
It works ok except that the color of the video is more blue than the reality. The problem seems to come from color format of images.
It seems that OpenCv is giving BGR images and ffmpeg+libx264 is expecting YUV420p. I've read that YUV420p correspond to YCbCr.
opencv has no conversion from BGR to YCbCr. It only has a conversion to YCrCb.
I have made some searchs and tried different alternatives to try converting opencv image to something that could be ok for ffmpeg+libx264. None is working. At this point, I am a bit lost and I would appreciate any pointer that could help me to fix this color issue.
You are right, the default pixel format of OpenCV is BGR.
The equivalent format on the ffmpeg side would be BGR24, so you don't need to convert it to YUV420p if you don't want to.
This post shows how to use a python application to capture frames from the webcam and write the frames to stdout. The purpose is to invoke this app on the cmd-line and pipe the result directly to the ffmpeg application, which stores the frames on the disk. Quite clever indeed!
capture.py:
import cv, sys
cap = cv.CaptureFromCAM(0)
if not cap:
sys.stdout.write("failed CaptureFromCAM")
while True :
if not cv.GrabFrame(cap) :
break
frame = cv.RetrieveFrame(cap)
sys.stdout.write( frame.tostring() )
And the command to be executed on the shell is:
python capture.py | ffmpeg -f rawvideo -pix_fmt bgr24 -s 640x480 -r 30 -i - -an -f avi -r 30 foo.avi
Where:
-r gives the frame rate coming off the camera
-an says "don't encode audio"
I tested this solution on my Mac OS X with OpenCV 2.4.2.
EDIT:
In case you haven't tried to record from the camera and use OpenCV to write the video to an mp4 file on the disk, here we go:
import cv, sys
cap = cv.CaptureFromCAM(0) # 0 is for /dev/video0
if not cap:
sys.stdout.write("!!! Failed CaptureFromCAM")
sys.exit(1)
frame = cv.RetrieveFrame(cap)
if not frame:
sys.stdout.write("!!! Failed to retrieve first frame")
sys.exit(1)
# Unfortunately, the following instruction returns 0
#fps = cv.GetCaptureProperty(cap, cv.CV_CAP_PROP_FPS)
fps = 25.0 # so we need to hardcode the FPS
print "Recording at: ", fps, " fps"
frame_size = cv.GetSize(frame)
print "Video size: ", frame_size
writer = cv.CreateVideoWriter("out.mp4", cv.CV_FOURCC('F', 'M', 'P', '4'), fps, frame_size, True)
if not writer:
sys.stdout.write("!!! Error in creating video writer")
sys.exit(1)
while True :
if not cv.GrabFrame(cap) :
break
frame = cv.RetrieveFrame(cap)
cv.WriteFrame(writer, frame)
cv.ReleaseVideoWriter(writer)
cv.ReleaseCapture(cap)
I've tested this with Python 2.7 on Mac OS X and OpenCV 2.4.2.
Have you tried switching the Cb/Cr channels in OpenCV using split and merge ?
Checked the conversion formulas present in: http://en.wikipedia.org/wiki/YCbCr?
The libx264 codec is able to process BGR images. No need to use any conversion to YCbCr. NO need to give a spcific pix_ftm to ffmpeg. I was using RGB and it was causing the blueish effect on the video.
The solution was simply to use the original image retuned by the camera without any conversion. :)
I tried this in my previous investigation and it was crashing the app. The solution is to copy the frame returned by the camera.
frame = opencv.QueryFrame(camera)
if not frame:
return None, None
# RGB : use this one for displaying on the screen
im_rgb = opencv.CreateImage(self.size, opencv.IPL_DEPTH_8U, 3)
opencv.CvtColor(frame, im_rgb, opencv.CV_BGR2RGB)
# BGR : Use this one for the video
im_bgr = opencv.CreateImage(self.size, opencv.IPL_DEPTH_8U, 3)
opencv.Copy(frame, im_bgr)
return im_rgb, im_bgr
I've already answered this here. But my VidGear Python Library automates the whole process of pipelining OpenCV frames into FFmpeg and also robustly handles the format conversion. Here's a basic python example:
# import libraries
from vidgear.gears import WriteGear
import cv2
output_params = {"-vcodec":"libx264", "-crf": 0, "-preset": "fast"} #define (Codec,CRF,preset) FFmpeg tweak parameters for writer
stream = cv2.VideoCapture(0) #Open live webcam video stream on first index(i.e. 0) device
writer = WriteGear(output_filename = 'Output.mp4', compression_mode = True, logging = True, **output_params) #Define writer with output filename 'Output.mp4'
# infinite loop
while True:
(grabbed, frame) = stream.read()
# read frames
# check if frame empty
if not is grabbed:
#if True break the infinite loop
break
# {do something with frame here}
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
# write a modified frame to writer
writer.write(gray)
# Show output window
cv2.imshow("Output Frame", frame)
key = cv2.waitKey(1) & 0xFF
# check for 'q' key-press
if key == ord("q"):
#if 'q' key-pressed break out
break
cv2.destroyAllWindows()
# close output window
stream.release()
# safely close video stream
writer.close()
# safely close writer
Source: https://github.com/abhiTronix/vidgear/wiki/Compression-Mode:-FFmpeg#2-writegear-classcompression-mode-with-opencv-directly
You can check out full VidGear Docs for more advanced applications and exciting features.
Hope that helps!