Could you please tell me, is there a way to get video streaming into Nifi? I am using Nifi "ExecuteProcess" to run python and OpenCV in order to capture the video stream but I am not able to make multiple flow files. My programs run the python code and then put all the captured frames in a single flow file. As I want to do analysis using YOLOV3 on live videos, I want each frame in a different flow file. The code is given below. My program runs the python code using "ExecuteProcess" and when the program finishes it stores everything that is on the output console to a single flow file but I want multiple flow files.
If there is a better way to get live video streaming using Nifi, please share it.
import cv2
cap = cv2.VideoCapture("videolink.mp4")
while (True):
ret, image = cap.read()
et, jpeg = cv2.imencode('.jpg', image)
jp = jpeg.tobytes()
print(jp)
cap.release()
cv2.destroyAllWindows()
i think the simplest way to make streaming:
1/ setup ListenHTTP processor that will receive video parts through incoming POST requests. and this will become your data ingestion point.
2/ add to your python script http-post method call instead of print like this:
import cv2
import requests
cap = cv2.VideoCapture("videolink.mp4")
while (True):
ret, image = cap.read()
et, jpeg = cv2.imencode('.jpg', image)
jp = jpeg.tobytes()
# PORT and BASE_PATH must be configured in ListenHTTP processor
requests.post('http://localhost:PORT/BASE_PATH', data = jp)
cap.release()
cv2.destroyAllWindows()
Related
I have a small OpenCV script that works with my windows laptop camera but doesn't work when I plug in an external Logitech C922 Pro HD stream webcam.
I have changed the numbers around in cap = cv2.VideoCapture(xxxNUMBERxxx) to open the camera with no luck. I have also installed the Logitech drivers with no luck. The rest of the internet seems to think it's a permissions issue, but I can't find any specific permission settings so I'm not sure that's the issue. I just want to be able to get a video stream from the camera.
The script is below:
import numpy as np
import cv2
cap = cv2.VideoCapture(2, cv2.CAP_DSHOW)
while(True):
# Capture frame-by-frame
ret, frame = cap.read()
# Our operations on the frame come here
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
# Display the resulting frame
cv2.imshow('frame',gray)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
Turns out to be a strange and silly solution. After downloading the 'Logi Capture' app, I had to keep the app running with the video streaming through this app whilst opening up the OpenCV app. Seems like a workaround instead of a solution but it works for me.
The solution is to completely uninstall the Logi Capture application.
I have an RTSP stream streaming to Amazon Kinesis Video Streams. I want to be able to run custom image processing on each frame of the stream. I have the custom image processing algorithm written in python, so naturally I want to be able to pull the KVS with python, and input it to the image processing algorithm. To start, I've tried making a python app which just displays the KVS stream by following advice from this post (however this post uses Amazon Rekognition for custom processing, my use case (custom python processing) is slightly different).
My questions are:
How come my HLS python KVS player is laggy / choppy (randomly pauses, then plays the past few seconds very quickly)? However, the HLS streamer available here looks quite good.
How can you effectively pull an Amazon Kinesis Video Stream for custom python processing? What are the pros/cons of using HLS vs Amazon's GetMedia API?
I also looked into using Amazon's GetMedia API, and found this post where someone attempted to use GetMedia but then ended up going with HLS.
I found the approach to using GetMedia for custom processing explained here (python) and here (c++).
Is it worth using Sagemaker for this? For my image processing, I only want to perform inference, I don't need to train the network, it is already trained.
Code for kvs_player.py:
import boto3
import cv2
STREAM_NAME = "ExampleStream"
STREAM_ARN = "MY_STREAM_ARN"
AWS_REGION = 'us-east-1'
def hls_stream():
kv_client = boto3.client("kinesisvideo", region_name=AWS_REGION)
endpoint = kv_client.get_data_endpoint(
StreamName=STREAM_NAME,
APIName="GET_HLS_STREAMING_SESSION_URL"
)['DataEndpoint']
print(endpoint)
# # Grab the HLS Stream URL from the endpoint
kvam_client = boto3.client("kinesis-video-archived-media", endpoint_url=endpoint, region_name=AWS_REGION)
url = kvam_client.get_hls_streaming_session_url(
StreamName=STREAM_NAME,
PlaybackMode="LIVE"
)['HLSStreamingSessionURL']
vcap = cv2.VideoCapture(url)
while(True):
# Capture frame-by-frame
ret, frame = vcap.read()
if frame is not None:
# Display the resulting frame
cv2.imshow('frame', frame)
# Press q to close the video windows before it ends if you want
if cv2.waitKey(22) & 0xFF == ord('q'):
break
else:
print("Frame is None")
break
# When everything done, release the capture
vcap.release()
cv2.destroyAllWindows()
print("Video stop")
if __name__ == '__main__':
hls_stream()
Some generic answers
You might want to do some debugging into your application to understand what "laggy" means. It could simply be the network latency, errors while transmitting or simply performance issues running decoding, python processing and later re-encoding.
GetMedia is a fast API and the Parser library https://github.com/aws/amazon-kinesis-video-streams-parser-library can be used to do real-time parsing. You might need to decode the video stream which is two orders of magnitude more CPU and IO intensive operation so you might want to use some hardware accelerated decoder (or at least a good software based decoder that takes advantages of SIMD/SSE2 instruction set). JCodec is certainly not the one.
It depends on your application really. At any rate, before the actual Sagemaker run, you need to retrieve the fragments, parse and decode them. There is a KVS Sagemaker KIT sample you can use for more serious inference applications.
For a more scoped and targeted questions you can go directly to GitHub page for the appropriate project/asset in question and cut an issue including detailed description and the verbose logs.
I get video streams from ip cameras using rtsp in python and want to get absolute timestamp for each frame in the sender reports. If I read a stream the camtime obviously starts from zero:
ret, image_np = cap.read()
camtime = cap.get(cv2.CAP_PROP_POS_MSEC)/1000.
I need to know when the frames are recorded since the cameras are located in different time zones and the frames are received with different delays. Can someone advise what library to use and how the python code should look like?
Thanks!
P.S. I can get camera streaming start time from ffmpeg in SDP (option "o").
I am opening my v4l2 device and both left and right streams are joined and opened at the same time, is there a way to split left and right sensor image frames and display them simultaneously using gstreamer?
EDIT1:
OK, So, I have a v4l2 device and a stereo camera which has both left and right streams writing to /dev/video0 and using gstreamer,I was able to view both the frames, I would like to know how to split left frame and right frame and display in separate windows. Also, I am trying out this script in opencv too, where I am only getting right video stream, I want to be able to view both streams in separate windows either in opencv or using gstreamer.
The below is openCV one
import os
import v4l2capture
import select
import numpy as np
import cv2
video = cv2.VideoCapture("/dev/video0")
while(True):
ret,frame = video.read()
cv2.imshow('whatever',frame)
key = cv2.waitKey(1) & 0xFF
if(key == ord("q")):
break
video.release()
cv2.destroyAllWindows()
The normal gstreamer application is just using a source and sink
gst-launch-1.0 v4l2src ! xvimagesink
I was able to get the answer to my question here,
https://www.stereolabs.com/docs/opencv/python/
I'm having problems to get a video stream from a IP camera I have. I'm using opencv to get the images from it. Here's the code i have:
import sys
import cv
video="http://prot-on.dyndns.org:8080/video2.mjpeg"
capture =cv.CaptureFromFile(video)
cv.NamedWindow('Video Stream', 1 )
while True:
# capture the current frame
frame = cv.QueryFrame(capture)
if frame is None:
break
else:
#detect(frame)
cv.ShowImage('Video Stream', frame)
if k == 0x1b: # ESC
print 'ESC pressed. Exiting ...'
break
Actually, this thing works, but it takes too much time to display the images. I'm guessing it's because of this error from ffmpeg.
[mjpeg # 0x8cd0940]max_analyze_duration reached
[mjpeg # 0x8cd0940]Estimating duration from bitrate, this may be inaccurate
I'm not a python expert so any help would be appreciated!
First, mjpeg is a relatively simple video format. If you read your IP camera's manual, it's very like that you can find how to display the video in browser with a bit JavaScript code. In fact, if you open link of http://prot-on.dyndns.org:8080/video2.mjpeg in Google Chrome, you would see the video without any problem. (Maybe you shouldn't leave the real URL of your camera)
Second, as far as I can see, the frame rate of your camera is pretty slow. That might due to Internet latency or your camera's setting. Compare what you see in Chrome with the video displayed by your code, if they are of same quality, then it's not your code's problem.