Good day dear community,
I have a system that is running a Nvidia NX system with an IntelSense 3D camera. The goal is to capture the video stream with Python and OpenCV on another computer that is connected over WiFi with the system.
On my computer I can access the videostream in my browser with XXX.XXX.XX.X:8080 over mjpg-streamer. The correct http-address is: http://XXX.XXX.XX.X:8080/stream.html
Unfortunately I had zero success with different approaches. First I tried to fetch the stream directly with OpenCV:
import cv2
stream = cv2.VideoCapture('http://192.168.12.1:8080/stream.html')
while True:
ret, frame = stream.read()
cv2.imshow('Video Stream Monitor', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
stream.release()
cv2.destroyAllWindows()
This didn't work. Too easy to be true.
Next I stumbled accross this thread and thought this may be more promising because in my first example I did nothing to convert the data. I updated my code:
import cv2
import urllib
import numpy as np
# stream = cv2.VideoCapture('http://192.168.12.1:8080/stream.html')
stream = urllib.request.urlopen('http://localhost:8080/frame.mjpg')
bytes = ''
while True:
bytes += stream.read(1024)
a = bytes.find('\xff\xd8')
b = bytes.find('\xff\xd9')
if a != -1 and b != -1:
jpg = bytes[a:b+2]
bytes = bytes[b+2:]
i = cv2.imdecode(np.fromstring(jpg, dtype=np.uint8), cv2.CV_LOAD_IMAGE_COLOR)
cv2.imshow('i', i)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
stream.release()
cv2.destroyAllWindows()
And the result was an Error:
URLError: <urlopen error [WinError 10061] Es konnte keine Verbindung hergestellt werden, da der Zielcomputer die Verbindung verweigerte>
Target computer denies the connection. And this is where I'm still stuck (I run Windows and it may be due to that, I will try switching to a Linux system). Although I think, that with the connection established, there will still be problems because I capture data from .html link and the code is requesting with a .mjpg ending.
Anyways, is there somebody that knows how I can get the stream from my system to my computer?
I also thought about connection the computer with a socket to the system ip and port to retrieve the data. Would that be even possible without setting up a server-client architecture on both systems? But that would be not my very first option because I already have latency because of the WiFi connection. Implementing a socket with server and client will make latency issues worse, so I'd like to retrieve the stream preferably directly.
Any help appreciated.
Everything I tried is stated with my code examples. Thus, this line is obligatory information.
Related
The code:
import cv2
cap= cv2.VideoCapture(1)
if not cap.isOpened():
raise IOError("Cannot open webcam")
cntr =0
while True:
ret,frame = cap.read()
cntr= cntr+1;
if ((cntr%30)==0):
cv2.putText(frame,'cool' ,(50,50), cv2.FONT_HERSHEY_SIMPLEX, 0.7,(0,0,255),2)
cv2.imshow('Text Detection Tutorial',frame)
if cv2.waitKey(2) & 0xff == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
The error in the jupyter-notebook window:
17 raise IOError("Cannot open webcam")
OSError: Cannot open webcam
I have two devices, a built-in webcam /dev/video0 and an USB webcam /dev/video1.
They both appear listed with ls -ltrh /dev/video*
Both cameras work fine with guvcview.
I use jupyter-notebook to run the code above.
When I use cap= cv2.VideoCapture(0) it works fine. But when I use cap= cv2.VideoCapture(1) for the USB camera I get the next error in the terminal:
[ WARN:0] global
/tmp/pip-req-build-ms668fyv/opencv/modules/videoio/src/cap_v4l.cpp
(893) open VIDEOIO(V4L2:/dev/video1): can't open camera by index
I tried to give all permissions to /dev/video1, sudo chmod 777 /dev/video1.
I'm using linux mint.
Linux creates 2 files in the /dev/video for each webcam. The lower numbered one (/dev/video0 in your case ) is for capturing stream and the following one (/dev/video1 in your case) includes some data about camera.
So when you tried with the index number 0, it works fine and opens the camera with VideoCapture(0). But what about the 2nd camera ? There may be some different reasons why it is missing:
Current driver systems doesn't support that ( not a video4linux device or old kernel version )
It is not usb-plugged camera.
As a result, its not bacause of VideoCapture can not read; its just because of you are trying to open it with wrong index which is data file of camera.
Note: I suggest you to try VideoCapture(-1) also, this also works in some cases interestingly. Sometimes for substreams of webcams or luckiliy opening a camera's substream which has its own driver.
Edit: To clarify: it does compile, it just crashes almost immediately after the stream loads. It does connect properly.
So, I've been trying for a very long time to complete this project of mine. What I'm trying to do is send a video feed over sockets using cv2. It works over LAN, not over WAN. I get the following error:
"ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host"
Code for client(sending video over):
import cv2
import numpy as np
import socket
import pickle
host = "<insert public ip of recipient>"
port = 7643
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # declares s object with two parameters
s.connect((host, port)) # connects to the host & port
cap = cv2.VideoCapture(1)
while cap.isOpened(): # while camera is being used
ret, frame = cap.read() # reads each frame from webcam
if ret:
encoded = pickle.dumps(cv2.imencode(".jpg", frame)[1]) # encoding each frame, instead of sending live video it is sending pictures one by one
s.sendall(encoded)
if cv2.waitKey(1) & 0xFF == ord("q"): # wait until key was pressed once and
break
cap.release()
cv2.destroyAllWindows()
Code for recipient(receiving video):
import cv2
import socket
import pickle
host = "192.168.1.186"
port = 7643
boo = True
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # declares s object with two parameters
s.bind((host, port)) # tells my socket object to connect to this host & port "binds it to it"
s.listen(10) # tells the socket how much data it will be receiving.
conn, addr = s.accept()
while boo:
try:
pictures = conn.recv(256000) # creates a pictures variable that receives the pictures with a max amount of 128000 data it can receive
decoded = pickle.loads(pictures) # decodes the pictures
frame = cv2.imdecode(decoded, cv2.IMREAD_COLOR) # translates decoded into frames that we can see!
cv2.imshow("unique", frame)
if cv2.waitKey(1) & 0xFF == ord("q"): # wait until q key was pressed once and
break
except:
print("Something is broken...")
boo = False
cv2.destroyAllWindows()
s.close()
You apparently got lucky when running this over your LAN. Your code is not correctly sending a stream of images from sender to recipient, because stream sockets like TCP are a little more complicated to use by their nature. The main issue is that your sender is not communicating where each image ends and the next begins, and your recipient similarly is not organizing the data it reads into individual full images.
That is to say, socket.sendall() does not communicate the end of its data to the recipient; you need to include that information in the actual data that you send.
Error handling
But before fixing that, you should fix your error handling on the recipient so that you get more useful error messages. When you write
except:
print("Something is broken...")
You're throwing away something that would have helped you more, like "EOFError: Ran out of input" or "_pickle.UnpicklingError". Don't throw that information away. Instead, print it:
except:
traceback.print_exc()
or re-raise it:
except Exception as err:
# do whatever you want to do first
raise err
or, since you want to let it crash your program, and just want to do cleanup first, do your cleanup in a finally clause, no need for except:
try:
# your code
finally:
# the cleanup
Stream sockets and the sender
Back to your socket code, you're using stream sockets. They send a stream of bytes, and while you can count on them arriving in the correct order, you can't count on when they'll arrive. If you send b"something" and then b"something else", you could receive b"somethingsomething else" all at once, b"somet" and then later b"hing", etc. Your receiver needs to know where the dividing line is between each message, so step one is making there be dividing lines between the messages. There are a few ways to do this:
Making all messages be the same size. Since you're encoding them as JPEGs which can have different sizes based on how it's compressed, that would be a little complicated and maybe not what you want anyway.
Sending an actual marker in bytes, like a newline b"\n" or b"\n\r". This is more complicated to make work for your situation.
Sending the size of each message before you send it. This should be the easiest for your case.
Of course if you're now sending the size of the message, that's just like another message, and your recipient needs to know where this size message ends. Once again you could end the size message with a newline:
s.sendall("{}\n".format(len(encoded)).encode("ascii"))
Or you could pack it into a fixed-length number of bytes, for example 4:
s.sendall(struct.pack("!i", len(encoded)))
The receiver
Your receiver code now needs to read full messages, despite the fact that socket.recv() can return partial messages, or parts of multiple messages together. You can keep a buffer of the incoming data. Add to the end, and then remove full messages from the front:
buf = ''
while boo:
new_data = s.recv(4096)
if not new_data:
# exit, because the socket has been closed
buf += new_data
# if there's a full message at the beginning of buf:
# remove that message, but leave the rest in buf
# process that message
# else:
# nothing, just go back to receiving more
Of course, to find your full message, first you need to get the full size message. If you encoded all your size messages as 4 bytes with struct.pack, just receive data until buf is 4 or more bytes long, then split it into the size and any remaining data:
message_size = struct.unpack("!i", buf[:4])[0]
buf = buf[4:]
Then do the same thing with the image message. Receive data until you have at least message_size bytes of data, split your buffer into the first image message, which you can decode and display, and keep the remainder in the buffer.
Security
The documentation for pickle says:
Warning: The pickle module is not secure. Only unpickle data you trust.
It is possible to construct malicious pickle data which will execute arbitrary code during unpickling. Never unpickle data that could have come from an untrusted source, or that could have been tampered with.
In your case, someone else could in theory connect to your IP on your chosen port and send whatever they wanted to your recipient. If this is just a toy project that wouldn't be left running all the time, the odds are low.
I'm fighting with Chinese IP camera modules, one of them has chipset IMX322 and has 2MPX. I managed to connect to this camera (there was no doc. how to do it ) but url above seems to work :
rtsp://<local_ip>/user=admin&password=&channel=1&stream=0.sdp
I use standard python loop to get data, but there is some problem and I'm really not sure where, it can be problem with camera (wrong header) or library under Nvidia Jetson TX2 and Ubuntu 16.4.
#!/usr/bin/env python
import sys
from time import sleep
import numpy as np
import cv2
import time
cap = cv2.VideoCapture('rtsp://192.168.1.11/user=admin&password=&channel=0&stream=0.sdp?tpc')
while(True):
ret, frame = cap.read()
if ret:
cv2.imshow("Image", cv2.resize(frame,(800,600)))
else:
print('no video')
#cap.set(cv2.CAP_PROP_POS_FRAMES, 0)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
The problem is that I'm getting unexpected behavior of VideoCapture(), it takes about 2 seconds to generate output frame in imgShow, but before that I'm receiving errors in command line level 2,3 times.
Invalid UE golomb code
Invalid UE golomb code
During extraction of correct frames from camera errors are still showing with frequency 1,2 per second. I tried to find any useful information in google but it looks like 90% of post are related with C++ or FFMPEG.
Well, it seems like my question had been asked many times before and unfortunately, no one replied. I hope someone will help.
I have an Easycap device that converts the analog images from my analog camera to digital signals through a USB port.
The device is identified to the system in the Device Manager under "Sound, Video and game controllers" category as "SMI Grabber Device".
I use a simple Python code to display the video from this device. I also have an embedded webcam in my laptop.
import numpy as np
import cv2
cap = cv2.VideoCapture(0)
while(True):
# Capture frame-by-frame
ret, frame = cap.read()
# Display the resulting frame
cv2.imshow('frame',frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
if cv2.waitKey(1) & 0xFF == ord('s'):
cv2.imwrite('screenshot.jpg',frame)
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
First, when I unplug the Easycap, CaptureVideo(0) returns the embedded webcam video stream. However, when I plug the Easycap, an error appears:
"Traceback (most recent call last):
File "C:\Users\DELL\Desktop\code\cam.py", line 10, in
cv2.imshow('frame',frame)
error: ......\src\opencv\modules\highgui\src\window.cpp:261: error: (-215) size.width>0 && size.height>0"
Notice that, any number except 0 makes the program display the webcam image. So if I tried cap = cv2.CaptureVideo(1), it will show the webcam, cap = cv2.CaptureVideo(20) is the same.
I also tried to enter "SMI Grabber Device" instead of 0 or 1 in the VideoCaptureconstructor function, but it didn't make any difference.
I'm using Windows 8, and I've installed the accompanying driver for Easycap. The software that comes with the driver (called ULead) works fine and display the CCTV camera video. I tried to display the images while I'm closing that program, and without, the result is the same.
I used before a C# program with Aforge library which had getCamList method or something which allowed me to choose the specific device I want to display from a comboBox. I can't find a similar function is opencv.
I'm using OpenCV 2.4.6. I didn't try the code on prior versions.
I really need to understand why this code doesn't work, knowing that I'm just a very beginner of opencv and image processing.
I hope someone can help.
I am using EasyCAP too.
You must check that ret is True.
I am use below code
while True:
ret, frame = vc.read()
if ret:
break
cv2.waitKey(10)
h, w = frame.shape[:2]
print h, w
while True:
ret, frame = vc.read()
if ret:
cv2.imshow(WID, frame)
if cv2.waitKey(1) == 27:
break
Let there be light!
On serious note, I struggled with the same problem and I hope this helps!
the original thread + answer
I have recently set up a Raspberry Pi camera and am streaming the frames over RTSP. While it may not be completely necessary, here is the command I am using the broadcast the video:
raspivid -o - -t 0 -w 1280 -h 800 |cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:8554/output.h264}' :demux=h264
This streams the video perfectly.
What I would now like to do is parse this stream with Python and read each frame individually. I would like to do some motion detection for surveillance purposes.
I am completely lost on where to start on this task. Can anyone point me to a good tutorial? If this is not achievable via Python, what tools/languages can I use to accomplish this?
Using the same method listed by "depu" worked perfectly for me.
I just replaced "video file" with "RTSP URL" of actual camera.
Example below worked on AXIS IP Camera.
(This was not working for a while in previous versions of OpenCV)
Works on OpenCV 3.4.1 Windows 10)
import cv2
cap = cv2.VideoCapture("rtsp://root:pass#192.168.0.91:554/axis-media/media.amp")
while(cap.isOpened()):
ret, frame = cap.read()
cv2.imshow('frame', frame)
if cv2.waitKey(20) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
Bit of a hacky solution, but you can use the VLC python bindings (you can install it with pip install python-vlc) and play the stream:
import vlc
player=vlc.MediaPlayer('rtsp://:8554/output.h264')
player.play()
Then take a snapshot every second or so:
while 1:
time.sleep(1)
player.video_take_snapshot(0, '.snapshot.tmp.png', 0, 0)
And then you can use SimpleCV or something for processing (just load the image file '.snapshot.tmp.png' into your processing library).
use opencv
video=cv2.VideoCapture("rtsp url")
and then you can capture framse. read openCV documentation visit: https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_gui/py_video_display/py_video_display.html
Depending on the stream type, you can probably take a look at this project for some ideas.
https://code.google.com/p/python-mjpeg-over-rtsp-client/
If you want to be mega-pro, you could use something like http://opencv.org/ (Python modules available I believe) for handling the motion detection.
Here is yet one more option.
It's much more complicated than the other answers.
But this way, with just one connection to the camera, you could "fork" the same stream simultaneously to several multiprocesses, to the screen, recast it into multicast, write it to disk, etc.
Of course, just in the case you would need something like that (otherwise you'd prefer the earlier answers)
Let's create two independent python programs:
Server program (rtsp connection, decoding) server.py
Client program (reads frames from shared memory) client.py
Server must be started before the client, i.e.
python3 server.py
And then in another terminal:
python3 client.py
Here is the code:
(1) server.py
import time
from valkka.core import *
# YUV => RGB interpolation to the small size is done each 1000 milliseconds and passed on to the shmem ringbuffer
image_interval=1000
# define rgb image dimensions
width =1920//4
height =1080//4
# posix shared memory: identification tag and size of the ring buffer
shmem_name ="cam_example"
shmem_buffers =10
shmem_filter =RGBShmemFrameFilter(shmem_name, shmem_buffers, width, height)
sws_filter =SwScaleFrameFilter("sws_filter", width, height, shmem_filter)
interval_filter =TimeIntervalFrameFilter("interval_filter", image_interval, sws_filter)
avthread =AVThread("avthread",interval_filter)
av_in_filter =avthread.getFrameFilter()
livethread =LiveThread("livethread")
ctx =LiveConnectionContext(LiveConnectionType_rtsp, "rtsp://user:password#192.168.x.x", 1, av_in_filter)
avthread.startCall()
livethread.startCall()
avthread.decodingOnCall()
livethread.registerStreamCall(ctx)
livethread.playStreamCall(ctx)
# all those threads are written in cpp and they are running in the
# background. Sleep for 20 seconds - or do something else while
# the cpp threads are running and streaming video
time.sleep(20)
# stop threads
livethread.stopCall()
avthread.stopCall()
print("bye")
(2) client.py
import cv2
from valkka.api2 import ShmemRGBClient
width =1920//4
height =1080//4
# This identifies posix shared memory - must be same as in the server side
shmem_name ="cam_example"
# Size of the shmem ringbuffer - must be same as in the server side
shmem_buffers =10
client=ShmemRGBClient(
name =shmem_name,
n_ringbuffer =shmem_buffers,
width =width,
height =height,
mstimeout =1000, # client timeouts if nothing has been received in 1000 milliseconds
verbose =False
)
while True:
index, isize = client.pull()
if (index==None):
print("timeout")
else:
data =client.shmem_list[index][0:isize]
img =data.reshape((height,width,3))
img =cv2.GaussianBlur(img, (21, 21), 0)
cv2.imshow("valkka_opencv_demo",img)
cv2.waitKey(1)
If you got interested, check out some more in https://elsampsa.github.io/valkka-examples/
Hi reading frames from video can be achieved using python and OpenCV . Below is the sample code. Works fine with python and opencv2 version.
import cv2
import os
#Below code will capture the video frames and will sve it a folder (in current working directory)
dirname = 'myfolder'
#video path
cap = cv2.VideoCapture("your rtsp url")
count = 0
while(cap.isOpened()):
ret, frame = cap.read()
if not ret:
break
else:
cv2.imshow('frame', frame)
#The received "frame" will be saved. Or you can manipulate "frame" as per your needs.
name = "rec_frame"+str(count)+".jpg"
cv2.imwrite(os.path.join(dirname,name), frame)
count += 1
if cv2.waitKey(20) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
Use in this
cv2.VideoCapture("rtsp://username:password#IPAddress:PortNO(rest of the link after the IPAdress)").