We Keep Coding

How can I run two python script using the same pi camera in raspberry pi?

I am currently working on a project where I need to have a live stream video and a motion detection that when a motion is detected, the raspberry pi records a 10 second video. I have put each functionality in different python files. I am using a single Pi camera. I have also created a file that consists of all the camera functionality so that I don't need to initialize the picamera twice. In the live stream file there is a button that allows the user to activate the motion detection functionality. When the button is clicked an error is shown "Failed to enable connection: Out of Resources"(Note that the live stream is already running.). I just wanted to know if it is possible to run two programs simultaneously using a single pi camera. If it is possible, how? Any help would be appreciated. Thank you in advance.

In general, AFAIK, it is not possible to access a single Raspberry Pi camera simultaneously from 2 separate processes.
I see a couple of possibilities which I'll describe in separate sections.
Use the shell and its tee command, to duplicate your video stream to 2 separate processes like this:
raspivid ... | tee >(MotionDetectionProcess) >(LiveStreamingProcess) > /dev/null
This is a bash "process substitution".
You might use raspividyuv or ffmpeg or some other program, or even one written by yourself, to read the camera and pass the data to stdout.
This approach is very quick to set up and simple - everything just reads or writes its stdin or stdout. On the downside, everything must be started at the same time and things run in "lock-step" with each other.
Run a single video capture process, that simply writes the video frames into multiprocessing shared memory.
Write your other two programs separately, but such that they "attach" to the shared memory and get the frames from there rather than directly from the camera.
This approach is initially a bit harder to program, but things are more decoupled, so you may choose not to bother running motion detection one day, or to start it later or sooner, and your live stream will be unaffected.

Multiple threads sending over one socket simultaneously?

I have two python programs. Program 1 displays videos in a grid with multiple controls on it, and Program 2 performs manipulations to the images and sends it back depending on the control pressed in Program 1.
Each video in the grid is running in its own thread, and each video has a thread in Program 2 for sending results back.
I'm running this on the same machine though and I was unable to get multiple socket connections working to and from the same address (localhost). If there's a way of doing that - please stop reading and tell me how!
I currently have one socket sitting independent of all of my video threads in Program 1, and in Program 2 I have multiple threads sending data to the one socket in an array with a flag for which video the data is for. The problem is when I have multiple threads sending data at the same time it seems to scramble things and stop working. Any tips on how I can achieve this?

Regarding If there's a way of doing that - please stop reading and tell me how!.
There's a way of doing it, assuming you are on Linux or using WSL on Windows, you could use the hostname -I commend which will output an IP that looks like 192.168.X.X.
You can use that IP in your python program by binding your server to that IP instead of localhost or 127.0.0.1.

How to do real-time video streaming frame by frame in python?

What I need is to readin one frame of a video, then send this frame as a frame in video, like P-frame, to the server, and server decode it based on the information of the frames it received before this frame.(So only the first frame is sent as complete I-frame, the rest are just P-frame to save bandwidth)
Then I need to do some real time operation on this frame.
But I try ffmpeg's and opencv's built in streaming protocal, it can only save a complete video file. But I need to do some real time manipulate as soon as I receive one fram, so I cant wait till I have received the whole file.
So what package should I use? Or there is not existing lib/software that could do this?

Extracting each individual frame from an H264 stream for real-time analysis with OpenCV

Problem Outline
I have an h264 real-time video stream (I'll call this "the stream") being captured in Process1. My goal is to extract each frame from the stream as it comes through and use Process2 to analyze it with OpenCV. (Process1 is nodejs, Process2 is Python)
Things I've tried, and their failure modes:
Send the stream directly from one Process1 to Process2 over a named fifo pipe:
I succeeded in directing the stream from Process1 into the pipe. However, in Process2 (which is Python) I could not (a) extract individual frames from the stream, and (b) convert any extracted data from h264 into an OpenCV format (e.g. JPEG, numpy array).
I had hoped to use OpenCV's VideoCapture() method, but it does not allow you to pass a FIFO pipe as an input. I was able to use VideoCapture by saving the h264 stream to a .h264 file, and then passing that as the file path. This doesn't help me, because I need to do my analysis in real time (i.e. I can't save the stream to a file before reading it in to OpenCV).
Pipe the stream from Process1 to FFMPEG, use FFMPEG to change the stream format from h264 to MJPEG, then pipe the output to Process2:
I attempted this using the command:
cat pipeFromProcess1.fifo | ffmpeg -i pipe:0 -f h264 -f mjpeg pipe:1 | cat > pipeToProcess2.fifo
The biggest issue with this approach is that FFMPEG takes inputs from Process1 until Process1 is killed, and only then does Process2 begin to receive the data.
Additionally, on the Process2 side, I still don't understand how to extract individual frames from the data coming over the pipe. I open the pipe for reading (as "f") and then execute data = f.readline(). The size of data varies drastically (some reads have length on the order of 100, others length on the order of 1,000). When I use f.read() instead of f.readline(), the length is much larger, on the order of 100,000.
If I were to know that I was getting the correct size chunk of data, I would still not know how to transform it into an OpenCV-compatible array because I don't understand the format it's coming over in. It's a string, but when I print it out it looks like this:
��_M~0A0����tQ,\%��e���f/�H�#Y�p�f#�Kus�} F����ʳa�G������+$x�%V�� }[����Wo �1'̶A���c����*�&=Z^�o'��Ͽ� SX-ԁ涶V&H|��$
~��<�E�� ��>�����u���7�����cR� �f�=�9 ��fs�q�ڄߧ�9v�]�Ӷ���& gr]�n�IRܜ�檯����
� ����+ �I��w�}� ��9�o��� �w��M�m���IJ ��� �m�=�Soՙ}S �>j �,�ƙ�'���tad =i ��WY�FeC֓z �2�g�;EXX��S��Ҁ*, ���w� _|�&�y��H��=��)� ���Ɗ3# �h���Ѻ�Ɋ��ZzR`��)�y�� c�ڋ.��v�!u���� �S�I#�$9R�Ԯ0py z ��8 #��A�q�� �͕� ijc �bp=��۹ c SqH
Converting from base64 doesn't seem to help. I also tried:
array = np.fromstring(data, dtype=np.uint8)
which does convert to an array, but not one of a size that makes sense based on the 640x368x3 dimensions of the frames I'm trying to decode.
Using decoders such as Broadway.js to convert the h264 stream:
These seem to be focused on streaming to a website, and I did not have success trying to re-purpose them for my goal.
Clarification about what I'm NOT trying to do:
I've found many related questions about streaming h264 video to a website. This is a solved problem, but none of the solutions help me extract individual frames and put them in an OpenCV-compatible format.
Also, I need to use the extracted frames in real time on a continual basis. So saving each frame as a .jpg is not helpful.
System Specs
Raspberry Pi 3 running Raspian Jessie
Additional Detail
I've tried to generalize the problem I'm having in my question. If it's useful to know, Process1 is using the node-bebop package to pull down the h264 stream (using drone.getVideoStream()) from a Parrot Bebop 2.0. I tried using the other video stream available through node-bebop (getMjpegStream()). This worked, but was not nearly real-time; I was getting very intermittent data streams. I've entered that specific problem as an Issue in the node-bebop repository.
Thanks for reading; I really appreciate any help anyone can give!

I was able to solve opening a Parrot Anafi stream with OpenCV (built with FFMPEG) in Python by setting the following environment variable:
export OPENCV_FFMPEG_CAPTURE_OPTIONS="rtsp_transport;udp"
FFMPEG defaults to TCP transport, but the feed from the drone is UDP so this sets the correct mode for FFMPEG.
Then use:
cv2.VideoCapture(<stream URI>, cv2.CAP_FFMPEG)
ret, frame = cap.read()
while ret:
cv2.imshow('frame', frame)
# do other processing on frame...
ret, frame = cap.read()
if (cv2.waitKey(1) & 0xFF == ord('q')):
break
cap.release()
cv2.destroyAllWindows()
as usual.
This should also work with a Parrot Bebop, but I don't have one to test it.

There are some suggestions online for piping the h264 stream into the opencv program using standard in:
some-h264-stream | ./opencv-program
where opencv-program contains something like:
VideoCapture cap("/dev/stdin");

How to capture network stream as image in Python?

In VLC I can do "Media -> Open Network Stream" and under "Please enter a network URL" I can put udp://#239.129.8.16:1234.
And this is opening local UDP video stream.
VLC is not related to my question, I have put it just for clarification.
How can I connect to "udp://#239.129.8.16:1234" network stream in Python, get image from it (screenshot) and save it in file?

I think neither network programming nor Python is the focus of your question here. At the core, you need to feed a video decoder with the binary data stream, make the video decoder collect a sufficient amount of data for decoding a single frame, let the decoder save this, and abort the operation.
I am quite sure that the command line tool avconv from the libav project can do everything you need. All you need to do is dig into the rather complex documentation and find the right command line parameters for your application scenario. From a quick glance, it looks you will need for instance
‘-vframes number (output)’
Set the number of video frames to record. This is an alias for -frames:v.
Also, you should definitely search the docs for this sentence:
If you want to extract just a limited number of frames, you can use
the above command in combination with the -vframes or -t option
It also looks like avconv can directly read from a UDP stream.
Note that VLC, the example you gave, uses libav at the core. Also note that if you really need to execute all this "from Python", then spawn avconv from within Python using the subprocess module.