I have a simple Python program creates an RTSP stream using gst-rtsp-server. It works, but as-is there's no error handling. If the pipeline has a typo or there's some issue connecting to the video source, I don't see a stack trace or any logging. Where would I hook in code to handle problems like this?
I should mention that I'm a complete beginner to the GObject world. I suspect there is a standard way for these libraries to report errors but I haven't been able to find anything in the documentation I've read about how that's done.
In case it's helpful, here is my code as I have it right now:
from threading import Thread
from time import sleep
import signal
import gi
gi.require_version("Gst", "1.0")
gi.require_version("GstRtsp", "1.0")
gi.require_version("GstRtspServer", "1.0")
from gi.repository import GLib, GObject, Gst, GstRtsp, GstRtspServer
PIPELINE = (
"( videotestsrc ! vp8enc ! rtpvp8pay name=pay0 pt=96 )")
def main():
GObject.threads_init()
Gst.init(None)
server = GstRtspServer.RTSPServer.new()
server.props.service = "3000"
server.attach(None)
loop = GLib.MainLoop.new(None, False)
def on_sigint(_sig, _frame):
print("Got a SIGINT, closing...")
loop.quit()
signal.signal(signal.SIGINT, on_sigint)
def run_main_loop():
loop.run()
main_loop_thread = Thread(target=run_main_loop)
main_loop_thread.start()
media_factory = GstRtspServer.RTSPMediaFactory.new()
media_factory.set_launch(PIPELINE)
media_factory.set_shared(True)
server.get_mount_points().add_factory("/test", media_factory)
print("Stream ready at rtsp://127.0.0.1:3000/test")
while loop.is_running():
sleep(0.1)
if __name__ == "__main__":
main()
So you can override do_handle_message in Gst.Bin in the following way:
import gi
gi.require_version("Gst", "1.0")
from gi.repository import Gst, GLib
Gst.init(None)
class SubclassBin(Gst.Bin):
def do_handle_message(self, message):
if message.type == Gst.MessageType.ERROR:
error, message = message.parse_error()
# TODO: Do something with the error
# Call the base Gst.Bin do_handle_message
super().do_handle_message(self, message)
subclass_bin = SubclassBin("mybin")
That said, I'm not sure how to tell GstRtspServer.RTSPMediaFactory to use SubclassBin instead of Gst.Bin because as far as I can tell, the only way to connect any Bin to RTSPMediaFactory is through the set_launch method which wants a pipeline string instead of a prebuilt bin. If there was a way of adding a prebuilt bin to RTSPMediaFactory, this would be a complete answer...but unfortunately this is as far as I can get.
Edit: I would recommend using bkanuka's solution (Subclassing Gst.Bin) instead of the below one, which has many drawbacks.
After some more experimentation with the GStreamer and the RTSP server library, the error handling situation is complicated.
The Normal Way
The canonical way to see errors on a GStreamer pipeline is to add a watcher to the pipeline's bus and listen for error messages.
def watcher(bus, message, *user_data);
if message.type == Gst.MessageType.ERROR:
error, message = message.parse_error()
# TODO: Do something with the error
my_pipeline.get_bus().add_watch(
GLib.PRIORITY_DEFAULT,
watcher,
None)
However, you cannot do this with pipelines you supply to a GstRtspServer. This is because GstRtspServer expects to be able to install its own watcher on the pipeline's bus and only one watcher can be attached to a bus at once. This is especially unfortunate because this prevents us from listening to any events on the pipeline, not just errors.
My Workaround
We can split the pipeline into two pieces: one that's in charge of the error-prone process of connecting to the source and decoding frames, and another that is in charge of encoding the resulting frames and payloading them for the GstRtspServer. We can then use the intervideo plugin to communicate between the two.
For example, let's say you're trying to stream from a file in the VP8 format. Our first pipeline that's in charge of reading and decoding frames would look like this:
filesrc location="{filepath}" ! decodebin ! intervideosink channel="file-channel"
... and our second pipeline that's in charge of encoding and payloading the frame would look like this:
intervideosrc channel="file-channel" ! videoconvert ! vp8enc deadline=1 ! rtpvp8pay name=pay0 pt=96
The key here is that only the second pipeline has to be managed by GstRtspServer, since it's the pipeline that provides the payloaded data. The first one is managed by us and we can attach our own watcher to it
that responds intelligently to errors and does whatever else we need. This of course is not a perfect solution because we aren't able to respond to errors related to encoding and payloading, but we've gained the ability to receive errors related to reading the file and decoding it. We're now also able to do other message-related tasks, like intercepting an end-of-stream message to loop the video file.
Related
I 've been trying to create a "communication wire" from my Raspberry Pi microphone (usb headsets) to my soundcard in order for the script to send whatever I say at microphone to my speakers through a pipeline.
Problem:
When i run the script below, the mic works and sends voice at my speakers for a couple of seconds. After that it stops repeating what i am saying.
The code i have is shown below:
#!/usr/bin/env python
import gi
import os
gi.require_version('Gst', '1.0')
gi.require_version('Gtk', '3.0')
from gi.repository import GObject, Gst, Gtk, GLib
#GObject.threads_init()
Gst.init(None)
loop = GLib.MainLoop ()
pipeline = Gst.parse_launch("autoaudiosrc ! audioconvert ! tee name=source ! queue ! vorbisenc ! oggmux ! filesink location=file.ogg source. ! queue ! audioconvert ! alsasink")
#autoaudiosrc = Gst.ElementFactory.make("autoaudiosrc", "autoaudiosrc")
#audioconvert = Gst.ElementFactory.make("audioconvert", "audioconvert")
#vorbisenc = Gst.ElementFactory.make("vorbisenc", "vorbisenc")
#oggmux = Gst.ElementFactory.make("oggmux", "oggmux")
#filesink = Gst.ElementFactory.make("filesink", "filesink")
#url = "1.ogg"
#filesink.set_property("location",url)
#pipeline.add( autoaudiosrc)
#pipeline.add( audioconvert)
#pipeline.add( vorbisenc)
#pipeline.add( oggmux)
#pipeline.add( filesink)
#autoaudiosrc.link( audioconvert)
#audioconvert.link( vorbisenc)
#vorbisenc.link( oggmux)
#oggmux.link( filesink)
pipeline.set_state(Gst.State.PLAYING)
loop.run()
pipeline.set_state(Gst.State.NULL)
Gtk.main()
At the start of the project I tried to create the pipeline with a different way (that's why I left in the code the comments in order for you to check my thought process), until i found a code that does my work here Simple microphone to speakers implementation so I tried to implement it to my project.
While debugging my first thought was that maybe pipeline closes after a while but some tests i did implied that this was not the case.
Please keep in mind that i am relatively new to GStreamer(been using it for less than 3 days)
so I might have some silly mistakes.
If you have the time please explain to me the solution of the problem (if you could spot it).
Thank you in advance.
**Edit 1: Just today I executed the code on a different raspberry Pi and I saw that a file is being created normally everytime the code executed that records my input whether I leave it open for 10 or 30 seconds (so the code on recorder's end is good). Probably the thing that needs to be fixed is the "play" code that plays the file that is created.
I tried the same piece of code on a Raspberry that i had previously installed Ubuntu 20. As expected the script was working fine. So in conclusion it is a Raspberry Pi OS issue and not an error on code. Also the simplest form of this pipeline can be achieved with:
pipeline = Gst.parse_launch("autoadiosrc ! pulsesink")
So the problem I'm trying to solve is to take an unknown of incoming URI streams and mix them into one outgoing stream. What I am essentially doing right now is using a for each loop where each iteration I create a source, add in elements like videoscale and the capsfilter, and then link it to the videomixer. Currently I have it working if I just pass in a usb webcam feed with autovideosrc. However, it completely falls apart when I try to use a uridecodebin, due to linking errors.
I know that normally when creating a uridecodebin you have to use the pad-added signal with something like this example.
#!/usr/bin/python
import pygst
pygst.require('0.10')
import gst
import pygtk
pygtk.require('2.0')
import gtk
# this is very important, without this, callbacks from gstreamer thread
# will messed our program up
gtk.gdk.threads_init()
def on_new_decoded_pad(dbin, pad, islast):
decode = pad.get_parent()
pipeline = decode.get_parent()
convert = pipeline.get_by_name('convert')
decode.link(convert)
pipeline.set_state(gst.STATE_PLAYING)
print "linked!"
def main():
pipeline = gst.Pipeline('pipleline')
filesrc = gst.element_factory_make("filesrc", "filesrc")
filesrc.set_property('location', 'C:/a.mp3')
decode = gst.element_factory_make("decodebin", "decode")
convert = gst.element_factory_make('audioconvert', 'convert')
sink = gst.element_factory_make("autoaudiosink", "sink")
pipeline.add(filesrc, decode, convert, sink)
gst.element_link_many(filesrc, decode)
gst.element_link_many(convert, sink)
decode.connect("new-decoded-pad", on_new_decoded_pad)
pipeline.set_state(gst.STATE_PAUSED)
gtk.main()
main()
However, every example of this that I have found has been assuming that I would always be connecting one uridecodebin to one other element. How would I go about implementing something where the element represented by 'convert' in this example would be different each time I call the function?
I've been trying to get an image to post to ROS (using Python/rospy), and while I think I have the method right, I'm having a hard time confirming it. Using
rosrun image_view image_view image:=(topic)
doesn't seem to show anything. I've also tried rqtbag, but I don't really know how that thing works, other than it doesn't show things published, anyways.
A few notes before pasting my current code:
The code I use right now is based off of code I have gotten to work previously. I've used a similar setup to post text images to ROS, and those output fairly reliably.
This is slightly modified here. Most of this code is part of an on_message function, since this all runs through MQTT when implemented. (The logic is acquire image on one system -> encode it -> transfer to other system -> decode -> publish to ROS.)
I'm using Python 2.7 on Ubuntu, and ROS Indigo.
Without further ado, my current code for publishing:
rospy.init_node('BringInAnImage', log_level = rospy.INFO)
def convert(messagepayload):
t = open('newpic.bmp','w')
t.write(messagepayload)
t.close()
def on_message(client, userdata, msg):
img = base64.b64decode(msg.payload)
convert(img)
time.sleep(5)
source = cv2.imread('newpic.bmp') #this should be a mat file
# talk to ROS
bridge = CvBridge()
pub2 = rospy.Publisher('/BringInAnImage', Image, queue_size = 10)
pub2.publish(bridge.cv2_to_imgmsg(source, "bgr8"))
print "uh..... done??"
I'm using a basic listening function to try and see what is going on (this is within a different script I execute in a separate terminal):
def listener():
rospy.init_node('listener', anonymous=True)
rospy.Subscriber("/BringInAnImage", Image, callback)
rospy.spin()
if __name__ == '__main__':
listener()
The callback just prints out that the image was received.
How to check if something is published on topic xyz
To check if a message is really published, you can use the rostopic command.
Run the following in a terminal to print everything that is published on the specified topic. This is the easiest way to check if there is something published.
rostopic echo <topic_name>
See the ROS wiki for more useful things rostopic can do.
Why is the image not received by image_view?
While you are doing it basically right, your images will not be received by any subscriber for a not so obvious but fatal problem in your code: You are using the publisher (pub2) immediately after initializing it. Subscribers need some time to register to the new publisher and will not be ready before you publish the image (see also this answer).
➔ Do not initialize a publisher just before you need it but do it right in the beginning, when initializing the node.
I am trying to save a stream from webcam as series of image using gstreamer. I have written this code so far...
#!/usr/bin/python
import sys, os
import pygtk, gtk, gobject
import pygst
pygst.require("0.10")
import gst
def __init__(self):
#....
# Code to create a gtk Window
#....
self.player = gst.Pipeline("player")
source = gst.element_factory_make("v4l2src", "video-source")
sink = gst.element_factory_make("xvimagesink", "video-output")
caps = gst.Caps("video/x-raw-yuv, width=640, height=480")
filter = gst.element_factory_make("capsfilter", "filter")
filter.set_property("caps", caps)
self.player.add(source, filter, sink)
gst.element_link_many(source, filter, sink)
After this, I am trying to create a signal over the bus to listen for any message from the source or the sink to indicate a new frame has been sent or received, so that it can be saved.
bus = self.player.get_bus()
bus.add_signal_watch()
bus.connect("message::any", self.save_file,"Save file")
where save_file is my callback, where I want to save the file.
def save_file(self, bus, msg):
print "SAVED A NEW FILE"
I have two questions,
How do I invoke this callback. The message::any is not working.
When this message is invoked, how do I get access to the image buffer.
UPDATE (4-12-2012):
Couple of links for reference
A python interface for v4l. But it has not been working for me. It seems to crash when i try to grab on 12.04 Ubuntu.
http://code.google.com/p/python-video4linux2/
A webcam viewer code for those interested. But this is not what I want since it uses gst-launch and does not provide the level of pipeline control I want to have. http://pygstdocs.berlios.de/pygst-tutorial/webcam-viewer.html
Gstreamer Bus is not intended to be used for this purpose. Messages that are put there signal rather some special event like end-of-stream, element state change and so on. Buffers (images) flowing through elements usualy don't generate any messages on the bus.
You may consider several possibilities:
make "tee" element before videosink and connect "multifilesink" in parallel to videosink (you may want to see some image encoders like pngenc or jpegenc and put one of them before multifilesink")
like before, but use "appsink" that allow you to handle buffers and do whatever-you-want with them
if you want to switch dumping on and off, consider using "valve" element
You may want to set "sync" property to false on your additional sink (Which cause buffers to be dumped as soon as possible without syncing to clock). Consider also adding some queues after tee (without this deadlock may occur during ready->paused transition).
I am not sure if my response after years will be useful for you. but hope, it will be useful for others.
to receive a message that you have received a buffer, you can use gstreamer probes.
It could be something similar:
def make_pipeline(self):
CLI2 = [
'v4l2src ! video/x-raw,format=RGB,width=640,height=480,framerate=30/1 ! ',
'videoconvert ! x264enc bitrate=128 ! mpegtsmux name="mux" ! hlssink name="sink"',
]
gcmd = ''.join(CLI2)
self.pipeline = Gst.parse_launch(gcmd)
self.hlssink = self.pipeline.get_by_name("sink")
self.hlssink.set_property("target-duration",2)
self.hlssink_pad = self.hlssink.get_static_pad("sink")
probe_id = self.hlssink_pad.add_probe(Gst.PadProbeType.EVENT_UPSTREAM,probe_callback)
and then, the probe callback function could be:
def probe_callback(hlssink_pad,info):
info_event = info.get_event()
info_structure = info_event.get_structure()
do_something_with_this_info
return Gst.PadProbeReturn.PASS
So, every time there is an event on either a source pad or sink pad, the probe callback function will be called in the main thread.
Hope this helps!
I'm trying to write a client in python 2.7 using Twisted. My code works just fine in linux (debian squeeze), but when I tried it on windows (xp and 7) I got a constant stream of error messages. A screenshot of these messages is here.
I have narrowed down the bug and was able to write a very stripped down version of my client that still contains the bug:
from twisted.internet.protocol import Protocol,ClientFactory
from twisted.protocols.basic import LineReceiver
from twisted.internet import reactor
class TheClient(LineReceiver):
def lineReceived(self,line):
print line
def connectionLost(self,reason):
reactor.stop()
class TheFactory(ClientFactory):
protocol = TheClient
class Test(object):
def doRead(self):
pass
def fileno(self):
return 0
def connectionLost(self,reason):
print 'connection lost'
def logPrefix(self):
return 'Client'
def main():
print 'starting'
test = Test()
reactor.addReader(test)
reactor.run()
if __name__ == '__main__':
main()
If the line containing 'reactor.addReader(test)' is commented out, I do not get any error messages. If I run this code on linux without commenting out any lines, I do not get any error messages.
I found this question, I don't think its the same problem, but as expected, it did not function properly on windows.
Is this code correct, and this is a windows bug, or do I have to do things differently for it to work in windows?
The Windows implementation of select only supports sockets. Presumably file descriptor 0 in your process does not represent a socket. More likely it represents something related to standard I/O.
If you'd just like to use standard I/O, then there's twisted.internet.stdio, though you may run into some rough edges with it on Windows (bug reports and fixes appreciated!).
If you're not interested in standard I/O and 0 was just an arbitrary test, you'll probably need to decide on what kind of input you're trying to do in particular. Depending on what kind of file descriptor you have, there will probably be a different approach to successfully reading from it.