This is the standard example of DBus service.
import dbus
import gobject
from dbus import service
# from gi._gobject import MainLoop
from dbus.mainloop.glib import DBusGMainLoop
class DBusServer(service.Object):
def __init__(self, name, object_path):
# super(service.Object, self).__init__(name, object_path)
dbus.service.Object.__init__(self, name, object_path)
#dbus.service.method("com.test", in_signature='s', out_signature="s")
def test(self, args):
return args + " Sent by dbus client"
#dbus.service.method("com.test", in_signature='s', out_signature="s")
def foo(self, args):
return "foo"
bus_loop = DBusGMainLoop(set_as_default=True)
session_bus = dbus.SessionBus()
session_name = service.BusName("com.test", session_bus)
dbus_server = DBusServer(session_name, "/test")
loop = gobject.MainLoop()
try:
loop.run()
except KeyboardInterrupt:
loop.quit()
I have questions regarding two mainloops used here
1. What is the role of each mainloop or event loop in this code (if I am using the correct terminology. They both are event loops I guess)
2. If my app is not a GUI app why should I need gobject mainloop or qt mainloop since that is required to catch user generated events from X11 library (in case of Linux)
3. Why can't I use an infinite loop which does nothing instead of gobject main loop (as follows)
while True:
pass
Below are summarized short answers, see the details at the end for more explanations.
Question 1:
There is only one mainloop being used in the code you posted, the loop = gobject.MainLoop(). Its role is to process events, e.g. D-Bus messages sent to your service.
Question 2:
You need a mainloop in order for your code to not block D-Bus messages coming from outside your service. The dbus-python binding does not have a mainloop implementation so you need to use a mainloop from some other library (which also needs to be one supported by dbus-python, in this case GLib).
Question 3:
An infinite loop like that will never allow for any code other than that particular loop to execute, and there would be no way of receiving D-Bus messages (or any other events).
Details:
Simply put, a mainloop typically processes other events when there is no code currently running in your program. In this case, the nature of D-Bus requires there to be a way for your code to handle events from an external source, i.e. D-Bus messages. This is the reason that you need to have a mainloop. You could base the event processing on a loop you write yourself, as long as you have code in the loop that can process external events. The reason people often use libraries for mainloops is because things tend to grow complex in 'real' applications or systems and it's often more robust to use something which has been tested, used, and improved by others for years (e.g. GLib).
The D-Bus binding dbus-python is implemented in a way that requires an external mainloop to be used, as it does not have its own mainloop implementation. A typical reason for designing a library to use an external mainloop is to let the user of a library decide what mainloop to use, since the mainloop could potentially be used for other things as well (e.g. process other events than D-Bus events). This increases the potential use of the library and reduces the dependencies introduced as a side-effect of using the library.
The library must however support the mainloop of choice which means that there must be some integration provided specifically for each mainloop intended to be used with the library. Typically, the internals of a library like this would be designed to use a class/type that is an abstraction of a mainloop so the code internals doesn't need to know about what mainloop has been integrated. This reduces the amount of code which needs to be mainloop implementation specific within the library.
If you examine the bus_loop object received when doing:
bus_loop = DBusGMainLoop(set_as_default=True)
... you will see that it is an object of type dbus.mainloop.NativeMainLoop which is the mainloop wrapper object in dbus-python.
Now, dbus-python currently only provides an integration for the GLib mainloop so the discussion above becomes a bit theoretical in this case. The way dbus-python integrates a GLib mainloop is to require the user to import and instantiate the GLib mainloop wrapper, like this:
from dbus.mainloop.glib import DBusGMainLoop
DBusGMainLoop(set_as_default=True)
This creates a GLib main context which is later used by a GLib mainloop to process events that dbus-python associates with that context. The last part of the mainloop integration is to require the user to start a GLib mainloop. When you import and run the gobject mainloop, like this:
loop = gobject.MainLoop()
loop.run()
The mainloop will process events from the context created previously. This is the 'link' between the D-Bus binding dbus-python and the GObject binding gobject (which gives access to the GLib mainloop).
Simplified to the extreme, one could say that the context created and set when doing:
from dbus.mainloop.glib import DBusGMainLoop
DBusGMainLoop(set_as_default=True)
is a list of events to process if they appear, and the mainloop created and started when doing:
loop = gobject.MainLoop()
loop.run()
is what makes something check that list. The fact that GLib is used for both things makes it work, again simply put.
If you want to dig further into this, read up on GLib and the GMainLoop and GMainContext and the concepts related to these. Hope this helps.
Related
I recently switched from wxPython to PyQT and can't find an equivalent of CallAfter. I need to use pubsub due to some imports and with wx I just sent messages with CallAfter -- is there a way to do something similar in PyQT? Basically, I want to inject something into the mainloop with pyQT.
EDIT FOR MORE INFO:
In my old GUI, using wxPython, I was using python-openzwave which uses an old dispatcher module. I would capture the old dispatcher signals and convert them to pubsub messages (for ease of use) and send the new messages with a CallAfter like this:
wx.CallAfter(pub.sendMessage, messagePack.signal, message = messagePack.message)
And then I was able to update the GUI by capturing the message and working directly on the gui elements because it essentially injected something into the mainloop.
Now, using pyqt, there is no callafter so, I have the same system setup without the callafter but the actions that have to occur after the message is received can't happen because it is in the middle of the mainloop.
The closest thing I can think of is using QTimer.singleShot with a short timeout, which will force it into the next event loop.
def other_function(self):
print 'other'
def my_function(self):
print 'one'
QTimer.singleShot(1, self.other_function)
print 'two'
Qt has the idea of an event loop, where it will check if there are events that need processing, like a button click, or part of a widget needs to be redrawn, etc. Typically, a function gets called as the result of an event. The QTimer.singleShot will stick your function call at the end of the list of things to be processed on the next cycle of the event loop.
But I agree with some of the comments that you probably could just use a separate QObject running in another thread to handle the openzwave events and re-dispatch the messages as Qt Signals, which the main thread can listen for and update the GUI.
I understand that one is not supposed to update the UI from other threads in gtk, or face consequences, but I am not sure how I can avoid that while using gstreamer.
My application crashes from time to time during video stream initialization with a following complaint:
[xcb] Unknown sequence number while processing queue
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
[xcb] Aborting, sorry about that.
python: ../../src/xcb_io.c:274: poll_for_event: Assertion `!xcb_xlib_threads_sequence_lost' failed.
In my code I have added gtk.thread_init() call right in the beginning of the GUI class:
import pygtk, gtk, gobject
gtk.gdk.threads_init()
(I have also tried gobject.threads_init(), but this doesnt seem any different). In a separate class, which is run in a separate thread, I start a gstreamer stream to a tcpserversink (this gstreamer thread is already a third thread, if someone is keeping count). And then another thread receives this data before pushing the data to xvimagesink in the end.
The xvimagesink needs a viewport, and I believe this gstreamer callback function is where sometimes gtk goes crazy, when I assign it:
def on_sync_message(self, bus, message):
...
if message_name == "prepare-xwindow-id":
# Assign the viewport
imagesink = message.src
imagesink.set_property("force-aspect-ratio", True)
imagesink.set_xwindow_id(self.window_handle.window.xid)
The self.window_handle is a pointer to self.movie_window = gtk.DrawingArea(), assigned during GUI initialization.
TL;DR Is there a safe way of using gtk with gstreamer, since I cannot avoid threading when calling gst.Pipeline("name").set_state(gst.STATE_PLAYING) and view will be a GTK drawing area ?
I think your issue is the fact that you are accessing unprotected Xlib/xcb from 2 threads - once impicitly from your Gtk+ UI and once in the thread where your gstreamer backend callback is executed - which by default is the mainloop you told gstreamer to use (or the thread default mainloop).
gtk.gdk.threads_init() is already called once the type system is initialized (if I recall correctly, correct me if I am wrong).
Use g_idle_add (or use GSource with a higher priority) (which are threadsafe) with a callback which schedules UI changes in the Gtk mainloop (run by gtk_main()).
Does anyone used PyQt with gevent?
How to link PyQt loop to the gevent?
http://www.gevent.org/ - coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on top of libevent event loop.
You can use a Qt IDLE "timer" to allow gevent for processing its microthreads while no Qt events handled for a short period of time, for example 10 milliseconds. It is still not perfect, since it does not give the "smoothest" possible integration. It is because we don't use a single event loop for both Qt and gevent, just "interleaving" them in time.
The correct solution would be to allow libevent to listen on new Qt events somehow, but I haven't been able to figure out how to do that in practice yet. Maybe having Qt to send something to gevent via a socket when a GUI event arrives into the event queue would help. Has anybody solved that?
Working example:
""" Qt - gevent event loop integration using a Qt IDLE timer
"""
import sys, itertools
import PySide
from PySide import QtCore, QtGui
import gevent
# Limit the IDLE handler's frequency while still allow for gevent
# to trigger a microthread anytime
IDLE_PERIOD = 0.01
class MainWindow(QtGui.QMainWindow):
def __init__(self, application):
QtGui.QMainWindow.__init__(self)
self.application = application
self.counter = itertools.count()
self.resize(400, 100)
self.setWindowTitle(u'Counting: -')
self.button = QtGui.QPushButton(self)
self.button.setText(u'Reset')
self.button.clicked.connect(self.reset_counter)
self.show()
def counter_loop(self):
while self.isVisible():
self.setWindowTitle(u'Counting: %d' % self.counter.next())
gevent.sleep(0.1)
def reset_counter(self):
self.counter = itertools.count()
def run_application(self):
# IDLE timer: on_idle is called whenever no Qt events left for processing
self.timer = QtCore.QTimer()
self.timer.timeout.connect(self.on_idle)
self.timer.start(0)
# Start counter
gevent.spawn(self.counter_loop)
# Start you application normally, but ensure that you stop the timer
try:
self.application.exec_()
finally:
self.timer.stop()
def on_idle(self):
# Cooperative yield, allow gevent to monitor file handles via libevent
gevent.sleep(IDLE_PERIOD)
def main():
application = QtGui.QApplication(sys.argv)
main_window = MainWindow(application)
main_window.run_application()
if __name__ == '__main__':
main()
I tried the following approach: to have a "PyQt backend" for gevent, ie. an implementation of the gevent loop making use of PyQt constructs like QSocketNotifier, QTimer, etc. instead of the libev loop. Finally I found it much easier than doing the opposite, and performance is very good (Qt's loop is based on the glib under Linux, it's not so bad).
Here is the link to the project on github for those interested:
https://github.com/mguijarr/qtgevent
This is just a start, but it works well for the tests I did. I would be happy if people with more experience with gevent and PyQt could contribute.
Here's how you would change pyqt by example's session1 to cooperate: https://github.com/traviscline/pyqt-by-example/commit/b5d6c61daaa4d2321efe89679b1687e85892460a
you sould avoid use app.exec_(), it is a loop function which use this function to process events:
http://doc.qt.nokia.com/stable/qcoreapplication.html#processEvents
so you can call processEvents directly.
I released a project named eventlet-pyqt. I hope it could be useful for the one who want to use greenlet in their PyQt application. I also tried gevent, but it was difficult for me to write a plugin for libevent because of my poor C language experience. The main problem using QApplicaton::processEvents() or a zero-interval QTimer is, the program run into infinite loop, causes 100% CPU core usage. To avoid this, I wrote a new hub to replace the select() function with PyQt's QSocketNotifier. Hope this message could help some one.
I have a Python script which uses Tkinter for the GUI. My little script should create a Toplevel widget every X seconds. When I run my code, the first Toplevel widget is created successfully, but when it tries to create a second one the program crashes.
What I am doing is using the after method to call the function startCounting every 5 seconds alongside root's mainloop. Every time this function is called, I append a Toplevel widget object into a list and start a new thread which hopefully will be running the new mainloop.
I would be very grateful if someone could figure this problem out. By the way, this is just a little script that I am currently using to solve my problem, which is preventing me from going on with my real school project.
The code:
import threading,thread
from Tkinter import *
def startCounting():
global root
global topLevelList
global classInstance
topLevelList.append (Toplevel())
topLevelList[len(topLevelList)-1].title("Child")
classInstance.append(mainLoopThread(topLevelList[len(topLevelList)-1]))
root.after(5000,startCounting)
class mainLoopThread(threading.Thread):
def __init__(self,toplevelW):
self.toplevelW = toplevelW
threading.Thread.__init__(self)
self.start()
def run(self):
self.toplevelW.mainloop()
global classInstance
classInstance = []
global topLevelList
topLevelList = []
global root
root = Tk()
root.title("Main")
startCounting()
root.mainloop()
Tkinter is designed to run from the main thread, only. See the docs:
Just run all UI code in the main
thread, and let the writers write to a
Queue object; e.g.
...and a substantial example follows, showing secondary threads writing requests to a queue, and the main loop being exclusively responsible for all direct interactions with Tk.
Many objects and subsystems don't like receiving requests from multiple various threads, and in the case of GUI toolkit it's not rare to need specfically to use the main thread only.
The right Python architecture for this issue is always to devote a thread (the main one, if one must) to serving the finicky object or subsystem; every other thread requiring interaction with said subsystem or object must them obtain it by queueing requests to the dedicated thread (and possibly waiting on a "return queue" for results, if results are required as a consequence of some request). This is also a very sound Python architecture for general-purpose threading (and I expound on it at length in "Python in a Nutshell", but that's another subject;-).
Tkinter has issues dealing with input from multiple threads, I use mtTkinter instead, you won't need to change any code and everything will work fine. Just import mtTkinter instead of Tkinter.
You can get it here:
http://tkinter.unpythonic.net/wiki/mtTkinter
Is there a reason you want (or think you need) one event loop per toplevel window? A single event loop is able to handle dozens (if not hundreds or thousands) of toplevel windows. And, as has been pointed out in another answer, you can't run this event loop in a separate thread.
So, to fix your code you need to only use a single event loop, and have that run in the main thread.
I am using PyGTK to build a GUI application. I want to update the textview widget from another thread but the widget is not getting updated everytime i try an update. What should i do to get a reliable GUI updating?
GTK+ is not thread-safe, so you should not simply call GUI update methods from other threads. glib.idle_add (or gobject.idle_add in older PyGTK versions) can be used for this purpose.
Instead of writing:
label.set_text("foo")
you would write:
glib.idle_add(label.set_text, "foo")
which causes the function call to be queued in GTK+.
If you need to run multiple statements, it's often easier to wrap them in a function:
def idle():
label1.set_text("foo")
label2.set_text("bar")
glib.idle_add(idle)
Make sure that the function passed to idle_add does not return True; otherwise it will be queued again.
Edit: As Daniel pointed out, you need to call gtk.gdk.threads_init() anywhere in your program first.
As stated in the previous answers, GTK is not "thread safe," but it is "thread-aware" - see this page on Threads: https://developer.gnome.org/gdk2/stable/gdk2-Threads.html
In order to modify GTK widgets from another thread you have to use GTK's locking. Call gtk.threads_init() immediately after importing the gtk module, and then you can update like so:
gtk.threads_enter()
# make changes...
gtk.threads_leave()
Note that the above will not work on Windows (see the link above). On Windows you must use gobject.idle_add() as explained above, though don't forget to put gobject.threads_init() directly after importing gobject in your code! The idle_add() function will execute the update itself in the main thread (the thread running gtk.main()).
the same may be achieved using gobject.idle_add method whose syntax is same as above,you have to import the module gobject
What Johannes said is correct, however since GTK is a wrapper for the glib and gobject things, you would actually want to use gtk.idle_add(). No need for the unnecessary imports.