I have one module which uses python "threading" for concurrency, and "signal" for shutdown hook:
signal.signal(signal.SIGINT, self.shutdownhook)
I have another module which uses dbus and gobject
dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
....
GObject.threads_init()
mainloop = GObject.MainLoop()
mainloop.run()
When I run them seperately, they both operate as expected and ctrl+c causes termination via "KeyboardInterrupt".
However, when I run them together the mainloop terminates but the shutdown hook is never called - the process does not terminate without kill -9 pid.
Can someone please explain why this occurs, and how best to integrate the two models
Here is a working example which highlights my problem. I cannot exit the program with just a CTRL+C and the shutdown hook is not called in this case either.
import threading
import signal
import sys
from gi.repository import GObject
def runMainloop():
print('running mainloop')
mainloop.run()
def shutdown():
print('shutdown')
def readInput():
print('readInput')
print(sys.stdin.readline())
if __name__ == '__main__':
signal.signal(signal.SIGINT, shutdown)
signal.signal(signal.SIGTERM, shutdown)
GObject.threads_init()
mainloop = GObject.MainLoop()
mainloopThread = threading.Thread(name='mainloop', target=runMainloop)
mainloopThread.setDaemon(True)
mainloopThread.start()
print('started')
inputThread = threading.Thread(name='input', target=readInput)
inputThread.start()
print('started input')
No one is interested, so let me try.
Just to be on the same page:
import signal
from gi.repository import GObject
GObject.threads_init()
mainloop = GObject.MainLoop()
signal.signal(signal.SIGINT, lambda n, f: mainloop.quit())
mainloop.run()
This code works:
import signal
from gi.repository import GObject
signal.signal(signal.SIGINT, lambda n, f: print("kill"))
GObject.threads_init()
mainloop = GObject.MainLoop()
mainloop.run()
I've first registered signal handler, and then initiated the loop. Strange is that it is not called. However result is - as expected...
As a side note - according to their doc... mainloop is deprecated. That is first thing.
Edit
Here is example with reading from stdin inside MainLoop:
import signal
import sys
from gi.repository import GObject, GLib
GObject.threads_init()
def readInput():
print('readInput\n')
while True:
input = sys.stdin.readline()
print(input)
if input.strip() == 'exit':
print('closing main loop')
mainloop.quit()
print('terminating thread')
return
if __name__ == '__main__':
signal.signal(signal.SIGINT, signal.SIG_DFL)
mainloop = GObject.MainLoop.new(None, False)
GObject.timeout_add(1000, readInput)
# inputThread = threading.Thread(name='input', target=readInput)
# inputThread.start()
# print('started input')
print('running mainloop\n')
try:
mainloop.run()
except KeyboardInterrupt:
mainloop.quit()
Adding .new(None, False) allows CTRL-C working normally. Took it from here, also here is another thread about integrating pulse audio controller with GLib/GObject loop. There are samples about integration dbus with the loop, but I'm not sure which way you wish to go...
Adding a timer makes the loop receive Unix signals.
import gi
from gi.repository import GLib
import signal
GLib.threads_init()
mainloop = GLib.MainLoop()
signal.signal(signal.SIGTERM, lambda signum, frame: mainloop.quit())
GLib.timeout_add(1000, lambda *args: (print("tick") or True))
try:
mainloop.run()
except KeyboardInterrupt:
print()
Related
I have a simple Python service, where there is a loop that performs some action infinitely. On various signals, sys.exit(0) is called, which causes SystemExit to be raised and then some cleanup should happen if it can.
In a test, i.e. standard unittest.TestCase, I would like to test that this cleanup happens and the loop exits. However, I'm stuck on even getting the signal to be triggered / SystemExit to be raised.
# service.py
import signal
import sys
import time
def main():
def signal_handler(signalnum, _):
# How to get this to block to run in a test?
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)
while True:
try:
print("Some action here")
time.sleep(10)
except SystemExit:
# How to get this to block to run in a test?
print("Some cleanup")
break
if __name__ == '__main__':
main()
How can the code enter the SystemExit handler / signal handler in the test environment? An alternative pattern would also be welcome.
You can trigger a SIGINT (or any signal) from another thread after some delay, which is received in the main thread. You can then assert on its effects just as in any other test, as below.
import os
import signal
import time
import threading
import unittest
from unittest.mock import (
Mock,
patch,
)
import service
class TestService(unittest.TestCase):
#patch('service.print')
def test_signal_handling(self, mock_print):
pid = os.getpid()
def trigger_signal():
while len(mock_print.mock_calls) < 1:
time.sleep(0.2)
os.kill(pid, signal.SIGINT)
thread = threading.Thread(target=trigger_signal)
thread.daemon = True
thread.start()
service.main()
self.assertEqual(mock_print.mock_calls[1][1][0], 'Some cleanup')
if __name__ == '__main__':
unittest.main()
Let's refactor that to make it easier to test:
def loop():
try:
print("Some action here")
except:
# clean up and re-raise
print("Some cleanup")
raise
def main():
def signal_handler(signalnum, _):
# How to get this to block to run in a test?
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)
while True:
try:
loop_body()
time.sleep(10)
except SystemExit:
break
if __name__ == '__main__':
main()
This doesn't allow easy testing of the signal handling code though. However, that amount is so small, rarely changed and strongly depends on the environment, that it is possible and perhaps even better to test manually.
For clarity, it could be useful to use a context handler, which is usually a good idea when you have setup/shutdown code. You don't mention the setup code, but my Crystall Ball (tm) tells me it exists. It could then be called like this:
try:
with my_service() as service:
while True:
service.run()
sleep(10)
except SystemExit:
# perform graceful shutdown on signal
pass
I'll leave the implementation of that context manager to you, but check out contextlib, which makes it easy and fun.
I have a GUI thread and Main thread. After closing a window I have method called inside the GUI thread. I would like to propagate this to Main thread to end its work. Main thread is doing several steps, so I am able to set stop_event, but I do not want to check after each line of code for Main thread if stop_event is set.
Thank you for your advices.
If your purpose is just to terminate main thread from the child thread, try the below.
import threading
import signal
import time
import os
def main():
threading.Thread(target=child).start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt as e:
# KeyboardInterrupt happens by `signal.SIGINT` from the child thread.
print('Main thread handle something before it exits')
print('End main')
def child():
print('Run child')
time.sleep(2)
# Send a signal `signal.SIGINT` to main thread.
# The signal only head for main thread.
os.kill(os.getpid(), signal.SIGINT)
print('End child')
if __name__ == '__main__':
main()
I need to run a gstreamer pipeline to perform video streaming. The GStreamer pipeline requires a GObject.MainLoop object which has a run() method that does not terminate until quit() is called.
For this I create a process (P2) from my main application process (P1), which runs the GObject.MainLoop instance in its main thread. The problem is that loop goes on indefinitly within the process P2 and I'm unable to exit/quit it from the main application process (P1).
Following is the section of code that might help understanding the scenario.
'''
start() spawns a new process P2 that runs Mainloop within its main thread.
stop() is called from P1, but does not quit the Mainloop. This is probably because
processes do not have shared memory
'''
from multiprocessing import Process
import gi
from gi.repository import GObject
class Main:
def __init__(self):
self.process = None
self.loop = GObject.MainLoop()
def worker(self):
self.loop.run()
def start(self):
self.process=Process(target=self.worker, args=())
self.process.start()
def stop(self):
self.loop.quit()
Next, I tried using a multiprocessing Queue for sharing the 'loop' variable between the processes, but am still unable to quit the mainloop.
'''
start() spawns a new process and puts the loop object in a multiprocessing Queue
stop() calls get() from the loop and calls the quit() method, though it still does not quit the mainloop.
'''
from multiprocessing import Process, Queue
import gi
from gi.repository import GObject
class Main:
def __init__(self):
self.p=None
self.loop = GObject.MainLoop()
self.queue = Queue()
def worker(self):
self.queue.put(self.loop)
self.loop.run()
def start(self):
self.p=Process(target=self.worker, args=())
self.p.start()
def stop(self):
# receive loop instance shared by Child Process
loop=self.queue.get()
loop.quit()
How do I call the quit method for the MainLoop object which is only accessible within the child Process P2?
Ok firstly we need to be using threads not processes. Processes will be in a different address space.
What is the difference between a process and a thread?
Try passing the main loop object to a separate thread that does the actual work. This will make your main method in to nothing but a basic GLib event processing loop, but that is fine and the normal behavior in many GLib applciations.
Lastly, we need to handle the race condition of the child process finishing its work before the main loop activates. We do this with the while not loop.is_running() snippet.
from threading import Thread
import gi
from gi.repository import GObject
def worker(loop):
while not loop.is_running():
print("waiting for loop to run")
print("working")
loop.quit()
print("quitting")
class Main:
def __init__(self):
self.thread = None
self.loop = GObject.MainLoop()
def start(self):
self.thread=Thread(target=worker, args=(self.loop,))
self.thread.start()
self.loop.run()
def main():
GObject.threads_init()
m = Main()
m.start()
if __name__ =='__main__' : main()
I extended multiprocessing.Process module in my class Main and overridden its run() method to actually run the GObject.Mainloop instance inside another thread (T1) instead of its main thread. And then implemented a wait-notify mechanism which will make the main thread of Process (P2) to go under wait-notify loop and used multiprocessing.Queue to forward messages to the main thread of P2 and P2 will be notified at the same time. For eg, stop() method, which will send the quit message to P2 for which a handler is defined in the overridden run() method.
This module can be extended to parse any number of messages to the Child Process provided their handlers are to be defined also.
Following is the code snippet which I used.
from multiprocessing import Process, Condition, Queue
from threading import Thread
import gi
from gi.repository import GObject
loop=GObject.MainLoop()
def worker():
loop.run()
class Main(Process):
def __init__(self, target=None, args=()):
self.target=target
self.args=tuple(args)
print self.args
self.message_queue = Queue()
self.cond = Condition()
self.thread = None
self.loop = GObject.MainLoop()
Process.__init__(self)
def run(self):
if self.target:
self.thread = Thread(target=self.target, args=())
print "running target method"
self.thread.start()
while True:
with self.cond:
self.cond.wait()
msg = self.message_queue.get()
if msg == 'quit':
print loop.is_running()
loop.quit()
print loop.is_running()
break
else:
print 'message received', msg
def send_message(self, msg):
self.message_queue.put(msg)
with self.cond:
self.cond.notify_all()
def stop(self):
self.send_message("quit")
self.join()
def func1(self):
self.send_message("msg 1") # handler is defined in the overridden run method
# few others functions which will send unique messages to the process, and their handlers
# are defined in the overridden run method above
This method is working fine for my scenerio but suggestions are welcomed if there is a better way to do the same.
I have the following two threads:
myThread = threading.Thread(target=sender.mainloop.run, daemon=True)
myThread.start()
myThread2 = threading.Thread(target=receiver.mainloop.run, daemon=True)
myThread2.start()
The targets are GObject.Mainloop() methods.
Afterwards my main program is in an infinite loop.
My problem is that when the execution is terminated by CTRL-C, Keyboardexception is raised for both threads, but the main program does not terminate.
Any ideas how could both the main program and the two threads be terminated by CTRL-C?
ctrl-c issues a SIGINT signal, which you can capture in your main thread for a callback. You can then run whatever shutdown code you want in the callback, maybe a sender/receiver.mainloop.quit() or something.
import threading
import signal
import sys
def loop():
while True:
pass
def exit(signal, frame):
sys.exit(0)
myThread = threading.Thread(target=loop)
myThread.daemon = True
myThread.start()
myThread2 = threading.Thread(target=loop)
myThread2.daemon = True
myThread2.start()
signal.signal(signal.SIGINT, exit)
loop()
I try to stop a gobject.MainLoop() after a few seconds.
I don't know if it's possible to set a timeout to this kind of loop, it would be perfect but I have not found that.
So, I tried to workaround this with threading but unfortunately, the main loop block others threads.
Here my code (I'm working with python 2.7):
import MediaCenter_dbusConfig
import dbus
import gobject
from dbus.mainloop.glib import DBusGMainLoop
from time import sleep
from threading import Thread
mainloop=0
class Timeout(Thread):
global mainloop
def __init__(self):
Thread.__init__(self)
def run(self):
global mainloop
i = 0
while i < 30:
sleep(1)
i += 1
mainloop.quit()
class Loop(Thread):
global mainloop
def __init__(self):
Thread.__init__(self)
def run(self):
global mainloop
sleep(5)
mainloop.run()
def catchall_detectedDevicePopUp_from_willShowPopup_signals_handler(popup_string):
global mainloop
if(popup_string == "DetectedDevicePopUp.qml") :
print(popup_string)
mainloop.quit()
def detectedDevicePopUp_detector() :
global mainloop
dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
bus=MediaCenter_dbusConfig.init() # basicly do a dbus.bus.BusConnection()
bus.add_signal_receiver(catchall_detectedDevicePopUp_from_willShowPopup_signals_handler, dbus_interface = "com.orange.mediacenter.apimanager", signal_name = "willShowPopup")
mainloop = gobject.MainLoop()
thread1 = Timeout()
thread2 = Loop()
thread1.start()
thread2.start()
thread1.join()
thread2.join()
Here I call detectedDevicePopUp_detector(). I'm waiting for a signal named willShowPopup. If I received a signal, I want to stop the loop and continue my program, and after 30s, if I have not received any signal, I want the same thing (stop the loop and continue my program) but here it doesn't work, my Timeout thread is blocked by my Loop thread.
Clarification: I can not edit the signals sent (I test an application).
Any ideas ?
As I understood the question, threading is not really wanted. Below is an example that uses gobject.timeout_add to add a maximum time that the mainloop will run if there is no signal to stop it:
import gobject
import dbus
import dbus.service
from dbus.mainloop.glib import DBusGMainLoop
DBusGMainLoop(set_as_default=True)
OPATH = "/com/example/StopLoop"
IFACE = "com.example.StopLoop"
BUS_NAME = "com.example.StopLoop"
TIMEOUT = 30 * 1000
class Example(dbus.service.Object):
def __init__(self, loop):
self.loop = loop
bus = dbus.SessionBus()
bus.request_name(BUS_NAME)
bus_name = dbus.service.BusName(BUS_NAME, bus=bus)
dbus.service.Object.__init__(self, bus_name, OPATH)
# Add a timeout for how long to run mainloop
# if no signal is received
self.setup_timeout(TIMEOUT)
# Listen to the "Stop" signal
self.listen_for_signal(bus)
def setup_timeout(self, timeout):
gobject.timeout_add(timeout, self.handler)
def listen_for_signal(self, bus):
bus.add_signal_receiver(self.handler, "Stop")
def handler(self):
# This handler is used for both timeout and signal
self.loop.quit()
if __name__ == "__main__":
loop = gobject.MainLoop()
a = Example(loop)
loop.run()
print "Exited mainloop, continuing program..."
If the Stop signal is received e.g. by doing:
dbus-send --session --type=signal --dest=com.example.StopLoop /com/example/StopLoop com.example.StopLoop.Stop
The mainloop will exit and the code will continue from where loop.run() was called.
If no signal is received the mainloop will be stopped by the timeout (30 seconds in this case) and continue from where loop.run() was called.