I have python code which runs continuously (collecting sensor data). It is supposed to be launched at boot using start-stop-daemon. However, I'd like to be able to kill the process gracefully, so I've started from the advice in the post How to process SIGTERM signal gracefully? and put my main loop in a separate thread. I'd like to be able to gracefully shut it down both when it is running as a daemon (the start-stop-daemon will send a kill signal) and when I launch it briefly for testing in a terminal myself (me pressing ctrl-c).
However, the signal handler doesn't seem to be called if I kill the process (even without using the thread, the "done (killed)" never ends up in the file I've redirected to). And when I press ctrl-c, the collecting just continues and keeps printing data in the terminal (or to the file I am redirecting to).
What am I doing wrong in the following code?
from threading import Thread
import time, sys, signal
shutdown_flag = False #used for gracefull shutdown
def main_loop():
while not shutdown_flag:
collect_data() # contains some print "data" statements
time.sleep(5)
print "done (killed)"
def sighandler(signum, frame):
print 'signal handler called with signal: %s ' % signum
global shutdown_flag
shutdown_flag = True
def main(argv=None):
signal.signal(signal.SIGTERM, sighandler) # so we can handle kill gracefully
signal.signal(signal.SIGINT, sighandler) # so we can handle ctrl-c
try:
Thread(target=main_loop, args=()).start()
except Exception, reason:
print reason
if __name__ == '__main__':
sys.exit(main(sys.argv))
You are terminating your main thread with this statement:
if __name__ == '__main__':
sys.exit(main(sys.argv))
So your signal handler never gets to run. The signal handler is part of the main thread not the main_loop thread you created. So once the main thread exits there's no signal handler function to call anymore.
You need something like this:
def sighandler(signum, frame):
print 'signal handler called with signal: %s ' % signum
global shutdown_flag
shutdown_flag = True
sys.exit() # make sure you add this so the main thread exits as well.
if __name__ == '__main__':
main(sys.argv)
while 1: # this will force your main thread to live until you terminate it.
time.sleep(1)
A simple test to see how many threads are running in your program is this:
def main_loop():
while not shutdown_flag:
collect_data() # contains some print "data" statements
time.sleep(5)
import threading
print threading.enumerate()
print "done (killed)"
Related
I have a program that has quite a few functions, each running on a separate thread.
When the user presses Ctrl+C, only 1 thread crashes with an exception, but because of this, the whole program may not work correctly.
Of course, I can write this construction in each function:
try:
do_something()
except KeyboardInterrupt as e:
pass
but, as I said, there are many functions, perhaps there is an option not to prescribe this construction in each function?
Or is it possible to disable Ctrl+C interrupt in cmd settings?
For example, in the registry. The program creates its own registry key in HKEY_CURRENT_USER\Console\MyProgrammKey
UPD 1
signal.signal(signal.SIGINT, signal.SIG_IGN)
It helped in almost all cases except one: a thread that has an infinite loop with the input() function anyway interrupts.
UPD 2
Here is a sample code
import signal, time
from threading import Thread
def one():
while True:
inp = input("INPUT: ")
def two():
while True:
print("I just printing...")
time.sleep(1)
if __name__ == '__main__':
signal.signal(signal.SIGINT, signal.SIG_IGN)
Thread(target=one).start()
Thread(target=two).start()
UPD 3
Screenshot of exception.
Ctrl+C will send SIGINT signal to program, so you could define a global signal handler to ignore that SIGINT, something like next:
test.py:
import signal, os, time
def handler(signum, frame):
pass
signal.signal(signal.SIGINT, handler)
time.sleep(10)
print("done")
During the program run, if you input Ctrl+c, the program will ignore it, and continue to run, finally print done:
$ python3 test.py
^Cdone
I am developing some code which I need to gracefully shutdown when a sigterm signal is sent from the command line in unix. I found this example https://stackoverflow.com/a/31464349/7019148 which works great, but there's one problem with it.
Code:
import signal
import time
class GracefulKiller:
def __init__(self):
signal.signal(signal.SIGTERM, self.exit_gracefully)
self.kill_now = False
def exit_gracefully(self, signum, frame):
self.kill_now = True
def run_something(self):
print("starting")
time.sleep(5)
print("ending")
if __name__ == '__main__':
killer = GracefulKiller()
print(os.getpid())
while True:
killer.run_something()
if killer.kill_now:
break
print("End of the program. I was killed gracefully :)")
When you pass the kill command kill -15 <pid>, the run_something method is interrupted and the process killed, gracefully. However, is there a way to do this so that the run_something method can complete before the process is killed? I.e. prevent the interruption?
Desired output:
>>> starting
*kill executed during the middle sleep*
>>> ending
>>> End of the program. I was killed gracefully :)
My use case is that this will be turned into a download script and if I want to terminate the process, I would like the process to finish downloading before terminating...
thread.join() waits till the thread finishes even if an exit signal was caught.
import threading
import Queue
import time
def download_for(seconds=5):
for i in range(seconds):
print("downloading...")
time.sleep(1)
print("finished download")
download_thread = threading.Thread(target=download_for, args=(3,))
download_thread.start()
# this waits till the thread finishes even if an exit signal was received
download_thread.join()
# this would just stop the download midway
# download_for(seconds=5)
The answer is in the original question. I am just leaving this here for future Google searchers.
I never had an issue in the first place, my terminal was just having a problem printing 'ending' following the kill command.
I am writing a python script that needs to run a thread which listens to a network socket.
I'm having trouble with killing it using Ctrl+c using the code below:
#!/usr/bin/python
import signal, sys, threading
THREADS = []
def handler(signal, frame):
global THREADS
print "Ctrl-C.... Exiting"
for t in THREADS:
t.alive = False
sys.exit(0)
class thread(threading.Thread):
def __init__(self):
self.alive = True
threading.Thread.__init__(self)
def run(self):
while self.alive:
# do something
pass
def main():
global THREADS
t = thread()
t.start()
THREADS.append(t)
if __name__ == '__main__':
signal.signal(signal.SIGINT, handler)
main()
Appreciate any advise on how to catch Ctrl+c and terminate the script.
The issue is that after the execution falls off the main thread (after main() returned), the threading module will pause, waiting for the other threads to finish, using locks; and locks cannot be interrupted with signals. This is the case in Python 2.x at least.
One easy fix is to avoid falling off the main thread, by adding an infinite loop that calls some function that sleeps until some action is available, like select.select(). If you don't need the main thread to do anything at all, use signal.pause(). Example:
if __name__ == '__main__':
signal.signal(signal.SIGINT, handler)
main()
while True: # added
signal.pause() # added
It's because signals can only be caught by main thread. And here main thread ended his life long time ago (application is waiting for your thread to finish). Try adding
while True:
sleep(1)
to the end of your main() (and of course from time import sleep at the very top).
or as Kevin said:
for t in THREADS:
t.join(1) # join with timeout. Without timeout signal cannot be caught.
In python, is it possible to make use of KeyboardInterrupt or CTRL+C to print a status message, possibly like printing content of a variable and then continuing with the execution?
Or will Interrupts always kill the process?
An example of what I would like to do:
def signal_handler(signum, frame):
global interrupted
interrupted = True
while true:
update(V)
if interrupted:
print V
You can do this using a signal handler:
import signal
def sigint_handler(signum, frame):
print "my_variable =", frame.f_locals.get("my_variable", None)
signal.signal(signal.SIGINT, sigint_handler)
Now interrupting the script calls the handler, which prints the variable, fishing it out of the current stack frame. The script then continues.
It can be done. The signal library provides this functionality, and it pretty much goes the way you prototyped.
import signal
interrupted = False
def signal_handler(signum, frame):
global interrupted
interrupted = True
signal.signal(signal.SIGINT, signal_handler)
while true:
update(V)
if interrupted:
print V
Pressing ctrl+c raises KeyboardInterrupt.
Catch KeyboardInterrupt and print a message from it, or use a state variable like you have above.
See: Capture Control-C in Python
I cannot get a way to terminate a thread that is hung in a socket.recvfrom() call. For example, ctrl+c that should trigger KeyboardInterrupt exception can't be caught. Here is a script I've used for testing:
from socket import *
from threading import Thread
from sys import exit
class TestThread(Thread):
def __init__(self,host="localhost",port=9999):
self.sock = socket(AF_INET,SOCK_DGRAM)
self.sock.bind((host,port))
super(TestThread,self).__init__()
def run(self):
while True:
try:
recv_data,addr = self.sock.recvfrom(1024)
except (KeyboardInterrupt, SystemExit):
sys.exit()
if __name__ == "__main__":
server_thread = TestThread()
server_thread.start()
while True: pass
The main thread (the one that executes infinite loop) exits. However the thread that I explicitly create, keeps hanging in recvfrom().
Please, help me resolve this.
Keyboard interrupts are always caught on the main thread -- never on "child" threads. To avoid server_thread keeping the process alive when the main thread exits, do
server_thread.daemon = True
before you call server_thread.start().
BTW, your while True: pass in the main thread is needlessly burning CPU cycles. You should at least change it to something like while True: time.sleep(1.0). But that doesn't change the semantics of your code -- just gets it down from 99% CPU or so, to (I'd guess) < 5%;-).
You should open a pipe from the main thread to the network thread and 'select' on both the socket and the pipe. When you want to terminate the network thread, just send a byte through the pipe from the main thread and act accordingly in the network thread.
Just my 2 cents.