I want to customize a signal handler on windows, and then use os.kill in other processes to notify the signal to execute the corresponding handler. But the result is that the signal still executes the default handler.
I wrote the following test code, the same code, in the window, did not print anything. In Linux, 'hi' is printed.
import os
import signal
def sigint(*args):
print('hi')
signal.signal(signal.SIGINT , sigint)
os.kill(os.getpid(), signal.SIGINT)
Why lead to such different results?
Related
I have a piece of Python code as below:
import sys
import signal
import atexit
def release():
print "Release resources..."
def sigHandler(signo, frame):
release()
sys.exit(0)
if __name__ == "__main__":
signal.signal(signal.SIGTERM, sigHandler)
atexit.register(release)
while True:
pass
The real code is far more complex than this snippets, but the structures are the same: i.e. main function maintains an infinite loop.
I need a signal callback to release the resources occupied, like DB handle.
Then I add a SIGTERM handler, in case the server is killed, which simply invoke the release function and then exit the process.
The atexit one aims to handling process complete successfully.
Now I have a problem I just want release to be invoked only once when the process is killed. Any improvement on my code?
Well, according to the documentation atexit handlers aren't executed if the program is killed by a signal not handled by Python, or in case of internal error, or if os._exit() is called. So I would use something like this (almost copied your code):
import sys
import signal
import atexit
def release():
print "Release resources..."
def sigHandler(signo, frame):
sys.exit(0)
if __name__ == "__main__":
atexit.register(release)
signal.signal(signal.SIGTERM, sigHandler)
while True:
pass
I've checked release() is called once and only once in case of both TERM (issued externally) and INTR signals (Ctrl-C from keyboard). If you need, you may install more signal handlers (e.g. for HUP etc). If you need "a more graceful shutdown", you should find a way to gracefully break the loop and/or install external "shutdown handlers" (in case of SIGKILL you won't get a chance to cleanly release resources) or simply make your application be ACID.
An application that asks for a keyboard interrupt. How can I send for a keyboard interrupt programmatically? I need it for automation.
Like <C-c> or <C-x>
KeyboardInterrupt
Code running in a separate thread can cause a KeyboardInterrupt to be generated in the main thread by calling thread.interrupt_main().
See https://docs.python.org/2/library/thread.html#thread.interrupt_main
Since you mention automation I assume you want a SendKeys for Python. Try this: http://rutherfurd.net/python/sendkeys/
My suggestion to solve this problem is to use the following code pattern. I used it to programmatically start a tensorboard server and shut it down by sending a CTRL-C when the object it belongs to is deleted. Generally speaking, this should work for any example that provokes a subprocess that is supposed to be send a KeyBoardInterrupt:
Import signal and subprocess
import signal
import subprocess
Create the subprocess using subprocess.Popen. Important: set the creationflags parameter to subprocess.CREATE_NEW_PROCESS_GROUP. This is necessary to later be able to send the KeyboardInterrupt event.
command= <enter your command that is supposed to be run in different process as a string>
process = subprocess.Popen(command.split(),stdout=subprocess.PIPE,stdin=subprocess.PIPE,creationflags=subprocess.CREATE_NEW_PROCESS_GROUP)
Wherever you want to send the keyboardInterrupt, do the following:
process.send_signal(signal.CTRL_C_EVENT)
That is it! Please see the the official subprocess documentation for insights on why the creationflags parameter of popen has to be set that way.
This is how the code looks for my example in a less generic way:
import signal
import subprocess
import time
class ExperimentTracker():
def __init__(self):
self.tensorboard_process=None
def __del__(self):
#shutdown tensorboard server and terminate process
self.tensorboard_process.send_signal(signal.CTRL_C_EVENT)
time.sleep(0.2)
self.tensorboard_process.kill()
def launch_tensorboard(self):
#launch tensorboard
bashCommand= "tensorboard --logdir runs"
self.tensorboard_process = subprocess.Popen(bashCommand.split(),stdout=subprocess.PIPE,stdin=subprocess.PIPE,creationflags=subprocess.CREATE_NEW_PROCESS_GROUP)
time.sleep(2) #sleep for 2 seconds to give tensorboard time to be launched
If say you want to run a program via shell ./program. In Linux what you could do is:
# Made a function to handle more complex programs which require multiple inputs.
run(){
./program
}
# Making a child process that will run the program while you stop it later.
run &
childPid=($!)
# Process id of the program which you want to interrupt (via `run`).
# Time after which you want to interrupt
sleep 5
# Actual command to send the interrupt
kill -SIGINT $childPid
Let me know if it works in windows as well.
This is a follow up to my other post Installing signal handler with Python. In short, Linux blocks all signals to PID 1 (including SIGKILL) unless Init has installed a signal handler for a particular signal; as to prevent kernel panic if someone were to send a termination signal to PID1. The issue I've been having, is it would seem that the signal module in Python doesn't install signal handlers in a way the system recognises. My Python Init script was seemingly, completely ignoring all signals as I think they were being blocked.
I seem to have found a solution; using ctypes to install the signal handlers with the signal() function in libc (in this case uClibc). Below is a python based test init. It opens a shell on TTY2 from which I can send signals to PID1 for testing. It seems to work in the KVM im using for testing (I'm willing to share the VM with anyone interested)
Is this the best way around this issue? Is there a 'better' way to install the signal handlers without the signal module? (I am not at all concerned with portably)
Is this a bug in Python?
#!/usr/bin/python
import os
import sys
import time
from ctypes import *
def SigHUP():
print "Caught SIGHUP"
return 0
def SigCHLD():
print "Caught SIGCHLD"
return 0
SIGFUNC = CFUNCTYPE(c_int)
SigHUPFunc = SIGFUNC(SigHUP)
SigCHLDFunc = SIGFUNC(SigCHLD)
libc = cdll.LoadLibrary('libc.so.0')
libc.signal(1, SigHUPFunc) # 1 = SIGHUP
libc.signal(17, SigCHLDFunc) # 17 = SIGCHLD
print "Mounting Proc: %s" % libc.mount(None, "/proc", "proc", 0, None)
print "forking for ash"
cpid = os.fork()
if cpid == 0:
os.closerange(0, 4)
sys.stdin = open('/dev/tty2', 'r')
sys.stdout = open('/dev/tty2', 'w')
sys.stderr = open('/dev/tty2', 'w')
os.execv('/bin/ash', ('ash',))
print "ash started on tty2"
print "sleeping"
while True:
time.sleep(0.01)
I did a bit of debugging under KVM and I found that the kernel is delivering signals to pid 1 when the signal handlers are installed by the standard signal module. However, when the signal is received "something" causes a clone of the process to be spawned, rather than printing the expected output.
Here is the strace output when I send HUP to the non-working init.sig-mod:
Which results in a new process running (pid 23) which is a clone of init.sig-mod:
I didn't have time to dig deeper into the cause, but this narrows things further. Probably something to do with Python's signal delivery logic (it registers a C hook which invokes your bytecode function when called). The ctypes technique bypasses this. The relevant Python source files are Python/pythonrun.c and Modules/signalmodule.c, in case you want to take a closer look.
Old Info -- I'm not sure this will solve your problem, but might get you closer. I
compared these different ways signal handlers are installed:
Installing a handler via Python's signal module.
Upstart's signal handlers.
Using ctypes to call the signal() syscall directly.
Some quick tests in C.
Both the ctypes-invoked signal() system call and Upstart's sigaction()
syscalls set the SA_RESTART flag when the handler is registered. Setting
this flag indicates that when a signal is received while the process is
executing or blocking inside certain syscalls (read, write, wait,
nanosleep, etc), after the signal handler completes the syscall should be
automatically restarted. The application won't be aware of this.
When Python's signal module registers a handler, it zeros the SA_RESTART
flag by calling siginterrupt(signum, 1). This says to the system "when a
system call is interrupted by a signal, after the signal handler completes
set errno to EINTR and return from the syscall". This leaves it up to the developer to
handle this and decide whether to restart the system call.
You can set the SA_RESTART flag by registering your signal this way:
import signal
signal.signal(signal.SIGHUP, handler)
signal.siginterrupt(signal.SIGHUP, False)
The issue was a compatibility issue with Python compiled against uClibc 0.9.31 with old linux threads. Compiling against 0.9.32-rc3 and using NPTL has fixed the issue.
(there is a follow up to this question here)
I am working on trying to write a Python based Init system for Linux but I'm having an issue getting signals to my Python init script. From the 'man 2 kill' page:
The only signals that can be sent to process ID 1, the init process, are those for which init has explicitly installed signal handlers.
In my Python based Init, I have a test function and a signal handler setup to call that function:
def SigTest(SIG, FRM):
print "Caught SIGHUP!"
signal.signal(signal.SIGHUP, SigTest)
From another TTY (the init script executes sh on another tty) if I send a signal, it is completely ignored and the text is never printed. kill -HUP 1
I found this issue because I wrote a reaping function for my Python init to reap its child processes as they die, but they all just zombied, it took awhile to figure out Python was never getting the SIGCHLD signal. Just to ensure my environment is sane, I wrote a C program to fork and have the child send PID 1 a signal and it did register.
How do I install a signal handler the system will acknowledge if signal.signal(SIG, FUNC) isn't working?
Im going to try using ctypes to register my handler with C code and see if that works, but I rather a pure Python answer if at all possible.
Ideas?
( I'm not a programmer, Im really in over my head here :p )
Test code below...
import os
import sys
import time
import signal
def SigTest(SIG, FRM):
print "SIGINT Caught"
print "forking for ash"
cpid = os.fork()
if cpid == 0:
os.closerange(0, 4)
sys.stdin = open('/dev/tty2', 'r')
sys.stdout = open('/dev/tty2', 'w')
sys.stderr = open('/dev/tty2', 'w')
os.execv('/bin/ash', ('ash',))
print "ash started on tty2"
signal.signal(signal.SIGHUP, SigTest)
while True:
time.sleep(5.0)
Signal handlers mostly work in Python. But there are some problems. One is that your handler won't run until the interpreter re-enters it's bytecode interpreter. if your program is blocked in a C function the signal handler is not called until it returns. You don't show the code where you are waiting. Are you using signal.pause()?
Another is that if you are in a system call you will get an exception after the singal handler returns. You need to wrap all system calls with a retry handler (at least on Linux).
It's interesting that you are writing an init replacement... That's something like a process manager. The proctools code might interest you, since it does handle SIGCHLD.
By the way, this code:
import signal
def SigTest(SIG, FRM):
print "SIGINT Caught"
signal.signal(signal.SIGHUP, SigTest)
while True:
signal.pause()
Does work on my system.
I am currently working on a wrapper for a dedicated server running in the shell. The wrapper spawns the server process via subprocess and observes and reacts to its output.
The dedicated server must be explicitly given a command to shut down gracefully. Thus, CTRL-C must not reach the server process.
If I capture the KeyboardInterrupt exception or overwrite the SIGINT-handler in python, the server process still receives the CTRL-C and stops immediately.
So my question is:
How to prevent subprocesses from receiving CTRL-C / Control-C / SIGINT?
Somebody in the #python IRC-Channel (Freenode) helped me by pointing out the preexec_fn parameter of subprocess.Popen(...):
If preexec_fn is set to a callable
object, this object will be called in
the child process just before the
child is executed. (Unix only)
Thus, the following code solves the problem (UNIX only):
import subprocess
import signal
def preexec_function():
# Ignore the SIGINT signal by setting the handler to the standard
# signal handler SIG_IGN.
signal.signal(signal.SIGINT, signal.SIG_IGN)
my_process = subprocess.Popen(
["my_executable"],
preexec_fn = preexec_function
)
Note: The signal is actually not prevented from reaching the subprocess. Instead, the preexec_fn above overwrites the signal's default handler so that the signal is ignored. Thus, this solution may not work if the subprocess overwrites the SIGINT handler again.
Another note: This solution works for all sorts of subprocesses, i.e. it is not restricted to subprocesses written in Python, too. For example the dedicated server I am writing my wrapper for is in fact written in Java.
Combining some of other answers that will do the trick - no signal sent to main app will be forwarded to the subprocess.
import os
from subprocess import Popen
def preexec(): # Don't forward signals.
os.setpgrp()
Popen('whatever', preexec_fn = preexec)
you can do something like this to make it work in windows and unix:
import subprocess
import sys
def pre_exec():
# To ignore CTRL+C signal in the new process
signal.signal(signal.SIGINT, signal.SIG_IGN)
if sys.platform.startswith('win'):
#https://msdn.microsoft.com/en-us/library/windows/desktop/ms684863(v=vs.85).aspx
#CREATE_NEW_PROCESS_GROUP=0x00000200 -> If this flag is specified, CTRL+C signals will be disabled
my_sub_process=subprocess.Popen(["executable"], creationflags=0x00000200)
else:
my_sub_process=subprocess.Popen(["executable"], preexec_fn = pre_exec)
After an hour of various attempts, this works for me:
process = subprocess.Popen(["someprocess"], creationflags=subprocess.DETACHED_PROCESS | subprocess.CREATE_NEW_PROCESS_GROUP)
It's solution for windows.
Try setting SIGINT to be ignored before spawning the subprocess (reset it to default behavior afterward).
If that doesn't work, you'll need to read up on job control and learn how to put a process in its own background process group, so that ^C doesn't even cause the kernel to send the signal to it in the first place. (May not be possible in Python without writing C helpers.)
See also this older question.