(there is a follow up to this question here)
I am working on trying to write a Python based Init system for Linux but I'm having an issue getting signals to my Python init script. From the 'man 2 kill' page:
The only signals that can be sent to process ID 1, the init process, are those for which init has explicitly installed signal handlers.
In my Python based Init, I have a test function and a signal handler setup to call that function:
def SigTest(SIG, FRM):
print "Caught SIGHUP!"
signal.signal(signal.SIGHUP, SigTest)
From another TTY (the init script executes sh on another tty) if I send a signal, it is completely ignored and the text is never printed. kill -HUP 1
I found this issue because I wrote a reaping function for my Python init to reap its child processes as they die, but they all just zombied, it took awhile to figure out Python was never getting the SIGCHLD signal. Just to ensure my environment is sane, I wrote a C program to fork and have the child send PID 1 a signal and it did register.
How do I install a signal handler the system will acknowledge if signal.signal(SIG, FUNC) isn't working?
Im going to try using ctypes to register my handler with C code and see if that works, but I rather a pure Python answer if at all possible.
Ideas?
( I'm not a programmer, Im really in over my head here :p )
Test code below...
import os
import sys
import time
import signal
def SigTest(SIG, FRM):
print "SIGINT Caught"
print "forking for ash"
cpid = os.fork()
if cpid == 0:
os.closerange(0, 4)
sys.stdin = open('/dev/tty2', 'r')
sys.stdout = open('/dev/tty2', 'w')
sys.stderr = open('/dev/tty2', 'w')
os.execv('/bin/ash', ('ash',))
print "ash started on tty2"
signal.signal(signal.SIGHUP, SigTest)
while True:
time.sleep(5.0)
Signal handlers mostly work in Python. But there are some problems. One is that your handler won't run until the interpreter re-enters it's bytecode interpreter. if your program is blocked in a C function the signal handler is not called until it returns. You don't show the code where you are waiting. Are you using signal.pause()?
Another is that if you are in a system call you will get an exception after the singal handler returns. You need to wrap all system calls with a retry handler (at least on Linux).
It's interesting that you are writing an init replacement... That's something like a process manager. The proctools code might interest you, since it does handle SIGCHLD.
By the way, this code:
import signal
def SigTest(SIG, FRM):
print "SIGINT Caught"
signal.signal(signal.SIGHUP, SigTest)
while True:
signal.pause()
Does work on my system.
Related
I have a piece of Python code as below:
import sys
import signal
import atexit
def release():
print "Release resources..."
def sigHandler(signo, frame):
release()
sys.exit(0)
if __name__ == "__main__":
signal.signal(signal.SIGTERM, sigHandler)
atexit.register(release)
while True:
pass
The real code is far more complex than this snippets, but the structures are the same: i.e. main function maintains an infinite loop.
I need a signal callback to release the resources occupied, like DB handle.
Then I add a SIGTERM handler, in case the server is killed, which simply invoke the release function and then exit the process.
The atexit one aims to handling process complete successfully.
Now I have a problem I just want release to be invoked only once when the process is killed. Any improvement on my code?
Well, according to the documentation atexit handlers aren't executed if the program is killed by a signal not handled by Python, or in case of internal error, or if os._exit() is called. So I would use something like this (almost copied your code):
import sys
import signal
import atexit
def release():
print "Release resources..."
def sigHandler(signo, frame):
sys.exit(0)
if __name__ == "__main__":
atexit.register(release)
signal.signal(signal.SIGTERM, sigHandler)
while True:
pass
I've checked release() is called once and only once in case of both TERM (issued externally) and INTR signals (Ctrl-C from keyboard). If you need, you may install more signal handlers (e.g. for HUP etc). If you need "a more graceful shutdown", you should find a way to gracefully break the loop and/or install external "shutdown handlers" (in case of SIGKILL you won't get a chance to cleanly release resources) or simply make your application be ACID.
I'm using a commercial application that uses Python as part of its scripting API. One of the functions provided is something called App.run(). When this function is called, it starts a new Java process that does the rest of the execution. (Unfortunately, I don't really know what it's doing under the hood as the supplied Python modules are .pyc files, and many of the Python functions are SWIG generated).
The trouble I'm having is that I'm building the App.run() call into a larger Python application that needs to do some guaranteed cleanup code (closing a database, etc.). Unfortunately, if the subprocess is interrupted with Ctrl+C, it aborts and returns to the command line without returning control to the main Python program. Thus, my cleanup code never executes.
So far I've tried:
Registering a function with atexit... doesn't work
Putting cleanup in a class __del__ destructor... doesn't work. (App.run() is inside the class)
Creating a signal handler for Ctrl+C in the main Python app... doesn't work
Putting App.run() in a Thread... results in a Memory Fault after the Ctrl+C
Putting App.run() in a Process (from multiprocessing)... doesn't work
Any ideas what could be happening?
This is just an outline- but something like this?
import os
cpid = os.fork()
if not cpid:
# change stdio handles etc
os.setsid() # Probably not needed
App.run()
os._exit(0)
os.waitpid(cpid)
# clean up here
(os.fork is *nix only)
The same idea could be implemented with subprocess in an OS agnostic way. The idea is running App.run() in a child process and then waiting for the child process to exit; regardless of how the child process died. On posix, you could also trap for SIGCHLD (Child process death). I'm not a windows guru, so if applicable and subprocess doesn't work, someone else will have to chime in here.
After App.run() is called, I'd be curious what the process tree looks like. It's possible its running an exec and taking over the python process space. If thats happening, creating a child process is the only way I can think of trapping it.
If try: App.run() finally: cleanup() doesn't work; you could try to run it in a subprocess:
import sys
from subprocess import call
rc = call([sys.executable, 'path/to/run_app.py'])
cleanup()
Or if you have the code in a string you could use -c option e.g.:
rc = call([sys.executable, '-c', '''import sys
print(sys.argv)
'''])
You could implement #tMC's suggestion using subprocess by adding
preexec_fn=os.setsid argument (note: no ()) though I don't see how creating a process group might help here. Or you could try shell=True argument to run it in a separate shell.
You might give another try to multiprocessing:
import multiprocessing as mp
if __name__=="__main__":
p = mp.Process(target=App.run)
p.start()
p.join()
cleanup()
Are you able to wrap the App.Run() in a Try/Catch?
Something like:
try:
App.Run()
except (KeyboardInterrupt, SystemExit):
print "User requested an exit..."
cleanup()
This is a follow up to my other post Installing signal handler with Python. In short, Linux blocks all signals to PID 1 (including SIGKILL) unless Init has installed a signal handler for a particular signal; as to prevent kernel panic if someone were to send a termination signal to PID1. The issue I've been having, is it would seem that the signal module in Python doesn't install signal handlers in a way the system recognises. My Python Init script was seemingly, completely ignoring all signals as I think they were being blocked.
I seem to have found a solution; using ctypes to install the signal handlers with the signal() function in libc (in this case uClibc). Below is a python based test init. It opens a shell on TTY2 from which I can send signals to PID1 for testing. It seems to work in the KVM im using for testing (I'm willing to share the VM with anyone interested)
Is this the best way around this issue? Is there a 'better' way to install the signal handlers without the signal module? (I am not at all concerned with portably)
Is this a bug in Python?
#!/usr/bin/python
import os
import sys
import time
from ctypes import *
def SigHUP():
print "Caught SIGHUP"
return 0
def SigCHLD():
print "Caught SIGCHLD"
return 0
SIGFUNC = CFUNCTYPE(c_int)
SigHUPFunc = SIGFUNC(SigHUP)
SigCHLDFunc = SIGFUNC(SigCHLD)
libc = cdll.LoadLibrary('libc.so.0')
libc.signal(1, SigHUPFunc) # 1 = SIGHUP
libc.signal(17, SigCHLDFunc) # 17 = SIGCHLD
print "Mounting Proc: %s" % libc.mount(None, "/proc", "proc", 0, None)
print "forking for ash"
cpid = os.fork()
if cpid == 0:
os.closerange(0, 4)
sys.stdin = open('/dev/tty2', 'r')
sys.stdout = open('/dev/tty2', 'w')
sys.stderr = open('/dev/tty2', 'w')
os.execv('/bin/ash', ('ash',))
print "ash started on tty2"
print "sleeping"
while True:
time.sleep(0.01)
I did a bit of debugging under KVM and I found that the kernel is delivering signals to pid 1 when the signal handlers are installed by the standard signal module. However, when the signal is received "something" causes a clone of the process to be spawned, rather than printing the expected output.
Here is the strace output when I send HUP to the non-working init.sig-mod:
Which results in a new process running (pid 23) which is a clone of init.sig-mod:
I didn't have time to dig deeper into the cause, but this narrows things further. Probably something to do with Python's signal delivery logic (it registers a C hook which invokes your bytecode function when called). The ctypes technique bypasses this. The relevant Python source files are Python/pythonrun.c and Modules/signalmodule.c, in case you want to take a closer look.
Old Info -- I'm not sure this will solve your problem, but might get you closer. I
compared these different ways signal handlers are installed:
Installing a handler via Python's signal module.
Upstart's signal handlers.
Using ctypes to call the signal() syscall directly.
Some quick tests in C.
Both the ctypes-invoked signal() system call and Upstart's sigaction()
syscalls set the SA_RESTART flag when the handler is registered. Setting
this flag indicates that when a signal is received while the process is
executing or blocking inside certain syscalls (read, write, wait,
nanosleep, etc), after the signal handler completes the syscall should be
automatically restarted. The application won't be aware of this.
When Python's signal module registers a handler, it zeros the SA_RESTART
flag by calling siginterrupt(signum, 1). This says to the system "when a
system call is interrupted by a signal, after the signal handler completes
set errno to EINTR and return from the syscall". This leaves it up to the developer to
handle this and decide whether to restart the system call.
You can set the SA_RESTART flag by registering your signal this way:
import signal
signal.signal(signal.SIGHUP, handler)
signal.siginterrupt(signal.SIGHUP, False)
The issue was a compatibility issue with Python compiled against uClibc 0.9.31 with old linux threads. Compiling against 0.9.32-rc3 and using NPTL has fixed the issue.
I am currently working on a wrapper for a dedicated server running in the shell. The wrapper spawns the server process via subprocess and observes and reacts to its output.
The dedicated server must be explicitly given a command to shut down gracefully. Thus, CTRL-C must not reach the server process.
If I capture the KeyboardInterrupt exception or overwrite the SIGINT-handler in python, the server process still receives the CTRL-C and stops immediately.
So my question is:
How to prevent subprocesses from receiving CTRL-C / Control-C / SIGINT?
Somebody in the #python IRC-Channel (Freenode) helped me by pointing out the preexec_fn parameter of subprocess.Popen(...):
If preexec_fn is set to a callable
object, this object will be called in
the child process just before the
child is executed. (Unix only)
Thus, the following code solves the problem (UNIX only):
import subprocess
import signal
def preexec_function():
# Ignore the SIGINT signal by setting the handler to the standard
# signal handler SIG_IGN.
signal.signal(signal.SIGINT, signal.SIG_IGN)
my_process = subprocess.Popen(
["my_executable"],
preexec_fn = preexec_function
)
Note: The signal is actually not prevented from reaching the subprocess. Instead, the preexec_fn above overwrites the signal's default handler so that the signal is ignored. Thus, this solution may not work if the subprocess overwrites the SIGINT handler again.
Another note: This solution works for all sorts of subprocesses, i.e. it is not restricted to subprocesses written in Python, too. For example the dedicated server I am writing my wrapper for is in fact written in Java.
Combining some of other answers that will do the trick - no signal sent to main app will be forwarded to the subprocess.
import os
from subprocess import Popen
def preexec(): # Don't forward signals.
os.setpgrp()
Popen('whatever', preexec_fn = preexec)
you can do something like this to make it work in windows and unix:
import subprocess
import sys
def pre_exec():
# To ignore CTRL+C signal in the new process
signal.signal(signal.SIGINT, signal.SIG_IGN)
if sys.platform.startswith('win'):
#https://msdn.microsoft.com/en-us/library/windows/desktop/ms684863(v=vs.85).aspx
#CREATE_NEW_PROCESS_GROUP=0x00000200 -> If this flag is specified, CTRL+C signals will be disabled
my_sub_process=subprocess.Popen(["executable"], creationflags=0x00000200)
else:
my_sub_process=subprocess.Popen(["executable"], preexec_fn = pre_exec)
After an hour of various attempts, this works for me:
process = subprocess.Popen(["someprocess"], creationflags=subprocess.DETACHED_PROCESS | subprocess.CREATE_NEW_PROCESS_GROUP)
It's solution for windows.
Try setting SIGINT to be ignored before spawning the subprocess (reset it to default behavior afterward).
If that doesn't work, you'll need to read up on job control and learn how to put a process in its own background process group, so that ^C doesn't even cause the kernel to send the signal to it in the first place. (May not be possible in Python without writing C helpers.)
See also this older question.
My python script intercepts the SIGINT signal with the signal process module to prevent premature exit, but this signal is passed to a subprocess that I open with Popen. is there some way to prevent passing this signal to the subprocess so that it also is not exited prematurely when the user presses ctrl-c?
Signal handlers are inherited when you start a subprocess, so if you use the signal module to ignore SIGINT (signal.signal(signal.SIGINT, signal.SIG_IGN)), then your child process automatically will also.
There are two important caveats, though:
You have to set the ignore handler before you spawn the child process
Custom signal handlers are reset to the default handlers, since the child process won't have access to the handler code to run it.
So if you need to customise your handling of SIGINT rather than just ignoring it, you probably want to temporarily ignore SIGINT while you spawn your child process, then (re)set your custom signal handler.
If you're trying to catch SIGINT and set a flag so you can exit at a safe point rather than immediately, remember that when you get to that safe point your code will have to manually clean up its descendants, since your child process and any processes it starts will be ignoring the SIGINT.
You are able to re-assign the role of ctrl-c using the tty module, which allows you to manipulate the assignment of signals. Be warned, however, that unless you put them back the way they were before you modified them, they will persist for the shell's entire session, even after the program exits.
Here is a simple code snippet to get you started that stores your old tty settings, re-assigns ctrl-c to ctrl-x, and then restores your previous tty settings upon exit.
import sys
import tty
# Back up previous tty settings
stdin_fileno = sys.stdin.fileno()
old_ttyattr = tty.tcgetattr(stdin_fileno)
try:
print 'Reassigning ctrl-c to ctrl-x'
# Enter raw mode on local tty
tty.setraw(stdin_fileno)
raw_ta = tty.tcgetattr(stdin_fileno)
raw_ta[tty.LFLAG] |= tty.ISIG
raw_ta[tty.OFLAG] |= tty.OPOST | tty.ONLCR
# ^X is the new ^C, set this to 0 to disable it entirely
raw_ta[tty.CC][tty.VINTR] = '\x18'
# Set raw tty as active tty
tty.tcsetattr(stdin_fileno, tty.TCSANOW, raw_ta)
# Dummy program loop
import time
for _ in range(5):
print 'doing stuff'
time.sleep(1)
finally:
print 'Resetting ctrl-c'
# Restore previous tty no matter what
tty.tcsetattr(stdin_fileno, tty.TCSANOW, old_ttyattr)
For python 2 codebase: subprocess is broken.
The right thing is
import subprocess32 as subprocess
See subprocess32
This is a backport of the Python 3 subprocess module for use on Python
2. This code has not been tested on Windows or other non-POSIX platforms.