I am currently working on a wrapper for a dedicated server running in the shell. The wrapper spawns the server process via subprocess and observes and reacts to its output.
The dedicated server must be explicitly given a command to shut down gracefully. Thus, CTRL-C must not reach the server process.
If I capture the KeyboardInterrupt exception or overwrite the SIGINT-handler in python, the server process still receives the CTRL-C and stops immediately.
So my question is:
How to prevent subprocesses from receiving CTRL-C / Control-C / SIGINT?
Somebody in the #python IRC-Channel (Freenode) helped me by pointing out the preexec_fn parameter of subprocess.Popen(...):
If preexec_fn is set to a callable
object, this object will be called in
the child process just before the
child is executed. (Unix only)
Thus, the following code solves the problem (UNIX only):
import subprocess
import signal
def preexec_function():
# Ignore the SIGINT signal by setting the handler to the standard
# signal handler SIG_IGN.
signal.signal(signal.SIGINT, signal.SIG_IGN)
my_process = subprocess.Popen(
["my_executable"],
preexec_fn = preexec_function
)
Note: The signal is actually not prevented from reaching the subprocess. Instead, the preexec_fn above overwrites the signal's default handler so that the signal is ignored. Thus, this solution may not work if the subprocess overwrites the SIGINT handler again.
Another note: This solution works for all sorts of subprocesses, i.e. it is not restricted to subprocesses written in Python, too. For example the dedicated server I am writing my wrapper for is in fact written in Java.
Combining some of other answers that will do the trick - no signal sent to main app will be forwarded to the subprocess.
import os
from subprocess import Popen
def preexec(): # Don't forward signals.
os.setpgrp()
Popen('whatever', preexec_fn = preexec)
you can do something like this to make it work in windows and unix:
import subprocess
import sys
def pre_exec():
# To ignore CTRL+C signal in the new process
signal.signal(signal.SIGINT, signal.SIG_IGN)
if sys.platform.startswith('win'):
#https://msdn.microsoft.com/en-us/library/windows/desktop/ms684863(v=vs.85).aspx
#CREATE_NEW_PROCESS_GROUP=0x00000200 -> If this flag is specified, CTRL+C signals will be disabled
my_sub_process=subprocess.Popen(["executable"], creationflags=0x00000200)
else:
my_sub_process=subprocess.Popen(["executable"], preexec_fn = pre_exec)
After an hour of various attempts, this works for me:
process = subprocess.Popen(["someprocess"], creationflags=subprocess.DETACHED_PROCESS | subprocess.CREATE_NEW_PROCESS_GROUP)
It's solution for windows.
Try setting SIGINT to be ignored before spawning the subprocess (reset it to default behavior afterward).
If that doesn't work, you'll need to read up on job control and learn how to put a process in its own background process group, so that ^C doesn't even cause the kernel to send the signal to it in the first place. (May not be possible in Python without writing C helpers.)
See also this older question.
Related
EDIT: Narrowed down problem from original version, originally assumed all SIGINT overrides were being ignored, but it's actually just the subprocess one, edited to reflect this.
I'd like to have python shutdown safely when receiving the SIGINT (Ctrl+C) from systemd. However, the command sudo systemctl kill --signal=SIGINT myapp ignores my subprocess.Popen(args, stdout=PIPE, stderr=PIPE, preexec_fn = os.setsid) line, which prevents the SIGINT from going to a called process (works when NOT using systemd), and crashes my program anyways.
Here's my setup (similar to this: How can I make a python daemon handle systemd signals?):
shutdown = False
def shutdown_handler(signal, frame):
global shutdown
is_thread = frame.f_code.co_name == "my_thread_func"
if shutdown:
logging.info("Force shutdown for process {0}".format(os.getpid()))
raise KeyboardInterrupt
else:
shutdown = True
if not is_thread:
logging.info("Shutdown signal received. Waiting for sweeps to finish.")
logging.info("Press Ctrl-C again to force shutdown.")
return
signal.signal(signal.SIGINT, shutdown_handler)
Elsewhere:
subprocess.Popen(args, stdout=PIPE, stderr=PIPE, preexec_fn = os.setsid)
When running NOT using systemd (as just python daemon.py), the Popen subprocess continues running as desired. But when using sudo systemctl kill --signal=SIGINT myapp, it sends the signal to the parent, child, and Popen (command line) processes.
systemd[1]: fi_iot.service: Sent signal SIGINT to main process 512562 (python3) on client request.
systemd[1]: fi_iot.service: Sending signal SIGINT to process 512978 (python3) on client request.
systemd[1]: fi_iot.service: Sending signal SIGINT to process 513023 (my-cli-tool) on client request.
Any one know why this is happening?
I'm also open to suggestions on alternative ways of implementing this (eg adding an ExecStop= arg to my system config, or using a custom signal instead of SIGINT), though I'd rather override as little default behavior as possible, I want sudo systemctl stop myapp to do what it's supposed to do without my custom code potentially messing things up or confusing others.
EDIT: It seems this issue is specific to how the Popen function is called, I might try setting it to SIGIGN and see it that works, an earlier version of this post indicated this was a broader issue than it appears to be.
In python3 ctl-c is raised as the error KeyboardInterrupt
to catch it use try, and except KeyboardInterrupt:
The best bet to catching it is to make a main method, and put the try except around where you call it.
Update:
Popen has a method called send_signal, so you can forward the systemd signal to it
relevant python docs
https://docs.python.org/3/library/subprocess.html#subprocess.Popen.send_signal
Solution: Using a different method of preventing subprocess.Popen from overriding signals worked:
def preexec_function():
# used by Popen to tell driver to ignore SIGINT
signal.signal(signal.SIGINT, signal.SIG_IGN)
proc = Popen(args, stdout=PIPE, stderr=PIPE, preexec_fn = preexec_function)
Now the subprocess ignores the signal, unlike copilot's preexec_fn=os.setsid, which doesn't do what I want from systemd, which is what I get for using GPT-3 generated code I don't understand.
I may look into using Showierdata9978's suggestion of using send_signal, which could allow me to send the interrupt when the second Ctrl+C is pressed, allowing it to shutdown safely despite the ignore.
I am developing a wrapper around docker compose with python.
However, I struggle with Popen.
Here is how I launch launch it :
import subprocess as sp
argList=['docker-compose', 'up']
env={'HOME': '/home/me/somewhere'}
p = sp.Popen(argList, env=env)
def handler(signum, frame):
p.send_signal(signum)
for s in (signal.SIGINT,):
signal.signal(s, handler) # to redirect Ctrl+C
p.wait()
Everything works fine, when I hit Ctrl+C, docker-compose kills gracelly the container, however, p.wait() never returns...
Any hint ?
NOTE : While writing the question, I though I needed to check if p.wait() does actually return and if the block is after (it's the last instruction in the script). Adding a print after it end in the process exiting normally, any further hints on this behavior ?
When I run your code as written, it works as intended in that it causes docker-compose to exit and then p.wait() returns. However, I occasionally see this behavior:
Killing example_service_1 ... done
ERROR: 2
I think that your code may end up delivering SIGINT twice to docker-compose. That is, I think docker-compose receives an initial SIGINT when you type CTRL-C, because it has the same controlling terminal as your Python script, and then you explicitly deliver another SIGINT in your handler function.
I don't always see this behavior, so it's possible my explanation is incorrect.
In any case, I think the correct solution here is imply to ignore SIGINT in your Python code:
import signal
import subprocess
argList = ["docker-compose", "up"]
p = subprocess.Popen(argList)
signal.signal(signal.SIGINT, signal.SIG_IGN) # to redirect Ctrl+C
p.wait()
With this implementation, your Python code ignores the SIGINT generated by CTRL-C, but it is received and processed normally by docker-compose.
This is a follow up to my other post Installing signal handler with Python. In short, Linux blocks all signals to PID 1 (including SIGKILL) unless Init has installed a signal handler for a particular signal; as to prevent kernel panic if someone were to send a termination signal to PID1. The issue I've been having, is it would seem that the signal module in Python doesn't install signal handlers in a way the system recognises. My Python Init script was seemingly, completely ignoring all signals as I think they were being blocked.
I seem to have found a solution; using ctypes to install the signal handlers with the signal() function in libc (in this case uClibc). Below is a python based test init. It opens a shell on TTY2 from which I can send signals to PID1 for testing. It seems to work in the KVM im using for testing (I'm willing to share the VM with anyone interested)
Is this the best way around this issue? Is there a 'better' way to install the signal handlers without the signal module? (I am not at all concerned with portably)
Is this a bug in Python?
#!/usr/bin/python
import os
import sys
import time
from ctypes import *
def SigHUP():
print "Caught SIGHUP"
return 0
def SigCHLD():
print "Caught SIGCHLD"
return 0
SIGFUNC = CFUNCTYPE(c_int)
SigHUPFunc = SIGFUNC(SigHUP)
SigCHLDFunc = SIGFUNC(SigCHLD)
libc = cdll.LoadLibrary('libc.so.0')
libc.signal(1, SigHUPFunc) # 1 = SIGHUP
libc.signal(17, SigCHLDFunc) # 17 = SIGCHLD
print "Mounting Proc: %s" % libc.mount(None, "/proc", "proc", 0, None)
print "forking for ash"
cpid = os.fork()
if cpid == 0:
os.closerange(0, 4)
sys.stdin = open('/dev/tty2', 'r')
sys.stdout = open('/dev/tty2', 'w')
sys.stderr = open('/dev/tty2', 'w')
os.execv('/bin/ash', ('ash',))
print "ash started on tty2"
print "sleeping"
while True:
time.sleep(0.01)
I did a bit of debugging under KVM and I found that the kernel is delivering signals to pid 1 when the signal handlers are installed by the standard signal module. However, when the signal is received "something" causes a clone of the process to be spawned, rather than printing the expected output.
Here is the strace output when I send HUP to the non-working init.sig-mod:
Which results in a new process running (pid 23) which is a clone of init.sig-mod:
I didn't have time to dig deeper into the cause, but this narrows things further. Probably something to do with Python's signal delivery logic (it registers a C hook which invokes your bytecode function when called). The ctypes technique bypasses this. The relevant Python source files are Python/pythonrun.c and Modules/signalmodule.c, in case you want to take a closer look.
Old Info -- I'm not sure this will solve your problem, but might get you closer. I
compared these different ways signal handlers are installed:
Installing a handler via Python's signal module.
Upstart's signal handlers.
Using ctypes to call the signal() syscall directly.
Some quick tests in C.
Both the ctypes-invoked signal() system call and Upstart's sigaction()
syscalls set the SA_RESTART flag when the handler is registered. Setting
this flag indicates that when a signal is received while the process is
executing or blocking inside certain syscalls (read, write, wait,
nanosleep, etc), after the signal handler completes the syscall should be
automatically restarted. The application won't be aware of this.
When Python's signal module registers a handler, it zeros the SA_RESTART
flag by calling siginterrupt(signum, 1). This says to the system "when a
system call is interrupted by a signal, after the signal handler completes
set errno to EINTR and return from the syscall". This leaves it up to the developer to
handle this and decide whether to restart the system call.
You can set the SA_RESTART flag by registering your signal this way:
import signal
signal.signal(signal.SIGHUP, handler)
signal.siginterrupt(signal.SIGHUP, False)
The issue was a compatibility issue with Python compiled against uClibc 0.9.31 with old linux threads. Compiling against 0.9.32-rc3 and using NPTL has fixed the issue.
(there is a follow up to this question here)
I am working on trying to write a Python based Init system for Linux but I'm having an issue getting signals to my Python init script. From the 'man 2 kill' page:
The only signals that can be sent to process ID 1, the init process, are those for which init has explicitly installed signal handlers.
In my Python based Init, I have a test function and a signal handler setup to call that function:
def SigTest(SIG, FRM):
print "Caught SIGHUP!"
signal.signal(signal.SIGHUP, SigTest)
From another TTY (the init script executes sh on another tty) if I send a signal, it is completely ignored and the text is never printed. kill -HUP 1
I found this issue because I wrote a reaping function for my Python init to reap its child processes as they die, but they all just zombied, it took awhile to figure out Python was never getting the SIGCHLD signal. Just to ensure my environment is sane, I wrote a C program to fork and have the child send PID 1 a signal and it did register.
How do I install a signal handler the system will acknowledge if signal.signal(SIG, FUNC) isn't working?
Im going to try using ctypes to register my handler with C code and see if that works, but I rather a pure Python answer if at all possible.
Ideas?
( I'm not a programmer, Im really in over my head here :p )
Test code below...
import os
import sys
import time
import signal
def SigTest(SIG, FRM):
print "SIGINT Caught"
print "forking for ash"
cpid = os.fork()
if cpid == 0:
os.closerange(0, 4)
sys.stdin = open('/dev/tty2', 'r')
sys.stdout = open('/dev/tty2', 'w')
sys.stderr = open('/dev/tty2', 'w')
os.execv('/bin/ash', ('ash',))
print "ash started on tty2"
signal.signal(signal.SIGHUP, SigTest)
while True:
time.sleep(5.0)
Signal handlers mostly work in Python. But there are some problems. One is that your handler won't run until the interpreter re-enters it's bytecode interpreter. if your program is blocked in a C function the signal handler is not called until it returns. You don't show the code where you are waiting. Are you using signal.pause()?
Another is that if you are in a system call you will get an exception after the singal handler returns. You need to wrap all system calls with a retry handler (at least on Linux).
It's interesting that you are writing an init replacement... That's something like a process manager. The proctools code might interest you, since it does handle SIGCHLD.
By the way, this code:
import signal
def SigTest(SIG, FRM):
print "SIGINT Caught"
signal.signal(signal.SIGHUP, SigTest)
while True:
signal.pause()
Does work on my system.
My python script intercepts the SIGINT signal with the signal process module to prevent premature exit, but this signal is passed to a subprocess that I open with Popen. is there some way to prevent passing this signal to the subprocess so that it also is not exited prematurely when the user presses ctrl-c?
Signal handlers are inherited when you start a subprocess, so if you use the signal module to ignore SIGINT (signal.signal(signal.SIGINT, signal.SIG_IGN)), then your child process automatically will also.
There are two important caveats, though:
You have to set the ignore handler before you spawn the child process
Custom signal handlers are reset to the default handlers, since the child process won't have access to the handler code to run it.
So if you need to customise your handling of SIGINT rather than just ignoring it, you probably want to temporarily ignore SIGINT while you spawn your child process, then (re)set your custom signal handler.
If you're trying to catch SIGINT and set a flag so you can exit at a safe point rather than immediately, remember that when you get to that safe point your code will have to manually clean up its descendants, since your child process and any processes it starts will be ignoring the SIGINT.
You are able to re-assign the role of ctrl-c using the tty module, which allows you to manipulate the assignment of signals. Be warned, however, that unless you put them back the way they were before you modified them, they will persist for the shell's entire session, even after the program exits.
Here is a simple code snippet to get you started that stores your old tty settings, re-assigns ctrl-c to ctrl-x, and then restores your previous tty settings upon exit.
import sys
import tty
# Back up previous tty settings
stdin_fileno = sys.stdin.fileno()
old_ttyattr = tty.tcgetattr(stdin_fileno)
try:
print 'Reassigning ctrl-c to ctrl-x'
# Enter raw mode on local tty
tty.setraw(stdin_fileno)
raw_ta = tty.tcgetattr(stdin_fileno)
raw_ta[tty.LFLAG] |= tty.ISIG
raw_ta[tty.OFLAG] |= tty.OPOST | tty.ONLCR
# ^X is the new ^C, set this to 0 to disable it entirely
raw_ta[tty.CC][tty.VINTR] = '\x18'
# Set raw tty as active tty
tty.tcsetattr(stdin_fileno, tty.TCSANOW, raw_ta)
# Dummy program loop
import time
for _ in range(5):
print 'doing stuff'
time.sleep(1)
finally:
print 'Resetting ctrl-c'
# Restore previous tty no matter what
tty.tcsetattr(stdin_fileno, tty.TCSANOW, old_ttyattr)
For python 2 codebase: subprocess is broken.
The right thing is
import subprocess32 as subprocess
See subprocess32
This is a backport of the Python 3 subprocess module for use on Python
2. This code has not been tested on Windows or other non-POSIX platforms.