So I have this code (partially taken from python docs):
import signal
def handler(signum, frame):
print 'Signal handler called with signal', signum
s = signal.signal(signal.SIGINT, handler)
some_fancy_code() # this code is using subprocess.Popen() to call another script
singal.signal(signal.SIGINT, s)
What I found right now is that if I do Ctrl+C in my program, it correctly enters that handler and prints. Now, what I thought is that after receiving Ctrl+C my handler will suppress default handler so for example my subprocess.Popen will not get the KeyboardInterrupt signal. But this is not the case.
But when we replace 'handler' with 'signal.SIG_IGN', this propagation never happens. Modified snippet:
import signal
s = signal.signal(signal.SIGINT, signal.SIG_IGN)
some_fancy_code() # this code is using subprocess.Popen() to call another script
singal.signal(signal.SIGINT, s)
Is this because SIG_IGN is some kind of 'magic' signal written in language itself? Or maybe there is a way to make similar suppression in my own handler?
After reading a bit of question on stack overflow I am bit confused. If someone could make clear for me why such difference in behaviour.
This is the specified POSIX behaviour of signals:
A child created via fork(2) inherits a copy of its parent's signal dis‐
positions. During an execve(2), the dispositions of handled signals
are reset to the default; the dispositions of ignored signals are left
unchanged.
When you execute (fork/execve) your another script in the first case, the SIGINT handler is reset to the default handler in the another script (default behaviour is to terminate the process) - of course, the another script could install its own handler and change this behaviour.
However, in the second case, you've configured SIGINT to be ignored. This behaviour will be propagated to the another script, as indicated in the definition above. Again, the another script could change this behaviour by installing its own handler.
So this has nothing to do with Python directly. It is the expected behaviour of the underlying operating system's POSIX signal handling implementation.
PS. If you're wondering what fork() and execve() are, fork() creates a copy of the running process (a child) and execve() replaces the current process with another. This is the underlying mechanism used by subprocess.Popen() to run the 'another script': first make a copy of the current process and then replace it with the target process.
Related
I have a program using tensorflow on a non-supported hardware, so everytime i run it, i get the "Illegal instruction (Core dumped)" error
my main goal is to capture this error. i don't want to solve it.
The error is not printed to the stderr of my program, it's printed to the stderr of bash.
then my program exists with code 33792 which is 132 (SIGILL)
And i cannot capture it using the method mentioned here, because i'm running my command using docker run and i can't pass it the curly brackets
Is there any way to capture the stdout of bash without the curly brackets?
Also how exactly is SIGILL generated? what exactly is happening behind the scenes?
Is SIGILL triggered in the parent process (bash in my case) and passed to the child process (my program)? or vice versa?
i tried adding a SIGILL handler in my program to see if i can capture it, but my program froze instead of printing the "illegal instruction" error.
I'm using Debian 11 and my program is written in python.
Edit:
The SIGILL kills my python program and my goal is to capture the SIGILL from inside my program, print some error and kill my program afterward.
I don't want the (Illegal instruction) error printed to be printed in the bash's stderr, I want it to be printed to my program's stderr or stdout.
Edit: here's the sigill handler I have in my code
def sigill_handler(sig, frame):
print("Illegal Instruction. terminating.")
signal.signal(signal.SIGILL, sigill_handler)
notice that this is the only signal I'm handling in my code
Citing https://docs.python.org/3/library/signal.html:
Execution of Python signal handlers
A Python signal handler does not get executed inside the low-level (C) signal handler. Instead, the low-level signal handler sets a flag which tells the virtual machine to execute the corresponding Python signal handler at a later point(for example at the next bytecode instruction). This has consequences:
It makes little sense to catch synchronous errors like SIGFPE or SIGSEGV that are caused by an invalid operation in C code. Python will return from the signal handler to the C code, which is likely to raise the same signal again, causing Python to apparently hang. From Python 3.3 onwards, you can use the faulthandler module to report on synchronous errors.
A long-running calculation implemented purely in C (such as regular expression matching on a large body of text) may run uninterrupted for an arbitrary amount of time, regardless of any signals received. The Python signal handlers will be called when the calculation finishes.
If the handler raises an exception, it will be raised “out of thin air” in the main thread. See the note below for a discussion.
According to https://docs.python.org/3/library/faulthandler.html, all the faulthandler can do is to dump a stack trace, so it does not help for your requirement.
What you could do is to run your possibly failing program from your own wrapper program where you can check the wait status and decide what you display to the user if the program was killed by SIGILL.
It would be better to check if your program runs on a supported platform before using any tensorflow functions.
After reading A LOT of data on the subject I still couldn't find any actual solution to my problem (there might not be any).
My problem is as following:
In my project I have multiple drivers working with various hardware's (IO managers, programmable loads, power supplies and more).
Initializing connection to these hardware's is costly (in time), and I cant open and then close the connection for every communication iteration between us.
Meaning I cant do this (Assuming programmable load implements enter / exit):
start of code...
with programmable_load(args) as program_instance:
programmable_load_instance.do_something()
rest of code...
So I went for a different solution :
class programmable_load():
def __init__(self):
self.handler = handler_creator()
def close_connection(self):
self.handler.close_connection()
self.handler = None
def __del__(self):
if (self.handler != None):
self.close_connection()
For obvious reasons I dont 'trust' the destructor to actually get called so I explicitly call close_connection() when I want to end my program (for all drivers).
The problem happens when I abruptly terminate the process, for example when I run via debug mode and quit debugging.
In these cases the process terminates without running through any destructors.
I understand that the OS will clear all memory unused at this point, but is there any way to clear the memory in an organized manner?
and if not, is there a way to make the quit debugging function pass through a certain set of functions? Does the python process know it got a quite debugging event or does it treat it as a normal termination?
Operating system: Windows
According to this documentation:
If a process is terminated by TerminateProcess, all threads of the
process are terminated immediately with no chance to run additional
code.
(Emphasis mine.) This implies that there is nothing you can do in this case.
As detailed here, signals don't work very well on ms-windows.
As was mentioned in a comment, you could use atexit to do the cleanup. But that only works if the process is asked to close (e.g. QUIT signal on Linux) and not just killed (as is likely the case when stopping the debugging session). Similarily if you force your computer to turn off (e.g. long press power button or remove power) then it won't be called either. There is no 'solution' to that for obvious reasons. Your program can't expect to be called when the power suddenly goes off or when it is forcefully killed. The point of forcefully killing is to definitely kill the process now. If it first called your clean-up code then you could delay that which defeats the purpose. That is why there are signals such as to ask your process to stop. This is not Python specific. The same concept also applies across operating systems.
Bonus (design suggestion, not a solution): I would argue that you can still make use of the context manager (using with). Your problem is not unique. Database connections are usually kept alive for longer as well. It is a question of the scope. Move the context further up to the application level. Then it is clear what the boundary is and you don't need any magic (you are probably also aware of #contextmanager to make that a breeze).
I haven't tested properly as I don't have wingide installed over here so I can't grant you this will work but what about using setconsolectrlhandler? For instance, try something like this:
import os
import sys
import win32api
if __name__ == "__main__":
def callback(sig, func=None):
print("Exit handler called!")
try:
win32api.SetConsoleCtrlHandler(callback, True)
except Exception as e:
print("Captured exception", e)
sys.exit(1)
print("Press to quit")
input()
print("Bye!")
It'll be able to handle CTRL+C and CTRL+BREAK signals:
I need to temporarily install a custom signal handler in my code. Afterwards, I need to restore control to the original signal handler. I've seen the following before in other StackOverflow answers:
# Save original signal handler
original_SIGINT_handler = signal.getsignal(signal.SIGINT)
# Install the new SIGINT handler
signal.signal(signal.SIGINT, handle_SIGINT)
# do stuff
# Return control to original signal handler
signal.signal(signal.SIGINT, original_SIGINT_handler)
However, this will not work if the original signal handler was registered from C. According to https://pymotw.com/2/signal/, if the original signal handler was registered from C, signal.getsignal(signal.SIGINT) will return None. Then, when we go to restore the signal with signal.signal(signal.SIGINT, None), the code fails with:
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object
How would I go about installing a temporary signal handler, if the original was defined in C to begin with?
Is it possible to "append" functionality to an existing signal handler? Given this other StackOverflow answer, I feel like there might be a hacky way to do so, since apparently Python signal handlers run after the C signal handler returns.
For more context, I'm writing a lldb Python script, and I need to temporarily capture ctrl+c while inside my script, which I'm doing by temporarily installing my own SIGINT handler. When leaving my script, I need to restore the original ctrl+c functionality, which lldb defined.
What is the workflow of processing a signal in python ? I set a signal handler, when the signal occur ,how does python invoke my function? Does the OS invoke it just like C program?
If I am in a C extend of python ,is it interrupted immediately ?
Now it's clear to me how does python process handle a signal . When you set a signal by the signal module , the module will register a function signal_handler(see $src/Modules/signalmodule.c) ,which set your handler and flag it as 1(Handlers[sig_num].tripped = 1;) , then call Py_AddPendingCall to tell python interpreter. The python interpreter will invoke Py_MakePendingCalls to call PyErr_CheckSignals which calls your function in main loop(see $src/Python/ceval.c).
communicate me if you want to talk about this : renenglish#gmail.com
If you set a Python code signal handler using the signal module the interpreter will only run it when it re-enters the byte-code interpreter. The handler is not run right away. It is placed in a queue when the signal occurs. If the code path is currently in C code, built-in or extension module, the handler is deferred until the C code returns control to the Python byte code interpreter. This can be a long time, and you can't really predict how long.
Most notably if you are using interactive mode with readline enabled your signal handler won't run until you give it some input to interpret. this is because the input code is in the readline library (C code) and doesn't return to the interpreter until it has a complete line.
Take a look at the signal module. If you invoke a signal to a python script, from my understanding if there is a handler for it will first process that signal, and potentially has the ability to handle and ignore certain signals. ie. instead of killing on a SIGKILL, you attempt to perform some shutdown cleanup work before killing.
I'd like to put my cmd.com window into a mode where Control-C does not generate a SIGINT signal to Python (ActiveState if it matters).
I know I can use the signal module to handle SIGINT. The problem is that handling SIGINT is too late; by the time it is handled, it has already interrupted a system call.
I'd like something equivalent to the *nix "raw" mode. Just let the input queue up and when it is safe for my application to read it, it will.
Maddeningly enough, msvcrt.getch() seems to return Control-C as a character. But that only works while the program is blocked by getch() itself. If I am in another system call (sleep, just to use an example), I get the SIGINT.
You need to call the win32 API function SetConsoleCtrlHandler with NULL (0) as its first parameter and TRUE (1) as its second parameter. If you're already using pywin32, win32.SetConsoleCtrlHandler is fine for the purpose, otherwise ctypes should work, specifically via ctypes.windll.kernel32.SetConsoleCtrlHandler(0, 1)/