We Keep Coding

How exactly is SIGILL generated?

I have a program using tensorflow on a non-supported hardware, so everytime i run it, i get the "Illegal instruction (Core dumped)" error
my main goal is to capture this error. i don't want to solve it.
The error is not printed to the stderr of my program, it's printed to the stderr of bash.
then my program exists with code 33792 which is 132 (SIGILL)
And i cannot capture it using the method mentioned here, because i'm running my command using docker run and i can't pass it the curly brackets
Is there any way to capture the stdout of bash without the curly brackets?
Also how exactly is SIGILL generated? what exactly is happening behind the scenes?
Is SIGILL triggered in the parent process (bash in my case) and passed to the child process (my program)? or vice versa?
i tried adding a SIGILL handler in my program to see if i can capture it, but my program froze instead of printing the "illegal instruction" error.
I'm using Debian 11 and my program is written in python.
Edit:
The SIGILL kills my python program and my goal is to capture the SIGILL from inside my program, print some error and kill my program afterward.
I don't want the (Illegal instruction) error printed to be printed in the bash's stderr, I want it to be printed to my program's stderr or stdout.
Edit: here's the sigill handler I have in my code
def sigill_handler(sig, frame):
print("Illegal Instruction. terminating.")
signal.signal(signal.SIGILL, sigill_handler)
notice that this is the only signal I'm handling in my code

Citing https://docs.python.org/3/library/signal.html:
Execution of Python signal handlers
A Python signal handler does not get executed inside the low-level (C) signal handler. Instead, the low-level signal handler sets a flag which tells the virtual machine to execute the corresponding Python signal handler at a later point(for example at the next bytecode instruction). This has consequences:
It makes little sense to catch synchronous errors like SIGFPE or SIGSEGV that are caused by an invalid operation in C code. Python will return from the signal handler to the C code, which is likely to raise the same signal again, causing Python to apparently hang. From Python 3.3 onwards, you can use the faulthandler module to report on synchronous errors.
A long-running calculation implemented purely in C (such as regular expression matching on a large body of text) may run uninterrupted for an arbitrary amount of time, regardless of any signals received. The Python signal handlers will be called when the calculation finishes.
If the handler raises an exception, it will be raised “out of thin air” in the main thread. See the note below for a discussion.
According to https://docs.python.org/3/library/faulthandler.html, all the faulthandler can do is to dump a stack trace, so it does not help for your requirement.
What you could do is to run your possibly failing program from your own wrapper program where you can check the wait status and decide what you display to the user if the program was killed by SIGILL.
It would be better to check if your program runs on a supported platform before using any tensorflow functions.

Control executed programm with python

I want to execute a testrun via bash, if the test needs too much time. So far, I found some good solutions here. But since the command kill does not work properly (when I use it correctly it says it is not used correctly), I decided to solve this problem using python. This is the Execution call I want to monitor:
EXE="C:/program.exe"
FILE="file.tpt"
HOME_DIR="C:/Home"
"$EXE" -vm-Xmx4096M --run build "$HOME_DIR/test/$FILE" "Auslieferung (ML) Execute"
(The opened *.exe starts a testrun which includes some simulink simulation runs - sometimes there are simulink errors - in this case, the execution time of the tests need too long and I want to restart the entire process).
First, I came up with the idea, calling a shell script containing these lines within a subprocess from python:
import subprocess
import time
process = subprocess.Popen('subprocess.sh', shell = True)
time.sleep(10)
process.terminate()
But when I use this, *.terminate() or *.kill() does not close the program I started with the subprocess call.
That´s why I am now trying to implement the entire call in python language. I got the following so far:
import subprocess
file = "somePath/file.tpt"
p = subprocess.Popen(["C:/program.exe", file])
Now I need to know, how to implement the second call "Auslieferung (ML) Execute" of the bash function. This call starts an intern testrun named "Auslieferung (ML) Execute". Any ideas? Or is it better to choose one of the other ways? Or can I get the "kill" option for bash somewhere, somehow?

Finding a line of a C module called by a python script that segfaults

I have a caller.py which repeatedly calls routines from some_c_thing.so, which was created from some_c_thing.c. When I run it, it segfaults - is there a way for me to detect which line of c code is segfaulting?

This might work:
make sure the native library is compiled with debug symbols (-g switch for gcc).
Run python under gdb and let it crash:
gdb --args python caller.py
run # tell gdb to run the program
# script runs and crashes
bt # print backtrace, which should show the crashing line
If crash happens in the native library code, then this should reveal the line.
If native library code just corrupts something or violates some postconditions, and crash happens in Python interpreter's code, then this will not be helpful. In that case your options are code review, adding debug prints (first step would be to just log entry and exit of each C function to detect which is the last C function called before crash, then adding more fine-grained logging for variable values etc), and finally using debugger to see what happens by using the usual debugger techniques (breakpoints, stepping, watches...).

Take Python and the .so file(s) out of the equation. See what params are being passed, if any, and call the routines from a debugger capable of stepping through C code and binaries.
Here is a link to an article describing a simple C debugging process, in case you're not familiar with debugging C (command line interface). Here is another link on using NetBeans to debug C. Also using Eclipse...
This could help: gdb: break in shared library loaded by python (might also turn out to be a dupe)

segfault... Check if the number of variables or the types of variables you passed to that c function (in .so) are correct. If not aligned, usually it's a segfault.

How does python process a signal?

What is the workflow of processing a signal in python ? I set a signal handler, when the signal occur ,how does python invoke my function? Does the OS invoke it just like C program?
If I am in a C extend of python ,is it interrupted immediately ?
Now it's clear to me how does python process handle a signal . When you set a signal by the signal module , the module will register a function signal_handler(see $src/Modules/signalmodule.c) ,which set your handler and flag it as 1(Handlers[sig_num].tripped = 1;) , then call Py_AddPendingCall to tell python interpreter. The python interpreter will invoke Py_MakePendingCalls to call PyErr_CheckSignals which calls your function in main loop(see $src/Python/ceval.c).
communicate me if you want to talk about this : renenglish#gmail.com

If you set a Python code signal handler using the signal module the interpreter will only run it when it re-enters the byte-code interpreter. The handler is not run right away. It is placed in a queue when the signal occurs. If the code path is currently in C code, built-in or extension module, the handler is deferred until the C code returns control to the Python byte code interpreter. This can be a long time, and you can't really predict how long.
Most notably if you are using interactive mode with readline enabled your signal handler won't run until you give it some input to interpret. this is because the input code is in the readline library (C code) and doesn't return to the interpreter until it has a complete line.

Take a look at the signal module. If you invoke a signal to a python script, from my understanding if there is a handler for it will first process that signal, and potentially has the ability to handle and ignore certain signals. ie. instead of killing on a SIGKILL, you attempt to perform some shutdown cleanup work before killing.

Have a Python CGI call a Perl CGI, passing original info (to limit searching private Mailman archives to logged-in users)

I need to have a Python CGI script do some stuff (a little bit of security checking), and then end up calling a Perl CGI script, passing anything it received (e.g., POST info) onto the Perl script.
For background, my reason for doing this is that I'm trying to integrate Swish searching with Mailman list archives.
Swish searching uses swish.cgi, a Perl script, but because these are private list archives I can't just allow people to call swish.cgi directly as recommended on this page: http://wpkg.org/Integrating_Mailman_with_a_Swish-e_search_engine#Mailman_configuration
I believe what I need to do is have the Mailman "private" cgi-bin file (written in Python) do its regular security checking (which calls a few Mailman/python modules) and THEN call on swish.cgi to do the search (after having verified that the user is on the mailing list).
Essentially, I believe the simplest solution would just be to protect access to the swish.cgi Perl script with a variant of the standard mailman cgi-bin/private Python script.
(I considered the idea that people could search with a non-protected swish.cgi, and people wouldn't be able to view the full results because those posts are already password-protected by default Mailman setup... but the problem is that even showing the Swish post excerpts in the search results could expose confidential information, so I must restrict access to even the search itself to just subscribers.)
If someone has a better idea of how to solve the overall problem without doing the Python-CGI-calls-Perl-CGI I'll be happy to consider that the "answer".
Just know that my goal is to make little (ideally no) changes to the standard Mailman installation. Copying the "private" cgi-bin script (whose source is mailman-2.1.12/Mailman/Cgi/private.py) and making changes to call swish.cgi is cool, but modifying the existing private cgi-bin script wouldn't really be cool.
Here's what I did to test the answer (using os.execv to replace the python script with the perl script, so that the perl script will inherit the python script's environment):
I created a pythontest script with:
import os
os.environ['FOO'] = 'BAR'
mydir = os.path.dirname(os.environ.get('SCRIPT_FILENAME'))
childprog = mydir + '/perltest'
childargs = []
os.execv(childprog, childargs)
Then a perltest script with:
print "Content-type: text/html\n\n";
while (($key,$value) = each %ENV) {
print "<p>$key=$value</p>\n";
}
Then I called http://myserver.com/cgi-bin/pythontest and saw that the environment printout included the custom FOO variable so the child perltest process had successfully inherited all the environment variables.

I'm just going to state the obvious here because I don't have any detailed knowledge about your specific environment.
If your python script is a genuine CGI and not a mod_python script or similar then it is just a regular process spawned to handle the one request. You can use os.execv to replace it with another process (e.g. the perl CGI) and the new process will inherit the current process' environment, stdin, stdout and stderr. This assumes that you don't need to read stdin for your security checks. It may also depend on whether your CGI is running in a restricted environment. execv is potentially dangerous and might be blocked in such an environment.
If you're running from a mod_python environment or if you need to peek at posted data (i.e. stdin) then the execv approach isn't available to you. You have two main alternatives.
You could run the perl CGI directly (e.g. look at the subprocess module) handing it a correct environment and feeding it the correct data to its stdin. You can the spool the returned data from its stdout raw (or cooked if needed) directly back to the web server.
Otherwise, you could make a local web request to run the CGI. This is likely to require a bit less knowledge about the server setup, but a bit more work in the python CGI to make and handle the HTTP request.