code in fork_child.py
from subprocess import Popen
child = Popen(["ping", "google.com"], stdout=subprocess.PIPE,stderr=subprocess.PIPE)
out, err = child.communicate()
I run it from a terminal window as -
$python fork_child.py
From another terminal window if I get the PID of fork_child.py and kill it with SIGTERM, "ping" doesn't get killed. How do I make sure that ping too gets killed when fork_child receives a SIGTERM ?
Children don't automatically die when the parent process is killed. They die if:
The parent forwards the signal and waits for the children to terminate
When the child tries to communicate with the parent, for example via stdio. That only works if the parent also created the file descriptors which the child uses.
The signals module contains examples how to write a signal handler.
So you need to:
collect all children in a list
install a signal handler
in the handler, iterate over all the child processes
For each child process, invoke child.terminate() followed by child.wait()
The wait() is necessary to allow the OS to garbage collect the child process. If you forget it, you may end up with zombie processes.
A simple way to kill the whole process tree in the shell is to kill its process group i.e., instead of kill $pid, run:
$ kill -TERM -$pid
Notice: the pid is negated.
Shell creates a new process group for each command (pipeline) therefore you won't kill innocent bystanders.
If descendant processes do not create their own independent process groups; they all die.
See Best way to kill all child processes.
Related
What I want? Create a script that starts and kill a communication protocol
What I have?
I have a python script that opens a shell script, and this shell script initialize the protocol. When I kill the parent process, everything goes fine (but in the final project, the parent process will have to stay alive), but when I kill the subprocess, it become a zombie function, and my protocol keep running.
Problems I believe can be: I'm "killing" the shell script (not the protocol, that's what I want)
The line I start the shell script:
`protocolProcess = subprocess.Popen(["sh", arquivo], cwd = localDoArquivo) #inicia o protocolo`
protocolProcessPID = protocolProcess.pid #armazena o pid do protocolProcess
The line I kill the shell script: os.kill(protocolPID, signal.SIGTERM)
Well, that's it! If anyone can help me, I'll be very grateful
Zombie processes are processes that have not yet been reaped by the parent process.
The parent process will hold onto those process handlers until the end of time, or until it reads the process exit status, or itself is killed.
It sounds like the parent process needs to have a better handle on how it spawns and reaps it's children. Simply killing a child process is not enough to free a zombie process.
I want to run a process/thread that continuously runs even after the main process is finished or is killed. How should I go about this?
Edit: If it makes a difference, the point of the child process is to monitor the status of an external process. When that process is done, the child process needs to kick off another process to the same external system. It needs to do so even after the main program finishes so that the user can continue working elsewhere while new jobs are being sent to the system as needed.
If you simply want to continue execution in a child process, and exit from the current process, you can simply use os.fork:
import os
import time
print('Before forking.')
if not os.fork():
for i in range(5):
print("Forkety fork, I'm forked!")
time.sleep(1)
Calling os.fork() will basically duplicate your process, but for the parent process, os.fork() will return the PID of the child process, while for the child process, it will return 0.
if not os.fork() will simply not enter the if branch in the parent process, since the result of calling os.fork() is non-zero (i.e. the PID of the child process).
In the child process, os.fork() returned zero, hence the child process enters the if branch.
If you want to run any executable (not necessarily Python code), your best choice is subprocess.Popen, as mentioned by #khachik.
I'm wondering if this is the correct way to execute a system process and detach from parent, though allowing the parent to exit without creating a zombie and/or killing the child process. I'm currently using the subprocess module and doing this...
os.setsid()
os.umask(0)
p = subprocess.Popen(['nc', '-l', '8888'],
cwd=self.home,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
os.setsid() changes the process group, which I believe is what lets the process continue running when it's parent exits, as it no longer belongs to the same process group.
Is this correct and also is this a reliable way of performing this?
Basically, I have a remote control utility that communicate through sockets and allows to start processes remotely, but I have to ensure that if the remote control dies, the processes it started continue running unaffected.
I was reading about double-forks and not sure if this is necessary and/or subprocess.POpen close_fds somehow takes care of that and all that's needed is to change the process group?
Thanks.
Ilya
For Python 3.8.x, the process is a bit different. Use the start_new_session parameter available since Python 3.2:
import shlex
import subprocess
cmd = "<full filepath plus arguments of child process>"
cmds = shlex.split(cmd)
p = subprocess.Popen(cmds, start_new_session=True)
This will allow the parent process to exit while the child process continues to run. Not sure about zombies.
The start_new_session parameter is supported on all POSIX systems, i.e. Linux, MacOS, etc.
Tested on Python 3.8.1 on macOS 10.15.5
popen on Unix is done using fork. That means you'll be safe with:
you run Popen in your parent process
immediately exit the parent process
When the parent process exits, the child process is inherited by the init process (launchd on OSX) and will still run in the background.
The first two lines of your python program are not needed, this perfectly works:
import subprocess
p = subprocess.Popen(['nc', '-l', '8888'],
cwd="/",
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
I was reading about double-forks and not sure if this is necessary
This would be needed if your parent process keeps running and you need to protect your children from dying with the parent. This answer shows how this can be done.
How the double-fork works:
create a child via os.fork()
in this child call Popen() which launches the long running process
exit child: Popen process is inherited by init and runs in the background
Why the parent has to immediately exit? What happens if it doesn't exit immediately?
If you leave the parent running and the user stops the process e.g. via ctrl-C (SIGINT) or ctrl-\ (SIGQUIT) then it would kill both the parent process and the Popen process.
What if it exits one second after forking?
Then, during this 1s period your Popen process is vulnerable to ctrl-c etc. If you need to be 100% sure then use the double forking.
On a Windows 7 machine:-
I have a main (Python) program that I start on a command prompt [main process].
This program spawns a child (Python) program [child process].
I close the command prompt.
Result:-
Child process ends immediately.
On the other hand if I end the main program from task manager, I observe that the child process is still running.
I was wondering why the 2 approaches do not have the same results? Is it sending some different signal in the two cases?
Comments to the question pointed me to get the answer.
I was using subprocess.Popen(args) to spawn the child process. This would spawn the child process successfully but the child process would be launched in the same command window as its parent.
Going through subprocess Popen doc, I found some additional arguments to be passed in order to launch child process in another command window.
Launching the child with the following arguments solved my problem.
subprocess.Popen(args, shell=True, creationflags=subprocess.CREATE_NEW_CONSOLE)
The last argument subprocess.CREATE_NEW_CONSOLE is only for windows.
I'm wondering if this is the correct way to execute a system process and detach from parent, though allowing the parent to exit without creating a zombie and/or killing the child process. I'm currently using the subprocess module and doing this...
os.setsid()
os.umask(0)
p = subprocess.Popen(['nc', '-l', '8888'],
cwd=self.home,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
os.setsid() changes the process group, which I believe is what lets the process continue running when it's parent exits, as it no longer belongs to the same process group.
Is this correct and also is this a reliable way of performing this?
Basically, I have a remote control utility that communicate through sockets and allows to start processes remotely, but I have to ensure that if the remote control dies, the processes it started continue running unaffected.
I was reading about double-forks and not sure if this is necessary and/or subprocess.POpen close_fds somehow takes care of that and all that's needed is to change the process group?
Thanks.
Ilya
For Python 3.8.x, the process is a bit different. Use the start_new_session parameter available since Python 3.2:
import shlex
import subprocess
cmd = "<full filepath plus arguments of child process>"
cmds = shlex.split(cmd)
p = subprocess.Popen(cmds, start_new_session=True)
This will allow the parent process to exit while the child process continues to run. Not sure about zombies.
The start_new_session parameter is supported on all POSIX systems, i.e. Linux, MacOS, etc.
Tested on Python 3.8.1 on macOS 10.15.5
popen on Unix is done using fork. That means you'll be safe with:
you run Popen in your parent process
immediately exit the parent process
When the parent process exits, the child process is inherited by the init process (launchd on OSX) and will still run in the background.
The first two lines of your python program are not needed, this perfectly works:
import subprocess
p = subprocess.Popen(['nc', '-l', '8888'],
cwd="/",
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
I was reading about double-forks and not sure if this is necessary
This would be needed if your parent process keeps running and you need to protect your children from dying with the parent. This answer shows how this can be done.
How the double-fork works:
create a child via os.fork()
in this child call Popen() which launches the long running process
exit child: Popen process is inherited by init and runs in the background
Why the parent has to immediately exit? What happens if it doesn't exit immediately?
If you leave the parent running and the user stops the process e.g. via ctrl-C (SIGINT) or ctrl-\ (SIGQUIT) then it would kill both the parent process and the Popen process.
What if it exits one second after forking?
Then, during this 1s period your Popen process is vulnerable to ctrl-c etc. If you need to be 100% sure then use the double forking.