We Keep Coding

Python how to kill root subprocess

I'm trying to launch a process with root privileges and kill it later on.
But for some reason, I can't get it to work.
Here is a small script to reproduce my problem (disclaimer: code is a bit dirty its only for bug reproduction):
import os
import time
import subprocess
command = ["sudo", "sleep", "25"]
process = subprocess.Popen(command,
bufsize=1,
stdin=open(os.devnull),
stderr=subprocess.PIPE,
stdout=subprocess.PIPE)
def kill():
pid = process.pid
cmd = "sudo kill %s" % pid
print(cmd)
print(os.system(cmd))
time.sleep(2)
kill()
stdout, stderr = process.communicate()
print("stdout: " + stdout)
print("stderr: " + stderr)
ret = process.wait()
print("ret: " + str(ret))
This code doesn't seem to be able to kill my subprocess, but when I launch os.system("sudo kill <pid>") in another python instance, it does work.

Problem here in your code is, that it does not close thread in kill function
Function kill does kill your subprocess command. But it does not end thread.
Note: Use -9 if you want to force fully kill the process.
Solution to your problem is. use process.wait() (this will close your thread) in your kill function.
def kill():
pid = process.pid
cmd = "sudo kill -9 %s" % pid . # -9 to kill force fully
print(cmd)
print(os.system(cmd))
print(process.wait()) # this will print -9 if killed force fully, else -15.

You may try this one too. Here what I tried to do is setting a session id to the group of processes that may get created during the subprocess call and when you want kill, a signal is sent to the process group leader, it's transmitted to all of the child processes of this group.
import signal
process = subprocess.Popen(command,
stderr=subprocess.PIPE,
stdout=subprocess.PIPE,
preexec_fn=os.setsid) # add session id to group
print(process.pid)
def kill():
pid = process.pid
cmd = "sudo kill %s" % pid
print(cmd)
os.killpg(os.getpgid(process.pid), signal.SIGTERM) # send signal to the group
time.sleep(2)
kill()

difference between subprocess check_output, Popen, getoutput python

I'm using python 3.6 in Windows, and my aim is to run a cmd command and save the output as a string in a variable.
I'm using subprocess and its objects like check_output, Popen and Communicate, and getoutput. But here is my problem with these:
subprocess.check_output the problem is if the code returns non-zero it raises an exception and I can't read the output, for example, executing the netstat -abcd.
stdout_value = (subprocess.check_output(command, shell=True, stdin=subprocess.PIPE, stderr=subprocess.DEVNULL, timeout=self.timeout)).decode()
subprocess.Popen and communicate() the problem is some commands like netstat -abcd returns empty from communicate().
self.process = subprocess.Popen(command, shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
stdin=subprocess.PIPE)
try:
self.process.wait(timeout=5)
stdout_value = self.process.communicate()[0]
except:
self.process.kill()
self.process.wait()
subprocess.getoutput(Command) is ok but there is no timeout so my code would block forever on executing some commands like netstat. I also tried to run it as a thread but the code is blocking and I can't stop the thread itself.
stdout_value = subprocess.getoutput(command)
What I want is to run any cmd commands (blocking like netstat or nonblocking like dir) with timeout for example if the user executes netstat it only shows the lines generated in timeout and then kills it.
Thanks.
EDIT------
According to Jean's answer, I rewrote the code but the timeout doesn't work in running some commands like netstat.
# command = "netstat"
command = "test.exe" # running an executable program can't be killed after timeout
self.process = subprocess.run(command, shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
stdin=subprocess.PIPE,
timeout=3,
universal_newlines=True
)
stdout_value = self.process.stdout

subprocess.run() with timeout doesn't seem to run properly on Windows.
You can try running the subprocess within a Timer-thread or in case of you dont need communicate(), you can do something like this:
import time
import subprocess
#cmd = 'cmd /c "dir c:\\ /s"'
#cmd = ['calc.exe']
cmd = ['ping', '-n', '25', 'www.google.com']
#_stdout = open('C:/temp/stdout.txt', 'w')
#_stderr = open('C:/temp/stderr.txt', 'w')
_stdout = subprocess.PIPE
_stderr = subprocess.PIPE
proc = subprocess.Popen(cmd, bufsize=0, stdout=_stdout, stderr=_stderr)
_startTime = time.time()
while proc.poll() is None and proc.returncode is None:
if (time.time() - _startTime) >= 5:
print ("command ran for %.6f seconds" % (time.time() - _startTime))
print ("timeout - killing process!")
proc.kill()
break
print (proc.stdout.read())
It works for all three commands on Win7/py3.6, but not for the 'killed-netstat' issue!

Timeout subprocess while reading stdout realtime

from subprocess import Popen, PIPE, STDOUT, TimeoutExpired
cmd = 'for x in $(seq 1 3); do echo stage $x; sleep 1; done'
proc = Popen(cmd, shell=True, close_fds=True,
stdout=PIPE, stderr=STDOUT,
universal_newlines=True, start_new_session=True)
for line in proc.stdout:
print(line.strip())
Default behavior of communicate is to spit out all stdout after the proc. is terminated. However I need access to stdout while proc. is still running, but terminate it, if it's running too long.
EDIT
The following seem to work the way I want, not sure if that's correct approach?
from os import killpg
from signal import SIGKILL
from concurrent import futures
from subprocess import Popen, PIPE, STDOUT, TimeoutExpired
cmd = 'for x in $(seq 1 3); do echo stage $x; sleep 1; done'
proc = Popen(cmd, shell=True, close_fds=True,
stdout=PIPE, stderr=STDOUT,
universal_newlines=True, start_new_session=True)
def handler(proc, timeout=None):
try:
proc.wait(timeout)
except TimeoutExpired as err:
print(err)
killpg(proc.pid, SIGKILL)
exe = futures.ThreadPoolExecutor(max_workers=1)
exe.submit(handler, proc, 2)
for line in proc.stdout:
print(line.strip())

How to validate that subprocess.Popen is non blocking

A couple of answers (first, second) have mentioned that subprocess.Popen is a non blocking call.
What can be a simple example which can validate it or can be used to explain it to a beginner.
I tried the following code. It shows that "Finish" is printed before printing output of ls -lrt but as soon as I add sleep 10 before ls -lrt in command, it waits for command to finish.
import logging
import os
import subprocess
import signal
import time
log = logging.getLogger(__name__)
class Utils(object):
#staticmethod
def run_command(cmnd, env=None, cwd=None, timeout=0):
p = subprocess.Popen(cmnd, shell=True, stdin=None, bufsize=-1, env=env,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
close_fds=True, cwd=cwd, preexec_fn=os.setsid)
#stdout_val = p.communicate()[0]
stdout_val = p.stdout.read()
return p.returncode, stdout_val.strip()
if __name__ == '__main__':
print "Start"
print "Invoke command"
status, output = Utils.run_command("ls -lrt") # line - 10
#status, output = Utils.run_command("sleep 10;ls -lrt") # line - 11
for i in xrange(10):
print "Finish"
print status
print output
EDIT 1: Replacing call p.communicate() with p.stdout.read() after suggestion.

Python popen python.exe

I'm rather puzzled by why the code below doesn't print stdout and exit, instead it hangs (on windows). Any reason why?
import subprocess
from subprocess import Popen
def main():
proc = Popen(
'C:/Python33/python.exe',
stderr=subprocess.STDOUT,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE
)
proc.stdin.write(b'exit()\r\n')
proc.stdin.flush()
print(proc.stdout.read(1))
if __name__=='__main__':
main()

Replace the following:
proc.stdin.flush()
with:
proc.stdin.close()
Otherwise the subprocess python.exe will wait forever stdin to be closed.
Alternative: using communicate()
proc = Popen(...)
out, err = proc.communicate(b'exit()\r\n')
print(out) # OR print(out[:1]) if you want only the first byte to be print.