I have created a simple echo.py like this:
import sys
while True:
s = sys.stdin.readline()
s = s.strip('\n')
if s == 'exit':
break
sys.stdout.write("You typed: %s\n" % s)
It works well on the terminal.
And another program to interact with echo.py named main.py
import subprocess
if __name__ == '__main__':
proc = subprocess.Popen(['python', 'echo.py'],stdin=subprocess.PIPE,
stdout=subprocess.PIPE,stderr=subprocess.PIPE)
proc.stdin.write(b'Hello\n')
proc.stdin.flush()
print(proc.stdout.readline())
proc.stdin.write(b'Hello World\n')
proc.stdin.flush()
print(proc.stdout.readline())
proc.terminate()
The main.py just hangs forever.
The thing is if I create subprocess with ['python', '-i'], it works.
Add sys.stdout.flush() to echo.py. Buffering works differently if you run the process with Popen instead of the terminal.
I believe the problem is the while loop.. You're opening a subprocess, writing to it and flushing, and it does all that, but never finishes readline() because of the loop.
Related
I want code like this:
if True:
run('ABC.PY')
else:
if ScriptRunning('ABC.PY):
stop('ABC.PY')
run('ABC.PY'):
Basically, I want to run a file, let's say abc.py, and based on some conditions. I want to stop it, and run it again from another python script. Is it possible?
I am using Windows.
You can use python Popen objects for running processes in a child process
So run('ABC.PY') would be p = Popen("python 'ABC.PY'")
if ScriptRunning('ABC.PY) would be if p.poll() == None
stop('ABC.PY') would be p.kill()
This is a very basic example for what you are trying to achieve
Please checkout subprocess.Popen docs to fine tune your logic for running the script
import subprocess
import shlex
import time
def run(script):
scriptArgs = shlex.split(script)
commandArgs = ["python"]
commandArgs.extend(scriptArgs)
procHandle = subprocess.Popen(commandArgs, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
return procHandle
def isScriptRunning(procHandle):
return procHandle.poll() is None
def stopScript(procHandle):
procHandle.terminate()
time.sleep(5)
# Forcefully terminate the script
if isScriptRunning(procHandle):
procHandle.kill()
def getOutput(procHandle):
# stderr will be redirected to stdout due "stderr=subprocess.STDOUT" argument in Popen call
stdout, _ = procHandle.communicate()
returncode = procHandle.returncode
return returncode, stdout
def main():
procHandle = run("main.py --arg 123")
time.sleep(5)
isScriptRunning(procHandle)
stopScript(procHandle)
print getOutput(procHandle)
if __name__ == "__main__":
main()
One thing that you should be aware about is stdout=subprocess.PIPE.
If your python script has a very large output, the pipes may overflow causing your script to block until .communicate is called over the handle.
To avoid this, pass a file handle to stdout, like this
fileHandle = open("main_output.txt", "w")
subprocess.Popen(..., stdout=fileHandle)
In this way, the output of the python process will be dumped into the file.(You will have to modily the getOutput() function too for this)
import subprocess
process = None
def run_or_rerun(flag):
global process
if flag:
assert(process is None)
process = subprocess.Popen(['python', 'ABC.PY'])
process.wait() # must wait or caller will hang
else:
if process.poll() is None: # it is still running
process.terminate() # terminate process
process = subprocess.Popen(['python', 'ABC.PY']) # rerun
process.wait() # must wait or caller will hang
So basically, i want python to run another programm and wait till that program is not visible in the taskmanger and then continue with the script.
Any Ideas?
As #eryksun suggested, the subprocess module can handle the waiting as well:
import subprocess
process = subprocess.Popen(["notepad.exe"], shell=False)
process.wait()
print ("notepad.exe closed")
You could use something like this, tracking the process id of the opened program:
import subprocess, win32com.client, time
wmi=win32com.client.GetObject('winmgmts:')
process = subprocess.Popen(["notepad.exe"], shell=False)
pid = process.pid
flag = True
while flag:
flag = False
for p in wmi.InstancesOf('win32_process'):
if pid == int(p.Properties_('ProcessId')):
flag = True
time.sleep(.1)
print ("notepad.exe closed")
Output when notepad is closed:
notepad.exe closed
>>>
Here's an example of a simple way to see if something is running on Windows that uses its built-in tasklist command:
import os
import subprocess
target = 'notepad.exe'
results = subprocess.check_output(['tasklist'], universal_newlines=True)
if any(line.startswith(target) for line in results.splitlines()):
print(target, 'is running')
else:
print(target, 'is *not* running')
It can be done with pywinauto:
from pywinauto import Application
app = Application().connect(process=pid) # or connect(title_re="") or other options
app.wait_for_process_exit(timeout=50, retry_interval=0.1)
I am trying to open a subprocess but have it be detached from the parent script that called it. Right now if I call subprocess.popen and the parent script crashes the subprocess dies as well.
I know there are a couple of options for windows but I have not found anything for *nix.
I also don't need to call this using subprocess. All I need is to be able to cal another process detached and get the pid.
With linux, it's no issue at all. Just Popen(). For example, here is a little dying_demon.py
#!/usr/bin/python -u
from time import sleep
from subprocess import Popen
print Popen(["python", "-u", "child.py"]).pid
i = 0
while True:
i += 1
print "demon: %d" % i
sleep(1)
if i == 3:
i = hurz # exception
spinning off a child.py
#!/usr/bin/python -u
from time import sleep
i = 0
while True:
i += 1
print "child: %d" % i
sleep(1)
if i == 20:
break
The child continues to count (to the console), while the demon is dying by exception.
I think this should do the trick: https://www.python.org/dev/peps/pep-3143/#reference-implementation
You can create daemon which will call your subprocess, passing detach_process=True.
This might do what you want:
def cmd_detach(*command, **kwargs) -> subprocess.CompletedProcess:
# https://stackoverflow.com/questions/62521658/python-subprocess-detach-a-process
# if using with ffmpeg remember to run it with `-nostdin`
stdout = os.open(os.devnull, os.O_WRONLY)
stderr = os.open(os.devnull, os.O_WRONLY)
stdin = os.open(os.devnull, os.O_RDONLY)
command = conform(command)
if command[0] in ["fish", "bash"]:
import shlex
command = command[0:2] + [shlex.join(command[2:])]
subprocess.Popen(command, stdin=stdin, stdout=stdout, stderr=stderr, close_fds=True, start_new_session=True, **kwargs)
return subprocess.CompletedProcess(command, 0, "Detached command is async")
On Windows you might need
CREATE_NEW_PROCESS_GROUP = 0x00000200
DETACHED_PROCESS = 0x00000008
creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP
instead of start_new_session=True
I managed to get it working by doing the following using python-daemon:
process = subprocess.Popen(["python", "-u", "Child.py"])
time.sleep(2)
process.kill()
Then in Child.py:
with daemon.DaemonContext():
print("Child Started")
time.sleep(30)
print "Done"
exit()
I do process.kill() because otherwise it creates a defunct python process. The main problem I have now is that the PID that popen returns does not match the final pid of the process. I can get by this by adding a function in Child.py to update a database with the pid.
Let me know if there is something that I am missing or if this is an ok method of doing this.
fork the subprocs using the NOHUP option
I am working on a python program which implements the cmd window.
I am using subproccess with PIPE.
If for example i write "dir" (by stdout), I use communicate() in order to get the response from the cmd and it does work.
The problem is that in a while True loop, this doesn't work more than one time, it seems like the subprocess closes itself..
Help me please
import subprocess
process = subprocess.Popen('cmd.exe', shell=False, stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=None)
x=""
while x!="x":
x = raw_input("insert a command \n")
process.stdin.write(x+"\n")
o,e=process.communicate()
print o
process.stdin.close()
The main problem is that trying to read subprocess.PIPE deadlocks when the program is still running but there is nothing to read from stdout. communicate() manually terminates the process to stop this.
A solution would be to put the piece of code that reads stdout in another thread, and then access it via Queue, which allows for reliable sharing of data between threads by timing out instead of deadlocking.
The new thread will read standard out continuously, stopping when there is no more data.
Each line will be grabbed from the queue stream until a timeout is reached(no more data in Queue), then the list of lines will be displayed to the screen.
This process will work for non-interactive programs
import subprocess
import threading
import Queue
def read_stdout(stdout, queue):
while True:
queue.put(stdout.readline()) #This hangs when there is no IO
process = subprocess.Popen('cmd.exe', shell=False, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
q = Queue.Queue()
t = threading.Thread(target=read_stdout, args=(process.stdout, q))
t.daemon = True # t stops when the main thread stops
t.start()
while True:
x = raw_input("insert a command \n")
if x == "x":
break
process.stdin.write(x + "\n")
o = []
try:
while True:
o.append(q.get(timeout=.1))
except Queue.Empty:
print ''.join(o)
I need to stop the service(runs at the background in another thread) that I issued through Popen in python after I got the result, but the following approach failed(just use ping for the sake of explanation):
class sample(threading.Thread):
def __init__(self, command, queue):
threading.Thread.__init__(self)
self.command = command;
self.queue = queue
def run(self):
result = Popen(self.command, shell=True, stdout=PIPE, stderr=STDOUT)
while True:
output = result.stdout.readline()
if not self.queue.empty():
result.kill()
break
if output != "":
print output
else:
break
def main():
q = Queue()
command = sample("ping 127.0.0.1", q)
command.start()
time.sleep(10)
q.put("stop!")
command.join()
if __name__ == "__main__":
main()
After running above program, when I pgrep for ping, it's still there. How can I kill the subprocess opened by Popen? Thanks.
PS: I also tried result.terminate(), but doesn't really solve the problem either.
You don't really need to run a subprocess from a thread. Try running the subprocess without a thread. Also, you specified shell=True, so it's running the command in a shell. So there are two new processes, the shell and the command. You can also remove the shell by making shell=False.