Is there a way to child.py sends stout "on the fly", when running?
Or main.py needs to wait child.py to terminate?
In these scripts, main.py needs to wait 5 seconds to start printing all lines.
I want that process.stdout.readline() get the last print in child.py when child.py still running.
main.py
import subprocess
import time
process = subprocess.Popen(["./child.py"], stdout=subprocess.PIPE)
i = 1
while i < 5:
print(process.stdout.readline()) #to print, child.py needs to terminate before
time.sleep(1)
i+=1
child.py
#!/usr/bin/env python3
# coding=utf-8
import sys
import time
def run():
i = 1
while i < 5:
time.sleep(1)
print(f'ok {i}')
i+=1
if __name__ == "__main__":
run()
In child.py you wrote this:
print(f'ok {i}')
Replace it with this:
print(f'ok {i}', flush=True)
When testing interactively isatty() returns True,
so child.py will default to unbuffered behavior.
Each line of output will appear immediately.
When running as a subprocess connected to a pipe,
you are seeing it default to buffered behavior.
Use a flush() call to defeat this.
Related
I am trying to run a python file with another python file in threading with subprocess. I am able to run the file, but not able to stop it.
What I want is
I want to run the test.py with my main.py in thread and stop it by entering stop in the console.
test.py
import time
while True:
print("Hello from test.py")
time.sleep(5)
main.py
import subprocess
import _thread
processes = []
def add_process(file_name):
p = subprocess.Popen(["python", file_name], shell=True)
processes.append(p)
print("Waiting the process...")
p.wait()
while True:
try:
# user input
ui = input("> ")
if ui == "start":
_thread.start_new_thread(add_process, ("test.py",))
print("Process started.")
elif ui == "stop":
if len(processes) > 0:
processes[-1].kill()
print("Process killed.")
else:
pass
except KeyboardInterrupt:
print("Exiting Program!")
break
Output
C:\users\me>python main2.py
> start
Process started.
> Waiting the process...
Hello from test.py, after 0 seconds
Hello from test.py, after 4 seconds
> stop
Process killed.
> Hello from test.py, after 8 seconds
Hello from test.py, after 12 seconds
> stopHello from test.py, after 16 seconds
Process killed.
> Hello from test.py, after 20 seconds
>
The program is still running even after I stop it with kill function, I have tried terminate also. I need to stop that, how can I. Or, is there any alternative module for subprocess to do this?
I suspect you have just started multiple processes. What happens if you replace your stop code with this:
for proc in processes:
proc.kill()
This should kill them all.
Note that you are on windows, and on windows terminate() is an alias for kill(). It is not a good idea to rely on killing processes gracelessly anyway, and if you need to stop your test.py you would be better off having some means of communicating with it and having it exit gracefully.
Incidentally, you don't want shell=True, and you're better off with threading than with _thread.
I am working on UBUNTU and I have file main.py with a code inside:
#!/usr/bin/env python3
# coding=utf-8
import os
import time
from subprocess import Popen, PIPE, call, signal
base_path = os.path.abspath('')
path_to_file = base_path + '/test_subprocess.py'
p = Popen(['gnome-terminal', "--", path_to_file])
time.sleep(2)
os.kill(p.pid, signal.SIGKILL)
I have test_subprocess.py with code like that:
#!/usr/bin/env python3
# coding=utf-8
import time
def print_message():
while True:
print('I am working!')
time.sleep(0.5)
print_message()
I tried to kill the subprocess but after
os.kill(p.pid, signal.SIGKILL)
subprocess is still working and prints 'I am working!'
How can I finish subprocess and how to close gnome terminal?
If I selected completely wrong way. Can you show me working example?
New version of test_subprocess.py
#!/usr/bin/env python3
# coding=utf-8
import sys
from subprocess import signal
import time
def print_message():
while True:
print('I am working!')
time.sleep(0.5)
if signal.SIGKILL: # it is braking a loop when parent process terminate!
print('I am killing self!')
break
print_message()
Should I do it like above?
You could try the following:
p = Popen(['gnome-terminal', "--", path_to_file])
PIDs = p.pid
os.system("kill {0}".format(PIDs))
Popen.pid The process ID of the child process.
Note that if you set the shell argument to True, this is the process
ID of the spawned shell.
http://docs.python.org/library/subprocess.html
This will at least kill the correct process. Not sure if it will close the terminal.
Edit: to kill the process and close the terminal:
p = Popen(['gnome-terminal', '--disable-factory', '-e', path_to_file], preexec_fn=os.setpgrp)
os.killpg(p.pid, signal.SIGINT)
Credit to https://stackoverflow.com/a/34690644/15793575, whih I modified for your command:
--disable-factory is used to avoid re-using an active terminal so that we can kill newly created terminal via the subprocess handle
os.setpgrp puts gnome-terminal in its own process group so that
os.killpg() could be used to send signal to this group
Popen.pid
The process ID of the child process.
Note that if you set the shell argument to True, this is the process
ID of the spawned shell.
Try setting the shell argument of the Popen constructor to False. (p = Popen(['gnome-terminal', "--", path_to_file]) -> p = Popen(['gnome-terminal', "--", path_to_file], shell=False)). I had a similar issue not long ago - this fixed it for me.
I have a main python script which calls several subscripts --> main.py
My first sub script subscript1.py runs a few lines of code then at the end, opens an external program (putty) using subprocess. The program to be opened is a data monitor which I want to keep open the whole time.
I want to return to main.py so that subscript2.py can be run.
Problem: python code doesn't resume until external program from subprocess is closed. How can I keep subprocess open but carry on with my python code?
main.py:
import subprocess
subprocess.call(['python', 'subscript1.py'])
subprocess.call(['python', 'subscript2.py'])
subscript1.py:
import subprocess
prog_path = 'C:/Programs/PUTTY.exe'
load_config = 'config_to_load'
... lines of code to check for a condition
if outcome_value == 1:
subprocess.run(prog_path, 'load', load_config)
else:
print("error message")
If I were you I will use multiprocessing.pool
from multiprocessing import Pool
import time
import subprocess
def do_wait(_):
subprocess.call(['python','-c',"import time;time.sleep(1)"])
st = time.time()
with Pool(5) as p:
print(p.map(do_wait, [1, 2, 3]))
diff = time.time() - st
print(f"total : {diff} sec")
I have a script "run.py" that must print "Hello", launch another script "run2.py", and then terminate (do not wait for run2.py to return).
run2.py is not in the local directory and is only required to print "Hello again".
How can I do this?
# run_path = "C:/Program Files (x86)/xxx/run.py"
# run2_path = "//network_share/folder/run2.py"
**run.py**
import os
print("Hello")
# What do I do here?
# os.execl("//network_share/folder/run2.py")
exit()
**run2.py**
print("Hello again")
This seems to work for a script I have in the same folder I'm running this one in.
This should verify that the first script finishes and doesn't linger while the second script runs in its own process. It is possible on some systems, due to their configuration, the child process will terminate when the parent does. But not in this case...
I put more time into this post to add code that shows how to check if the parent process is still running. This would be a good way for the child to ensure it's exited. Also shows how to pass parameters to the child process.
# launch.py
import subprocess as sp
import os
if __name__ == '__main__':
sp.Popen(['ps']) # Print out runniing processes.
print("launch.py's process id is %s." % os.getpid())
# Give child process this one's process ID in the parameters.
sp.Popen(['python3', 'runinproc.py', str(os.getpid())])
# ^^^ This line above anwers the main question of how to kick off a
# child Python script.
print("exiting launch.py")
Other script.
# runinproc.py
import time
import subprocess as sp
import sys
import os
def is_launcher_running():
try:
# This only checks the status of the process. It doesn't
# kill it, or otherwise affect it.
os.kill(int(sys.argv[1]), 0)
except OSError:
return False
else:
return True
if __name__ == '__main__':
print("runinproc.py was launched by process ID %s" % sys.argv[1])
for i in range(100):
if is_launcher_running():
# Is launch.py still running?
print("[[ launch.py is still running... ]]")
sp.Popen(['ps']) # Print out the running processes.
print("going to sleep for 2 seconds...")
time.sleep(2)
Bash output:
Todds-iMac:pyexperiments todd$ python3 launch.py
launch.py process id is 40975.
exiting launch.py
Todds-iMac:pyexperiments todd$ runinproc.py was launched by process ID 40975
going to sleep for 2 seconds...
PID TTY TIME CMD
PID TTY TIME CMD
40866 ttys000 0:00.09 -bash
40866 ttys000 0:00.09 -bash
40977 ttys000 0:00.04 /Library/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/C
40977 ttys000 0:00.04 /Library/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/C
going to sleep for 2 seconds...
PID TTY TIME CMD
40866 ttys000 0:00.09 -bash
40977 ttys000 0:00.04 /Library/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/C
going to sleep for 2 seconds...
PID TTY TIME CMD
40866 ttys000 0:00.09 -bash
40977 ttys000 0:00.04 /Library/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/C
going to sleep for 2 seconds...
Note that the first call to the shell, ps from launch.py is executed after launch.py exited. That's why it doesn't show up in the printed process list.
subprocess is your friend, but if you need to not wait, check out the P_NOWAIT--replacing example code in https://docs.python.org/3/library/subprocess.html
EG:
pid = Popen(["/bin/mycmd", "myarg"]).pid
I don't think .communicate is what you need this time around - isn't it more for waiting?
The cleanest way to do this (since both scripts are written in pure Python) is to import the other script as a module and execute its content, placed within a function:
run.py
import os
import sys
sys.path.append("//network_share/folder/")
import run2
print("Hello")
run2.main()
exit()
run2.py
def main():
print("Hello again")
I have created a simple echo.py like this:
import sys
while True:
s = sys.stdin.readline()
s = s.strip('\n')
if s == 'exit':
break
sys.stdout.write("You typed: %s\n" % s)
It works well on the terminal.
And another program to interact with echo.py named main.py
import subprocess
if __name__ == '__main__':
proc = subprocess.Popen(['python', 'echo.py'],stdin=subprocess.PIPE,
stdout=subprocess.PIPE,stderr=subprocess.PIPE)
proc.stdin.write(b'Hello\n')
proc.stdin.flush()
print(proc.stdout.readline())
proc.stdin.write(b'Hello World\n')
proc.stdin.flush()
print(proc.stdout.readline())
proc.terminate()
The main.py just hangs forever.
The thing is if I create subprocess with ['python', '-i'], it works.
Add sys.stdout.flush() to echo.py. Buffering works differently if you run the process with Popen instead of the terminal.
I believe the problem is the while loop.. You're opening a subprocess, writing to it and flushing, and it does all that, but never finishes readline() because of the loop.