When I run a shell command in Python it does not show the output until the command is finished. the script that I run takes a few hours to finish and I'd like to see the progress while it is running.
How can I have python run it and show the outputs in real-time?
Use the following function to run your code. In this example, I want to run an R script with two arguments. You can replace cmd with any other shell command.
from subprocess import Popen, PIPE
def run(command):
process = Popen(command, stdout=PIPE, shell=True)
while True:
line = process.stdout.readline().rstrip()
if not line:
break
print(line)
cmd = f"Rscript {script_path} {arg1} {arg2}"
run(cmd)
Related
I wrote a Python script to run a terminal command that belongs to a 3rd party program.
import subprocess
DETACHED_PROCESS = 0x00000008
command = 'my cmd command'
process = subprocess.Popen(
args=command,
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
encoding="utf-8",
creationflags=DETACHED_PROCESS
)
code = process.wait()
print(process.stdout.readlines())
# Output: []
This script basically runs the command successfully. However, I'd like to print the output but process.stdout.readlines() prints an empty list.
I need to run the subprocess with creationflags due to 3rd party program's terminal command.
I've also tried creationflags=subprocess.CREATE_NEW_CONSOLE. It works but process takes too long because of 3rd party program's terminal command.
Is there a way to print the output of subprocess by using creationflags=0x00000008 ?
By the way, I can use subprocess.run etc to run the command also but I'm wondering if I can fix this.
Thank you for your time!
Edit:
I'm sorry I forgot to say I can get output if i write "dir" etc. as a command. However, I can't get any output when I write a command such as: command = '"program.exe" test'
I'm not sure that this works for your specific case, but I use subprocess.check_output when I need to capture subprocess output.
import subprocess
DETACHED_PROCESS = 0x00000008
command = 'command'
process = subprocess.check_output(
args=command,
shell=True,
stderr=subprocess.STDOUT,
encoding="utf-8",
creationflags=DETACHED_PROCESS
)
print(process)
This just returns a string of stdout.
I want to execute a set of statements which is a combination of shell and python code in python script in a child process. I am using subprocess.call() method but it only takes one shell command as input. I want to execute some python code after the shell command in the child process and exit the child process once shell+python code has finished execution.
command = "./darknet detector demo cfg/coco.data cfg/yolov3-tiny.cfg yolov3-tiny.weights {0}".format(latest_subdir)
proc = subprocess.Popen([command], stdout=subprocess.PIPE, shell=True)
(out, err) = proc.communicate()
with open(result_file, 'w') as fout:
fout.write(out)
os.system('mv {0} {1}'.format(latest_subdir,processed_dir))
s3_util.upload_results([result_file])
When you send a string to os.system, you use the syntax of the system command shell. For instance, this works just fine on Linux:
os.system('ls; wc -l so.py; echo "done"')
Separate the commands with semicolons.
"I have an issue executing commands in nested adb sub shell in python. executing "command_two" in adb shell opens a sub console in command line (and the console waits for input). how do i execute commands (give input to the console) in that console using the python.
Code:
R = subprocess.Popen('adb shell', stdin=subprocess.PIPE)
R.communicate('command_one\ncommand_two\n)
Please try this:
import time
R = subprocess.Popen('adb shell', shell=True, stdin=subprocess.PIPE)
R.communicate('command_one\n')
time.sleep(2)
R.communicate('command_two\n')
I'm new to python, still learning
What i need to do is to fork a complex command to background and continue th execution of my main program, something like this:
I do this from the linux command line (and works ok)
./pgm1 arg1 arg2 arg3 | ./pgm22 arg21 arg22 arg23 arg24 &
so the program goes to background and i can coninue my work.
How can i run the above command in my python program?
Many thanks
You can PIPE the output of the first command to the second using subprocess.Popen:
from subprocess import PIPE,Popen
p = Popen(["./pgm1" ,"arg1" ,"arg2" ,"arg3" ],stdout=PIPE)
p1 = Popen( ["./pgm22", "arg21", "arg22", "arg23" ,"arg24"],stdin=p.stdout,stdout=PIPE)
p.stdout.close()
Popen does not wait for the command to finish.
I use "spim" emulator to emulate the mips architecture. The way it works is that I should first have a "filename.asm" file, I then type "spim" in bash to open the command line interpreter for spim, then I can use the spim commands like loading the file and running it, etc..
I am trying to write a python script that opens the spim command line interpreter and starts typing spim commands in it. Is this possible?
Thanks.
This is going to depend on spim, which I'm not familiar with, but if you can pipe something to it, you can do the same in Python
Check out http://docs.python.org/library/subprocess.html
Something like this will get you started:
proc = subprocess.Popen('spim',shell = True,stdin = subprocess.PIPE)
proc.stdin.write("Hello world")
from subprocess import Popen, PIPE, STDOUT
# Open Pipe to communicate with spim process.
p = Popen(['spim'], stdout=PIPE, stdin=PIPE, stderr=STDOUT, shell=True)
# Write a "step 1" command to spim.
p.stdin.write('step 1\n')
p.stdin.close()
# Get the spim process output.
spim_stdout = p.stdout.read()
print(spim_stdout)