We Keep Coding

Run a program in the background and then open another program using subprocess

On the terminal, I have two programs to run using subprocess
First, I will call ./matrix-odas & so the first program will run in the background and I can then type the second command. The first command will return some messages.
The second command ~/odas/bin/odaslive -vc ~/odas/config/odaslive/matrix_creator.cfg will open the second program and it will keep running and keep printing out text. I'd like to use subprocess to open these programs and capture both outputs.
I have never used subprocess before and following tutorials, I am writing the script on Jupyter notebook (python 3.7) in order to see the output easily.
from subprocess import Popen, PIPE
p = Popen(["./matrix-odas", "&"], stdout=PIPE, stderr=PIPE, cwd=wd, universal_newlines=True)
stdout, stderr = p.communicate()
print(stdout)
This is the code that i tried to open the first program. But Jupyter notebook always gets stuck at p.communicate() and I can't see the messages. Without running the first program in the background, I won't be able to get the command prompt after the messages are printed.
I would like to know what subprocess function should I use to solve this issue and which platform is better to test subprocess code. Any suggestions will be appreciated. Thank you so much!

From this example at the end of this section of the docs
with Popen(["ifconfig"], stdout=PIPE) as proc:
log.write(proc.stdout.read())
it looks like you can access stdout (and I would assume stderr) from the object directly. I am not sure whether you need to use Popen as a context manager to access that property or not.

Restart ruby script after crashing (python)

I am new to Python. I want to make it show Ruby puts and also to make my ruby script start all over when it crashes:
import subprocess
cmd = "ruby script.rb"
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
output, errors = p.communicate()
Any help?

I don't think Python is a good solution for this problem. It's much better to write a simple CMD (if you're using Windows) or Bash (if you're using Unix-like OS) script:
#!/bin/bash
trap handler SIGINT
RUN=true
handler() {
RUN=false
}
while [ ${RUN} = true ]; do
python child.py
done
It gives you realtime output, which is much harder to achieve in Python. And automatically restarts script, when it exits.

Run a perl script from my python script, print the output and wait for it to finish

I have a python script which at one point is required to run a perl script, wait for it to finish, then continue.
As this case will only occur on a windows machine, I thought I could simply open a new cmd and run the perl script there, but I'm having difficulties doing so.
import os
os.system("start /wait cmd /c {timeout 10}")
should open a new cmd and sleep for 10 seconds, but it closes right away. I don't want to put the perl script in position of the timeout 10, as it is quite resource intensive.
Another idea was to use a subprocess with call or Popen and wait.
perl_script = subprocess.call(['script.pl', params])
But I'm not sure what would happen to the stdout of the perl script in such a case.
I know the location and the parameters of the perl script.
How can I run a perl script from my python script, print the output (a lot) and wait for it to finish?
edit:
As suggested by #rchang, I added the subprocess with communicate as following and it works just as intended.
import subprocess, sys
perl = "C:\\perl\\bin\\perl.exe"
perl_script "C:\\scripts\\perl\\flamethrower.pl"
params = " --mount-doom-hot"
pl_script = subprocess.Popen([perl, perl_script, params], stdout=sys.stdout)
pl_script.communicate()
These are my first lines of perl, just a quick copy/past script to test this.
print "Hello Perld!\n";
sleep 10;
print "Bye Perld!\n";

import subprocess
import sys
perl_script = subprocess.Popen(["script.pl", params], stdout=sys.stdout)
perl_script.communicate()
This should hook up the stdout of the subprocess to the stdout stream of the Python script, provided you won't actually need the Python script to output anything else meaningful during execution that may not be related to the subprocess output.

You could try:
perl_script = subprocess.check_output(["script.pl", params])
print perl_script

Opening an .exe and passing commands to it via Python subprocess?

So I have my .exe program that opens but i want to pass strings to it from my python script.
Im opening the exe like this
import subprocess
p = subprocess.Popen("E:\Work\my.exe", shell=True)
#let user fill in some tables
p.communicate("userInfo")
I want to pass a string to this program while having it just run in the background and not take over any ideas?

From the Python documentation for Using the subprocess module:
You don’t need shell=True to run a batch file, nor to run a
console-based executable.
From the Python documentation for Popen Objects :
Note that if you want to send data to the process’s stdin, you need to
create the Popen object with stdin=PIPE. Similarly, to get anything
other than None in the result tuple, you need to give stdout=PIPE
and/or stderr=PIPE too.
Code example:
from subprocess import Popen, PIPE
p = Popen(r"E:\Work\my.exe", stdin=PIPE)
p.communicate("userInfo")

Reading process output

I'm writing a simple wrapper over python debugger (pdb) and I need to parse pdb output. But I have a problem reading text from process pipe.
Example of my code:
import subprocess, threading, time
def readProcessOutput(process):
while not process.poll():
print(process.stdout.readline())
process = subprocess.Popen('python -m pdb script.py', shell=True, universal_newlines=True,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE)
read_thread = threading.Thread(target=readProcessOutput, args=(process,))
read_thread.start()
while True:
time.sleep(0.5)
When i execute given command (python -m pdb script.py) in OS shell I get results like this:
> c:\develop\script.py(1)<module>()
-> print('hello, world!')
(Pdb)
But when i run my script i get only two lines, but can't get pdb prompt. Writing commands to stdin after this has no effect. So my question is:
why I cannot read third line? How can I avoid this problem and get correct output?
Platform: Windows XP, Python 3.3

The third line can not be read by readline() because it is not terminated yet by the end of line. You see usually the cursor after "(pdb) " until you write anything + enter.
The communication to processes that have some prompt is usually more complicated. It proved to me to write also an independent thread for data writer first for easier testing the communication in order to be sure that the main thread never freezes if too much is tried to be written or read. Then it can be simplified again.