We Keep Coding

Run a program in the background and then open another program using subprocess

On the terminal, I have two programs to run using subprocess
First, I will call ./matrix-odas & so the first program will run in the background and I can then type the second command. The first command will return some messages.
The second command ~/odas/bin/odaslive -vc ~/odas/config/odaslive/matrix_creator.cfg will open the second program and it will keep running and keep printing out text. I'd like to use subprocess to open these programs and capture both outputs.
I have never used subprocess before and following tutorials, I am writing the script on Jupyter notebook (python 3.7) in order to see the output easily.
from subprocess import Popen, PIPE
p = Popen(["./matrix-odas", "&"], stdout=PIPE, stderr=PIPE, cwd=wd, universal_newlines=True)
stdout, stderr = p.communicate()
print(stdout)
This is the code that i tried to open the first program. But Jupyter notebook always gets stuck at p.communicate() and I can't see the messages. Without running the first program in the background, I won't be able to get the command prompt after the messages are printed.
I would like to know what subprocess function should I use to solve this issue and which platform is better to test subprocess code. Any suggestions will be appreciated. Thank you so much!

From this example at the end of this section of the docs
with Popen(["ifconfig"], stdout=PIPE) as proc:
log.write(proc.stdout.read())
it looks like you can access stdout (and I would assume stderr) from the object directly. I am not sure whether you need to use Popen as a context manager to access that property or not.

How to use Popen with an interactive command? nslookup, ftp

Is there any way to use Popen with interactive commands? I mean nslookup, ftp, powershell... I read the whole subprocess documentation several times but I can't find the way.
What I have (removing the parts of the project which aren't of interest here) is:
from subprocess import call, PIPE, Popen
command = raw_input('>>> ')
command = command.split(' ')
process = Popen(command, stdout=PIPE, stderr=PIPE, shell=True)
execution = process.stdout.read()
error = process.stderr.read()
output = execution + error
process.stderr.close()
process.stdout.close()
print(output)
Basically, when I try to print the output with a command like dir, the output is a string, so I can work with the .read() on it. But when I try to use nslookup for example, the output isn't a string, so it can't be read, and the script enters in a deadlock.
I know that I can invoke nslookup in non-interactive mode, but that's not the point. I want to remove all the chances of a deadlock, and make it works with every command you can run in a normal cmd.
The real way the project works is through sockets, so the raw_input is a s.recv() and the output is sending back the output, but I have simplified it to focus on the problem.

Run a python script inside a running program

I am running a python script that launches a executable called ./abc This executable enters inside of a program and waits for a command like so:
$./abc
abc > \\waits for a command here.
What I would like to do is to enter a couple of commands like:
$./abc
abc > read_blif alu.blif
abc > resyn2
What I have so far is as follows:
import os
from array import *
os.system('./abc')
for file in os.listdir("ccts/"):
print 'read_blif ' + file + '\n'
print 'resyn2\n'
print 'print_stats\n'
print 'if -K 6\n'
print 'print_stats\n'
print 'write_blif ' + file.split('.')[0] + 'mapped.blif\n'
This however will do the following:
abc > \\stays idle and waits until I ^C and then it prints
read ...blif
resyn2
...
It prints just to the terminal. How do I make it execute this inside the program and wait until it sees the next abc > to run the next command.
Thanks

I have done something similar using subprocess.
import subprocess
cmd = './command_to_execute'
pro = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)
result = pro.stdout.read()
This will execute the command specified by cmd, then reads the result into result. It will wait for there to be a result printed to the console before executing anything after the result assignment. I believe this might be what you want though your description was a bit vague.

You may be looking for the pexpect module. Here is the basic example from pexpect's documentation
# This connects to the openbsd ftp site and
# downloads the recursive directory listing.
import pexpect
child = pexpect.spawn('ftp ftp.openbsd.org')
child.expect('Name .*: ')
child.sendline('anonymous')
child.expect('Password:')
child.sendline('noah#example.com')
child.expect('ftp> ')
child.sendline('lcd /tmp')
I think it will work the same way with abc >, if your OS is compatible with pexpect.

You need to spawn a new process using subprocess library and make two pipes one for stdin and one for stdout. Using this pipes ( that are represented in python as files) you can communicate with your process
Here is an example:
import subprocess
cmd = './full/path/to/your/abc/executable'
pro = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stdin=subprocess.PIPE)
pro.stdin.write("read_blif alu.blif \n")
pro.stdout.read(3)
You can use pro.communicate but I assume you need to take the output for every command you input. something like:
abc > command1
ouput1
abc > command2
output2 -part1
output2 -part2
output2 -part3
In this way I think the PIPE approach is more useful.
Use the dir function in python to find more info about the pro object and what methods and attributes are available dir(proc). Don't forget the help built in that will display the docstrings help(pro) or help(pro.stdin).
You are making a mistake when you run os.system this will run you program in the background an you won't have control over it. Maybe you would like to look into what input/output stream.
More reading can be done here.

If you want to pipe commands into the input of an executable, the easiest way would be to use the subprocess module. You can input into the stdin of the executable and get its output with Popen.communicate.
import subprocess
for f in os.listdir('ccts/'):
p = subprocess.Popen('./abc', stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p.communicate('read_blif ' + f + '\n'
'resyn2\n'
'print_stats\n'
'if -K 6\n'
'print_stats\n'
'write_blif ' + f.split('.')[0] + 'mapped.blif\n')
# stdout will be what the program outputs and stderr will be any errors it outputs.
This will, however, close stdin of the subprocess everytime, but is the only way to reliable communicate without deadlocking. According to https://stackoverflow.com/a/28616322/5754656, you should use pexpect for an "interactive session-like" program. This is avoided here by having multiple subprocesses, assuming you can have different children run the program.
I assume that you only need the stdouts of print_stats, so you can do (as an example, you might want to handle errors):
import subprocess
def helper_open(cmd='./abc'):
return subprocess.Popen(cmd, stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
for f in in os.listdir('cts/'):
helper_open().communicate('read_blif' + f + '\n'
'resyn2\n')
stats, _ = helper_open().communicate('print_stats\n')
# Do stuff with stats
stats, _ = helper_open().communicate('if -K 6\nprint_stats\n')
# Do more stuff with other stats
helper_open().communicate('write_blif ' + f.split('.')[0] + 'mapped.blif\n')

os.popen not working as .exe

When I run this code as a application(.exe) file it returns a blank output window.
import os
r = os.popen('cmd').read()
print(r)
(Output Window)
Could someone fix my code or suggest an altenative?
Edit:
My aim is to run the program as an executable, run the commandin the DOS console and return the output of the command.
Thanks

os.popen:
Open a pipe to or from command. The return value is an open file
object connected to the pipe, which can be read or written depending
on whether mode is 'r' (default) or 'w'.
If you just want to call an app:
os.popen('app_you_want_call.exe')
If you are trying to play interactively with the windows command line, you must do something like:
import subprocess
proc = subprocess.Popen('cmd.exe', stdin=subprocess.PIPE)
proc.stdin.write("YOUR COMMAND HERE")
More about subprocess

Opening another command line interpreter and typing commands

I use "spim" emulator to emulate the mips architecture. The way it works is that I should first have a "filename.asm" file, I then type "spim" in bash to open the command line interpreter for spim, then I can use the spim commands like loading the file and running it, etc..
I am trying to write a python script that opens the spim command line interpreter and starts typing spim commands in it. Is this possible?
Thanks.

This is going to depend on spim, which I'm not familiar with, but if you can pipe something to it, you can do the same in Python
Check out http://docs.python.org/library/subprocess.html
Something like this will get you started:
proc = subprocess.Popen('spim',shell = True,stdin = subprocess.PIPE)
proc.stdin.write("Hello world")

from subprocess import Popen, PIPE, STDOUT
# Open Pipe to communicate with spim process.
p = Popen(['spim'], stdout=PIPE, stdin=PIPE, stderr=STDOUT, shell=True)
# Write a "step 1" command to spim.
p.stdin.write('step 1\n')
p.stdin.close()
# Get the spim process output.
spim_stdout = p.stdout.read()
print(spim_stdout)