We Keep Coding

How to execute multiple commands to a command line program one by one using python?

How to execute multiple commands to a command line program(.exe) one by one using python. I need to send commands and need to read the reply on the command line for each command. All in one session since login and settings has to be done.
I tried the below code(python 3.6) but it's not working
from subprocess import Popen, PIPE
process = Popen( "cmd.exe", shell=False, universal_newlines=True,
stdin=PIPE, stdout=PIPE, stderr=PIPE )
cmd= "\"C:\temp\demo.exe\"\n"
out, err = process.communicate(cmd)
print(out)
out, err = process.communicate( 'login\n' )
print(out)

this way you can execute multiple commands
import subprocess
command_list=["C:\\temp\\demo.exe","echo hello"]
for command in command_list:
proc = subprocess.Popen(command, shell=True,
stdin=subprocess.PIPE, stdout=subprocess.PIPE)
out, err = proc.communicate()
print("{} : {}".format(command,out.decode()))
Because shell=True is not the need for cmd.exe

echo'ing commands (with returns) into Popen stdin

I'd like to run an fdisk function in python, but the returns are making this not work...
command = ['echo', '-e', "'o\nn\np\n1\n\n\nw'", '|', 'sudo', 'fdisk', '/dev/xvdm']
p = subprocess.Popen(command, stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
output, err = p.communicate()
This gives the the (incorrect) output of:
b"'o\nn\np\n1\n\n\nw' | sudo fdisk /dev/xvdm\n"
What is the equivalent?

Why not just run fdisk and send it the input yourself?
command = ['sudo', 'fdisk', '/dev/xvdm']
p = subprocess.Popen(command, stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
output, err = p.communicate(b"o\nn\np\n1\n\n\nw")

You can not use a pipe (|) in a command like this. The pipe is given as argument to the program ("echo" in your case).
scnerd give you the best way/answer to send input text to fdisk.
If you really want to keep the pipe, you shall run a "bash" program with argument "-c" (command), and give in parameter the command (including your pipe) :
command = ['bash', '-c', "echo -e 'o\nn\np\n1\n\n\nw' | sudo fdisk /dev/xvdm"]
p = subprocess.Popen(command, stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
output, err = p.communicate()

Subprocess doesn't show data from tcpdump in realtime. It shows with a pause about 10-20 seconds

So i wanna get all data from tcpdump and add some logic in the future.
I haven't such problem yet with subprocess' pipes.
I wrote code and ran tcpdump and run.py in parallel.
run.py:
from subprocess import Popen, PIPE
# process = Popen(['/usr/bin/sudo', '/usr/sbin/tcpdump', '-i', 'wlan0'], bufsize=1, stdout=PIPE, stderr=PIPE)
process = Popen('sudo tcpdump -i wlan0', bufsize=1, universal_newlines=True, shell=True, stdout=PIPE, stderr=PIPE)
while True:
print(process.stdout.readline())
Output looks like this:
I tried different values for bufsize and other but behavior hasn't changed.
How can i get output as fast as tcpdump gets with Subprocess.Popen?

It's stdio buffering in tcpdump process.
By default stdio sets the buffering mode to _IOFBF(full) on redirected streams.
Luckily tcpdump has -l option which switches the mode to line-buffered:
process = Popen('sudo tcpdump -l -i wlan0', bufsize=1, universal_newlines=True,
shell=True, stdout=PIPE, stderr=PIPE)
Andrea's solution also works but mine would work on windows too.

Try: sudo stdbuf -oL tcpdump -i wlan0
It works for me
from subprocess import Popen, PIPE
# process = Popen(['/usr/bin/sudo', '/usr/sbin/tcpdump', '-i', 'wlan0'], bufsize=1, stdout=PIPE, stderr=PIPE)
process = Popen('sudo stdbuf -oL tcpdump -i wlan0', bufsize=1, universal_newlines=True, shell=True, stdout=PIPE, stderr=PIPE)
while True:
print(process.stdout.readline())

Python Popen empty stdout with top

I try to measure CPU usage via top and python:
#!/usr/bin/python
import subprocess
from subprocess import PIPE, Popen
proc = subprocess.Popen("top -c -b -n 1 | grep /usr/local/bin/wineserver | grep -v grep | awk '{print $9}'", shell=True, stdout=PIPE, stderr=PIPE)
stdout, stderr = proc.communicate()
print len(stdout)
print len(stderr)
Output:
0
0
If I run the cmd via shell I get:
54
It seems the piping is the issue but I am not sure.
Solution:
os.system("top -c -b -n 1 | grep /usr/local/bin/wineserver | grep -v grep | awk '{print $9}' > top")
stdout = open("top").read().strip("\n")

When run interactively, top limits its display to your screen width. When run through Popen with stdout=PIPE, top is not running in a terminal and reverts to its default column width. This can be changed with an environment variable.
You could ditch the shell completely and process with python:
#!/usr/bin/python
import subprocess
from subprocess import PIPE, Popen
import os
myenv = os.environ.copy()
myenv["COLUMNS"] = "512"
proc = subprocess.Popen(["top", "-c", "-b", "-n", "1"], stdout=PIPE, stderr=PIPE, env=myenv)
for line in proc.stdout:
columns = print line.strip().split()
if columns[-1] == '/usr/local/bin/wineserver':
print columns
proc.wait()
Alternately, you can get CPU information through ps and use its filtering and output format specifiers to just grab the information you want. Below I use filters to display CPU and command line for "wineserver" only.
#!/usr/bin/python
import subprocess
from subprocess import PIPE, Popen
proc = subprocess.Popen(["ps", "-ww", "--no-headers", "-C", "wineserver", "-o", "pcpu args"],
stdout=PIPE, stderr=PIPE, env=myenv)
for line in proc.stdout:
pcpu, command = print line.strip().split(" ", 1)
print pcpu, command
proc.wait()

subprocess pid different from ps output

Why is it that the subprocess pid (Popen.pid) has different value from that the ps command returns?
I've noticed this when ps called both from inside python (with subprocess.call()) and from another terminal.
Here's a simple python file to test:
#!/usr/bin/python3
'''
Test subprocess termination
'''
import subprocess
command = 'cat'
#keep pipes so that cat doesn't complain
proc = subprocess.Popen(command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
stdin=subprocess.PIPE,
shell=True)
print('pid = %d' % proc.pid)
subprocess.call("ps -A | grep -w %s" % command,
shell=True)
proc.terminate()
proc.wait() # make sure its dead before exiting pytyhon
Usually the pid reported by ps is 1 or 2 more than that reported by Popen.pid.

Because the command is run with shell=True, the pid returned by subprocess is that of the shell process used to run the command.