When I use stdout in my Popen with stdin my process does not run.
def program():
p = subprocess.Popen(link, shell=True, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
out = p.communicate(longquery)
print out
When I run this my out comes out as (None, None) although I do have an output that shows up on the console. Why is this happening and how can I save the output that is coming out?
You need to pipe the output data per the manual:
"to get anything other than None in the result tuple, you need to give stdout=PIPE and/or stderr=PIPE too."
def program():
p = subprocess.Popen(link, shell=True, stdin=subprocess.PIPE, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
out = p.communicate(longquery)
print out
You need to set stdout=PIPE as well.
Change your code to:
from subprocess import Popen, PIPE, STDOUT
def program():
p = Popen(link, shell=True, stdin=PIPE, stderr=STDOUT, stdout=PIPE)
out = p.communicate(longquery)
print out
Related
Am seeing garbled output with below code. Can someone help on how to synchronize the output?
import subprocess
from subprocess import Popen, PIPE
def do_shell(cmd):
p = Popen('/bin/bash', shell=False, universal_newlines=True, stdin=PIPE, stdout=PIPE, stderr=PIPE)
o, e = p.communicate(cmd)
p.wait()
print('Output: ' + o.decode('utf8'))
Is it possible to wait for 5 sec after each command execution passed to communicate?
Note:Here cmd is string of commands (cmd=cmd1\ncmd2\ncmd3\n)
I want to interact with a process.
I can start the process and print out the first two lines (something like 'process successfully started').
Now I want to send a new command to the process which should return again something like 'command done' but nothing happens.
Please help me.
import subprocess
def PrintAndPraseOutput(output, p):
print(output)
if 'sucessfully' in output:
p.stdin.write('command')
cmd = ["./programm"]
p = subprocess.Popen(cmd, universal_newlines=True, shell=False, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
while p.poll() is None:
output = p.stdout.readline()
PrintAndPraseOutput(output, p)
Update:
same problem, no output after 'process successfully started'
import subprocess
def print_and_parse_output(output, p):
print(output)
if 'successfully' in output:
p.stdin.write('command\n')
with subprocess.Popen(["./programm"], universal_newlines=True, shell=False, stdout=subprocess.PIPE, stdin=subprocess.PIPE) as proc:
while proc.poll() is None:
output = proc.stdout.readline()
print_and_parse_output(output, proc)
Your I/O should be line buffered, so PrintAndPraseOutput should send a '\n' at the end of the string.
BTW, you have a couple of spelling errors. That function should be named print_and_parse_output to conform to PEP-0008, and "successfully" has 2 c's.
def print_and_parse_output(output, p):
print(output)
if 'successfully' in output:
p.stdin.write('command\n')
When using subprocess like this it's a good idea to put it in a with statement. From the subprocess.Popen` docs:
Popen objects are supported as context managers via the with
statement: on exit, standard file descriptors are closed, and the
process is waited for.
with Popen(["ifconfig"], stdout=PIPE) as proc:
log.write(proc.stdout.read())
I'm rather puzzled by why the code below doesn't print stdout and exit, instead it hangs (on windows). Any reason why?
import subprocess
from subprocess import Popen
def main():
proc = Popen(
'C:/Python33/python.exe',
stderr=subprocess.STDOUT,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE
)
proc.stdin.write(b'exit()\r\n')
proc.stdin.flush()
print(proc.stdout.read(1))
if __name__=='__main__':
main()
Replace the following:
proc.stdin.flush()
with:
proc.stdin.close()
Otherwise the subprocess python.exe will wait forever stdin to be closed.
Alternative: using communicate()
proc = Popen(...)
out, err = proc.communicate(b'exit()\r\n')
print(out) # OR print(out[:1]) if you want only the first byte to be print.
Im executing a module popen1.py and that calls popen2.py using the subprocess module,
but the output of popen2.py is not being displayed..When I display the child process id , its being displayed..Where does the output will be printed for popen2.py
call
child = subprocess.Popen(['python', 'popen2.py',"parm1='test'","parm='test1'"], shell=True,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
After the process completes, you can read child.stdout and child.stderr to get the data (since you pass subprocess.PIPE)
Alternatively, you can use oudata,errdata = child.communicate() which will wait for the subprocess to finish and then give you it's output as strings.
From a design perspective, it's better to import. I would refactor popen2.py as follows:
#popen2.py
# ... stuff here
def run(*argv):
#...
if __name__ == '__main__':
import sys
run(sys.argv[1:])
Then you can just import and run popen2.py in popen1.py:
#popen1.py
import popen2
popen2.run("parm1=test","parm=test1")
You can use child.stdout/child.stdin to comunicate with the process:
child = subprocess.Popen(['python', 'popen2.py',"parm1='test'","parm='test1'"], shell=True,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print child.stdout.readlines()
I'm running into some difficulties getting output from a subprocess stdout pipe. I'm launching some third party code via it, in order to extract log output. Up until a recent update of the third party code, everything worked fine. After the update, python has started blocking indefinitely, and not actually showing any output. I can manually launch the third party app fine and see output.
A basic version of the code I'm using:
import subprocess, time
from threading import Thread
def enqueue_output(out):
print "Hello from enqueue_output"
for line in iter(out.readline,''):
line = line.rstrip("\r\n")
print "Got %s" % line
out.close()
proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, bufsize=1)
thread = Thread(target=enqueue_output, args=(proc.stdout,))
thread.daemon = True
thread.start()
time.sleep(30)
This works perfectly if I substitute third_party.exe for this script:
import time, sys
while True:
print "Test"
sys.stdout.flush()
time.sleep(1)
So I'm unclear as to magic needs to be done to get this working with the original command.
These are all variants of the subprocess.Popen line I've tried with no success:
proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, bufsize=0)
proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, shell=True)
proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, creationflags=subprocess.CREATE_NEW_CONSOLE)
si = subprocess.STARTUPINFO()
si.dwFlags = subprocess.STARTF_USESTDHANDLES | subprocess.STARTF_USESHOWWINDOW
proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, startupinfo=si)
Edit 1:
I can't actually use .communicate() in this case. The app I'm launching remains running for long periods of time (days to weeks). The only way I could actually test .communicate() would be to kill the app shortly after it launches, which I don't feel would give me valid results.
Even the non-threaded version of this fails:
import subprocess, time
from threading import Thread
proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print "App started, reading output..."
for line in iter(proc.stdout.readline,''):
line = line.rstrip("\r\n")
print "Got: %s" % line
Edit 2:
Thanks to jdi, the following works okay:
import tempfile, time, subprocess
w = "test.txt"
f = open("test.txt","a")
p = subprocess.Popen("third_party.exe", shell=True, stdout=f,
stderr=subprocess.STDOUT, bufsize=0)
time.sleep(30)
with open("test.txt", 'r') as r:
for line in r:
print line
f.close()
First I would recommend that you simplify this example to make sure you can actually read anything. Remove the complication of the thread from the mix:
proc = subprocess.Popen("third_party.exe", stdout=subprocess.PIPE, bufsize=1)
print proc.communicate()
If that works, great. Then you are having problems possibly with how you are reading the stdout directly or possibly in your thread.
If this does not work, have you tried piping stderr to stdout as well?
proc = subprocess.Popen("third_party.exe",
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, bufsize=1)
Update
Since you say communicate() is deadlocking, here is another approach you can try to see if its a problem with the internal buffer of subprocess...
import tempfile
import subprocess
w = tempfile.NamedTemporaryFile()
p = subprocess.Popen('third_party.exe', shell=True, stdout=w,
stderr=subprocess.STDOUT, bufsize=0)
with open(w.name, 'r') as r:
for line in r:
print line
w.close()
args = ['svn','log','-v']
def foo(info=''):
import logging
import subprocess
import tempfile
try:
pipe = subprocess.Popen(args,bufsize = 0,\
stdout = subprocess.PIPE,\
stderr=subprocess.STDOUT)
except Exception as e:
logging.error(str(e))
return False
while 1:
s = pipe.stdout.read()
if s:
print s,
if pipe.returncode is None:
pipe.poll()
else:
break
if not 0 == pipe.returncode:
return False
return True
print foo()
This one should works,not thread,temp file magic.