The following script (which should take the output from p1 and pipe it to p2, and then output the result in the terminal) doesn't seem to work as expected.
Code as follows :
#!/binr/bin/python
# Script to lauch mosquitto_sub to monitor a mqtt topic -- SYNTAX : mosk topic
import sys
import subprocess
total = len(sys.argv)
cmdargs = str(sys.argv)
print ("The total numbers of args passed to the script: %d " % total)
print ("Args list: %s " % cmdargs)
# Pharsing args one by one
print ("Script name: %s" % str(sys.argv[0]))
print ("First argument: %s" % str(sys.argv[1]))
path = str(sys.argv[1])
print (path)
p1 = subprocess.Popen(['mosquitto_sub','-h','192.168.1.100','-t',path,'-v'], shell=False, stdout=subprocess.PIPE)
p2 = subprocess.Popen(['ts'], stdin=p1.stdout, shell=False, stdout=subprocess.PIPE)
for line in p2.stdout:
sys.stdout.write(line)
with an input as follows "./mosk2.py test/+" and whilst publishing MQTT via mosquitto on the relevant topics, I never get the expected output in the terminal
Solved - I ended up stepping neatly around the problem (cheating) as follows :
cmd = "mosquitto_sub -h 192.168.1.100 -v -t " + topic + " | ts"
print "The command which was executed is : " , cmd
def run_command(command):
process = subprocess.Popen(command, stdout=subprocess.PIPE, shell=True)
while True:
output = process.stdout.readline()
if output == '' and process.poll() is not None:
break
if output:
print output.strip()
rc = process.poll()
return rc
run_command(cmd) #This is the lauch of the actual command
Related
Here is my code:
class TestRegisters():
def application(self):
return(Popen(["i2c_control.exe"], shell=True, bufsize=0, stdout=PIPE, stdin=PIPE, stderr=PIPE, universal_newlines=True))
def run_read_command(self, address, size):
cmd = "read %s %s" % (address, size)
out = self.application().communicate(cmd)[0]
print('output is: {}\n'. format(out))
# print('error is: \n', err)
return(out)
test = TestRegisters()
for key in register:
address = key
size = int(register[key][1])*4
access = register[key][0]
data_type = register[key][2]
if access == 'Read' or access == 'Read/Write':
test.application()
read = test.run_read_command(address, size)
It prints the output correctly in terminal if I don't PIPE stdout, but when I PIPE stdout, communicate[0] prints empty output as following:
output is:
output is:
output is:
output is:
Not sure what am I doing wrong. Any help is appreciated, thanks.
In Python3 using subprocess.Popen, I would like to capture the output and command return code for this "nc -z 192.168.25.14 22" command. Here is my sample code:
#!/usr/bin/env python
import urllib.request, urllib.error, urllib.parse
import subprocess
import time
# set up null file for pipe messages
nul_f = open('/dev/null', 'w')
# try loop for clean breakout with cntl-C
try:
with open('/mnt/usbdrive/output/Urls.txt') as f:
for line in f:
data = line.split()
commands = ['nc', '-vZ', data[1], data[0]]
print(commands)
try:
ncdmp = subprocess.Popen(commands , stderr=subprocess.PIPE, stdout=subprocess.PIPE,)
except OSError:
print ("error: popen")
exit(-1) # if the subprocess call failed, there's not much point in continuing
ncdmp.wait()
if ncdmp.returncode != 0:
print(" os.wait:exit status != 0\n")
else:
print ("os.wait:", ncdmp.pid, ncdmp.returncode)
print("STDERR is ", ncdmp.stderr)
print("STDOUT is ", ncdmp.stdout)
print("STDIN is ", ncdmp.stdin)
except KeyboardInterrupt:
print('Done', i)
# clean up pipe stuff
ncdmp.terminate()
ncdmp.kill()
nul_f.close()
and an example of the output is
*Commands is ['nc', '-vZ', '192.168.25.14', '22']
os.wait:exit status != 0
STDERR is <_io.BufferedReader name=12>
STDOUT is <_io.BufferedReader name=9>
STDIN is <_io.BufferedWriter name=8>*
I'm assuming that I have an error in my code or logic, but I can not figure it out. I have used similar code for other commands like ssh and ls without issues. For this "nc" command I get the same set of output/messages regardless of whether or not there is an open port 22 at the host address.
Thanks...RDK
OK, as I did not get any useful replies to this question, I changed the code from subprocess.Popen to subprocess.run as shown below. This modification worked for my requirements as ncdup.stderr contained the information I was looking for.
commands = shlex.split("nc -vz " + "-w 5 " + data[1] + " " + data[0])
try:
ncdmp = subprocess.run(commands , capture_output=True)
except OSError:
print ("error: popen")
exit(-1)
err_line = str(ncdmp.stderr)
For example:
#!/usr/bin/env python3
# cmd.py
import time
for i in range(10):
print("Count %d" % i)
time.sleep(1)
#!/usr/bin/env python3
import subprocess
# useCmd.py
p = subprocess.Popen(['./cmd.py'], stdout=subprocess.PIPE)
out, err = p.communicate()
out = out.decode()
print(out)
In useCmd.py I can print out the output of cmd.py, but only after it's finished outputting. How can I print out it in realtime and still have it stored in a string? (sort of like tee in bash.)
If you don't have to deal with stdin, you could avoid using communicate that is blocking, and read directly from the process stdout until your stdout ends:
p = subprocess.Popen(['python', 'cmd.py'], stdout=subprocess.PIPE)
# out, err = p.communicate()
while True:
line = p.stdout.readline()
if line != '':
print line,
else:
break
related
The below python code is running in my windows 7(32 bit) machine. But it showing following error in windows 7 (64 bit) machine,
proc.stdin.write('%d\n' % i)
invalid argument
Code:
import subprocess
print 'One line at a time:'
proc = subprocess.Popen('python repeater.py',
shell=True,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
)
for i in range(10):
proc.stdin.write('%d\n' % i)
output = proc.stdout.readline()
print output.rstrip()
remainder = proc.communicate()[0]
print remainder
print
print 'All output at once:'
proc = subprocess.Popen('python repeater.py',
shell=True,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
)
for i in range(10):
proc.stdin.write('%d\n' % i)
output = proc.communicate()[0]
print output
Thanks
I am a Python newbie writing a Python (2.7) script that needs to exec a number of external applications, one of which writes a lot of output to its stderr stream. What I am trying to figure out is a concise and succinct way (in Python) to get the last N lines from that subprocess' stderr output stream.
Currently, I am running that external application from my Python script like so:
p = subprocess.Popen('/path/to/external-app.sh', stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p.communicate()
if p.returncode != 0:
print "ERROR: External app did not complete successfully (error code is " + str(p.returncode) + ")"
print "Error/failure details: ", stderr
status = False
else:
status = True
I'd like to capture the last N lines of output from its stderr stream so that they can be written to a log file or emailed, etc.
N = 3 # for 3 lines of output
p = subprocess.Popen(['/path/to/external-app.sh'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p.communicate()
if p.returncode != 0:
print ("ERROR: External app did not complete successfully "
"(error code is %s)" % p.returncode)
print "Error/failure details: ", '\n'.join(stderr.splitlines()[-N:])
status = False
else:
status = True
If the whole output can't be stored in RAM then:
import sys
from collections import deque
from subprocess import Popen, PIPE
from threading import Thread
ON_POSIX = 'posix' in sys.builtin_module_names
def start_thread(func, *args):
t = Thread(target=func, args=args)
t.daemon = True
t.start()
return t
def consume(infile, output):
for line in iter(infile.readline, ''):
output(line)
infile.close()
p = Popen(['cat', sys.argv[1]], stdout=PIPE, stderr=PIPE,
bufsize=1, close_fds=ON_POSIX)
# preserve last N lines of stdout, print stderr immediately
N = 100
queue = deque(maxlen=N)
threads = [start_thread(consume, *args)
for args in (p.stdout, queue.append), (p.stderr, sys.stdout.write)]
for t in threads: t.join() # wait for IO completion
print ''.join(queue), # print last N lines
retcode = p.wait()