I have written a Python script which will execute a shell command and print the output to stdout.
The shell command itself connects to a backend service and prints the output (line by line)
Issue: Only the first line of the output is printed by the script. The expected output is a multi line output as shown below:
//+This is line one of output (/something/something)
This is line 2 of output…
This is line 3 of output
…
…
…
Completed in 946 miliseconds
I have tried using the following ways to get the output of the command:
Method 1:
import os
import sys
something = sys.argv[1]
command = "here is the command part 1" + something + "command part 2"
os.system(command)
Output of the command (only the first line is printed):
//+This is line one of output (/something/something)
Method 2:
import sys
import subprocess
something = sys.argv[1]
command = "here is the command part 1" + something + "command part 2"
print subprocess.check_output(command, shell=True)
Output of the command:
//+This is line one of output (/something/something)
Method 3:
import sys
import subprocess
something = sys.argv[1]
command = "here is the command part 1" + something + "command part 2"
p = subprocess.Popen(command, stdout=subprocess.PIPE, shell=True)
(output, err) = p.communicate()
print output
How can I print the complete output?
Update: Per the suggestions in the comments below, I redirected the output of the command to a file from within the Python script as shown below:
import os
import sys
something = sys.argv[1]
ofile = sys.argv[2]
command = "here is the command part 1" + something + "command part 2 >> " + ofile
os.system(command)
f = open(ofile, "r")
data = f.read()
f.close()
print data
It still prints only the first line of the output.
However, if I redirect the output of the command from the command line to an output file, the complete output is captured.
Thanks.
Related
I am writing a python script for automation.
I need to run a linux shell command (Program: dvbv5-zap) and wait for specific command output (DVR interface '/dev/dvb/adapter0/dvr0' can now be opened). When command outputs this string python should run another shell program.
I don't know how to capture subprocess cli output, I tried with .stdout.readline(), and I got nothing.
I run a command with subprocess.Popen(['dvbv5-zap', 'args'], stdout=subprocess.PIPE)
I found my answer here: https://fredrikaverpil.github.io/2013/10/11/catching-string-from-stdout-with-python/
Code snippet:
# Imports
import os, sys, subprocess
# Build command
command = [ 'python', os.join.path('/path/to', 'scriptFile.py') ]
# Execute command
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
# Read stdout and print each new line
sys.stdout.flush()
for line in iter(p.stdout.readline, b''):
# Print line
sys.stdout.flush()
print(">>> " + line.rstrip())
# Look for the string 'Render done' in the stdout output
if 'Render done' in line.rstrip():
# Write something to stdout
sys.stdout.write('Nice job on completing the render, I am executing myFunction()\n' )
sys.stdout.flush()
# Execute something
myFunction()
I have written a Python code to generate a shell script and then run the script using subprocess.
The script file is created, but when I try to run from within the code, it is not doing anything. If I try to run the same script with a file that I have created outside the script, it is working as expected.
Here is my code :
import subprocess
import os
cwd = os.getcwd()
file_name = cwd + "/cmd_file_from_python"
fd = open(file_name,"w")
fd.write("#!/usr/local/bin/tcsh -f\n")
fd.write("echo 'PRINT FROM CMD_FILE_FROM_PYTHON'\n")
fd.close
os.chmod(file_name, 0o777)
cmd=file_name
p = subprocess.Popen(cmd,executable='/bin/ksh', shell=True, stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
(stdout,stderr) = p.communicate()
p_status = p.wait()
print "Command output : ", stdout
print "Command outerr : ", stderr
print "Command exit status/return code : ", p_status
print "================================================================"
file_name = cwd + "/cmd_file"
cmd = file_name
p = subprocess.Popen(cmd,executable='/bin/ksh', shell=True, stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
(stdout,stderr) = p.communicate()
p_status = p.wait()
print "Command output : ", stdout
print "Command outerr : ", stderr
print "Command exit status/return code : ", p_status
and the output :
Command output :
Command outerr :
Command exit status/return code : 0
================================================================
Command output : PRINT FROM CMD_FILE
Command outerr :
Command exit status/return code : 0
Here is the code of the script which I created outside the code:
$ cat cmd_file
#!/usr/local/bin/tcsh -f
echo 'PRINT FROM CMD_FILE'
IF I check both files, they only difference is the print :
$ diff cmd_file_from_python cmd_file
2c2
< echo 'PRINT FROM CMD_FILE_FROM_PYTHON'
---
> echo 'PRINT FROM CMD_FILE'
Your file is empty while the program is running:
fd = open(file_name,"w")
fd.write("#!/usr/local/bin/tcsh -f\n")
fd.write("echo 'PRINT FROM CMD_FILE_FROM_PYTHON'\n")
fd.close
Note the lack of call parens on fd.close; you never actually closed the file, so the entire contents of the file were likely sitting in Python's buffer, and never go to disk until the program ends (when the CPython reference interpreter, as an implementation detail, goes through and cleans up globals, closing the open files for you by side-effect; it might never reach disk in another interpreter).
To fix, actually call close. Or even better, switch to the much safer with statement approach, where the close is implicit, and automatic, occurring even if an exception or return causes you to exit the code early:
with open(file_name, "w") as fd:
fd.write("#!/usr/local/bin/tcsh -f\n")
fd.write("echo 'PRINT FROM CMD_FILE_FROM_PYTHON'\n")
# No need to call close; file is guaranteed closed when you exit with block
I'm trying to execute a command using Popen and search through the output using a regex to find a UserID. When I run the code below and have if reg_ex.search(line) code block commented out, the command runs successfully (although I don't have my userID). As soon as I uncomment out the regex code, the command itself fails.
Any ideas why this regex if statement makes the command itself fail when I'm just iterating through output and any idea how to get the userId I need from the output? I've already confirmed the regex itself works when run separately (although for those that want it, the output line for regex will look like this u'userId': u'userID_in–question'}.
import re
import subprocess
#command I want to run
command = "add-user " + FirstName + " " + LastName + " " + Email + " " + Phone
#Regex to find userId in output
reg_ex_find = re.compile('userId')
reg_ex_replace = re.compile("(""u'userId""|\s|:|}|')")
#call command
process = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
for line in iter(process.stdout.readline, b''):
print line
if reg_ex_find.search(line):
UserId = line
reg_ex_replace.sub('',UserId)
process.stdout.close()
process.wait()
print process.returncode
Following is the code which I am trying to execute using python
from subprocess import Popen, PIPE
cmd = 'if (-e "../a.txt") then \n ln -s ../a.txt . \n else \n echo "file is not present " \n endif'
ret_val = subprocess.call(cmd,shell="True")
When executed gives following error message
/bin/sh: -c: line 5: syntax error: unexpected end of file
In sh scripts, if is terminated with fi, not endif.
http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.html
Or just write the darn code in Python:
import os
if os.path.exists('../a.txt'):
print 'Exists'
os.symlink('../a.txt', 'a.txt')
else:
print 'Does not exist'
os.path.exists ()
os.symlink()
If you really want to run tcsh commands, then:
import shlex
import subprocess
args = ['tcsh', '-c'] + shlex.split(some_tcsh_command)
ret = suprocess.call(args)
I have a python script which tries to run an external command and look for result of the command. And it needs to use the value 'count=' from the output of the external command
COUNT_EXP = re.compile("count=(.*)")
cmd = [] # external command
p = subprocess.Popen(cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
for line in iter(p.stdout.readline, b''):
result = COUNT_EXP.match(line)
if result:
print "count= " + result.group(1)
return int(result.group(1))
When I try to run my script, my external command ("cmd") got execute and I see count=10 in the shell. But why my python can't find that and print out "count= 10? in the 'if' clause above?
p = subprocess.Popen(['python','blah.py'],stdout=subprocess.PIPE)
while True:
line = proc.stdout.readline()
if len(line) != 0:
print "success" #if this code works, expanding to your regex should also work
I wrote the following C program:
#include "stdio.h"
int main() {
printf ("count=199");
return 0;
}
... which I called countOutput.c and the following Python script, modified from yours:
import subprocess, re
COUNT_EXP = re.compile("count=(.*)")
cmd = "./countOutput" # external command
p = subprocess.Popen(cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
for line in iter(p.stdout.readline, b''):
result = COUNT_EXP.match(line)
if result:
print "count is equal to " + result.group(1)
... which I called countTest.py, and then ran:
$ python countTest.py
count is equal to 199
... which all works as expected. I'd therefore tend to agree with #kichik in thinking that the external command that you're using may be writing to stderr rather than stdout.
It might be printing that to stderr. Try redirecting that one to PIPE as well and read data from there. You can also append 2>&1 to the end of the command to get stderr redirected to stdout by the shell. You might have to add shell=True for that.