So, I'm trying to check if the command was successful or not when doing subprocess command.
I'm really bad at explaining but just look at my example:
Here's my code
output = subprocess.getoutput("sdf")
print(output)
I want to check if the output is:
'sdf' is not recognized as an internal or external command,
operable program or batch file.
I tried this code:
error_temp = fr"'sdf' is not recognized as an internal or external command, operable program or batch file."
if output == error_temp:
print("'sdf' was not recognized by this system, please register this command and try again later.")
else:
print(output)
But it's not really working, I think it's got to do with a skip line in the output...
Any help is appreciated, thanks.
EDIT:
I fixed this problem thanks to #Cristian
Here's my updated code:
status = subprocess.getstatusoutput("sdf")
print(status[0])
You can use the getstatusoutput function from the same package. It returns a tuple with the exit code and the message. If the exit code is 0, it is considered as a successful completion. Other codes indicate an abnormal completion.
I just want to show an alternative to getstatusoutput, an older method that always launches a shell to execute your program, which may be inefficient if you do not need the facilities that a shell provides (such as wildcard expansion).
The following uses subprocess.run (which can also use a shell to execute your program if you specify shell=True). The first example does not capture the output from the executed program and the second example does. The program being run is a small Python program, test.py, executed with the command python test.py
test.py
print('It works.\n')
Example 1 -- Do not capture output
import subprocess
completed_process = subprocess.run(['python', 'test.py'])
print(completed_process.returncode)
Prints:
It works.
0
Example 2 -- Capture output
import subprocess
completed_process = subprocess.run(['python', 'test.py'], capture_output=True, text=True)
print(completed_process.returncode)
print(completed_process.stdout)
Prints:
0
It works.
Related
Using Python 3, I want to execute an external program, interact with it by providing some text into standard input, and then print the result.
As an example, I created the following external program, called test.py:
print('Test Program')
print('1 First option, 2 Second Option')
choice = input()
if choice == '1':
second_param = input('Insert second param: ')
result = choice + ' ' + second_param
print(result)
If I run this program directly, it works as expected. If I provide the input 1 and then 2, the result is 1 2.
I want to run this program in another script and interact with it to print the same result.
After reading the documentation for subprocess, and checking out similar questions on SO, I ended up with the following:
EXTERNAL_PROG = 'test.py'
p = Popen(['py', EXTERNAL_PROG], stdout=PIPE, stdin=PIPE, shell=True)
print(p.stdout.readline().decode('utf-8'))
print(p.stdout.readline().decode('utf-8'))
p.stdin.write(b'1\n')
p.stdin.write(b'2\n')
print(p.stdout.readline().decode('utf-8'))
However, when I run the code, the program freezes after printing 1 First option, 2 Second Option, and I need to restart my shell. This is probably caused by the fact that subprocess.stdout.readline() expects to find a newline character, and the prompt for the second param doesn’t contain one.
I found 2 SO questions that talk about something similar but I couldn’t get it to work.
Here, the answer recommends using the pexpect module. I tried to adapt the code to my situation but it didn’t work.
Here, the suggestion is to use -u, but adding it didn’t change anything.
I know that a solution can be found by modifying test.py, but this is not possible in my case since I need to use another external program and this is just a minimal example based on it.
If you have fixed input to your program (means input not changing at run time) then this solution can be relevant.
Answer
First create file.
Input file. name it input.txt and put 1 2 in it
command = "python test.py < input.txt > output.txt 2>&1"
# now run this command
os.system(command)
When you run this, you will find output.txt in same directory. If your program is executed successfully then output.txt contains output of code test.py but if your code gives any error then error is in output.txt.
Answer As You Want
main.py become
import sys
from subprocess import PIPE, Popen
EXTERNAL_PROG = 'test.py'
p = Popen(['python3', EXTERNAL_PROG], stdout=PIPE, stdin=PIPE, stderr=PIPE)
print(p.stdout.readline())
print(p.stdout.readline())
p.stdin.write(b'1\n')
p.stdin.write(b'2\n')
p.stdin.flush()
print(p.stdout.readline())
print(p.stdout.readline())
This question already has answers here:
Running shell command and capturing the output
(21 answers)
Closed 2 years ago.
I want to get the stdout in a variable after running the os.system call.
Lets take this line as an example:
batcmd="dir"
result = os.system(batcmd)
result will contain the error code (stderr 0 under Windows or 1 under some linux for the above example).
How can I get the stdout for the above command without using redirection in the executed command?
If all you need is the stdout output, then take a look at subprocess.check_output():
import subprocess
batcmd="dir"
result = subprocess.check_output(batcmd, shell=True)
Because you were using os.system(), you'd have to set shell=True to get the same behaviour. You do want to heed the security concerns about passing untrusted arguments to your shell.
If you need to capture stderr as well, simply add stderr=subprocess.STDOUT to the call:
result = subprocess.check_output([batcmd], stderr=subprocess.STDOUT)
to redirect the error output to the default output stream.
If you know that the output is text, add text=True to decode the returned bytes value with the platform default encoding; use encoding="..." instead if that codec is not correct for the data you receive.
These answers didn't work for me. I had to use the following:
import subprocess
p = subprocess.Popen(["pwd"], stdout=subprocess.PIPE)
out = p.stdout.read()
print out
Or as a function (using shell=True was required for me on Python 2.6.7 and check_output was not added until 2.7, making it unusable here):
def system_call(command):
p = subprocess.Popen([command], stdout=subprocess.PIPE, shell=True)
return p.stdout.read()
import subprocess
string="echo Hello world"
result=subprocess.getoutput(string)
print("result::: ",result)
I had to use os.system, since subprocess was giving me a memory error for larger tasks. Reference for this problem here. So, in order to get the output of the os.system command I used this workaround:
import os
batcmd = 'dir'
result_code = os.system(batcmd + ' > output.txt')
if os.path.exists('output.txt'):
fp = open('output.txt', "r")
output = fp.read()
fp.close()
os.remove('output.txt')
print(output)
I would like to expand on the Windows solution. Using IDLE with Python 2.7.5, When I run this code from file Expts.py:
import subprocess
r = subprocess.check_output('cmd.exe dir',shell=False)
print r
...in the Python Shell, I ONLY get the output corresponding to "cmd.exe"; the "dir" part is ignored. HOWEVER, when I add a switch such as /K or /C ...
import subprocess
r = subprocess.check_output('cmd.exe /K dir',shell=False)
print r
...then in the Python Shell, I get all that I expect including the directory listing. Woohoo !
Now, if I try any of those same things in DOS Python command window, without the switch, or with the /K switch, it appears to make the window hang because it is running a subprocess cmd.exe and it awaiting further input - type 'exit' then hit [enter] to release. But with the /K switch it works perfectly and returns you to the python prompt. Allrightee then.
Went a step further...I thought this was cool...When I instead do this in Expts.py:
import subprocess
r = subprocess.call("cmd.exe dir",shell=False)
print r
...a new DOS window pops open and remains there displaying only the results of "cmd.exe" not of "dir". When I add the /C switch, the DOS window opens and closes very fast before I can see anything (as expected, because /C terminates when done). When I instead add the /K switch, the DOS window pops open and remain, AND I get all the output I expect including the directory listing.
If I try the same thing (subprocess.call instead of subprocess.check_output) from a DOS Python command window; all output is within the same window, there are no popup windows. Without the switch, again the "dir" part is ignored, AND the prompt changes from the python prompt to the DOS prompt (since a cmd.exe subprocess is running in python; again type 'exit' and you will revert to the python prompt). Adding the /K switch prints out the directory listing and changes the prompt from python to DOS since /K does not terminate the subprocess. Changing the switch to /C gives us all the output expected AND returns to the python prompt since the subprocess terminates in accordance with /C.
Sorry for the long-winded response, but I am frustrated on this board with the many terse 'answers' which at best don't work (seems because they are not tested - like Eduard F's response above mine which is missing the switch) or worse, are so terse that they don't help much at all (e.g., 'try subprocess instead of os.system' ... yeah, OK, now what ??). In contrast, I have provided solutions which I tested, and showed how there are subtle differences between them. Took a lot of time but...
Hope this helps.
commands also works.
import commands
batcmd = "dir"
result = commands.getoutput(batcmd)
print result
It works on linux, python 2.7.
On the terminal, I have two programs to run using subprocess
First, I will call ./matrix-odas & so the first program will run in the background and I can then type the second command. The first command will return some messages.
The second command ~/odas/bin/odaslive -vc ~/odas/config/odaslive/matrix_creator.cfg will open the second program and it will keep running and keep printing out text. I'd like to use subprocess to open these programs and capture both outputs.
I have never used subprocess before and following tutorials, I am writing the script on Jupyter notebook (python 3.7) in order to see the output easily.
from subprocess import Popen, PIPE
p = Popen(["./matrix-odas", "&"], stdout=PIPE, stderr=PIPE, cwd=wd, universal_newlines=True)
stdout, stderr = p.communicate()
print(stdout)
This is the code that i tried to open the first program. But Jupyter notebook always gets stuck at p.communicate() and I can't see the messages. Without running the first program in the background, I won't be able to get the command prompt after the messages are printed.
I would like to know what subprocess function should I use to solve this issue and which platform is better to test subprocess code. Any suggestions will be appreciated. Thank you so much!
From this example at the end of this section of the docs
with Popen(["ifconfig"], stdout=PIPE) as proc:
log.write(proc.stdout.read())
it looks like you can access stdout (and I would assume stderr) from the object directly. I am not sure whether you need to use Popen as a context manager to access that property or not.
I'm struggling to get some python script to start a subprocess, wait until it completes and then retrieve the required data. I'm quite new to Python.
The command I wish to run as a subprocess is
./bin.testing/Eva -t --suite="temp0"
Running that command by hand in the Linux terminal produces:
in terminal mode
Evaluation error = 16.7934
I want to run the command as a python sub-process, and receive the output back. However, everything I try seems to skip the second line (ultimately, it's the second line that I want.) At the moment, I have this:
def job(self,fen_file):
from subprocess import Popen, PIPE
from sys import exit
try:
eva=Popen('{0}/Eva -t --suite"{0}"'.format(self.exedir,fen_file),shell=True,stdout=PIPE,stderr=PIPE)
stdout,stderr=eva.communicate()
except:
print ('Error running test suite '+fen_file)
exit("Stopping")
print(stdout)
.
.
.
return 0
All this seems to produce is
in terminal mode
0
with the important line missing. The print statement is just so I can see what I am getting back from the sub-process -- the intention is that it will be replaced with code that processes the number from the second line and returns the output (here I'm just returning 0 just so I can get this particular bit to work first. The caller of this function prints the result, which is why there is a zero at the end of the output.) exedir is just the directory of the executable for the sub-process, and fen-file is just an ascii file that the sub-process needs. I have tried removing the 'in terminal mode' from the source code of the sub-process and re compiling it, but that doesn't work -- it still doesn't return the important second line.
Thanks in advance; I expect what I am doing wrong is really very simple.
Edit: I ought to add that the subprocess Eva can take a second or two to complete.
Since the 2nd line is an error message, it's probably stored in your stderr variable!
To know for sure you can print your stderr in your code, or you can run the program on the command line and see if the output is split into stdout and stderr. One easy way is to do ./bin.testing/Eva -t --suite="temp0" > /dev/null. Any messages you get are stderr since stdout is redirected to /dev/null.
Also, typically with Popen the shell=True option is discouraged unless really needed. Instead pass a list:
[os.path.join(self.exedir, 'Eva'), '-t', '--suite=' + fen_file], shell=False, ...
This can avoid problems down the line if one of your arguments would normally be interpreted by the shell. (Note, I removed the ""'s, because the shell would normally eat those for you!)
Try using subprocess check_output.
output_lines = subprocess.check_output(['./bin.testing/Eva', '-t', '--suite="temp0"'])
for line in output_lines.splitlines():
print(line)
I am trying to compile a set of lines and execute them and append the output to text file. Instead of writing the same thing, I used a python script to compile and execute in background.
import subprocess
subprocess.call(["ifort","-openmp","mod1.f90","mod2.f90","pgm.f90","-o","op.o"])
subprocess.call(["nohup","./op.o",">","myout.txt","&"])
The program pgm.f90 is getting compliled using the ifort compiler, but the ouput is not getting appended to myout.txt. Instead it is appending output to nohup.out and the program is not running in the background even after specifying "&" in the python script.
What obvious error have I made here?
Thanks in advance
You can call a subprocess as if you were in the shell by using Popen() with the argument shell=True:
subprocess.Popen("nohup ./op.o > myout.txt &", shell=True)
This issue is that when you supply arguments as a list of elements, the subprocess library bypasses the shell and uses the exec syscall to directly run your program (in your case, "nohup"). Thus, rather than the ">" and "&" operators being interpreted by the shell to redirect your output and run in the background, they are being passed as literal arguments to the nohup command.
You can tell subprocess to execute your command via the shell, but this starts a whole extra instance of shell and can be wasteful. For a workaround, use the built-in redirection functionality in subprocess instead of using the shell primitives:
p = subprocess.Popen(['nohup', "./op.o"],
stdout=open('myout.txt', 'w'))
# process is now running in the background.
# if you want to wait for it to finish, use:
p.wait()
# or investigate p.poll() if you want to check to see if
# your process is still running.
For more information: http://docs.python.org/2/library/subprocess.html