I am trying to create a function that can run any shell command and return the stdout of that command, without worrying about any exceptions that could be thrown. When testing the code written below with incorrect commands like xyz testing, I get a FileNotFoundError rather that the CalledProcessError even though the command xyz testing returns a non-zero return code when I ran it in the shell. But, when I run the same function with a similar invalid command like ls -xyz I get a CalledProcessError as expected. Does anyone know why the exception is not being thrown in a consistent manner.
try:
cmd_result = subprocess.run(
cmd,
capture_output=True,
text=True,
check=True,
)
except subprocess.CalledProcessError as exc:
click.echo(exc.stderr)
click.echo(cmd)
sys.exit(1)
Thank you
I tried putting the shell=True option, and then the error situation swaps (xyz testing) passes and ls -xyz fails.
I have been stuck on what else I can try.
A couple of things are happening. With check=True, subprocess raises CalledProcessError if the command is executed and returns a non-zero exit status. When shell=False, the file to execute couldn't be found, so you got the FileNotFound error code. The program wasn't run, there was no exit code to examine, so CalledProcessError was not an option.
When shell=True, subprocess runs an intermediate shell. Now its the shell return code that matters and that can be different than the called process. Subprocess found the intermediate shell, so there is no FileNotFound error, but couldn't find the program so returned a non-zero value, generating the other exception. Consider the command "xyz testing;exit 0" this would not generate a called process error even though xyz does not exist. That's because the shell returned a 0 exit code.
Related
I am currently trying to get the process id of a process started with subprocess.check_call.
i.e.
from subprocess import check_output
# I want to retrieve the PID of this process:
try:
p = check_output(['some broken program'])
except:
if CalledProcessError: # but Popen does not throw me a CalledProcessError even if program crashes
print("triage some stuff")
print(p.pid) # this doesn't work unless its Popen
I have tried using Popen which works perfectly, however, it doesn't seem to be able to catch when a program is terminated i.e. CalledProcessError.
Can anyone advise, whether there is a way to get around either problem? Thanks!
You have to import CalledProcessError too. Code (with check_output): from subprocess import check_output, CalledProcessError
To detect a specific exception you have to use the following syntax:
try:
statement
except ExceptionName:
another_statement
In your situation:
try:
p = check_output(['some broken program'])
except CalledProcessError:
print("triage some stuff")
You cannot reference p inside the except block as p may be undefined.
Moreover, as stated in this post, subprocess.run() is recommended over subprocess.check_output()
I have a very simple piece of code
import subprocess
print(subprocess.check_output(["pidof","ffmpeg"]))
that should print the PID(s) of processes named ffmpeg (see here). However, I always get the following error:
subprocess.CalledProcessError: Command '['pidof', 'ffmpeg']' returned non-zero exit status 1
for python2 and python3. What am I doing wrong?
From man pidof:
EXIT STATUS
0 At least one program was found with the requested name.
1 No program was found with the requested name.
You just don't have any process named ffmpeg.
you can use try except to avoid the execution from blocking
use this
import subprocess
try:
print(subprocess.check_output(["pidof","ffmpeg"]))
except subprocess.CalledProcessError:
print("no process named ffmpeg")
you are getting error because if pidof ffmpeg gives no output and using
print(subprocess.check_output(["pidof","ffmpeg"])) we are expecting output from that command.
also you can use
print(subprocess.getoutput("pidof ffmpeg"))
which will give no error even if output from that command is none
if you check the library method check_output you can find this
def check_output(*popenargs, timeout=None, **kwargs):
r"""Run command with arguments and return its output.
If the exit code was non-zero it raises a CalledProcessError. The
CalledProcessError object will have the return code in the returncode
attribute and output in the output attribute.
The arguments are the same as for the Popen constructor.... """
Is there a doc somewhere which indicates what the different return codes of python's subprocess check_output() command means? I'm seeing the returncode equal 3, and I have no idea what is setting that value and/or what it means.
Example code:
try:
output = subprocess.check_output(cmd,
stderr=subprocess.STDOUT)
except subprocess.CalledProcessError as e:
print e.returncode
This code is printing 3.
The Python subprocess.check_output() doesn't itself return a code, it returns the output of the command being run. The doco can be found here.
If you're seeing an error code in that string, it's almost certainly specific to whatever you're trying to run, not a result of subprocess itself.
If, however, you're capturing the CalledProcessError exception caused by a non-zero return code (which can be extracted from the returncode attribute), that's still specific to whatever you're running.
In both cases, the documentation you need to check is that of whatever tool subprocess is running.
For example, I have a command commandA and want to get the the exit code after commandA is executed. CommandA is expected to be failed, so the exit code we should get is 1.
If I type command in the terminal as commandA;echo $?, a 1 get displayed on the screen. However, when I do it with python, things went wrong.
I have tried to call commandA with os.system(commandA) or subprocess.call(commandA.split()), and then call os.popen('echo $?').read(), results are 0.
os.popen('commandA;echo $?').read() gives me a correct result but the process of commandA is not displayed in the screen, which is what I don't want it happens.
subprocess.call returns the exit code directly:
exit_code = subprocess.call(commandA.split())
The reason your attempts with echo $? are not working is that both echo (typically) and $? (certainly) are constructs of the shell, and don't exist in Python.
It kind of depends on python version. You can do:
result=subprocess.check_output(['commandA'],Shell='True')
For 3.x you do:
result = subprocess.run(['ls', '-l'], stdout=subprocess.PIPE)
Or you can do with a try catch to see only errors. Something like:
try:
output = subprocess.check_output(["command"])
except subprocess.CalledProcessError as e:
errorCode = e.returncode
allocating I want to launch a process and retrieve the stdout and stderr.
I don t really care about getting this in real time.
I wanted to use subprocess.check_ouput(), but the process might fail.
After reading StackOverflow and the Python docs I added a try .. catch block:
def execute(cmd,timeinsec=60):
print("execute ",cmd, " with time out ",timeinsec)
try:
output = subprocess.check_output(cmd, stderr=subprocess.STDOUT, timeout=timeinsec,universal_newlines=True)
except subprocess.TimeoutExpired:
print("timeout expired")
return "",2
except subprocess.CalledProcessError:
print("called process failed")
return "",1
print ('The command returned the following back to python:'+output)
return output,0
But when I print the output with output.decode('utf-8')
I just get the first line of the output.
Note : I'm running this in the MSys environment distributed with Msysgit 1.8 on windows.
Do you have any idea of what can be wrong?
Do you know any better way to do this?
You must be using python-3.x. Please tag your question accordingly. Also, I am not sure why you are calling read() method of output. The output is a byte string and does not have a read() method. The following code works for me:
#! /usr/bin/env python3
import subprocess
try :
retcode = 0
cmd = ["/bin/ls", "/usr/local"]
output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
except subprocess.CalledProcessError as e:
output = e.output
retcode = e.returncode
print(output.decode('utf-8'))
print(retcode)
The output is:
bin
etc
games
include
lib
man
sbin
share
src
0
If I trigger an error by replacing /usr/local with /usr/localfoo (which does not exist), then the output is:
/bin/ls: cannot access /usr/localfoo: No such file or directory
2
Finally, you can add universal_newlines=True to check_output() call and not have to worry about calling decode() on the output:
...
output = subprocess.check_output(cmd, stderr=subprocess.STDOUT,universal_newlines=True)
...
...
print(output)
Please take the above example and see if you can make it reproduce your problem. If you can reproduce the problem, please post your code, its output, and all error messages (copy, paste, and reformat for SO).
Solution
The problem was that the application in windows launched in the subprocess was allocating a console (built using Visual) and the stdout of the process was already redirected outside, and only one print was done in the original cout before this redirection