I have a strange issue here - I have an application that I'm attempting to launch from python, but all attempts to launch it from within a .py script fail without any discernable output. Testing from within VSCode debugger. Here's some additional oddities:
When I swap in notepad.exe into the .py instead of my target applications path, notepad launches ok.
When I run the script line by line from the CLI (start by launching python, then type out the next 4-5 lines of Python), the script works as expected.
Examples:
#This works in the .py, and from the CLI
import subprocess
cmd = ['C:\\Windows\\system32\\notepad.exe', 'C:\\temp\\myfiles\\test_24.xml']
pipe = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
pipe.wait()
print(pipe)
#This fails in the .py, but works ok when pasted in line by line from the CLI
import subprocess
cmd = ['C:\\temp\\temp_app\\target_application.exe', 'C:\\temp\\myfiles\\test_24.xml']
pipe = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
pipe.wait()
print(pipe)
The result is no output when running the .py
I've tried several other variants, including the following:
import subprocess
tup = 'C:\\temp\\temp_app\\target_application.exe C:\temp\test\test_24.xml'
proc = subprocess.Popen(tup)
proc.wait()
(stdout, stderr) = proc.communicate()
print(stdout)
if proc.returncode != 0:
print("The error is: " + str(stderr))
else:
print("Executed: " + str(tup))
Result:
None
The error is: None
1.082381010055542
Now this method indicates there is an error because we are returning something other than 0 and printing "The error is: None", and this is because stderror is "None". So - is it throwing an error without giving an error?
stdout is also reporting "None".
So, lets try check_call and see what happens:
print("Trying check_call")
try:
subprocess.check_call('C:\\temp\\temp_app\\target_application.exe C:\\temp\\test\\test_24.xml', shell=True)
except subprocess.CalledProcessError as error:
print(error)
Results:
Trying check_call
Command 'C:\temp\temp_app\target_application.exe C:\temp\test\test_24.xml' returned non-zero exit status 1.
I've additionally tried subprocess.run, although it is missing the wait procedure I was hoping to use.
import subprocess
tup = 'C:\\temp\\temp_app\\target_application.exe C:\temp\test\test_24.xml'
proc = subprocess.run(tup, check=True)
proc.wait()
(stdout, stderr) = proc.communicate()
print(stdout)
if proc.returncode != 0:
print("The error is: " + str(stderr))
else:
print("Executed: " + str(tup))
What reasons might be worth chasing, or what other ways of trying to catch an error might work here? I don't know how to interpret "`" as an error result.
I am trying to create subprocess to remove python a package and return stdout and stderr. I do following but I wonder is it safe to use? Today I faced /bin/bash: resource temporarily unavailable error while I am using. And when I do ps ux I saw lots of bin/bash process.
I think that function cause lots of bash terminal in the background. How should I safely close the subprocess after I got stdout and stderr? Documentation says run method is the recommended way.
def run_subprocess_command(process_command):
response = {"stdout": "", "stderr": "", "exception": ""}
try:
plugin_install_feedback.send(
sender="", message="Package install starting..")
p = subprocess.run(process_command,
universal_newlines=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=180)
response["stdout"] = p.stdout
response["stderr"] = p.stderr
return response
except Exception as err:
response["exception"] = err
return response
I am converting bash code to python.
I use mkdir of bash through subprocess.run() in python.
In the following example, subprocess.run() raise an exception.
However I could not check what the error is because I could not
get an resultant object returned by subprocess.run().
Are there any smart ways to know what the error was?
Or should not I use 'try exception' here?
import sys
import subprocess
directory = '/tmp/test_dir'
options = ''
try:
result=subprocess.run(['mkdir', options, directory], check=True)
except subprocess.CalledProcessError as ex:
print("In this example, subprocess.run() above raise an exception CalledProcessError.")
# print("I would like to check result.returncode = {0}. But it failed because object 'result' is not defined.".format(result.returncode))
except Exception as ex:
sys.stderr.write("This must not happen.")
sys.exit(1)
Thank you very much.
you can always do
import subprocess
# make the subprocess
pr = subprocess.Popen(['your', 'command', 'here'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
# blocks until process finishes
out, err = pr.communicate()
# check the return code
if pr.returncode != 0:
sys.stderr.write("oh no")
I'm trying to do a Bitcoin payment from within Python. In bash I would normally do this:
bitcoin sendtoaddress <bitcoin address> <amount>
So for example:
bitcoin sendtoaddress 1HoCUcbK9RbVnuaGQwiyaJGGAG6xrTPC9y 1.4214
If it is successful I get a transaction id as output, but if I try to transfer an amount larger than my bitcoin balance, I get the following output:
error: {"code":-4,"message":"Insufficient funds"}
In my Python program I now try to do the payment as follows:
import subprocess
try:
output = subprocess.check_output(['bitcoin', 'sendtoaddress', address, str(amount)])
except:
print "Unexpected error:", sys.exc_info()
If there's enough balance it works fine, but if there's not enough balance sys.exc_info() prints out this:
(<class 'subprocess.CalledProcessError'>, CalledProcessError(), <traceback object at 0x7f339599ac68>)
It doesn't include the error which I get on the command line though. So my question is; how can I get the outputted error ({"code":-4,"message":"Insufficient funds"}) from within Python?
According to the subprocess.check_output() docs, the exception raised on error has an output attribute that you can use to access the error details:
try:
subprocess.check_output(...)
except subprocess.CalledProcessError as e:
print(e.output)
You should then be able to analyse this string and parse the error details with the json module:
if e.output.startswith('error: {'):
error = json.loads(e.output[7:]) # Skip "error: "
print(error['code'])
print(error['message'])
I don't think the accepted solution handles the case where the error text is reported on stderr. From my testing the exception's output attribute did not contain the results from stderr and the docs warn against using stderr=PIPE in check_output(). Instead, I would suggest one small improvement to J.F Sebastian's solution by adding stderr support. We are, after all, trying to handle errors and stderr is where they are often reported.
from subprocess import Popen, PIPE
p = Popen(['bitcoin', 'sendtoaddress', ..], stdout=PIPE, stderr=PIPE)
output, error = p.communicate()
if p.returncode != 0:
print("bitcoin failed %d %s %s" % (p.returncode, output, error))
As mentioned by #Sebastian the default solution should aim to use run():
https://docs.python.org/3/library/subprocess.html#subprocess.run
Here a convenient implementation (feel free to change the log class with print statements or what ever other logging functionality you are using):
import subprocess
def _run_command(command):
log.debug("Command: {}".format(command))
result = subprocess.run(command, shell=True, capture_output=True)
if result.stderr:
raise subprocess.CalledProcessError(
returncode = result.returncode,
cmd = result.args,
stderr = result.stderr
)
if result.stdout:
log.debug("Command Result: {}".format(result.stdout.decode('utf-8')))
return result
And sample usage (code is unrelated, but I think it serves as example of how readable and easy to work with errors it is with this simple implementation):
try:
# Unlock PIN Card
_run_command(
"sudo qmicli --device=/dev/cdc-wdm0 -p --uim-verify-pin=PIN1,{}"
.format(pin)
)
except subprocess.CalledProcessError as error:
if "couldn't verify PIN" in error.stderr.decode("utf-8"):
log.error(
"SIM card could not be unlocked. "
"Either the PIN is wrong or the card is not properly connected. "
"Resetting module..."
)
_reset_4g_hat()
return
Trying to "transfer an amount larger than my bitcoin balance" is not an unexpected error. You could use Popen.communicate() directly instead of check_output() to avoid raising an exception unnecessarily:
from subprocess import Popen, PIPE
p = Popen(['bitcoin', 'sendtoaddress', ..], stdout=PIPE)
output = p.communicate()[0]
if p.returncode != 0:
print("bitcoin failed %d %s" % (p.returncode, output))
Since Python 3.5, subprocess.run() supports check argument:
If check is true, and the process exits with a non-zero exit code, a CalledProcessError exception will be raised. Attributes of that exception hold the arguments, the exit code, and stdout and stderr if they were captured.
A simple example that will raise and print out CalledProcessError:
import subprocess
try:
subprocess.run("exit 1", shell=True, check=True, timeout=15, capture_output=True)
except subprocess.CalledProcessError as e:
print(e) # Output: Command 'exit 1' returned non-zero exit status 1.
There are good answers here, but in these answers, there has not been an answer that comes up with the text from the stack-trace output, which is the default behavior of an exception.
If you wish to use that formatted traceback information, you might wish to:
import traceback
try:
check_call( args )
except CalledProcessError:
tb = traceback.format_exc()
tb = tb.replace(passwd, "******")
print(tb)
exit(1)
As you might be able to tell, the above is useful in case you have a password in the check_call( args ) that you wish to prevent from displaying.
This did the trick for me. It captures all the stdout output from the subprocess(For python 3.8):
from subprocess import check_output, STDOUT
cmd = "Your Command goes here"
try:
cmd_stdout = check_output(cmd, stderr=STDOUT, shell=True).decode()
except Exception as e:
print(e.output.decode()) # print out the stdout messages up to the exception
print(e) # To print out the exception message
Based on the answer of #macetw I print the exception directly to stderr in a decorator.
Python 3
from functools import wraps
from sys import stderr
from traceback import format_exc
from typing import Callable, Collection, Any, Mapping
def force_error_output(func: Callable):
#wraps(func)
def forced_error_output(*args: Collection[Any], **kwargs: Mapping[str, Any]):
nonlocal func
try:
func(*args, **kwargs)
except Exception as exception:
stderr.write(format_exc())
stderr.write("\n")
stderr.flush()
raise exception
return forced_error_output
Python 2
from functools import wraps
from sys import stderr
from traceback import format_exc
def force_error_output(func):
#wraps(func)
def forced_error_output(*args, **kwargs):
try:
func(*args, **kwargs)
except Exception as exception:
stderr.write(format_exc())
stderr.write("\n")
stderr.flush()
raise exception
return forced_error_output
Then in your worker just use the decorator
#force_error_output
def da_worker(arg1: int, arg2: str):
pass
I think most of previous answers are correct, in my case I needed to do this on Windows server and command was a Powershell, for that this worked really nicely for me:
try:
print("inpgoress")
cmd_exec="Get-Date"
print(cmd_aws)
subprocess.run(['powershell', '-Command', cmd_exec],shell=False,check=True,capture_output=True,text=True,encoding="utf-8")
except Exception as e:
print(e)
print("ERROR: something went wrong executing powershell command")
raise e
The subprocess invoked needs to be told to capture the output in the invoked program and raise the exception. It's simple to do it.
Firstly, Use
subprocess.run() instead of subprocess.call()
Let's assume u wanna python script called "Vijay.py".
For raising the exception, use the following;
subprocess.run("py vijay.py", check=True, capture_output=True, shell=True)
The above method then can be put in try and except block to immediately raise the error or can use sys.exit(1) :any non-zero exit is fine
try:
subprocess.call("py vijay.py", check=True, capture_output=True, shell=True)
except Exception as e:
print("Exception raised: ", e)
and body of vijay.py can be as follows;
vijay.py
try:
Your code is here...
except Exception as e:
sys.exit(1) // or can even use raise Exception("ur own exception to raise:)
enter code here
I am writing a script which can take a file name as input, compile it and run it.
I am taking the name of a file as input(input_file_name). I first compile the file from within python:
self.process = subprocess.Popen(['gcc', input_file_name, '-o', 'auto_gen'], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT, shell=False)
Next, I'm executing the executable using the same(Popen) call:
subprocess.Popen('./auto_gen', stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT, shell=False)
In both cases, I'm catching the stdout(and stderr) contents using
(output, _) = self.process.communicate()
Now, if there is an error during compilation, I am able to catch the error because the returncode is 1 and I can get the details of the error because gcc sends them on stderr.
However, the program itself can return a random value even on executing successfully(because there might not be a "return 0" at the end). So I can't catch runtime errors using the returncode. Moreover, the executable does not send the error details on stderr. So I can't use the trick I used for catching compile-time errors.
What is the best way to catch a runtime error OR to print the details of the error? That is, if ./auto_gen throws a segmentation fault, I should be able to print either one of:
'Runtime error'
'Segmentation Fault'
'Program threw a SIGSEGV'
Try this. The code runs a subprocess which fails and prints to stderr. The except block captures the specific error exit code and stdout/stderr, and displays it.
#!/usr/bin/env python
import subprocess
try:
out = subprocess.check_output(
"ls non_existent_file",
stderr=subprocess.STDOUT,
shell=True)
print 'okay:',out
except subprocess.CalledProcessError as exc:
print 'error: code={}, out="{}"'.format(
exc.returncode, exc.output,
)
Example output:
$ python ./subproc.py
error: code=2, out="ls: cannot access non_existent_file: No such file or directory
"
If ./autogen is killed by a signal then self.process.returncode (after .wait() or .communicate()) is less than zero and its absolute value reports the signal e.g., returncode == -11 for SIGSERV.
please check following link for runtime errors or output of subprocess
https://www.endpoint.com/blog/2015/01/28/getting-realtime-output-using-python
def run_command(command):
process = subprocess.Popen(shlex.split(command),
stdout=subprocess.PIPE)
while True:
output = process.stdout.readline()
if output == '' and process.poll() is not None:
break
if output:
print output.strip()
rc = process.poll()
return rc