So, I have some services running and the whatever is printed in the console is scraped by datadog agents to show in the Datadog UI. The problem is sometimes when there is an unexpected error the stacktrace is multiline and that shows up as different logs in datadog. Is there any way to automatically intercept any stdout and convert that to multiline ?
I could always try except everything and take care of that in my logger but try excepting every piece of code is probably not ideal ?
#Lenormju comment works perfectly!
I would add, if you have a logger configured to output json then you may want to change the print statement to logger.error.
logger = logging.getLogger(__name__)
def single_line_excepthook(ex_type, ex_value, ex_traceback):
logger.error(
"%s\n%s", repr(ex_value), "|".join(traceback.format_tb(ex_traceback))
)
output
{"asctime": "2021-10-16 22:06:02,310", "levelname": "ERROR", "name": "test_file", "filename": "test_file.py", "lineno": 27, "dd.service": null, "dd.env": null, "dd.version": null, "dd.trace_id": null, "dd.span_id": null,
"message": "ZeroDivisionError('division by zero')\n File \"<string>\", line 1, in <module>\n| File \"/code/test_file.py\", line 126, in consume\n 1 / 0\n"}
What you want is to change the sys.excepthook :
This function prints out a given traceback and exception to sys.stderr.
When an exception is raised and uncaught, the interpreter calls sys.excepthook with three arguments, the exception class, exception instance, and a traceback object. In an interactive session this happens just before control is returned to the prompt; in a Python program this happens just before the program exits. The handling of such top-level exceptions can be customized by assigning another three-argument function to sys.excepthook.
Default behavior is to print on multiple lines :
Traceback (most recent call last):
File "C://PycharmProjects/stackoverflow/67925591.py", line 15, in <module>
throw()
File "C://PycharmProjects/stackoverflow/67925591.py", line 10, in throw
1/0
ZeroDivisionError: division by zero
But using this code :
import sys
import traceback
def single_line_excepthook(ex_type, ex_value, ex_traceback):
print(repr(ex_value) + "|".join(traceback.format_tb(ex_traceback)).replace("\n", ""), file=sys.stderr)
def throw():
1/0
sys.excepthook = single_line_excepthook
throw()
I get a single line output :
ZeroDivisionError('division by zero',) File "C://PycharmProjects/stackoverflow/67925591.py", line 15, in <module> throw()| File "C://PycharmProjects/stackoverflow/67925591.py", line 10, in throw 1/0
Related
I am trying to use sys.excepthook
With below hook
def foo(type, value, traceback):
# how to print the line that except occurs
sys.excepthook = foo
and use like below
$ python3
>>> text that cause error
How to define foo such that text that cause error being printed?
EDIT
Let me add the full story, to make it clear. (deserve downvotes? -) )
What I want is not print the line, is
get the line
if the line match some criterion, modify then exec
eg,
if type
>>> import requests
>>> edit requests
get the line that exception occurs, i.e, edit requests
then exec edit(find_file(request))
where edit() use subprocess.call(), find_file use inspect to find the file where an object being defined.
Yes, I know ipython magics, and use it regularlly. this time I am ask to how to define it.
You can use the modul traceback and the passed traceback object's tb_lineno attribute:
import sys
import traceback as tb
def foo(type, value, traceback):
# how to print the line that except occurs
print("The line where the exception occurs: {}".format(tb.linecache.getline(tb.extract_tb(sys.last_traceback)[0].filename, traceback.tb_lineno)))
sys.excepthook = foo
int("text")
Out:
The line where the exception occurs: int("text")
as the the line int("text") line raised an exeption.
We can use the traceback module to help us.
def foo(type, value, trace):
print(trace.msg)
The traceback object will have lots of information about where the error occurred.
Why isn't the sys.excepthook function called if I try to execute this code?
import sys;
def MyExcepthook(ex_cls, ex, tb):
print("Oops! There's an Error.\n");
a=open("./ERR.txt","w"); #Fixed as suggested by unutbu BUT the problem is the same!
a.write("Oops! There's an Error.\n");
a.close();
sys.excepthook = MyExcepthook;
def main():
print(1/0);
if (__name__=="__main__"):
main();
Output:
Traceback (most recent call last):
File "C:\Users\Path\to\my\python\file.py", line 13, in <module>
main();
File "C:\Users\Path\to\my\python\file.py", line 10, in main
print(1/0);
ZeroDivisionError: division by zero
Expected Output (by print):
Oops! There's an Error.
and a new file (Err.txt) should be created (by open)
The print function doesn't show the text and the file is not created because the sys.excepthook function is not called - why?
-->EDIT
My Problem is caused by a bug in idle-python 3.4 because now i tried to run the code by interpreter python (command line) and it works! this makes my Question useless if not to warn about this bug in idle-python 3.4 i'm sorry and thanks for your help!
[SOLUTION] if someone has my same problem => Try to Run your code by command line! and not from IDE.
Your custom excepthook must not itself raise an exception:
a=open("./ERR.txt") # opens the file in read mode
should be
a=open("./ERR.txt", 'w') # open the file in write mode.
When the custom excepthook raises an exception, you should see
something like
Oops! There's an Error.
Error in sys.excepthook:
...
IOError: [Errno 2] No such file or directory: './ERR.txt'
Original exception was:
...
ZeroDivisionError: integer division or modulo by zero
PS. Don't forget to delete all those unnecessary semicolons!
import sys
def my_excepthook(ex_cls, ex, tb):
msg = "Oops! There's an Error.\n"
print(msg)
with open("./ERR.txt", 'w') as a:
a.write(msg)
sys.excepthook = my_excepthook
def main():
print(1/0)
if __name__=="__main__":
main()
I am writing a Python script that uses 3rd party modules from GDAL. The GDAL functions do not raise exceptions when an error occurs, but do send messages to stdout. Generally, the errors that occur with GDAL functions do not warrant stopping the process and I don't need to know about the error having occurred.
Is there a way I can intercept the messages that are being sent to stdout before they are printed in the console? The GDAL messages interfere with the messages that I have provided in my own code.
As described in "Python Gotchas", you can turn on Exceptions using gdal.UseExceptions(), e.g.:
from osgeo import gdal
dsrc = gdal.Open('nonexist')
# ... silence
gdal.UseExceptions()
dsrc = gdal.Open('nonexist')
# Traceback (most recent call last):
# File "<interactive input>", line 1, in <module>
# RuntimeError: `nonexist' does not exist in the file system,
# and is not recognised as a supported dataset name.
You could always then use a try except block get the actual error message string:
try:
dsrc = gdal.Open('nonexist')
except RuntimeError as e:
print(str(e))
which will print the error message:
`nonexist' does not exist in the file system,
and is not recognised as a supported dataset name.
I am building or trying to build a python script which check's a list of ip addresses (ips.txt) for a specific program using the wmi python module. However, no matter how I handle the exceptions on assets with no RPC service running the script stops running on an error. I am using python 2.7.5
Can I catch and pass the error's to proceed?
Can I catch the error and print or return a note that the ip was not alive or rpc was not running?
Thank you in advance
Here is my code:
import wmi
list = open("ips.txt")
for line in list.readlines():
asset = line.strip('\n')
c = wmi.WMI(asset)
try:
for process in c.Win32_Process (name="SbClientManager.exe"):
print asset, process.ProcessId, process.Name
except Exception:
pass
I have tried handling the exceptions in multiple way's to continue parsing my list, but the script continues to error out with the following:
Traceback (most recent call last):
File ".\check_service.py", line 12, in <module>
c = wmi.WMI(asset)
File "C:\Python27\lib\site-packages\wmi.py", line 1290, in connect
handle_com_error ()
File "C:\Python27\lib\site-packages\wmi.py", line 241, in handle_com_error
raise klass (com_error=err)
wmi.x_wmi: <x_wmi: Unexpected COM Error (-2147023174, 'The RPC server is unavailable.', None, None)>
Ultimately, I am just trying to continue the script and catch the error. Maybe a note stating that IP was not responsive would be helpful. Here are the exceptions samples that I have tried:
except Exception:
sys.exc_clear()
except:
pass
except wmi.x_wmi, x:
pass
The traceback you pasted says that the error is in the c = wmi.WMI(asset) line. You need to put that line inside the try block.
Like so:
import wmi
list = open("ips.txt")
bad_assets = []
for line in list.readlines():
asset = line.strip('\n')
try:
c = wmi.WMI(asset)
for process in c.Win32_Process (name="SbClientManager.exe"):
print asset, process.ProcessId, process.Name
except Exception:
bad_assets.append(asset)
Also, trying to catch the right exception is recommended.
I know that print(e) (where e is an Exception) prints the occurred exception
but, I was trying to find the python equivalent of Java's e.printStackTrace() that exactly traces the exception to what line it occurred and prints the entire trace of it.
Could anyone please tell me the equivalent of e.printStackTrace() in Python?
import traceback
traceback.print_exc()
When doing this inside an except ...: block it will automatically use the current exception. See http://docs.python.org/library/traceback.html for more information.
There is also logging.exception.
import logging
...
try:
g()
except Exception as ex:
logging.exception("Something awful happened!")
# will print this message followed by traceback
Output:
ERROR 2007-09-18 23:30:19,913 error 1294 Something awful happened!
Traceback (most recent call last):
File "b.py", line 22, in f
g()
File "b.py", line 14, in g
1/0
ZeroDivisionError: integer division or modulo by zero
(From http://blog.tplus1.com/index.php/2007/09/28/the-python-logging-module-is-much-better-than-print-statements/ via How to print the full traceback without halting the program?)
e.printStackTrace equivalent in python
In Java, this does the following (docs):
public void printStackTrace()
Prints this throwable and its backtrace to the standard error stream...
This is used like this:
try
{
// code that may raise an error
}
catch (IOException e)
{
// exception handling
e.printStackTrace();
}
In Java, the Standard Error stream is unbuffered so that output arrives immediately.
The same semantics in Python 2 are:
import traceback
import sys
try: # code that may raise an error
pass
except IOError as e: # exception handling
# in Python 2, stderr is also unbuffered
print >> sys.stderr, traceback.format_exc()
# in Python 2, you can also from __future__ import print_function
print(traceback.format_exc(), file=sys.stderr)
# or as the top answer here demonstrates, use:
traceback.print_exc()
# which also uses stderr.
Python 3
In Python 3, we can get the traceback directly from the exception object (which likely behaves better for threaded code).
Also, stderr is line-buffered, but the print function gets
a flush argument, so this would be immediately printed to stderr:
print(traceback.format_exception(None, # <- type(e) by docs, but ignored
e, e.__traceback__),
file=sys.stderr, flush=True)
Conclusion:
In Python 3, therefore, traceback.print_exc(), although it uses sys.stderr by default, would buffer the output, and you may possibly lose it. So to get as equivalent semantics as possible, in Python 3, use print with flush=True.
Adding to the other great answers, we can use the Python logging library's debug(), info(), warning(), error(), and critical() methods. Quoting from the docs for Python 3.7.4,
There are three keyword arguments in kwargs which are inspected: exc_info which, if it does not evaluate as false, causes exception information to be added to the logging message.
What this means is, you can use the Python logging library to output a debug(), or other type of message, and the logging library will include the stack trace in its output. With this in mind, we can do the following:
import logging
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
def f():
a = { 'foo': None }
# the following line will raise KeyError
b = a['bar']
def g():
f()
try:
g()
except Exception as e:
logger.error(str(e), exc_info=True)
And it will output:
'bar'
Traceback (most recent call last):
File "<ipython-input-2-8ae09e08766b>", line 18, in <module>
g()
File "<ipython-input-2-8ae09e08766b>", line 14, in g
f()
File "<ipython-input-2-8ae09e08766b>", line 10, in f
b = a['bar']
KeyError: 'bar'