We Keep Coding

Intercept python stdout and convert to single line

So, I have some services running and the whatever is printed in the console is scraped by datadog agents to show in the Datadog UI. The problem is sometimes when there is an unexpected error the stacktrace is multiline and that shows up as different logs in datadog. Is there any way to automatically intercept any stdout and convert that to multiline ?
I could always try except everything and take care of that in my logger but try excepting every piece of code is probably not ideal ?

#Lenormju comment works perfectly!
I would add, if you have a logger configured to output json then you may want to change the print statement to logger.error.
logger = logging.getLogger(__name__)
def single_line_excepthook(ex_type, ex_value, ex_traceback):
logger.error(
"%s\n%s", repr(ex_value), "|".join(traceback.format_tb(ex_traceback))
)
output
{"asctime": "2021-10-16 22:06:02,310", "levelname": "ERROR", "name": "test_file", "filename": "test_file.py", "lineno": 27, "dd.service": null, "dd.env": null, "dd.version": null, "dd.trace_id": null, "dd.span_id": null,
"message": "ZeroDivisionError('division by zero')\n File \"<string>\", line 1, in <module>\n| File \"/code/test_file.py\", line 126, in consume\n 1 / 0\n"}

What you want is to change the sys.excepthook :
This function prints out a given traceback and exception to sys.stderr.
When an exception is raised and uncaught, the interpreter calls sys.excepthook with three arguments, the exception class, exception instance, and a traceback object. In an interactive session this happens just before control is returned to the prompt; in a Python program this happens just before the program exits. The handling of such top-level exceptions can be customized by assigning another three-argument function to sys.excepthook.
Default behavior is to print on multiple lines :
Traceback (most recent call last):
File "C://PycharmProjects/stackoverflow/67925591.py", line 15, in <module>
throw()
File "C://PycharmProjects/stackoverflow/67925591.py", line 10, in throw
1/0
ZeroDivisionError: division by zero
But using this code :
import sys
import traceback
def single_line_excepthook(ex_type, ex_value, ex_traceback):
print(repr(ex_value) + "|".join(traceback.format_tb(ex_traceback)).replace("\n", ""), file=sys.stderr)
def throw():
1/0
sys.excepthook = single_line_excepthook
throw()
I get a single line output :
ZeroDivisionError('division by zero',) File "C://PycharmProjects/stackoverflow/67925591.py", line 15, in <module> throw()| File "C://PycharmProjects/stackoverflow/67925591.py", line 10, in throw 1/0

Is it possible to continue execution after error in Jupyter? [duplicate]

How do I print the error/exception in the except: block?
try:
...
except:
print(exception)

For Python 2.6 and later and Python 3.x:
except Exception as e: print(e)
For Python 2.5 and earlier, use:
except Exception,e: print str(e)

The traceback module provides methods for formatting and printing exceptions and their tracebacks, e.g. this would print exception like the default handler does:
import traceback
try:
1/0
except Exception:
traceback.print_exc()
Output:
Traceback (most recent call last):
File "C:\scripts\divide_by_zero.py", line 4, in <module>
1/0
ZeroDivisionError: division by zero

In Python 2.6 or greater it's a bit cleaner:
except Exception as e: print(e)
In older versions it's still quite readable:
except Exception, e: print e

Python 3: logging
Instead of using the basic print() function, the more flexible logging module can be used to log the exception. The logging module offers a lot extra functionality, for example, logging messages...
into a given log file, or
with timestamps and additional information about where the logging happened.
For more information check out the official documentation.
Usage
Logging an exception can be done with the module-level function logging.exception() like so:
import logging
try:
1/0
except BaseException:
logging.exception("An exception was thrown!")
Output
ERROR:root:An exception was thrown!
Traceback (most recent call last):
File ".../Desktop/test.py", line 4, in <module>
1/0
ZeroDivisionError: division by zero
Notes
the function logging.exception() should only be called from an exception handler
the logging module should not be used inside a logging handler to avoid a RecursionError (thanks #PrakharPandey)
Alternative log-levels
It's also possible to log the exception with another log level but still show the exception details by using the keyword argument exc_info=True, like so:
logging.critical("An exception was thrown!", exc_info=True)
logging.error ("An exception was thrown!", exc_info=True)
logging.warning ("An exception was thrown!", exc_info=True)
logging.info ("An exception was thrown!", exc_info=True)
logging.debug ("An exception was thrown!", exc_info=True)
# or the general form
logging.log(level, "An exception was thrown!", exc_info=True)
Name and description only
Of course, if you don't want the whole traceback but only some specific information (e.g., exception name and description), you can still use the logging module like so:
try:
1/0
except BaseException as exception:
logging.warning(f"Exception Name: {type(exception).__name__}")
logging.warning(f"Exception Desc: {exception}")
Output
WARNING:root:Exception Name: ZeroDivisionError
WARNING:root:Exception Desc: division by zero

(I was going to leave this as a comment on #jldupont's answer, but I don't have enough reputation.)
I've seen answers like #jldupont's answer in other places as well. FWIW, I think it's important to note that this:
except Exception as e:
print(e)
will print the error output to sys.stdout by default. A more appropriate approach to error handling in general would be:
except Exception as e:
print(e, file=sys.stderr)
(Note that you have to import sys for this to work.) This way, the error is printed to STDERR instead of STDOUT, which allows for the proper output parsing/redirection/etc. I understand that the question was strictly about 'printing an error', but it seems important to point out the best practice here rather than leave out this detail that could lead to non-standard code for anyone who doesn't eventually learn better.
I haven't used the traceback module as in Cat Plus Plus's answer, and maybe that's the best way, but I thought I'd throw this out there.

In case you want to pass error strings, here is an example from Errors and Exceptions (Python 2.6)
>>> try:
... raise Exception('spam', 'eggs')
... except Exception as inst:
... print type(inst) # the exception instance
... print inst.args # arguments stored in .args
... print inst # __str__ allows args to printed directly
... x, y = inst # __getitem__ allows args to be unpacked directly
... print 'x =', x
... print 'y =', y
...
<type 'exceptions.Exception'>
('spam', 'eggs')
('spam', 'eggs')
x = spam
y = eggs

One has pretty much control on which information from the traceback to be displayed/logged when catching exceptions.
The code
with open("not_existing_file.txt", 'r') as text:
pass
would produce the following traceback:
Traceback (most recent call last):
File "exception_checks.py", line 19, in <module>
with open("not_existing_file.txt", 'r') as text:
FileNotFoundError: [Errno 2] No such file or directory: 'not_existing_file.txt'
Print/Log the full traceback
As others already mentioned, you can catch the whole traceback by using the traceback module:
import traceback
try:
with open("not_existing_file.txt", 'r') as text:
pass
except Exception as exception:
traceback.print_exc()
This will produce the following output:
Traceback (most recent call last):
File "exception_checks.py", line 19, in <module>
with open("not_existing_file.txt", 'r') as text:
FileNotFoundError: [Errno 2] No such file or directory: 'not_existing_file.txt'
You can achieve the same by using logging:
try:
with open("not_existing_file.txt", 'r') as text:
pass
except Exception as exception:
logger.error(exception, exc_info=True)
Output:
__main__: 2020-05-27 12:10:47-ERROR- [Errno 2] No such file or directory: 'not_existing_file.txt'
Traceback (most recent call last):
File "exception_checks.py", line 27, in <module>
with open("not_existing_file.txt", 'r') as text:
FileNotFoundError: [Errno 2] No such file or directory: 'not_existing_file.txt'
Print/log error name/message only
You might not be interested in the whole traceback, but only in the most important information, such as Exception name and Exception message, use:
try:
with open("not_existing_file.txt", 'r') as text:
pass
except Exception as exception:
print("Exception: {}".format(type(exception).__name__))
print("Exception message: {}".format(exception))
Output:
Exception: FileNotFoundError
Exception message: [Errno 2] No such file or directory: 'not_existing_file.txt'

Expanding off of the "except Exception as e:" solution here is a nice one liner which includes some additional info like the type of error and where it occurred.
try:
1/0
except Exception as e:
print(f"{type(e).__name__} at line {e.__traceback__.tb_lineno} of {__file__}: {e}")
Output:
ZeroDivisionError at line 48 of /Users/.../script.py: division by zero

Try this
try:
print("Hare Krishna!")
except Exception as er:
print(er)

One liner error raising can be done with assert statements if that's what you want to do. This will help you write statically fixable code and check errors early.
assert type(A) is type(""), "requires a string"

I would recommend using a try-except statement. Also, rather than using a print statement, a logging exception logs a message with level ERROR on the logger, which I find is more effective than a print output. This method should only be called from an exception handler, as it is here:
import logging
try:
*code goes here*
except BaseException:
logging.exception("*Error goes here*")
There's good documentation on this python page if you want to learn more about logging and debugging.

python: how to get line text where exception occur, not line number

I am trying to use sys.excepthook
With below hook
def foo(type, value, traceback):
# how to print the line that except occurs
sys.excepthook = foo
and use like below
$ python3
>>> text that cause error
How to define foo such that text that cause error being printed?
EDIT
Let me add the full story, to make it clear. (deserve downvotes? -) )
What I want is not print the line, is
get the line
if the line match some criterion, modify then exec
eg,
if type
>>> import requests
>>> edit requests
get the line that exception occurs, i.e, edit requests
then exec edit(find_file(request))
where edit() use subprocess.call(), find_file use inspect to find the file where an object being defined.
Yes, I know ipython magics, and use it regularlly. this time I am ask to how to define it.

You can use the modul traceback and the passed traceback object's tb_lineno attribute:
import sys
import traceback as tb
def foo(type, value, traceback):
# how to print the line that except occurs
print("The line where the exception occurs: {}".format(tb.linecache.getline(tb.extract_tb(sys.last_traceback)[0].filename, traceback.tb_lineno)))
sys.excepthook = foo
int("text")
Out:
The line where the exception occurs: int("text")
as the the line int("text") line raised an exeption.

We can use the traceback module to help us.
def foo(type, value, trace):
print(trace.msg)
The traceback object will have lots of information about where the error occurred.

Why doesn't sys.excepthook work?

Why isn't the sys.excepthook function called if I try to execute this code?
import sys;
def MyExcepthook(ex_cls, ex, tb):
print("Oops! There's an Error.\n");
a=open("./ERR.txt","w"); #Fixed as suggested by unutbu BUT the problem is the same!
a.write("Oops! There's an Error.\n");
a.close();
sys.excepthook = MyExcepthook;
def main():
print(1/0);
if (__name__=="__main__"):
main();
Output:
Traceback (most recent call last):
File "C:\Users\Path\to\my\python\file.py", line 13, in <module>
main();
File "C:\Users\Path\to\my\python\file.py", line 10, in main
print(1/0);
ZeroDivisionError: division by zero
Expected Output (by print):
Oops! There's an Error.
and a new file (Err.txt) should be created (by open)
The print function doesn't show the text and the file is not created because the sys.excepthook function is not called - why?
-->EDIT
My Problem is caused by a bug in idle-python 3.4 because now i tried to run the code by interpreter python (command line) and it works! this makes my Question useless if not to warn about this bug in idle-python 3.4 i'm sorry and thanks for your help!
[SOLUTION] if someone has my same problem => Try to Run your code by command line! and not from IDE.

Your custom excepthook must not itself raise an exception:
a=open("./ERR.txt") # opens the file in read mode
should be
a=open("./ERR.txt", 'w') # open the file in write mode.
When the custom excepthook raises an exception, you should see
something like
Oops! There's an Error.
Error in sys.excepthook:
...
IOError: [Errno 2] No such file or directory: './ERR.txt'
Original exception was:
...
ZeroDivisionError: integer division or modulo by zero
PS. Don't forget to delete all those unnecessary semicolons!
import sys
def my_excepthook(ex_cls, ex, tb):
msg = "Oops! There's an Error.\n"
print(msg)
with open("./ERR.txt", 'w') as a:
a.write(msg)
sys.excepthook = my_excepthook
def main():
print(1/0)
if __name__=="__main__":
main()

Printing an exception

I'm writing a little script which catches an error (or exception). But when the exception occurs I want to have all information like the Traceback, exception name and exception message. It should also act if the exception hadn't been caught but the following code shouldn't be affected (i.d the error should be appear but script doesn't stop working).
For example: in the following code a exception will be thrown. If that happens (and only if it happens) I want to make "cleanup".
try:
1 / 0
except Exception as e:
# Printing the Exception like it would have been done if the exception hadn't been caught:
# Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
# ZeroDivisionError: integer division or modulo by zero
# With the traceback, the exception name and the exception message.
# Doing some additional stuff.
pass
I'm not going to use a logger because the script is very smart (no longer than 100 lines) and it will only be used by me.
Edit: I'm using python 2.x

You'll want to use the traceback module:
import traceback
try:
raise Exception('what')
except Exception:
print(traceback.format_exc())

You can solve:
It should also act if the exception hadn't been caught.
with
try:
1 / 0
except Exception as e:
do_cleanup(e)
raise