I would like to disable the stack trace that is printed when there is an exception raised.
Whenever the code calls the logger.exception method, the stack trace is automatically printed.
That's because the default value of the exc_info parameter of the .exception method is True.
See the source code:
def exception(msg, *args, exc_info=True, **kwargs):
"""
Log a message with severity 'ERROR' on the root logger, with exception
information. If the logger has no handlers, basicConfig() is called to add
a console handler with a pre-defined format.
"""
error(msg, *args, exc_info=exc_info, **kwargs)
To prevent this, you can send exc_info=False to the .exception method like this:
try:
raise Exception("Huston we have a problem!")
except Exception as ex:
logger.exception(f"Looks like they have a problem: {ex}", exc_info=False)
While this seems like working, it is bad to force users to write exc_info=False each time they use that method. Therefore, in order to take this burden from the shoulders of the programmers, you can monkey patch the .exception method and make it act like a regular .error method like this:
# somewhere in the start of your program
# money patch the .exception method
logger.exception = logger.error
try:
raise Exception("Huston we have a problem!")
except Exception as ex:
logger.exception(f"Looks like they have a problem: {ex}")
Looking around I found the following solution / workaround:
sys.tracebacklimit = 0
Related
What is the best practice in Python for reporting an error which occurs when closing a resource?
In particular, if I implement __enter__ and __exit__, I can use
with my_library.connect(endpoint) as connection:
connection.store_data("...")
But what should I do if closing my connection and persisting the changes fails, e.g. due to a network outage? I know that I could technically raise an error from within __exit__ but is this best practice/idiomatic in Python? Or should I, e.g., provide a separate persistChanges method, let the __exit__ swallow all errors, and then write in the documentation "if you don't call persistChanges you might lose your changes in error cases"?
My specific use case is: I am providing a Python API to other developers, and I wonder how to handle this case of "error on closing the resource" such that my API follows Python best practices/meets the expectations of Python devs using my library.
I would recommend making a custom error/warning class for your library. this can be very, very simple. There already exists a set of built-in exceptions that you can extend from. Based on your description above, I would recommend extending the RuntimeError like this:
class MyLibraryConnectionError(RuntimeError):
pass
or, if you want to only throw a warning, using the ResourceWarning like this:
class MyLibraryConnectionWarning(ResourceWarning):
pass
There is also the RuntimeWarning that could be extended for similar effect.
If you feel that ResourceWarning, RuntimeWarning, and RuntimeError don't accurately describe the exception, you can also just have them inherit directly from Exception or Warning, depending on whether you want them to only be flagged in Developer mode(Warning), or if you want the full exception functionality.
You can throw these like any other exception:
throw MyLibraryConnectionError("The underlying resource failed to close")
throw MyLibraryConnectionWarning("The underlying resource failed to close")
or even catch your dependency's thrown exception:
def __exit__(...):
try:
# potentially dangerous code with connections
underlyingConnection.close()
except TheErrorYouSeeThrown as e: # you can probably make this Exception instead of TheErrorYouSeeThrown. The more specific the better, so you don't accidentally catch random errors you didn't mean to.
throw MyLibraryConnectionError(e.message) # or whatever this needs to be, depending on the thrown exception type
Then your users could implement like so:
try:
with my_library.connect(endpoint) as connection:
connection.store_data("...")
except my_library.MyLibraryConnectionError as e:
# handle appropriately, or ignore
except Exception as e:
# Handle other errors that happen, that your library doesn't cause.
I wrote a cronjob that iterates through a list of accounts and performs some web call for them (shown below):
for account in self.ActiveAccountFactory():
try:
self.logger.debug('Updating %s', account.login)
self.update_account_from_fb(account)
self.check_balances()
self.check_rois()
except Exception,e:
self.logger.exception(traceback.format_exc())
Because this job is run by heroku one every 10 minutes, I do not want the entire job to fail just because one account is running into issues (it happens). I placed a try catch clause here so that this task is "fault-tolerant".
However, I noticed that when I am testing, this try/catch block is giving me cryptic problems because of the task is allowed to continue executing even though there is some serious error.
What is the best way to disable a try/except block during testing?
I've though about implementing the code directly like this:
for account in self.ActiveAccountFactory():
self.logger.debug('Updating %s', account.login)
self.update_account_from_fb(account)
self.check_balances()
self.check_rois()
self.logger.exception(traceback.format_exc())
in my test cases but then this makes my tests very clumsy as I am copying large amounts of code over.
What should I do?
First of all: don't swallow all exceptions using except Exception. It's bad design. So cut it out.
With that out of the way:
One thing you could do is setup a monkeypatch for the logger.exception method. Then you can handle the test however you see fit based on whether it was called, whether it's creating a mock logger, or a separate testing logger, or a custom testing logger class that stops the tests when certain exceptions occur. You could even choose to end the testing immediately by raising an error.
Here is an example using pytest.monkeypatch. I like pytest's way of doing this because they already have a predefined fixture setup for it, and no boilerplate code is required. However, there are others ways to do this as well (such as using unittest.mock.patch as part of the unitest module).
I will call your class SomeClass. What we will do is create a patched version of your SomeClass object as a fixture. The patched version will not log to the logger; instead, it will have a mock logger. Anything that happens to the logger will be recorded in the mock logger for inspection later.
import pytest
import unittest.mock as mock # import mock for Python 2
#pytest.fixture
def SomeClassObj_with_patched_logger(monkeypatch):
##### SETUP PHASE ####
# create a basic mock logger:
mock_logger = mock.Mock(spec=LoggerClass)
# patch the 'logger' attribute so that when it is called on
# 'some_class_instance' (which is bound to 'self' in the method)
# things are re-routed to mock_logger
monkeypatch.setattr('some_class_instance.logger', mock_logger)
# now create class instance you will test with the same name
# as the patched object
some_class_instance = SomeClass()
# the class object you created will now be patched
# we can now send that patched object to any test we want
# using the standard pytest fixture way of doing things
yield some_class_instance
###### TEARDOWN PHASE #######
# after all tests have been run, we can inspect what happened to
# the mock logger like so:
print('\n#### ', mock_logger.method_calls)
If call.exception appears in the method calls of the mock logger, you know that method was called. There are a lot of other ways you could handle this as well, this is just one.
If you're using the logging module, LoggerClass should just be logging.Logger. Alternatively, you can just do mock_logger = mock.Mock(). Or, you could create your own custom testing logger class that raises an exception when its exception method is called. The sky is the limit!
Use your patched object in any test like so:
def test_something(SomeClassObj_with_patched_logger):
# no need to do the line below really, just getting
# a shorter variable name
my_obj = SomeClassObj_with_patched_logger
#### DO STUFF WITH my_obj #####
If you are not familiar with pytest, see this training video for a little bit more in depth information.
try...except blocks are difficult when you are testing because they catch and try to dispose of errors you would really rather see. As you have found out. While testing, for
except Exception as e:
(don't use Exception,e, it's not forward-compatible) substitute an exception type that is really unlikely to occur in your circumstances, such as
except AssertionError as e:
A text editor will do this for you (and reverse it afterwards) at the cost of a couple of mouse-clicks.
You can make callables test-aware by add a _testing=False parameter. Use that to code alternate pathways in the callable for when testing. Then pass _testing=True when calling from a test file.
For the situation presented in this question, putting if _testing: raise in the exception body would 'uncatch' the exception.
Conditioning module level code is tricker. To get special behavior when testing module mod in package pack, I put
_testing = False # in `pack.__init__`
from pack import _testing # in pack.mod
Then test_mod I put something like:
import pack
pack._testing = True
from pack import mod
The Python 2.7 unittest docs say:
All the assert methods (except assertRaises(), assertRaisesRegexp()) accept a msg argument that, if specified, is used as the error message on failure
… but what if I want to specify the error message for assertRaises() or assertRaisesRegexp()?
Use case: when testing various values in a loop, if one fails I’d like to know which one:
NON_INTEGERS = [0.21, 1.5, 23.462, math.pi]
class FactorizerTestCase(unittest.TestCase):
def test_exception_raised_for_non_integers(self):
for value in NON_INTEGERS:
with self.assertRaises(ValueError):
factorize(value)
If any of these fails, I get:
AssertionError: ValueError not raised
which isn’t too helpful for me to work out which one failed… if only I could supply a msg= argument like I can with assertEqual() etc!
(I could of course break these out into separate test functions — but maybe there are loads of values I want to test, or it requires some slow/expensive setup, or it’s part of a longer functional test)
I’d love it if I could easily get it to report something like:
AssertionError: ValueError not raised for input 23.462
— but it’s also not a critical enough thing to warrant reimplementing/extending assertRaises() and adding a load more code to my tests.
You could also fallback to using self.fail which feels annoying, but looks a bit less hacky I think
for value in NON_INTEGERS:
with self.assertRaises(ValueError) as cm:
factorize(value)
self.fail('ValueError not raised for {}'.format(value))
1. Easiest (but hacky!) way to do this I’ve found is:
for value in NON_INTEGERS:
with self.assertRaises(ValueError) as cm:
cm.expected.__name__ = 'ValueError for {}'.format(value) # custom failure msg
factorize(value)
which will report this on failure:
AssertionError: ValueError for 23.462 not raised
Note this only works when using the with … syntax.
It works because the assertRaises() context manager does this internally:
exc_name = self.expected.__name__
…
raise self.failureException(
"{0} not raised".format(exc_name))
so could be flaky if the implementation changes, although the Py3 source is similar enough that it should work there too (but can’t say I’ve tried it).
2. Simplest way without relying on implementation is to catch the error and re-raise it with an improved message:
for value in NON_INTEGERS:
try:
with self.assertRaises(ValueError) as cm:
factorize(value)
except AssertionError as e:
raise self.failureException('{} for {}'.format(e.message, value)), sys.exc_info()[2]
The sys.exc_info()[2] bit is to reuse the original stacktrace, but this syntax is Py2 only. This answer explains how to do this for Py3 (and inspired this solution).
But this is already making the test hard to read, so I prefer the first option.
The ‘proper’ solution would require writing a wrapped version of both assertRaises AND the _AssertRaisesContext class, which sounds like overkill when you could just throw in some logging when you get a failure.
I use this instead of assertRaises:
def test_empty_username(self):
# noinspection PyBroadException
try:
my_func(username="")
except Exception:
# If it does, we are OK.
return
# If not, we are here.
self.fail("my_func() must reject empty username.")
In Python 3, unittest now exposes the error message as a proper, publicly accessible property of the _AssertRaisesContext instance you get from with self.assertRaises(). So in Python 3, you can do this:
with self.assertRaises(ValueError) as assertion:
assertion.msg = f"ValueError not raised for input {value}"
factorize(value)
I don't have control over a python library that needlessly raises a certain exception.
Is there a way to handle it with a "placebo" Exception subclass? i.e.
class MyException(Exception):
def __init__(self, value):
# literally do nothing. return to program flow
pass
raises a certain exception
You have a certain exception, you should handle the exception by specifying the exception class in the except clause of a try-except block.
The placebo approach (i.e. except Exception...) is only going to silence other exceptions since exceptions typically derive from Exception or isn't going to work at all (with your new exception class) since the raised exception is apparently not derived from the new exception class.
Bear in mind:
Errors should never pass silently unless explicitly silenced
So I don't see why you would want to undo a raised exception without a try-except. The exception was not raised for nothing.
If you're hoping to avoid the use of try-except every time the function from this library is called, then you could write a wrapper function that wraps the call with a try-except, and then use the new function henceforth.
from somelib import func
from somelib import SomeException
def wrapper_func(*args, **kwargs):
try:
func(*args, **kwargs)
except SomeException:
pass
While the other answer is correct, if you're dealing with some strange/buggy module you can give fuckitpy a try!
Note that it is generally a very bad idea to let the exceptions silently pass through.
Anyhow, the basic way to use fuckitpy is (quoting from the docs):
import fuckit
#import some_shitty_module
fuckit('some_shitty_module')
some_shitty_module.some_function()
Also from the docs:
Still getting errors? Chain fuckit calls. This module is like violence: if it doesn't work, you just need more of it.
import fuckit
fuckit(fuckit('some_shitty_module'))
# This is definitely going to run now.
some_shitty_module.some_function()
fuckitpy (GitHub): https://github.com/ajalt/fuckitpy
I am trying to assertRaise the exception inside a function where a condition raises a custom exception message .
Function:
if not Cart.objects.filter(member=member).count():
raise CartDoesNotExist("Cart Does Not Exist for Member: %s ( %id )." % (member.email,member.id))
Now , i am able to successfully produce the required condition to get to the raise statement.
So , my testcase looks like this :
def Order_CartDoesNotExist(self):
self.assertRaises(CartDoesNotExist,Order.objects.create_order(member=self.member2,member_slot=self.memslot,order_type="Normal"))
When i run the test , the output is an Error . It gives the same error CartDoesNotExist.....
So my question is , how to raise these kind of exceptions ? How to cover these situations in our unittest? I do not want to escape these conditions as they are important and increase code coverage?
Thank you all.
Your code is calling create_order directly, which raises the exception. You need to change how it is called. In Python 2.7, you can use this:
with self.assertRaises(CartDoesNotExist):
Order.objects.create_order(member=self.member2, member_slot=self.memslot, order_type="Normal"))
Here the context manager allows you to call your code directly, and the context manager will handle the exception for you.
If you are running with 2.6 or below:
self.assertRaises(CartDoesNotExist, Order.objects.create_order, member=self.member2, member_slot=self.memslot, order_type="Normal")
Here you aren't calling your function, you are passing it to assertRaises, along with the arguments it needs, and assertRaises will call the code and deal with the exception properly.