In my core application I have a custom Exception 'timeout' which when caught should return a message and trigger 'sys.exit()' to terminate the application.
timeout Exception:
class timeout(Exception):
def __init__(self, msg):
super().__init__(msg)
Core Application:
try:
s3Client.put_object(Bucket = bucket, Key = key, Body = body)
except timeout as error:
print(error)
sys.exit()
In the above example I am using boto3 AWS module, but this could be substituted with any other. During the execution of this boto3 function, the timeout error will be raised by the core application.
What I would expect, is for the timeout error to be raised (triggered by an Alarm from the Signal module FYI) in the Core Application, '' to be printed, and 'sys.exit()' to be triggered, terminating the application.
Instead, the Core Application raises the custom 'timeout()' exception, but the boto3 function is not configured to handle it and returns HTTPClientError('An HTTP Client raised an unhandled exception: Integration Timeout'). My Core Application in turn is not configured to handle 'HTTPClientError()', so the timeout logic does not execute.
I want a way for my 'timeout()' Exception to be handled in the Core Application, even if the module throws a different exception.
Considered modifying the module to add my custom Exception, seems hacky.
Thanks
A bit hacky, but if you don't want to modify the module this could be an option:
timeout Exception:
class timeout(Exception):
exception_identifier = "timeout"
def __init__(self, msg):
super().__init__(msg + f" exception_identifier={self.exception_identifier}")
Core Application:
import timeout
try:
s3Client.put_object(Bucket = bucket, Key = key, Body = body)
except timeout as error:
print(error)
sys.exit()
except HTTPClientError as error:
if f"exception_identifier={timeout.exception_identifier}" in error.msg:
print(error)
sys.exit()
else:
raise
Maybe there is a better way to extract if the original Exception was a timeout than comparing the strings. You can also use an exception_identifier like a long ID-string, that is unlikely to clash with other Exceptions that may use "timeout" in the message.
Related
Having read through http://krondo.com/an-introduction-to-asynchronous-programming-and-twisted/, I have a basic understanding of Twisted. I have built a testing infrastructure that a few other people also use. From time to time, we experience 'Unhandled error in Deferred' type errors if there are bugs or typos in the code. The problem is that there isn't always an accompanying stack trace with these errors making it hard for people to debug.
I have reproduced the problem with the following simple code:
from twisted.internet import defer, reactor
from twisted.internet.task import deferLater
def sleep(delay):
"""Twisted safe sleep.
When using Twisted, it is not safe to call time.sleep(). So
we have this function to emulate the behavior.
"""
return deferLater(reactor, delay, lambda: None)
#defer.inlineCallbacks
def run_test():
print 'run_test: start'
bug()
yield sleep(1)
print 'run_test: stop'
#defer.inlineCallbacks
def run_tests():
def err(arg):
print 'err', arg
return arg
def success(arg):
print 'success', arg
return arg
d = run_test()
#d.addCallbacks(success, err)
try:
yield d
finally:
reactor.stop()
reactor.callWhenRunning(run_tests)
reactor.run()
When I run this code, I see the following output:
run_test: start
Unhandled error in Deferred:
And if I uncomment the addCallbacks() line above, then I get some stack trace information:
run_test: start
err [Failure instance: Traceback: <type 'exceptions.NameError'>: global name 'bug' is not defined
/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py:1406:unwindGenerator
/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py:1260:_inlineCallbacks
tmp.py:34:run_tests
/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py:1406:unwindGenerator
--- <exception caught here> ---
/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py:1260:_inlineCallbacks
tmp.py:18:run_test
]
Unhandled error in Deferred:
My question is, is there some way to get the stack trace without having to add callbacks at all defer sites?
The Deferred class has tons of try/except magic, to the point that the errors are pretty well abstracted from the end user and only easily attained via addErrback. If errors are occurring, it's a sign you have broken functionality. In synchronous apps, you might encapsulate the "problem section" in a try/except block. In the case of inlineCallbacks, the same technique can be utilized:
try:
run_test()
except Exception as err:
# handle error here
finally:
reactor.stop()
Since the bug occurs in the run_test() function, catch the exception when running that function, then handle the error according to your requirements.
However, if you don't plan to "handle" errors, but rather you want a record that an error has occurred, then you should consider using Twisted logger functionality. This will catch your unhandled tracebacks and log them somewhere.
I have a typical Pyramid+SQLAlchemy+Postgres app. In stress testing or during moments of exceptional load and with low max_connections setting in PG it might happen that OperationalException is raised:
OperationalError: (psycopg2.OperationalError) FATAL: sorry, too many clients already
Now, obviously I do not want to do this everywhere:
try:
DBSession.query(Item)...
except OperationalError as e:
log.error(...)
Is there some way of catching this exception "globally" to be properly handled?
My app uses ZopeTransactionExtension in typical Pyramid manner:
DBSession = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))
I managed to develop a tween that can do this (example):
def catch_pg_exc_tween_factory(handler, registry):
def catch_pg_exc_tween_clos(request):
response = None
try:
response = handler(request)
except Exception as e:
log.error('\n\n\n +++ problem: %s', e)
return response
return catch_pg_exc_tween_clos
The strange thing is that nothing but explicit tween ordering in development.ini works (no amount of over= or under= tuning of config.add_tween call seems to have worked):
pyramid.tweens = pyramid_debugtoolbar.toolbar_tween_factory
pyramid.tweens.excview_tween_factory
pyramid_tm.tm_tween_factory
mypkg.util.tweens.catch_pg_exc_tween_factory
In my Python app, I have been working with Raven+Sentry to catch exception and sent it to sentry at the time of the exception occur, code below:
if __name__ == '__main__':
from raven import Client
client = Client(dsn='<MY SENTRY DSN>')
try:
MyApp().run()
except:
import traceback
traceback.print_exc()
ident = client.get_ident(client.captureException())
print "Exception caught; reference is %s" % ident
It will sent the exception directly to Senty backend just after the application crashed. What I want to accomplished is, saving the exception first to local file and then send it later on when the app starting.
Do Sentry+Raven support this kind of functionality?
I writing little customized ftp server and I need to suppress printing exceptions (well, one specific type of exception) to console but I want server to send "550 Requested action not taken: internal server error" or something like that to client.
However, when I catch exception using addErrback(), than I don't see exception in console but client gets OK status..
What could I do?
When you catch an error in errback handler, you should then inspect the type of the Failure and based on internal logic of your application send the Error as an FTP error message to the client
twisted.protocol.ftp.FTP handles this with self.reply(ERROR_CODE, "description")
So your code could look something like this:
from twisted.internet import ftp
MY_ERROR = ftp.REQ_ACTN_NOT_TAKEN
def failureCheck(failureInstance):
#do some magic to establish if we should reply an Error to this failure
return True
class myFTP(ftp.FTP):
def myActionX(self):
magicResult = self.doDeferredMagic()
magicResult.addCallback(self.onMagicSuccess)
magicResult.addErrback(self.onFailedMagic)
def onFailedMagic(self,failureInstance):
if failureCheck(failureInstance):
self.reply(MY_ERROR,'Add relevant failure information here')
else:
#do whatever other logic here
pass
I've recently started developing my first web app with GAE and Python, and it is a lot of fun.
One problem I've been having is exceptions being raised when I don't expect them (since I'm new to web apps). I want to:
Prevent users from ever seeing exceptions
Properly handle exceptions so they don't break my app
Should I put a try/except block around every call to put and get?
What other operations could fail that I should wrap with try/except?
You can create a method called handle_exception on your request handlers to deal with un-expected situations.
The webapp framework will call this automatically when it hits an issue
class YourHandler(webapp.RequestHandler):
def handle_exception(self, exception, mode):
# run the default exception handling
webapp.RequestHandler.handle_exception(self,exception, mode)
# note the error in the log
logging.error("Something bad happend: %s" % str(exception))
# tell your users a friendly message
self.response.out.write("Sorry lovely users, something went wrong")
You can wrap your views in a method that will catch all exceptions, log them and return a handsome 500 error page.
def prevent_error_display(fn):
"""Returns either the original request or 500 error page"""
def wrap(self, *args, **kwargs):
try:
return fn(self, *args, **kwargs)
except Exception, e:
# ... log ...
self.response.set_status(500)
self.response.out.write('Something bad happened back here!')
wrap.__doc__ = fn.__doc__
return wrap
# A sample request handler
class PageHandler(webapp.RequestHandler):
#prevent_error_display
def get(self):
# process your page request