In order to log "some dependency, somewhere deep" errors that trigger a server error 500, without stack traces in the console log on production instances due to DEBUG=False, I implemented the standard custom 500 handler that a fair number of Stackoverflow questions about printing a stack trace for a 500 error recommend:
import sys
import traceback
def server_error_500_handler(request):
type, value, tb = sys.exc_info()
print('\n----intercepted 500 error stack trace----')
print(value)
print(type)
print(traceback.format_exception(type, value, tb))
print('----\n')
However, these then also all say to end with render(request, '500.html'), whereas I don't want to serve a custom 500 page, but want the code to go "back" (if there is such a thing) to just serving whatever Django itself does already. Is there some way to make it do that? Or, is there some way to listen for the 500 event without hijacking the 500 error return codepath?
Instead of making a custom 500 handler, make a custom middleware of your own and implement a process_exception method in it:
import traceback
class Log500ErrorsMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
response = self.get_response(request)
return response
def process_exception(self, request, exception):
print('\n----intercepted 500 error stack trace----')
print(exception)
print(type(exception))
tb = exception.__traceback__
print(traceback.format_exception(type(exception), exception, tb))
print('----\n')
return None # Let other middlewares do further processing
Then add it to the MIDDLEWARE setting, all the way at the end, since the middlewares are run in bottom-up order during the response/exception phase, so if you put it last it will always get run (some middleware can decide to short circuit and return a response otherwise, so having anything after it might prevent it from running).
MIDDLEWARE = [
...
'path.to.Log500ErrorsMiddleware',
]
I have a simple code using flask:
#app.route('/foo/<arg>')
#app.cache.memoize()
def foo_response(arg):
return 'Hello ' + arg
This is working great while my redis server (cache server) is up.
If the redis server goes down, an exception is raised every time I query /foo/<arg>, which is understandable.
How (and where) can I handle that exception (à la try-except) in order to not use the redis server if it is down at that moment?
It is actually implemented this way. By checking the source of memoize() in Flask-Cache package you see
try:
cache_key = decorated_function.make_cache_key(f, *args, **kwargs)
rv = self.cache.get(cache_key)
except Exception:
if current_app.debug:
raise
logger.exception("Exception possibly due to cache backend.")
return f(*args, **kwargs)
This means if you are on production i.e. app.debug=False you will see the exception log and the function will be called normally.
I'm trying to setup email error logging and I want to test it.
Whats an easy way to trigger a 500 error in Django? This surprisingly has not been discussed here yet.
A test view like this will work:
from django.http import HttpResponse
def my_test_500_view(request):
# Return an "Internal Server Error" 500 response code.
return HttpResponse(status=500)
or use the baked in error class:
from django.http import HttpResponseServerError
def my_test_500_view(request):
# Return an "Internal Server Error" 500 response code.
return HttpResponseServerError()
Raising an appropriate Exception is the most robust way to test this. Returning an HttpResponse of any variety, as in #agconti 's answer, will not allow you to test the behavior of error handling. You'll just see a blank page. Raising an exception triggers both the correct response code and the internal Django behavior you expect with a "real" error.
Response code 500 represents the server not knowing what to do. The easiest way is to write into your test or test-view raise Exception('Make response code 500!').
Most other errors should be raised with exceptions designed to trigger specific response codes. Here's the complete list of Django exceptions that you might raise.
Old question, but I hope this helps someone in the future. The accepted answer doesn't really answer the original question...
You can do this with django rest framework easily by raising ApiException:
from rest_framework.exceptions import APIException
try:
...
except ...:
raise APIException('custom exception message')
This will return a response with status 500 from your view. Pretty useful if you are calling another function from your API and you don't want to manage return values from this function to decide if returning a response with status 500.
You can use it like this:
raise ApiException
And the default response message (at the time of writing this) will be 'A server error occurred.'.
There's a few predefined ApiException subclasses to raise different kinds of errors, check the file rest_framework.exceptions, if you need one that is not in there, you can extend ApiException like this:
from rest_framework.exceptions import APIException
class YourCustomApiExceptionName(APIException):
def __init__(self, status, detail):
self.status_code = status
self.detail = detail
Usage:
raise YourCustomApiExceptionName(100, 'continue')
You can also define custom status and detail but wouldn't make much sense in most cases.
I'm writing a bottle application with beaker session middleware.
My code is has this:
#bottle.route('/')
def slash():
try:
beaker_session = request.environ['beaker.session']
except:
#redirect('/login')
abort(401, "Failed beaker_session in slash")
try:
name = beaker_session['name']
except:
redirect('/login')
for each route request except /login. I know there is a bottle hook system to do things before requests, but I'm not sure how best to use it to check if someone is logged in or not.
I'm fairly new to python webapps using bottle. Not to many people are using it with beaker session middleware so I don't have a lot of examples to go by.
Thanks for any help or pointers!
PS. The entire code for this is in this repo: https://github.com/curtisgithub/labinski/blob/master/labinski.py
I know there is a bottle hook system to do things before requests, but I'm not sure how best to use it to check if someone is logged in or not.
You can use the before_request hook to run code prior to each request, but checking authentication here only makes sense if you expect all access to be authenticated. You could do something like this:
#bottle.hook('before_request')
def setup_request():
try:
beaker_session = request.environ['beaker.session']
except:
#redirect('/login')
abort(401, "Failed beaker_session in slash")
try:
name = beaker_session['name']
except:
redirect('/login')
...but without some extra code this is going to result in a redirect loop when someone actually requests /login. So you can add this to the hook, maybe:
if request.urlparts.path == '/login':
continue
Another solution is to implement similar functionality using a Python decorator, which lets you control access on a method-by-method basis. For example, you could say something like:
#route('/')
#authenticated
def index():
return 'This is /.'
#route('/login')
def login():
return 'This is login.'
And then your authenticated decorator would look very much like the hook:
def authenticated(func):
def wrapped(*args, **kwargs):
try:
beaker_session = request.environ['beaker.session']
except:
abort(401, "Failed beaker_session in slash")
try:
name = beaker_session['name']
return func(*args, **kwargs)
except:
redirect('/login')
return wrapped
I'm struggling to produce the same behavior in web service code that uses Deferred objects as in code that does not. My objective is to write a decorator that will delegate processing of any method (which is decoupled from Twisted) to the Twisted thread pool, so that the reactor is not blocked, without changing any of that method's semantics.
When an instance of class echo below is exposed as a web service, this code:
from twisted.web import server, resource
from twisted.internet import defer, threads
from cgi import escape
from itertools import count
class echo(resource.Resource):
isLeaf = True
def errback(self, failure): return failure
def callback1(self, request, value):
#raise ValueError # E1
lines = ['<html><body>\n',
'<p>Page view #%s in this session</p>\n' % (value,),
'</body></html>\n']
return ''.join(lines)
def callback2(self, request, encoding):
def execute(message):
#raise ValueError # E2
request.write(message.encode(encoding))
#raise ValueError # E3
request.finish()
#raise ValueError # E4
return server.NOT_DONE_YET
return execute
def render_GET(self, request):
content_type, encoding = 'text/html', 'UTF-8'
request.setHeader('Content-Type', '%s; charset=%s' %
tuple(map(str, (content_type, encoding))))
s = request.getSession()
if not hasattr(s, 'counter'):
s.counter = count(1)
d = threads.deferToThread(self.callback1, request, s.counter.next())
d.addCallback(self.callback2(request, encoding))
d.addErrback(self.errback)
#raise ValueError # E5
return server.NOT_DONE_YET
will display an HTML document to the browser when all the raise statements are commented out, and display a nicely formatted stack trace (which Twisted does for me) when the raise statement labelled "E5" is included. That is what I want. Likewise, if I do not use Deferred objects at all and place all the behavior from callback1 and callback2 within render_GET(), an exception raised anywhere within render_GET will produce the desired stack trace.
I am trying to write code that will respond to the browser immediately, not cause resource leaks within Twisted, and cause the browser stack trace to also be displayed in the cases where any of the raise statements "E1" to "E3" is included in the deferred code--though of course I understand that the stack traces themselves will be different. (The "E4" case I don't care about as much.) After reading the Twisted documentation and other questions on this site I am unsure how to achieve this. I would have thought that adding an errback should facilitate this, but evidently not. There must be something about Deferred objects and the twisted.web stack that I'm not understanding.
The effects on logging I document here may be affected by my use of the PythonLoggingObserver to bridge Twisted logging to the standard logging module.
When "E1" is included, the browser waits until the reactor is shut down, at which point the ValueError exception with stack trace is logged and the browser receives an empty document.
When "E2" is included, the ValueError exception with stack trace is logged immediately, but the browser waits until the reactor shuts down at which point it receives an empty document.
When "E3" is included, the ValueError exception with stack trace is logged immediately, the browser waits until the reactor shuts down, and at that point receives the intended document.
When raise statement "E4" is included, the intended document is returned to the browser immediately, and the ValueError exception with stack trace is logged immediately. (Is there any possibility of a resource leak in this case?)
Ok, after reading your question several times, I think I understand what your asking. I have also reworked you code to make a little better than your original answer. This new answer should show off all the powers of deferred's.
from twisted.web import server, resource
from twisted.internet import defer, threads
from itertools import count
class echo(resource.Resource):
isLeaf = True
def errback(self, failure, request):
failure.printTraceback() # This will print the trace back in a way that looks like a python exception.
# log.err(failure) # This will use the twisted logger. This is the best method, but
# you need to import twisted log.
request.processingFailed(failure) # This will send a trace to the browser and close the request.
return None # We have dealt with the failure. Clean it out now.
def final(self, message, request, encoding):
# Message will contain the message returned by callback1
request.write(message.encode(encoding)) # This will write the message and return it to the browser.
request.finish() # Done
def callback1(self, value):
#raise ValueError # E1
lines = ['<html><body>\n',
'<p>Page view #%s in this session</p>\n' % (value,),
'</body></html>\n']
return ''.join(lines)
#raise ValueError # E4
def render_GET(self, request):
content_type, encoding = 'text/html', 'UTF-8'
request.setHeader('Content-Type', '%s; charset=%s' %
tuple(map(str, (content_type, encoding))))
s = request.getSession()
if not hasattr(s, 'counter'):
s.counter = count(1)
d = threads.deferToThread(self.callback1, s.counter.next())
d.addCallback(self.final, request, encoding)
d.addErrback(self.errback, request) # We put this here in case the encoding raised an exception.
#raise ValueError # E5
return server.NOT_DONE_YET
Also I recommend that you read the krondo tutorial. It will teach you everything you need to know about deferred.
Edit:
Have modified the code above to fix some silly bugs. Also improved it. If an exception happens anywhere (except in self.errback, but we need some level of trust) then it will be passed to self.errback which will log or print the error in twisted and then send the trace to the browser and close the request. This should stop any resource leaks.
I figured it out by digging through the Twisted source. The necessary insight is that the reactor and Deferred callback/errback chain logic is decoupled from the request object, which is how data gets back to the browser. The errback is necessary, but cannot merely propagate the Failure object down the chain as in the original code I posted. The errback must report the error to the browser.
The below code meets my requirements (never keeps the browser waiting, always gives the stack trace, does not require a reactor restart to get things going again) and will allow me to decorate blocking methods and thereby delegate them to threads to keep the reactor responsive to other events (such methods will essentially take the place of callback1 here). However, I did find that in the below code, uncommenting the "E4" raise statement produces very strange behavior on subsequent browser requests (partial data from previous requests returned to the browser; deadlock).
Hopefully others will find this to be a useful Deferred example.
from twisted.web import server, resource
from twisted.internet import defer, threads
from itertools import count
class echo(resource.Resource):
isLeaf = True
def errback(self, request):
def execute(failure):
request.processingFailed(failure)
return failure
return execute
def callback1(self, value):
#raise ValueError # E1
lines = ['<html><body>\n',
'<p>Page view #%s in this session</p>\n' % (value,),
'</body></html>\n']
return ''.join(lines)
def callback2(self, request, encoding):
def execute(message):
#raise ValueError # E2
request.write(message.encode(encoding))
#raise ValueError # E3
request.finish()
#raise ValueError # E4
return server.NOT_DONE_YET
return execute
def render_GET(self, request):
content_type, encoding = 'text/html', 'UTF-8'
request.setHeader('Content-Type', '%s; charset=%s' %
tuple(map(str, (content_type, encoding))))
s = request.getSession()
if not hasattr(s, 'counter'):
s.counter = count(1)
d = threads.deferToThread(self.callback1, s.counter.next())
eback = self.errback(request)
d.addErrback(eback)
d.addCallback(self.callback2(request, encoding))
d.addErrback(eback)
#raise ValueError # E5
return server.NOT_DONE_YET