We Keep Coding

Hook for processing (not catching) exception in Flask

I have a Flask server.
Whenever the code inside my handler throws an exception, Flask catches it, and returns an HTML page to the client with a 5XX error.
The problem is that I don't notice this. I just got an email from someone using my API saying that they were getting 504 errors, and I didn't know about it until they told me.
In other non-Flask parts of my application I wrote a custom decorator to catch all exceptions, send an email to me, then re-throw. I would like something similar for my Flask app.
I want to find a way to have Flask call a function of mine every time my handler code throws an exception, before it returns a response to the client. I do not wish to modify the response that gets sent to the client. I don't want to change how Flask handles errors, or how it catches them. I just want some way of being notified, and then Flask can continue doing the default error handling behavior.
I suppose I could put a decorator over every single route handler to catch and rethrow exceptions before Flask sees them, but that's messy. I just know I'll forget one of them, especially when I add new ones in the future.
MWE
A buggy application:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
assert False, "buggy code here"
return "hello"
def error_handler(exc_type, exc_val, exc_tb):
send_email(exc_type, exc_val, exc_tb)
# This is the part I don't know
# I want something along the lines of:
app.config['ERROR_HOOK'] = error_handler

from flask import Flask
app = Flask(__name__)
app.debug = False
app.config['PROPAGATE_EXCEPTIONS'] = True
#app.errorhandler(Exception)
def all_exception_handler(error):
print(str(error))
#app.errorhandler(404)
def page_not_found(error):
return 'This page does not exist', 404
you can define a function for each specific error you want to catch #app.my_custom_errorhandler(code_or_exception)
The argument to your error handler function will be an Exception.

Request timed out: timeout('timed out') in Python's HTTPServer

I am trying to create a simple HTTP server that uses the Python HTTPServer which inherits BaseHTTPServer. [https://github.com/python/cpython/blob/main/Lib/http/server.py][1]
There are numerous examples of this approach online and I don't believe I am doing anything unusual.
I am simply importing the class via:
"from http.server import HTTPServer, BaseHTTPRequestHandler"
in my code.
My code overrides the do_GET() method to parse the path variable to determine what page to show.
However, if I start this server and connect to it locally (ex: http://127.0.0.1:50000) the first page loads fine. If I navigate to another page (via my first page links) that too works fine, however, on occasion (and this is somewhat sporadic), there is a delay and the server log shows a Request timed out: timeout('timed out') error. I have tracked this down to the handle_one_request method in the BaseHTTPServer class:
def handle_one_request(self):
"""Handle a single HTTP request.
You normally don't need to override this method; see the class
__doc__ string for information on how to handle specific HTTP
commands such as GET and POST.
"""
try:
self.raw_requestline = self.rfile.readline(65537)
if len(self.raw_requestline) > 65536:
self.requestline = ''
self.request_version = ''
self.command = ''
self.send_error(HTTPStatus.REQUEST_URI_TOO_LONG)
return
if not self.raw_requestline:
self.close_connection = True
return
if not self.parse_request():
# An error code has been sent, just exit
return
mname = 'do_' + self.command ## the name of the method is created
if not hasattr(self, mname): ## checking that we have that method defined
self.send_error(
HTTPStatus.NOT_IMPLEMENTED,
"Unsupported method (%r)" % self.command)
return
method = getattr(self, mname) ## getting that method
method() ## finally calling it
self.wfile.flush() #actually send the response if not already done.
except socket.timeout as e:
# a read or a write timed out. Discard this connection
self.log_error("Request timed out: %r", e)
self.close_connection = True
return
You can see where the exception is thrown in the "except socket.timeout as e:" clause.
I have tried overriding this method by including it in my code but it is not clear what is causing the error so I run into dead ends. I've tried creating very basic HTML pages to see if there was something in the page itself, but even "blank" pages cause the same sporadic issue.
What's odd is that sometimes a page loads instantly, and almost randomly, it will then timeout. Sometimes the same page, sometimes a different page.
I've played with the http.timeout setting, but it makes no difference. I suspect it's some underlying socket issue, but am unable to diagnose it further.
This is on a Mac running Big Sur 11.3.1, with Python version 3.9.4.
Any ideas on what might be causing this timeout, and in particular any suggestions on a resolution. Any pointers would be appreciated.

After further investigation, this particular appears to be an issue with Safari. Running the exact same code and using Firefox does not show the same issue.

Flask File Size Limit - 413 Error Closes Connection

I am setting a file limit in Flask. When files are uploaded it correctly blocks files over 4mb and returns a 413 error. However this is closing the connection on the server which means that the redirect doesn't work. What I want to do is keep the connection open on this error so I can then redirect the user.
My file size limit is set as:
app.config['MAX_CONTENT_LENGTH'] = 4 * 1024 * 1024
The code which captures the exception is:
try:
form = FileUploadForm();
except Exception as e:
print(e)
flash(u"File size too large: please choose a file under 4mb","danger")
return redirect(url_for("home"))
This is the exception returned:
413 Request Entity Too Large: The data value transmitted exceeds the capacity limit.
The code "return redirect(url_for("home"))" doesn't work, I believe because the connection is closed on error 413. This is Flask running in development mode.
I have tried the code in the suggested answer, but that doesn't seem to work. My code now looks like:
#app.errorhandler(413)
def largefile_error(e):
print("Large file")
return redirect(url_for("addsign")), 413
#app.route('/addsign', methods=['GET', 'POST'])
def addsign():
if not current_user.is_authenticated:
flash(f"Please login to upload a sign","danger")
return redirect(url_for('home'))
form = FileUploadForm()
In the console I can see the following output.
Large file
127.0.0.1 - - [13/Jul/2020 18:02:34] "POST /addsign HTTP/1.1" 413 -

from flask import redirect
#app.errorhandler(413)
def largefile_error(e):
return redirect(url_for("your_function_name")), 413
you can use this for reference

flask caching - handle exception when redis service is down

I have a simple code using flask:
#app.route('/foo/<arg>')
#app.cache.memoize()
def foo_response(arg):
return 'Hello ' + arg
This is working great while my redis server (cache server) is up.
If the redis server goes down, an exception is raised every time I query /foo/<arg>, which is understandable.
How (and where) can I handle that exception (à la try-except) in order to not use the redis server if it is down at that moment?

It is actually implemented this way. By checking the source of memoize() in Flask-Cache package you see
try:
cache_key = decorated_function.make_cache_key(f, *args, **kwargs)
rv = self.cache.get(cache_key)
except Exception:
if current_app.debug:
raise
logger.exception("Exception possibly due to cache backend.")
return f(*args, **kwargs)
This means if you are on production i.e. app.debug=False you will see the exception log and the function will be called normally.

Finding the real error in a Webtest test failure

I'm using Python and Webtest to test a WSGI application. I found that exceptions raised in the handler code tend to be swallowed by Webtest, which then raises a generic:
AppError: Bad response: 500 Internal Server Error
How do I tell it to raise or print the original error that caused this?

While clj's answer certainly works, you may still want to access the response in your test case. To do this, you can use expect_errors=True (from the webtest documentation) when you make your request to the TestApp, and that way no AppError will be raised. Here is an example where I am expecting a 403 error:
# attempt to access secure page without logging in
response = testapp.get('/secure_page_url', expect_errors=True)
# now you can assert an expected http code,
# and print the response if the code doesn't match
self.assertEqual(403, response.status_int, msg=str(response))

Your WSGI framework and server contains handlers which catch exceptions and performs some action (render a stacktrace in the body, log the backtrace to a logfile, etc). Webtest, by default, does not show the actual response, which might be useful if your framework renders a stacktrace in the body. I use the following extension to Webtest when I need to look at the body of the response:
class BetterTestApp(webtest.TestApp):
"""A testapp that prints the body when status does not match."""
def _check_status(self, status, res):
if status is not None and status != res.status_int:
raise webtest.AppError(
"Bad response: %s (not %s)\n%s", res.status, status, res)
super(BetterTestApp, self)._check_status(status, res)
Getting more control over what happens to the exception depends on what framework and server you are using. For the built in wsgiref module you might be able to override error_output to achieve what you want.