I am trying to catch an exception when using a Response object. My use case is that I want to stream data like described in the official documentation and that there is the possibility of an exception when I get the data that I want to stream:
# test.py
from flask import Flask, Response, jsonify
import werkzeug
app = Flask(__name__)
#app.route("/")
def hello():
def generate():
raise Exception
for i in range(0,10):
yield '1'
try:
return Response(generate())
except Exception:
return jsonify("error") # expected this but instead 500 server error is returned
if __name__ == '__main__':
app.run()
When I then run the server and request the data, I would expect to see "error" but instead an Internal Server Error message from Flask is shown.
FLASK_APP=test.py flask run
curl http://localhost:5000/
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to
complete your request. Either the server is overloaded or there is
an error in the application.</p>
In some way the exception gets raised, as we can see it in stdout but does not get catched by Flask.
I also tried to make sure that the exception does not get catched by setting passthrough_errors option of the werkzeug server. Then the exception does not get catched in Werkzeug e.g. here or here. Unfortunately this did not helped for catching it in the Flask application above:
app.run(passthrough_errors=True)
Output of flask --version:
Flask 1.0.2
Python 3.7.0 (default, Aug 22 2018, 15:22:33)
[Clang 9.1.0 (clang-902.0.39.2)]
UPDATE:
This problem also occurs when I am not using the streaming:
from flask import Flask, Response, jsonify
app = Flask(__name__)
#app.route("/")
def hello():
try:
return Response(2)
except Exception as error:
return jsonify({'error': error})
if __name__ == '__main__':
app.run(passthrough_errors=False)
A TypeError gets raised on Flask side, because the Integer is not iterable but the exception does not get catched.
FLASK_APP=test.py flask run
curl http://localhost:5000/
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to
complete your request. Either the server is overloaded or there is
an error in the application.</p>
In case you come here from a web search because your exceptions in a generator don't get rendered on the client side: That's because the pattern of generators in responses is really HTTP chunked streaming. This technique results in one HTTP response being sent to the client in chunks. The HTTP response code (e.g. 200 for success) is returned first. So if any of the following chunks trigger an exception in your code, Flask has no possibility of returning that to the browser.
Let's change your code and run it again:
#app.route("/")
def hello():
def generate():
raise Exception
yield '1'
try:
resp = Response(generate())
data = resp.data
return resp
except Exception:
return jsonify("error") # expected this but instead 500 server error is returned
This code will correctly return "error", the previous one also "correctly" raises a 500 server error. To understand why, you have to think of the execution flow. In your initial code, return Response(generate()) will return immediately without executing the generator function. The consumer, in this case probably werkzeug, will try to read data or a similar property which will then cause the generator to run. By that time your function is already executed and of course no exception has happened, since you are returning the generator wrapped in a Response. Hope this helps.
Update
The above code only demonstrates the problem and explains why the exception is not caught in the original code. If you have an error in the middle of your stream IMHO the application should give a 500 server error and die. Moving the exception to the generator however will let you catch such errors:
#app.route("/")
def hello():
def generate():
try:
raise Exception('some error')
for i in range(0,10):
yield '1'
except Exception as e:
yield str(e)
return Response(generate())
And you can't use jsonify because the generator is being consumed outside of your application context.
Related
I have a Flask server.
Whenever the code inside my handler throws an exception, Flask catches it, and returns an HTML page to the client with a 5XX error.
The problem is that I don't notice this. I just got an email from someone using my API saying that they were getting 504 errors, and I didn't know about it until they told me.
In other non-Flask parts of my application I wrote a custom decorator to catch all exceptions, send an email to me, then re-throw. I would like something similar for my Flask app.
I want to find a way to have Flask call a function of mine every time my handler code throws an exception, before it returns a response to the client. I do not wish to modify the response that gets sent to the client. I don't want to change how Flask handles errors, or how it catches them. I just want some way of being notified, and then Flask can continue doing the default error handling behavior.
I suppose I could put a decorator over every single route handler to catch and rethrow exceptions before Flask sees them, but that's messy. I just know I'll forget one of them, especially when I add new ones in the future.
MWE
A buggy application:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
assert False, "buggy code here"
return "hello"
def error_handler(exc_type, exc_val, exc_tb):
send_email(exc_type, exc_val, exc_tb)
# This is the part I don't know
# I want something along the lines of:
app.config['ERROR_HOOK'] = error_handler
from flask import Flask
app = Flask(__name__)
app.debug = False
app.config['PROPAGATE_EXCEPTIONS'] = True
#app.errorhandler(Exception)
def all_exception_handler(error):
print(str(error))
#app.errorhandler(404)
def page_not_found(error):
return 'This page does not exist', 404
you can define a function for each specific error you want to catch #app.my_custom_errorhandler(code_or_exception)
The argument to your error handler function will be an Exception.
I have a flask app that receives bad requests from another software. I would like to manually handle these requests as I can then still call the relevant functions. A sample bad request looks like this GET GET / HTTP/1.1 with the additional GET in front
Hence, I tried to add a custom error handler as shown in the documentation. However, I am unable to make it work and the default error handler is used instead. This is my code:
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello, World!'
#app.errorhandler(400)
def handle_bad_request(e):
print('Custom handler active')
return 'bad request!', 400
if __name__ == '__main__':
app.run()'
I used Packetsender to send similar bad requests, and Flask receives them and uses the default error handler as I can see in my console:
127.0.0.1 - - [18/Jan/2021 17:49:28] code 400, message Bad request syntax ('GET GET / HTTP/1.1')
127.0.0.1 - - [18/Jan/2021 17:49:28] "GET GET / HTTP/1.1" HTTPStatus.BAD_REQUEST
So, why is my custom error handler not used? Is this somehow related to the ridiculously incorrect syntax of the requests? Any feedback would be helpful
Error handlers are for unhandled errors in the application code. Sending a bad HTTP request causes an error at the HTTP server layer (the Werkzeug development server in your case, or an HTTP server like Nginx in production). There is no way for Flask or any other WSGI application to handle the errors you're seeing, as the request is invalid, so it doesn't make it to the application layer.
From the uwsgi documentation:
def application(env, start_response):
start_response('200 OK', [('Content-Type','text/html')])
return [b"Hello World"]
Is it possible to respond to http request(close http connection) and continue execution flow(without any usage of threads/queues/external services etc)?
like this:
def application(env, start_response):
start_response('200 OK', [('Content-Type','text/html')])
end_response(b"Hello World")
#HTTP connection is closed
#continue execution..
TL;DR - if you're using Django, you can skip to the end of the answer.
Yes, there is a way to do this. You can hook on to a .close() method of the object returned by your application callable (or alternatively, wsgi.file_wrapper returned through the env.
Check PEP333, specifically the server side spec part: https://peps.python.org/pep-0333/#the-server-gateway-side:
result = application(environ, start_response)
try:
for data in result:
if data: # don't send headers until body appears
write(data)
if not headers_sent:
write('') # send headers now if body was empty
finally:
if hasattr(result, 'close'):
result.close()
As you can see, result.close() will be called at the very end, by which point the data is already sent.
This is, of course, just some reference code, and it does not deal with upstream and terminating the connection before continuing execution. But well-behaved wsgi servers, such as uwsgi and (presumably) gunicorn, do.
They will signal to the upstream that the request has finished sending by closing the socket (or whatever the upstream protocol requires), and then call .close().
If you are using Django, you are already set, because it has request_finished signal. Here's how it works, it hooks to .close() as described above:
https://github.com/django/django/blob/57c7220280db19dc9dda0910b90cf1ceac50c66f/django/http/response.py#L323
Unfortunately, there is no way to continue the code execution after you have returned the response. It would be much easier if you use multithreading but if not you can workaround it in Flask by adding an AJAX call to your HTML response which will send a POST request to one of the server extra route whose handler function will be the execution code you want after returning the response. Here's one of the possible approach using Flask:
myflaskapp.py
from flask import Flask, render_template_string
import time
app = Flask(__name__)
#app.route('/run', methods=['POST'])
def run():
# this is where you put your "continue execution..." code
# below code is used to test if it runs after HTTP connection close
time.sleep(8)
print('Do something')
return ''
#app.route('/')
def index():
return render_template_string('''
Hello World!
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
<script>
$(function() {
$.ajax({
type: "POST",
url: "{{ url_for('run') }}"
});
})
</script>
''')
if __name__ == "__main__":
app.run(host='0.0.0.0')
You can run the server on port 9091 with the command:
uwsgi --http 127.0.0.1:9091 --wsgi-file myflaskapp.py --callable app
To test if it is working or not, you can go to the address localhost:9091. If everything works well, you should see that the page is loaded immediately while the terminal will only print out Do something after 8 seconds have passed, indicating the function run executes after the HTTP connection is closed.
I want to add global exception handling object to my Flask webproject. In main module, where application class is created I've added code to override sys.excepthook. Here is simple test code:
import sys
def my_exception_hook(exception_type, value, traceback):
print "My exception handler"
print " Exception type:", exception_type
print " Exception instance:", value
print " Traceback:", traceback
sys.__excepthook__(exception_type, value, traceback)
sys.excepthook = my_exception_hook
from flask import Flask
import requests
app = Flask(__name__)
#app.route('/')
def index():
#Exception raised here "requests.exceptions.MissingSchema" not captured by my handler
r = requests.get('ololo', auth=('user', 'pass'))
return "Hello, world!"
#raise Exception("AAA") #Exception raised here is successfully goes to my_exception_hook
app.run(debug=True, port=5001)
I do not want and have no possibility to envelop each call or requests module with try/catch. Also I want to handle other eaception, for example, mongoDB connection problem which may occured spontaneousely (not wheen I creating connection), please do not suggest it.
Flask already handles exceptions in views for you. Using sys.excepthook is not the right way to handle this, the hook will never be called.
Specify a custom exception handler with Flask instead, see the Redirects and Errors section of the Quickstart manual:
from flask import render_template
#app.errorhandler(500)
def page_not_found(error):
return 'some error message'
Flask also uses the logging module to record exceptions; configure the logging module to write anything of severity logging.ERROR to the console or a log file.
Since you use app.run(debug=True, port=5001), you'll already see exceptions printed to the console.
Note that the 500 error page is only ever invoked when debug is not set to True however; otherwise the Werkzeug debug view is invoked instead.
I'm using Python and Webtest to test a WSGI application. I found that exceptions raised in the handler code tend to be swallowed by Webtest, which then raises a generic:
AppError: Bad response: 500 Internal Server Error
How do I tell it to raise or print the original error that caused this?
While clj's answer certainly works, you may still want to access the response in your test case. To do this, you can use expect_errors=True (from the webtest documentation) when you make your request to the TestApp, and that way no AppError will be raised. Here is an example where I am expecting a 403 error:
# attempt to access secure page without logging in
response = testapp.get('/secure_page_url', expect_errors=True)
# now you can assert an expected http code,
# and print the response if the code doesn't match
self.assertEqual(403, response.status_int, msg=str(response))
Your WSGI framework and server contains handlers which catch exceptions and performs some action (render a stacktrace in the body, log the backtrace to a logfile, etc). Webtest, by default, does not show the actual response, which might be useful if your framework renders a stacktrace in the body. I use the following extension to Webtest when I need to look at the body of the response:
class BetterTestApp(webtest.TestApp):
"""A testapp that prints the body when status does not match."""
def _check_status(self, status, res):
if status is not None and status != res.status_int:
raise webtest.AppError(
"Bad response: %s (not %s)\n%s", res.status, status, res)
super(BetterTestApp, self)._check_status(status, res)
Getting more control over what happens to the exception depends on what framework and server you are using. For the built in wsgiref module you might be able to override error_output to achieve what you want.