I am setting a file limit in Flask. When files are uploaded it correctly blocks files over 4mb and returns a 413 error. However this is closing the connection on the server which means that the redirect doesn't work. What I want to do is keep the connection open on this error so I can then redirect the user.
My file size limit is set as:
app.config['MAX_CONTENT_LENGTH'] = 4 * 1024 * 1024
The code which captures the exception is:
try:
form = FileUploadForm();
except Exception as e:
print(e)
flash(u"File size too large: please choose a file under 4mb","danger")
return redirect(url_for("home"))
This is the exception returned:
413 Request Entity Too Large: The data value transmitted exceeds the capacity limit.
The code "return redirect(url_for("home"))" doesn't work, I believe because the connection is closed on error 413. This is Flask running in development mode.
I have tried the code in the suggested answer, but that doesn't seem to work. My code now looks like:
#app.errorhandler(413)
def largefile_error(e):
print("Large file")
return redirect(url_for("addsign")), 413
#app.route('/addsign', methods=['GET', 'POST'])
def addsign():
if not current_user.is_authenticated:
flash(f"Please login to upload a sign","danger")
return redirect(url_for('home'))
form = FileUploadForm()
In the console I can see the following output.
Large file
127.0.0.1 - - [13/Jul/2020 18:02:34] "POST /addsign HTTP/1.1" 413 -
from flask import redirect
#app.errorhandler(413)
def largefile_error(e):
return redirect(url_for("your_function_name")), 413
you can use this for reference
Related
I have a flask app that receives bad requests from another software. I would like to manually handle these requests as I can then still call the relevant functions. A sample bad request looks like this GET GET / HTTP/1.1 with the additional GET in front
Hence, I tried to add a custom error handler as shown in the documentation. However, I am unable to make it work and the default error handler is used instead. This is my code:
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello, World!'
#app.errorhandler(400)
def handle_bad_request(e):
print('Custom handler active')
return 'bad request!', 400
if __name__ == '__main__':
app.run()'
I used Packetsender to send similar bad requests, and Flask receives them and uses the default error handler as I can see in my console:
127.0.0.1 - - [18/Jan/2021 17:49:28] code 400, message Bad request syntax ('GET GET / HTTP/1.1')
127.0.0.1 - - [18/Jan/2021 17:49:28] "GET GET / HTTP/1.1" HTTPStatus.BAD_REQUEST
So, why is my custom error handler not used? Is this somehow related to the ridiculously incorrect syntax of the requests? Any feedback would be helpful
Error handlers are for unhandled errors in the application code. Sending a bad HTTP request causes an error at the HTTP server layer (the Werkzeug development server in your case, or an HTTP server like Nginx in production). There is no way for Flask or any other WSGI application to handle the errors you're seeing, as the request is invalid, so it doesn't make it to the application layer.
I want to have a custom error page for internal errors. Right know i'm testing it by restarting my database service and trying to load a page before it connects again. The queries return an OperationalError
I've disabled debug mode so that i don't get the stack trace but instead of the custom 500 page I get the default text on white background.
I'm not sure if this is the correct way to link a sqlalchemy error to the custom 500 page. If I don't define a event listener for sqlalchemy the flask error handler won't get called.
#db.event.listens_for(db.engine, "handle_error")
def handle_exception(context):
internal_error(context.original_exception)
#app.errorhandler(500)
def internal_error(error):
db.session.rollback()
return render_template('500.html'), 500
Removing the sqlalchemy event or reraising the error there also doesn't work.
127.0.0.1 - - [11/Jul/2019 17:29:04] "GET /products HTTP/1.1" 500 -
Error on request:
...stack strace...
sqlalchemy.exc.OperationalError: (psycopg2.errors.AdminShutdown) terminating connection due to administrator command
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
UPDATE: Now that I thik about it. I don't think is has anything to do with sqlalchemy. I tried dividing by zero to see if I will get my custom page but the same thing happens.
I am trying to catch an exception when using a Response object. My use case is that I want to stream data like described in the official documentation and that there is the possibility of an exception when I get the data that I want to stream:
# test.py
from flask import Flask, Response, jsonify
import werkzeug
app = Flask(__name__)
#app.route("/")
def hello():
def generate():
raise Exception
for i in range(0,10):
yield '1'
try:
return Response(generate())
except Exception:
return jsonify("error") # expected this but instead 500 server error is returned
if __name__ == '__main__':
app.run()
When I then run the server and request the data, I would expect to see "error" but instead an Internal Server Error message from Flask is shown.
FLASK_APP=test.py flask run
curl http://localhost:5000/
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to
complete your request. Either the server is overloaded or there is
an error in the application.</p>
In some way the exception gets raised, as we can see it in stdout but does not get catched by Flask.
I also tried to make sure that the exception does not get catched by setting passthrough_errors option of the werkzeug server. Then the exception does not get catched in Werkzeug e.g. here or here. Unfortunately this did not helped for catching it in the Flask application above:
app.run(passthrough_errors=True)
Output of flask --version:
Flask 1.0.2
Python 3.7.0 (default, Aug 22 2018, 15:22:33)
[Clang 9.1.0 (clang-902.0.39.2)]
UPDATE:
This problem also occurs when I am not using the streaming:
from flask import Flask, Response, jsonify
app = Flask(__name__)
#app.route("/")
def hello():
try:
return Response(2)
except Exception as error:
return jsonify({'error': error})
if __name__ == '__main__':
app.run(passthrough_errors=False)
A TypeError gets raised on Flask side, because the Integer is not iterable but the exception does not get catched.
FLASK_APP=test.py flask run
curl http://localhost:5000/
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to
complete your request. Either the server is overloaded or there is
an error in the application.</p>
In case you come here from a web search because your exceptions in a generator don't get rendered on the client side: That's because the pattern of generators in responses is really HTTP chunked streaming. This technique results in one HTTP response being sent to the client in chunks. The HTTP response code (e.g. 200 for success) is returned first. So if any of the following chunks trigger an exception in your code, Flask has no possibility of returning that to the browser.
Let's change your code and run it again:
#app.route("/")
def hello():
def generate():
raise Exception
yield '1'
try:
resp = Response(generate())
data = resp.data
return resp
except Exception:
return jsonify("error") # expected this but instead 500 server error is returned
This code will correctly return "error", the previous one also "correctly" raises a 500 server error. To understand why, you have to think of the execution flow. In your initial code, return Response(generate()) will return immediately without executing the generator function. The consumer, in this case probably werkzeug, will try to read data or a similar property which will then cause the generator to run. By that time your function is already executed and of course no exception has happened, since you are returning the generator wrapped in a Response. Hope this helps.
Update
The above code only demonstrates the problem and explains why the exception is not caught in the original code. If you have an error in the middle of your stream IMHO the application should give a 500 server error and die. Moving the exception to the generator however will let you catch such errors:
#app.route("/")
def hello():
def generate():
try:
raise Exception('some error')
for i in range(0,10):
yield '1'
except Exception as e:
yield str(e)
return Response(generate())
And you can't use jsonify because the generator is being consumed outside of your application context.
I wrote a general error handler for a flask application like this
def error_handler(error):
if isinstance(error, HTTPException):
description = error.get_description(request.environ)
code = error.code
name = error.name
else:
description = ("We encountered an error "
"while trying to fulfill your request")
code = 500
name = 'Internal Server Error'
templates_to_try = ['errors/error{}.html'.format(code), 'errors/generic_error.html']
return render_template(templates_to_try,
code=code,
name=Markup(name),
description=Markup(description),
error=error)
def init_app(app):
''' Function to register error_handler in app '''
for exception in default_exceptions:
app.register_error_handler(exception, error_handler)
app.register_error_handler(Exception, error_handler)
which I registered with the app as
error_handler.init_app(app)
but in case of a 413 error (Request Entity Too Large) I do not get redirected to the error handler? Instead, I can create an additional error handler like this
#app.errorhandler(413)
def request_entity_too_large(error):
return 'File Too Large', 413
which catches the error fine.
I found that when I raise the RequestEntityTooLarge error artificially within my app, the error handler works fine. So it must have to do with the fact that the error gets raise within the werkzeuge package
RequestBase._load_form_data(self)
File "/usr/local/lib/python2.7/site-packages/werkzeug/wrappers.py", line 385, in _load_form_data
mimetype, content_length, options)
File "/usr/local/lib/python2.7/site-packages/werkzeug/formparser.py", line 197, in parse
raise exceptions.RequestEntityTooLarge()
RequestEntityTooLarge: 413 Request Entity Too Large: The data value transmitted exceeds the capacity limit.
Does anybody know why my first solution cannot capture 413 errors? But my second solution can? How would I need to modify my error_handler to capture the 413 error?
ok I found the solution. Changing the error_handler to
return render_template(templates_to_try,
code=code,
name=Markup(name),
description=Markup(description),
error=error), code
does solve the problem... not sure exactly why though
The problem lies in the flask development server. It is not really a fully fledged server and falls short in that aspect. You don't have to worry about it because in production WSGI server etc it will work as expected with a normal error handler.
To quote flask documentation:
When using the local development server, you may get a connection
reset error instead of a 413 response. You will get the correct status
response when running the app with a production WSGI server.
I am writing a web application with Pyramid and would like to restrict the maximum length for POST requests, so that people can't post huge amount of data and exhaust all the memory on the server. However I looked pretty much everywhere I could think of (Pyramid, WebOb, Paster) and couldn't find any option to accomplish this. I've seen that Paster has limits for the number of HTTP headers, length each header, etc., but I didn't see anything for the size of the request body.
The server will be accepting POST requests only for JSON-RPC, so I don't need to allow huge request body sizes. Is there a way in the Pyramid stack of accomplishing this?
Just in case this is not obvious from the rest, a solution which has to accept and load the whole request body into memory before checking the length and returning a 4xx error code defeats the purpose of what's I'm trying to do, and is not what I'm looking for.
Not really a direct answer to your question. As far as I know, you can create a wsgi app that will load the request if the body is below the configuration setting you can pass it to the next WSGI layer. If it goes above you can stop reading and return an error directly.
But to be honest, I really don't see the point to do it in pyramid. For example, if you run pyramid behind a reverse proxy with nginx or apache or something else.. you can always limit the size of the request with the frontend server.
unless you want to run pyramid with Waitress or Paster directly without any proxy, you should handle body size in the front end server that should be more efficient than python.
Edit
I did some research, it isn't a complete answer but here is something that can be used I guess. You have to read environ['wsgi_input'] as far as I can tell. This is a file like object that receives chunk of data from nginx or apache for example.
What you really have to do is read that file until max lenght is reached. If it is reached raise an Error if it isn't continue the request.
You might want to have a look at this answer
You can do it in a variety of ways here's a couple of examples. one using wsgi middleware based on webob(installed when you install pyramid among other things). and one that uses pyramids event mechanism
"""
restricting execution based on request body size
"""
from pyramid.config import Configurator
from pyramid.view import view_config
from pyramid.events import NewRequest, subscriber
from webob import Response, Request
from webob.exc import HTTPBadRequest
import unittest
def restrict_body_middleware(app, max_size=0):
"""
this is straight wsgi middleware and in this case only depends on
webob. this can be used with any wsgi compliant web
framework(which is pretty much all of them)
"""
def m(environ, start_response):
r = Request(environ)
if r.content_length <= max_size:
return r.get_response(app)(environ, start_response)
else:
err_body = """
request content_length(%s) exceeds
the configured maximum content_length allowed(%s)
""" % (r.content_length, max_size)
res = HTTPBadRequest(err_body)
return res(environ, start_response)
return m
def new_request_restrict(event):
"""
pyramid event handler called whenever there is a new request
recieved
http://docs.pylonsproject.org/projects/pyramid/en/1.2-branch/narr/events.html
"""
request = event.request
if request.content_length >= 0:
raise HTTPBadRequest("too big")
#view_config()
def index(request):
return Response("HI THERE")
def make_application():
"""
make appplication with one view
"""
config = Configurator()
config.scan()
return config.make_wsgi_app()
def make_application_with_event():
"""
make application with one view and one event subsriber subscribed
to NewRequest
"""
config = Configurator()
config.add_subscriber(new_request_restrict, NewRequest)
return config.make_wsgi_app()
def make_application_with_middleware():
"""
make application with one view wrapped in wsgi middleware
"""
return restrict_body_middleware(make_application())
class TestWSGIApplication(unittest.TestCase):
def testNoRestriction(self):
app = make_application()
request = Request.blank("/", body="i am a request with a body")
self.assert_(request.content_length > 0, "content_length should be > 0")
response = request.get_response(app)
self.assert_(response.status_int == 200, "expected status code 200 got %s" % response.status_int)
def testRestrictedByMiddleware(self):
app = make_application_with_middleware()
request = Request.blank("/", body="i am a request with a body")
self.assert_(request.content_length > 0, "content_length should be > 0")
response = request.get_response(app)
self.assert_(response.status_int == 400, "expected status code 400 got %s" % response.status_int)
def testRestrictedByEvent(self):
app = make_application_with_event()
request = Request.blank("/", body="i am a request with a body")
self.assert_(request.content_length > 0, "content_length should be > 0")
response = request.get_response(app)
self.assert_(response.status_int == 400, "expected status code 400 got %s" % response.status_int)
if __name__ == "__main__":
unittest.main()