Flask Middleware with both Request and Response - python

I want to create a middleware function in Flask that logs details from the request and the response. The middleware should run after the Response is created, but before it is sent back. I want to log:
The request's HTTP method (GET, POST, or PUT)
The request endpoint
The response HTTP status code, including 500 responses. So, if an exception is raised in the view function, I want to record the resulting 500 Response before the Flask internals send it off.
Some options I've found (that don't quite work for me):
The before_request and after_request decorators. If I could access the request data in after_request, my problems still won't be solved, because according to the documentation
If a function raises an exception, any remaining after_request functions will not be called.
Deferred Request Callbacks - there is an after_this_request decorator described on this page, which decorates an arbitrary function (defined inside the current view function) and registers it to run after the current request. Since the arbitrary function can have info from both the request and response in it, it partially solves my problem. The catch is that I would have to add such a decorated function to every view function; a situation I would very much like to avoid.
#app.route('/')
def index():
#after_this_request
def add_header(response):
response.headers['X-Foo'] = 'Parachute'
return response
return 'Hello World!'
Any suggestions?


My first answer is very hacky. There's actually a much better way to achieve the same result by making use of the g object in Flask. It is useful for storing information globally during a single request. From the documentation:
The g name stands for “global”, but that is referring to the data being global within a context. The data on g is lost after the context ends, and it is not an appropriate place to store data between requests. Use the session or a database to store data across requests.
This is how you would use it:
#app.before_request
def gather_request_data():
g.method = request.method
g.url = request.url
#app.after_request
def log_details(response: Response):
g.status = response.status
logger.info(f'method: {g.method}\n url: {g.url}\n status: {g.status}')
return response
Gather whatever request information you want in the function decorated with #app.before_request and store it in the g object.
Access whatever you want from the response in the function decorated with #app.after_request. You can still refer to the information you stored in the g object from step 1. Note that you'll have to return the response at the end of this function.


you can use flask-http-middleware for it link
from flask import Flask
from flask_http_middleware import MiddlewareManager, BaseHTTPMiddleware
app = Flask(__name__)
class MetricsMiddleware(BaseHTTPMiddleware):
def __init__(self):
super().__init__()
def dispatch(self, request, call_next):
url = request.url
response = call_next(request)
response.headers.add("x-url", url)
return response
app.wsgi_app = MiddlewareManager(app)
app.wsgi_app.add_middleware(MetricsMiddleware)
#app.get("/health")
def health():
return {"message":"I'm healthy"}
if __name__ == "__main__":
app.run()
Every time you make request, it will pass the middleware


Okay, so the answer was staring me in the face the whole time, on the page on Deferred Request Callbacks.
The trick is to register a function to run after the current request using after_this_request from inside the before_request callback. This is the code snippet of what worked for me:
#app.before_request
def log_details():
method = request.method
url = request.url
#after_this_request
def log_details_callback(response: Response):
logger.info(f'method: {method}\n url: {url}\n status: {response.status}')
These are the steps:
Get the required details from the response in the before_request callback and store them in some variables.
Then access what you want of the response in the function you decorate with after_this_request, along with the variables you stored the request details in earlier.

Related

Is there a way to convert local flask http request to a function call?

Currently, in our system, we are calling the endpoints even in the same flask application by a HTTP request. All the requests is called through a make_request wrapper method as shown below:
def make_request(url, body, http_type="get"):
http_fn = getattr(requests, http_type)
response = http_fn(url, headers=headers, json=body)
return response.status_code, response
Hence I'm trying to convert all local requests within the same flask application to a direct method call so that any endpoint within the same flask application is called this way:
def make_request(url, body, http_type="get"):
# Figure out If its local request call the function of the endpoint and construct the response
# If not make an http request
return response.status_code, response
EDIT: Tried searching in the url_map to find the method associated with the endpoint but the function returned in not in a callabale state. Any points on how we can call the method from here?
for rule in current_app.url_map.iter_rules():
if my_url in rule.rule:
endpoint = rule.endpoint
for key, view in current_app.view_functions.items():
if key == endpoint:
# Found the view function, need to know how to call
# the right method( GET, POST etc)
view contains the following:
{
'view_class': <class 'endpoints.attribute_endpoints.AttributeEndpoint'>,
'methods': {'GET', 'PUT', 'POST'}, 'provide_automatic_options': None, '__wrapped__': <function View.as_view.<locals>.view at 0x10c9190d0>}

If I understand correctly, what you're trying to achieve is calling a flask endpoint internally without going over http. Look at the solution below and let me know if it is does what you want.
Old Code:
#app.route('/someRoute', methods=['GET'])
def some_route_function():
json_object = request.get_json()
my_number = json_object['myNumber']
my_number = my_number**2
return jsonify(my_number=my_number)
New Code:
def square_number_func(number):
return number**2
#app.route('/someRoute', methods=['GET'])
def some_route_function():
json_object = request.get_json()
my_number = json_object['myNumber']
my_number = square_number_func(my_number)
return jsonify(my_number=my_number)
def my_non_flask_function():
my_number = 17
my_number = square_number_func(my_number)
This way you get the functionality you need without having to rely on Flask's request object, nor having to call flask via http.
Edit: If you need to figure out if it's an internal flask function then you compare it against a list of functions in your local global scope or in your flask app, as it does store your routes. You can even store the function parameters so you know what to call. Finally, you can map each endpoint to another function if you want inside a dictionary or something, such as {some_route_function: square_number_func} so that you can tell which function to substitute for the http call.

Is there a way to set a cookie in #before.request?

I want my application to be able to detect the user's language to serve the corresponding page to them. My idea was to use #before.request to read the Accepted-Languages header, match it against the app's supported languages, and set a cookie if needed, but it seems that the last step is not achievable.
Here is the code for that:
#app.before_request
def before_request_callback():
if request.cookies.get('lang'):
pass
else:
lang = request.accept_languages.best_match(supported_languages)
print(lang)
#I would like to set the cookie here
I thought about setting the cookie in the response object directly in the decorator (resp.set_cookie()) and thus reformatted all the returns in my app to look like this
#app.route("/")
def hello():
resp = make_response(render_template('index.html'))
return resp
to maybe be able to fetch the cookie and attach it to this response, but since the response is created afterwards in the endpoitn function, i have no idea how to do that either.
I also thought about creating the response directly in the decorator, but since i need the return condition, i don't know if thats possible

I think you're trying to use a cookie when you don't need to. As you noted, you can only set cookies on the response. You send the browser cookies in the response and then it re-sends them to you in any subsequent request. But it's already sending you the Accept-Language. So there's no point in setting a cookie on the request. It's already arrived, and already contains what you need.
Instead of getting hung up on setting cookies, just consult the request at the point that you're generating the response to ensure that the served content is in the appropriate language.
from flask import request
#app.route("/")
def hello():
lang = request.accept_languages.best_match(supported_languages)
return render_template(f'{lang}/index.html')

Passing session from template view to python requests api call

I want to make multiple internal REST API call from my Django TemplateView, using requests library. Now I want to pass the session too from template view to api call. What is the recommended way to do that, keeping performance in mind.
Right now, I'm extracting cookie from the current request object in template view, and passing that to requests.get() or requests.post() call. But problem with that is, I would have to pass request object to my API Client, which I don't want.
This the current wrapper I'm using to route my requests:
def wrap_internal_api_call(request, requests_api, uri, data=None, params=None, cookies=None, is_json=False, files=None):
headers = {'referer': request.META.get('HTTP_REFERER')}
logger.debug('Request API: %s calling URL: %s', requests_api, uri)
logger.debug('Referer header sent with requests: %s', headers['referer'])
if cookies:
csrf_token = cookies.get('csrftoken', None)
else:
csrf_token = request.COOKIES.get('csrftoken', None)
if csrf_token:
headers['X-CSRFToken'] = csrf_token
if data:
if is_json:
return requests_api(uri, json=data, params=params, cookies=cookies if cookies else request.COOKIES, headers=headers)
elif not files:
return requests_api(uri, data=data, params=params, cookies=cookies if cookies else request.COOKIES, headers=headers)
else:
return requests_api(uri, data=data, files=files, params=params, cookies=cookies if cookies else request.COOKIES,
headers=headers)
else:
return requests_api(uri, params=params, cookies=cookies if cookies else request.COOKIES, headers=headers)
Basically I want to get rid of that request parameter (1st param), because then to call it I've to keep passing request object from TemplateViews to internal services. Also, how can I keep persistent connection across multiple calls?

REST vs Invoking the view directly
While it's possible for a web app to make a REST API call to itself. That's not what REST is designed for. Consider the following from: https://docs.djangoproject.com/ja/1.9/topics/http/middleware/
As you can see a django request/response cycle has quite a bit of overhead. Add to this the overhead of webserver and wsgi container. At the client side you have the overhead associated with the requests library, but hang on a sec, the client also happens to be the same web app so it become s part of the web app's overhead too. And there is the problem of peristence (which I will come to shortly).
Last but not least, if you have a DNS round robin setup your request may actually go out on the wire before coming back to the same server. There is a better way, to invoke the view directly.
To invoke another view without the rest API call is really easy
other_app.other_view(request, **kwargs)
This has been discussed a few times here at links such as Django Call Class based view from another class based view and Can I call a view from within another view? so I will not elaborate.
Persistent requests
Persistent http requests (talking about python requests rather than django.http.request.HttpRequest) are managed through session objects (again not to be confused with django sessions). Avoiding confusion is really difficult:
The Session object allows you to persist certain parameters across
requests. It also persists cookies across all requests made from the
Session instance, and will use urllib3's connection pooling. So if
you're making several requests to the same host, the underlying TCP
connection will be reused, which can result in a significant
performance increase
Different hits to your django view will probably be from different users so you don't want to same cookie reused for the internal REST call. The other problem is that the python session object cannot be persisted between two different hit to the django view. Sockets cannot generally be serialized, a requirement for chucking them into memcached or redis.
If you still want to persist with internal REST
I think #julian 's answer shows how to avoid passing the django request instance as a parameter.

If you want to avoid passing the request to wrap_internal_api_call, all you need to do is do a bit more work on the end of the TemplateView where you call the api wrapper. Note that your original wrapper is doing a lot of cookies if cookies else request.COOKIES. You can factor that out to the calling site. Rewrite your api wrapper as follows:
def wrap_internal_api_call(referer, requests_api, uri, data=None, params=None, cookies, is_json=False, files=None):
headers = {'referer': referer}
logger.debug('Request API: %s calling URL: %s', requests_api, uri)
logger.debug('Referer header sent with requests: %s', referer)
csrf_token = cookies.get('csrftoken', None)
if csrf_token:
headers['X-CSRFToken'] = csrf_token
if data:
if is_json:
return requests_api(uri, json=data, params=params, cookies=cookies, headers=headers)
elif not files:
return requests_api(uri, data=data, params=params, cookies=cookies, headers=headers)
else:
return requests_api(uri, data=data, files=files, params=params, cookies=cookies, headers=headers)
else:
return requests_api(uri, params=params, cookies=cookies, headers=headers)
Now, at the place of invocation, instead of
wrap_internal_api_call(request, requests_api, uri, data, params, cookies, is_json, files)
do:
cookies_param = cookies or request.COOKIES
referer_param = request.META.get['HTTP_REFERER']
wrap_internal_api_call(referer_param, requests_api, uri, data, params, cookies_param, is_json, files)
Now you are not passing the request object to the wrapper anymore. This saves a little bit of time because you don't test cookies over and over, but otherwise it doesn't make a difference for performance. In fact, you could achieve the same slight performance gain just by doing the cookies or request.COOKIES once inside the api wrapper.
Networking is always the tightest bottleneck in any application. So if these internal APIs are on the same machine as your TemplateView, your best bet for performance is to avoid doing an API call.

Basically I want to get rid of that request parameter (1st param), because then to call it I've to keep passing request object from TemplateViews to internal services.
To pass function args without explicitly passing them into function calls you can use decorators to wrap your functions and automatically inject your arguments. Using this with a global variable and some django middleware for registering the request before it gets to your view will solve your problem. See below for an abstracted and simplified version of what I mean.
request_decorators.py
REQUEST = None
def request_extractor(func):
def extractor(cls, request, *args, **kwargs):
global REQUEST
REQUEST = request # this part registers request arg to global
return func(cls, request, *args, **kwargs)
return extractor
def request_injector(func):
def injector(*args, **kwargs):
global REQUEST
request = REQUEST
if len(args) > 0 and callable(args[0]): # to make it work with class methods
return func(args[0], request, args[1:], **kwargs) # class method
return func(request, *args, **kwargs) # function
return injector
extract_request_middleware.py
See the django docs for info on setting up middleware
from request_decorators import request_extractor
class ExtractRequest:
#request_extractor
def process_request(self, request):
return None
internal_function.py
from request_decorators import request_injector
#request_injector
def internal_function(request):
return request
your_view.py
from internal_function import internal_function
def view_with_request(request):
return internal_function() # here we don't need to pass in the request arg.
def run_test():
request = "a request!"
ExtractRequest().process_request(request)
response = view_with_request(request)
return response
if __name__ == '__main__':
assert run_test() == "a request!"

Override Falcon's default error handler when no route matches

When Falcon(-Framework) could not find a route for a specific request, 404 is returned. How can I override this default handler? I want to extend the handler with a custom response.

The default handler when no resource matches is the path_not_found responder:
But as you can see in the _get_responder method of falcon API, it can't be override without some monkey patching.
As far as I can see, there are two different ways to use a custom handler:
Subclass the API class, and overwrite the _get_responder method so it calls your custom handler
Use a default route that matches any route if none of the application ones are matched. You probably prefer to use a sink instead of a route, so you capture any HTTP method (GET, POST...) with the same function.
I would recommend the second option, as it looks much neater.
Your code would look like:
import falcon
class HomeResource:
def on_get(self, req, resp):
resp.body = 'Hello world'
def handle_404(req, resp):
resp.status = falcon.HTTP_404
resp.body = 'Not found'
application = falcon.API()
application.add_route('/', HomeResource())
# any other route should be placed before the handle_404 one
application.add_sink(handle_404, '')

There is a better solution here.
def custom_response_handler(req, resp, ex, params):
resp.status = falcon.HTTP_404
resp.text = "custom text response"
app = falcon.App()
app.add_error_handler(HTTPRouteNotFound, custom_response_handler)

Flask - access the request in after_request or teardown_request

I want to be able to access the request object before I return the response of the HTTP call.
I want access to the request via "teardown_request" and "after_request":
from flask import Flask
...
app = Flask(__name__, instance_relative_config=True)
...
#app.before_request
def before_request():
# do something
#app.after_request
def after_request(response):
# get the request object somehow
do_something_based_on_the_request_endpoint(request)
#app.teardown_request
def teardown_request(response):
# get the request object somehow
do_something_based_on_the_request_endpoint(request)
I saw that I can add the request to g and do something like this:
g.curr_request = request
#app.after_request
def after_request(response):
# get the request object somehow
do_something_based_on_the_request_endpoint(g.curr_request)
But the above seems a bit strange. I'm sure that there's a better way to access the request.
Thanks

The solution is simple -
from flask import request
#app.after_request
def after_request(response):
do_something_based_on_the_request_endpoint(request)
return response

Also try teardown_request(exception). This executes "regardless of whether there was an exception or not". Check the documentation: http://flask.pocoo.org/docs/0.12/api/#flask.Flask.teardown_request

Categories