I am developing a REST API with python and flask, I leave the project here Github project
I added error handlers to the application but when I run an abort function, it gives me a default message from Flask, not the structure I am defining.
I will leave the path to the handlers and where I run the abort from.
Handlers abort(400)
Flask message
Ok, the solution was told to me that it could be in another question.
What to do is to overwrite the handler function of the Flask Api object.
With that, you can configure the format with which each query will be answered, even the ones that contain an error.
def response_structure(code_status: int, response=None, message=None):
if code_status == 200 or code_status == 201:
status = 'Success'
else:
status = 'Error'
args = dict()
args['status'] = status
if message is not None:
args['message'] = message
if response is not None:
args['response'] = response
return args, code_status
class ExtendAPI(Api):
def handle_error(self, e):
return response_structure(e.code, str(e))
Once the function is overwritten, you must use this new one to create
users_bp = Blueprint('users', __name__)
api = ExtendAPI(users_bp)
With this, we can then use the flask functions to respond with the structure that we define.
if request.args.get('name') is None:
abort(400)
Response JSON
{
"response": "400 Bad Request: The browser (or proxy) sent a request that this server could not understand.",
"status": "Error"
}
Related
I have a python client application that calls (http) a python Flask api.
Both of these applications are logging to Azure Application insights using the opencensus libraries.
I want to do the logging in a fashion so that I can correlate the logs in ApplicationInsights end to end.
Python client app
For example, when the client app initiates an HTTP GET call to the Flask API, it generates an http request dependency log entry in ApplicationInsights.
The app also logs individual entries about the http request and http response into the trace table.
Flask API
I am logging the incoming HTTP request in the Flask API using request decorator, and also logging the HTTP response using a request decorator.
Also the actual method ( that the Flask routing invokes ) has its own logging.
Note These are logs go into trace table.
Expectation
I am trying to get the logs generated from the Flask API have a correlation with the log generated from the client application.
Current behaviour
Logs of Python client app
The logs in the dependency table have a operation_Id - All good!
The logs in the trace table have the same operation_Id and operation_ParentId as above - All good!
Logs of Flask api
The logs in the request table have the same operation_Id as above - All good!
The logs in the trace table generated by the before_request, after_request decorators - The operation_Id and operation_ParentId are blank. - Problematic!
The logs in the trace table generated by the logging statements inside the route/methods - The operation_Id and operation_ParentId are blank. - Problematic!
Help please
I can see that Traceparent http header is coming in as part of the http request in the Flask API, but looks like logging is ignoring this.
How do I get the logging statements to use the Traceparent data so that operation_Id and operation_ParentId show up correctly in the traces table for the Flask API?
Flask API Code
import flask
from flask import request, jsonify
import logging
import json
import requests
from opencensus.ext.azure.log_exporter import AzureLogHandler,AzureEventHandler
from opencensus.ext.flask.flask_middleware import FlaskMiddleware
from opencensus.ext.azure.trace_exporter import AzureExporter
from opencensus.trace.samplers import ProbabilitySampler, AlwaysOnSampler
from opencensus.trace.tracer import Tracer
from opencensus.trace import config_integration
import os
logger = logging.getLogger()
class MyJSONEncoder(flask.json.JSONEncoder):
def default(self, obj):
if isinstance(obj, decimal.Decimal):
# Convert decimal instances to strings.
return str(obj)
if isinstance(obj, datetime.datetime):
return obj.strftime(strftime_iso_regular_format_str)
return super(MyJSONEncoder, self).default(obj)
# Initialize logging with Azure Application Insights
class CustomDimensionsFilter(logging.Filter):
"""Add custom-dimensions like run_id in each log by using filters."""
def __init__(self, custom_dimensions=None):
"""Initialize CustomDimensionsFilter."""
self.custom_dimensions = custom_dimensions or {}
def filter(self, record):
"""Add the default custom_dimensions into the current log record."""
dim = {**self.custom_dimensions, **
getattr(record, "custom_dimensions", {})}
record.custom_dimensions = dim
return True
APPLICATION_INSIGHTS_CONNECTIONSTRING=os.getenv('APPLICATION_INSIGHTS_CONNECTIONSTRING')
modulename='FlaskAPI'
APPLICATION_NAME='FlaskAPI'
ENVIRONMENT='Development'
def callback_function(envelope):
envelope.tags['ai.cloud.role'] = APPLICATION_NAME
return True
logger = logging.getLogger(__name__)
log_handler = AzureLogHandler(
connection_string=APPLICATION_INSIGHTS_CONNECTIONSTRING)
log_handler.addFilter(CustomDimensionsFilter(
{
'ApplicationName': APPLICATION_NAME,
'Environment': ENVIRONMENT
}))
log_handler.add_telemetry_processor(callback_function)
logger.addHandler(log_handler)
azureExporter = AzureExporter(
connection_string=APPLICATION_INSIGHTS_CONNECTIONSTRING)
azureExporter.add_telemetry_processor(callback_function)
tracer = Tracer(exporter=azureExporter, sampler=AlwaysOnSampler())
app = flask.Flask("app")
app.json_encoder = MyJSONEncoder
app.config["DEBUG"] = True
middleware = FlaskMiddleware(
app,
exporter=azureExporter,
sampler=ProbabilitySampler(rate=1.0),
)
config_integration.trace_integrations(['logging', 'requests'])
def getJsonFromRequestBody(request):
isContentTypeJson = request.headers.get('Content-Type') == 'application/json'
doesHaveBodyJson = False
if isContentTypeJson:
try:
doesHaveBodyJson = request.get_json() != None
except:
doesHaveBodyJson = False
if doesHaveBodyJson == True:
return json.dumps(request.get_json())
else:
return None
def get_properties_for_customDimensions_from_request(request):
values = ''
if len(request.values) == 0:
values += '(None)'
for key in request.values:
values += key + ': ' + request.values[key] + ', '
properties = {'custom_dimensions':
{
'request_method': request.method,
'request_url': request.url,
'values': values,
'body': getJsonFromRequestBody(request)
}}
return properties
def get_properties_for_customDimensions_from_response(request,response):
request_properties = request_properties = get_properties_for_customDimensions_from_request(request)
request_customDimensions = request_properties.get('custom_dimensions')
response_properties = {'custom_dimensions':
{
**request_customDimensions,
'response_status':response.status,
'response_body':response.data.decode('utf-8')
}
}
return response_properties
# Useful debugging interceptor to log all values posted to the endpoint
#app.before_request
def before():
properties = get_properties_for_customDimensions_from_request(request)
logger.warning("request {} {}".format(
request.method, request.url), extra=properties)
# Useful debugging interceptor to log all endpoint responses
#app.after_request
def after(response):
response_properties = get_properties_for_customDimensions_from_response(request,response)
logger.warning("response: {}".format(
response.status
),extra=response_properties)
return response
#app.route('/api/{}/status'.format("v1"), methods=['GET'])
def health_check():
message = "Health ok!"
logger.info(message)
return message
if __name__ == '__main__':
app.run()
References used
Microsoft's guidance on Application Insights Log Correlation
My code repository where I have tested and reproduced the problem
I am working on a simple service with my show_greeting endpoint handling Get request while set_greeting is my Post.
The purpose of this app is that when "header_message: {header parameter}" is sent to set_greeting, {header parameter} will be returned in the header for responses to show_greeting and to reset {header parameter}, "clear" would reset header_message and header.
I have tried using global variables but encountered an error with shadowing from outside the scope and am not sure which approach to take for this. For now, I would like to learn how to return {header parameter} from my /show_greeting endpoint.
Edit: The /show_greeting endpoint returns holiday_message from the request. The header that I would like to send in addition to holiday_message is "header_message".
My code is as follows:
from flask import Flask, request, make_response, Response
app = Flask(__name__)
#app.route('/show_greeting', methods=['GET'])
def show_greeting():
received = request.args
(I do not know how to set header here from header_message in set_greeting)
return received['holiday_message']
#app.route('/set_greeting', methods=['POST'])
def set_greeting():
posted = request.args
if 'header_message' in posted:
(I attempted save_message = posted['header_message'] here but this approach failed)
return "Header Message Set"
else:
return "Please Send A Header Message"
if __name__ == '__main__':
app.run()
My recommendation is to use the session object. It stores the data in a cookie, which is sent with every request.
If a cookie is not desired, there are other options for saving sessions. For this, however, an extension will be necessary.
Saving with global variables should also work, but is not recommended.
A file or a database can also be used if the data is to be saved across multiple requests from many users.
The data of the post body can be accessed via request.form, while the url parameters of a get request can be queried via request.args.
from flask import Flask
from flask import request, session
app = Flask(__name__)
app.secret_key = b'your secret here'
#app.route('/show_greeting', methods=['GET'])
def show_greeting():
received = request.args
# get the saved message or an empty string if no message is saved
header_message = session.get('header_message', '')
return f"{received['holiday_message']} - {header_message}"
#app.route('/set_greeting', methods=['POST'])
def set_greeting():
posted = request.form
if 'header_message' in posted:
# store the message
session['header_message'] = posted['header_message']
return "Header Message Set"
else:
# clear the message
session.pop('header_message', None)
return "Please Send A Header Message"
Much success in your further steps.
If I understood your problem, you can work with "g" the flask global object.
Check this code, I expect it will fix your issue.
from flask import g # Added
from flask import Flask, request, make_response, Response
app = Flask(__name__)
#app.route('/show_greeting', methods=['GET'])
def show_greeting():
received = request.args
return g.saved_message # Modified
#app.route('/set_greeting', methods=['POST'])
def set_greeting():
posted = request.args
if 'message' in posted:
g.saved_message = posted['request'] # Added
return "Message Set"
else:
return "Please Send A Greeting Message"
if __name__ == '__main__':
app.run()
For returning a 400/500 response to clients in a flask webapp, I've seen the following conventions:
Abort
import flask
def index(arg):
return flask.abort("Invalid request", 400)
Tuple
def index(arg):
return ("Invalid request", 400)
Response
import flask
def index(arg):
return flask.Response("Invalid request", 400)
What are the difference and when would one be preferred?
Related question
Coming from Java/Spring, I am used to defining a custom exception with a status code associated with it and then anytime the application throws that exception, a response with that status code is automatically returned to the user (instead of having to explicitly catch it and return a response as shown above). Is this possible in flask? This is my little wrapped attempt
from flask import Response
class FooException(Exception):
""" Binds optional status code and encapsulates returing Response when error is caught """
def __init__(self, *args, **kwargs):
code = kwargs.pop('code', 400)
Exception.__init__(self)
self.code = code
def as_http_error(self):
return Response(str(self), self.code)
Then to use
try:
something()
catch FooException as ex:
return ex.as_http_error()
The best practice is to create your custom exception classes and then registering with Flask app through error handler decorator. You can raise a custom exception from business logic and then allow the Flask Error Handler to handle any of custom defined exceptions. (Similar way it's done in Spring as well.)
You can use the decorator like below and register your custom exception.
#app.errorhandler(FooException)
def handle_foo_exception(error):
response = jsonify(error.to_dict())
response.status_code = error.status_code
return response
You can read more about it here Implementing API Exceptions
I have a Flask application that returns both HTML pages and JSON responses to API requests. I want to change what an error handler returns based on the content type of the request. If the client requests application/json, I want to return a jsonify response, otherwise I want to return a render_template response. How can I detect what was requested and change the response appropriately?
The current error handlers I have only return an HTML response.
def register_errorhandlers(app):
"""Register error handlers."""
def render_error(error):
"""Render error template."""
# If a HTTPException, pull the `code` attribute; default to 500
error_code = getattr(error, 'code', 500)
return render_template('{0}.html'.format(error_code)), error_code
for errcode in [401, 404, 500]:
app.errorhandler(errcode)(render_error)
Use request.content_type to get the content type the client sent with the request. Use request.accept_mimetypes the get the mimetypes the client indicated it can accept in a response. Use these to determine what to return.
from flask import request, jsonify, render_template
if request.accept_mimetypes.accept_json:
return jsonify(...)
else:
return render_template(...)
I used the after_request decorator to do this and checked the content type:
#app.after_request
def after_request_helper(resp):
if resp.content_type == "text/html":
# If a HTTPException, pull the `code` attribute; default to 500
error_code = getattr(error, 'code', 500)
return render_template('{0}.html'.format(error_code)), error_code
else:
return app.errorhandler(errcode)(render_error)
A more detailed answer:
def wants_json_response():
return request.accept_mimetypes['application/json'] >= \
request.accept_mimetypes['text/html']
The wants_json_response() helper function compares the preference for JSON or HTML selected by the client in their list of preferred formats. If JSON rates higher than HTML, then it is necessary to return a JSON response.
Otherwise, return the original HTML responses based on templates.
For the JSON responses would slightly supplement the function with one condition:
if wants_json_response(): which is what you need. So the answer is in that.
If the condition is true we could write a function that would generate a response:
def api_error_response(status_code, message=None):
payload = {'error': HTTP_STATUS_CODES.get(status_code, 'Unknown error')}
if message:
payload['message'] = message
response = jsonify(payload)
response.status_code = status_code
return response
This function uses the handy HTTP_STATUS_CODES dictionary from Werkzeug (a core dependency of Flask) that provides a short descriptive name for each HTTP status code.
For easier and faster understanding, 'error' is used to represent errors, so you only need to worry about the numeric status code and the optional long description.
The jsonify() function returns a Flask Response object with a default status code of 200, so after the response is created, it is necessary to set the status code to the correct one for the error.
So if we put it all together now it would look like this:
# app/__init__.py
import requests
def register_errorhandlers(app):
from .errors import render_error
for e in [
requests.codes.INTERNAL_SERVER_ERROR,
requests.codes.NOT_FOUND,
requests.codes.UNAUTHORIZED,
]:
app.errorhandler(e)(render_error)
and
# app/errors.py
import requests
from flask import render_template, request, jsonify
from werkzeug.http import HTTP_STATUS_CODES
from .extensions import db
def api_error_response(status_code, message=None):
payload = {'error': HTTP_STATUS_CODES.get(status_code, 'Unknown error')}
if message:
payload['message'] = message
response = jsonify(payload)
response.status_code = status_code
return response
def wants_json_response():
return request.accept_mimetypes['application/json'] >= \
request.accept_mimetypes['text/html']
def render_error(e):
if requests.codes.INTERNAL_SERVER_ERROR == e.code:
db.session.rollback()
if wants_json_response():
return api_error_response(e.code)
else:
return render_template(f'{e.code}.html'), e.code
Additionally
Then they could use the response generation for other cases as well.
The most common error that the API is going to return is going to be
the code 400, which is the error for “bad request”. This is the error
that is used when the client sends a request that has invalid data in it.
In order to generate messages to the function below even easier in these cases, we forward only the required description - message.
def bad_request(message):
return api_error_response(400, message)
I hope this will help in approaching with errors :)
I am currently trying to write some unit tests for my Flask application. In many of my view functions (such as my login), I redirect to a new page. So for example:
#user.route('/login', methods=['GET', 'POST'])
def login():
....
return redirect(url_for('splash.dashboard'))
I'm trying to verify that this redirect happens in my unit tests. Right now, I have:
def test_register(self):
rv = self.create_user('John','Smith','John.Smith#myschool.edu', 'helloworld')
self.assertEquals(rv.status, "200 OK")
# self.assert_redirects(rv, url_for('splash.dashboard'))
This function does make sure that the returned response is 200, but the last line is obviously not valid syntax. How can I assert this? My create_user function is simply:
def create_user(self, firstname, lastname, email, password):
return self.app.post('/user/register', data=dict(
firstname=firstname,
lastname=lastname,
email=email,
password=password
), follow_redirects=True)
Flask has built-in testing hooks and a test client, which works great for functional stuff like this.
from flask import url_for, request
import yourapp
test_client = yourapp.app.test_client()
with test_client:
response = test_client.get(url_for('whatever.url'), follow_redirects=True)
# check that the path changed
assert request.path == url_for('redirected.url')
For older versions of Flask/Werkzeug the request may be available on the response:
from flask import url_for
import yourapp
test_client = yourapp.app.test_client()
response = test_client.get(url_for('whatever.url'), follow_redirects=True)
# check that the path changed
assert response.request.path == url_for('redirected.url')
The docs have more information on how to do this, although FYI if you see "flaskr", that's the name of the test class and not anything in Flask, which confused me the first time I saw it.
Try Flask-Testing
there is api for assertRedirects you can use this
assertRedirects(response, location)
Checks if response is an HTTP redirect to the given location.
Parameters:
response – Flask response
location – relative URL (i.e. without http://localhost)
TEST script:
def test_register(self):
rv = self.create_user('John','Smith','John.Smith#myschool.edu', 'helloworld')
assertRedirects(rv, url of splash.dashboard)
One way is to not follow the redirects (either remove follow_redirects from your request, or explicitly set it to False).
Then, you can simply replace self.assertEquals(rv.status, "200 OK") with:
self.assertEqual(rv.status_code, 302)
self.assertEqual(rv.location, url_for('splash.dashboard', _external=True))
If you want to continue using follow_redirects for some reason, another (slightly brittle) way is to check for some expected dashboard string, like an HTML element ID in the response of rv.data. e.g. self.assertIn('dashboard-id', rv.data)
You can verify the final path after redirects by using Flask test client as a context manager (using the with keyword). It allows keeping the final request context around in order to import the request object containing request path.
from flask import request, url_for
def test_register(self):
with self.app.test_client() as client:
user_data = dict(
firstname='John',
lastname='Smith',
email='John.Smith#myschool.edu',
password='helloworld'
)
res = client.post('/user/register', data=user_data, follow_redirects=True)
assert res.status == '200 OK'
assert request.path == url_for('splash.dashboard')