I'm currently refactoring a larger code base which uses Flask. I see a lot of duplication around parameter checks. It's similar to this:
#api.route("/foo/bar/<string:content_id>")
def get_foo(content_id):
try:
content = get_content(content_id)
except:
return jsonify({"status": 404, "error": "not found"}), 404
...
#api.route("/foo/bar/<string:content_id>/update")
def update_foo(content_id):
try: # This is exactly the same!
content = get_content(content_id) # It's just copy-pasted
except: # and this way a lot of duplication
return jsonify({"status": 404, "error": "not found"}), 404 # is added! I want to reduce that.
...
The checks are longer and there are way more of them, but I hope this illustrates the issue well.
Is it possible to factor this check out? Maybe by using a decorator? How would that look like?
What I want
I would love to have something like this:
def get_content_validator():
try:
content = get_content(content_id)
except:
return jsonify({"status": 404, "error": "not found"}), 404
return content
#api.route("/foo/bar/<get_content_validator(string:content_id)>")
def get_foo(content_id):
...
#api.route("/foo/bar/<get_content_validator(string:content_id)>/update")
def update_foo(content):
...
The issue with that is that in one case the value is returned and the normal execution of get_foo / update_foo is done and in other cases the function body of get_foo / update_foo is not executed. This could be possible by checking the return type. If it's a "response" return type, the response is given. In all other cases it's just returned. But that sounds as if it could result in other issues.
I'd just make a decorator:
from functools import wraps
def validated_content(func):
#wraps(func)
def wrapper(content_id, *args, **kwargs):
try:
content = get_content(content_id)
except:
return jsonify(...)
return func(content, *args, **kwargs)
return wrapper
And then your handlers become:
#api.route("...")
#validated_content
def get_foo(content):
# do stuff with content
Related
I want to design an error handler function in view and use it in different main functions in view.
This error handler function will gather required error info, log the error to the system, and render those info to an error handler page.
Codes in view.py look like:
def error_manage(request, error_title, error_detail):
error_time = datetime.now()
info_table = {'error_title': error_title,
'error_detail': error_detail,
'error_time ': error_time ,
}
logger.error(str(error_detail))
return render(request,'home/error_manage.html', {'error_title': error_title,
'info_table ': display_table})
def my_page_1(request):
try:
var1 = my_calculation_func()
except Exception as ex:
error_manage(request, "Unexpected Error Happened.", ex)
var2 = my_calculation_func2()
var3 = my_calculation_func3()
return render(request,'home/my_page_1.html', {'var1': var1 ,'var2': var2 ,'var3': var3})
The problem here is that when the error occurs in var1, the program will jump into error_manage but will not stop in the return of this function and also will not successfully render into error_manage.html.
it will later jump out of this function and continue run var2 and var3 until it meets the final return in my_page_1().
I know I can simply put the return statement in error_manage to my_page_1 under Exception.
But this error handler will be used in many other functions in view.py and I don't want to repeat it over and over again.
Is there a good solution for this problem? Thank you.
Here's an example of the decorator approach:
class MyError(Exception):
pass
def error_handler(func):
def decorated(*args, **kwargs):
try:
return func(*args, **kwargs)
except MyError as e:
return f"Oh no! {e} happened!"
return decorated
#error_handler
def my_view(request):
if "error" in request:
raise MyError("request was rubbish")
return "Great success!"
for r in ["Fine", "no problem", "An error happened"]:
print(my_view(r))
So running this, we get
Great success!
Great success!
Oh no! request was rubbish happened!
So essentially we use the decorator to catch and handle MyError exceptions, other exceptions will not be handled and will bubble up out of the view as normal to be handled by django's [default] view error handling.
You should make a custom middleware to handle exceptions. See Writing your own middleware [Django docs]:
class MyMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
response = self.get_response(request)
return response
def process_exception(request, exception):
error_time = datetime.now()
info_table = {
'error_detail': exception,
'error_time ': error_time ,
}
logger.error(str(exception))
return render(request,'home/error_manage.html', {'info_table ': display_table})
Now you need to add this middleware to the MIDDLEWARE list in the settings:
MIDDLEWARE = [
# Default middlewares
'path.to.MyMiddleware',
]
Now in your view you should simply not catch the Exception and let the middleware handle it.
Note: With this your approach of passing error_title won't work but you can try to re-raise a new exception from the old one if you
need some customization: raise Exception("Unexpected Error Happened.") from ex
I have a class that handles the API calls to a server. Certain methods within the class require the user to be logged in. Since it is possible for the session to run out, I need some functionality that re-logins the user once the session timed out. My idea was to use a decorator. If I try it like this
class Outer_Class():
class login_required():
def __init__(self, decorated_func):
self.decorated_func = decorated_func
def __call__(self, *args, **kwargs):
try:
response = self.decorated_func(*args, **kwargs)
except:
print('Session probably timed out. Logging in again ...')
args[0]._login()
response = self.decorated_func(*args, **kwargs)
return response
def __init__(self):
self.logged_in = False
self.url = 'something'
self._login()
def _login(self):
print(f'Logging in on {self.url}!')
self.logged_in = True
#this method requires the user to be logged in
#login_required
def do_something(self, param_1):
print('Doing something important with param_1')
if (): #..this fails
raise Exception()
I get an error. AttributeError: 'str' object has no attribute '_login'
Why do I not get a reference to the Outer_Class-instance handed over via *args? Is there another way to get a reference to the instance?
Found this answer How to get instance given a method of the instance? , but the decorated_function doesn't seem to have a reference to it's own instance.
It works fine, when Im using a decorator function outside of the class. This solves the problem, but I like to know, if it is possible to solve the this way.
The problem is that the magic of passing the object as the first hidden parameter only works for a non static method. As your decorator returns a custom callable object which is not a function, it never receives the calling object which is just lost in the call. So when you try to call the decorated function, you only pass it param_1 in the position of self. You get a first exception do_something() missing 1 required positional argument: 'param_1', fall into the except block and get your error.
You can still tie the decorator to the class, but it must be a function to have self magic work:
class Outer_Class():
def login_required(decorated_func):
def inner(self, *args, **kwargs):
print("decorated called")
try:
response = decorated_func(self, *args, **kwargs)
except:
print('Session probably timed out. Logging in again ...')
self._login()
response = decorated_func(self, *args, **kwargs)
return response
return inner
...
#this method requires the user to be logged in
#login_required
def do_something(self, param_1):
print('Doing something important with param_1', param_1)
if (False): #..this fails
raise Exception()
You can then successfully do:
>>> a = Outer_Class()
Logging in on something!
>>> a.do_something("foo")
decorated called
Doing something important with param_1
You have the command of
args[0]._login()
in the except. Since args[0] is a string and it doesn't have a _login method, you get the error message mentioned in the question.
Is it possible to develop a decorator to determine if a function successfully completed without crashing ? Something like the decorator below but it must detect if the function ran successfully.
def functionStartDecorator():
print("Function has started")
def decoratorWrapper(functionToBeDecorated):
def wrapper(*args, **kwargs):
#Todo log function has run
return functionToBeDecorated(*args, **kwargs)
return wrapper
return decoratorWrapper
As said in comments, simplest would be to wrap your function in a try/except. If your function returns nothing and simply operate by side-effect, it should be straightforward to have the decorated function return the status of that function run:
def tryfn(f):
def decorated(*args, **kwargs):
try:
f(*args, **kwargs)
return 'Ran successfully'
except Exception as e:
return 'Error: {}'.format(e)
return decorated
You can then tweak the exact return type of decorated: maybe return a boolean, maybe append the status of the function to the return of f, maybe log things... The core principle will probably still be around a try/except within the decorated function. For instance, if you want to return both your return value (or None if it failed) and whether this was a success:
def tryfn(f):
def decorated(*args, **kwargs):
try:
res = f(*args, **kwargs)
return True, res
except:
return False, None
return decorated
Note that in this case, you're losing the information about the exact error, so you might want to expand the return to include success, failure, error, etc... that becomes a trade-off between convenience and completeness, which will depend on your exact problem.
I did something like this. Do you see any issues with this code ?
def functionStartDecorator():
def decoratorWrapper(functionToBeDecorated):
def wrapper(*args, **kwargs):
try:
print("Function has started")
result=functionToBeDecorated(*args, **kwargs)
print("Function has complete")
return result
except:
print ("Function failed")
return wrapper
return decoratorWrapper
#functionStartDecorator()
def simplefunc():
return "somthing"
This pattern is from the django docs:
class SimpleTest(unittest.TestCase):
def test_details(self):
client = Client()
response = client.get('/customer/details/')
self.assertEqual(response.status_code, 200)
From: https://docs.djangoproject.com/en/1.8/topics/testing/tools/#default-test-client
If the test fails, the error message does not help very much. For example if the status_code is 302, then I see 302 != 200.
The question is now: Where does the wrong HTTPResponse get created?
I would like to see the stacktrace of the interpreter where the wrong HTTPResponse object get created.
I read the docs for the assertions of django but found no matching method.
Update
This is a general question: How to see the wanted information immediately if the assertion fails? Since these assertions (self.assertEqual(response.status_code, 200)) are common, I don't want to start debugging.
Update 2016
I had the same idea again, found the current answer not 100% easy. I wrote a new answer, which has a simple to use solution (subclass of django web client): Django: assertEqual(response.status_code, 200): I want to see useful stack of functions calls
I think it could be achieved by creating a TestCase subclass that monkeypatches django.http.response.HttpResponseBase.__init__() to record a stack trace and store it on the Response object, then writing an assertResponseCodeEquals(response, status_code=200) method that prints the stored stack trace on failure to show where the Response was created.
I could actually really use a solution for this myself, and might look at implementing it.
Update:
Here's a v1 implementation, which could use some refinement (eg only printing relevant lines of the stack trace).
import mock
from traceback import extract_stack, format_list
from django.test.testcases import TestCase
from django.http.response import HttpResponseBase
orig_response_init = HttpResponseBase.__init__
def new_response_init(self, *args, **kwargs):
orig_response_init(self, *args, **kwargs)
self._init_stack = extract_stack()
class ResponseTracebackTestCase(TestCase):
#classmethod
def setUpClass(cls):
cls.patcher = mock.patch.object(HttpResponseBase, '__init__', new_response_init)
cls.patcher.start()
#classmethod
def tearDownClass(cls):
cls.patcher.stop()
def assertResponseCodeEquals(self, response, status_code=200):
self.assertEqual(response.status_code, status_code,
"Response code was '%s', expected '%s'" % (
response.status_code, status_code,
) + '\n' + ''.join(format_list(response._init_stack))
)
class MyTestCase(ResponseTracebackTestCase):
def test_index_page_returns_200(self):
response = self.client.get('/')
self.assertResponseCodeEquals(response, 200)
How do I see the traceback if the assertion fails without debugging
If the assertion fails, there isn't a traceback. The client.get() hasn't failed, it just returned a different response than you were expecting.
You could use a pdb to step through the client.get() call, and see why it is returning the unexpected response.
Maybe this could work for you:
class SimpleTest(unittest.TestCase):
#override_settings(DEBUG=True)
def test_details(self):
client = Client()
response = client.get('/customer/details/')
self.assertEqual(response.status_code, 200, response.content)
Using #override_settings to have DEBUG=True will have the stacktrace just as if you were running an instance in DEBUG mode.
Secondly, in order to provide the content of the response, you need to either print it or log it using the logging module, or add it as your message for the assert method. Without a debugger, once you assert, it is too late to print anything useful (usually).
You can also configure logging and add a handler to save messages in memory, and print all of that; either in a custom assert method or in a custom test runner.
I was inspired by the solution that #Fush proposed but my code was using assertRedirects which is a longer method and was a bit too much code to duplicate without feeling bad about myself.
I spent a bit of time figuring out how I could just call super() for each assert and came up with this. I've included 2 example assert methods - they would all basically be the same. Maybe some clever soul can think of some metaclass magic that does this for all methods that take 'response' as their first argument.
from bs4 import BeautifulSoup
from django.test.testcases import TestCase
class ResponseTracebackTestCase(TestCase):
def _display_response_traceback(self, e, content):
soup = BeautifulSoup(content)
assert False, u'\n\nOriginal Traceback:\n\n{}'.format(
soup.find("textarea", {"id": "traceback_area"}).text
)
def assertRedirects(self, response, *args, **kwargs):
try:
super(ResponseTracebackTestCase, self).assertRedirects(response, *args, **kwargs)
except Exception as e:
self._display_response_traceback(e, response.content)
def assertContains(self, response, *args, **kwargs):
try:
super(ResponseTracebackTestCase, self).assertContains(response, *args, **kwargs)
except Exception as e:
self._display_response_traceback(e, response.content)
I subclassed the django web client, to get this:
Usage
def test_foo(self):
...
MyClient().get(url, assert_status=200)
Implementation
from django.test import Client
class MyClient(Client):
def generic(self, method, path, data='',
content_type='application/octet-stream', secure=False,
assert_status=None,
**extra):
if assert_status:
return self.assert_status(assert_status, super(MyClient, self).generic, method, path, data, content_type, secure, **extra)
return super(MyClient, self).generic(method, path, data, content_type, secure, **extra)
#classmethod
def assert_status(cls, status_code, method_pointer, *args, **kwargs):
assert hasattr(method_pointer, '__call__'), 'Method pointer needed, looks like the result of a method call: %r' % (method_pointer)
def new_init(self, *args, **kwargs):
orig_response_init(self, *args, **kwargs)
if not status_code == self.status_code:
raise HTTPResponseStatusCodeAssertionError('should=%s is=%s' % (status_code, self.status_code))
def reraise_exception(*args, **kwargs):
raise
with mock.patch('django.core.handlers.base.BaseHandler.handle_uncaught_exception', reraise_exception):
with mock.patch.object(HttpResponseBase, '__init__', new_init):
return method_pointer(*args, **kwargs)
Conclusion
This results in a long exception if a http response with a wrong status code was created. If you are not afraid of long exceptions, you see very fast the root of the problem. That's what I want, I am happy.
Credits
This was based on other answers of this question.
I am currently using the following to raise a HTTP bad request:
raise tornado.web.HTTPError(400)
which returns a html output:
<html><title>400: Bad Request</title><body>400: Bad Request</body></html>
Is it possible to return just the HTTP response code with a custom body?
You may simulate RequestHandler.send_error method:
class MyHandler(tornado.web.RequestHandler):
def get(self):
self.clear()
self.set_status(400)
self.finish("<html><body>My custom body</body></html>")
Tornado calls RequestHandler.write_error to output errors, so an alternative to VisioN's approach would be override it as suggested by the Tornado docs. The advantage to this approach is that it will allow you to raise HTTPError as before.
The source for RequestHandler.write_error is here. Below you can see an example of a simple modification of write_error that will change the set the status code and change the output if you provide a reason in kwargs.
def write_error(self, status_code, **kwargs):
if self.settings.get("serve_traceback") and "exc_info" in kwargs:
# in debug mode, try to send a traceback
self.set_header('Content-Type', 'text/plain')
for line in traceback.format_exception(*kwargs["exc_info"]):
self.write(line)
self.finish()
else:
self.set_status(status_code)
if kwargs['reason']:
self.finish(kwargs['reason'])
else:
self.finish("<html><title>%(code)d: %(message)s</title>"
"<body>%(code)d: %(message)s</body></html>" % {
"code": status_code,
"message": self._reason,
})
It's better to use the standard interface and define your custom message on the HTTPError.
raise tornado.web.HTTPError(status_code=code, log_message=custom_msg)
You can then parse the error in your RequestHandler and check for the message:
class CustomHandler(tornado.web.RequestHandler):
def write_error(self, status_code, **kwargs):
err_cls, err, traceback = kwargs['exc_info']
if err.log_message and err.log_message.startswith(custom_msg):
self.write("<html><body><h1>Here be dragons</h1></body></html>")
def write_error(self, status_code, **kwargs):
#Function to display custom error page defined in the handler.
#Over written from base handler.
data = {}
data['code'] = status_code
data['message'] = httplib.responses[status_code]
# your other conditions here to create data dict
self.write(TEMPLATES.load('error.html').generate(data=data))
when ever self.send_error() call is initiated write_error() function is called by the request handler. So you can create your custom error data dict here and render it to your custom error page.
http.responses[status_code] returns the error code text like "page not found" based on the status code.
Also you can override get_error_html method in your handler.
For example:
import tornado.web
class CustomHandler(tornado.web.RequestHandler):
def get_error_html(self, status_code, **kwargs);
self.write("<html><body><h1>404!</h1></body></html>")
...
def get(self):
...
This exchange clarifies some of the approaches suggested here, and discounts the reason keyword (which I was thinking about trying).
Q: (by mrtn)
"I want to use raise tornado.web.HTTPError(400, reason='invalid request') to pass a custom reason to the error response, and I hope to do this by overriding the write_error (self, status_code, **kwargs) method.
"But it seems that I can only access self._reason inside write_error, which is not what I want. I also tried kwargs['reason'] but that does not exist."
A: (by Tornado lead developer #bendarnell)
"The exception that exposed the error is available to write_error as an exc_info triple in the keyword arguments. You can access the reason field with something like this:
if "exc_info" in kwargs:
e = kwargs["exc_info"][1]
if isinstance(e, tornado.web.HTTPError):
reason = e.reason
"But note that the reason field is essentially deprecated (it is not present in HTTP/2), so it's probably not the best way to do whatever you're trying to do here (HTTPError's log_message field is a little better, but still not ideal). Just raise your own exception instead of using HTTPError; your write_error override can use self.set_status(400) when it sees the right kind of exception."
For json error response i use follow template:
Request handler:
import json
from tornado.web import RequestHandler
from src.lib.errors import HTTPBadRequest
class JsonHandler(RequestHandler):
def prepare(self):
content_type = ''
if "Content-Type" in self.request.headers:
content_type = self.request.headers['Content-Type']
if content_type == 'application/json':
try:
self.request.body = json.loads(self.request.body.decode('utf-8'))
except ValueError:
raise HTTPBadRequest
def write_error(self, *args, **kwargs):
err_cls, err, traceback = kwargs['exc_info']
self.set_status(err.status_code)
if err.description:
self.write_json(err.description)
self.finish()
def set_default_headers(self):
self.set_header('Content-Type', 'application/json')
def write_json(self, response):
self.write(json.dumps(response))
Errors handler:
from typing import Any
from tornado import httputil
class BaseHTTPError(Exception):
def __init__(
self, status_code: int = 500, description=None, *args: Any, **kwargs: Any
) -> None:
if description is None:
description = {}
self.status_code = status_code
self.description = description
self.args = args
self.kwargs = kwargs
def __str__(self) -> str:
message = "HTTP %d: %s" % (
self.status_code,
httputil.responses.get(self.status_code, "Unknown"),
)
return message
class HTTPBadRequest(BaseHTTPError):
def __init__(self, *args, **kwargs):
super().__init__(status_code=400, description={"error": "Bad Request"}, *args, **kwargs)