Django exception line number logging through middleware - python

I am trying to create Django middleware that logs specific information about exceptions that are raised from within views. I am having trouble getting the line number on which the exception was raised.
Specifically, the end of the traceback shows File "<string>", line None. Where is this "<string>" coming from, and why does it show line number None? I get this same result whether I navigate to /view_raises_exception/ through my tests or through a browser window. I'm sure I could probably dig through the traceback in my process_exception() method to get the line number, but I'm wondering why exception.lineno isn't giving me the result I expect.
My setup:
Ubuntu 14.04.3 LTS (Cloud9)
Python 3.4.3
Django 1.9
View:
def exception_view(request):
raise SyntaxError('Custom error message...')
Urls:
from django.conf.urls import url
from . import views
urlpatterns = [
url(r'^view_raises_exception/', views.exception_view),
url(r'^view_raises_no_exceptions/', views.no_exception_view),
]
Middleware file:
from django.http import HttpResponse
class ExceptionLoggingMiddleware(object):
def process_exception(self, request, exception):
name = type(exception).__name__
message = str(exception)
line_no = exception.lineno
print('***INSIDE PROCESS_EXCEPTIONS***')
print('Name:', name)
print('Message:', message)
print('Line Number:', line_no)
# return HttpResponse('Exception handled.')
Server log:
System check identified no issues (0 silenced).
September 09, 2016 - 22:58:40
Django version 1.9, using settings 'middlewares_library.settings'
Starting development server at http://0.0.0.0:8080/
Quit the server with CONTROL-C.
***INSIDE PROCESS_EXCEPTIONS***
Name: SyntaxError
Message: Custom error message...
Line Number: None
Internal Server Error: /view_raises_exception/
Traceback (most recent call last):
File "/home/ubuntu/.virtualenvs/env3/lib/python3.4/site-packages/django/core/handlers/base.py", line 149, in get_response
response = self.process_exception_by_middleware(e, request)
File "/home/ubuntu/.virtualenvs/env3/lib/python3.4/site-packages/django/core/handlers/base.py", line 147, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/ubuntu/workspace/middlewares_library/library/views.py", line 6, in exception_view
raise SyntaxError('Custom error message...')
File "<string>", line None
SyntaxError: Custom error message...
[09/Sep/2016 22:58:43] "GET /view_raises_exception/ HTTP/1.1" 500 68472
Thanks in advance!

Related

Internal Server Error, rather than raised AuthError response from Auth0

I have my flask-restful app.py which contains all of my main functions. I have created a server.py file as instructed from here: https://auth0.com/docs/quickstart/backend/python
In my app.py file from server i import AuthError and requires_auth. I have then put #requires_auth in front of my functions.
When I have a valid jwt, it works perfectly. When the jwt is not valid it fails. Failing is good, because the requests shouldn't work. But the response i get from my api is "Internal Server Error" rather than the detailed response in the raise AuthError section in the server.py file.
I get 2 errors:
Traceback (most recent call last):
File "C:\Users\ME\code\server.py", line 88, in decorated
issuer="https://"+AUTH0_DOMAIN+"/"
File "C:\Users\ME\lib\site-packages\jose\jwt.py", line 150, in decode
options=defaults)
File "C:\Users\ME\lib\site-packages\jose\jwt.py", line 457, in _validate_claims
_validate_exp(claims, leeway=leeway)
File "C:\Users\ME\lib\site-packages\jose\jwt.py", line 299, in _validate_exp
raise ExpiredSignatureError('Signature has expired.')
jose.exceptions.ExpiredSignatureError: Signature has expired.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\ME\lib\site-packages\flask\app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "C:\Users\ME\lib\site-packages\flask\app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "C:\Users\ME\lib\site-packages\flask_restful\__init__.py", line 480, in wrapper
resp = resource(*args, **kwargs)
File "C:\Users\ME\lib\site-packages\flask\views.py", line 88, in view
return self.dispatch_request(*args, **kwargs)
File "C:\Users\ME\lib\site-packages\flask_restful\__init__.py", line 595, in dispatch_request
resp = meth(*args, **kwargs)
File "C:\Users\ME\code\server.py", line 92, in decorated
"description": "token is expired"}, 401)
server.AuthError: ({'code': 'token_expired', 'description': 'token is expired'}, 401)
How do i get the AuthError as the response to the call, rather than just my Internal Server Error?
Thanks!
There is an issue with this specific tutorial in Auth0, it instructs you to include the error handler in auth.py:
#app.errorhandler(AuthError)
def handle_auth_error(ex):
response = jsonify(ex.error)
response.status_code = ex.status_code
return response
Instead, you have to include this handler in your app.py, where you actually use #requires_auth.
Notice that to do so, you need to add relevant imports:
from auth import AuthError
from flask import jsonify
Notice: To be able to import from auth.py you need to add an empty file __init__.py in the same directory.
Try setting app.config[“PROPAGATE_EXCEPTIONS”] = True
Uou could maybe use an errorhandler to explicitly catch those errors and return some explicit json based on them.

How to test get request in Django with a data?

I am now writing tests for my web application in Django. I have an URL 127.0.0.1/home/device/(?P<item>[^/]+). I was testing an invalid URL path.
Here item is a device name in the database. Consider an invalid device and while testing I have given the following code:
response=self.client.get("/health/errorfiles/",{'item':'testdevice_R'})
This give me a 404 response. The same thing I have tried with:
response=self.client.get("/health/errorfiles/testdevice_R/")
But this time, the test runner executes my view function and gives a TypeError since it is not an invalid device.
In both of the methods, which one is correct usage for a get request?
views.py
def showfiles(request,item):
if request.user.is_anonymous(): ## check if the user is valid
return HttpResponseRedirect("/")
db = MySQLdb.connect(host='localhost',user=root,passwd='123mcvis',db='test_db')
s = db.cursor()
username=request.user
s.execute("Select tzone,type from device where user='%s' and device='%s'"%(username,item)
tz=s.fetchone()
timezone.activate(tz[0])
s.close()
return render(request,"Home/showdevices.html")
This is my view function and since the device is invalid, it shows typ error.
class showfile_test(TestCase):
def test_invalid(self):
response=self.client.get("/health/errorfiles/testdevice_R/")
self.assertEqual(response.status_code,404)
Traceback
ERROR: test_invalid (HealthApp.tests2.showfile_test)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/vishnumc/vishnu/project/django/official/version6.2.3/HealthApp/tests2.py", line 162, in test_showfiles_with_invalid_deviceid
response=self.client.get("/health/errorfiles/invaliddevice/")
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/test/client.py", line 529, in get
**extra)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/test/client.py", line 333, in get
return self.generic('GET', path, secure=secure, **r)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/test/client.py", line 409, in generic
return self.request(**r)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/test/client.py", line 494, in request
six.reraise(*exc_info)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner
response = get_response(request)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/vishnumc/vishnu/project/env/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/vishnumc/vishnu/project/django/official/version6.2.3/HealthApp/views.py", line 118, in showfiles
timezone.activate(tz[0]) ##setting current timezone to user's time zone for display timestamps
TypeError: 'NoneType' object has no attribute '__getitem__'
The first example is a GET request to /health/errorfiles/?item=testdevice_R. In the view, you would then find item in request.GET.
The second example is a GET request to /health/errorfiles/testdevice_R/. There is no data in request.GET and Django will pass item to your view since you have a named group (?P<item>[^/]+) in your URL pattern.
You should use the second version, because you want to test the URL pattern r'/home/device/(?P<item>[^/]+)'.
The second version of your test has uncovered problems in your view. You need to fix the view so that it doesn't raise TypeError.
Ideally, you shouldn't be writing raw SQL like that. Django allows you to do something like the following:
from .models import Device
from django.shortcuts import get_object_or_404, render
def showhome(request, item):
device = get_object_or_404(Device, user=request.user, item=item)
if device is not None:
timezone.activate(device.tzone)
return render(request,"Home/showdevices.html", {'device': device})
If you must use raw SQL then don't use string substitution %(username,item). Your current code exposes you to an SQL injection account. You should change it to:
s.execute("Select tzone,type from device where user=%s" and device=%s, (username, item))
Your code then has handle the case where tz is None, to avoid the TypeError.
tz = s.fetchone()
if tz is not None:
timezone.activate(tz[0])
else:
# Decide what to do if no items returned

Google AppEngine with Service Account but invalid Credentials

I have tried to get a group settings with group settings API using Google App Engine 1.7.5 with python 2.5 following a simple example here.
this is my app.yaml:
application: {{APP_ID}}
version: 1
runtime: python
api_version: 1
handlers:
- url: /.*
script: main.py
and my main.py is like following:
def main():
application = webapp.WSGIApplication([('/', TestHandler)])
run_wsgi_app(application)
class TestHandler(webapp.RequestHandler):
"""Handles a request for the query page."""
def get(self):
self.response.out.write(Test())
def Test():
credentials = AppAssertionCredentials(scope='https://www.googleapis.com/auth/apps.groups.settings')
http = credentials.authorize(httplib2.Http(memcache))
service = build('groupssettings', 'v1', http=http)
group = service.groups().get(groupUniqueId='{{GROUP_ID}}').execute()
logging.error(group)
if __name__=="__main__":
main()
and this is the stacktrace telling me invalid credentials!
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\ext\webapp\_webapp25.py", line 714, in __call__
handler.get(*groups)
File "D:\Projets\GS\main.py", line 26, in get
self.response.out.write(Test())
File "D:\Projets\GS\main.py", line 41, in Test
group = service.groups().get(groupUniqueId='{{GROUP_ID}}').execute()
File "D:\Projets\GS\oauth2client\util.py", line 132, in positional_wrapper
return wrapped(*args, **kwargs)
File "D:\Projets\GS\apiclient\http.py", line 723, in execute
raise HttpError(resp, content, uri=self.uri)
HttpError: <HttpError 401 when requesting https://www.googleapis.com/groups/v1/groups/{{GROUP_ID}}?alt=json returned "Invalid Credentials">
INFO 2014-11-21 15:51:45,423 dev_appserver.py:3104] "GET / HTTP/1.1" 500 -
INFO 2014-11-21 15:51:45,611 dev_appserver.py:3104] "GET /favicon.ico HTTP/1.1" 404 -
Have anyone got this error before or have an idea of the root cause?
Have you turned on access to the groups API through the developer console for the project? This is a simple step that is often missed by people trying to get APIs going.
Also, I'd recommend using the decorator patterns for OAuth as described here to simplify the reasoning around authorization flows.
In addition, your app.yaml should specify python27 rather than just python. App Engine runs python27.

Error when using Django Rest Framework's APITestCase

I'm trying to run some tests for my Django-Rest-Framework API but am stuck on an error. When I run the following tests, I get the following errors.
Traceback (most recent call last):
File "C:\Users\Bill\SD\DjangoApps\vidapp\startapp\tests.py", line 21, in test_get_user
response = self.client.get('/user/1/')
File "C:\Anaconda\lib\site-packages\django\test\client.py", line 473, in get
response = super(Client, self).get(path, data=data, **extra)
File "C:\Anaconda\lib\site-packages\django\test\client.py", line 280, in get
return self.request(**r)
File "C:\Anaconda\lib\site-packages\rest_framework\test.py", line 143, in request
return super(APIClient, self).request(**kwargs)
File "C:\Anaconda\lib\site-packages\rest_framework\test.py", line 95, in request
request = super(APIRequestFactory, self).request(**kwargs)
File "C:\Anaconda\lib\site-packages\django\test\client.py", line 444, in request
six.reraise(*exc_info)
File "C:\Anaconda\lib\site-packages\django\core\handlers\base.py", line 114, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
TypeError: __init__() takes exactly 1 argument (2 given)
Test Cases:
class UserTestCase(APITestCase):
def setUp(self):
helper.reset_test_db()
def test_get_user(self):
response = self.client.get('/user/1/')
print response.content
self.assertEqual(response.data, {'fname':'Generic','lname':'Name','token':'token1'})
URL Config:
url(r'^user/new/$', 'startapp.views.new_user'),
url(r'^user/1/$', 'startapp.views.get_1'),
Views:
class get_1(APIView):
def get(self, request):
user = db_models.User.objects.get(pk=1)
if(user is not None):
serial_user = serial.UserSerializer(user)
return Response(serial_user.data)
else:
return Response(status.HTTP_404_NOT_FOUND)
I know the view itself works because I tested that separately. The data is definitely present since helper.reset_test_db() puts it there (I know I should be using fixtures but this is for testing so I went with the simple route). The same error occurs for POST and other commands or when I use Django's TestCase instead of APITestCase. While this is my first time using Django's TestCase, I read both the Django and Django rest documents but can't seem to figure out this issue.
The view in your case is a class-based view.
So you have to add it to the urlconfig with as_view:
url(r'^user/1/$', startapp.views.get_1.as_view()),

Django how to redirect python import error to custom error page

I need to redirect django 'import error' to a custom error page.All other errors except import error is redrecting to custom error page
Already in DEBUG= False
Error displayed in browser is
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/django/core/servers/basehttp.py", line 283, in run
self.result = application(self.environ, self.start_response)
File "/usr/lib/python2.7/dist-packages/django/core/handlers/wsgi.py", line 272, in __call__
response = self.get_response(request)
File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 169, in get_response
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 214, in handle_uncaught_exception
if resolver.urlconf_module is None:
File "/usr/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 274, in _get_urlconf_module
self._urlconf_module = import_module(self.urlconf_name)
File "/usr/lib/python2.7/dist-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/project/backup/project/hg/rel0129/hg/src/customer_/../customer_/hg/urls.py", line 4, in <module>
from lostpassword.views import recaptcha, reset
File "/project/backup/project/hg/rel0129/hg/../customer_/hg/lostpassword/views.py", line 12, in <module>
from hg.utils.common import __render, __redirect
hg/utils/common.py", line 15, in <module>
from urllibwrapper import URLClient, ClientResponse
ImportError: No module named urlwrapper
The error needs to be redirect to custom error page
Thanks in advance
Django will already show you an error page when there is an import error; what error page is shown depends on the value of DEBUG.
If DEBUG is set to True, you are shown the traceback and some debug information.
If DEBUG is False, Django shows you the default 500 error page. You can set a custom 500 error response by either providing a 500.html page in the root of your templates directory, or by providing a handler500 string in your root URLConf.
See the 500 error view documentation.
However, an import error in your project can prevent Django from being configured correctly, and the custom 500 error handler won't be found. There is such a thing as 'too broken', and an early import exception is such a situation. The exception then bypasses the Django error handling altogether and drops down to the WSGI server level instead.
In such a situation, provided DEBUG is set to False, the user is shown the default 500 error message, a simple text message:
A server error occurred. Please contact the administrator.
Note that older Django releases (prior to 1.4) the built-in server can still show the traceback of import exceptions even when debug mode is off, because these older versions use their own simple HTTP server.
You can upgrade your Django server to 1.4 or newer in that case, and / or not use the built-in server. Use a proper production deployment instead.
To raise a custom error page you need to define a template and then add the following into your urls.py:
handler500 = 'mysite.views.my_custom_error_view'
To see more (including the traceback - NOT recommended if DEBUG = False) have a look at this answer to Django 500 message in custom template.
More information in the django docs.
This server_error view should suffice for 99% of Web applications, but if you want to override the view, you can specify handler500 in your root URLconf, like so:
handler500 = 'mysite.views.my_custom_error_view'
Behind the scenes, Django determines the 500 view by looking for handler500 in your root URLconf, and falling back to django.views.defaults.server_error if you did not define one.
One thing to note about 500 views:
If DEBUG is set to True (in your settings module), then your 500 view will never be used, and the traceback will be displayed instead, with some debug information.

Categories