I'm running Django 1.0 and I'm close to deploying my app. As such, I'll be changing the DEBUG setting to False.
With that being said, I'd still like to include the stacktrace on my 500.html page when errors occur. By doing so, users can copy-and-paste the errors and easily email them to the developers.
Any thoughts on how best to approach this issue?
Automatically log your 500s, that way:
You know when they occur.
You don't need to rely on users sending you stacktraces.
Joel recommends even going so far as automatically creating tickets in your bug tracker when your application experiences a failure. Personally, I create a (private) RSS feed with the stacktraces, urls, etc. that the developers can subscribe to.
Showing stack traces to your users on the other hand could possibly leak information that malicious users could use to attack your site. Overly detailed error messages are one of the classic stepping stones to SQL injection attacks.
Edit (added code sample to capture traceback):
You can get the exception information from the sys.exc_info call. While formatting the traceback for display comes from the traceback module:
import traceback
import sys
try:
raise Exception("Message")
except:
type, value, tb = sys.exc_info()
print >> sys.stderr, type.__name__, ":", value
print >> sys.stderr, '\n'.join(traceback.format_tb(tb))
Prints:
Exception : Message
File "exception.py", line 5, in <module>
raise Exception("Message")
As #zacherates says, you really don't want to display a stacktrace to your users. The easiest approach to this problem is what Django does by default if you have yourself and your developers listed in the ADMINS setting with email addresses; it sends an email to everyone in that list with the full stack trace (and more) everytime there is a 500 error with DEBUG = False.
If we want to show exceptions which are generated , on ur template(500.html) then we could write your own 500 view, grabbing the exception and passing it to your 500 template.
Steps:
# In views.py:
import sys,traceback
def custom_500(request):
t = loader.get_template('500.html')
print sys.exc_info()
type, value, tb = sys.exc_info()
return HttpResponseServerError(t.render(Context({
'exception_value': value,
'value':type,
'tb':traceback.format_exception(type, value, tb)
},RequestContext(request))))
# In Main urls.py:
from django.conf.urls.defaults import *
handler500 = 'project.web.services.views.custom_500'
# In Template(500.html):
{{ exception_value }}{{value}}{{tb}}
more about it here: https://docs.djangoproject.com/en/dev/topics/http/views/#the-500-server-error-view
You could call sys.exc_info() in a custom exception handler. But I don't recommend that. Django can send you emails for exceptions.
I know this is an old question, but these days I would recommend using a service such as Sentry to capture your errors.
On Django, the steps to set this up are incredibly simple. From the docs:
Install Raven using pip install raven
Add 'raven.contrib.django.raven_compat' to your settings.INSTALLED_APPS.
Add RAVEN_CONFIG = {"dsn": YOUR_SENTRY_DSN} to your settings.
Then, on your 500 page (defined in handler500), pass the request.sentry.id to the template and your users can reference the specific error without any of your internals being exposed.
Related
Someone recently said to me "In Python it is better to try and ask for forgiveness later instead of begging for permission" which I found quite funny but also relates to my question
I'm creating a personal assistant of sorts called Ada and was being pedantic about performance. From what I gather, using a try statement is faster when it works then checking and then executing. E.G: (The second one being slower if the directory DOESNT EXIST???)
import os
try:
os.makedirs("Test")
except FileExistsError:
pass
# VS
if not os.path.exists("Test"):
os.makedirs("Test")
So when creating coding, you need to weigh up whats more likely. In my example it would be the file is more likely to NOT exist so I should use a try block which yields better performance then an If-Else
I was wondering if this is of any benefit to try (pun intended) this method of the default If-Else?
P.S (This question isn't a duplicate of Python if vs try-except since that's not specifying about comparing the probabilities of the try: code block)
My current code if anyone's interested: (Created a folder in AppData called Ada where a Config.ini file is made)
import os
AppDataDirectory = os.getenv("AppData")
AdaDirectory = AppDataDirectory + "\Ada"
ConfigFile = AdaDirectory + "\CONFIG.ini"
try:
File = open(ConfigFile, "r")
except FileNotFoundError:
try:
os.makedirs(AdaDirectory)
except FileExistsError:
print("Config File Missing")
# Setup Config File
The performance of try vs. if-else depends.
Generally, it is considered more pythonic to use try blocks because:
You do an action
If it fails then do something else
The benefit here is that if the action fails you can specify an exact error and react accordingly or keep the exception at the Base Exception class.
Using if-else isn't necessarily non-pythonic but with them python (this gets longer if there are elif statements),
Checks the boolean value of a statement
If boolean value is desired, then perform another action
If boolean value is not desired then do something else
Ultimately try blocks are recommended because you do something and specify an exact response if the action fails as opposed to setting up a variety of conditions.
A real time use case example of try-catch && if-else with Django:
I created a custom user model in account app. So if i try to import it from default user model like.
from django.contrib.auth.models import User
I will get a error like:
AttributeError: Manager isn't available; 'auth.User' has been swapped for 'account.User
and this will hang the process.
#from account.models import User # this is the correct import
if User.objects.filter(username="milon").exits():
print(f"Super User with username {env('SUPER_USER_NAME')} is already exit!!")
else:
superuser = User.objects.create_superuser(
username="milon", email="alim.abdul#gmail.com", password="******")
superuser.save()
But if I use try-catch block.
try:
superuser = User.objects.create_superuser(
username="milon"), email="alim.abdul#gmail.com", password="******")
superuser.save()
except IntegrityError:
print(f"Super User with username {env('SUPER_USER_NAME')} is already exit!!")
except Exception as e:
print(e)
We will get bellow error and this will not hang the process.
Manager isn't available; 'auth.User' has been swapped for 'account.User'
This example was tried in docker where i was used django & react separate project in docker container. For if-else docker was hanged the other processes. But using try-catch others process was not hanged in docker container.
I am overwriting connection.notices of psycopg2.
My goal is to send a notice from a PostgreSQL trigger to my application.
In python I want to write the current stacktrace to a log file to see which python code is triggering the trigger in the database.
It works, but unfortunately I can't extract the whole stacktrace.
I only get the lines below psycopg, not the above (lines of the callers).
Here is my code:
# Overwriting connetion.notices via Django
class MyAppConfig(AppConfig):
def ready(self):
connection_created.connect(connection_created_check_for_notice_in_connection)
class ConnectionNoticeList(object):
def append(self, message):
if not 'some_magic_of_db_trigger' in message:
return
logger.warn('%s %s' % (message, ''.join(traceback.format_stack())))
def connection_created_check_for_notice_in_connection(sender, connection, **kwargs):
connection.connection.notices=ConnectionNoticeList()
I see this in the logs:
'NOTICE: some_magic_of_db_trigger: 17909
File "/snap/pycharm-community/128/helpers/pycharm/_jb_pytest_runner....ork/foo/apps.py", line 47, in append
logger.warn(\'%s %s\' % (message, \'\'.join(traceback.format_stack())))
'
traceback.format_stack() inside ConnectionNoticeList.append() extracts not the callers.
Is there a way to get the lines of the upper methods?
(This is related to an older question: Log Stacktrace of current Python Interpreter via PostgreSQL trigger)
Above code works. I see the whole traceback.
I don't know why the traceback was cut in PyCharm. In production I could see the whole traceback and I could find the broken code which modified the data in way which should not happen.
Many thanks to Daniele Varrazzo who provided the hint to overwrite connection.notices:
https://www.postgresql.org/message-id/CA%2Bmi_8a5rij_5xchPkYPFS3moAtw9i6TrwKndynWfbJ%3DkFA4fg%40mail.gmail.com
Is there a way to show the errors that occurred on the server during a StaticLiveServerTestCase directly in the test feedback? That being, when some server function call errors and page just doesn't show up, the test execution by default has no knowledge of the server error. Is there someway to pass that output onto the testing thread?
Preferably the these errors would show up in the same place that errors directly in the test code execution show up. If this isn't (easily) possible though, what's the next best way to quickly see those server errors?
Thanks!
Code (as requested):
class TestFunctionalVisitor(StaticLiveServerTestCase):
def setUp(self):
self.browser = webdriver.Firefox()
def tearDown(self):
self.browser.quit()
def test_visitor(self):
self.browser.get(self.live_server_url)
self.assertEqual(self.browser.title, "Something")
...
class Home(TemplateView):
template_name = 'home.html'
def get_context_data(self):
context = {}
MyModel = None
context['my_models'] = MyModel.objects.all()
return context
This has been significantly altered to make it simple and short. But when MyModel is None and tries to call objects.all() the server has a server 500 error, but all I get is the "Something" not in self.browser.title error from the test output, when I'd like to see the NoneType has no... error in the test output.
To see the errors immediately, run the test in DEBUG mode:
from django.test.utils import override_settings
#override_settings(DEBUG=True)
class DjkSampleTestCase(StaticLiveServerTestCase):
# fixtures = ['club_app_phase01_2017-01-09_13-30-19-169537.json']
reset_sequences = True
But one should also configure logging of server-side errors, either via custom django.core.handlers.base.BaseHandler class handle_uncaught_exception() method implementation or via sentry.
I use to override the default logger using:
import logging
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s -%(filename)s:%(lineno)d - %(message)s')
This will display stderr on your terminal. You can even do:
logging.debug('My var %s', var)
I only do this for debugging, if you want to use logging for non-debugging things I'd suggest to create custom loggers.
More details about logging:
https://docs.djangoproject.com/en/1.10/topics/logging/
https://docs.python.org/3/library/logging.html
This is exactly why it is recommended to have much more unit tests than integration and UI/end-to-end. Aside from other things, the latter don't provide you with a specific feedback and you often need more time debugging and investigating why a UI test failed. On the other hand, when a unit test fails, it is usually a failure or an exception pointing you to a specific line in the code - you get the "What went wrong" answer right away.
In other words, the point is: Cover this particular problem with unit tests leaving your UI test as is.
To help you gather more information about the "Something" not in self.browser.title failure, turn the logging on and log as much details as possible. You may also use the built-in Error Reporting and, for instance, let Django send you an email on 500 error. In other words, collect all the details and troubleshoot the failure manually.
I have just started to use peewee in python. But as i am saving table data using .save() function. There is an error in the line. and control does not go to the next line.
Just wanted to know how can know what the error is. Although i have narrowed down to the line as below
try:
with database.transaction():
driver = Driver()
driver.person = person
driver.qualification = form.getvalue('qualification')
driver.number = form.getvalue('phone')
driver.license = form.getvalue('issu')
driver.audited_by = 0
print "this line prints"
driver.save()
print "this one does not print"
print "Success"
except:
print "Error"
I have used print statements i was able to figure out the error in in the line driver.save(). But how to check what exactly is the error?
Peewee logs queries at the DEBUG level to the peewee namespace, so you just have to configure logging as desired. Per the docs:
import logging
logger = logging.getLogger('peewee')
logger.addHandler(logging.StreamHandler())
logger.setLevel(logging.DEBUG)
This is specified in the peewee documentation here.
In the future, you should also include the traceback when you're asking for help debugging an error. The traceback tells you, as best as it can, exactly what went wrong.
If you want to do some debugging, you can check out pdb (or ipdb if you use iPython):
https://docs.python.org/2/library/pdb.html
In my python script I would like to trap a "Data truncated for column 'xxx'" warning durnig my query using MySql.
I saw some posts suggesting the code below, but it doesn' work.
Do you know if some specific module must be imported or if some option/flag should be called before using this code?
Thanks all
Afeg
import MySQLdb
try:
cursor.execute(some_statement)
# code steps always here: No Warning is trapped
# by the code below
except MySQLdb.Warning, e:
# handle warnings, if the cursor you're using raises them
except Warning, e:
# handle warnings, if the cursor you're using raises them
Warnings are just that: warnings. They get reported to (usually) stderr, but nothing else is done. You can't catch them like exceptions because they aren't being raised.
You can, however, configure what to do with warnings, and turn them off or turn them into exceptions, using the warnings module. For instance, warnings.filterwarnings('error', category=MySQLdb.Warning) to turn MySQLdb.Warning warnings into exceptions (in which case they would be caught using your try/except) or 'ignore' to not show them at all. You can (and probably should) have more fine-grained filters than just the category.
Raise MySQL Warnings as errors:
import warnings, MySQLdb
warnings.filterwarnings('error', category=MySQLdb.Warning)
To ignore instead of raising an error, replace "error" with "ignore".
Handle them in a try-except block like:
try:
# a MySQL DB operation that raises a warning
# for example: a data truncated warning
except Warning as a_warning:
# do something here
I would try first to set the sql_mode. You can do this at the session level. If you execute a:
SET ##sql_mode:=TRADITIONAL;
(you could also set it at the server level, but you need access to the server to do that. and, that setting an still be overridden at the session level, so the bottom line is, always set it at the session level, immediately after establishing the connection)
then many things that are normally warnings become errors. I don't know how those manifest themselves at the python level, but the clear advantage is that the changes are not stored in the database. See: http://dev.mysql.com/doc/refman/5.1/en/server-sql-mode.html#sqlmode_traditional
If you want to trap it to ignore it see "Temporarily suppressing warnings" in Python's documentation:
https://docs.python.org/2/library/warnings.html#temporarily-suppressing-warnings
import warnings
with warnings.catch_warnings():
warnings.simplefilter("ignore")
# Put here your query raising a warning
Else, just read the doc about "warnings" module of Python, you can transform them into exception if you want to catch them later, etc... it's all here: https://docs.python.org/2/library/warnings.html
Just to add to Thomas Wouters reply, there is no need to import the warnings module to turn them into errors. Just run your script with "-W error" (or ignore) as flag for Python.
Have you tried using MySQL's SHOW WARNINGS command?
Stop using MySQLdb. It has such stupid behavior as truncating data and issuing only a warning. Use oursql instead.