There are many situations where I know an error will occur and want to pass additional data to Sentry. I still want the exception to get raised (so the rest of my code stops) and I only want one error in sentry.
For example, let's say I'm making an HTTP call and, in the event that the HTTP call fails, I want an error including the response text sent to Sentry:
import requests
resp = requests.post(url, json=payload)
if resp.ok:
return resp.json()
try:
text = resp.json()
except json.JSONDecodeError:
text = resp.text
# TODO: add `text` to Sentry error
resp.raise_for_status()
How do I do this using the Sentry Python SDK?
Rejected solutions:
Sentry's logging: this results in two errors in sentry (one for the log statement and one for the raised exception)
capture_expection: this results in two errors in sentry (one for the captured exception and one for the raised exception)
Adding extra details to the exception message: this breaks Sentry's error grouping because each error has a unique name
Large or Unpredictable Data: set_context
If you need to send a lot of data or you don't know the contents of your data, the function you are looking for is Sentry's set_context. You want to call this function right before your exception gets raised. Note that context objects are limited in size to 8kb.
Note: you should only call set_context in a situation where an exception will definitely get raised, or else the extra information you set may get added to other (irrelevant) errors in Sentry.
For example:
import requests
import sentry_sdk
resp = requests.post(url, json=payload)
if resp.ok:
return resp.json()
try:
text = resp.json()
except json.JSONDecodeError:
text = resp.text
# vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
sentry_sdk.set_context("Payload", {"text": text})
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
resp.raise_for_status()
This attaches it as additional data to your Sentry error, listed just after the breadcrumbs:
Small, Predictable Data: set_tag
If your data is small and predictable (such as the HTTP status code), you can use sentry's set_tag. It's best to do this within a push_scope block so that the tag is just set for the area of your code that may go wrong. Note that tags keys are limited to 32 characters and tag values are limited in size to 200 characters.
Tags show up at the top of the view of a sentry error.
For example:
import requests
from sentry_sdk import push_scope
resp = requests.post(url, json=payload)
if resp.ok:
return resp.json()
# vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
with push_scope() as scope:
sentry_sdk.set_tag("status", resp.status_code)
resp.raise_for_status()
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Related
I'm trying to create a post request in python, but I get an internal server error when issuing the request.
I'm trying to intercept it with a try-statement, but that doesn't seem to work.
import logging
import requests
logging.basicConfig(filename='python.log', filemode='w', level=logging.DEBUG)
url = "https://redacted-url.com/my-api/check_email"
json = {"email":request.params["email"].strip(), "list":request.params["list"]}
headers = {"Content-Type":"application/json", "Accept": "text/plain"}
try:
r = requests.post(url, headers=headers, json=json)
except requests.exceptions.RequestException as e:
logging.error(e, exc_info=True)
A: I have no idea where that logging-file would be stored. Do I have to add the full server-part? What If I just use «python.log»? Where would it be stored?
B: the try/except doesn't seem to work, I still get an internal server error
C: the error definitely occurs on the line r = requests.post(url, headers=headers, json=json). If I comment that out, the error doesn't occur.
D: Since I don't get an error that's meaningful: What am I doing wrong with that request? This is actually my main problem, but it would be nice to figure out how to log that error and how to intercept it.
Last but not least: If I run the same command from the terminal, the request is processed fine. WTH???
I'm setting up a small Python script so my colleagues can collect data from a certain internal API based on a few inputs using the following code:
url = "https://....."
params = dict(...)
client = BackendApplicationClient(client_id=client_id)
client.prepare_request_body(scope=[])
session = OAuth2Session(client=client)
response = session.get(url=url, params=params, verify=session.verify)
where the params are based on the manual inputs. I can guarantee some of the inputs will not conform to the API's requirements fully (like lower case letters where upper case is needed, etc.). In this case, the API will return a response with status 400:
>> response
<Response [400]>
>> response.text
{"statusCode":400,"errorMessage":"Bad Request","errors": ...}
>> response.status_code
400
I thought I could capture this with response.raise_for_status(), but no Exception is raised, and the returned value is None:
>> response.raise_for_status()
None
Why is this? I thought the raise_for_status function was supposed to raise an Exception based on the response's status_code
raise_for_status() on a response from the requests module will raise an HTTPError exception if the HTTP status code is 400. This is a peculiarity of OAuth2Session which you can read about here
Need to capture the response body for a HTTP error in python. Currently using the python request module's raise_for_status(). This method only returns the Status Code and description. Need a way to capture the response body for a detailed error log.
Please suggest alternatives to python requests module if similar required feature is present in some different module. If not then please suggest what changes can be done to existing code to capture the said response body.
Current implementation contains just the following:
resp.raise_for_status()
I guess I'll write this up quickly. This is working fine for me:
try:
r = requests.get('https://www.google.com/404')
r.raise_for_status()
except requests.exceptions.HTTPError as err:
print(err.request.url)
print(err)
print(err.response.text)
you can do something like below, which returns the content of the response, in unicode.
response.text
or
try:
r = requests.get('http://www.google.com/nothere')
r.raise_for_status()
except requests.exceptions.HTTPError as err:
print(err)
sys.exit(1)
# 404 Client Error: Not Found for url: http://www.google.com/nothere
here you'll get the full explanation on how to handle the exception. please check out Correct way to try/except using Python requests module?
You can log resp.text if resp.status_code >= 400.
There are some tools you may pick up such as Fiddler, Charles, wireshark.
However, those tools can just display the body of the response without including the reason or error stack why the error raises.
Using Python 2.7, Django on Google App Engine. I'm trying to do some simple URL checking, including checking a JSON data payload, and return a meaningful error to the user. What I have coded is basically this:
from django.core.exceptions import SuspiciousOperation
...
def check(self, request):
json_data = json.loads(request.body)
if not json_data:
raise SuspiciousOperation('Required JSON data not found in the POST request.')
...
But, when I test this in debug mode (DEBUG = True in settings.py) by omitting the JSON data, instead of returning a HTTP 400 as I expect from SuspiciousOperation, I get an HTTP 500 that contains my error message "Required JSON data not found in the POST request." The same thing occurs if I check for a valiud URL with URLValidator(): I can correctly test for a good or bad URL with the URLValidator(), but if I try to raise a custom message on a bad URL with SuspiciousOperation I get HTTP 500 instead of 400.
How can I return a meaningful error to my caller without the server error obfuscating everything when Debug is turned back off and crashing the process in the process? Is SuspiciousOperation not supported by GAE?
There was an issue raised about this on Django's bug tracker and it looks like it was fixed in 1.6 but not backported. Indeed, SuspiciousOperation is handled by a catch-all in 1.5.11 (django/django/core/handlers/base.py line 173):
except: # Handle everything else, including SuspiciousOperation, etc.
# Get the exception info now, in case another exception is thrown later.
signals.got_request_exception.send(sender=self.__class__, request=request)
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
According to the urllib2 documentation,
Because the default handlers handle redirects (codes in the 300 range), and codes in the 100-299 range indicate success, you will usually only see error codes in the 400-599 range.
And yet the following code
request = urllib2.Request(url, data, headers)
response = urllib2.urlopen(request)
raises an HTTPError with code 201 (created):
ERROR 2011-08-11 20:40:17,318 __init__.py:463] HTTP Error 201: Created
So why is urllib2 throwing HTTPErrors on this successful request?
It's not too much of a pain; I can easily extend the code to:
try:
request = urllib2.Request(url, data, headers)
response = urllib2.urlopen(request)
except HTTPError, e:
if e.code == 201:
# success! :)
else:
# fail! :(
else:
# when will this happen...?
But this doesn't seem like the intended behavior, based on the documentation and the fact that I can't find similar questions about this odd behavior.
Also, what should the else block be expecting? If successful status codes are all interpreted as HTTPErrors, then when does urllib2.urlopen() just return a normal file-like response object like all the urllib2 documentation refers to?
You can write a custom Handler class for use with urllib2 to prevent specific error codes from being raised as HTTError. Here's one I've used before:
class BetterHTTPErrorProcessor(urllib2.BaseHandler):
# a substitute/supplement to urllib2.HTTPErrorProcessor
# that doesn't raise exceptions on status codes 201,204,206
def http_error_201(self, request, response, code, msg, hdrs):
return response
def http_error_204(self, request, response, code, msg, hdrs):
return response
def http_error_206(self, request, response, code, msg, hdrs):
return response
Then you can use it like:
opener = urllib2.build_opener(self.BetterHTTPErrorProcessor)
urllib2.install_opener(opener)
req = urllib2.Request(url, data, headers)
urllib2.urlopen(req)
As the actual library documentation mentions:
For 200 error codes, the response object is returned immediately.
For non-200 error codes, this simply passes the job on to the protocol_error_code handler methods, via OpenerDirector.error(). Eventually, urllib2.HTTPDefaultErrorHandler will raise an HTTPError if no other handler handles the error.
http://docs.python.org/library/urllib2.html#httperrorprocessor-objects
I personally think it was a mistake and very nonintuitive for this to be the default behavior.
It's true that non-2XX codes imply a protocol level error, but turning that into an exception is too far (in my opinion at least).
In any case, I think the most elegant way to avoid this is:
opener = urllib.request.build_opener()
for processor in opener.process_response['https']: # or http, depending on what you're using
if isinstance(processor, urllib.request.HTTPErrorProcessor): # HTTPErrorProcessor also for https
opener.process_response['https'].remove(processor)
break # there's only one such handler by default
response = opener.open('https://www.google.com')
Now you have the response object. You can check it's status code, headers, body, etc.