We Keep Coding

How to handle an error that is within an except block that returns?

Background
I am writing an AWS Lambda function that uses the Boto3 SDK to access AWS services in Python.
Context
My main function/lambda handler makes an API request. I check the status code from the request and raise custom exceptions based on the response code. For each except I create a descriptive message to send an email via SNS to notify me about the error.
What I've tried
I read this thread on custom python exceptions with error codes and error messages
I read this thread on nested excpetions but did not really understand how to apply it to my situation.
I also read this thread on handling exceptions that occur within except clause but I do not understand how to translate it.
Question
How do I handle the SNS sending if that raises an error and return the proper information?
class Error(Exception):
pass
class Error_401(Error):
pass
class Error_403(Error):
pass
class Error_404(Error):
pass
class Error_500(Error):
pass
# This syntax is most likely wrong, I just pasted it
# from link 1
class SNS_Send_Error(Error):
def __init__(self, code):
self.code = code
def __str__(self):
return repr(self.code)
def create_sns_client(arn,message):
try:
sns = boto3.client("sns")
res = sns.publish(
TargetArn=arn,
Message=json.dumps({"default": json.dumps(message)}),
MessageStructure="json"
)
except ClientError as e:
code = e.reponse['ResponseMetadata']['HTTStatusCode']
message = e.response['Error']['Code']
print(message)
raise SNS_Send_Error('SNS failed to send message. \
Responed with error code: {}'.fomat(code),code)
return res
def status_codes(arg):
if arg == 401:
raise Error_401
elif arg == 403:
raise Error_403
elif arg == 404:
raise Error_404
elif arg == 500:
raise Error_500
def lambda_handler(event, context):
# Other Code ...
# Request data from GH API
try:
res = request_gh_data(gh_secret, jobId)
status = res.status_code
gh_response = res.json()
status_codes(status)
except Error_401 as e:
message = generate_error(gh_response['message'], context)
create_sns_client(message)
return {
'statusCode': status,
'body': message
}
except Error_403 as e:
message = generate_error(gh_response['message'], context)
create_sns_client(message)
return {
'statusCode': status,
'body': message
}
except Error_404 as e:
message = generate_error(gh_response['message'], context)
create_sns_client(message)
return {
'statusCode': status,
'body': message
}
except Error_500 as e:
message = generate_error(gh_response['message'], context)
create_sns_client(message)
return {
'statusCode': status,
'body': message
}
except SNS_Send_Error as e:
code = e.reponse['ResponseMetadata']['HTTStatusCode']
message = e.response['Error']['Code']
print(message)
return {
"statusCode": code,
"body": message
}
# Additional Code
return {
'statusCode': 200,
'body': "Success!"
}
We can see that in each except block, I send a message based on the type of error that occurs. How should I handle the message sending error in the most "Pythonic" manner.

You can use except without specifying the exception type to handle any kind of Exception. Then you can get what was the cause of the exception with e.__class__.
try:
# code to be executed
except Exception as e:
# you can give your message
# or do the task you want
# based on the exception class
print('exception raised: ', e.__class__)

Raising a server error to the client with grpc

Let's consider a simple service:
service Something {
rpc Do(Request) returns Response;
}
message Request {
string field = 1;
}
message Response {
string response = 1;
}
Assume I have to do some checking on the Request.field, I want to raise a client error if the field is invalid:
class MyService(proto_pb2.SomethingServicer):
def Do(self, request, context):
if not is_valid_field(request.field):
raise ValueError("Damn!") # Or something like that
return proto_pb2.Response(response="Yeah!")
With the following client:
channel = grpc.insecure_channel(...)
stub = proto_pb2.SomethingStub(channel)
try:
response = stub.Do(proto_pb2.Request(field="invalid"))
except grpc.RpcError as e:
print(e)
<_Rendezvous of RPC that terminated with (StatusCode.UNKNOWN, Exception calling application: Damn!)>
So I can technically handle errors. My issue is... is there a better way? Is there a good way to change the message description? Can we change the status code?

Yes, there is a better way. You may change the status details using the ServicerContext.set_details method and you may change the status code using the ServicerContext.set_code method. I suspect that your servicer will look something like
class MyService(proto_pb2.SomethingServicer):
def Do(self, request, context):
if not is_valid_field(request.field):
context.set_code(grpc.StatusCode.INVALID_ARGUMENT)
context.set_details('Consarnit!')
return proto_pb2.Response()
return proto_pb2.Response(response='Yeah!')
.

There's a new method for this too, context.abort() - it'll actually raise an exception to terminate the RPC call:
grpc.ServicerContext.abort()

So at gRPC side someone can abort context using:
grpc.ServicerContext.abort()
At client side (python):
try:
result = {'msg', 'success'}
except grpc.RpcError as e:
if e.code() == grpc.StatusCode.INVALID_ARGUMENT:
result = {'msg', 'invalid arg error'}
elif e.code() == grpc.StatusCode.ALREADY_EXISTS:
result = {'msg', 'already exists error'}

Give Flask error handler access to request

I'm trying to make a custom error page in Flask, and I'd like to give the error handler access to the request that generated the API call that caused the error so that the error page it returns can change depend on the circumstances. For instance, say there are two endpoints:
(1) #app.route('/get_item')
(2) #app.route('/submit_item')
If an error occurs during a call to get_item, I want to display a certain error page ("Sorry, an error occurred...") however, if an error occurs during a call to submit_item, I want it to say something more informative, like:
"An error occured! Please contact us.
Your user id: request.json['userid']
Your submission id: request.json['submission']"
Is it possible to allow the error handler to have access to this, or do I just have to wrap the whole of submit_item in try/except statements?

You can use the request context in the error handler function,
something along those lines:
from flask import request
def exception_handler(*args):
if request.url.endswith('submit_item'):
return "MyMoreDescriptiveErrorMessage", 500
else:
return "Something wrong happened", 500

I would probably create a custom exception and specify an error handler for it similar to this example.
class CustomException(Exception):
def __init__(self, message=None, status_code=None, payload=None):
Exception.__init__(self)
if message is None:
message = "Sorry, an error occurred..."
self.message = message
if status_code is not None:
self.status_code = status_code
self.payload = payload
#app.errorhandler(CustomException)
def handle_custom(error):
response = render_template('error.html', message=error.message)
response.status_code = error.status_code
return response
#app.route('/submit_item')
def submit_item():
message = "An error occured! Userid: %(userid)d, submission: %(submission_id)d"
message = message % (request.json)
raise CustomException(message)

Python Flask Mailchimp signup

I'm trying to use PostMonkey & Flask to HTTP GET an email address (from a from on my website) and then subscribe this to the specified list.
It works and sends the email requesting the user to confirm the subscription, but either server error 500's or when debug mode is on it comes up with
TypeError: signup() takes no arguments (2 given)
Here's my code:
#app.route("/signup", methods=['GET'])
def signup():
try:
email = request.args.get('email')
pm.listSubscribe(id="cdc2ba625c", email_address=email)
except MailChimpException, e:
print e.code
print e.error
return redirect("/")
return signup
I'm not sure what's causing it and it's been bugging me for a while!

If anyone is interested the problem was related to my 'Return' statement, turns out flask doesn't like returning nothing.
#app.route('/signup', methods=['POST'])
def signup():
try:
email = request.form['email']
#email = request.args.get('email')
pm.listSubscribe(id="cdc2ba625c", email_address=email, double_optin=False)
except MailChimpException, e:
print e.code
print e.error
return redirect("/")
return render_template('index.html')
Thanks to all those that commented back

Python: Getting the error message of an exception

In python 2.6.6, how can I capture the error message of an exception.
IE:
response_dict = {} # contains info to response under a django view.
try:
plan.save()
response_dict.update({'plan_id': plan.id})
except IntegrityError, e: #contains my own custom exception raising with custom messages.
response_dict.update({'error': e})
return HttpResponse(json.dumps(response_dict), mimetype="application/json")
This doesnt seem to work. I get:
IntegrityError('Conflicts are not allowed.',) is not JSON serializable

Pass it through str() first.
response_dict.update({'error': str(e)})
Also note that certain exception classes may have specific attributes that give the exact error.

Everything about str is correct, yet another answer: an Exception instance has message attribute, and you may want to use it (if your customized IntegrityError doesn't do something special):
except IntegrityError, e: #contains my own custom exception raising with custom messages.
response_dict.update({'error': e.message})

You should use unicode instead of string if you are going to translate your application.
BTW, Im case you're using json because of an Ajax request, I suggest you to send errors back with HttpResponseServerError rather than HttpResponse:
from django.http import HttpResponse, HttpResponseServerError
response_dict = {} # contains info to response under a django view.
try:
plan.save()
response_dict.update({'plan_id': plan.id})
except IntegrityError, e: #contains my own custom exception raising with custom messages.
return HttpResponseServerError(unicode(e))
return HttpResponse(json.dumps(response_dict), mimetype="application/json")
and then manage errors in your Ajax procedure.
If you wish I can post some sample code.

Suppose you raise error like this
raise someError("some error message")
and 'e' is catched error instance
str(e) returns:
[ErrorDetail(string='some error message', code='invalid')]
but if you want "some error message" only
e.detail
will gives you that (actually gives you a list of str which includes "some error message")

This works for me:
def getExceptionMessageFromResponse( oResponse ):
#
'''
exception message is burried in the response object,
here is my struggle to get it out
'''
#
l = oResponse.__dict__['context']
#
oLast = l[-1]
#
dLast = oLast.dicts[-1]
#
return dLast.get( 'exception' )