correct way for catching aws boto3 errors - python

why to use botocore.exceptions.ClientError for catching instead of python inbuilt Exception, what is the recommended way and what are the advantages of using botocore exceptions
I have the following codes
except botocore.exceptions.ClientError as error:
logger.info(error)
except Exception as error:
logger.info(str(error))
For the above code snippets the output through logger is same when object doesn't exists or wrong key is given
INFO:root:An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

If you use botocore.exceptions.ClientError then you will get specifically identifies the error as an AWS client error and this error helps you to solve more errors that are related to AWS libraries in the future.
Also you can use simple python Exception error handling. But I think you need to use botocore.exceptions.ClientError handler for dealing specific error resolutions in the future.

Related

How can i receive errors having status code 500 only on self hosted sentry

I have hosted sentry, now sentry report two types of errors :
1) exceptions that are not handled by any exception view(4XX)
2) exceptions whose exception view returns a status code of 500
I only want to receive those exceptions whose exception view returns a status code of 500.
I am unable to find any option to do it.
Just for reference, i am using sentry to track issue in Pyramid(python) project.

Properly catch boto3 Errors

I am developing a django app which communicates with several Amazon Web Services.
So far I am having trouble dealing with and catching exceptions thrown by the boto3 client. What I am doing seems unnecessarily tedious:
Example:
client = boto3.client('sns')
client.create_platform_endpoint(PlatformApplicationArn=SNS_APP_ARN, Token=token)
this might throw an botocore.errorfactory.InvalidParameterException if e.g. the token is bad.
client.get_endpoint_attributes(EndpointArn=endpoint_arn)
might throw an botocore.errorfactory.NotFoundException.
First, I can't find these Errors anywhere in code, so they are probably generated somewhere. Bottom line: I can't import it and catch it as usual.
Second, I found one way to catch the error here using:
try:
# boto3 stuff
except botocore.exceptions.ClientError as e:
if e.response['Error']['Code'] == 'NotFound':
# handle exception
else:
raise e
But I have to remove the Exception part of the error name. Seems very random and I have no clue whether I would remove the Error in botocore.exceptions.ParamValidationError if I wanted to catch that one. So it's hard to generalize.
Another way to catch the error is using the boto3 client object I got:
try:
# boto3 stuff
except client.exceptions.NotFoundException as e:
# handle exception
This seems the cleanest way so far. But I don't always have the boto3 client object at hand where I want to catch the error. Also I am still only trying things out, so it's mostly guess work.
Does anybody know how boto3 errors are supposed to be handled?
Or can point me towards some coherent documentation which mentions the errors above? Thanks
You've summarized the situation well. The old boto had a simple hardcoded approach to supporting AWS APIs. boto3, in what appears to be an attempt to reduce the overhead of keeping Python client synced with evolving features on the various apis, has been more squishy around exceptions, so the ClientError approach you outlined above used to be the canonical way.
In 2017 they introduced the second mechanism you highlight: 'modeled' exceptions available on the client.
I am not familiar with SNS but in my experience with other AWS products, the ClientError naming matches up with the HTTP apis, which tend to be well documented. So I would start here: https://docs.aws.amazon.com/sns/latest/api/Welcome.html
It looks like the new-style modeled exceptions are generated from service definition files that live in botocore module. I can't find any documentation about it, but go browse around the AWS service models in https://github.com/boto/botocore/tree/master/botocore/data.
Also, it's good to know that if you are not (in contrast to OP's code) dealing directly with the low-level client, but instead are using a high-level AWS ServiceResource object, a low-level client is still easily available at my_service_resource.meta.client so you can handle exceptions like this:
try:
my_service_resource.do_stuff()
except my_service_resource.meta.client.exceptions.NotFoundException as e:
# handle exception
Use Boto3 exceptions: https://www.programcreek.com/python/example/97944/boto3.exceptions
client = boto3.client('logs')
try:
client.create_log_group(logGroupName=LOG_GROUP)
except client.exceptions.ResourceAlreadyExistsException:
pass

azure-storage-python - Exception handling with create_blob_from_path

If for some reason the create_blob_from_path method runs into issues, what would be the proper way to handle the exception? As in, when you are writing your 'except' block, it would be except ??what?? as e: ? I want to be able to handle that exception when a blob upload fails so it can email me via sendmail so I know a backup that is attempting to be offloaded to Azure storage has failed.
Thank you!
I looked for & searched some exception names on the source codes of Azure Python Storage SDK, there are not any exceptions defined about failure of uploading blob. However, per my experience, Azure Storage SDK for Python wrapped REST APIs of Storage Services via the python package requests to do related operations. So any failure will cause the requests exceptions, you can try to catch the requests root exception requests.exceptions.RequestException as below to do the next action like sendmail.
import requests
try:
blobservice.create_blob_from_path(container_name, blob_name, file_path)
except requests.exceptions.RequestException:
sendmain(...)

Which exception should I raise for a REST API error?

I am writing a simple API client in Python and I am wondering what exception should I raise if the remote server is not happy.
The API itself is very poorly documented (I don't know all the possible error messages so I can't just define custom classes for all of them), I am letting Requests handle HTTP-level errors which raises HTTPError in those cases, but what should I raise if the server just supplies an error key in the JSON response?
I am currently using Exception but that feels quite broad, I'd like to know whether there's a better alternative.
Regards.
Yes, just raising Exception is likely too broad.
Usually a module should define his own base exception class:
class MyAPIClientError(Exception):
pass
And then you can subclass that:
class RemoteServerNotHappyError(MyAPIClientError):
pass
RemoteServerNotHappyError should probably mention something about what the server's json returned that was not expected by your API client class. Add in the relevant error messages as you see fit.
This means users of your library can catch these specific exceptions (which they might know how to handle), without having to catch every exception (they surely don't know how to handle every possible failure mode).

Don't send exceptions to the client

It seems that any except thrown in my WAMP server is caught by Autobahn and sent to the client. There are two problems with this: a) if something goes wrong on the server, it needs to be logged on the server so it can be fixed by the team, and b) those exceptions could potentially be very revealing (esp. exceptions from the ORM) and I don't want them to be forwarded to a malicious third party.
Can I prevent this behavior? If I can, is there a way to explicitly send the error messages that should be sent to the client?
Updated answer:
AutobahnPython no longer sends tracebacks by default.
When a procedure of a Callee is invoked that raises an exception, only the exception is forwarded - not a full traceback. To enable forwarding of tracebacks, set traceback_app = True on the Callee's ApplicationSession.
Deprecated answer: The following original answer does only apply to WAMP v1 code in AutobahnPython, which has been deprecated and removed.
The last version of AutobahnPython with WAMP1 code was 0.8.15, which you can find here and here.
You can override the onAfterCallError hook on your class deriving from autobahn.wamp.WampProtocol (see here or the generated docs).
In your override you can do your custom logging and/or modification of the error object that is the basis for the WAMP RPC error return sent to the caller.
try:
main()
except Exception, e:
log(str(e))

Categories