Using DynamoDB, I know I can specify a conditional expression to control updates, for example, using attribute_not_exists() to prevent overwriting an existing item. However, is there any way to check the result of this? I.e. if there was indeed an existing item and the create failed, I'd like to know that so I can return an error code in my HTTP response. However, looking at the documentation in Boto3, by default put_item returns nothing, so I'm unsure of how I'd be able to monitor the success of the operation. Anyone found a way to do this?
To provide a code example
import boto3
from botocore.exceptions import ClientError
dynamodb = boto3.client('dynamodb')
try:
response = dynamodb.put_item(
TableName='my-table',
Item={
"MyPartitionKey": {
"S": 'some-unique-value'
}
},
ConditionExpression="attribute_not_exists(MyPartitionKey)",
)
print('this worked!')
except ClientError as e:
if e.response['Error']['Code'] != 'ConditionalCheckFailedException':
print('This was not a unique key')
else:
print('some other error')
Found it, ConditionalCheckFailedException is thrown. Disappointed the docs don't mention this, other kinds of exceptions are detailed in the boto3 docs, but not this one!
Related
I was wondering how to validate that a user has correctly entered their spotify developer profile information correctly. I have a file, credentials.py, with 3 constants needed to get a spotify API token. In main.py, I use these constants like so.
def authenticate_user():
if not credentials.SPOTIPY_CLIENT_SECRET or not credentials.SPOTIPY_CLIENT_ID:
raise RuntimeError("Spotify credentials have not been set! Check credentials.py")
os.environ["SPOTIPY_CLIENT_ID"] = credentials.SPOTIPY_CLIENT_ID
os.environ["SPOTIPY_CLIENT_SECRET"] = credentials.SPOTIPY_CLIENT_SECRET
os.environ["SPOTIPY_REDIRECT_URI"] = credentials.SPOTIPY_REDIRECT_URI
sp = spotipy.Spotify(auth_manager=spotipy.SpotifyOAuth())
return sp
The resulting variable, sp, does not seem to have any methods to verify that it has successfully been validated. On attempting to use any methods, you are sent to a webpage with the corresponding spotify endpoint, showing that the request failed. When exiting the page, no error is thrown and the program hangs. How do I throw an error on invalid information being used to authenticate with the spotify servers?
You can check if the spotipy api returns a boolean upon verification, just make an if statement that looks like this:
If (sp == True):
Print('credentials valid')
Base on Spotify docs here
you should get a Spotify.oauth2.SpotifyOauthError
then you can surround the code with a try and except.
you can do it like this:
try:
sp = spotipy.Spotify(auth_manager=spotipy.SpotifyOAuth())
return sp
except SpotifyOauthError as e:
raise RuntimeError(f"Spotify credentials didn't work, error={e}")
just don't forget to import the exception from Spotify.oauth2.SpotifyOauthError
I am trying to use moto to mock acm client in a python code but it doesn't seem to work. Coming from a dependency injection mindset this idea of mock decorator isn't clear to me.
def handle_custom_resource_request(event, context):
try:
cert_arn = event['PhysicalResourceId']
acm_client.delete_certificate(CertificateArn=cert_arn)
except Exception as e:
pass
finally:
pass
#mock_acm
def test_handle_custom_resource_request():
event = {
'RequestType': 'Delete',
'PhysicalResourceId': 'arn:aws:acm:eu-west-1:123456789:certificate/DummyCertificate',
'ResponseURL': 'http://localhost'
}
context = {}
response = cert_requester.handle_custom_resource_request(event, context)
print(response)
I want to mock this acm client that it should return a value of True for example when the delete_certificate method is called. However it seems like there is no mocking happening in this test and I am getting the following error
botocore.exceptions.ClientError: An error occurred (ExpiredTokenException) when calling the DeleteCertificate operation: The security token included in the request is expired
Is there way to mock return values like we have in other languages testing frameworks.
To avoid your tests reaching out to AWS, the mock has to be started before any boto3-clients are created.
From your example code, the easiest way would be to simply move the imports around:
#mock_acm
def test_handle_custom_resource_request():
import cert_requester
event = {
'RequestType': 'Delete',
'PhysicalResourceId': 'arn:aws:acm:eu-west-1:123456789:certificate/DummyCertificate',
'ResponseURL': 'http://localhost'
}
context = {}
response = cert_requester.handle_custom_resource_request(event, context)
print(response)
The documentation has a bit more information on this:
http://docs.getmoto.org/en/latest/docs/getting_started.html#what-about-those-pesky-imports
I want to mock this acm client that it should return a value of True for example when the delete_certificate method is called.
Think of Moto as a local, offline equivalent of AWS. The moment you have a unit test that passes against AWS, using boto3-requests to verify it does what it's supposed to do, you can slap the mock_..-decorator on it. With the decorator on it, you can trust that the tests still verify that the behaviour is correct, but without the cost of running it against AWS.
For example, if you want to know whether a certificate was successfully deleted, you can use the following boto3-requests:
Call describe_certificate, and verify it throws an error (because it no longer exists)
Call list_certificates, and verify the deleted certificate no longer shows up
So the final test could look something like this:
def test_handle_custom_resource_request():
# GIVEN
# A certificate exists
# WHEN
# Our business logic should delete the certificate with that ARN
import cert_requester
event = {
'RequestType': 'Delete',
'PhysicalResourceId': 'arn:aws:acm:eu-west-1:123456789:certificate/DummyCertificate',
'ResponseURL': 'http://localhost'
}
context = {}
response = cert_requester.handle_custom_resource_request(event, context)
print(response)
# THEN
# the certificate should be deleted
certificates = boto3.client("acm").list_certificates()
assert "DummyCertificate" not in certificates
When I'm using nova.keypairs.create() and I pass it an invalid public key, I get the following:
BadRequest: Keypair data is invalid: failed to generate fingerprint (HTTP 400) (Request-ID: req-12bc6440-f042-4687-9ee9-d89e7edc260d)
I tried doing the following and for obvious reasons (it's a unique exception to OpenStack) it didn't work:
try:
nova.keypairs.create(name=keyname, public_key=key)
except BadRequest:
raise cherrypy.HTTPError(400, "Invalid public key")
How can I use OpenStack specific exceptions such as BadRequest within my own try and except statements?
You will need to import the exceptions for nova package. Going through github for the package, it looks like you will need to do:
from nova.exception import *
Note that the exception you are seeing is actually InvalidKeypair exception, which itself subclasses from exception class Invalid, the BadRequest message is just the template text for it.
So, your complete code would look something like:
from nova.exception import *
# You can import specific ones if you are confident about them
try:
nova.keypairs.create(name=keyname, public_key=key)
except InvalidKeypair:
raise cherrypy.HTTPError(400, "Invalid public key")
What is the best way to do error handling when getting an object from S3 using Python boto3?
My approach:
from botocore.exceptions import ClientError
import boto3
s3_client = boto3.client('s3')
try:
s3_object = s3_client.get_object("MY_BUCKET", "MY_KEY")
except ClientError, e:
error_code = e.response["Error"]["Code"]
# do error code checks here
I am not sure if ClientError is the best Exception to use here. I know there is a Boto3Error class, but I do not think you can do error code checks similarly to ClientError.
I think your approach is sufficient. If you can narrow your errors to a few, you can break it down into if blocks, and handle accordingly.
except ClientError as e:
error_code = e.response["Error"]["Code"]
if error_code == "AccessDenied":
# do code
elif error_code == "InvalidLocationConstraint":
# do more code
This is just an experimental approach. Because most error responses are API-driven, I don't think you'll find them anywhere directly in the code (ie: doing except AccessDenied:). You can find all error responses for Amazon S3 here.
I test URLs provided by people with urlfetch to catch wrong links.
result = urlfetch.fetch(url)
When I provide an URL such as «http://qwerty.uiop» the log says there was «DNSLookupFailedError», but this code wouldn't catch it:
except urlfetch.DNSLookupFailedError:
self.error(400)
self.response.out.write(
'Sorry, there was a problem with URL "' + url + '"')
I also tried "except urlfetch.Error:" and "except urlfetch.DownloadError:"
What am I doing wrong, and is there another way to accomplish what I'm trying to do?
In the local developer environment and in production I actually see a different exception: DownloadError. Catching that worked fine for me.
try:
result = urlfetch.fetch('http://qwerty.uiop')
except urlfetch.DownloadError:
self.response.write('Oops!')