We Keep Coding

AWS RDS snapshot by python Lamda - python

I am trying to create/delete Snapshot of my AWS RDS through function of AWS Lamda (python 3.6), but I don't know where I am doing wrong, this function also delete Snapshot from 7 days old, so anyone can suggest me on this
import boto3
import datetime
def lambda_handler(event, context):
print("Connecting to RDS")
client = boto3.client('rds')
dbInstances = ['magento-live']
for dbInstance in dbInstances:
print("RDS snapshot backups started at %s...\n" % datetime.datetime.now())
client.create_db_snapshot(
DBInstanceIdentifier=dbInstance,
DBSnapshotIdentifier=dbInstance+'{}'.format(datetime.datetime.now().strftime("%y-%m-%d-%H")),
Tags=[
{
'Key': 'NI',
'Value': 'NIRDS'
},
]
)
for snapshot in client.describe_db_snapshots(DBInstanceIdentifier=dbInstance, MaxRecords=50)['DBSnapshots']:
createTs = snapshot['SnapshotCreateTime'].replace(tzinfo=None)
if createTs < datetime.datetime.now() - datetime.timedelta(days=7):
print("Deleting snapshot id:", snapshot['DBSnapshotIdentifier'])
client.delete_db_snapshot(
DBSnapshotIdentifier=snapshot['DBSnapshotIdentifier']
)
But always I am getting below error
{
"errorMessage": "An error occurred (InvalidParameterValue) when calling the CreateDBSnapshot operation: The specified instance is a member of a cluster and a snapshot cannot be created directly. Please use the CreateDBClusterSnapshot API instead.",
"errorType": "ClientError",
"stackTrace": [
[
"/var/task/lambda_function.py",
19,
"lambda_handler",
"'Value': 'NIRDS'"
],
[
"/var/runtime/botocore/client.py",
312,
"_api_call",
"return self._make_api_call(operation_name, kwargs)"
],
[
"/var/runtime/botocore/client.py",
605,
"_make_api_call",
"raise error_class(parsed_response, operation_name)"
]
]
}

Aron is right. You have use a different method (that applies to aurora cluster and not a DB instance).
Here is a lambda function that worked for me:
def lambda_handler(event, context):
print("Connecting to RDS")
client = boto3.client('rds')
print("RDS snapshot backups stated at %s...\n" % datetime.datetime.now())
client.create_db_cluster_snapshot(
DBClusterIdentifier='enter-your-cluster-name-her',
DBClusterSnapshotIdentifier='enter-your-cluster-name-here-%s' % datetime.datetime.now().strftime("%y-%m-%d-%H"),
Tags=[
{
'Key': 'ENV',
'Value': 'dev'
},
]
)

Related

I have set up an AWS PinPoint project and I'm trying to test it by sending an event from a lambda function:
import boto3
import datetime
import time
client = boto3.client('pinpoint')
app_id = '1234abcd'
endpoint_id = 'test_endpoint'
address = 'test#test.com'
def lambda_handler(event, context):
response = client.put_events(
ApplicationId = app_id,
EventsRequest={
'BatchItem': {
endpoint_id: {
'Endpoint': {
'ChannelType': 'EMAIL',
'Address': address,
'Attributes': {
'Cart': ['Hat'],
'Purchased': ['No']
}
},
'Events':{
'cart-event-2': {
'Attributes':{
'AddedToCart': 'Hat'
},
'EventType': 'AddToCartEvent',
'Metrics': {
'price': 29.95
},
'Timestamp': datetime.datetime.fromtimestamp(time.time()).isoformat()
}
}
}
}
}
)
return response
But I am receiving an error that the resource cannot be found, even though I can see it in Pin Point console:
{
"errorMessage": "An error occurred (NotFoundException) when calling the PutEvents operation: Resource not found",
"errorType": "NotFoundException",
"requestId": "xxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx",
"stackTrace": [
" File \"/var/task/lambda_function.py\", line 12, in lambda_handler\n response = client.put_events(\n",
" File \"/var/runtime/botocore/client.py\", line 391, in _api_call\n return self._make_api_call(operation_name, kwargs)\n",
" File \"/var/runtime/botocore/client.py\", line 719, in _make_api_call\n raise error_class(parsed_response, operation_name)\n"
]
}

Turns out I was just in the wrong region on my AWS account. 🧠
I created the AWS pinpoint project in one region but was trying to send events to the project from another AWS region, which was why I was getting the NotFoundException.

Below is the simple code where I'm trying to pull SnapshotId from describe_volumes. However, I get a KeyError with not much information to go off of. Please let me know what I'm doing wrong, thank you
import boto3
ec2 = boto3.client('ec2')
def lambda_handler(event, context):
snapshot_id = ec2.describe_volumes(VolumeIds=['vol-xxxx'])
print(snapshot_id['SnapshotId'])
In the above code, I get the following error:
{
"errorMessage": "'SnapshotId'",
"errorType": "KeyError",
"requestId": "6b99ce8b-092e-49b8-89b3-72381129e9cc",
"stackTrace": [
" File \"/var/task/lambda_function.py\", line 7, in lambda_handler\n print(snapshot_id['SnapshotId'])\n"
]
}

The response synthax for 'describe_volumes' method is:
{
'Volumes': [
{
'Attachments': [
.
.
,
],
'AvailabilityZone': 'string',
'SnapshotId': 'string',
.
.
.
},
],
'NextToken': 'string'
}
So when you attempt to read the SnapshotId attribute it does not work because it is not on the root level. Considering it is a list of volumes you can iterate it and implement the logic that you need. For example:
response_describe_volumes = ec2.describe_volumes(VolumeIds=['vol-xxxxxxxxxxx'])
for volume in response_describe_volumes['Volumes']:
print(volume['SnapshotId'])
#TODO
Reference:
Boto3 describe_volumes

I'm using urllib3 library in Lambda and python3 code that fetches the webhook url of MSTeams from AWS Secret Manager and sends a http post request to publish a notification.
My webhook url starts with https and looks like this "https://outlook.office.com/webhook/.......". On executing the lambda function, I get an error as shown below LocationParseError Failed to parse:
Code
import urllib3
http = urllib3.PoolManager()
MSTEAMS_WEBHOOK_SECRET_NAME = os.getenv('MSTEAMS_WEBHOOK_SECRET_NAME')
HOOK_URL = get_secret(MSTEAMS_WEBHOOK_SECRET_NAME,"eu-west-1")
def get_secret(secret_name, region_name):
# Create a Secrets Manager client
session = boto3.session.Session()
client = session.client(
service_name='secretsmanager',
region_name=region_name
)
get_secret_value_response = client.get_secret_value(
SecretId=secret_name,
VersionStage="AWSCURRENT"
)
if 'SecretString' in get_secret_value_response:
secret = get_secret_value_response['SecretString']
return secret
else:
decoded_binary_secret = base64.b64decode(get_secret_value_response['SecretBinary'])
return decoded_binary_secret
def lambda_handler(event, context):
message = {
"#context": "https://schema.org/extensions",
"#type": "MessageCard",
"themeColor": data["colour"],
"title": title,
"text": "accountId:\n" + account_id + " <br/>\n"
}
webhook_encoded_body = json.dumps(message).encode('utf-8')
response = http.request('POST',HOOK_URL, body=webhook_encoded_body)
errorMessage
{
"errorMessage": "Failed to parse: {\"msteams-secret\":\"https://outlook.office.com/webhook/dxxxxxx#d779xxxxx-xxxxxx/IncomingWebhook/axxxxxx5/ca746326-bxxx-4xxx-8x-xxxxx\"}",
"errorType": "LocationParseError",
"stackTrace": [
[
"/var/task/lambda_function.py",
145,
"lambda_handler",
"resp = http.request('POST',HOOK_URL, body=webhook_encoded_body)"
],
[
"/var/runtime/urllib3/request.py",
80,
"request",
"method, url, fields=fields, headers=headers, **urlopen_kw"
],
[
"/var/runtime/urllib3/request.py",
171,
"request_encode_body",
"return self.urlopen(method, url, **extra_kw)"
],
[
"/var/runtime/urllib3/poolmanager.py",
324,
"urlopen",
"u = parse_url(url)"
],
[
"/var/runtime/urllib3/util/url.py",
392,
"parse_url",
"return six.raise_from(LocationParseError(source_url), None)"
],
[
"<string>",
3,
"raise_from",
""
]
]
}

Here is how I solved it
Deployed the lambda zip file again, with correct dependencies like requests, urllib3 in the same folder
Apparently, I was trying to store the secret as key/value pair in AWS Secret manager so it was not able to parse a dictionary. I changed the secret type to plaintext

I have written a python script to get instance information over email with cron setup and populate metrics as well. With the following code i can see all the logs in cloudwatch logs console. However "dimension" never gets created under cloudwatch events section and not triggering any mail as well.
import boto3
import json
import logging
from datetime import datetime
logger = logging.getLogger()
logger.setLevel(logging.INFO)
def post_metric(example_namespace, example_dimension_name, example_metric_name, example_dimension_value, example_metric_value):
cw_client = boto3.client("cloudwatch")
response = cw_client.put_metric_data(
Namespace=example_namespace,
MetricData=[
{
'MetricName': example_metric_name,
'Dimensions': [
{
'Name': example_dimension_name,
'Value': example_dimension_value
},
],
'Timestamp': datetime.datetime.now(),
'Value': int(example_metric_value)
},
]
)
def lambda_handler(event, context):
logger.info(event)
ec2_client = boto3.client("ec2")
sns_client = boto3.client("sns")
response = ec2_client.describe_instances(
Filters=[
{
'Name': 'tag:Name',
'Values': [
'jenkins-slave-*'
]
}
]
)['Reservations']
for reservation in response:
ec2_instances = reservation["Instances"]
for instance in ec2_instances:
myInstanceId = (instance['InstanceId'])
myInstanceState = (instance['State']['Name'])
myInstance = \
(
{
'InstanceId': (myInstanceId),
'InstanceState': (myInstanceState),
}
)
logger.info(json.dumps(myInstance)
post_metric("Jenkins", "ciname", "orphaned-slaves", myInstanceId, 1)
# Send message to SNS (Testing purpose)
SNS_TOPIC_ARN = 'arn:aws:sns:us-east-1:1234567890:example-instance-alarms'
sns_client.publish(
TopicArn = SNS_TOPIC_ARN,
Subject = 'Instance Info: ' + myInstanceId,
Message = 'Instance id: ' + myInstanceId
)
Can anyone please help if i am missing anything here. Thanks in advance.

You forgot to add required fields such as EvaluationPeriods, AlarmName and etc. to your put_metric_data according to documentation.
You can use this for an example.

I got the following error when trying to set a VPC's tags after creating it (once out of a few dozen tries. It works most of the time, but not always):
botocore.exceptions.ClientError: An error occurred (InvalidVpcID.NotFound) when calling the CreateTags operation: The vpc ID 'vpc-4240de24' does not exist
I checked afterwards, and the VPC vpc-4240de24 did exist, so CreateTags was called too early.
The error occurred in the following method:
def create_vpc(self, region, vpc_name):
"""create VPC in region and attach tags (threaded)"""
ec2_client = aws.ec2_client(region) # Custom method, essentially calls boto3.client('ec2')
vpc_id = ec2_client.create_vpc(
CidrBlock="172.31.0.0/16",
AmazonProvidedIpv6CidrBlock=False
)["Vpc"]["VpcId"]
# TODO: Attach tags on VPC creation when (if) it becomes supported
ec2_client.get_waiter("vpc_exists").wait(VpcIds=[vpc_id])
ec2_client.create_tags(
Resources=[vpc_id],
Tags=[
{
"Key": "Namespace",
"Value": config.NAMESPACE
},
{
"Key": "Name",
"Value": vpc_name
}
]
)
I do not understand how getting that error is even possible. Shouldn't the vpc_exists waiter return only when the VPC exists, and raise the WaiterError exception otherwise? I would set a sleep for 1 second after the waiter, but is there something I'm not doing correctly?

You can now tag the VPC on creation
response = client.create_vpc(
CidrBlock='string',
AmazonProvidedIpv6CidrBlock=True|False,
Ipv6Pool='string',
Ipv6CidrBlock='string',
DryRun=True|False,
InstanceTenancy='default'|'dedicated'|'host',
Ipv6CidrBlockNetworkBorderGroup='string',
TagSpecifications=[
{
'ResourceType': 'vpc',
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
},
]
)