We Keep Coding

AWS Lambda function for AWS Metric using Autoscaling groups

I am in the midst of coding a lambda function which will create an alarm based upon some disk metrics. The code so far looks like this:
import collections
from datetime import datetime
import calendar
def lambda_handler(event, context):
client = boto3.client('cloudwatch')
alarm = client.put_metric_alarm(
AlarmName='Disk Monitor',
MetricName='disk_used_percent',
Namespace='CWAgent',
Statistic='Maximum',
ComparisonOperator='GreaterThanOrEqualToThreshold',
Threshold=60.0,
Period=10,
EvaluationPeriods=3,
Dimensions=[
{
'Name': 'InstanceId',
'Value': '{instance_id}'
},
{
'Name': 'AutoScalingGroupName',
'Value': '{instance_id}'
},
{
'Name': 'fstype',
'Value': 'xfs'
},
{
'Name': 'path',
'Value': '/'
}
],
Unit='Percent',
ActionsEnabled=True)
As seen, {instance_id} is a variable because the idea is that this will be used for every instance. However, I am wondering how I would code the same for AutoScalingGroupName because I require this to be a variable also. I know that that the below pulls out the AutoScalingGroupName for me, but how would I add that to the above block in terms of syntax, is my problem:
aws autoscaling describe-auto-scaling-instances --output text --query "AutoScalingInstances[?InstanceId == '<instance_dets>'].{AutoScalingGroupName:AutoScalingGroupName}"
For example, would I add a block beginning as below:
def lambda_handler(event, context):
client = boto3.client('autoscaling')
And if so, how would I then code what is needed in terms of syntax to get the 'Value': '{AutoScalingGroupName}' by which I mean a variable to hold the ASG?

describe_auto_scaling_instances takes InstanceIds as a parameter. So if you know your instance_id you can find its asg as follows:
client = boto3.client('autoscaling')
response = client.describe_auto_scaling_instances(
InstanceIds=[instance_id])
asg_name = ''
if response['AutoScalingInstances']:
asg_name = response['AutoScalingInstances'][0]['AutoScalingGroupName']
print(asg_name)

Error when trying to add a Security group to an EC2 Instance

sg = ec2.create_security_group(GroupName='MyWebServer', Description = 'WebServer', VpcId='vpc-0dea879f34afff60d')
instance = ec2.create_instances(
ImageId='ami-0fc970315c2d38f01',
MinCount=1,
MaxCount=1,
InstanceType='t2.nano',
SecurityGroups=[ sg.group_id ]
)
I'm trying to create an instance and attach a security group to it,
It's giving me an error when I run this code of ''(InvalidParameterValue) when calling the RunInstances operation: Value () parameter groupId is invalid. The value cannot be empty."
It creates the security group but doesn't create the instance when called. Any solutions or help would be appreciated.

We can access group Id with sg['GroupId']
import boto3
ec2 = boto3.client('ec2', region_name='us-east-1')
sg = ec2.create_security_group(GroupName='MyWebServer', Description = 'WebServer', VpcId='vpc-0dea879f34afff60d')
response = ec2.run_instances(
ImageId='ami-0fc970315c2d38f01',
InstanceType='t2.micro',
MaxCount=1,
MinCount=1,
SecurityGroupIds=[
sg['GroupId']
],
)

Try this:
instance = ec2.create_instances(
ImageId='ami-0fc970315c2d38f01',
MinCount=1,
MaxCount=1,
InstanceType='t2.nano',
SecurityGroupIds=[ sg.group_id ]
)

// BELOW CODE WILL CREATE AWS INSTANCE AND LINK IT WITH SECURITY GROUP
// PROVIDER MENTIONING THE REGION AND THE ACCESS KEY FOR THE CLOUD
provider "aws" {
# Configuration options
region= "us-east-2"
access_key= "XXXX"
secret_key= "XXXXXX"
}
// CREATING THE AMI INSTANCE AND ASSOCIATING TO THE SECURITY GROUP
resource "aws_instance" "base" {
ami = "ami-0277b52859bac6f4b"
instance_type = "t2.micro"
associate_public_ip_address = true
key_name = "Linux" // ASSOCIATING THE EXISTING LOGIN KEY PAIR NAME
tags = {
Name="terraform"
}
//ASSOCIATING THE EXISTING SECURITY GROUP TO THE INSTANCE
vpc_security_group_ids = [ aws_security_group.customSecGrp.id ]
}
// CREATING A SECURITY WITH INBOUND AND OUTBOUND PORTS
resource "aws_security_group" "customSecGrp" {
name = "customSecGrp"
description = "Security group allowing Inbound"
tags = {
Name = "customSecGrp"
}
ingress {
description = "SSL TLS from VPC"
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
ingress {
cidr_blocks = [ "0.0.0.0/0" ]
description = "tomcat port from vpc"
from_port = 8080
protocol = "tcp"
self = false
to_port = 8080
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}

How to set tags for AWS EC2 instance in boto3

I am new to Boto3, and wanted to create a VPC, subnets, and some ec2 instances. The basic architecture is having a VPC, 2 subnets within 2 different availability zones (us-east-1a and b), and applying a security group which allows SSH and ping.
My problem is how to specify additional options for each resources. The Python SDK (unlike how Javadoc works) doesn't show the required arguments and example options, so I'm confused.
How can I specify tags for resources? (e.g. ec2 instance). I need to set name, owner, etc.
instances2 = ec2.create_instances(ImageId='ami-095575c1a372d21db', InstanceType='t2.micro', MaxCount=1, MinCount=1, NetworkInterfaces=[{'SubnetId': subnet2.id, 'DeviceIndex': 0, 'AssociatePublicIpAddress': True, 'Groups': [sec_group.group_id]}])
instances2[0].wait_until_running()
print(instances1[0].id)

You need the TagSpecifications argument with 'ResourceType' set to 'instance':
TagSpecifications=[
{
'ResourceType': 'instance',
'Tags': [
{
'Key': 'name',
'Value': 'foobar'
},
{
'Key': 'owner',
'Value': 'me'
},
]
},
],
It is in the docs but you do need to know what you're looking for...

Following works for me with python 3.7 and boto3 1.12.39
Reference - https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html?highlight=create_instances#EC2.ServiceResource.create_instances
AMI = os.environ['AMI']
INSTANCE_TYPE = os.environ['INSTANCE_TYPE']
KEY_NAME = os.environ['KEY_NAME']
SUBNET_ID = os.environ['SUBNET_ID']
TAG_SPEC = [
{
"ResourceType":"instance",
"Tags": [
{
"Key": "Name",
"Value": "EC2_INSTANCE_TEST_AUTOGENERATED"
}
]
}
]
ec2 = boto3.resource('ec2')
instance = ec2.create_instances(
ImageId = AMI,
InstanceType = INSTANCE_TYPE,
KeyName = KEY_NAME,
SubnetId = SUBNET_ID,
TagSpecifications = TAG_SPEC,
MaxCount = 1,
MinCount = 1
)

boto3 vpc_exists waiter returns before VPC exists?

I got the following error when trying to set a VPC's tags after creating it (once out of a few dozen tries. It works most of the time, but not always):
botocore.exceptions.ClientError: An error occurred (InvalidVpcID.NotFound) when calling the CreateTags operation: The vpc ID 'vpc-4240de24' does not exist
I checked afterwards, and the VPC vpc-4240de24 did exist, so CreateTags was called too early.
The error occurred in the following method:
def create_vpc(self, region, vpc_name):
"""create VPC in region and attach tags (threaded)"""
ec2_client = aws.ec2_client(region) # Custom method, essentially calls boto3.client('ec2')
vpc_id = ec2_client.create_vpc(
CidrBlock="172.31.0.0/16",
AmazonProvidedIpv6CidrBlock=False
)["Vpc"]["VpcId"]
# TODO: Attach tags on VPC creation when (if) it becomes supported
ec2_client.get_waiter("vpc_exists").wait(VpcIds=[vpc_id])
ec2_client.create_tags(
Resources=[vpc_id],
Tags=[
{
"Key": "Namespace",
"Value": config.NAMESPACE
},
{
"Key": "Name",
"Value": vpc_name
}
]
)
I do not understand how getting that error is even possible. Shouldn't the vpc_exists waiter return only when the VPC exists, and raise the WaiterError exception otherwise? I would set a sleep for 1 second after the waiter, but is there something I'm not doing correctly?

You can now tag the VPC on creation
response = client.create_vpc(
CidrBlock='string',
AmazonProvidedIpv6CidrBlock=True|False,
Ipv6Pool='string',
Ipv6CidrBlock='string',
DryRun=True|False,
InstanceTenancy='default'|'dedicated'|'host',
Ipv6CidrBlockNetworkBorderGroup='string',
TagSpecifications=[
{
'ResourceType': 'vpc',
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
},
]
)

How to find instances that DONT have a tag using Boto3

I'm trying to find instances that DONT have a certain tag.
For instance I want all instances that don't have the Foo tag.
I also want instances that don't have the Foo value equal to Bar.
This is what I'm doing now:
import boto3
def aws_get_instances_by_name(name):
"""Get EC2 instances by name"""
ec2 = boto3.resource('ec2')
instance_iterator = ec2.instances.filter(
Filters=[
{
'Name': 'tag:Name',
'Values': [
name,
]
},
{
'Name': 'tag:Foo',
'Values': [
]
},
]
)
return instance_iterator
This is returning nothing.
What is the correct way?

Here's some code that will display the instance_id for instances without a particular tag:
import boto3
instances = [i for i in boto3.resource('ec2', region_name='ap-southeast-2').instances.all()]
# Print instance_id of instances that do not have a Tag of Key='Foo'
for i in instances:
if i.tags is not None and 'Foo' not in [t['Key'] for t in i.tags]:
print i.instance_id