Im trying to copy the specified ec2 tags to their respective volumes. the function is invoked when the instance state changed to 'running'. However I don't want the code to run on every instance--for the first version of the code, when it as invoked my a single instance, it ran on all instances. even those that were already tagged. I want to have it run only for the specific instances that are booting up.
with some changes, Im getting error: ec2.Instance' object is not iterable
im really new to python and not sure how to proceed. Any inputs from you bright minds?
----HERE IS MY LAMBDA CODE----
import boto3
import json
def lambda_handler(event, context):
# is_test = context.function_name == 'test' # this value is injected by SAM local
instance = boto3.resource('ec2').Instance(id=event["detail"]["instance-id"])
tags_to_use = ['Stack-Name', 'StackID']
for instance in instance:
tags = instance.tags
to_tag = [t for t in tags if t['Key'] in tags_to_use]
for vol in instance.volumes.all():
print(f"Tagging volume {vol.id} from instance {instance.id}")
vol.create_tags(Tags=to_tag)
Related
I'm working on a project to automate the deployment of VMs in GCP using Python. I recently figured out how to create a custom image using Python and I'm now at the step where I need to create the VM. I have the example template from the Google documentation but I'm stuck on a particular method and don't understand the argument that it wants.
I can successfully get the image from the family, create and attached disk from the image, but when I get to create_instance function I'm unsure of how it wants me to reference the disk. disks: List[compute_v1.AttachedDisk]. I keep getting google.cloud.compute.v1.Instance.disks is not iterable when I try and specify the name or path.
Any help or guidance is appreciated.
import re
import sys
from typing import Any, List
import warnings
from google.api_core.extended_operation import ExtendedOperation
from google.cloud import compute_v1
def get_image_from_family(project: str, family: str) -> compute_v1.Image:
"""
Retrieve the newest image that is part of a given family in a project.
Args:
project: project ID or project number of the Cloud project you want to get image from.
family: name of the image family you want to get image from.
Returns:
An Image object.
"""
image_client = compute_v1.ImagesClient()
# List of public operating system (OS) images: https://cloud.google.com/compute/docs/images/os-details
newest_image = image_client.get_from_family(project=project, family=family)
return newest_image
def disk_from_image(
disk_type: str,
disk_size_gb: int,
boot: bool,
source_image: str,
auto_delete: bool = True,
) -> compute_v1.AttachedDisk:
"""
Create an AttachedDisk object to be used in VM instance creation. Uses an image as the
source for the new disk.
Args:
disk_type: the type of disk you want to create. This value uses the following format:
"zones/{zone}/diskTypes/(pd-standard|pd-ssd|pd-balanced|pd-extreme)".
For example: "zones/us-west3-b/diskTypes/pd-ssd"
disk_size_gb: size of the new disk in gigabytes
boot: boolean flag indicating whether this disk should be used as a boot disk of an instance
source_image: source image to use when creating this disk. You must have read access to this disk. This can be one
of the publicly available images or an image from one of your projects.
This value uses the following format: "projects/{project_name}/global/images/{image_name}"
auto_delete: boolean flag indicating whether this disk should be deleted with the VM that uses it
Returns:
AttachedDisk object configured to be created using the specified image.
"""
boot_disk = compute_v1.AttachedDisk()
initialize_params = compute_v1.AttachedDiskInitializeParams()
initialize_params.source_image = source_image
initialize_params.disk_size_gb = disk_size_gb
initialize_params.disk_type = disk_type
boot_disk.initialize_params = initialize_params
# Remember to set auto_delete to True if you want the disk to be deleted when you delete
# your VM instance.
boot_disk.auto_delete = auto_delete
boot_disk.boot = boot
return boot_disk
def wait_for_extended_operation(
operation: ExtendedOperation, verbose_name: str = "operation", timeout: int = 300
) -> Any:
"""
This method will wait for the extended (long-running) operation to
complete. If the operation is successful, it will return its result.
If the operation ends with an error, an exception will be raised.
If there were any warnings during the execution of the operation
they will be printed to sys.stderr.
Args:
operation: a long-running operation you want to wait on.
verbose_name: (optional) a more verbose name of the operation,
used only during error and warning reporting.
timeout: how long (in seconds) to wait for operation to finish.
If None, wait indefinitely.
Returns:
Whatever the operation.result() returns.
Raises:
This method will raise the exception received from `operation.exception()`
or RuntimeError if there is no exception set, but there is an `error_code`
set for the `operation`.
In case of an operation taking longer than `timeout` seconds to complete,
a `concurrent.futures.TimeoutError` will be raised.
"""
result = operation.result(timeout=timeout)
if operation.error_code:
print(
f"Error during {verbose_name}: [Code: {operation.error_code}]: {operation.error_message}",
file=sys.stderr,
flush=True,
)
print(f"Operation ID: {operation.name}", file=sys.stderr, flush=True)
raise operation.exception() or RuntimeError(operation.error_message)
if operation.warnings:
print(f"Warnings during {verbose_name}:\n", file=sys.stderr, flush=True)
for warning in operation.warnings:
print(f" - {warning.code}: {warning.message}", file=sys.stderr, flush=True)
return result
def create_instance(
project_id: str,
zone: str,
instance_name: str,
disks: List[compute_v1.AttachedDisk],
machine_type: str = "n1-standard-1",
network_link: str = "global/networks/default",
subnetwork_link: str = None,
internal_ip: str = None,
external_access: bool = False,
external_ipv4: str = None,
accelerators: List[compute_v1.AcceleratorConfig] = None,
preemptible: bool = False,
spot: bool = False,
instance_termination_action: str = "STOP",
custom_hostname: str = None,
delete_protection: bool = False,
) -> compute_v1.Instance:
"""
Send an instance creation request to the Compute Engine API and wait for it to complete.
Args:
project_id: project ID or project number of the Cloud project you want to use.
zone: name of the zone to create the instance in. For example: "us-west3-b"
instance_name: name of the new virtual machine (VM) instance.
disks: a list of compute_v1.AttachedDisk objects describing the disks
you want to attach to your new instance.
machine_type: machine type of the VM being created. This value uses the
following format: "zones/{zone}/machineTypes/{type_name}".
For example: "zones/europe-west3-c/machineTypes/f1-micro"
network_link: name of the network you want the new instance to use.
For example: "global/networks/default" represents the network
named "default", which is created automatically for each project.
subnetwork_link: name of the subnetwork you want the new instance to use.
This value uses the following format:
"regions/{region}/subnetworks/{subnetwork_name}"
internal_ip: internal IP address you want to assign to the new instance.
By default, a free address from the pool of available internal IP addresses of
used subnet will be used.
external_access: boolean flag indicating if the instance should have an external IPv4
address assigned.
external_ipv4: external IPv4 address to be assigned to this instance. If you specify
an external IP address, it must live in the same region as the zone of the instance.
This setting requires `external_access` to be set to True to work.
accelerators: a list of AcceleratorConfig objects describing the accelerators that will
be attached to the new instance.
preemptible: boolean value indicating if the new instance should be preemptible
or not. Preemptible VMs have been deprecated and you should now use Spot VMs.
spot: boolean value indicating if the new instance should be a Spot VM or not.
instance_termination_action: What action should be taken once a Spot VM is terminated.
Possible values: "STOP", "DELETE"
custom_hostname: Custom hostname of the new VM instance.
Custom hostnames must conform to RFC 1035 requirements for valid hostnames.
delete_protection: boolean value indicating if the new virtual machine should be
protected against deletion or not.
Returns:
Instance object.
"""
instance_client = compute_v1.InstancesClient()
# Use the network interface provided in the network_link argument.
network_interface = compute_v1.NetworkInterface()
network_interface.name = network_link
if subnetwork_link:
network_interface.subnetwork = subnetwork_link
if internal_ip:
network_interface.network_i_p = internal_ip
if external_access:
access = compute_v1.AccessConfig()
access.type_ = compute_v1.AccessConfig.Type.ONE_TO_ONE_NAT.name
access.name = "External NAT"
access.network_tier = access.NetworkTier.PREMIUM.name
if external_ipv4:
access.nat_i_p = external_ipv4
network_interface.access_configs = [access]
# Collect information into the Instance object.
instance = compute_v1.Instance()
instance.network_interfaces = [network_interface]
instance.name = instance_name
instance.disks = disks
if re.match(r"^zones/[a-z\d\-]+/machineTypes/[a-z\d\-]+$", machine_type):
instance.machine_type = machine_type
else:
instance.machine_type = f"zones/{zone}/machineTypes/{machine_type}"
if accelerators:
instance.guest_accelerators = accelerators
if preemptible:
# Set the preemptible setting
warnings.warn(
"Preemptible VMs are being replaced by Spot VMs.", DeprecationWarning
)
instance.scheduling = compute_v1.Scheduling()
instance.scheduling.preemptible = True
if spot:
# Set the Spot VM setting
instance.scheduling = compute_v1.Scheduling()
instance.scheduling.provisioning_model = (
compute_v1.Scheduling.ProvisioningModel.SPOT.name
)
instance.scheduling.instance_termination_action = instance_termination_action
if custom_hostname is not None:
# Set the custom hostname for the instance
instance.hostname = custom_hostname
if delete_protection:
# Set the delete protection bit
instance.deletion_protection = True
# Prepare the request to insert an instance.
request = compute_v1.InsertInstanceRequest()
request.zone = zone
request.project = project_id
request.instance_resource = instance
# Wait for the create operation to complete.
print(f"Creating the {instance_name} instance in {zone}...")
operation = instance_client.insert(request=request)
wait_for_extended_operation(operation, "instance creation")
print(f"Instance {instance_name} created.")
return instance_client.get(project=project_id, zone=zone, instance=instance_name)
def create_from_custom_image(
project_id: str, zone: str, instance_name: str, custom_image_link: str
) -> compute_v1.Instance:
"""
Create a new VM instance with custom image used as its boot disk.
Args:
project_id: project ID or project number of the Cloud project you want to use.
zone: name of the zone to create the instance in. For example: "us-west3-b"
instance_name: name of the new virtual machine (VM) instance.
custom_image_link: link to the custom image you want to use in the form of:
"projects/{project_name}/global/images/{image_name}"
Returns:
Instance object.
"""
disk_type = f"zones/{zone}/diskTypes/pd-standard"
disks = [disk_from_image(disk_type, 10, True, custom_image_link, True)]
instance = create_instance(project_id, zone, instance_name, disks)
return instance
create_instance('my_project', 'us-central1-a', 'testvm-01','?')
I am trying to create 2 instances of linux on amazon ec2 there is one instance which I am starting in script
import sys
import boto3
instance_id = "i-03e7f6391a0f523ee"
action = 'ON'
ec2 = boto3.client('ec2')
if action == 'ON':
response = ec2.start_instances(InstanceIds=[instance_id], DryRun=False)
x2=boto3.resource('ec2')
instance=x2.Instance(instance_id)
foo=instance.wait_until_running('self',Filters=[{'Name':'instance-state-name','Values':['running']}])
print ("instance is now running")
else:
response = ec2.stop_instances(InstanceIds=[instance_id], DryRun=False)
print(response)
ec2 = boto3.resource('ec2', region_name="ap-south-1")
ami-ImageId="ami-00b6a8a2bd28daf19"
#creating instances
ec2.create_instances(ImageId=ami-ImageId,MinCount=1,MaxCount=2,KeyName="datastructutre key",InstanceType="t2.micro")
#using filter to retrive instance status
instances = ec2.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])
for instance in instances:
print(instance.id, instance.instance_type)
#Checking health status of instances
for status in ec2.meta.client.describe_instance_status()['InstanceStatuses']:
print(status)
However this line
ami-ImageId="ami-00b6a8a2bd28daf19"
is giving me errors instead of double quotes I have also tried single quotes
ami-ImageId='ami-00b6a8a2bd28daf19'
but in both the cases error is same, the error is
Syntax error: can't assign to operator
What is the mistake in assigning ami-id in this way? It is normal string being assigned to a variable.
The problem is not in the assignment. It's in the variable name. The - in ami-ImageId is an operator. Use a different variable name e.g. ami_ImageId
The expression "ami-ImageId" is a substraction of ami and ImageId variables but not a correct single variable name. Use ami_ImageId correct name for example.
I am trying to writing a program to read a configuration file but while testing it am having this error:
self.connection_attempts = self.config_file.get('CONNECTION_ATTEMPTS', 'TIME')
AttributeError: 'list' object has no attribute 'get'
I ma pretty sure it is something I don't get, but it is few hours I am trying to understand where the problem is.
My __init__ method looks like this:
import simpleconfigparser
class ReportGenerator:
def __init__(self):
self.config_parser = simpleconfigparser.configparser()
self.config_file = config_parser.read('config.ini')
self.connection_attempts = config_file.get('CONNECTION_ATTEMPTS', 'TIME')
self.connection_timeout = config_file.get('CONNECTION_TIMEOUT', 'TIMEOUT')
self.report_destination_path = config_file.get('REPORT', 'REPORT_PATH')
This code uses the SimpleConfigParser package.
You want config_parser.get() not config_file.get(). config_parser.read() simply returns the list of config files successfully read after populating the config object. (Usually it is called config or cfg, not config_parser).
This list (config_file) serves no purpose in your code and you might as well not capture it at all.
from simpleconfigparser import simpleconfigparser
TIME = 5
TIMEOUT = 10
REPORT_PATH = '/tmp/'
class ReportGenerator:
def __init__(self):
self.config = simpleconfigparser()
config.read('config.ini')
self.connection_attempts = config.get('CONNECTION_ATTEMPTS', TIME)
self.connection_timeout = config.get('CONNECTION_TIMEOUT', TIMEOUT)
self.report_destination_path = config.get('REPORT', REPORT_PATH)
My guess would also be, that you use the default value in .get() the wrong way, but i cannot be certain with the information you have given.
I have been trying to enable ELB connection draining using the modify_lb_attribute method in the python boto module; however I haven't been able to get it working. According to the documentation here http://boto.readthedocs.org/en/latest/ref/elb.html I should be able to call it like his:
modify_lb_attribute(load_balancer_name, attribute, value)
Here is an example:
modify_lb_attribute('my-elb', 'connectionDraining', 120)
When I do this however I receive the following error:
File "/Library/Python/2.7/site-packages/boto/ec2/elb/init.py", line 421, in modify_lb_attribute
value.enabled and 'true' or 'false'
AttributeError: 'NoneType' object has no attribute 'enabled'
I have been able to get it to work successfully with crossZoneLoadBalancing.
For example this works:
modify_lb_attribute('my-elb', 'crossZoneLoadBalancing', 'true')
Any help or suggestions would be appreciated.
Thanks
Working syntax for instantiating a ConnectionDrainingAttribute and passing it a to a load balancer:
from boto.ec2.elb.attributes import ConnectionDrainingAttribute
import boto.ec2.elb
connection = boto.ec2.elb.connect_to_region("region")
cda = ConnectionDrainingAttribute(connection)
cda.enabled = True
cda.timeout = 120
connection.modify_lb_attribute(
load_balancer_name='my-elb',
attribute='connectionDraining',
value=cda
)
More information about the ConnectionDrainingAttribute class can be found here in the boto docs.
When you modify the connectionDraining attribute of a load balancer, there are actually two values you can supply. The first is a boolean indicating whether you are enabling or disabling the connection draining feature. The second is an integer indicating the timeout which obviously applies only if connection draining is being enabled.
To allow you to specify both of these values, boto defines a ConnectionDrainingAttribute class in boto.ec2.elb.attributes. You must pass an instance of this class as the value to modify_elb_attribute, e.g.:
from boto.ec2.elb.attributes import ConnectionDrainingAttribute
cda = ConnectionDrainingAttribute()
cda.enabled = True
cda.timeout = 120
...
modify_lb_attribute('my-elb', cda)
I'm trying to obtain tags from instances in my AWS account using Python's boto library.
While this snippet works correctly bringing all tags:
tags = e.get_all_tags()
for tag in tags:
print tag.name, tag.value
(e is an EC2 connection)
When I request tags from individual instances,
print vm.__dict__['tags']
or
print vm.tags
I'm getting an empty list (vm is actually an instance class).
The following code:
vm.__dict__['tags']['Name']
of course results in:
KeyError: 'Name'
My code was working until yesterday and suddenly I'm not able to get the tags from an instance.
Does anybody know whether there is a problem with AWS API?
You have to be sure that the 'Name' tag exists before accessing it. Try this:
import boto.ec2
conn=boto.ec2.connect_to_region("eu-west-1")
reservations = conn.get_all_instances()
for res in reservations:
for inst in res.instances:
if 'Name' in inst.tags:
print "%s (%s) [%s]" % (inst.tags['Name'], inst.id, inst.state)
else:
print "%s [%s]" % (inst.id, inst.state)
will print:
i-4e444444 [stopped]
Amazon Linux (i-4e333333) [running]
Try something like this:
import boto.ec2
conn = boto.ec2.connect_to_region('us-west-2')
# Find a specific instance, returns a list of Reservation objects
reservations = conn.get_all_instances(instance_ids=['i-xxxxxxxx'])
# Find the Instance object inside the reservation
instance = reservations[0].instances[0]
print(instance.tags)
You should see all tags associated with instance i-xxxxxxxx printed out.
For boto3 you will need to do this.
import boto3
ec2 = boto3.resource('ec2')
vpc = ec2.Vpc('<your vpc id goes here>')
instance_iterator = vpc.instances.all()
for instance in instance_iterator:
for tag in instance.tags:
print('Found instance id: ' + instance.id + '\ntag: ' + tag)
It turned out to be an error in my code. I did not consider the case of having one instance without the tag 'Name'.
There was one instance without the tag "Name" and my code was trying to get this tag from every instance.
When I ran this piece of code in an instance without the tag 'Name',
vm.__dict__['tags']['Name']
I got: KeyError: 'Name'. vm is a AWS instance.
With the instances that actually had this tag set, I didn't have any problem.
Thank you for your help and sorry for asking when it was only my own mistake.