(cross posted to boto-users)
Given an image ID, how can I delete it using boto?
You use the deregister() API.
There are a few ways of getting the image id (i.e. you can list all images and search their properties, etc)
Here is a code fragment which will delete one of your existing AMIs (assuming it's in the EU region)
connection = boto.ec2.connect_to_region('eu-west-1', \
aws_access_key_id='yourkey', \
aws_secret_access_key='yoursecret', \
proxy=yourProxy, \
proxy_port=yourProxyPort)
# This is a way of fetching the image object for an AMI, when you know the AMI id
# Since we specify a single image (using the AMI id) we get a list containing a single image
# You could add error checking and so forth ... but you get the idea
images = connection.get_all_images(image_ids=['ami-cf86xxxx'])
images[0].deregister()
(edit): and in fact having looked at the online documentation for 2.0, there is another way.
Having determined the image ID, you can use the deregister_image(image_id) method of boto.ec2.connection ... which amounts to the same thing I guess.
With newer boto (Tested with 2.38.0), you can run:
ec2_conn = boto.ec2.connect_to_region('xx-xxxx-x')
ec2_conn.deregister_image('ami-xxxxxxx')
or
ec2_conn.deregister_image('ami-xxxxxxx', delete_snapshot=True)
The first will delete the AMI, the second will also delete the attached EBS snapshot
For Boto2, see katriels answer. Here, I am assuming you are using Boto3.
If you have the AMI (an object of class boto3.resources.factory.ec2.Image), you can call its deregister function. For example, to delete an AMI with a given ID, you can use:
import boto3
ec2 = boto3.resource('ec2')
ami_id = 'ami-1b932174'
ami = list(ec2.images.filter(ImageIds=[ami_id]).all())[0]
ami.deregister(DryRun=True)
If you have the necessary permissions, you should see an Request would have succeeded, but DryRun flag is set exception. To get rid of the example, leave out DryRun and use:
ami.deregister() # WARNING: This will really delete the AMI
This blog post elaborates on how to delete AMIs and snapshots with Boto3.
Script delates the AMI and associated Snapshots with it. Make sure you have right privileges to run this script.
Inputs - Please pass region and AMI ids(n) as inputs
import boto3
import sys
def main(region,images):
region = sys.argv[1]
images = sys.argv[2].split(',')
ec2 = boto3.client('ec2', region_name=region)
snapshots = ec2.describe_snapshots(MaxResults=1000,OwnerIds=['self'])['Snapshots']
# loop through list of image IDs
for image in images:
print("====================\nderegistering {image}\n====================".format(image=image))
amiResponse = ec2.deregister_image(DryRun=True,ImageId=image)
for snapshot in snapshots:
if snapshot['Description'].find(image) > 0:
snap = ec2.delete_snapshot(SnapshotId=snapshot['SnapshotId'],DryRun=True)
print("Deleting snapshot {snapshot} \n".format(snapshot=snapshot['SnapshotId']))
main(region,images)
using the EC2.Image resource you can simply call deregister():
Example:
for i in ec2res.images.filter(Owners=['self']):
print("Name: {}\t Id: {}\tState: {}\n".format(i.name, i.id, i.state))
i.deregister()
See this for using different filters:
What are valid values documented for ec2.images.filter command?
See also: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Image.deregister
Related
I've written a function that returns the Width, Height, and Dominant Color of an image, and I don't know how to store the results into a database. Here's the code I have so far:
import os
from colorthief import ColorThief
from PIL import Image
os.chdir("x")
def get_details():
for f in os.listdir("x"):
if f.endswith(".jpg"):
i = Image.open(f)
fn, fext = os.path.splitext(f)
print(f"The file name is: {fn}{fext}")
width = i.width
print(f"The image width is: {width}px")
height = i.height
print(f"The Image height is: {height}px")
ct = ColorThief(f)
dominant_color = ct.get_color(quality=1)
domcol = dominant_color
def rgb_to_hex(domcol):
return str("#%02x%02x%02x" % domcol).upper()
print(f"The Dominant Color is: {rgb_to_hex(domcol)}")
get_details()
There are 3 general options to make a local "database":
Use a SQL database such as sqlite3. This solution needs some configurations it is a relatively simple solution.
Use a NoSQL type database such as MongoDB but it is more complicated to setup.
Final option (and least suggested but the simplest) is to store the values as a file using pickle library.
Ultimately, it depends on what 'database' you want to use to store the data.
A relational database like mySQL or SQL Server or Oracle will have a connection string.
A simple text file won't have a connection string like a database, but will have a file path.
There are other storage options, too.
Regardless of what and where you decide your database will be, you need a way to access it from your code. For many, many storage options, you should be able to find some exiting Python libraries to use to access them.
Again, without specifics as to what and where you want to store this data, we can only generalize, but the basic idea will be you'll need to 'open' or access your database or file, you'll need to insert a record or row, and then you'll need to save your changes. Once you decide on your platform, you can find your access library, and that library documentation should detail how to use it to save your data.
I started trying Amazon Rekognition to compare faces called from a Lambda execution. my model starts from user uploading an image and S3 will send an event to trigger a lambda that directly fetches the two closest images in the bucket to compare faces, but I can't read the image from the address S3's URI to Lambda to compare faces which has to create a test to read two images from S3 up. Do you guys have a way to get the URI address from S3 to Lambda to compare faces?
This is my test.
{
"sourceImage": "source.jpg",
"targetImage": "target.jpg"
}
This is the main program
import json
import boto3
s3 = boto3.resource('s3')
def lambda_handler(event, context):
print(event)
dump = json.loads(json.dumps(event))
sourceImage = dump['sourceImage']
targetImage = dump['targetImage']
bucket='your_name'
client = boto3.client('rekognition')
faceComparison= client.compare_faces(
SourceImage={'S3Object': {'Bucket':bucket,'Name':str(sourceImage)}},
TargetImage={'S3Object': {'Bucket':bucket,'Name':str(targetImage)}}
)
res = {
"faceRecognition": faceComparison
}
return res
You cannot use URI to access objects on s3 unless it is a public object/bucket. There are 2 alternatives you can use:
You can use the download_fileobj method from boto3 to get streamingbody of the object and pass that to your function.
You can use download_file method to download the file to /tmp location in lambda and then give path of that file to your function.
If you want to perform a Lambda function that reads objects from Amazon S3, use the Amazon S3 API to read the object bytes and DO NOT use an Object URI.
This AWS tutorial performs a very similar use case:
The Lambda function reads all images in a given Amazon S3 bucket. For each object in the bucket, it passes the image to the Amazon Rekognition service to detect PPE information. The results are stored as records in an Amazon DynamoDB table and then emailed to a recipient.
So instead of comparing faces, it detects PPE gear. Its implemented in Java, but you can port it to your programming language. It will point you in the right direction:
Creating an AWS Lambda function that detects images with Personal Protective Equipment
I botched a Firebase cloud function and accidentally created 1.9 million images stored in gs://myapp.appspot.com//tmp/. That double slash is accurate--the server was writing to /tmp/, which I guess results in the path mentioned above.
I'm now wanting to delete those files (they're all nonsense). I tried using the Python wrapper like so:
export GOOGLE_APPLICATION_CREDENTIALS="../secrets/service-account.json"
Then:
from google.cloud import storage
storage_client = storage.Client()
bucket = storage_client.get_bucket('tmp')
blobs = bucket.list_blobs(bucket='tmp', prefix='')
for blob in blobs:
print(' * deleting', blob)
blob.delete()
But this throws:
google.api_core.exceptions.Forbidden: 403 GET https://storage.googleapis.com/storage/v1/b/tmp?projection=noAcl: firebase-adminsdk-yr6f8#myapp.iam.gserviceaccount.com does not have storage.buckets.get access to tmp.
Does anyone know how to allow the admin credentials to delete from /tmp/? Any pointers would be hugely helpful!
I was able to reproduce this problem with gsutil command:
gsutil cp ~/<example-file> gs://<my-project-name>.appspot.com//tmp/
First of all, in my Firebase console I am able to do it with one tick (whole folder) not sure if you consider this.
Anyway if you want to have it done with API I have found following solution.
I think (comparing to my test) bucket name should be: myapp.appspot.com
If you print the blobs in python you will get something like this: <Blob: <my-project-name>.appspot.com, /tmp/<example-file>, 1584700014023619>
The 2nd value is name property of blob object. I noticed that in this situation its blobs name starts with /tmp/
Code that works on my side is:
from google.cloud import storage
storage_client = storage.Client()
bucket = storage_client.get_bucket('myapp.appspot.com')
blobs = bucket.list_blobs()
for blob in blobs:
if blob.name.startswith('/tmp/'):
print(' * deleting', blob)
blob.delete()
I don't think its very elegant solution, but for the one time fix maybe be good enough.
I hope it will help!
I made this code in python 2.7 for downloading bing traffic flow map (specific area) every x minutes.
from cStringIO import StringIO
from PIL import Image
import urllib
import time
i = 1
end = time.time() + 60*24*60
url = 'https://dev.virtualearth.net/REST/V1/Imagery/Map/AerialWithLabels/45.8077453%2C15.963863/17?mapSize=500,500&mapLayer=TrafficFlow&format=png&key=Lt2cLlR9OcfEnMLv5qyd~YbPpC6zOQdhTMcwsKCwlgQ~Am2YLG00hHI6h7W1IPq31VOzqEXKAhedzHfknCejIrdQF_iVrQS82AUdjBT0YMtt'
while True:
buffer = StringIO(urllib.urlopen(url).read())
image = Image.open(buffer)
image.save('C:\Users\slika'+str(i)+'.png')
i=i+1
if time.time()>end:
break
time.sleep(60*10)
This is one of the images i got traffic flow
Now my question is can i convert only traffic flow lines (green,yellow, orange, red) and assign them attributes (1,2,3,4) or ('No traffic' , 'Light' , 'Moderate' , 'Heavy') into shape file for usage in QGIS. What modules should i look for and is it even possible. Any idea or sample code would be much helpful.
This is against the terms of use of Bing Maps.
Also, I notice that you are using a Universal Windows App key. Those keys are to only be used in public facing Windows apps that anyone has access to. These keys cannot be used in GIS/business apps. Use a Dev/Test key or upgrade to an Enterprise account.
I have a vm with disk 1,2,3,4, I want to do some image operations:
Q1: How can i capture the image only contain system disk and disk 3?
Q2: If I achieve the image production described in Q1, can I use this
image install or reload a vm? How SL api to do with the disk 3 in the
image ?
Q3: Can I make a snapshot image only for disk 3?
Q4: If I achieve the image described in Q3, how can I use this
snapshot to init a disk ?
at moment to create the image template you can specify the block devices that you want in the image template you can do that using the API and the portal.
this is an example using the API
"""
Create image template.
The script creates a standard image template, it makes
a call to the SoftLayer_Virtual_Guest::createArchiveTransaction method
sending the IDs of the disks in the request.
For more information please see below.
Important manual pages:
https://sldn.softlayer.com/reference/services/SoftLayer_Virtual_Guest
https://sldn.softlayer.com/reference/services/SoftLayer_Virtual_Guest/createArchiveTransaction
https://sldn.softlayer.com/reference/datatypes/SoftLayer_Virtual_Guest_Block_Device
License: http://sldn.softlayer.com/article/License
Author: SoftLayer Technologies, Inc. <sldn#softlayer.com>
"""
import SoftLayer
# Your SoftLayer API username and key.
USERNAME = 'set me'
API_KEY = 'set me'
# The virtual guest ID you want to create a template
virtualGuestId = 4058502
# The name of the image template
groupName = 'my image name'
# An optional note for the image template
note = 'an optional note'
"""
Build a skeleton SoftLayer_Virtual_Guest_Block_Device object
containing the disks you want to the image.
In this case we are going take an image template of 2 disks
from the virtual machine.
"""
blockDevices = [
{
"id": 4667098,
"complexType": "SoftLayer_Virtual_Guest_Block_Device"
},
{
"id": 4667094,
"complexType": "SoftLayer_Virtual_Guest_Block_Device"
}
]
# Declare a new API service object
client = SoftLayer.Client(username=USERNAME, api_key=API_KEY)
try:
# Creating the transaction for the image template
response = client['SoftLayer_Virtual_Guest'].createArchiveTransaction(groupName, blockDevices, note, id=virtualGuestId)
print(response)
except SoftLayer.SoftLayerAPIError as e:
"""
# If there was an error returned from the SoftLayer API then bomb out with the
# error message.
"""
print("Unable to create the image template. faultCode=%s, faultString=%s" % (e.faultCode, e.faultString))
You only need to get the block devices ID (or disks), for that you can call this method:
http://sldn.softlayer.com/reference/services/SoftLayer_Virtual_Guest/getBlockDevices
There is some rules for the block devices:
Only block devices of type disk can be captured.
The block device of swap type cannot not be included in the list of block devices to capture.(this is is the disk number 1).
The block device which contains the OS must be included (this is the disk number 0).
The block devices which contain metadata cannot be included in the image.
When you are ordening a new device using this image template you need to keep in mind this:
If you are using the placeOrder method you need to make sure that you are adding the prices for the extra disks.
If you are using the createObject method, the number of disk will be taken from the image template, so it is not neccesary to specify the extra disks.
And also you can use the images templates in reloads, but the reload only afects to the disk wich contains the OS. so If you have a Vitrual machine which contains 3 disks and performs a reload only the disk which contains the OS is afected even if the image template has 3 disks.
In case there are errors in your order due to lack of disk capacity or other issues, at provisioning time there will be errors and the VSI will not be provisioned, likely a ticket will be opened and some softlayer employee will inform you about that.
Regards