I’m trying to use Python to create EC2 instances but I keep getting these errors.
Here is my code:
#!/usr/bin/env python
import boto3
ec2 = boto3.resource('ec2')
instance = ec2.create_instances(
ImageId='ami-0922553b7b0369273',
MinCount=1,
MaxCount=1,
InstanceType='t2.micro')
print instance[0].id
Here are the errors I'm getting
Traceback (most recent call last):
File "./createinstance.py", line 8, in <module>
InstanceType='t2.micro')
File "/usr/lib/python2.7/site-packages/boto3/resources/factory.py", line 520, in do_action
response = action(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/boto3/resources/action.py", line 83, in __call__
response = getattr(parent.meta.client, operation_name)(**params)
File "/usr/lib/python2.7/site-packages/botocore/client.py", line 320, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/lib/python2.7/site-packages/botocore/client.py", line 623, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidAMIID.NotFound) when calling the RunInstances operation: The image id '[ami-0922553b7b0369273]' does not exist
I also get an error when trying to create a key pair
Here's my code for creating the keypair
import boto3
ec2 = boto3.resource('ec2')
# create a file to store the key locally
outfile = open('ec2-keypair.pem','w')
# call the boto ec2 function to create a key pair
key_pair = ec2.create_key_pair(KeyName='ec2-keypair')
# capture the key and store it in a file
KeyPairOut = str(key_pair.key_material)
print(KeyPairOut)
outfile.write(KeyPairOut)
response = ec2.instance-describe()
print response
Here's are the error messages
./createkey.py: line 1: import: command not found
./createkey.py: line 2: syntax error near unexpected token `('
./createkey.py: line 2: `ec2 = boto3.resource('ec2')'
What I am I missing?
For your first script, one of two possibilities could be occurring:
1. The AMI you are referencing by the ID is not available because the key is incorrect or the AMI doesn't exist
2. AMI is unavailable in the region that your machine is setup for
You most likely are running your script from a machine that is not configured for the correct region. If you are running your script locally or on a server that does not have roles configured, and you are using the aws-cli, you can run the aws configure command to configure your access keys and region appropriately. If you are running your instance on a server with roles configured, your server needs to be ran in the correct region, and your roles need to allow access to EC2 AMI's.
For your second question (which in the future should probably be posted separate), your syntax error in your script is a side effect of not following the same format for how you wrote your first script. It is most likely that your python script is not in fact being interpreted as a python script. You should add the shebang at the top of the file and remove the spacing preceding your import boto3 statement.
#!/usr/bin/env python
import boto3
# create a file to store the key locally
outfile = open('ec2-keypair.pem','w')
# call the boto ec2 function to create a key pair
key_pair = ec2.create_key_pair(KeyName='ec2-keypair')
# capture the key and store it in a file
KeyPairOut = str(key_pair.key_material)
print(KeyPairOut)
outfile.write(KeyPairOut)
response = ec2.instance-describe()
print response
Related
i am trying to connect & list the blob container using python runbook , it gives below error
Error:
Failed
Traceback (most recent call last): File "C:\Temp\o54f0ym2.hp4\64f6d97c-4b5b-4ab5-80b2-e9425a2f829e", line 5, in for blob in blob_list: File "C:\WPy64-3800\python-3.8.0.amd64\lib\site-packages\azure\core\paging.py", line 128, in next return next(self._page_iterator) File "C:\WPy64-3800\python-3.8.0.amd64\lib\site-packages\azure\core\paging.py", line 76, in next self._response = self._get_next(self.continuation_token) File "C:\WPy64-3800\python-3.8.0.amd64\lib\site-packages\azure\storage\blob_list_blobs_helper.py", line 100, in _get_next_cb process_storage_error(error) File "C:\WPy64-3800\python-3.8.0.amd64\lib\site-packages\azure\storage\blob_shared\response_handlers.py", line 185, in process_storage_error exec("raise error from None") # pylint: disable=exec-used # nosec File "", line 1, in azure.core.exceptions.HttpResponseError: This request is not authorized to perform this operation.RequestId:78fa857f-001e-0057-5cbb-f2616a000000Time:2022-11-07T15:15:39.7021443ZErrorCode:AuthorizationFailureContent: AuthorizationFailureThis request is not authorized to perform this operation.RequestId:78fa857f-001e-0057-5cbb-f2616a000000Time:2022-11-07T15:15:39.7021443Z
i am using below code :
from azure.storage.blob import ContainerClient
container = ContainerClient.from_connection_string(conn_str="", container_name="")
blob_list = container.list_blobs()
for blob in blob_list:
print(blob.name + '\n')
Any help appreciated. Thanks in advance
I tried to reproduce the same in my environment and got the similar error like below:
The error usually occurs if the required permissions are not granted to perform the action.
I created an Azure Storage container and uploaded test blob:
To resolve the error, make sure to generate the connection string like below:
I am able to get the list of blobs inside the container successfully when I passed the connection string with the required permissions like below:
from azure.storage.blob import ContainerClient
container = ContainerClient.from_connection_string(conn_str="", container_name="")
blob_list = container.list_blobs()
for blob in blob_list:
print(blob.name + '\n')
from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient
from msrest.authentication import CognitiveServicesCredentials
from azure.cognitiveservices.vision.customvision import prediction
from PIL import Image
endpoint = "https://southcentralus.api.cognitive.microsoft.com/"
project_id = "projectidhere"
prediction_key = "predictionkeyhere"
predict = CustomVisionPredictionClient(prediction_key, endpoint)
with open("c:/users/paul.barbin/pycharmprojects/hw3/TallowTest1.jpg", mode="rb") as image_data:
tallowresult = predict.detect_image(project_id, "test1", image_data)
Python 3.7, and I'm using Azure Custom Vision 3.1? (>azure.cognitiveservices.vision.customvision) (3.1.0)
Note that I've seen the same question on SO but no real solution. The posted answer on the other question says to use the REST API instead.
I believe the error is in the endpoint (as stated in the error), and I've tried a few variants - with the slash, without, using an environment variable, without, I've tried appending various strings to my endpoint but I keep getting the same message. Any help is appreciated.
Full error here:
Traceback (most recent call last):
File "GetError.py", line 15, in <module>
tallowresult = predict.detect_image(project_id, "test1", image_data)
File "C:\Users\paul.barbin\PycharmProjects\hw3\.venv\lib\site-packages\azure\cognitiveservices\vision\customvision\prediction\operations\_custom_vision_
prediction_client_operations.py", line 354, in detect_image
request = self._client.post(url, query_parameters, header_parameters, form_content=form_data_content)
File "C:\Users\paul.barbin\PycharmProjects\hw3\.venv\lib\site-packages\msrest\service_client.py", line 193, in post
request = self._request('POST', url, params, headers, content, form_content)
File "C:\Users\paul.barbin\PycharmProjects\hw3\.venv\lib\site-packages\msrest\service_client.py", line 108, in _request
request = ClientRequest(method, self.format_url(url))
File "C:\Users\paul.barbin\PycharmProjects\hw3\.venv\lib\site-packages\msrest\service_client.py", line 155, in format_url
base = self.config.base_url.format(**kwargs).rstrip('/')
KeyError: 'Endpoint'
CustomVisionPredictionClient takes two required, positional parameters: endpoint and credentials. Endpoint needs to be passed in before credentials, try swapping the order:
predict = CustomVisionPredictionClient(endpoint, prediction_key)
There is this document from the Firebase about how to write and deploy the cloud function in Nodejs but can anyone help me out getting that very document in python. I am getting confused as I am a newbie in this field?
However, I tried to write my cloud function which looks like the below but constantly getting some errors that I am going to mention below:
import json
import firebase_admin
from firebase_admin import credentials
from firebase_admin import db
def go_firebase(request):
cred = credentials.Certificate('firebasesdk.json')
firebase_admin.initialize_app(cred, {
'databaseURL' : 'https://firebaseio.com/'
})
ref=db.reference('agents')
snapshot = ref.order_by_key().get()
for key, val in snapshot.items():
kw=val
dictfilt = lambda x, y: dict([ (i,x[i]) for i in x if i in set(y) ])
wanted_keys = ("address","name","phone","uid")
result = dictfilt(kw, wanted_keys)
data= json.dumps(result, sort_keys=True)
return data
And after deploying the function with http trigger, in the log, it is saying:
severity: "ERROR"
textPayload: "Traceback (most recent call last):
File "/env/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 313, in run_http_function
result = _function_handler.invoke_user_function(flask.request)
File "/env/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 215, in invoke_user_function
return call_user_function(request_or_event)
File "/env/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 208, in call_user_function
return self._user_function(request_or_event)
File "/user_code/main.py", line 6, in go_firebase
cred = credentials.Certificate('firebasesdk.json')
File "/env/lib/python3.7/site-packages/firebase_admin/credentials.py", line 83, in __init__
with open(cert) as json_file:
FileNotFoundError: [Errno 2] No such file or directory: 'firebasesdk.json'
I have no idea why it is saying file not found because I have that json file in the same path where I am executing the function! I am using google cloud shell!
Can anyone be kind enough to tell me where I am going wrong?
The .json file that you are referring to is probably the dependencies file for Node.js Cloud Function.
How it works
Every Google Cloud Function has an extra file (additionally to the main code) that has all the libraries to be installed. For example if you are using requests library in your code then there is no way of running pip install requests before executing the main code. Therefore you add this library in the additional file and the Cloud Function will first read that file during deployment and will try to install all the libraries mentioned there.
For Node.js code the file with the libraries is a .json file. For the Python it is a requirements.txt file. For more information, you can refer to The Python Runtime documentation.
I'm trying to execute a Lambda Script in Python with an imported library, however I'm getting permission errors.
I am also getting some alerts about the database, but database queries are called after the subprocess so I don't think they are related. Could someone explain why do I get error?
Alert information
Alarm:Database-WriteCapacityUnitsLimit-BasicAlarm
State changed to INSUFFICIENT_DATA at 2016/08/16. Reason: Unchecked: Initial alarm creation
Lambda Error
[Errno 13] Permission denied: OSError Traceback (most recent call last):File "/var/task/lambda_function.py", line 36, in lambda_handler
xml_output = subprocess.check_output(["./mediainfo", "--full", "--output=XML", signed_url])
File "/usr/lib64/python2.7/subprocess.py", line 566, in check_output process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 710, in __init__ errread, errwrite) File "/usr/lib64/python2.7/subprocess.py", line 1335, in _execute_child raise child_exception
OSError: [Errno 13] Permission denied
Lambda code
import logging
import subprocess
import boto3
SIGNED_URL_EXPIRATION = 300 # The number of seconds that the Signed URL is valid
DYNAMODB_TABLE_NAME = "TechnicalMetadata"
DYNAMO = boto3.resource("dynamodb")
TABLE = DYNAMO.Table(DYNAMODB_TABLE_NAME)
logger = logging.getLogger('boto3')
logger.setLevel(logging.INFO)
def lambda_handler(event, context):
"""
:param event:
:param context:
"""
# Loop through records provided by S3 Event trigger
for s3_record in event['Records']:
logger.info("Working on new s3_record...")
# Extract the Key and Bucket names for the asset uploaded to S3
key = s3_record['s3']['object']['key']
bucket = s3_record['s3']['bucket']['name']
logger.info("Bucket: {} \t Key: {}".format(bucket, key))
# Generate a signed URL for the uploaded asset
signed_url = get_signed_url(SIGNED_URL_EXPIRATION, bucket, key)
logger.info("Signed URL: {}".format(signed_url))
# Launch MediaInfo
# Pass the signed URL of the uploaded asset to MediaInfo as an input
# MediaInfo will extract the technical metadata from the asset
# The extracted metadata will be outputted in XML format and
# stored in the variable xml_output
xml_output = subprocess.check_output(["./mediainfo", "--full", "--output=XML", signed_url])
logger.info("Output: {}".format(xml_output))
save_record(key, xml_output)
def save_record(key, xml_output):
"""
Save record to DynamoDB
:param key: S3 Key Name
:param xml_output: Technical Metadata in XML Format
:return:
"""
logger.info("Saving record to DynamoDB...")
TABLE.put_item(
Item={
'keyName': key,
'technicalMetadata': xml_output
}
)
logger.info("Saved record to DynamoDB")
def get_signed_url(expires_in, bucket, obj):
"""
Generate a signed URL
:param expires_in: URL Expiration time in seconds
:param bucket:
:param obj: S3 Key name
:return: Signed URL
"""
s3_cli = boto3.client("s3")
presigned_url = s3_cli.generate_presigned_url('get_object', Params={'Bucket': bucket, 'Key': obj},
ExpiresIn=expires_in)
return presigned_url
I'm fairly certain that this is a restriction imposed by the lambda execution environment, but it can be worked around by executing the script through the shell.
Try providing shell=True to your subprocess call:
xml_output = subprocess.check_output(["./mediainfo", "--full", "--output=XML", signed_url], shell=True)
I encountered a similar situation. I was receiving the error:
2016-11-28T01:49:01.304Z d4505c71-b50c-11e6-b0a1-65eecf2623cd Error: Command failed: /var/task/node_modules/youtube-dl/bin/youtube-dl --dump-json -f best https://soundcloud.com/bla/blabla
python: can't open file '/var/task/node_modules/youtube-dl/bin/youtube-dl': [Errno 13] Permission denied
For my (and every other) Node Lambda project containing third party libraries, there will be a directory called "node_modules" (most tutorials, such as this one, will detail how this directory is created) that has all the third party packages and their dependencies. The same principles apply to the other supported languages (currently Python and Java). THESE ARE THE FILES THAT AMAZON IS ACTUALLY PUTTING ON THE LAMBDA AMIS AND ATTEMPTING TO USE. So, to fix the issue, run this on the node_modules directory (or whatever directory your third party libraries live in):
chmod -R 777 /Users/bla/bla/bla/lambdaproject/node_modules
This command means making the file readable, writable and executable by all users. Which is apparently what the servers that execute Lambda functions need, in order to work. Hopefully this helps!
I wrote a simple script with the python amazon api. I copied the example but when I run the script I received the following error:
for book in api.item_search('Books', Publisher='Galileo Press'):
return paginator(self.call, **operators)
File "/usr/local/lib/python2.7/dist-packages/amazonproduct/processors/__init__.py",
line 88, in __init__
self.page(kwargs.get(self.counter, 1))
root = self.fun(*self.args, **self.kwargs) mazonproduct.errors.InvalidClientTokenId: InvalidClientTokenId: The
AWS Access Key Id you provided does not exist in our records. File
"/usr/local/lib/python2.7/dist-packages/amazonproduct/api.py", line
529, in item_search
return paginator(self.call, **operators) File "/usr/local/lib/python2.7/dist-packages/amazonproduct/processors/__init__.py",
line 88, in __init__
self.page(kwargs.get(self.counter, 1)) File "/usr/local/lib/python2.7/dist-packages/amazonproduct/processors/__init__.py",
line 121, in page
root = self.fun(*self.args, **self.kwargs) File "/usr/local/lib/python2.7/dist-packages/amazonproduct/api.py", line
334, in call
return self._parse(e.fp) File "/usr/local/lib/python2.7/dist-packages/amazonproduct/api.py", line
277, in _parse
raise _e(errors[e.code]) amazonproduct.errors.InvalidClientTokenId: InvalidClientTokenId: The
AWS Access Key Id you provided does not exist in our records.
AWS Access Key Id you provided does not exist in our records.
Go to aws.amazon.com and either create a new account or login to your account. When in the dashboard, click on your name on the menu bar located at the upper right corner of the screen, then select Security Credentials. Expand the line item that says Access Keys (Access Key ID and Secret Access Key), and create a new Access Key ID. This will also supply you with your Secret Key, make sure to record both.
Create a file ~/.amazon-product-api containing the following data:
[Credentials]
access_key = <your access key>
secret_key = <your secret key>
associate_tag = <your associate id>
Then your program should run.