I'm looking to create an AWS system with one master EC2 instance which can create other instances.
For now, I managed to create python files with boto able to create ec2 instances.
The script works fine in my computer environment but when I try to deploy it using Amazon BeanStalk with Django (Python 3.4 included) the script doesn't work. I can't configure aws cli (and so Boto) through SSL because the only user I can access is ec2-user and the web server uses another user.
I could simply handwrite my access ID key and password on the python file but that would not be secure. What can I do to solve this problem?
I also discovered AWS cloudformation today, is it a better idea to create new instances with that rather than with the boto function run?
This sounds like an AWS credentials question, not specifically a "create ec2 instances" question. The answer is to assign the appropriate AWS permissions to the EC2 instance via an IAM role. Then your boto/boto3 code and/or the AWS CLI running on that instance will have permissions to make the necessary AWS API calls without having an access key and secret key stored in your code.
Related
I have a temporary access key and secret access key that I have to use in Amazon S3. I never used AWS before so I'm very confused. How do I use my credentials? Do I need to make my own AWS account and use the credentials in there? I'm given a bucket name and bucket location for Amazon S3. Can someone help?
As you are new to Amazon Web Services and looks like you want to use Python, here is a good place to start with that will get you up and running.
See:
Quickstart
This guide details the steps needed to install or update the AWS SDK for Python.
The SDK is composed of two key Python packages: Botocore (the library providing the low-level functionality shared between the Python SDK and the AWS CLI) and Boto3 (the package implementing the Python SDK itself).
I am having three instances in AWS. I have to start each instance by logging into the account and then start it manually. I want to start it using python by just running a script so that I don't need to login to the AWS account and start the service manually. Here are my instance types.
Is there any way I could do it? I am new to AWS so not finding a way to do it.
The easiest method would be to use the AWS Command-Line Interface (CLI):
aws ec2 start-instances --instance-ids i-11111 i-2222 i-3333
If you have not previously used the AWS CLI, you will first need to run aws configure and provide your IAM User credentials (Access Key + Secret Key).
You can also use a Python script to do this, using the boto3 SDK and the start_instances() command.
My setup is: the code is in the private repository in Github which I run from AWS EC2.
I have this doubt where should I store the API and database credentials. My feeling at the moment is that no credentials should be stored in the code, instead, I should use the AWS Secret Manager to access them but then, you also connect to AWS. What is your view on it? A disclosure, I am starting with Python, so, please, be gentle.
Never store your secrets in code. In your case I would recommend AWS Secret Manager (Or secret parameters in AWS System Manager Parameter Store) and store your secrets there.
I would recommend to create an IAM role for your EC2 which has a policy which allows the role to read the correct secrets from AWS Secret Manager. Connect the role with an instance profile and the instance profile with the EC2. This is done automatically in the AWS console but not when your using CloudFormation. An instance profile is kind of a wrapper around a role that allows the role to be attached to an instance.
In this flow your EC2 instance will be allowed to read the secrets from system manager by using the instance profile and role. Roles are the recommended way to make AWS resources interact with each other because it uses temporary credentials and restricts access.
With the above setup you should be able to read the secrets from within your code like explained here. You can use boto3 (AWS SDK for Python) to interact from within the EC2 to the secrets manager.
I am an absolute beginner in AWS: I have created a key and an instance, the python script I want to run in the EC2 environment needs to loop through around 80,000 filings, tokenize the sentences in them, and use these sentences for some unsupervised learning.
This might be a duplicate; but I can't find a way to copy these filings to the EC2 environment and run the python script in EC2, I am also not very sure as to how I can use boto3. I am using Mac OS. I am just looking for any way to speed things up. Thank you so so much! I am forever grateful!!!
Here's what I tried recently:
Create the bucket and keep the bucket accessible for public.
Create the role and add HTTP option.
Upload all the files and make sure the files are public accessible.
Get the HTTP link of the S3 file.
Connect the instance through putty.
wget copies the file into EC2
instance.
If your files are in zip format, one time copy enough to move all the files into instance.
Here's one way that might help:
create a simple IAM role that allows S3 access to the bucket holding your files
apply that IAM role to the running EC2 instance (or launch a new instance with the IAM role)
install the awscli on the EC2 instance
SSH to the instance and sync the S3 files to the EC2 instance using aws s3 sync
run your app
I'm assuming you've launched EC2 with enough diskspace to hold the files.
I am looking to launch an AWS instance by deploying a script. However, I do not fully understand what this means. What should be in the script in order to launch it and how do I approach this in order to meet the following requirements?
User specifies AWS credentials in a separate key file;
User invokes termination script and pass the instance ID from
command line;
Termination script shuts down AWS instance.
Upon completion, the termination script returns message indicating
whether the termination process has been completed successfully
I would appreciate some help in understanding what exactly a deployment script it and what language I should write it in. I have been coding thus far in Python and have created a script that creates an instance. But I am not sure how this is different from deploying an instance.
The usage of the expressions "create an instance" and "deploy an instance" can mean the same thing or different things. Depends on the engineer's viewpoint.
Basically creating an EC2 instance means the AWS definition of launching an EC2 instance. Deploying an EC2 instance may include additional configuration details such as patching the OS, installing software and applications, etc. It is up to you to decide which is which and how each should be done.
When deploying an EC2 instance, I prefer to configure a machine exactly the way that I want with OS patches, software and my applications. Then I create an AMI. When I then launch a new EC2 instance, I use my hand created AMI. Then the new EC2 instance is exactly what I want. No long deployment phase.
Best practices when writing scripts. Do not store your Amazon credentials in your scripts, source code, random files, etc. Install the Amazon CLI (Command Line Internface) tool and then configure the CLI with your credentials. Now your credentials are stored in a well defined location with the added benefit that Amazon SDKs, scripts, etc. will know how to find the credentials and will automatically load and use them.
The easiest way of writing scripts to manage AWS services is to use the AWS CLI. Just about anything that you can do in the Amazon Management Console, you can do with the CLI. The CLI works on Windows, Linux and Mac OS.
AWS Command Line Interface
Here is a CLI example that will terminate an EC2 instance. Replace with your instance ID:
aws ec2 terminate-instances --instance-ids i-1234567890abcdef0
Writing your scripts in Python is another good idea. Managing AWS services with Python is very easy; there are lots of examples available on the Internet; and Python is just so easy and quick to develop Amazon apps. Use the Boto3 library and not the older Boto library. I use Python 3.x for all new development, but be aware that there is a lot of already created work on the Internet for AWS that runs under Python 2.x.
CLI EC2 Commands