I'm creating an Amazon Managed Airflow (MWAA) using CDK with the setting of webserver_access_mode='PRIVATE_ONLY'. In this mode, AWS creates a VPC interface endpoint and binds an IP address, from the selected VPC private subnets, to them as explained here: https://docs.aws.amazon.com/mwaa/latest/userguide/configuring-networking.html
Now, I want to use those IPs to add a listener to an existing load balancer that I can then use to connect to a VPN, but this doesn't seem to be available as an output attribute/property of aws_cdk.aws_mwaa.CfnEnvironment: https://docs.aws.amazon.com/cdk/api/v1/python/aws_cdk.aws_mwaa/CfnEnvironment.html#aws_cdk.aws_mwaa.CfnEnvironment.NetworkConfigurationProperty
My question is, is there a way to obtain those IPs associated with the aws_cdk.aws_mwaa.CfnEnvironment? Right now I am looking up the results manually after the deployment with CDK and creating the listener but I would prefer to fully automate it in the same CDK construct.
I struggled with this same problem for some time. In the end I used a Custom Resource in my CFN template, passing it the URL of the MWAA webserver. In the Python code associated with the Custom Resource (Lambda) I do a socket.gethostbyname_ex() call, passing the URL as an argument. This call will return a tuple that that you'll have to parse to extract the endpoint addresses.
I made good use of the crhelper libraries (https://aws.amazon.com/blogs/infrastructure-and-automation/aws-cloudformation-custom-resource-creation-with-python-aws-lambda-and-crhelper/), which made things a lot easier.
In the end, I used a lambda function to resolve the webserver URL and register the IP addresses to the target group. The approach is described in the following AWS blog post: https://aws.amazon.com/blogs/networking-and-content-delivery/hostname-as-target-for-network-load-balancers/
The implementation of the lambda function is also available through the following AWS sample code: https://github.com/aws-samples/hostname-as-target-for-elastic-load-balancer
Related
I have the following script
import redis
client = redis.Redis.from_url('redis://xxx.amazonaws.com:6379')
client.ping()
This works when I run it on a throwaway EC2 instamce
However when I run it locally or on a local server I get
redis.exceptions.ConnectionError: Error 11 connecting to xxx.amazonaws.com:6379. Resource temporarily unavailable.
Is this something to do with the VPC? If so, what is the way around it?
Thanks
Elasticache Redis is a VPC only service. ie. You can only conenct to it from resources within your VPC such as an EC2 instance or a Lambda function.
If you want to connect from outside, You will first need something to gain you VPC access like an AWS VPN or a Transit Gateway.
I think this link discusses it - Accessing ElastiCache resources from outside AWS - https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/accessing-elasticache.html#access-from-outside-aws
To add more context if you may be unaware,
VPC - Virtual Private Cloud ; basically an atmosphere for all your Cloud resources. Every region you choose to work with will have a default VPC. These Default VPCs have specific IP-address blocks associated to them. When you create a resource within a VPC, one of the IP-address will get associated with one of your resource.
Subents - These are partitions of your VPC. By default, all your Subnets within a VPC are present in different Availability Zones of AWS Datacenter in that region. Eg; N. Virginia has 6 AZs meaning it has 6 distinct locations where your resource can be present. In the default VPCs, each subnet represents one of those locations. When you select a subnet in default VPC, you're basically selecting your AZ.
NOTE - In custom made VPC, you can have subnets in the same AZ. That's totally on how you design it.
If you're new to all this, you might want to consider going through AWS docs - https://docs.aws.amazon.com/vpc/latest/userguide/how-it-works.html
They can be very comprehensive. Get some popcorn. :)
Cheers.
is this possible to get a api gateway using it's arn and then add more endpoints to it ? for example I have a api gateway and it's root path is '/path-one' and more api's are attached to it like '/path-one/one' etc not I want to get this api gateway using it's
rest_api_root_resource_id and then add new api like '/path-one/two' to same path . is this possible ? how i can achieve using python cdk
FOR Example as the same way we access lambda function using arn
self.my_lambda = _lambda.Function.from_function_arn(self, "my-lambda", my_lambda_arn)
in same type of thing for api gateway I am finding. any help would be highly appericiated.
When trying to use requests in my function, it only timeout without additional error. I was trying to use rapidapi for amazon. I already have the host and key, but when I test my function it always timeout. My zip file and my function were on the same directory and I knew that my code was correct.
I just figured out that my VPC configuration in Lambda was can only access within the resources of the VPC. I just removed the VPC and it now runs. But when your lambda function will connect to your database, you need to add and configure your VPC.
Few days back I asked a question to stack overflow community at about custom construct library Question
Although I didn't got the exact answer I was looking for. But somehow I managed to create a custom construct library. But now I have another query on how to host the CDK App as an API.
Below is the snapshot of custom construct library:
test_ec2 = ec2.Instance(self, "APIInstance",
vpc=my_vpc,
machine_image=ec2.AmazonLinuxImage(
generation=ec2.AmazonLinuxGeneration.AMAZON_LINUX
),
key_name="test-cdk",
instance_name=inst-name
)
If I want to host above AWS CDK Application as an API that will accept a string for the variable inst-name and will create a EC2 instance. I tried creating it as an lambda function but not sure how to manage "node dependencies and python dependencies" at same time.
Can it be done using an already created EC2 Instance(attaching an IAM Role with permission of cloudformation) and accepting HTTP requests (But don't know how?) Is this making sense?
Thank You in advance to all the devs.
There are many ways to do this. I think the simplest would be to syntesize your cloudformation templates and publish them to S3 ahead of time and use API gateway with a REST API and with AWS request type integration that would create the Cloudformation stack.
Here's a tutorial that explains how to build a REST API with AWS API integration: https://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started-aws-proxy.html
Just this is for SNS:ListTopics action, but you would want cloudformation:CreateStack action instead.
I have two simple lambda functions. Lambda 1 is invoking lambda 2 (both do a simple print for text).
If both lambdas are outside of a VPC then the invocation succeeds, however as soon as I set them both in to access a VPC (I need to test within a VPC as the full process will be wtihin a VPC) the invocation times out.
Do I have to give my lambda access to the internet to be able invoke a second lambda within the same VPC?
If your lambda functions are inside a VPC you need to configure your both lambda functions into private subnet not public subnet. That is the AWS recommended way.
If you are invoking the second Lambda from the first using Amazon API Gateway, then your Lambda will need to have access to the internet. Follow this guide to configure a NAT Gateway (last step).
regarding the VPC: in order to connect to your VPC and access resources there, the Lambdas must reside in the same region as your VPC and also be configured access to your VPC.
Please follow the steps provided in this AWS Guide: Configuring a Lambda Function to Access Resources in an Amazon VPC. This guide advises to use AWS CLI commands to do this and does not show how to configure it through the console.
You will need to be familiar with Amazon networking particulars (VPCs, Security Groups and Subnets), IAM security for the VPC and have a CLI environment setup. You are going to grant the Lambda Function access to this VPC using IDs and IAM execution roles via the CLI.