When trying to use requests in my function, it only timeout without additional error. I was trying to use rapidapi for amazon. I already have the host and key, but when I test my function it always timeout. My zip file and my function were on the same directory and I knew that my code was correct.
I just figured out that my VPC configuration in Lambda was can only access within the resources of the VPC. I just removed the VPC and it now runs. But when your lambda function will connect to your database, you need to add and configure your VPC.
Related
I suspect this might be IAM/VPC issue but perhaps not? My Lambda function calls Cognito's sign_up to register. The lambda function is behind an API GET endpoint.
I am able to call the endpoint locally and it creates the Cognito user successfully. However, when I deploy it to a staging environment, it doesn't seem to be able to call Cognito anymore. I peeked at the Cloudwatch logs and there is no error reported.
Staging environment has a lambda function behind a VPC that also has a MySQL database as well. I am able to pull data from this database from this lambda function. From that same environment, I am now trying to call Cognito user pool.
I also checked that the permission IAM role for this lambda function has CognitoPowerUser permissions.
I am not sure what other IAM roles/permissions I am missing here, it seems to work fine when I call it locally but not from the said staging environment.
My assumption would also be that this is an issue with your Lambda function in the VPC being unable to reach the public internet due to your VPC set up.
I would encourage you to review this blog to ensure you are set up correctly from a network perspective.
https://aws.amazon.com/premiumsupport/knowledge-center/internet-access-lambda-function/
I'm creating an Amazon Managed Airflow (MWAA) using CDK with the setting of webserver_access_mode='PRIVATE_ONLY'. In this mode, AWS creates a VPC interface endpoint and binds an IP address, from the selected VPC private subnets, to them as explained here: https://docs.aws.amazon.com/mwaa/latest/userguide/configuring-networking.html
Now, I want to use those IPs to add a listener to an existing load balancer that I can then use to connect to a VPN, but this doesn't seem to be available as an output attribute/property of aws_cdk.aws_mwaa.CfnEnvironment: https://docs.aws.amazon.com/cdk/api/v1/python/aws_cdk.aws_mwaa/CfnEnvironment.html#aws_cdk.aws_mwaa.CfnEnvironment.NetworkConfigurationProperty
My question is, is there a way to obtain those IPs associated with the aws_cdk.aws_mwaa.CfnEnvironment? Right now I am looking up the results manually after the deployment with CDK and creating the listener but I would prefer to fully automate it in the same CDK construct.
I struggled with this same problem for some time. In the end I used a Custom Resource in my CFN template, passing it the URL of the MWAA webserver. In the Python code associated with the Custom Resource (Lambda) I do a socket.gethostbyname_ex() call, passing the URL as an argument. This call will return a tuple that that you'll have to parse to extract the endpoint addresses.
I made good use of the crhelper libraries (https://aws.amazon.com/blogs/infrastructure-and-automation/aws-cloudformation-custom-resource-creation-with-python-aws-lambda-and-crhelper/), which made things a lot easier.
In the end, I used a lambda function to resolve the webserver URL and register the IP addresses to the target group. The approach is described in the following AWS blog post: https://aws.amazon.com/blogs/networking-and-content-delivery/hostname-as-target-for-network-load-balancers/
The implementation of the lambda function is also available through the following AWS sample code: https://github.com/aws-samples/hostname-as-target-for-elastic-load-balancer
I have a Lambda function that has a Python handler that submits a job to AWS Batch via boto3 client:
client = boto3.client('batch', 'us-east-1')
def handle_load(event, context):
hasher = hashlib.sha1()
hasher.update(str(time.time()).encode())
job_name = f"job-{hasher.hexdigest()[:10]}"
job_queue = os.environ.get("job_queue")
job_definition = os.environ.get("job_definition")
logger.info(f"Submitting job named '{job_name}' to queue '{job_queue}' "
f"with definition '{job_definition}'")
response = client.submit_job(
jobName=job_name,
jobQueue=job_queue,
jobDefinition=job_definition,
)
logger.info(f"Submission successful, job ID: {response['jobId']}")
I can see this Lambda function submit the Batch job in CloudWatch logs but it always times out before the response comes back. I never see these jobs show up in the queue, so I'm not sure where things go after they are submitted, it seems that the Lambda is always timing out before the response comes back, I have little else to go on.
I have successfully added a job to the queue via AWS CLI, using the same queue and definition ARNs that are used in the Lambda's Python code. This job can be seen in the queue under the runnable tab (presumably the job will be started at some point in the near future).
The job submission with AWS CLI comes back instantly, so there must be something amiss on the Lambda configuration preventing the job submission. Perhaps I'm not using the correct role for the Lambda that submits the job, or have some other permissions that are amiss causing the timeout? The Lambda has permission for the batch:SubmitJob action allowed on all resources.
If an AWS Lambda function is not connected to a VPC, then by default it is connected to the Internet. This means it can call AWS API functions, which resides on the Internet.
If your Lambda function is configured to use a VPC, it will not have Internet access by default. This is good for connecting to other resources in a VPC, but if you wish to communicate with an AWS service, you'll either:
A NAT Gateway in a public subnet, with the Lambda function connected to a private subnet that has a Route Table rule that points to the NAT Gateway, or
A VPC endpoint that connects to the desired service. Unfortunately, AWS Batch does not have a VPC Endpoint.
So, if your Lambda function does not need to connect to other resources in the VPC, you can disconnect it and it should work. Otherwise, use a NAT Gateway.
Based on the comments. Lambda in a VPC does not have access to internet. You need to setup internet gateway in public subnet and NAT gateway in private subnet with your lambda to be able to access AWS Batch endpoints. Alternatively have to use VPC interface endpoint for AWS Batch. From docs:
Connect your function to private subnets to access private resources. If your function needs internet access, use NAT. Connecting a function to a public subnet does not give it internet access or a public IP address.
Also you need to add permissions to your lambda's execution role to be able to create network interface in VPC:
ec2:CreateNetworkInterface
ec2:DescribeNetworkInterfaces
ec2:DeleteNetworkInterface
I have two simple lambda functions. Lambda 1 is invoking lambda 2 (both do a simple print for text).
If both lambdas are outside of a VPC then the invocation succeeds, however as soon as I set them both in to access a VPC (I need to test within a VPC as the full process will be wtihin a VPC) the invocation times out.
Do I have to give my lambda access to the internet to be able invoke a second lambda within the same VPC?
If your lambda functions are inside a VPC you need to configure your both lambda functions into private subnet not public subnet. That is the AWS recommended way.
If you are invoking the second Lambda from the first using Amazon API Gateway, then your Lambda will need to have access to the internet. Follow this guide to configure a NAT Gateway (last step).
regarding the VPC: in order to connect to your VPC and access resources there, the Lambdas must reside in the same region as your VPC and also be configured access to your VPC.
Please follow the steps provided in this AWS Guide: Configuring a Lambda Function to Access Resources in an Amazon VPC. This guide advises to use AWS CLI commands to do this and does not show how to configure it through the console.
You will need to be familiar with Amazon networking particulars (VPCs, Security Groups and Subnets), IAM security for the VPC and have a CLI environment setup. You are going to grant the Lambda Function access to this VPC using IDs and IAM execution roles via the CLI.
I use aws lambda to perform custom actions as Auto Scaling terminates instances. It looks like this
def scaledown_handler(event, context):
# customs actions
client = boto3.client('autoscaling')
response = client.complete_lifecycle_action(LifecycleHookName=event['detail']['LifecycleHookName'],
LifecycleActionToken=event['detail']['LifecycleActionToken'],
AutoScalingGroupName=event['detail']['AutoScalingGroupName'],
LifecycleActionResult='CONTINUE',
InstanceId=event['detail']['EC2InstanceId'])
The problem is that the function just hangs on client.complete_lifecycle_action() and finishes by timeout without any response and my ec2 instances are always "Waiting for Terminate Lifecycle Action".
aws autoscaling complete-lifecycle-action in aws CLI works fine, but i need to be done this from AWS lambda. How can I find out why does complete_lifecycle_action() hang without a response?
If you don't have a NAT gateway in your VPC then the Lambda function won't have access to anything outside the VPC. The AWS API exists outside your VPC, so the Lambda function is getting a network timeout trying to access it.
You have to add a NAT Gateway to your VPC in order for Lambda functions (and other things in your VPC that don't have a public IP) to access anything outside the VPC.
You need to use put_lifecycle_hook() API.
http://www.callumpember.com/auto-scaling-lifecycle-hooks/
On this link, you can get the complete python script for executing the custom actions before terminating the instance.