I'm developing a lambda function that I write in DynamoDB. On one hand I have created a layer that has a script with the functions of DynamoDB:
class DynamoHandler():
def __init__(self):
self.resource = boto3.resource('dynamodb', region_name = 'eu-west-1')
self.__table = None
def set_table(self, table_name: str):
table = self.resource.Table(table_name)
table.table_arn
self.__table = table
def insert(self, item, **kwargs):
self.__check_table()
return self.__table.put_item(
Item=item,
**kwargs
)
In the lambda I write the following code:
from dynamo_class import DynamoHandler
db = DynamoHandler()
db.set_table(TABLE NAME)
db.insert(msg)
And I get the error:
[ERROR] EndpointConnectionError: Could not connect to the endpoint URL: "https://dynamodb.eu-west-1.amazonaws.com/"
Do you know how can I solve this problem?
I have searched for similar errors but they occurred when the region was not specified, in my case in the DynamoDB class I assign the region "eu-west-1".
The timeout occurs most likely because lambda in a VPC has no internet nor public IP address. From docs:
Connecting a function to a public subnet doesn't give it internet access or a public IP address.
Subsequently, the lambda function can't connect to DynamoDB endpoint.
There are two ways to rectify the issue:
place the lambda in a private subnet and setup NAT gateway to enable lambda access the internet.
Use VPC Gateway for DynamoDB which would be better in this case, as for DynamoDB gateway there are no extra charges.
In addition to the great answer by Marcin above, have you checked that the Security Group associated with the function has the correct egress rules that allow the network interface to connect to either the DynamoDB or its NAT gateway?
Related
I am getting a mysterious error of Tasked timing out as shown in the screenshot I attached
Here I tried to connect with my rds database with boto3 provided all the required params like secret keys arn etc but still, nothing seems to work.Connected with proper vpc's and given role to lambda with all required policies
here is the code on lamda
import json
import boto3
rds_data = boto3.client('rds-data')
database_name = ''
db_clust_arn = ''
db_secret_arn = ''
def lambda_handler(event, context):
sql = """"
response = rds_data.execute_statement(
resourceArn = db_clust_arn,
secretArn = db_secret_arn,
database = database_name,
sql = sql,
)
print(str(response))
enter code here
You need to use the RDS Lambda Proxy.
When you put your lambda in the VPC it won't have access to make HTTP calls by default so the request will never be routed to RDS, even though they are in the same VPC. That's because Lambda doesn't have an ENI attached which can route your requests.
I have an RDS instance my_instance and a lambda. Both sit on the same VPC.
The RDS has a SC my-security-group-rds and lambda another security group my-security-group-lambda.
my-security-group-lambda has outbound rule without constraints. All protocols with IP 0.0.0.0/0.
my-security-group-rds has an inbound rule that allows any access from my-security-group-lambda.
Inside the lambda I want to perform DNS resolution of the hostname:
def dns_lookup(domainname,record_type,*dnsserver):
"""
Get dns lookup results
:param domain:
:return: list of dns lookup results
"""
try:
lookup_result_list = []
myResolver = dns.resolver.Resolver()
if not dnsserver:
lookupAnswer = myResolver.query(domainname, record_type)
else:
myResolver.nameservers = dnsserver[0]
lookupAnswer = myResolver.query(domainname,record_type)
for answer in lookupAnswer:
lookup_result_list.append(str(answer))
return lookup_result_list
except Exception as err:
logger.exception("Not able to lookup DNS:{}".format(err))
raise
I first perform the dns_lookup on the rds regional server, which works out nicely.
listOfAuthoritiveNames = dns_lookup("eu-west-1.rds.amazonaws.com", "NS")
Then I try to perform dns lookup on my instance:
dns_lookup("my-database.dfafda.eu-west-1.rds.amazonaws.com", "A", listOfAuthoritiveNames)
This gives a timeout. If I use a wrong hostname for the database it says that there is no record, so it would seem that it has access to the nameservers.
Any idea what might be missing?
I'm trying to fetch my bookings collection from AWS Document db cluster. I've maintained the credentials using AWS Secret Manager and passing them to connect. However, it gives me a Task Timed Out Error of 900 seconds. (I increased the time limit to 15 mins since it gave the same error with lesser duration)
The error is as such:
{
"errorMessage": "2021-08-19T09:05:22.872Z a96e95cb-4c42-4880-b339-9cb29e83c1ec Task timed out after 900.10 seconds"
}
Code snippet:
def lambda_handler(event, context):
db = create_mongo_connection()
print(db)
print("aaaaa") # this gets printed -- debugging
bookings = db.bookings.find({}) # bookings collection not fetched
print("bbbbb") # this does not get printed -- debugging
#configuration settings maintained in environment variables
mongoconfig = os.environ['mongoconfig']
def create_mongo_connection():
try :
secretsmanager = get_secret()
SecretString = json.loads(secretsmanager)
username = SecretString['username']
password = SecretString['password']
host = SecretString['host']
port = SecretString['port']
mongoclient = MongoClient(host, port, username=username, password=password,
authSource='admin', ssl_ca_certs='rds-combined-ca-bundle.pem',retryWrites='false')
dbname = mongoconfig['db_name']
print(dbname)
return mongoclient[dbname]
except Exception as e:
print("Exception : ", e, "\nTraceback : ", format_exc())
The API endpoints for the Secrets Manager live on the Internet. It sounds like the Lambda function is not able to access the Internet.
When an AWS Lambda function is connected to a VPC, it can access resources in the VPC. However, to access the Internet:
The Lambda function must be in a private subnet, and
A NAT Gateway must be in a public subnet
If the Lambda function does not require access to resources in the VPC, then simply disconnect the Lambda function from the VPC and it will receive Internet access automatically.
There is another option, which is to Use Secrets Manager with VPC endpoints - AWS Secrets Manager, which creates a tunnel between a VPC and the Secrets Manager.
I have published a Lex bot and a Lambda function on the same region. I am trying to interact with Lex from Lambda using following code.
import boto3
client = boto3.client('lex-runtime')
def lambda_handler(event, context):
response = client.post_text(
botName='string',
botAlias='string',
userId='string',
# sessionAttributes={
# 'string': 'string'
# },
# requestAttributes={
# 'string': 'string'
# },
inputText='entity list'
)
return response
While testing the code my lambda function is getting timed out. Kindly let me know if you need to know anything else.
Error Message:
"errorMessage": "2021-04-30T07:09:45.715Z <req_id> Task timed out after 183.10 seconds"
Lambda in a VPC, including a default VPC, does not have internet access. Since your lambda in a default VPC, it will fail to connect to any lex-runtime AWS endpoint, resulting in a timeout.
Since lex does not support VPC endpoints, the only way for your lambda to access the lex-runtime is:
disassociated your function from VPC. If you do this, your function will be able to connect to the internet, and subsequently to lex-runtime.
create a private subnet in a default VPC, and setup a NAT gateway. Then place your function in the new subnet. This way your function will be able to connect to the internet using NAT.
More info is in:
How do I give internet access to a Lambda function that's connected to an Amazon VPC?
My AWS Lambda function code works fine when I run it outside of an Amazon Virtual Private Cloud (Amazon VPC). However, when I configure my function to connect to a VPC, I get function timeout errors. How do I fix these?
def get_db_connection_config():
# Create a Secrets Manager client.
session = boto3.session.Session()
client = session.client(
service_name='secretsmanager',
region_name=region_name
)
# In this sample we only handle the specific exceptions for the 'GetSecretValue' API.
# See https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html
# We rethrow the exception by default.
try:
logger.info("Retrieving MySQL database configuration...")
get_secret_value_response = client.get_secret_value(
SecretId=secret_name
)
except ClientError as error:
logger.error(error)
sys.exit()
else:
# Decrypts secret using the associated KMS CMK.
# Depending on whether the secret is a string or binary, one of these fields will be populated.
if 'SecretString' in get_secret_value_response:
secret = get_secret_value_response['SecretString']
return json.loads(secret)
else:
return base64.b64decode(get_secret_value_response['SecretBinary'])
When a Lambda resides in AWS network it is able to use the internet to connect to these services, however once it joins your VPC outbound internet traffic is also routed through your VPC. As there is presumably no outbound internet connectivity the Lambda is unable to reach the internet.
If your function needs internet access, use network address translation (NAT). Connecting a function to a public subnet doesn't give it internet access or a public IP address.
For your Lambda to be able to communicate with other AWS services when it resides within a VPC, one of the following must be in place.
The first option is that you create either a NAT gateway or a NAT instance, and then add this to the route table which your Lambda resides in. To be clear this subnet should be a private subnet only as by utilizing a NAT for a 0.0.0.0/0 record it will stop inbound traffic to instances which have a public IP address that share the same subnet.
The second option is that you utilize VPC endpoints for the services, by doing this any traffic that would have previously traversed the public internet will instead use a private connection directly to the AWS service itself. Please note that not every AWS service is covered yet for this.