I am trying to upload a python lambda function via a zip file. I copied all of my python files into a directory called "lambda", installed all of the libraries directly into that directory, did chmod -R 755 . to make sure all of those files are executable, and then zipped the directory with zip -r ../analysis.zip .
The file that holds my lambda function is called "main.py" and the lambda function is called "handler", so by AWS Lambda convention, I set the file it should be looking for to main.handler in the AWS Lambda page. I check my cloudwatch logs for this lambda function and still get an error saying aws cannot find the main module and also cannot find some regex._regex module as well.
This is the official error:
[ERROR] Runtime.ImportModuleError: Unable to import module 'main': No module named 'regex._regex'
Does anyone know what can be the problem? I have deployed aws lambda functions before using the same process and this is the first time I am getting this issue.
Lambda operates on a Python function/method not at the file. So the function handler must point to a actual function/method not a file.
So within your main.py file, there must be a function e.g. test_function, and your handler has to be main.test_function. The name of the function in AWS is irrelevant to the function.
Hope that helps.
By the description and error message your naming convention seems right.
The error message indicates that you're missing the regex module. However, from your description you seem to have packaged the dependancies correctly.
Usually, it would then be a case of a runtime miss match. However, I have had struggles with regex and lambda when runtimes do match. By default now, I don't go above python 3.6 at the moment. I have struggled with other dependancies on lambda, such as pickle, on higher versions recently. Whilst everything seems to operate fine on 3.6.
I got rid of the regex error on lambda with python 3.6 by downloading the taar.gz file from pypi and running setup.py... rather than pip3 install. It's a bit of pain, but it worked.
Related
I am trying to import python-dependency-injector package within lambda function.
To run lambda function, I have zipped my project contents together with packages in /opt/anaconda3/envs/.../python3.9/site-packages to deploy a fast-api(with mangum) app.
Use of any other packages works fine, but weirdly using this package (written in Cython) results in below error :\
{"errorMessage": "Unable to import module 'main': cannot import name 'providers' from 'dependency_injector' (/var/task/dependency_injector/init.py)", "errorType": "Runtime.ImportModuleError", "requestId": "5c63c01b-5be1-4481-adf8-691167cb54bd", "stackTrace": []}
I am not sure whether this is a known issue when using packages written in Cython or there are any workarounds.
I have also tried manually importing the package (that was in site-packages) within a newly created EC2 (AWS linux) and also my local (Mac)
Without importing this specific package, everything else works fine.
Would really appreciate if someone can guide me how to resolve this issue.
This has been already asked a number of times, but I have tried everything suggested plus more, and nothing seems to work.
My setup: an application on Lambda, with python functions and deployed via CloudPipeline. This is the full error I get (and all I can see in the logs):
{
"errorMessage": "Unable to import module 'lambda_functions.function_one': No module named 'lambda_functions'",
"errorType": "Runtime.ImportModuleError"
}
lambda_functions is a directory, and function_one is the name of the python file with the handler in it. The full function invocation path in my template.yml is: lambda_functions.function_one.lambda_handler. I do have a __init__.py in that dir.
I installed the AWS SAM tools and I can invoke the function locally fine. I have also downloaded the zipped project from S3 and checked permissions etc.
The logs show that the requirements are installed correctly, but even then, just to make sure I tried commenting out everything in the function file except for a bare handler, no dependencies at all, and still fails.
Any ideas on why Lambda can fail to find my module?
I do not think it is supported to put the file in a subdirectory, so you probably have to make sure that the file function_one.py file is in the root of the zip file.
I think posting the question here served as rubber duck debugging! After hours trying to make it work, I have just found the problem: the Code URI entry in the template was pointing to a full path to a S3 bucket, like:
s3://aws-eu-west-2-575-foo-foo-pipe/622d85cc8
But I just need a ./, like this:
Resources:
getAllItemsFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: ./
Handler: function_one.lambda_handler
I hope this helps someone else and they don't have to spend hours trying to debug this. Lambda is nice when it all works well, but man it is hard to debug.
I am working on a project that is using AWS CodeBuild to deploy a Serverless (SLS) function that is written in Python.
The deployment works fine within code build. It successfully creates the function and I can view the lambda within the Lambda AWS UI. Whenever the function is triggered, I get the error seen below:
Runtime.ImportModuleError: Unable to import module 'some/function': attempted relative import with no known parent package
It is extremely frustrating as I know the function exists at that directory listed above. During the CodeBuild script, I can ls into the directory and confirm that it indeed exists. The function is defined in my serverless.yml file as follows:
functions:
file-blaster:
runtime: python3.7
handler: some/function.function_name
events:
- existingS3:
bucket: some_bucket
events:
- s3:ObjectCreated:*
rules:
- prefix: ${opt:stage}/some/prefix
Sadly, I haven't been able to crack this one. Has anyone had a similar experience while working with SLS and python in the cloud?
It seems odd that SLS would build and deploy successfully, but the Lambda itself cant find the function.
This will be a short answer for what is a somewhat longer discussion on Python imports. You can do the research yourself on the hectic and confusing battle between relative and absolute imports as a design for a python project.
The Gist:
It is necessary to understand that the base of the python importing for SLS functions IS where the serverless.yml file exists (I imagine that it is similar to having a main.py that calls the other files that are referenced as "functions" in the sls yml). For my case above, I did not structure the imports using absolute imports when I had my issues. I switched all of my imports to have absolute paths, so when I moved the package around, it would continue to work.
The error that I was given Runtime.ImportModuleError: Unable to import module 'some/function': attempted relative import with no known parent package was really poor to describe the actual issue. The error should have included that the packages being used by some/function were not found when attempting a relative import because that was the actual problem that needed fixing.
Hopefully this helps someone else out someday. Let me know if I can provide more information where I haven't already.
I think you need to change your handler property from :
handler: some/function.function_name
to
handler: some/function.{lambda handler name}
like, my folder structure is:
- some
- function1.py
then my template will be:
functions:
file-blaster:
runtime: python3.7
handler: some/function1.lambda_handler
for more details check here https://serverless.com/framework/docs/providers/aws/guide/functions/
My Python script has been tested on multiple local computers and works as expected. But when the script is zipped up and uploaded to AWS Lambda I get Runtime.ImportModuleError: Unable to import module 'testsdk': No module named 'jsonpickle' when I test the code. testsdk is the desired python file that should be executed.
The python version on my local PC is 3.7.0 which is the same Python version as AWS Lambda.
Any help with this would be much appreciated!
I faced the same issue and solved it by correcting the directory structure.
Inside the lambda function are two equal folder names. I fixed the issue by deleting the first folder.
I followed this to set up twilio: https://www.fullstackpython.com/blog/send-sms-text-messages-python.html
The imports seem to be working when I run locally using python send_sms.py
Then, I use Apache Nifi ExecuteScript processor to execute the send_sms.py file and assume it should be same as if I am running the file locally.
It shows me the error:
error when calling the metaclass bases function() argument 1 must be code not str
When I am trying to: from twilio.rest import TwilioRestClient.
Twilio was installed at path /sendsms/lib/python2.7/site-packages, so I set the Module Directory to this path
Does anyone know what is wrong here? I am really stuck and please help.
ExecuteScript uses Jython (not Python) to execute pure Python scripts, and as such any imported packages (and their dependencies) must be pure Python modules as well. I am guessing that TwilioRestClient (or its dependencies) include a non-pure Python module (compiled C, e.g.). For these cases, Jython (and thus ExecuteScript) will not work.
An alternative is to use the ExecuteStreamCommand processor, with which you can shell out to your Python interpreter (and script).