Because Google Functions does not support Python, I was hoping to use google-cloud-datastore in AWS Lambda but hit the same error as
AWS Lambda to Firestore error: cannot import name 'cygrpc'
google-cloud-storage works just fine in Lambda so core packaging is not the issue but could pip'ing datastore miss some dependencies ?
setting env.var GOOGLE_CLOUD_DISABLE_GRPC=true does not help as error occurs at import itself. I wonder if this is not a design flaw: you would want to import any gRPC-related libs only if enabled (which is the default). Seems cygprc is being loaded regardless.
I could try downgrade the datastore module to a version which only supported the http api but not sure which one - and that would require most likely to change storage version too
If we managed to get datastore lib to work, our next attempt will be bigquery - should work as doesn't use gRPC (yet?)
Related
I am trying to scrape one website stuck with "errorMessage": "Unable to import module 'lambda_function': No module named 'requests'",
How to import any module in AWS lambda which throw the Unable to import error.
Disclaimer : I am not running on EC2 instance
i need to put x.text in the s3 bucket
code is below
import requests
x = requests.get('https://w3schools.com/python/demopage.htm')
print(x.text)
You have 2 choices for packaging extra python dependencies:
Run a pip install and zip the contents along with your Lambda function. Upload this Zip to your Python function.
Create a Lambda layer that you can then use with your Lambda function.
Regarding requests it is no longer included in the base Lambda setup.
AWS have a blog post that explains how to include these files in your codebase.
It also includes the AWS Arns for public Labda layers containing the requests dependency, although it does cap the SDK at a slightly older version.
In order to use external packages in AWS Lambda, as stated in official documentation, you should pack your dependencies along with your code and upload it all together.
I wish to create issues in JIRA from an AWS Lambda function. I am getting stuck on the first couple of steps for OAuth where I need to use JIRA's AppLinks to pair with an application. Do I need to build a custom client? If so, how? If not, what do I need to do?
Any help is appreciated, thanks!
I found this question via Google looking for an answer to the same problem. I'm adding an answer in case someone else stumbles upon this question looking for an answer.
The jira python package does not include cryptography which you'll need to add as a pip requirement.
pycrypto - This is required for the RSA-SHA1 used by OAuth. Please note that it’s not installed automatically, since it’s a fairly cumbersome process in Windows. On Linux and OS X, a pip install pycrypto should do it.
https://jira.readthedocs.io/en/master/installation.html#dependencies
Depending on your platform it may not be as simple as adding the above and deploying due to native code requirements
cryptography contains native code and that code is compiled for the architecture (and OS) of the current machine. The AWS lambda documentation currently appears to claim that zipping up your local directory will work, but that's patently untrue.
https://github.com/pyca/cryptography/issues/3051#issuecomment-234093766
To resolve you can compile your function's requirements on an EC2 AMI or docker container.
If you are using serverless you can use the serverless-python-requirements plugin which is what I used to resolved these issue for me.
In your serverless.yml file you can specify compiling packages via docker.
custom:
pythonRequirements:
dockerizePip: true
found via https://www.npmjs.com/package/serverless-python-requirements#cross-compiling
AWS Lambda is showing me 'no module import error' for package pyodbc in my lambda function.
I've been using another library without error like this only getting error for this 'pyodbc' lib.
I've added pyodbc installing with pip to my python code directory and upload
them in zip to aws lambda.
You would need to ensure that the native libraries that pyodbc requires are present in the Lambda execution environment - either bundled as part of your deployment package or as a separate Lambda layer.
See https://medium.com/#narayan.anurag/breaking-the-ice-between-aws-lambda-pyodbc-6f53d5e2bd26 for more details about this.
I'm trying to use AWS Lambda to transfer data from my S3 bucket to Couchbase server, and I'm writing in Python. So I need to import couchbase module in my Python script. Usually if there are external modules used in the script, I need to pip install those modules locally and zip the modules and script together, then upload to Lambda. But this doesn't work this time. The reason is the Python client of couchbase works with the c client of couchbase: libcouchbase. So I'm not clear what I should do. When I simply add in the c client package (with that said, I have 6 package folders in my deployment package, the first 5 are the ones installed when I run "pip install couchbase": couchbase, acouchbase, gcouchbase, txcouchbase, couchbase-2.1.0.dist-info; and the last one is the c client of Couchbase I installed: libcouchbase), lambda doesn't work and said:
"Unable to import module 'lambda_function': libcouchbase.so.2: cannot open shared object file: No such file or directory"
Any idea on how I can get the this work? With a lot of thanks.
Following two things worked for me:
Manually copy /usr/lib64/libcouchbase.so.2 into ur project folder
and zip it with your code before uploading to AWS Lambda.
Use Python 2.7 as runtime on the AWS Lambda console to connect to couchbase.
Thanks !
Unfortunately AWS Lambda does not support executing C-based python modules, like the Couchbase SDK.
Your best bet would be to use a pure-python client. The easiest way to do this would be to use the unofficial memcached client https://github.com/couchbase/couchbase-cli/blob/master/cb_bin_client.py which uses server-side moxi to handle memcached clients on port 11211.
I am deploying a Django app on Google App Engine, but I get an error when importing django-widget-tweaks:
appcfg.py: error: Error parsing ./app.yaml: the library "django-widget-tweaks" is not supported
Is there any way to fix this, apart from not using the library?
You can install third-party libraries yourself. Since this is not one of the runtime-provided third-party libraries you have to fulfill the following criteria:
The library must be implemented as pure Python code (no C extensions).
The code is uploaded to App Engine with your application code, and
counts toward file quotas.
Use pip to install the library and the
vendor module to enable importing packages from the third-party
library directory
From what I can tell the module you want to use is fully implemented in Python, so this should be straight forward. Consult the docs on vendoring for more information on how to do this.