I try the example on: https://github.com/Azure-Samples/ms-identity-python-webapp with Windows 10, but I get an error with ModuleNotFoundError: No module named 'flask_caching.backends.filesystem' (Flask-Caching is already installed with pip).
Version:
Python 3.9.9,
Flask 1.1.4 and
Werkzeug 1.0.1.
I only changed the code with Client_ID, CLient_Secret and domain name in app_config.py.
Has anybody an idea?
The error ModuleNotFoundError means python interpreter cannot find the libraries which you are referring to in the code although the module is already installed.
Common causes of this error:
Using modules meant for different python versions but Installing python 2.x modules in python 3.x and vice a versa.
When not properly setting PATH variable.
(Or)
If you are using a python virtual environment. It need to be installed after creating a virtual environment as commented by #grumpyp . The libraries will reside inside the folder created for the virtual environment.
And can installed according to requirements.txt file
pip install virtualenv
It requires activation and dedicated installation of modules inside the virtual environment.
Refer this blog for more details to do
pip install -r requirements.txt
Other reference :Set Up a Virtual Python Environment (Windows)
(or )
This may not be your query but Just to make it a bit easy You can try this way when trying out your sample project to compare with manually configured one.
The quick start: "Add sign-in with Microsoft to a Python web app" that you are using ,can be directly configured in portal quickstart like below where every thing is configured including client id ,tenant id etc directly.
Just register the app with name and account type and follow the steps below for direct configuration .
Go to quickstart page of app
Select Python as platform for web application
Just follow the steps to configure azure ad inside app directly
There after following the steps , I checked the versions with pip freeze and
versions i have: Python 3.9.7, Flask 1.1.4 and Werkzeug 1.0.1.
quickstart-v2-python-webapp | microsoftdocs
I am learning how to use Azure functions and using my web scraping script in it.
It uses BeautifulSoup (bs4) and pymysql modules.
It works fine when I tried it locally in the virtual environment as per this MS guide:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-create-first-azure-function-azure-cli?pivots=programming-language-python&tabs=cmd%2Cbrowser#run-the-function-locally
But when I create the function App and publish the script to it, Azure Functions logs give me this error:
Failure Exception: ModuleNotFoundError: No module named 'pymysql'.
It must happen when attempting to import it.
I really don't know how to proceed, where should I specify what modules it needs to install?
You need to check if you have generated the requirements.txt which includes all of the information of the modules. When you deploy the function to azure, it will install the modules by the requirements.txt automatically.
You can generate the information of modules in requirements.txt file by the command below in local:
pip freeze > requirements.txt
And then deploy the function to azure by running the publish command:
func azure functionapp publish hurypyfunapp --build remote
For more information about deploy python function from local to auzre, please refer to this tutorial.
By the way, if you use consumption plan for your python function, the "Kudu" is not available for us. If you want to use "Kudu", you need to create app service plan for it but not consumption plan.
Hope it helps~
You need to upload the installed modules when deploying to azure. You can upload them using Kudu:
https://github.com/projectkudu/kudu/wiki/Kudu-console
as an alternative, you can also use Kudu and run pip install using the console:
Install python packages from the python code itself with the following snippet: (Tried and verified on Azure functions)
def install(package):
# This function will install a package if it is not present
from importlib import import_module
try:
import_module(package)
except:
from sys import executable as se
from subprocess import check_call
check_call([se,'-m','pip','-q','install',package])
for package in ['beautifulsoup4','pymysql']:
install(package)
Desired libraries mentioned the list gets installed when the azure function is triggered for the first time. for the subsequent triggers, you can comment/ remove the installation code.
I download (pip install pysftp) and make a zip file and upload in a lambda function.
but it is not working in a lambda function. throwing error.
Response:
{
"errorMessage": "Unable to import module 'lambda_function': cannot import name '_bcrypt' from 'bcrypt' (./lib/bcrypt/__init__.py)",
"errorType": "Runtime.ImportModuleError"
}
Many thank you in advance.
The operation "pip install pysftp" is to be performed in a linux distribution for preparing the lib for aws lambda. I had used ubuntu in docker on windows to perform pip install on mounted volume to generate the lib.
Try reinstalling the packages and uploading the new packages. If it still shows error, move your development environment from Windows to Linux.
Similar kind of error for your reference: [1]: https://forums.aws.amazon.com/thread.jspa?messageID=804753&tstart=0
Since you need to troubleshoot module dependencies, the python runtime environment of AWS Lambda must be inspected.
In your AWS Lambda, print the modules that are loaded, and therefore available to the other modules that your code imports.
def lambda_handler(event, context):
print (help("modules"))
Running this in a python interpreter is illuminating.
python
help("modules")
You will see Please wait a moment while I gather a list of all available modules... and then a big list of available modules that are importable.
You will find that you are missing bcrypt, for within that module as taught by help(bcrypt) you will find the missing dependency _bcrypt.
Should bcrypt be available to the lambda or just a python interpreter, it is found in this manner.
>>> bcrypt._bcrypt
<module 'bcrypt._bcrypt' from '/usr/local/lib/python2.7/site-packages/bcrypt/_bcrypt.so'>
To install everything in the same directory, use the below command
pip install pysftp -t .
After that, zip the entire dir & upload to Lambda console.
Little old trick that helpssfor those who using Lambda for the 1st time...
I'm trying to install and run pandas on an Amazon Lambda instance. I've used the recommended zip method of packaging my code file model_a.py and related python libraries (pip install pandas -t /path/to/dir/) and uploaded the zip to Lambda. When I try to run a test, this is the error message I get:
Unable to import module 'model_a': C extension:
/var/task/pandas/hashtable.so: undefined symbol: PyFPE_jbuf not built.
If you want to import pandas from the source directory, you may need
to run 'python setup.py build_ext --inplace' to build the C extensions
first.
Looks like an error in a variable defined in hashtable.so that comes with the pandas installer. Googling for this did not turn up any relevant articles. There were some references to a failure in numpy installation but nothing concrete. Would appreciate any help in troubleshooting this! Thanks.
I would advise you to use Lambda layers to use additional libraries. The size of a lambda function package is limited, but layers can be used up to 250MB (more here).
AWS has open sourced a good package, including Pandas, for dealing with data in Lambdas. AWS has also packaged it making it convenient for Lambda layers. You can find instructions here.
I have successfully run pandas code on lambda before. If your development environment is not binary-compatible with the lambda environment, you will not be able to simply run pip install pandas -t /some/dir and package it up into a lambda .zip file. Even if you are developing on linux, you may still run into compatability issues.
So, how do you get around this? The solution is actually pretty simple: run your pip install on a lambda container and use the pandas module that it downloads/builds instead. When I did this, I had a build script that would spin up an instance of the lambci/lambda container on my local system (a clone of the AWS Lambda container in docker), bind my local build folder to /build and run pip install pandas -t /build/. Once that's done, kill the container and you have the lambda-compatible pandas module in your local build folder, ready to zip up and send to AWS along with the rest of your code.
You can do this for an arbitrary set of python modules by making use of a requirements.txt file, and you can even do it for arbitrary versions of python by first creating a virtual environment on the lambci container. I haven't needed to do this for a couple of years, so maybe there are better tools by now, but this approach should at least be functional.
If you want to install it directly through the AWS Console, I made a step-by-step youtube tutorial, check out the video here: How to install Pandas on AWS Lambda
I got this error in Google App Engine's Python have used Google Translate API,
But I don't know how to fix,
<module>
from apiclient.discovery import build
ImportError: No module named apiclient.discovery
I'll try to set environment which indicates to Google App Engine SDK,
And upload to Google Apps Engine again, always get the error,
Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this error message and the query that caused it.
Please tell me how to fix,
Thanks
UPDATE : Fixed
Follow Nijjin's help,
I fixed problems by adding the following folders,
apiclient, gflags, httplib2, oauth2client, uritemplate
If you still got problem, please consider below Answer of this page to get more info. ex. : Varum answer, etc ...
You should be able to get these dependencies with this simple install:
sudo pip install --upgrade google-api-python-client
This is described on the quick start page for python.
apiclient was the original name of the library.
At some point, it was switched over to be googleapiclient.
If your code is running on Google App Engine, both should work.
If you are running the application yourself, with the google-api-python-client installed, both should work as well.
Although, if we take a look at the source code of the apiclient package's __init__.py module, we can see that the apiclient module was simply kept around for backwards-compatibility.
Retain apiclient as an alias for googleapiclient.
So, you really should be using googleapiclient in your code, since the apiclient alias was just maintained as to not break legacy code.
# bad
from apiclient.discovery import build
# good
from googleapiclient.discovery import build
If none of the above solutions work for you, consider if you might have installed python through Anaconda. If this is the case then installing the google API library with conda might fix it.
Run:
python --version
If you get something like
Python 3.6.4 :: Anaconda, Inc.
Then try:
conda install google-api-python-client
As bgoodr has pointed out in a comment you might need to specify the channel (think repository) to get the google API library. At the time of writing this means running the command:
conda install -c conda-forge google-api-python-client
See more at https://anaconda.org/conda-forge/google-api-python-client
apiclient is not in the list of third party library supplied by the appengine runtime: http://developers.google.com/appengine/docs/python/tools/libraries27 .
You need to copy apiclient into your project directory & you need to copy these uritemplate & httplib2 too.
Note: Any third party library that are not supplied in the documentation list must copy to your appengine project directory
Make sure you only have google-api-python-client installed. If you have apiclient installed, it will cause a collision. So, run the following:
sudo pip uninstall apiclient
I fixed the problem by reinstalling the package with:
pip install --force-reinstall google-api-python-client
For app engine project you gotta install the lib locally by typing
pip install -t lib google-api-python-client
read more here
There is a download for the Google API Python Client library that contains the library and all of its dependencies, named something like google-api-python-client-gae-<version>.zip in the downloads section of the project. Just unzip this into your App Engine project.
for python3 this worked for me:
sudo pip3 install --upgrade google-api-python-client
I had the same problem because of a bug in the installation of the URITemplate module.
This solved the problem:
pip install --force-reinstall uritemplate.py
I got this same error when working on a project to parse recent calendar events from Google Calendar.
Using the standard install with pip did not work for me, here is what I did to get the packages I needed.
Go directly to the source, here is a link for the google-api-python-client, but if you need a different language it should not be too different.
https://github.com/google/google-api-python-client
Click on the green "Clone or Download" button near the top left and save it as a zip file. Move the zip to your project folder and extract it there. Then cut all the files from the folder it creates back into the root of your project folder.
Yes, this does clutter your work space, but many compilers have ways to hide files.
After doing this the standard
from googleapiclient import discovery
works great.
Hope this helps.
"google-api-python-client" requires:
pip install uritemplate.py
to fix problem on GAE Development Server:
from googleapiclient.discovery import build
ImportError: No module named googleapiclient.discovery
I installed google-api-python-client using pip but it is still showing me error so I try upgrading it and it help me to get out of error
if you are using "windows" then
pip install --upgrade google-api-python-client
will help you because it help me so it will help you too :)
I encountered the same issue.
This worked:
>>> import pkg_resources
>>> pkg_resources.require("google-api-python-client")
[google-api-python-client 1.5.3 (c:\python27), uritemplate 0.6 (c:\python27\lib\site-packages\uritemplate-0.6-py2.7.egg), six 1.10.0 (c:\python27\lib\site-packages\six-1.10.0-py2.7.egg), oauth2client 3.0.0 (c:\python27\lib\site-packages\oauth2client-3.0.0-py2.7.egg), httplib2 0.9.2 (c:\python27\lib\site-packages\httplib2-0.9.2-py2.7.egg), simplejson 3.8.2 (c:\python27\lib\site-packages\simplejson-3.8.2-py2.7-win32.egg), six 1.10.0 (c:\python27\lib\site-packages\six-1.10.0-py2.7.egg), rsa 3.4.2 (c:\python27\lib\site-packages\rsa-3.4.2-py2.7.egg), pyasn1-modules 0.0.8 (c:\python27\lib\site-packages\pyasn1_modules-0.0.8-py2.7.egg), pyasn1 0.1.9 (c:\python27\lib\site-packages\pyasn1-0.1.9-py2.7.egg)]
>>> from apiclient.discovery import build
>>>
It only worked with me when I used sudo:
sudo pip install --upgrade google-api-python-client
I was getting the same error, even after following Google's guide at https://developers.google.com/drive/api/v3/quickstart/python, then I realized I had to invoke like this:
python3 quickstart.py
Instead of:
python quickstart.py <-- WRONG
(Note the "3")
Worked flawlessly.
I'm using Ubuntu 18.04.4 LTS.
use this
pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
The same error can be seen if you are creating a Python module and your executing the script after installing it via pip or pipx command.
In this case ensure you have declared what the project minimally needs to run correctly into install_requires section of your setup.py file, so in this case:
install_requires=[
"google-api-python-client>=1.12.3",
"google-auth-httplib2>=0.0.4",
"google-auth-oauthlib>=0.4.1"
]
This can also happen if the interpreter on your IDE is pointing to the wrong virtual environment. In VSCODE I've set it manually to the correct interpreter and my problem was solved.
(May 2021) It's been about 8 years since the original question, and since then, several product changes have occurred, so new developers arriving here looking to use the Google Translate API on Python App Engine have a few changes to make:
Product info/costs: The Google Translate API is now available as the Google Cloud Translation API. It's not free (meaning you need to create a billing account backed by a financial instrument like a credit card), but you get a quota of translated characters per month. See its pricing page for more info. Similarly, while you used to be able to create an App Engine app without a credit card, you can no longer do so via the new policy as of Nov 2019. It still has a generous "Always Free" tier quota which you must exceed in order to incur charges. Also see the App Engine pricing page for more info.
Client libraries: Rather than using apiclient or googleapiclient which are part of the Google APIs client library which is a low-level, multi-product, platform-level client library, we recommend the Google Cloud client libraries which are higher-level and product-focused. That means there's a specific Cloud Translation client library (actually two: basic/v2/Python 2 or advanced/v3/Python 3) — these are higher-level and much easier to use:
Add client lib: pip install -U pip google-cloud-translate (or pip3)
With it, your code sample can be as simple as:
'translate_demo.py - demo the Cloud Translation API'
from __future__ import print_function
import google.auth
from google.cloud import translate
TRANSLATE = translate.TranslationServiceClient()
_, PROJECT_ID = google.auth.default()
PARENT = 'projects/{}'.format(PROJECT_ID)
TARGET_LANG = 'es'
TEXT = 'Hello world'
DATA = {
'parent': PARENT,
'contents': [TEXT],
'target_language_code': TARGET_LANG,
}
try: # Python 3/advanced/v3
rsp = TRANSLATE.translate_text(request=DATA)
except TypeError: # Python 2/basic/v2
rsp = TRANSLATE.translate_text(**DATA)
print(TEXT, '=', rsp.translations[0].translated_text)
It also works on Python 2 and 3 without any modification:
$ python2 translate_demo.py
Hello world = Hola Mundo
$ python3 translate_demo.py
Hello world = Hola Mundo
This code snippet can be adapted for App Engine fairly easily (more below), especially if you're prototyping since you can take advantage of the default service account so you don't have to muck around with service accounts, like making a new one, creating a public/private key-pair, and having to download the JSON credentials file and pointing the GOOGLE_APPLICATION_CREDENTIALS environment variable to it, etc. When you're ready to go into production and need to create your own service account, then check out this page in the docs.
Furthermore, there has been significant changes in App Engine itself: the original Python 2 App Engine service had a bunch of built-in proprietary APIs (Datastore, Memcache, Task Queues, etc.). Due to user feedback regarding "vendor lock-in," the next generation Python 3 App Engine service was made to free developers from those services. Instead, you'd leverage any equivalent productized services, i.e., Cloud Datastore, Cloud Memorystore, and Cloud Tasks instead. The Google Cloud team has created a migration guide and I've augmented that guide with hands-on tutorials, code samples, and videos to help folks migrate to these unbundled services as you port your app to Python 3.
If you're considering Google Cloud serverless compute platforms beyond App Engine, such as Cloud Functions (FaaS) or Cloud Run (containerized/managed CaaS), then check out this Translation API sample app I created (where I basically stole the above code snippet from) that can be deployed 8 different ways, Python 2 and 3, locally with Flask's development server, to App Engine, Cloud Functions, or Cloud Run, all with just minor config changes. It's meant to show flexibility in our platforms as well as to help users understand the differences between them better.