I am creating a Python Web App in Google App engine.
When I
sudo pip install
a third party library and then try to import it, I get the error 'ImportError: No module named x'. Where x is the name of that library. In my case as an example: Boto, Boto3, Fask etc.
If I go into shell in GAE and type python >> import X the library can be used inside the python environment. When deploying the app though or running the app in the virtaul server in Google App Engine I get the module import error.
I even tried methods like: python >> import sys >> sys.path.insert(0, "path_here")
export PYTHONPATH and selected where those libraries are located
I even followed several Q&A here in Stackoverflow without any success, can somebody please give me a proper way to fix the import error in Google App Engine?
FYI
I am not using any local environment in my pc, I am working directly through the GAE bash console, the launch code editor in GAE and I am running the command dev_appserver.py $PWD
When I do
pip freeze
I can see that the modules are currently installed and deployed on the GAE virtual environment. Is there a problem with my path? What's the best approach to make GAE load my already installed third party libraries.
UPDATE:
Importing the library directly on the python shell from Google App Engine works just fine. Importing the library on my python app index.py file results in the error.
Python import directly from Shell
Python import to the index.py file
Though this is an old thread, adding this answer now:
Run command : gcloud components list
This will show the different components installed and not in your environment.
Install app-engine-python components if not installed:
gcloud components install app-engine-python
gcloud components install app-engine-python-extras
If it doesn't work:
In Windows, uninstall and download and install Google-sdk (check the python version you need). Delete all the files the installer ask you to delete in the last step and run the gcloud component commands again.
Related
I try the example on: https://github.com/Azure-Samples/ms-identity-python-webapp with Windows 10, but I get an error with ModuleNotFoundError: No module named 'flask_caching.backends.filesystem' (Flask-Caching is already installed with pip).
Version:
Python 3.9.9,
Flask 1.1.4 and
Werkzeug 1.0.1.
I only changed the code with Client_ID, CLient_Secret and domain name in app_config.py.
Has anybody an idea?
The error ModuleNotFoundError means python interpreter cannot find the libraries which you are referring to in the code although the module is already installed.
Common causes of this error:
Using modules meant for different python versions but Installing python 2.x modules in python 3.x and vice a versa.
When not properly setting PATH variable.
(Or)
If you are using a python virtual environment. It need to be installed after creating a virtual environment as commented by #grumpyp . The libraries will reside inside the folder created for the virtual environment.
And can installed according to requirements.txt file
pip install virtualenv
It requires activation and dedicated installation of modules inside the virtual environment.
Refer this blog for more details to do
pip install -r requirements.txt
Other reference :Set Up a Virtual Python Environment (Windows)
(or )
This may not be your query but Just to make it a bit easy You can try this way when trying out your sample project to compare with manually configured one.
The quick start: "Add sign-in with Microsoft to a Python web app" that you are using ,can be directly configured in portal quickstart like below where every thing is configured including client id ,tenant id etc directly.
Just register the app with name and account type and follow the steps below for direct configuration .
Go to quickstart page of app
Select Python as platform for web application
Just follow the steps to configure azure ad inside app directly
There after following the steps , I checked the versions with pip freeze and
versions i have: Python 3.9.7, Flask 1.1.4 and Werkzeug 1.0.1.
quickstart-v2-python-webapp | microsoftdocs
I am learning how to use Azure functions and using my web scraping script in it.
It uses BeautifulSoup (bs4) and pymysql modules.
It works fine when I tried it locally in the virtual environment as per this MS guide:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-create-first-azure-function-azure-cli?pivots=programming-language-python&tabs=cmd%2Cbrowser#run-the-function-locally
But when I create the function App and publish the script to it, Azure Functions logs give me this error:
Failure Exception: ModuleNotFoundError: No module named 'pymysql'.
It must happen when attempting to import it.
I really don't know how to proceed, where should I specify what modules it needs to install?
You need to check if you have generated the requirements.txt which includes all of the information of the modules. When you deploy the function to azure, it will install the modules by the requirements.txt automatically.
You can generate the information of modules in requirements.txt file by the command below in local:
pip freeze > requirements.txt
And then deploy the function to azure by running the publish command:
func azure functionapp publish hurypyfunapp --build remote
For more information about deploy python function from local to auzre, please refer to this tutorial.
By the way, if you use consumption plan for your python function, the "Kudu" is not available for us. If you want to use "Kudu", you need to create app service plan for it but not consumption plan.
Hope it helps~
You need to upload the installed modules when deploying to azure. You can upload them using Kudu:
https://github.com/projectkudu/kudu/wiki/Kudu-console
as an alternative, you can also use Kudu and run pip install using the console:
Install python packages from the python code itself with the following snippet: (Tried and verified on Azure functions)
def install(package):
# This function will install a package if it is not present
from importlib import import_module
try:
import_module(package)
except:
from sys import executable as se
from subprocess import check_call
check_call([se,'-m','pip','-q','install',package])
for package in ['beautifulsoup4','pymysql']:
install(package)
Desired libraries mentioned the list gets installed when the azure function is triggered for the first time. for the subsequent triggers, you can comment/ remove the installation code.
I'm trying to install the vimpdb lib but it's working, even though I successfully installed vimpdb using pip install I always get this error:
import vimpdb; vimpdb.set_trace();
ImportError: No module named vimpdb
I'm running the code locally but when I run the same code as a simple script (without using localhost) it imports correctly, it only throws an error when I start a server and begin to try using this plugin.
Any ideas?
Thansk!
App Engine won't import Python modules on your Python path. You need to actually include the module within the App Engine project.
For example, in the same directory as app.yaml, you could add a symbolic link similar to this:
vimpdb -> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/vimpdb
Or you could copy the vimpdb directory to that location.
This should be pretty basic as I've installed lots of python packages, but I can't for the life of my get google apis client library for python to install.
I'm pressure sure I've gone through the instructions on this website properly:
https://developers.google.com/api-client-library/python/start/installation
To summarize I've done the following:
$ easy_install --upgrade google-api-python-client
this seems to work fine, doesn't report any errors or warnings
downloaded and unzipped google-api-python-client-gae-1.2.zip into the directory where my project is
but if I open an iPython session in the folder where I unpacked the full dependencies I can't do the basic imports such as:
import google.appengine.api it just says "No Module named google.appengine.api"
I checked in my site-packages folder and google_api_python_client-1.2-py2.7.egg is there. But it doesn't show up in sys.path
when I do sys.path.append('C:\Anaconda\Lib\site-packages\google_api_python_cli
ent-1.2-py2.7.egg') it adds the correct path, but the import still doesn't work.
EDIT: This fixed my problem
Adding the Google SDK to my Python path did the trick. I don't know why the installer didn't do this when I ran it. but hey, this worked
So if I run:
sys.path.append('C:\\Program Files (x86)\\Google\\google_appengine')
from google.appengine import api
works!
Without knowing all the steps you took to install app engine and the client APIs, my only recommendation is to install (or reinstall) the Google App Engine Python SDK from here: https://developers.google.com/appengine/downloads#Google_App_Engine_SDK_for_Python. Looks like you're on windows, so you would grab the MSI.
Just make sure you launch the AppEngineLauncher application after installation as it will give you the option to create symlinks so you can run commands from terminal.
I'm trying to teach myself python using Google's AppEngine, and I can't get the dev server running. I get this error:
Traceback (most recent call last):
File "/opt/google_appengine/google_appengine_1.2.7/dev_appserver.py",
line 60, in
run_file(file, globals()) File
"/opt/google_appengine/google_appengine_1.2.7/dev_appserver.py",
line 57, in run_file
execfile(script_path, globals_) File
"/opt/google_appengine/google_appengine_1.2.7/google/appengine/tools/dev_appserver_main.py",
line 65, in
from google.appengine.tools import os_compat ImportError: cannot import
name os_compat
Ubuntu 9.10 comes with python2.6 (didn't work), and I installed python2.5 (didn't work), and have tried running it with python dev_appserver.py helloWorld (didn't work) as well as running dev_appserver.py after editing the first line to be:
#!/usr/bin/env python2.5
I can't seem to find anything online with this error. The only problem I've found is about using python 2.5, and I think I've solved that.
Kyle suggested I need to set my PYTHONPATH variable. After running
export PYTHONPATH=/opt/google_appengine/google_appengine_1.2.7
I still get the same error trying to run dev_appserver.py. Am I setting PYTHONPATH wrong? Alternatively, how do I uninstall the protocol buffers python project? I have no use for Ubuntu One and had already uninstalled it.
The problem appears to be the fact that Karmic Koala 9.10 (the latest version of Ubuntu) ships with Ubuntu One, a python app that depends on Google's protocol buffers library. The python-protobuf package provides the google.protobuf package in /usr/lib/pymodules/python2.6.
Unfortunately, the AppEngine SDK includes another package called google.appengine. So somewhere in your code, the google package is being imported, and the package that contains protobuf is being found on PYTHONPATH first. Python caches the first package it finds in sys.modules, so the second google package in the SDK will never be imported.
You could move the google AppEngine SDK up to the front of your PYTHONPATH. That should ensure that Python finds the google.appengine package instead of the package provided by python-protobuf.
PYTHONPATH=/opt/google_appengine/google_appengine_1.2.7 \
python dev_appserver.py helloWorld
This is a bug that should be reported to the AppEngine SDK project.
Update: I've submitted a bug against the AppEngine API.
It was a file permission problem. os_compat.py wasn't readable by user, just by root. I'm not sure if I screwed this up, or if the permissions by default don't have read-all, but that was the fix.
I hate to accept my own answer after Kyle gave such a good response, but I don't need the $PYTHONPATH fix to make it work now that I did sudo chown -R +r /opt/google_appengine/google_appengine_1.2.7
With that error, Python is saying that it can't find or read the name that it's trying to import. Since the import of os_compat is the very first executable line of AppEngine's dev_appserver.py, I suspect that there's a problem with the way that your paths are configured.
The latest version of Ubuntu (10.10) has also removed Python 2.5 - making it a pain to install the App Engine development environment.
I (finally) got my environment working (including using App Engine Helper for unit testing). I built this bash script which might be useful to others. It installs:
sqlite
libsqlite
pep8
mock
OpenSSL
Python 2.5.2
Python SSL Library
Django 1.1 (latest version in production)
App Engine
App Engine Helper
http://pageforest.googlecode.com/hg/tools/pfsetup
Ubuntu 11.04 comes with python 2.6 as the default version. It is suggested to use Google app engine with version 2.5. I am using it though for many years with python 2.6 without any issues.
What you need to do in order to execute it smoothly with python 2.6 is to edit google/appengine/tools/dev_appserver.py and add these three lines
'_counter',
'_fastmath',
'strxor',
after 'XOR', and before '_Crypto_Cipher__AES', around line ~1350.
If you are now using Google Cloud SDK, put this into ~/.profile.
export CLOUDSDK_ROOT_DIR="/path/to/google/cloud/sdk/"
export APPENGINE_HOME="${CLOUDSDK_ROOT_DIR}/platform/appengine-java-sdk"
export GAE_SDK_ROOT="${CLOUDSDK_ROOT_DIR}/platform/google_appengine"
# The next line enables Java libraries for Google Cloud SDK
export CLASSPATH="${APPENGINE_HOME}/lib":${CLASSPATH}
# The next line enables Python libraries for Google Cloud SDK
export PYTHONPATH=${GAE_SDK_ROOT}:${PYTHONPATH}
# * OPTIONAL STEP *
# If you wish to import all Python modules, you may iterate in the directory
# tree and import each module.
#
# * WARNING *
# Some modules have two or more versions available (Ex. django), so the loop
# will import always its latest version.
for module in ${GAE_SDK_ROOT}/lib/*; do
if [ -r ${module} ]; then
PYTHONPATH=${module}:${PYTHONPATH}
fi
done
unset module
Do not put inside ~/.bashrc because, every time you open a bash session, all those modules will be added again and again into your PYTHONPATH environment variable.