I am using a couple of google libs in order to authenticate with firebase in a Python + GAE app.
I have configured the requirements.txt with the following content:
google-auth==1.0.1
requests==2.14.2
requests-toolbelt==0.7.1
This is what I am importing:
import google.auth.transport.requests
When I run pip install, they do get installed locally and I get no errors.
local libs screenshot
But when I try to deploy this application to Google App Engine, all those external libs get the same errors. GAE doesn't find the files:
ImportError: No module named auth.transport.requests
You need to provide your library directory to the google.appengine.ext.vendor.add() method.
Create a file named appengine_config.py in the same folder as your app.yaml file.
Edit the appengine_config.py file and provide your library directory to the vendor.add() method.
# appengine_config.py
from google.appengine.ext import vendor
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
https://cloud.google.com/appengine/docs/standard/python/tools/using-libraries-python-27#installing_a_third-party_library
Related
I am using google-auth to allow firebase authentication in my GAE project.
Everything works fine when I run the code locally using dev_appserver.py or when I deploy it to google app engine.
But I get this ImportError exceptions when I try to use Django's manage.py script to create/run migrations.
ImportError: Could not import 'firebase.authentication.FirebaseAuthentication' for API setting 'DEFAULT_AUTHENTICATION_CLASSES'. ImportError: No module named auth.transport.requests.
The google-auth module is installed under lib directory and has this structure:
- lib
- google
- auth
- oauth2
These import cause the ImportErrors:
import google.auth.transport.requests
from google.oauth2 import id_token
My guess is that there might be naming conflicts as other imports work fine.
Please help!
If you want to use 3rd party libraries that are not included in this list, then you'll have to add them manually since you have done that by adding lib folder and including all the package folder follow these steps.
Create your_app_directory/appengine_config.py file.
Add these following lines inside that file
from google.appengine.ext import vendor
vendor.add('lib')
This should solve the import error problem.
As the title says.
So I've added the following to appengine_config.py with no luck:
# appengine_config.py
from google.appengine.ext import vendor
# Add any libraries install in the "lib" folder.
vendor.add('lib')
I did a print sys.path and verified the lib dir contains google/cloud/bigquery
I can import it if I run python myself:
from google.cloud import bigquery
print bigquery.__path__
['/usr/local/lib/python2.7/dist-packages/google/cloud/bigquery']
From a Google App Engine end points:
import google.cloud;print google.cloud.__path__
['/usr/local/lib/python2.7/dist-packages/google/cloud']
Big Query is at that system location. /usr/local/lib/python2.7/dist-packages/google/cloud/biqguery/ exists. However if I try the following from app engine end point:
from google.cloud import bigquery
ImportError: No module named google.cloud.bigquery
The file in question includes and that does not appear to help:
from __future__ import absolute_import
Update
I setup a venv and install everything like this: pip install -t lib/ -r requirements.txt --upgrade. From there if I try import google; print google.__path__ I get:
['lib/google', '/usr/lib/google-cloud-sdk/platform/google_appengine/google']
I think the second path might be the cause.
I've reviewed the following with no success:
Error importing Google Cloud Bigquery api module in python app
https://github.com/GoogleCloudPlatform/google-cloud-python/issues/2366
How to import BigQuery in AppEngine for Python
You need to install google-cloud-bigquery into you app's lib dir, that's where the development server is looking at, not on your system's libs. From Installing a third-party library:
Create a directory to store your third-party libraries, such as lib/.
mkdir lib
Use pip (version 6 or later) with the -t <directory> flag to copy the libraries into the folder you created in the previous
step. For example:
pip install -t lib/ <library_name>
I want to use two Python libraries (Google's Cloud Library, and their Cloud SDK) in a single application, but they have conflicting names (they both use google in their base import names and do not use relative imports internally). How can I use them in a single app?
Changing the library's code to use proper relative imports is not practical. Also, I know I can use virtualenv to access these libraries from separate python applications, but how do I access them from within the same python app?
Details of the Naming Conflict
Here are some of the details on the import. When I import a module from the Cloud Library (I run import google.cloud.datastore), there is an exception about another import within that library:
>>> import libs.google.cloud.datastore
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "C:\[ProjectDIR]\libs\google\cloud\datastore\__init__.py", line 52, in <module>
from google.cloud.datastore.batch import Batch
ImportError: No module named cloud.datastore.batch
The library is trying to do an absolute import, rather than a relative one. The reason that the Google Cloud Library cannot import google.cloud.datastore.batch is because google is already defined in the SDK, there is a naming conflict:
>>> print google.__path__
['C:\\Program Files (x86)\\Google\\Cloud SDK\\google-cloud-sdk\\platform\\google_appengine\\google']
Because the Cloud Library uses absolute imports, and the name google is already defined in the SDK, then the import fails.
The google packages take care to register themselves as a namespace package. With a properly set up sys.path there is no conflict here.
You need to set up your library environment correctly. Add a appengine_config.py file in the root of your project with:
from google.appengine.ext import vendor
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
This adds the lib subdirectory in the right location of sys.path. See the Installing a third-party library section in the Developing Python Apps on App Engine How-To.
From here on out imports of google.cloud just work:
$ ls -1d lib *.py *.yaml
app.yaml
appengine_config.py
lib
main.py
$ pip install -t lib google-cloud
# installing into the lib subdirectory
$ cat main.py
import google
from google.cloud import datastore
from google.appengine.api import memcache
import os.path
here = os.path.dirname(os.path.abspath(__file__))
def app(*args, **kwargs):
return '''
google: {}<br />
google.cloud.datastore: {}<br />
google.appengine.api.memcache: {}'''.format(
os.path.relpath(google.__file__, here),
os.path.relpath(datastore.__file__, here),
os.path.relpath(memcache.__file__, here))
and in the browser I am served:
google: ../google-cloud-sdk/platform/google_appengine/google/__init__.py
google.cloud.datastore: lib/google/cloud/datastore/__init__.pyc
google.appengine.api.memcache: ../google-cloud-sdk/platform/google_appengine/google/appengine/api/memcache/__init__.pyc
I'm running into this weird error with a Cloud Endpoints app that I'm just starting to write. I'm not sure if Google changed their libraries or not, but I think this should work?
In my app.yaml I've got...
libraries:
- name: webapp2
version: "latest"
- name: endpoints
version: "latest"
And then in my main.py I call:
import endpoints
Result:
ImportError: No module named endpoints
Why would app engine be telling me that endpoints doesn't exist? I can see the endpoints folder in the directory itself...
I got the same thing on a fresh install of the SDK (previously worked).
You need to explicitly add the relevant GAE libraries to your PYTHONPATH.
For example if you're using virtualenvwrapper (you should be):
$ add2virtualenv /path/to/google-cloud-sdk/platform/google_appengine/lib/endpoints-1.0
$ add2virtualenv /path/to/google-cloud-sdk/platform/google_appengine/lib/protorpc-1.0
I'm following this tutorial
https://developers.google.com/bigquery/articles/dashboard#downloadinstalllibraries
Here they say that
cd source/
enable-app-engine-project hellodashboard/
which will copy some 60 files into the project directory.But i saw only some 10 files copied.And the oauth2client folder in project directory only has __init__.pyfile
Since i'm using
from oauth2client.appengine import oauth2decorator_from_clientsecrets
When i run my program it shows an error
from oauth2client.appengine import oauth2decorator_from_clientsecrets
ImportError: No module named appengine
So i manually copied all files from oauth2client zip to the project oauth2client directory.
Now when i run my program it doesn't show any error and seems to run fine.Is it some thing messy with the enable-app-engine-project command or am i doing some thing wrong.
enable-app-engine-project tries to locate sources based on the path(s) where the modules are installed on your local system. Have you installed these modules locally?
As an alternative, you can simply copy the following directories into your App Engine project manually:
SOURCES = [
'gflags',
'gflags_validators',
'httplib2',
'oauth2client',
'oauth2',
'apiclient',
'uritemplate',
]