On Ubuntu 16.04, am suddenly getting import errors from the local GAE development server.
The local dev server starts up, including the admin interface, but app no longer loads.
Native python imports of the same library on same machine (in this case "from google.cloud import datastore") work fine.
The GAE standard app does run when deployed, but development just got a little challenging.
google-cloud is version 0.27.0
gcloud components is 172.0.1
python is Anaconda 2.7.13
GAE is standard
I have confirmed to the best of my middling abilities that $PATH is correct for all named libraries.
I have removed and re-added all the named libraries to no effect.
cachetools(2.0.1) it should probably be noted, is installed as a dependency of the google cloud libraries, so I don't think this is addressable through requirements.txt or "libraries" in app.yaml.
I did recently go through a cycle of removing and adding libraries to fix a problem with apache_beam 2.0.1, so I may have jacked up something else, but am not sure where to look.
Suggestions deeply appreciated. Full traceback (from admin, same as from app):
Traceback (most recent call last):
File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/runtime/request_handler.py", line 232, in handle_interactive_request
exec(compiled_code, self._command_globals)
File "<string>", line 1, in <module>
File "/home/brian/anaconda3/lib/python2.7/site-packages/google/cloud/datastore/__init__.py", line 61, in <module>
from google.cloud.datastore.client import Client
File "/home/brian/anaconda3/lib/python2.7/site-packages/google/cloud/datastore/client.py", line 23, in <module>
from google.cloud.client import ClientWithProject
File "/home/brian/anaconda3/lib/python2.7/site-packages/google/cloud/client.py", line 27, in <module>
from google.oauth2 import service_account
File "/home/brian/anaconda3/lib/python2.7/site-packages/google/oauth2/service_account.py", line 79, in <module>
from google.auth import jwt
File "/home/brian/anaconda3/lib/python2.7/site-packages/google/auth/jwt.py", line 49, in <module>
import cachetools
ImportError: No module named cachetools
The stacktrace indicates you're running the libraries from the local system installation (the site-packages dir), not from your app.
For standard env GAE apps you need to install the dependencies inside your app and they will be uploaded to GAE together with your app code.
More specifically you need to use the -t <your_app_lib_dir> option for the pip installation. From Installing a third-party library:
Use pip (version 6 or later) with the -t <directory> flag to copy the libraries into the folder you created in the previous
step. For example:
pip install -t lib/ <library_name>
I addressed my problem through the requirements.txt file in app root directory.
I had: google-cloud==0.22.0
and changed it to: google-cloud==0.27.0
which fixed it.
Related
I am attempting to use the Google Cloud Datastore, but importing google.cloud.datastore gives an ImportError:
ERROR 2018-03-13 19:28:29,013 wsgi.py:263]
Traceback (most recent call last):
File "/home/<user>/Software/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/home/<user>/Software/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/home/<user>/Software/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/home/<user>/Projects/<my_project>/main.py", line 1, in <module>
from my_project import app
File "/home/<user>/Projects/<my_project>/<my_project>/__init__.py", line 2, in <module>
from my_project.submit.controllers import submit
File "/home/<user>/Projects/<my_project>/<my_project>/submit/controllers.py", line 6, in <module>
from . import model_datastore
File "/home/<user>/Projects/<my_project>/<my_project>/submit/model_datastore.py", line 2, in <module>
from google.cloud import datastore
File "/home/<user>/Projects/<my_project>/env/local/lib/python2.7/site-packages/google/cloud/datastore/__init__.py", line 57, in <module>
from pkg_resources import get_distribution
ImportError: No module named pkg_resources
I am on Linux Mint attempting to run a Google App Engine local dev server.
I am using a virtual environment. Both setuptools and pkg_resources are installed and updated in the virtual environment. When I enter the python cmd line interpreter from the virtual env and import pkg_resources, it works fine. When I run the google app engine dev server by doing dev_appserver app.yaml, everything works fine until I access the page that activates the handler that imports datastore, then I get this error.
None of the other similar posts about 'pkg_resources` import error were helpful.
If there is any other info I can provide that would help, please let me know. Thanks!
In the standard environment you need to install all your dependencies inside your application. See Using third-party libraries.
Whatever you have installed in the local environment (virtual or not) doesn't matter, GAE doesn't know how to use those and your app might not work properly locally and definitely won't work when deployed on GAE.
You traceback indicates that you're loading the datastore library from your virtual env, not from your app, which is most likely why it doesn't work:
.../env/local/lib/python2.7/site-packages/google/cloud/datastore/__init__.py
You need to fix your app dependency installation.
This discussion might be of interest as well: No module named warnings when starting GAE inside virtualenv locally
I would like to use Google Cloud Storage Client Library Functions.
For that I have to import the cloudstorag. To get the cloudstorage I download Google Cloud Storage client library.
I try to import cloudstorage using python -c "import cloudstorage". I get the following error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "cloudstorage/__init__.py", line 20, in <module>
from .api_utils import RetryParams
File "cloudstorage/api_utils.py", line 45, in <module>
from google.appengine.api import app_identity
ImportError: No module named google.appengine.api
Am I missing something?
When you execute python -c "import cloudstorage" you're attempting to run a standalone application. But the GCS library you're trying to use is for a (standard environment) GAE application, which cannot be executed as a standalone app, it needs to run in a GAE sandbox (locally that's dev_appserver.py). See GAE: AssertionError: No api proxy found for service "datastore_v3".
And the library needs to be installed inside your GAE app, see Copying a third-party library.
If you're not developing a standard env GAE app and indeed you want to write a standalone one, you're not looking at the right documentation. You need to use a different library than the GAE-specific one(s). See Cloud Storage Client Libraries
You can add this lines, which will add the path of the sdk tools:
import pkgutil
import google
google.__path__ = pkgutil.extend_path(google.__path__, google.__name__)
For unittesting, it could be useful to run in standalone mode.
Looks like gcloud is not installed on your system.
pip install --upgrade gcloud
pip install --upgrade google-api-python-client
TL;DR: Yampy uses relative imports... is there some setting I can change that would make it work as-is (without having to refactor every import in the project)?
Windows 7, Python 3.4.3, PyCharm 2016.1.4, Yampy 1.0
Should be fairly simple question -- hoping someone has encountered this before. I am following the quickstart guide. Someone asked the same question a year ago with less information, but there was no answer.
I created a virtualenv, activated, and installed yampy. That gave the import error below, so I uninstalled and installed again:
(MyVenv) C:\Users\me>pip install yampy
Collecting yampy
Using cached yampy-1.0.tar.gz
Requirement already satisfied (use --upgrade to upgrade): requests in c:\virtual environments\myvenv\lib\site-packages (from yampy)
Installing collected packages: yampy
Running setup.py install for yampy ... done
Successfully installed yampy-1.0
That created the following directory:
C:\Virtual Environments\MyVenv\Lib\site-packages\yampy
__pycache_ _ (contains appropriate .pyc files)
apis
__pycache_ _ (contains appropriate .pyc files)
__init_ _.py
messages.py
users.py
utils.py
__init_ _.py
authenticator.py
client.py
constants.py
errors.py
models.py
yammer.py
Contents of __init_ _.py:
"""
The official Python client for Yammer's API
"""
from authenticator import Authenticator
from client import Client
from yammer import Yammer
The issue:
When I import yampy from the python shell, I get the following traceback (this was from PyCharm shell, but same issue in command-line shell):
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "C:\Program Files (x86)\JetBrains\PyCharm 2016.1\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
File "C:\Virtual Environments\myvenv\lib\site-packages\yampy\__init__.py", line 22, in <module>
from authenticator import Authenticator
File "C:\Program Files (x86)\JetBrains\PyCharm 2016.1\helpers\pydev\_pydev_bundle\pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
That doesn't make any sense to me since everything seems to be there. Thanks!
Update:
If I change the _init _ file to use absolute imports (from yampy.authenticator import .. instead of from authenticator import..), it solves the immediate issue, but creates a string of import errors throughout the project. Is there some setting I can change that would require a minimum of refactoring?
Should have checked GitHub first: there is an open issue about this that has been open since Feb 2015. Apparently the project is in Python 2. Going to try forking and updating to Python 3..
Update: made the changes manually, took about 5 minutes. Imports now!
Better Update: Anthony Shaw (tonybaloney on Github) published a package for Python 3 called yampy3.
I have installed qpid-0.22 on sles11 sp2 X86_64, the broker works fine.
Then I installed qpid-python client and set the env variable.
PYTHONPATH=/home/zdx/qpid/qpid-0.22/python/:/usr/local/lib/python2.7:/usr/local/lib/python2.7/site-packages:/home/zdx/qpid/qpid-0.22/python
But the python client doesn't work, including qpid-config tool and qpid-python client test examples.
When I ran this kind of script, it showed following exception:
Traceback (most recent call last):
File "/usr/local/bin/qpid-config", line 31, in
from qpid.messaging import Connection
File "/usr/local/lib/python2.7/site-packages/qpid/init.py", line 20, in
import connection
File "/usr/local/lib/python2.7/site-packages/qpid/connection.py", line 20, in
import datatypes, session
File "/usr/local/lib/python2.7/site-packages/qpid/session.py", line 26, in
from ops import Command, MessageTransfer
ImportError: cannot import name MessageTransfer
It indicate that class or module MessageTransfer does not exist in ops module,
and I look into the python module ops.py, there is none class MessageTransfer.
what is the problem with it? thanks.
Even though you installed the command line tools properly, sometimes you will get this error.
This means that you need to install the python-qpid bindings and their libraries.
If you have epel repository in your /etc/yum.repos.d/ , You can directly install the package by using yum like this.
# yum search python-qpid
In search results, select package according to your Operating system (32-bit/64-bit).
And then install the package.
# yum install python-qpid..... (python-qpid-proton.x86_64, etc..)
If you don't have epel, first get epel into your /etc/yum.repos.d/ and then install the package
Is CherryPy broken? I just set it up and tried to use the routes dispatcher but it has an import error, my code is as follows:
import cherrypy
mapper = cherrypy.dispatch.RoutesDispatcher()
The error is:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jwesonga/environments/cherrypy/lib/python2.6/site-packages/CherryPy-3.2.2-py2.6.egg/cherrypy/_cpdispatch.py", line 463, in __init__
import routes
ImportError: No module named routes
I'm on a Mac and I tried both 3.2.2 and 3.0 using virtualenv for the latter.
I have successfully used CherryPy with the routes dispatcher under OS X.
The error you've shown is:
ImportError: No module named routes
This is pretty clear -- Python can't find the routes modules. Have you installed it? This is not part of CherryPy, it's a separate module that you will need to install. If you're using MacPorts, you should be able to:
port install py-routes
(Or py25-routes or py26-routes depending on which Python you're using). If you're using virtualenv, you can simply run:
easy_install routes