We Keep Coding

Python shell working differently on different location in the same machine inside same virtual environment

I have a cloud instance of a Linux machine (openSuSE) with multiple users.
I have created a virtual environment and installed all my required libraries (including Klein).
I have two users "a" and "b".
While logged in as "a" and inside virtualenv, when I open python shell at home directory and type
import klein
it imports normally.
Now when I change directory to
/home/b/
and run the same (open python shell, import klein) while being in the same virtualenv, it gives me an error.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/a/.local/lib/python3.6/site-packages/klein/__init__.py", line 3, in <module>
from klein._plating import Plating
File "/home/a/.local/lib/python3.6/site-packages/klein/_plating.py", line 16, in <module>
from .app import _call
File "/home/a/.local/lib/python3.6/site-packages/klein/app.py", line 19, in <module>
from twisted.internet import endpoints, reactor
File "/home/a/.local/lib/python3.6/site-packages/twisted/internet/endpoints.py", line 58, in <module>
from twisted.protocols.tls import TLSMemoryBIOFactory
File "/home/a/.local/lib/python3.6/site-packages/twisted/protocols/tls.py", line 63, in <module>
from twisted.internet._sslverify import _setAcceptableProtocols
File "/home/a/.local/lib/python3.6/site-packages/twisted/internet/_sslverify.py", line 158, in <module>
verifyHostname, VerificationError = _selectVerifyImplementation()
File "/home/a/.local/lib/python3.6/site-packages/twisted/internet/_sslverify.py", line 141, in _selectVerifyImplementation
from service_identity import VerificationError
File "/home/a/.conda/envs/mm/lib/python3.6/site-packages/service_identity/__init__.py", line 7, in <module>
from . import cryptography, pyopenssl
File "/home/a/.conda/envs/mm/lib/python3.6/site-packages/service_identity/cryptography.py", line 16, in <module>
from .exceptions import SubjectAltNameWarning
File "/home/a/.conda/envs/mm/lib/python3.6/site-packages/service_identity/exceptions.py", line 21, in <module>
#attr.s
AttributeError: module 'attr' has no attribute 's'
Command "which python" gives same address at both location which is my virtualenv python address and that should be expected.
But what causes this weird python shell behavior.
Thank you

I solved it and a very shameful reason caused the error.
One of the modules Twisted uses is "attr" module. I had named one of my files attr.py and that is what was causing all the error.
I myself am not deleting this question if moderation has no problem, maybe somebody like me might be stuck at the same situation. It may help them.
Never name your python files same as that of any standard module unless overriding.
Also if your issue persists, then Jean's answer will definitely resolve it.

There can be multiple different Python packages that provide the same Python module. For example, there are at least two packages that provide the attr module:
https://pypi.org/project/attr/
https://pypi.org/project/attrs/
It's possible you've installed the wrong package based on the requirements. You can check what you have installed with pip freeze.

"ImportError: No module named pkg_resources" when importing google.cloud.datastore

I am attempting to use the Google Cloud Datastore, but importing google.cloud.datastore gives an ImportError:
ERROR 2018-03-13 19:28:29,013 wsgi.py:263]
Traceback (most recent call last):
File "/home/<user>/Software/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/home/<user>/Software/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/home/<user>/Software/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/home/<user>/Projects/<my_project>/main.py", line 1, in <module>
from my_project import app
File "/home/<user>/Projects/<my_project>/<my_project>/__init__.py", line 2, in <module>
from my_project.submit.controllers import submit
File "/home/<user>/Projects/<my_project>/<my_project>/submit/controllers.py", line 6, in <module>
from . import model_datastore
File "/home/<user>/Projects/<my_project>/<my_project>/submit/model_datastore.py", line 2, in <module>
from google.cloud import datastore
File "/home/<user>/Projects/<my_project>/env/local/lib/python2.7/site-packages/google/cloud/datastore/__init__.py", line 57, in <module>
from pkg_resources import get_distribution
ImportError: No module named pkg_resources
I am on Linux Mint attempting to run a Google App Engine local dev server.
I am using a virtual environment. Both setuptools and pkg_resources are installed and updated in the virtual environment. When I enter the python cmd line interpreter from the virtual env and import pkg_resources, it works fine. When I run the google app engine dev server by doing dev_appserver app.yaml, everything works fine until I access the page that activates the handler that imports datastore, then I get this error.
None of the other similar posts about 'pkg_resources` import error were helpful.
If there is any other info I can provide that would help, please let me know. Thanks!

In the standard environment you need to install all your dependencies inside your application. See Using third-party libraries.
Whatever you have installed in the local environment (virtual or not) doesn't matter, GAE doesn't know how to use those and your app might not work properly locally and definitely won't work when deployed on GAE.
You traceback indicates that you're loading the datastore library from your virtual env, not from your app, which is most likely why it doesn't work:
.../env/local/lib/python2.7/site-packages/google/cloud/datastore/__init__.py
You need to fix your app dependency installation.
This discussion might be of interest as well: No module named warnings when starting GAE inside virtualenv locally

Matplotlib with Google App Engine local development server

I want to use matplotlib in my Google App Engine project. I followed the steps, described here in the official docs. What I did:
1) Created a directory named lib in my application root directory.
2) Created a file appengine_config.py in my application root directory and added there these lines:
from google.appengine.ext import vendor
vendor.add('lib')
3) Since the docs say, that the only version of matplotlib working is 1.2.0, I executed the following command in the Terminal:
pip install -t lib matplotlib==1.2.0
There is also step 0 in the docs, which says
Use pip to install the library and the vendor module to enable importing packages from the third-party library directory.
But I don't understand what it actually means. If this is something essential, please, explain to me what it's meant here. I found this answer here on stackoverflow, and It seems there is nothing different from what I have done.
Also, I added
libraries:
- name: matplotlib
version: "1.2.0"
to app.yaml.
So, after all these steps I add the line
import matplotlib
to main.py and start a local server with
python ~/path/google_appengine/dev_appserver.py app.yaml
But when I try to access http://localhost:8080/, the error is raised:
raise ImportError('No module named %s' % fullname)
ImportError: No module named _ctypes
The whole output, if needed, looks like this:
ERROR 2016-08-11 16:26:51,621 wsgi.py:263]
Traceback (most recent call last):
File "/home/magnitofon/Загрузки/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/home/magnitofon/Загрузки/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/home/magnitofon/Загрузки/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/home/magnitofon/realec-inspector/main.py", line 20, in <module>
import matplotlib
File "/home/magnitofon/realec-inspector/lib/matplotlib/__init__.py", line 151, in <module>
from matplotlib.rcsetup import (defaultParams,
File "/home/magnitofon/realec-inspector/lib/matplotlib/rcsetup.py", line 20, in <module>
from matplotlib.colors import is_color_like
File "/home/magnitofon/realec-inspector/lib/matplotlib/colors.py", line 52, in <module>
import numpy as np
File "/home/magnitofon/Загрузки/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py", line 705, in load_module
module = self._find_and_load_module(fullname, fullname, [module_path])
File "/home/magnitofon/Загрузки/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py", line 446, in _find_and_load_module
return imp.load_module(fullname, source_file, path_name, description)
File "/usr/local/lib/python2.7/dist-packages/numpy/__init__.py", line 180, in <module>
from . import add_newdocs
File "/usr/local/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/usr/local/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 8, in <module>
from .type_check import *
File "/usr/local/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "/usr/local/lib/python2.7/dist-packages/numpy/core/__init__.py", line 22, in <module>
from . import _internal # for freeze programs
File "/usr/local/lib/python2.7/dist-packages/numpy/core/_internal.py", line 14, in <module>
import ctypes
File "/usr/lib/python2.7/ctypes/__init__.py", line 10, in <module>
from _ctypes import Union, Structure, Array
File "/home/magnitofon/Загрузки/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py", line 963, in load_module
raise ImportError('No module named %s' % fullname)
ImportError: No module named _ctypes
What am I doing wrong?

matplotlib is one of the Google-provided 3rd party libs, so you should be following just the Requesting a library instructions and
not the Installing a library ones.
Sadly they're now both on the same documentation page, called Using Built-in Libraries in Python 2.7 - very confusing for the unaware as the vendoring technique should be used for libraries which are not GAE built-in/provided. Filed Issue 13202.
Note: pay attention to the Using libraries with the local development server section, it applies to matplotlib. You may need to install some packages on your system, but not in the application itself (which could negatively affect your deployment on GAE) - they need to be accessible by the development server, not directly by your application.
Duh, I just noticed the Using matplotlib section, on the same page :)
It mentions:
Note: The experimental release of matplotlib is not supported on the development server. You can still add matplotlib to the
libraries list, but it will raise an ImportError exception when
imported.

Some searching into this issue surfaced this (old) recipe:
http://code.activestate.com/recipes/578393-gae-matplotlib-demo/
be sure to read the comments.
However I tried, I was not able to make it work. It be this approach just does not work with the current version of the app engine sandbox anymore, or I just was not able to follow all the steps in the recipe correctly.
Maybe an even better read are the comments of Matt Giuca (a google engineer who ported matplotlib to work in GAE production env.) and his pull request in github.
https://github.com/matplotlib/matplotlib/issues/1823/
His comment of April 17, 2013 gives a recipe for matching the dev_appserver (version 1.77) to work with the matplotlib locally.
Maybe this helps someone dealing with this issue.

Google app engine + python (django) deployment error: Error loading MySQLdb module

I am creating an app on Google App Engine. I am using Django 1.4 and Python 2.7. Everything works fine on localhost. But after deployment it is not running, I keep getting this on admin logs:
2012-12-15 15:02:41.870
/base/python27_runtime/python27_lib/versions/1/lib/cacerts/urlfetch_cacerts.txt missing; without this urlfetch will not be able to validate SSL certificates.
W 2012-12-15 15:02:41.870
No ssl package found. urlfetch will not be able to validate SSL certificates.
E 2012-12-15 15:02:46.086
Traceback (most recent call last):
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 196, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 266, in _LoadHandler
__import__(cumulative_path)
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/djangoappengine/main/__init__.py", line 28, in <module>
setup_env()
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/djangoappengine/boot.py", line 82, in setup_env
setup_logging()
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/djangoappengine/boot.py", line 130, in setup_logging
if not settings.DEBUG:
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/utils/functional.py", line 276, in __getattr__
self._setup()
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/conf/__init__.py", line 42, in _setup
self._wrapped = Settings(settings_module)
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/conf/__init__.py", line 87, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/settings.py", line 6, in <module>
import django.db.backends.mysql.base
File "/base/data/home/apps/s~cloudwallforever/1.363864476397206865/django/db/backends/mysql/base.py", line 14, in <module>
raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb
Any ideas why it is happening?

The 'Django Support' documentation hints at the solution, but doesn't make it explicit:
Since the standard django.db.backends.mysql backend uses MySQLdb internally, app.yaml must reference MySQLdb in the list of libraries.
Adding the following to app.yaml seems to fix the ImportError:
libraries:
- name: MySQLdb
version: "latest"
Note that MySQLdb is not currently included in the list of available third party libraries. I tried it on a whim and it seems to have fixed the issue for me, YMMV.enter link description here

As the Cloud SQL docs clearly state, you should be using ''google.appengine.ext.django.backends.rdbms' as your database engine setting.

If it's not working after adding the following to your app.yaml:
libraries:
- name: MySQLdb
version: "latest"
...then take the following steps:
Ensure MySQLdb is installed pip install mysql-python on Mac
Ensure the GAE devserver is pointing to your virtualenv/pip'd python 2.7 installation vs. Mac/*nix's default version as pip may not have installed it there. I resolved Can't find module: MySQLdb by updating the GAE devserver to use: /usr/local/bin/python2.7 e.g. in GAE devserver preferences for Python Path
Note: the preferences can be accessed through the application menus:
If you're using Windows: look under File > Preferences...
If you're using Mac, look under GoogleAppengineLauncher > Preferences...

Python with Google App Engine. AttributeError: 'module' object has no attribute 'HTTPSHandler' error message

I’m new to Python and I’m currently following the Head First Python book. I’m trying to put an example exercise to work but I’m stuck with an error from the GAE.
I installed both Python 2.5 and GAE on my Kubuntu 10.10 S.O.
First I create a folder named “mygaetest” with the files sayhello.py and app.yaml inside.
Because there is no graphical front end for Linux, I start my GAE webapp from the command line this way:
python2.5
google_appengine/dev_appserver.py
home/lucas/workspace/Python/Chapter10/src/mygaetest/
But then the following error is shown:
Traceback (most recent call last):
File
"google_appengine/dev_appserver.py",
line 78, in
run_file(__file__, globals())
File
"google_appengine/dev_appserver.py",
line 74, in run_file
execfile(script_path, globals_)
File
"/home/lucas/Desarrollo/Python/App
Engine/google_appengine/google/appengine/tools/dev_appserver_main.py",
line 118, in
from google.appengine.tools import appcfg
File
"/home/lucas/Desarrollo/Python/App
Engine/google_appengine/google/appengine/tools/appcfg.py",
line 68, in
from google.appengine.tools import appengine_rpc
File
"/home/lucas/Desarrollo/Python/App
Engine/google_appengine/google/appengine/tools/appengine_rpc.py",
line 27, in
import fancy_urllib
File
"/home/lucas/Desarrollo/Python/App
Engine/google_appengine/lib/fancy_urllib/fancy_urllib/init.py",
line 341, in
class FancyHTTPSHandler(urllib2.HTTPSHandler):
AttributeError: 'module' object has no
attribute 'HTTPSHandler'
Can someone tell me what to do in order to solve that error?
Many thanks in advance.

Welcome to Python, App Engine, and Stack Overflow!
Your Python installation apparently lacks SSL support. To add support, install this.
http://code.google.com/p/googleappengine/issues/detail?id=19

I received this error after upgrading Mac OSX to 10.10 (Yosemite), I needed the new build of python 2.7:
$ brew update && brew upgrade python
==> Upgrading 1 outdated package, with result:
python 2.7.8_2
==> Upgrading python
...
==> Summary
/usr/local/Cellar/python/2.7.8_2: 4777 files, 76M

I am using Python 2.7 and I used the following instructions to solve the problem:
http://paltman.com/2007/11/15/getting-ssl-support-in-python-251/