I am trying to test 3rd party code with Anaconda 4.2 / Python 3.5 When I execute tests I get following exception:
Traceback (most recent call last):
File "pyspark/sql/tests.py", line 25, in <module>
import subprocess
File "/home/user/anaconda3/lib/python3.5/subprocess.py", line 364, in <module>
import signal
File "/home/user/anaconda3/lib/python3.5/signal.py", line 3, in <module>
from functools import wraps as _wraps
File "/home/user/anaconda3/lib/python3.5/functools.py", line 22, in <module>
from types import MappingProxyType
File "/home/user/Spark/spark-2.1.0-bin-hadoop2.7/python/pyspark/sql/types.py", line 22, in <module>
import calendar
File "/home/user/anaconda3/lib/python3.5/calendar.py", line 10, in <module>
import locale as _locale
File "/home/user/anaconda3/lib/python3.5/locale.py", line 108, in <module>
#functools.wraps(_localeconv)
AttributeError: module 'functools' has no attribute 'wraps'
Normally I would assume some module is shadowing built-in modules but as far as I can tell this is not the issue:
I logged module path (functools.__file__) from the tests and it yields expected path. Also there is nothing strange in the path I get in the exception.
To exclude possible module corruption I tested completely new Anaconda installation.
When I execute tests, with the same configuration and path, from IPython shell (%run pyspark/sql/tests.py) problem disappears.
functools.wraps can be imported in the shell started in the same directory and with the same configuration.
When I replace Python 3 environment with Python 2 environment problem disappears.
Problem cannot be reproduced with environment create using virtualenv.
With different version of the same project I get:
Traceback (most recent call last):
File "pyspark/sql/tests.py", line 25, in <module>
import pydoc
File "/home/user/anaconda3/lib/python3.5/pydoc.py", line 55, in <module>
import importlib._bootstrap
File "/home/user/anaconda3/lib/python3.5/importlib/__init__.py", line 57, in <module>
import types
File "/home/user/Spark/spark-1.6.3-bin-hadoop2.6/python/pyspark/sql/types.py", line 22, in <module>
import calendar
File "/home/user/anaconda3/lib/python3.5/calendar.py", line 10, in <module>
import locale as _locale
File "/home/user/anaconda3/lib/python3.5/locale.py", line 19, in <module>
import functools
File "/home/user/anaconda3/lib/python3.5/functools.py", line 22, in <module>
from types import MappingProxyType
ImportError: cannot import name 'MappingProxyType'
Is there something obvious I missed here?
Edit:
Dockerfile which can be used to reproduce the problem:
FROM debian:latest
RUN apt-get update
RUN apt-get install -y wget bzip2
RUN wget https://repo.continuum.io/archive/Anaconda3-4.2.0-Linux-x86_64.sh
RUN bash Anaconda3-4.2.0-Linux-x86_64.sh -b -p /anaconda3
RUN wget ftp://ftp.piotrkosoft.net/pub/mirrors/ftp.apache.org/spark/spark-2.1.0/spark-2.1.0-bin-hadoop2.7.tgz
RUN tar xf spark-2.1.0-bin-hadoop2.7.tgz
ENV PATH /anaconda3/bin:$PATH
ENV SPARK_HOME /spark-2.1.0-bin-hadoop2.7
ENV PYTHONPATH $PYTHONPATH:$SPARK_HOME/python/lib/py4j-0.10.4-src.zip:$SPARK_HOME/python
WORKDIR /spark-2.1.0-bin-hadoop2.7
RUN python python/pyspark/sql/tests.py
I suspect this is happening because, the functools module of python3 has the following import: from types import MappingProxyType and, instead of picking up this module from ${CONDA_PREFIX}/lib/python3.5/types.py, it tries to import the module from inside the sql directory: ${SPARK_HOME}/python/pyspark/sql/types.py . The functools module of python2 does not have this import and hence does not throw the error.
A workaround to this, is to somehow import the required types module first and then invoke the script. As a proof of concept:
(root) ~/condaexpts$ PYTHONPATH=$SPARK_HOME/python/lib/py4j-0.10.4-src.zip:$SPARK_HOME/python python
Python 3.5.2 |Anaconda 4.2.0 (64-bit)| (default, Jul 2 2016, 17:53:06)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import types
>>> import os
>>> sqltests=os.environ['SPARK_HOME'] + '/python/pyspark/sql/tests.py'
>>> exec(open(sqltests).read())
.....Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
17/01/30 05:59:43 WARN SparkContext: Support for Java 7 is deprecated as of Spark 2.0.0
17/01/30 05:59:44 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
...
----------------------------------------------------------------------
Ran 128 tests in 372.565s
Also note that there is nothing special about conda. One can see the same thing in a normal virtualenv (with python3):
~/condaexpts$ virtualenv -p python3 venv
Running virtualenv with interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in venv/bin/python3
Also creating executable in venv/bin/python
Installing setuptools, pip...done.
~/condaexpts$ source venv/bin/activate
(venv)~/condaexpts$ python --version
Python 3.4.3
(venv)~/condaexpts$ python $WORKDIR/python/pyspark/sql/tests.py
Traceback (most recent call last):
File "/home/ubuntu/condaexpts/spark-2.1.0-bin-hadoop2.7/python/pyspark/sql/tests.py", line 26, in <module>
import pydoc
File "/usr/lib/python3.4/pydoc.py", line 59, in <module>
import importlib._bootstrap
File "/home/ubuntu/condaexpts/venv/lib/python3.4/importlib/__init__.py", line 40, in <module>
import types
File "/home/ubuntu/condaexpts/spark-2.1.0-bin-hadoop2.7/python/pyspark/sql/types.py", line 22, in <module>
import calendar
File "/usr/lib/python3.4/calendar.py", line 10, in <module>
import locale as _locale
File "/home/ubuntu/condaexpts/venv/lib/python3.4/locale.py", line 20, in <module>
import functools
File "/home/ubuntu/condaexpts/venv/lib/python3.4/functools.py", line 22, in <module>
from types import MappingProxyType
ImportError: cannot import name 'MappingProxyType'
Related
Motivation: The package of interest only works with GCC compiled Python, and I would like to use an IDE (Spyder) to interpret the GCC compiled Python which is not the default MSC version of Python included with Anaconda 3 on windows.
Issue: DLL load failed while importing [package name]: The specified procedure could not be found.
System information:
Windows 10 Home Build 19043.1526 with WSL Enabled.
Bash: MSYS2 MinGW x64
Spyder 5.1.5
gcc version 11.2.0 targeting x86_64-w64-ming32
Python 3.9.10 (main, Jan 25 2022, 17:58:23) [GCC 11.2.0 64 bit (AMD64)]
Reproducible Example:
Download Anaconda 3 which includes Spyder and default MSC compiled
Python
Use Msys2 64 bit with Mingw64 to obtain GCC compiled python. To do
so, you would need to use pacman to obtain mingw-w64-x86_64-toolchain
and python for mingw64.
Install common python packages such as numpy, scipy, and matplotlib
using pacman. You will also need spyder dependencies such as
spyder-kernels and IPython. If the dependencies are not satisfied, Spyder will fail to start the kernel.
At this stage, running python in mingw64 (by typing "python" in
bash) should have no problems. Importing packages should also have
no problems. e.g, import matplotlib.pyplot
Try using Anaconda 3 Spyder to interpret the GCC python by going to:
Tool > Preferences > Python interpreter > Select Use the following
python interpreter > Set the location to
"path to
msys2"\mingw64\bin\python.exe
Depending on the matplotlib graphical settings, you may or may not
already receive an error in the console regarding a DLL load
failure. If not, then importing matplotlib.pyplot or importing
scipy.optimize, should give you the error. However, starting GCC
python in mingw64 will have no problem importing these packages.
Errors:
Python 3.9.10 (main, Jan 25 2022, 17:58:23) [GCC 11.2.0 64 bit (AMD64)]
Type "copyright", "credits" or "license" for more information.
IPython -- An enhanced Interactive Python.
Output from spyder call 'show_mpl_backend_errors':
=========================================================================
NOTE: The following error appeared when setting your Matplotlib backend!!
=========================================================================
Traceback (most recent call last):
File "C:\msys64\mingw64\lib\python3.9\site-packages\spyder_kernels\console\kernel.py", line 739, in _set_mpl_backend
get_ipython().run_line_magic(magic, backend)
File "C:\msys64\mingw64\lib\python3.9\site-packages\IPython\core\interactiveshell.py", line 2364, in run_line_magic
result = fn(*args, **kwargs)
File "C:\msys64\mingw64\lib\python3.9\site-packages\decorator.py", line 232, in fun
return caller(func, *(extras + args), **kw)
File "C:\msys64\mingw64\lib\python3.9\site-packages\IPython\core\magic.py", line 187, in <lambda>
call = lambda f, *a, **k: f(*a, **k)
File "C:\msys64\mingw64\lib\python3.9\site-packages\IPython\core\magics\pylab.py", line 99, in matplotlib
gui, backend = self.shell.enable_matplotlib(args.gui.lower() if isinstance(args.gui, str) else args.gui)
File "C:\msys64\mingw64\lib\python3.9\site-packages\ipykernel\zmqshell.py", line 601, in enable_matplotlib
gui, backend = super(ZMQInteractiveShell, self).enable_matplotlib(gui)
File "C:\msys64\mingw64\lib\python3.9\site-packages\IPython\core\interactiveshell.py", line 3533, in enable_matplotlib
from matplotlib_inline.backend_inline import configure_inline_support
File "C:\msys64\mingw64\lib\python3.9\site-packages\matplotlib_inline\backend_inline.py", line 7, in <module>
from matplotlib.backends.backend_agg import ( # noqa
File "C:\msys64\mingw64\lib\python3.9\site-packages\matplotlib\backends\backend_agg.py", line 35, in <module>
from PIL import Image
File "C:\msys64\mingw64\lib\python3.9\site-packages\PIL\Image.py", line 89, in <module>
from . import _imaging as core
ImportError: DLL load failed while importing _imaging: The specified procedure could not be found.
import matplotlib.pyplot
Traceback (most recent call last):
File "<ipython-input-1-864e826dab68>", line 1, in <module>
import matplotlib.pyplot
File "C:\msys64\mingw64\lib\python3.9\site-packages\matplotlib\pyplot.py", line 36, in <module>
import matplotlib.colorbar
File "C:\msys64\mingw64\lib\python3.9\site-packages\matplotlib\colorbar.py", line 44, in <module>
import matplotlib.contour as contour
File "C:\msys64\mingw64\lib\python3.9\site-packages\matplotlib\contour.py", line 17, in <module>
import matplotlib.text as text
File "C:\msys64\mingw64\lib\python3.9\site-packages\matplotlib\text.py", line 16, in <module>
from .textpath import TextPath # Unused, but imported by others.
File "C:\msys64\mingw64\lib\python3.9\site-packages\matplotlib\textpath.py", line 11, in <module>
from matplotlib.mathtext import MathTextParser
File "C:\msys64\mingw64\lib\python3.9\site-packages\matplotlib\mathtext.py", line 27, in <module>
from PIL import Image
File "C:\msys64\mingw64\lib\python3.9\site-packages\PIL\Image.py", line 89, in <module>
from . import _imaging as core
ImportError: DLL load failed while importing _imaging: The specified procedure could not be found.
import scipy.optimize
Traceback (most recent call last):
File "<ipython-input-2-77fb0d19797b>", line 1, in <module>
import scipy.optimize
File "C:\msys64\mingw64\lib\python3.9\site-packages\scipy\optimize\__init__.py", line 413, in <module>
from ._linprog import linprog, linprog_verbose_callback
File "C:\msys64\mingw64\lib\python3.9\site-packages\scipy\optimize\_linprog.py", line 22, in <module>
from ._linprog_highs import _linprog_highs
File "C:\msys64\mingw64\lib\python3.9\site-packages\scipy\optimize\_linprog_highs.py", line 20, in <module>
from ._highs._highs_wrapper import _highs_wrapper
ImportError: DLL load failed while importing _highs_wrapper: The specified procedure could not be found.
Today I installed python 3.7 from apt-get to try out the new dataclasses module. I installed it seperately because python3.6 wasn't upgradeable to 3.7.
When I type: python3.7 --version, it gives me: >>> Python 3.7.0a2 as my current version.
The problem is that I can't seem to import dataclasses.
my import statement is: from dataclasses import dataclass as instructed here
This is the error message it's giving me:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'dataclasses'
Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
from apport.fileutils import likely_packaged, get_recent_crashes
File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
from apport.report import Report
File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in <module>
import apport.fileutils
File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in <module>
from apport.packaging_impl import impl as packaging
File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 24, in <module>
import apt
File "/usr/lib/python3/dist-packages/apt/__init__.py", line 23, in <module>
import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'
Original exception was:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'dataclasses'
I even tried installing the dataclasses module with pip3: pip3 install dataclasses. Not sure if that's necessary though.
Any suggestion on what might be the problem?
[EDIT] Just tried it with a python3.6 console and it worked fine .. weird.
As suggested by #wim python3.7 -m venv venv_dir
This command will:
Use python3.7 to run the command
The -m flag tells the interpreter to run the next argument as a script
venv is a module, and because of the -m flag it will be run as a script
Finally, the venv_dir is given to the venv module as an argument which
this module will use to create a virtual environment directory at
Once this command is run now you'll have a nice sandbox for messing around/testing the dataclasses module.
To activate this virtual environment be sure to run source venv_dir/bin/activate before you begin. This command will run the script at venv_dir/bin/activate to set up the necessary environment variables and other things for you
To deactivate, simply run deactivate after activating
I'm using python 3.5.2 (64 bit) within a operating system Linux Mint 18 with a kernel release 4.4.0-109-generic.
A long time ago I installed PyQt4 through sudo apt-get install python3-pyqt4 and everything worked perfectly, untill I recently installed through pip PyQt5.
Since then, I'm able to import PyQt4 and import PyQt4.pyqtconfig but not the modules like import PyQt4.QtCore, import PyQt4.QtGui or import PyQt4.Qt for which exits python with 'Segmentation Fault'. I've also realized that the command (within python enviroment) help('modules') gives same output after some seconds.
I've tried to remove PyQt5 with pip and reinstall PyQt4 from terminal (version 4.11.4+dfsg-1build4), but nothing seems to change. I also can see the directory from the PyQt4 from /usr/lib/python3/dist-packages/PyQt4
I've also tried to create a python virtual environment with virtualenv env --python=python3.5 inside a folder and copy the whole directory from PyQt4 to the dist-packages of the environment and also install sip within it. Using sip 4.19.8 version from pip.
Another thing I've tried is to use gdb python3 to run a python program that only contains the code:
#!/usr/bin/python3
import PyQt4.QtCore
What it returns is this:
$ gdb -q python3
Reading symbols from python3...Reading symbols from /usr/lib/debug/.build-id/59/a8ef36ca241df24686952480966d7bc0d7c6ea.debug...done.
done.
(gdb) run FaultPy35.py
Starting program: /usr/bin/python3 FaultPy35.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Program received signal SIGSEGV, Segmentation fault.
PyType_IsSubtype () at ../Objects/typeobject.c:1343
1343 ../Objects/typeobject.c: No such file or directory.
(gdb) c
Continuing.
Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.
(gdb)
I'm really new to gdb but apparently the problem goes beyond python packages...
Note: I've checked via dpkg-query -L python3-pyqt4 what the package python3-pyqt4 has installed in the following list:
/.
/usr
/usr/share
/usr/share/doc
/usr/share/doc/python3-pyqt4
/usr/share/doc/python3-pyqt4/THANKS
/usr/share/doc/python3-pyqt4/copyright
/usr/share/doc/python3-pyqt4/changelog.Debian.gz
/usr/lib
/usr/lib/python3
/usr/lib/python3/dist-packages
/usr/lib/python3/dist-packages/PyQt4
/usr/lib/python3/dist-packages/PyQt4/QtHelp.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/uic
/usr/lib/python3/dist-packages/PyQt4/uic/Compiler
/usr/lib/python3/dist-packages/PyQt4/uic/Compiler/qtproxies.py
/usr/lib/python3/dist-packages/PyQt4/uic/Compiler/proxy_metaclass.py
/usr/lib/python3/dist-packages/PyQt4/uic/Compiler/compiler.py
/usr/lib/python3/dist-packages/PyQt4/uic/Compiler/indenter.py
/usr/lib/python3/dist-packages/PyQt4/uic/Compiler/misc.py
/usr/lib/python3/dist-packages/PyQt4/uic/Compiler/qobjectcreator.py
/usr/lib/python3/dist-packages/PyQt4/uic/Compiler/__init__.py
/usr/lib/python3/dist-packages/PyQt4/uic/objcreator.py
/usr/lib/python3/dist-packages/PyQt4/uic/Loader
/usr/lib/python3/dist-packages/PyQt4/uic/Loader/loader.py
/usr/lib/python3/dist-packages/PyQt4/uic/Loader/qobjectcreator.py
/usr/lib/python3/dist-packages/PyQt4/uic/Loader/__init__.py
/usr/lib/python3/dist-packages/PyQt4/uic/driver.py
/usr/lib/python3/dist-packages/PyQt4/uic/properties.py
/usr/lib/python3/dist-packages/PyQt4/uic/uiparser.py
/usr/lib/python3/dist-packages/PyQt4/uic/pyuic.py
/usr/lib/python3/dist-packages/PyQt4/uic/port_v3
/usr/lib/python3/dist-packages/PyQt4/uic/port_v3/invoke.py
/usr/lib/python3/dist-packages/PyQt4/uic/port_v3/ascii_upper.py
/usr/lib/python3/dist-packages/PyQt4/uic/port_v3/proxy_base.py
/usr/lib/python3/dist-packages/PyQt4/uic/port_v3/load_plugin.py
/usr/lib/python3/dist-packages/PyQt4/uic/port_v3/as_string.py
/usr/lib/python3/dist-packages/PyQt4/uic/port_v3/string_io.py
/usr/lib/python3/dist-packages/PyQt4/uic/port_v3/__init__.py
/usr/lib/python3/dist-packages/PyQt4/uic/__init__.py
/usr/lib/python3/dist-packages/PyQt4/uic/widget-plugins
/usr/lib/python3/dist-packages/PyQt4/uic/widget-plugins/phonon.py
/usr/lib/python3/dist-packages/PyQt4/uic/widget-plugins/qtwebkit.py
/usr/lib/python3/dist-packages/PyQt4/uic/widget-plugins/qtdeclarative.py
/usr/lib/python3/dist-packages/PyQt4/uic/widget-plugins/.noinit
/usr/lib/python3/dist-packages/PyQt4/uic/widget-plugins/qaxcontainer.py
/usr/lib/python3/dist-packages/PyQt4/uic/widget-plugins/qscintilla.py
/usr/lib/python3/dist-packages/PyQt4/uic/icon_cache.py
/usr/lib/python3/dist-packages/PyQt4/uic/exceptions.py
/usr/lib/python3/dist-packages/PyQt4/QtSvg.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/Qt.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/pyqtconfig.py
/usr/lib/python3/dist-packages/PyQt4/QtScriptTools.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/pyqtconfig_nd5.py
/usr/lib/python3/dist-packages/PyQt4/QtDesigner.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/QtCore.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/QtDeclarative.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/QtGui.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/QtNetwork.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/QtXml.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/QtWebKit.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/QtDBus.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/QtAssistant.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/__init__.py
/usr/lib/python3/dist-packages/PyQt4/QtXmlPatterns.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/QtTest.cpython-35m-x86_64-linux-gnu.so
/usr/lib/python3/dist-packages/PyQt4/QtScript.cpython-35m-x86_64-linux-gnu.so
Any Ideas?
Update (25/04/18)
I tried the python 3.5 debug build at import time:
$ python3-dbg
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import PyQt4
>>> import PyQt4.QtCore
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'PyQt4.QtCore'
Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
from apport.fileutils import likely_packaged, get_recent_crashes
File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
from apport.report import Report
File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in <module>
import apport.fileutils
File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in <module>
from apport.packaging_impl import impl as packaging
File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 23, in <module>
import apt
File "/usr/lib/python3/dist-packages/apt/__init__.py", line 23, in <module>
import apt_pkg
ImportError: No module named 'apt_pkg'
Original exception was:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'PyQt4.QtCore'
>>>
Yesterday updated my ubuntu 17.04 to ubuntu 17.10. any comments? Appear when I try to run server in pycharm. #django project.
bash -cl "/home/encuentrum/venv-encuentrum3/bin/python /usr/share/pycharm/helpers/pycharm/django_manage.py check /home/encuentrum/GitLab/encuentrum3/ENCUENTRUM/packers_"
Traceback (most recent call last):
File "/usr/share/pycharm/helpers/pycharm/django_manage.py", line 5, in <module>
from pycharm_run_utils import adjust_django_sys_path
File "/usr/share/pycharm/helpers/pycharm/pycharm_run_utils.py", line 4, in <module>
import imp
File "/home/encuentrum/venv-encuentrum3/lib/python3.6/imp.py", line 19, in <module>
from importlib._bootstrap import _ERR_MSG, _exec, _load, _builtin_from_name
File "/home/encuentrum/venv-encuentrum3/lib/python3.6/importlib/__init__.py", line 57, in <module>
import types
File "/home/encuentrum/venv-encuentrum3/lib/python3.6/types.py", line 171, in <module>
import functools as _functools
File "/home/encuentrum/venv-encuentrum3/lib/python3.6/functools.py", line 23, in <module>
from weakref import WeakKeyDictionary
File "/home/encuentrum/venv-encuentrum3/lib/python3.6/weakref.py", line 12, in <module>
from _weakref import (
ImportError: cannot import name '_remove_dead_weakref'
Maybe you have mixed your multiple Python installations, the newer version of weakref are not compatible with older version python binary, try to remove any one (the older one is recommended) of Python installation.
Analysis
For my case, I have installed older version Python （3.5.1) before, and upgrade my Debian installation. The newer Debian upgrade it's Python3.5 to 3.5.3 which have _remove_dead_weakref in _weakref in its Python binary
When I type $ where python3.5, I get
/usr/local/bin/python3.5
/usr/local/bin/python3.5
/usr/bin/python3.5
The /usr/local/bin/python3.5 is my own older installation, and /usr/bin/python3.5 is Debian offical Python3.5
When I update my Python3.5 installation by apt-get, apt-get execute python3.5 -E -S /usr/lib/python3.5/py_compile.py $files (post-install script) in the deb package.`, it triggers the weakref issue, here is my log
Setting up python3.5-minimal (3.5.3-1+deb9u1) ...
Traceback (most recent call last):
File "/usr/lib/python3.5/py_compile.py", line 6, in <module>
import importlib._bootstrap_external
File "/usr/lib/python3.5/importlib/__init__.py", line 57, in <module>
import types
File "/usr/lib/python3.5/types.py", line 166, in <module>
import functools as _functools
File "/usr/lib/python3.5/functools.py", line 23, in <module>
from weakref import WeakKeyDictionary
File "/usr/lib/python3.5/weakref.py", line 12, in <module>
from _weakref import (
ImportError: cannot import name '_remove_dead_weakref'
I tested Python 3.5.1 and Python 3.5.3 with same import action, here are the compares
Official Python 3.5.3 from apt-get
Python 3.5.3 (default, Sep 27 2018, 17:25:39)
[GCC 6.3.0 20170516] on linux
>>> from _weakref import _remove_dead_weakref
>>>
My own Python 3.5.1 installation
Python 3.5.1 (default, Apr 23 2016, 16:40:21)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> from _weakref import _remove_dead_weakref
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name '_remove_dead_weakref'
>>>
So, I confirm that python3.5 in /usr/local/bin/ cannot use _remove_dead_weakref.
But which python did apt-get use in post-installation script? Try it.
$ which python3.5
/usr/local/bin/python3.5
So, here is why. The post-installation script use my custom installation of python, along with newer python library (/usr/lib/python3.5/weakref.py)
Fix it!
As I said, disable older version of python
sudo mv /usr/local/bin/python3.5 /usr/local/bin/python3.5.bak
Test
$ which python3.5
/usr/bin/python3.5
Adding to #Comzyh answer, this indeed is due to mixed python versions when an upgrade happen due to any reason. A quick fix is to remove your venv python binary i.e. rm <path-to-your-env>/bin/python and copying your system python binary to your venv like cp /usr/bin/python <path-to-your-env>/bin/python. This will fix the weak ref error
From command line I can't import appengine, this might be something with my python path:
$ python
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from google.appengine.ext import db
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "google/appengine/ext/db/__init__.py", line 98, in <module>
from google.appengine.api import datastore
File "google/appengine/api/datastore.py", line 62, in <module>
from google.appengine.datastore import datastore_query
File "google/appengine/datastore/datastore_query.py", line 64, in <module>
from google.appengine.datastore import datastore_index
File "google/appengine/datastore/datastore_index.py", line 60, in <module>
from google.appengine.api import validation
File "google/appengine/api/validation.py", line 51, in <module>
import yaml
ImportError: No module named yaml
>>>
I don't want duplicate installations, I want to point the Python interpretor to where the missing module is. How do I make the interpretor find the app engine modules from the command prompt? In the application these imports are working.
appending:
/usr/local/google_appengine/:/usr/local/google_appengine/lib/:/usr/local/google_appengine/lib/yaml/
to your PYTHONPATH environment variable should do the trick (your SDK location may vary).
For appengine 1.9.6 google has created a new directory "yaml-3.10" that contains the yaml module. I added "[appengine install directory]/google_appengine/lib/yaml-3.10" to PYTHONPATH in my .bashrc file and that solved this problem. BTW: I use Ubuntu 14.04 LTS.
yaml is not installed in your current setup. yaml package is included with google_appengine in the lib folder. the setup.py script in the folder will add the yaml package to your current python
cd google_appengine/lib/yaml
sudo python setup.py install