[OS info: 5.14.14-arch1-1 x86_64 GNU/Linux]
After creating a Python v3.7.0 virtual-env with Pyenv, with the --no-site-packages option, I install packages with pip, using:
$ python -m -pip install <package-name>
An example is either Pandas v1.1.5 or Torch v1.7.0+cpu installed in the virtual-env. Then, starting a Python 3.7.0 shell in console, simply by issuing python in terminal in that environment, I try to import those packages. The result is:
$ python
Python 3.7.0 (default, Oct 20 2021, 09:16:31)
[GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
+>>> import faulthandler; faulthandler.enable()
+>>> import torch
Fatal Python error: Segmentation fault
Current thread 0x00007fac608f22c0 (most recent call first):
File "/home/USER/.pyenv/versions/3.7.0/lib/python3.7/ctypes/__init__.py", line 273 in _reset_cache
File "/home/USER/.pyenv/versions/3.7.0/lib/python3.7/ctypes/__init__.py", line 543 in <module>
File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
File "<frozen importlib._bootstrap_external>", line 728 in exec_module
File "<frozen importlib._bootstrap>", line 677 in _load_unlocked
File "<frozen importlib._bootstrap>", line 967 in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 983 in _find_and_load
File "/home/USER/.pyenv/versions/3.7.0/lib/python3.7/site-packages/torch/__init__.py", line 16 in <module>
File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
File "<frozen importlib._bootstrap_external>", line 728 in exec_module
File "<frozen importlib._bootstrap>", line 677 in _load_unlocked
File "<frozen importlib._bootstrap>", line 967 in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 983 in _find_and_load
File "<stdin>", line 1 in <module>
Segmentation fault
$
The interesting thing is that if in that same virtual-env, I put together an iPython kernel, like so:
$ ipython kernel install --user --name <my_project> --display-name "<Python3.7.0 (my_project)>"
and import the same package(s), say from a Jupyter notebook running that kernel, there is absolutely no issue. This tells me that there is no version issue (although dependency issues could occur still, I suppose. The segmentation fault only occurs in the Python shell in console. Also when I try importing from my pyenv global environment (3.10.0), there seems to be no issue whatsoever.
Any help to understand why and how to resolve the issue is welcome.
This is a tentative answer as I lack of a definite proof of solution. Anyone more comfortable than I am with debugging is welcome to add to it or to write their own answer.
(i) I originally thought something along the line of a corrupted ~/.inputrc might be responsible for the "hoopla" along the lines of this answer on SO. However nothing had changed in that ~/.inputrc file in past months. So the issue was more likely to be a problematic update sequence.
(ii) Looking at lines 273 and 543 in /home/USER/.pyenv/versions/3.7.0/lib/python3.7/ctypes/__init__.py the _reset_cache() function seems to trigger the segfault, just as module ctypes is imported for instance in ../torch/__init__.py.
(iii) updating pyenv to the latest version build (v2.1.0-1) and, a day later, gdbm aka GNU dbm (v1.22-1), the database function library on which Python depends, seems to have solved the issue.(Co-)incidentally both packages are maintained by the same person. ;-)
Still unsure about exactly what was at play here. Consulting the GDBM documentation did not help. Sorry about the incomplete answer.
Related
I tried to follow the instructions at this page to build a custom Julia system image, in order to speed up the initialization phase when using PyJulia.
The command python3 -m julia.sysimage sys.so successfully builds a sys.so image, but then jl = Julia(sysimage="sys.so") seems to fail, since I still get:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
File "/home/jar/.local/lib/python3.7/site-packages/julia/core.py", line 248, in load_module
elif self.julia.isafunction(juliapath):
File "/home/jar/.local/lib/python3.7/site-packages/julia/core.py", line 239, in julia
self.__class__.julia = julia = Julia()
File "/home/jar/.local/lib/python3.7/site-packages/julia/core.py", line 483, in __init__
raise UnsupportedPythonError(jlinfo)
julia.core.UnsupportedPythonError: It seems your Julia and PyJulia setup are not supported.
Julia executable:
julia
Python interpreter and libpython used by PyCall.jl:
/usr/bin/python3
/usr/lib/x86_64-linux-gnu/libpython3.6m.so.1.0
Python interpreter used to import PyJulia and its libpython.
/usr/bin/python3.7
None
Your Python interpreter "/usr/bin/python3.7"
is statically linked to libpython. Currently, PyJulia does not fully
support such Python interpreter.
which is the same issue I am trying to solve by compiling this custom image in the first place.
What am I doing wrong? I would like to avoid the Julia(compiled_modules=False) workaround, because it takes literally forever.
It seems related to this issue, and the following workaround to load the custom image seems to work:
from julia.api import LibJulia
api = LibJulia.load()
api.sysimage = "PATH/TO/CUSTOM/sys.so"
api.init_julia()
from julia import Main
i need some help with my leaflet config.
i'm getting this error when running the server since i added leaflet to my django-3.04 project.
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/Users/marcusbey/.local/share/virtualenvs/rb-website-fKSjEdfu/src/django-leaflet/leaflet/__init__.py", line 80, in <module>
if DEFAULT_PRECISION is not None and not (isinstance(DEFAULT_PRECISION, six.integer_types) and (4 <= DEFAULT_PRECISION <= 12)):
NameError: name 'six' is not defined
I installed six and add it to the settings.py file but still getting that same error. i don't know where else to look and what to do.
:/
i followed this answer and it finally worked.
go to https://pypi.org/project/six/#files
download "six-1.14.0.tar.gz (33.9 kB)"
unzip it, copy and paste "six.py" into your source directory.
import "six" module into your source code (import six)
run source script.
https://stackoverflow.com/a/61327529/8571945
Thanks guys
Django used to come packaged with six, but version 3.0 has dropped it since Python 2 support has been completely removed: https://docs.djangoproject.com/en/3.0/releases/3.0/#removed-private-python-2-compatibility-apis
For now, if you need django-leaflet, you may have to stay on Django version 2.2. However, it looks like they're in the process of supporting Django 3. As of this writing, the last commit from 20 days ago is, Drop support for Python 2:
https://github.com/makinacorpus/django-leaflet/commit/844887affe607d3f115920c862f8ea2b45e19ed8
It may be worth trying an install from this commit into your virtual environment to see if it solves the issue:
pip install git+https://github.com/makinacorpus/django-leaflet.git#844887affe607d3f115920c862f8ea2b45e19ed8
I attempted it into a venv, and it didn't seem to require six any more.
Either way, it looks like they're working on a release for Django 3.0, it just isn't available from PyPI for pip install quite yet. Good luck!
I am new to Python and am trying to setup environment so I can use MS SQL. My simple python code fails on: import pyodbc with following error:
ImportError: DLL load failed while importing pyodbc: The specified module could not be found.
Background:
I installed Python 3.8.1 for windows 64-bit.
Installed pyodbc (pip install pyodbc) with no issue.
Installed Microsoft ODBC Driver 17 for SQL Server for Windows 64-bit
TRACE:
Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
File "dbtest.py", line 10, in <module>
import pyodbc
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 657, in _load_unlocked
File "<frozen importlib._bootstrap>", line 556, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 1101, in create_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: DLL load failed while importing pyodbc: The specified module could not be found.
I fixed my issue by downgrading pyodbc package from 4.0.28 to 4.0.27 and now it works.
(It's frustrating when newest packages don't work together)
I used a following command:
pip install --upgrade pyodbc==4.0.27
The root cause could be because your IDE (VS Code in many cases) has the wrong installation of Python installed. You may have to use some of the other fixes like deleting temp files, uninstalling/reinstalling PYODBC or downgrading.
If none of those work, you may have the wrong installation of Python selected in VS Code.
To change, go to VS Code and hit "Ctrl+Shift+P" then type "Python: S"
Select the python interpreter line and a the options will change to show you all the registered versions of Python installed. In my case I wanted the version I manually installed from Python in the root of my C drive. Selecting this version allowed my installation of PYODBC to be found and imported without further tweaking.
I'm trying to use CNTK in Python/VS2017. I'm experienced in VS but new to Python and CNTK.
I've installed CNTK into Anaconda 4.1.1 and I've created a custom Anaconda environment pointing to C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\cntk-py35.
If I open an Interactive Window from the "Python Environments" list in VS2017, I can import CNTK.
However, if I create an empty Python project in VS2017, then open an Interactive Window from the same Anaconda environment shown in the project "Python Environments" list, I get a "module not found".
Any help is appreciated,
Bill
Traceback (most recent call last):
File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\cntk-py35\lib\site-packages\cntk\cntk_py.py", line 18, in swig_import_helper
return importlib.import_module(mname)
File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\cntk-py35\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 986, in _gcd_import
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 956, in _find_and_load_unlocked
ImportError: No module named 'cntk._cntk_py'
This seems like a flaw in VS2017 to me, but this is a solution, if not a good one.
In the Python project properties I changed the Working director from "." to the full path of the Anaconda environment.
I am using Python 3.4.3 | Anaconda 2.3.0 (x86_64) on OS X 10.10.5 and following these instructions to install MATLAB Engine for Python. Installation seems to succeed, but upon launching python & attempting to import, I see the following:
Python 3.4.3 |Anaconda 2.3.0 (x86_64)| (default, Oct 20 2015, 14:27:51)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import matlab.engine
Traceback (most recent call last):
File "/Users/tyler/.local/lib/python3.4/site-packages/matlab/engine/__init__.py", line 42, in <module>
pythonengine = importlib.import_module("matlabengineforpython"+_PYTHONVERSION)
File "/Users/tyler/anaconda/lib/python3.4/importlib/__init__.py", line 109, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
File "<frozen importlib._bootstrap>", line 2224, in _find_and_load_unlocked
ImportError: No module named 'matlabengineforpython3_4'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/tyler/.local/lib/python3.4/site-packages/matlab/engine/__init__.py", line 58, in <module>
pythonengine = importlib.import_module("matlabengineforpython"+_PYTHONVERSION)
File "/Users/tyler/anaconda/lib/python3.4/importlib/__init__.py", line 109, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 1191, in _load_unlocked
File "<frozen importlib._bootstrap>", line 1161, in _load_backward_compatible
File "<frozen importlib._bootstrap>", line 539, in _check_name_wrapper
File "<frozen importlib._bootstrap>", line 1715, in load_module
File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
ImportError: dlopen(/Applications/MATLAB_R2015b.app/extern/engines/python/dist/matlab/engine/maci64/matlabengineforpython3_4.so, 2): Library not loaded: #rpath/libpython3.4m.dylib
Referenced from: /Applications/MATLAB_R2015b .app/extern/engines/python/dist/matlab/engine/maci64/matlabengineforpython3_4.so
Reason: image not found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/tyler/.local/lib/python3.4/site-packages/matlab/engine/__init__.py", line 60, in <module>
raise EnvironmentError('The installation of MATLAB Engine for Python is '
OSError: The installation of MATLAB Engine for Python is corrupted. Please reinstall it or contact MathWorks Technical Support for assistance.
While researching this issue, I found several other references to this exact issue on MathWorks' website, all without a resolution. The final link suggests the problem may be with dynamic linking.
Given how many scientific toolboxes are only available in MATLAB, it would be of great use to be able to call this engine using Anaconda. While there are other packages available that support this functionality, they invariably have limitations or other complexity like a client-server architecture.
Any suggestions are most welcome.
According to my communication with MathWorks support, MATLAB Engine for Python is incompatible with Anaconda. Fortunately, this is incorrect.
Thanks to some direction from the Anaconda mailing list, I found a simple solution:
sudo ln -s ~/anaconda/lib/libpython3.4m.dylib /usr/lib
After this, matlab.engine imports successfully. Hope this helps someone!
Edit: As of OS X 10.11 El Capitan, the workaround needs to be tweaked due to the Rootless feature, as root no longer has permission to modify /usr/lib
sudo ln -s ~/anaconda/lib/libpython3.4m.dylib /usr/local/lib
If you're trying to install matlab engine in a conda environment, follow the below steps:
cd "matlabroot\extern\engines\python"
python setup.py install --prefix="installdir"
Eg
cd /opt/MATLAB/R2019a/extern/engines/python
python setup.py install --prefix="/home/nagabhushan/anaconda3/"
Now, matlab engine package is actually installed under the separate
anaconda3/lib/python3.6/site-package/matlab
So you need to manually move that matlab folder to the site-package folder your python is using. If you have multiple virtual envs, move it to the corresponding site-package folder of the virtual env you will run program.
Eg
anaconda3/envs/my_env/lob/python3.6/site-packages/matlab
References:
https://stackoverflow.com/a/41800724/3337089
https://stackoverflow.com/a/56553740/3337089
Edit 1 (03-Mar-2020):
I tried activating my environment and then simply installing matlab engine and it worked! I'm using Python 3.7
python setup.py install
Note that this initially gave error that error: You do not have write permission in build/lib/matlab/engine/, but changing the permission of build directory recursively worked: sudo chmod -R 777 build/