I'm using pybind11 to create a Python wrapper for a small C++ class.
I'm getting the following error when importing the DLL (running python -v to show Traceback):
>>> from a_py import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 657, in _load_unlocked
File "<frozen importlib._bootstrap>", line 556, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 1101, in create_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: DLL load failed while importing a_py: The specified module could not be found.
The class has just two members - types from the PCL library:
class A
{
pcl::NormalEstimation< pcl::PointXYZ, pcl::Normal> normalEstimation_;
pcl::PointCloud<pcl::Normal>::Ptr normals_;
};
If I remove the first member I can successfully import the module from the DLL.
This is the pybind11 code:
namespace py = pybind11;
PYBIND11_MODULE(a_py, m)
{
py::class_<A>(m, "A");
}
PCL is found using CMake: find_package(PCL REQUIRED)
This happens on both Windows and Linux with the latest PCL (1.10) and older versions too.
(Adding #define PCL_NO_PRECOMPILE before the #includes does not help.)
Update:
I opened a GitHUb issue about this, since this seems like a spurious internal runtime dependency.
The error “The specified module could not be found” is a bit misleading on Windows because it means either the DLL you are trying to load or any of its dependencies cannot be located.
Windows searches for DLLs in the order and paths described here: https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order but usually you have to just ensure that the DLLs you depend on are in the same folder as the DLL you are loading.
Try to open the your DLL in this tool (a more modern version of DependencyWalker): https://github.com/lucasg/Dependencies and look for DLLs which cannot be found.
Related
I tried to follow the instructions at this page to build a custom Julia system image, in order to speed up the initialization phase when using PyJulia.
The command python3 -m julia.sysimage sys.so successfully builds a sys.so image, but then jl = Julia(sysimage="sys.so") seems to fail, since I still get:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
File "/home/jar/.local/lib/python3.7/site-packages/julia/core.py", line 248, in load_module
elif self.julia.isafunction(juliapath):
File "/home/jar/.local/lib/python3.7/site-packages/julia/core.py", line 239, in julia
self.__class__.julia = julia = Julia()
File "/home/jar/.local/lib/python3.7/site-packages/julia/core.py", line 483, in __init__
raise UnsupportedPythonError(jlinfo)
julia.core.UnsupportedPythonError: It seems your Julia and PyJulia setup are not supported.
Julia executable:
julia
Python interpreter and libpython used by PyCall.jl:
/usr/bin/python3
/usr/lib/x86_64-linux-gnu/libpython3.6m.so.1.0
Python interpreter used to import PyJulia and its libpython.
/usr/bin/python3.7
None
Your Python interpreter "/usr/bin/python3.7"
is statically linked to libpython. Currently, PyJulia does not fully
support such Python interpreter.
which is the same issue I am trying to solve by compiling this custom image in the first place.
What am I doing wrong? I would like to avoid the Julia(compiled_modules=False) workaround, because it takes literally forever.
It seems related to this issue, and the following workaround to load the custom image seems to work:
from julia.api import LibJulia
api = LibJulia.load()
api.sysimage = "PATH/TO/CUSTOM/sys.so"
api.init_julia()
from julia import Main
I have installed the bloomberg Python API and set the BLPAPI_ROOT to the VC++ folder.
However, when I import blpapi, I got the following error.
How to get rid of these errors?
Thank you very much.
import blpapi
Traceback (most recent call last):
File "C:\Users\user\AppData\Roaming\Python\Python36\site-packages\blpapi\internals.py", line 39, in swig_import_helper
return importlib.import_module(mname)
File "C:\Program Files\WinPython-64bit-3.6.2.0Qt5\python-3.6.2.amd64\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 978, in _gcd_import
File "<frozen importlib._bootstrap>", line 961, in _find_and_load
File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 648, in _load_unlocked
File "<frozen importlib._bootstrap>", line 560, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 922, in create_module
File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
ImportError: DLL load failed: The specified procedure could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\user\AppData\Roaming\Python\Python36\site-packages\blpapi\__init__.py", line 4, in <module>
from .internals import CorrelationId
File "C:\Users\user\AppData\Roaming\Python\Python36\site-packages\blpapi\internals.py", line 42, in <module>
_internals = swig_import_helper()
File "C:\Users\user\AppData\Roaming\Python\Python36\site-packages\blpapi\internals.py", line 41, in swig_import_helper
return importlib.import_module('_internals')
File "C:\Program Files\WinPython-64bit-3.6.2.0Qt5\python-3.6.2.amd64\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '_internals'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\user\AppData\Roaming\Python\Python36\site-packages\blpapi\__init__.py", line 9, in <module>
raise debug_load_error(error)
ImportError: No module named '_versionhelper'
Could not open the C++ SDK library.
Download and install the latest C++ SDK from:
http://www.bloomberg.com/professional/api-library
If the C++ SDK is already installed, please ensure that the path to the library
was added to PATH before entering the interpreter.
I did 2 things to solve an issue similar to that:
1- I installed Microsoft Visual Studio with the following components
C++/CLI Support
VC++ 2015.3 v14.00 (v140) toolset for desktop
Visual C++ MFC for x86 and x64
Visual C++ ATL for x86 and x64
2- I manually copied the .dll files in C++API\lib (blpapi3_32.dll and blpapi3_64.dll in my case) into C:\windows\system32 where all the dll files that system uses.
Also, I copied the dll files in in C++API\lib into C:\blp\DAPI, replacing the new ones with the old ones.
Please set BLPAPI_ROOT environment variable to the location where the blpapi C++ SDK is located.
An alternative option is to use conda, as discussed here and here.
If you use conda you don't need to worry about downloading the C++ SDK and properly configuring the BLPAPI_ROOT. From the conda-forge channel this can be done using
conda install -c conda-forge blpapi
I'm trying to use CNTK in Python/VS2017. I'm experienced in VS but new to Python and CNTK.
I've installed CNTK into Anaconda 4.1.1 and I've created a custom Anaconda environment pointing to C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\cntk-py35.
If I open an Interactive Window from the "Python Environments" list in VS2017, I can import CNTK.
However, if I create an empty Python project in VS2017, then open an Interactive Window from the same Anaconda environment shown in the project "Python Environments" list, I get a "module not found".
Any help is appreciated,
Bill
Traceback (most recent call last):
File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\cntk-py35\lib\site-packages\cntk\cntk_py.py", line 18, in swig_import_helper
return importlib.import_module(mname)
File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\cntk-py35\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 986, in _gcd_import
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 956, in _find_and_load_unlocked
ImportError: No module named 'cntk._cntk_py'
This seems like a flaw in VS2017 to me, but this is a solution, if not a good one.
In the Python project properties I changed the Working director from "." to the full path of the Anaconda environment.
I am using Python 3.4.3 | Anaconda 2.3.0 (x86_64) on OS X 10.10.5 and following these instructions to install MATLAB Engine for Python. Installation seems to succeed, but upon launching python & attempting to import, I see the following:
Python 3.4.3 |Anaconda 2.3.0 (x86_64)| (default, Oct 20 2015, 14:27:51)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import matlab.engine
Traceback (most recent call last):
File "/Users/tyler/.local/lib/python3.4/site-packages/matlab/engine/__init__.py", line 42, in <module>
pythonengine = importlib.import_module("matlabengineforpython"+_PYTHONVERSION)
File "/Users/tyler/anaconda/lib/python3.4/importlib/__init__.py", line 109, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
File "<frozen importlib._bootstrap>", line 2224, in _find_and_load_unlocked
ImportError: No module named 'matlabengineforpython3_4'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/tyler/.local/lib/python3.4/site-packages/matlab/engine/__init__.py", line 58, in <module>
pythonengine = importlib.import_module("matlabengineforpython"+_PYTHONVERSION)
File "/Users/tyler/anaconda/lib/python3.4/importlib/__init__.py", line 109, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 1191, in _load_unlocked
File "<frozen importlib._bootstrap>", line 1161, in _load_backward_compatible
File "<frozen importlib._bootstrap>", line 539, in _check_name_wrapper
File "<frozen importlib._bootstrap>", line 1715, in load_module
File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
ImportError: dlopen(/Applications/MATLAB_R2015b.app/extern/engines/python/dist/matlab/engine/maci64/matlabengineforpython3_4.so, 2): Library not loaded: #rpath/libpython3.4m.dylib
Referenced from: /Applications/MATLAB_R2015b .app/extern/engines/python/dist/matlab/engine/maci64/matlabengineforpython3_4.so
Reason: image not found
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/tyler/.local/lib/python3.4/site-packages/matlab/engine/__init__.py", line 60, in <module>
raise EnvironmentError('The installation of MATLAB Engine for Python is '
OSError: The installation of MATLAB Engine for Python is corrupted. Please reinstall it or contact MathWorks Technical Support for assistance.
While researching this issue, I found several other references to this exact issue on MathWorks' website, all without a resolution. The final link suggests the problem may be with dynamic linking.
Given how many scientific toolboxes are only available in MATLAB, it would be of great use to be able to call this engine using Anaconda. While there are other packages available that support this functionality, they invariably have limitations or other complexity like a client-server architecture.
Any suggestions are most welcome.
According to my communication with MathWorks support, MATLAB Engine for Python is incompatible with Anaconda. Fortunately, this is incorrect.
Thanks to some direction from the Anaconda mailing list, I found a simple solution:
sudo ln -s ~/anaconda/lib/libpython3.4m.dylib /usr/lib
After this, matlab.engine imports successfully. Hope this helps someone!
Edit: As of OS X 10.11 El Capitan, the workaround needs to be tweaked due to the Rootless feature, as root no longer has permission to modify /usr/lib
sudo ln -s ~/anaconda/lib/libpython3.4m.dylib /usr/local/lib
If you're trying to install matlab engine in a conda environment, follow the below steps:
cd "matlabroot\extern\engines\python"
python setup.py install --prefix="installdir"
Eg
cd /opt/MATLAB/R2019a/extern/engines/python
python setup.py install --prefix="/home/nagabhushan/anaconda3/"
Now, matlab engine package is actually installed under the separate
anaconda3/lib/python3.6/site-package/matlab
So you need to manually move that matlab folder to the site-package folder your python is using. If you have multiple virtual envs, move it to the corresponding site-package folder of the virtual env you will run program.
Eg
anaconda3/envs/my_env/lob/python3.6/site-packages/matlab
References:
https://stackoverflow.com/a/41800724/3337089
https://stackoverflow.com/a/56553740/3337089
Edit 1 (03-Mar-2020):
I tried activating my environment and then simply installing matlab engine and it worked! I'm using Python 3.7
python setup.py install
Note that this initially gave error that error: You do not have write permission in build/lib/matlab/engine/, but changing the permission of build directory recursively worked: sudo chmod -R 777 build/
After the installation of motionless library, I try to run my code and the following error message occurs.
**Traceback (most recent call last):
File "C:\Users\Kevin\Downloads\tracker.py", line 4, in <module>
from motionless import CenterMap
File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
File "<frozen importlib._bootstrap>", line 2222, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 2164, in _find_spec
File "<frozen importlib._bootstrap>", line 1940, in find_spec
File "<frozen importlib._bootstrap>", line 1916, in _get_spec
File "<frozen importlib._bootstrap>", line 1897, in _legacy_get_spec
File "<frozen importlib._bootstrap>", line 863, in spec_from_loader
File "<frozen importlib._bootstrap>", line 904, in spec_from_file_location
File "C:\Python34\lib\site-packages\motionless-1.1-py3.4.egg\motionless.py",
line 55
if label and (len(label) <> 1 or not label in Marker.LABELS):
^
SyntaxError: invalid syntax**
Motionless hasn't been updated since 06/08/2010 according to the PyPi Package Index.
I've downloaded it and get the same error immediately just by running:
import motionless
print(motionless.__version__)
It's also not flagged as being Python 3.4 compatible in PyPi; if you are running the latest version of Python this is likely the issue. Have you tried running it with Python 2.7 instead?
Edit: Looking at the Python 2.7 docs; it states here that != and <> are equvilent, however <> is deprecated. In the Python 3.4 docs it states here that only != is supported, no mention of <> so I imagine it's been removed.
You could try instead:
Raising an issue on the GitHub Repo; the author may still be updating the library and not know it's incompatible with Python 3.4
Checking out the code yourself from the GitHub Repo and manually fixing the problem (Check out 2to3 for automatically doing this. It will convert all <> usage to != for you)