I tried to follow the instructions at this page to build a custom Julia system image, in order to speed up the initialization phase when using PyJulia.
The command python3 -m julia.sysimage sys.so successfully builds a sys.so image, but then jl = Julia(sysimage="sys.so") seems to fail, since I still get:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
File "/home/jar/.local/lib/python3.7/site-packages/julia/core.py", line 248, in load_module
elif self.julia.isafunction(juliapath):
File "/home/jar/.local/lib/python3.7/site-packages/julia/core.py", line 239, in julia
self.__class__.julia = julia = Julia()
File "/home/jar/.local/lib/python3.7/site-packages/julia/core.py", line 483, in __init__
raise UnsupportedPythonError(jlinfo)
julia.core.UnsupportedPythonError: It seems your Julia and PyJulia setup are not supported.
Julia executable:
julia
Python interpreter and libpython used by PyCall.jl:
/usr/bin/python3
/usr/lib/x86_64-linux-gnu/libpython3.6m.so.1.0
Python interpreter used to import PyJulia and its libpython.
/usr/bin/python3.7
None
Your Python interpreter "/usr/bin/python3.7"
is statically linked to libpython. Currently, PyJulia does not fully
support such Python interpreter.
which is the same issue I am trying to solve by compiling this custom image in the first place.
What am I doing wrong? I would like to avoid the Julia(compiled_modules=False) workaround, because it takes literally forever.
It seems related to this issue, and the following workaround to load the custom image seems to work:
from julia.api import LibJulia
api = LibJulia.load()
api.sysimage = "PATH/TO/CUSTOM/sys.so"
api.init_julia()
from julia import Main
Related
I'm using pybind11 to create a Python wrapper for a small C++ class.
I'm getting the following error when importing the DLL (running python -v to show Traceback):
>>> from a_py import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 657, in _load_unlocked
File "<frozen importlib._bootstrap>", line 556, in module_from_spec
File "<frozen importlib._bootstrap_external>", line 1101, in create_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: DLL load failed while importing a_py: The specified module could not be found.
The class has just two members - types from the PCL library:
class A
{
pcl::NormalEstimation< pcl::PointXYZ, pcl::Normal> normalEstimation_;
pcl::PointCloud<pcl::Normal>::Ptr normals_;
};
If I remove the first member I can successfully import the module from the DLL.
This is the pybind11 code:
namespace py = pybind11;
PYBIND11_MODULE(a_py, m)
{
py::class_<A>(m, "A");
}
PCL is found using CMake: find_package(PCL REQUIRED)
This happens on both Windows and Linux with the latest PCL (1.10) and older versions too.
(Adding #define PCL_NO_PRECOMPILE before the #includes does not help.)
Update:
I opened a GitHUb issue about this, since this seems like a spurious internal runtime dependency.
The error “The specified module could not be found” is a bit misleading on Windows because it means either the DLL you are trying to load or any of its dependencies cannot be located.
Windows searches for DLLs in the order and paths described here: https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order but usually you have to just ensure that the DLLs you depend on are in the same folder as the DLL you are loading.
Try to open the your DLL in this tool (a more modern version of DependencyWalker): https://github.com/lucasg/Dependencies and look for DLLs which cannot be found.
We are deploying python based Azure Function using Azure CLI but getting this below error while importing cv2 library.
ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory
We are not using any docker image. Would prefer a non-docker based solution.
Exception: ImportError: libgthread-2.0.so.0: cannot open shared object file: No such file or directory
Stack: File "/usr/local/lib/python3.6/site-packages/azure_functions_worker/dispatcher.py", line 239, in _handle__function_load_request
func_request.metadata.entry_point)
File "/usr/local/lib/python3.6/site-packages/azure_functions_worker/loader.py", line 66, in load_function
mod = importlib.import_module(fullmodname)
File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/site/wwwroot/FaceFinderHttpTrigger/__init__.py", line 11, in <module>
import cv2
File "/home/site/wwwroot/.python_packages/lib/python3.6/site-packages/cv2/__init__.py", line 3, in <module>
from .cv2 import *
There is a similar SO thread Azure Functions: In a Python function under Linux, how do I import a non-standard module such as opencv/imutils? with the same issue as yours.
Due to the libgthread shared library is a Linux base library, per my experience, it seems to be caused by Azure CLI could not package these Python packages with native dependencies as Azure Function. So the only solution I think is to build your Azure Function as a Docker container by yourself as the offical document Create a function on Linux using a custom image said, or try to use Azure CLI command func azure functionapp publish with --build-native-deps option to automatically build and configure the dependencies using a Docker container.
Also as I known, there is a blog OPENCV WITH AZURE FUNCTIONS which seems to work fine with opencv for Azure Functions, but it just run on local development environment, not on cloud, as my view. So I still recommend you to deploy your app as a Docker contaier for the app using native dependencies.
I have a code for predicting some value that uses xgboost package in the code. When I run it in PyCharm, it runs as expected.
The problem is when I make an executable file using pyinstaller. It will make the exe without any error, but when I run it the following error is raised:
Traceback (most recent call last):
File "test_fraud.py", line 3, in <module>
import xgboost
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in
_find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File
"C:\Users\ShubhamSingh\PycharmProjects\cfna_scoring\venv\lib\site-
packages\PyInstaller\loader\pyimod03_importers.py", line 627, in
exec_module
exec(bytecode, module.__dict__)
File "lib\site-packages\xgboost\__init__.py", line 11, in <module>
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in
_find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "C:\Users\ShubhamSingh\PycharmProjects\cfna_scoring\venv\lib\site-
packages\PyInstaller\loader\pyimod03_importers.py", line 627, in
exec_module
exec(bytecode, module.__dict__)
File "lib\site-packages\xgboost\core.py", line 161, in <module>
File "lib\site-packages\xgboost\core.py", line 123, in _load_lib
File "lib\site-packages\xgboost\libpath.py", line 48, in find_lib_path
xgboost.libpath.XGBoostLibraryNotFound: Cannot find XGBoost Library in
the candidate path, did you install compilers and run build.sh in root
path?
List of candidates:
C:\Users\SHUBHA~1\AppData\Local\Temp\_MEI11402\xgboost\xgboost.dll
C:\Users\SHUBHA~1\AppData\Local\Temp\_MEI11402\xgboost
\../../lib/xgboost.dll
C:\Users\SHUBHA~1\AppData\Local\Temp\_MEI11402\xgboost\./lib/xgboost.dll
C:\Users\SHUBHA~1\AppData\Local\Temp\_MEI11402\xgboost\xgboost.dll
C:\Users\SHUBHA~1\AppData\Local\Temp\_MEI11402\xgboost
\../../windows/x64/Release/xgboost.dll
C:\Users\SHUBHA~1\AppData\Local\Temp\_MEI11402\
xgboost\./windows/x64/Release/xgboost.dll
[6564] Failed to execute script test_fraud
What's wrong here?
It seems that Pyinstaller can't find the xgboost.dll, VERSION files. So you need to add them manually to your output package. I also suggest you use a try/except block to see what is going on.
Suppose this simple example:
import traceback
try:
import xgboost
input("xgboost imported successfully!")
except Exception:
traceback.print_exc()
input("Import Error!")
I suggest you use an env to build your script so, you need to add the xgboost directory located at <path_to_venv>/xgboost and VERSION file located at <path_to_venv>/Lib/site-packages/xboost. Next, add them as a data-file with pyinstaller. Launch your env and execute the below command (My virtualenv named as env):
├───myscript.py
├───env
Command:
(env) > pyinstaller myscript.py -F --add-data "./env/xgboost/*;xgboost/" --add-data "./env/Lib/site-packages/xgboost/VERSION;xgboost/"
I'm trying to use CNTK in Python/VS2017. I'm experienced in VS but new to Python and CNTK.
I've installed CNTK into Anaconda 4.1.1 and I've created a custom Anaconda environment pointing to C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\cntk-py35.
If I open an Interactive Window from the "Python Environments" list in VS2017, I can import CNTK.
However, if I create an empty Python project in VS2017, then open an Interactive Window from the same Anaconda environment shown in the project "Python Environments" list, I get a "module not found".
Any help is appreciated,
Bill
Traceback (most recent call last):
File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\cntk-py35\lib\site-packages\cntk\cntk_py.py", line 18, in swig_import_helper
return importlib.import_module(mname)
File "C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\cntk-py35\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 986, in _gcd_import
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 956, in _find_and_load_unlocked
ImportError: No module named 'cntk._cntk_py'
This seems like a flaw in VS2017 to me, but this is a solution, if not a good one.
In the Python project properties I changed the Working director from "." to the full path of the Anaconda environment.
After the installation of motionless library, I try to run my code and the following error message occurs.
**Traceback (most recent call last):
File "C:\Users\Kevin\Downloads\tracker.py", line 4, in <module>
from motionless import CenterMap
File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
File "<frozen importlib._bootstrap>", line 2222, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 2164, in _find_spec
File "<frozen importlib._bootstrap>", line 1940, in find_spec
File "<frozen importlib._bootstrap>", line 1916, in _get_spec
File "<frozen importlib._bootstrap>", line 1897, in _legacy_get_spec
File "<frozen importlib._bootstrap>", line 863, in spec_from_loader
File "<frozen importlib._bootstrap>", line 904, in spec_from_file_location
File "C:\Python34\lib\site-packages\motionless-1.1-py3.4.egg\motionless.py",
line 55
if label and (len(label) <> 1 or not label in Marker.LABELS):
^
SyntaxError: invalid syntax**
Motionless hasn't been updated since 06/08/2010 according to the PyPi Package Index.
I've downloaded it and get the same error immediately just by running:
import motionless
print(motionless.__version__)
It's also not flagged as being Python 3.4 compatible in PyPi; if you are running the latest version of Python this is likely the issue. Have you tried running it with Python 2.7 instead?
Edit: Looking at the Python 2.7 docs; it states here that != and <> are equvilent, however <> is deprecated. In the Python 3.4 docs it states here that only != is supported, no mention of <> so I imagine it's been removed.
You could try instead:
Raising an issue on the GitHub Repo; the author may still be updating the library and not know it's incompatible with Python 3.4
Checking out the code yourself from the GitHub Repo and manually fixing the problem (Check out 2to3 for automatically doing this. It will convert all <> usage to != for you)