Python version: 3.8.1
Spyder version: 3.3.6
Qt version: 5.12.9
Wrapper: develop using PyBind11
I am wrapping a dll develop in C++ which use Qt dlls to be used with Python. I wrote the wrapper with Visual Studio 2019 using the compiler MSVC (as my dll is compiled with MSVC). After generating the solution in VS2019 I obtain a .pyd file which can be import with python.
It works good when I use python on line command:
Start cmd.exe
$python
import MyLibName
I can use the functions/classes ...
But if I try with Spyder, I get the following error:
ImportError: DLL load failed while importing PyStack: The specified module could not be found..
So here are my questions :
Is there a way to get more information about ImportError like the name of the missing dll or something?
I don't understand why the issue only happen with spyder. I tried with IPython Qt Console and it work. Does spyder use a embeded python version or something ?
I don't fully understand how dll shall be managed, I mean shall I provide dll like libGLESV2.dll with the .pyd or just give a path where to find it ?
Thank you in advance.
My guess
I think I find out which part of Qt/python is producing this issue, but I still don't know how to solve it.
My dll use signals/slots which need an event loop to be performed. If an event loop is already running the dll will try to use it, if the loop version (ex : PyQt5==5.14.1) isn’t the same as mine (ex Qt==5.15.1) import will be impossible.
Note that the reverse is true, if I run my dll an then try to start a loop with %gui qt the command will throw an error.
How to reproduce the issue :
Compile a Qt project available here.
Copy the output dll in the folder PyMyStack/dependencies of the VS Project (available here)
Compile the VS project.
Open an IPython console (without using qt has event loop)
Import the module created with VS (Import PyMyStack)
Run the magic command %gui qt
Last command shall print : ERROR:root:DLL load failed while importing QtSvg: The specified procedure could not be found.
How to hide/solve the problem:
Disclaimer : The solutions presented here are surely not the best, if you know a better one please share it ☺
If you just want to import your lib in Spyder, you can use another event loop. Here are the steps to change this:
In Spyder menus go to Tools→Preferences
Select “IPython console”
Go to “Graphics” tab and change the backend combo box to any other values than Qt or Automatic
If you want to use Qt event loop you will have to update it. You can do this with pip command, but remember than Spyder is not compatible with some version. Here is the pip command:
Pip install PyQt5==X.Y.Z
Where X and Y are the same version use to compile your Qt project. The last digit version seems to not be important.
Related
I'm trying to embed python with numpy in a C++ application. I'm using Windows 10 and Visual Studio 2015.
Currently I have Anaconda and WinPython installed (because I'm using Python scripts that work only with one or the other). I don't have any environment variable related to python.
For my C++ application, I'm using the WinPython python, that have numpy and a handful of other packages installed. I managed to embed python and numpy in my application when using Visual studio, both for Debug and Release. Everything is working, python is initialized and I can use numpy array and functions. WinPython is correctly used. As a simple test in my code I have:
_putenv_s("PYTHONPATH", ".");
Py_InitializeEx(0);
PyRun_SimpleString("import sys");
PyRun_SimpleString("print(sys.path)");
PyRun_SimpleString("print(sys.prefix)");
PyRun_SimpleString("print(sys.executable)");
PyRun_SimpleString("import importlib.machinery");
PyRun_SimpleString("print(importlib.machinery.all_suffixes())");
init_numpy2();
That prints:
['C:\\DevC++\\Tesseler-Cmake\\build', 'C:\\Git\\WPy64-3741\\python-3.7.4.amd64\\python37.zip', 'C:\\Git\\WPy64-3741\\python-3.7.4.amd64\\DLLs', 'C:\\Git\\WPy64-3741\\python-3.7.4.amd64\\lib', 'C:\\DevC++\\Tesseler-Cmake\\build\\Release', 'C:\\Git\\WPy64-3741\\python-3.7.4.amd64', 'C:\\Git\\WPy64-3741\\python-3.7.4.amd64\\lib\\site-packages', 'C:\\Git\\WPy64-3741\\python-3.7.4.amd64\\lib\\site-packages\\win32', 'C:\\Git\\WPy64-3741\\python-3.7.4.amd64\\lib\\site-packages\\win32\\lib', 'C:\\Git\\WPy64-3741\\python-3.7.4.amd64\\lib\\site-packages\\Pythonwin']
C:\Git\WPy64-3741\python-3.7.4.amd64
C:\DevC++\Tesseler-Cmake\build\Release\Tesseler.exe
['.py', '.pyw', '.pyc', '.cp37-win_amd64.pyd', '.pyd']
I then set-up an installer using Wix in Release, and checked that the Winpython python37.dll is shipped with my application. But when I run my program, I'm getting this error when calling import_numpy2():
['C:\\Tesseler', 'C:\\Tesseler\\python37.zip', 'C:\\Users\\Florian\\anaconda3\\Lib', 'C:\\Users\\Florian\\anaconda3\\DLLs', 'C:\\Users\\Florian\\anaconda3', 'C:\\Users\\Florian\\anaconda3\\lib\\site-packages', 'C:\\Users\\Florian\\anaconda3\\lib\\site-packages\\win32', 'C:\\Users\\Florian\\anaconda3\\lib\\site-packages\\win32\\lib', 'C:\\Users\\Florian\\anaconda3\\lib\\site-packages\\Pythonwin']
C:\Users\Florian\anaconda3
C:\Tesseler\Tesseler.exe
['.py', '.pyw', '.pyc', '.cp37-win_amd64.pyd', '.pyd']
ModuleNotFoundError: No module named 'numpy'
I don't understand why anaconda is added to the sys.path and sys.prefix since I never have any reference to it in my visual studio project, nor any environment variable referencing it.
I see why using a dll from anaconda could lead to some problem but I checked with Process Explorer that my application is using the python37.dll shipped with it, which is the case.
Any idea what could cause this error?
Update:
Following ideas described in this thread, I managed to make it work by creating a python subfolder and copying the whole numpy, scipy, pandas and statsmodels folders in it (these 4 modules are needed by my script). These folders were copied from C:\Git\WPy64-3741\python-3.7.4.amd64\Lib\site-packages. I also added this python subfolder to PYTHONPATH: _putenv_s("PYTHONPATH", ".;./python");
Anyway, if someone has a better solution I'm eager to hear it as I find it annoying to have to bundle these modules (more than 600 Mo) when my application is roughly 20 Mo...
I'm working on a project that requires C++ to call a program written in Python that relies on Python exclusive modules.
The project is handled using Qt Creator, and Python 3.7.5 and its packages are installed via Miniconda. I've gotten a basic embedding working using Pybind11 where basic interfacing works, however, most external modules cannot be imported.
For example, when importing Numpy through Pybind11, the following error is thrown (reduced for brevity):
Importing the numpy c-extensions failed.
Original error was: /home/brentnallt/miniconda3/envs/car_class_nogpu/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so: undefined symbol: PyMemoryView_FromObject
A similar error occurs when importing tensorflow through Pybind11:
ImportError: /home/brentnallt/miniconda3/envs/car_class_nogpu/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so: undefined symbol: PyUnicode_FromFormat
It appears to be a problem with Python's C API being found when reading C extension shared libraries. However, modules like lxml which use C source files import just fine. Additionally, I can import problem modules in projects separate from the project I'm working on, implying it's a setup problem. Note that this test project setup doesn't actually use any QT functionality, whereas the main one does.
My PYTHONHOME environment variable looks like:
['/home/brentnallt/miniconda3/envs/car_class_nogpu/lib/python3.7', '/home/brentnallt/miniconda3/envs/car_class_nogpu/lib/python3.7/site-packages', '/home/brentnallt/miniconda3/envs/car_class_nogpu/lib/python37.zip', '/home/brentnallt/miniconda3/envs/car_class_nogpu/lib/python3.7/lib-dynload', '.']
Are there any special considerations I have to make when embedding with Qt Creator? Or is this likely a different problem from a setup error?
Maybe you can consider using PythonQt as an alternative module for calling and importing python libraries from Qt application.
I've used it a lot in my projects and it never failed, but never used it with any kinda data scientific modules maybe you could give it a chance
https://mevislab.github.io/pythonqt/
I am trying to run a simple python program, importing the paraview.simple module:
from paraview.simple import *
I get the following error:
Error: Could not import vtkCommonComputationalGeometry
I looked at similar posts on different fora and tried everything that was suggested there, but nothing worked for me. My Python path includes:
ParaView-5.7.0-MPI-Linux-Python2.7-64bit/bin/
ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib
ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib/python2.7/site-packages/
My LD_LIBRARY_PATH includes:
ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib/python2.7/site-packages/vtkmodules/
Does anybody know how to fix it?
Update:
I think there is an underline issue regarding the Unicode variant my python interpreter is using. I now get the following error:
Unicode error
ImportError: /home/nick/ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib/python2.7/site-packages/vtkmodules/vtkCommonCorePython.so: undefined symbol: PyUnicodeUCS2_DecodeUTF8
Does anybody know a fix?
You may want to use the pvpython program that is bundled with ParaView. It is basically a python interpreter but already setup with the correct paths.
If you want to use an external interpreter, you have to setup the PYTHONPATH environment variable to ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib/python2.7/site-packages/ and the LD_LIBRARY_PATH (on linux, PATH on windows) to ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib.
See also the ParaViewTutorial pdf from the download page (https://www.paraview.org/download/), at 3.1 Starting the Python Interpreter
I have a big C++ module with Python 3 bindings using Boost.Python, that I compile to a .so file using CMake on macOS.
When I try to import it in the REPL, everything seems to work fine:
>>>import myModule
>>>
However, as soon as I run the import statement, the famous rocket icon of Python shows up in the Dock and stays there jumping for some minutes and stops after. Obviously then, I cannot access any of the functions defined in my module, so the import looks fine but doesn't actually do anything.
I tried looking in the Console and saw that whenever I import myModule, I get two launchservicesd[83]: SecTaskLoadEntitlements failed error=22.
It brought me to this and that related questions but I can't find what the exact problem is.
The C++ module is huge so I just can't look at the code and find the problem, thus I'm asking for any hints about at least how to debug that problem.
I can suggest the following steps:
Try to import that module though local python session. So, run interactive python interpreter, and 'import myModule'.
If bad, try to check:
are python version, with which myMoudle was linked with, is similiar to used interpreter
check that build architectires are the same
check that you can load even simple boost.python example module
If ok, check that you have correctly set up module search path in your python code.
I am using python 2.6.5 on an Ubuntu intalled server.
I need to integrate an API for our applicaion, in that case, i needed to use a DLL given to me by the API provider. Their example of code about api integration is written in Visual Basic... I made a search on google and found some examples of using ctypes , and i try using cdll and pydll, which caused the following error...
OSError: /home//some.dll: invalid ELF header
One possibility is using IronPython, but i do not have much information about ironpython so i am not sure if it will handle my needs completely..
Is there any available module that let me use that dll on python (or aynthing that i am missing from the exixting ones). It is hard to upgrade my python version?
DLLs may be windows creatures, but if a DLL is 'pure .NET' and doesn't utilize executables specific to windows etc., then it can work often in Linux, through Mono. (mono ipy.exe).
Ironpython's System and similiar windows modules are customized to be os agnostic (to a untested degree).
I have successfully run NHibernate, FluentNHibernate, log4net, and a few other commonly used DLLS in Ubuntu.
import clr
import sys
sys.path.append(os.path.abspath('./DLL')) #where your dlls are
clr.AddReference('System')
clr.AddReference('FluentNHibernate')
from FluentNHibernate.Cfg.Db import PostgreSQLConfiguration
The key seems to be to import DLLs in this fashion. If a dll imports another (fluentnhibernate imports nhibernate), you don't need to import Nhibernate for example.
DLLs are Windows creatures. The only way you'll be able to use a DLL is by using a Windows build of Python. You'll be able to run Windows Python on Ubuntu by having Windows installed inside a virtual machine. You also might be able to run it using Wine.
An alternative, of course, is to ask your API provider if they have a Linux version of the API.
First, check if your DLL is a .NET Assembly file. An "Assembly DLL file" has nothing to do with the assembler. It's simply a way the .NET framework stores its bytecode inside a DLL file!
Do file library.dll in Linux. If it says something like this:
PE32 executable (DLL) (console) Intel 80386 Mono/.Net assembly, for MS Windows
then you're lucky: it's an assembly file. You can run it on Linux.
Install Mono. Install Python.NET. Forget IronPython: it's dead.
Now, in Python.NET, you can do this:
import clr
clr.AddReference('./library.dll')
# the library has just registered a namespace we can use
from LibraryName import *
but how do you know what to import?
Auto-complete.
Or use monop tool to inspect the DLL like this:
$ monop -r library.dll
Assembly Information:
LibraryName
Version=9.9.3.0
Culture=neutral
PublicKeyToken=null
LibraryName.ClassName
...
$ monop -r library.dll LibraryName.ClassName
public class ClassName {
public ClassName (string inputString);
...
}
and it will tell you everything about that library