We Keep Coding

Python/C++ wrapper Using external dll with Pybind11

Python version: 3.8.1
Spyder version: 3.3.6
Qt version: 5.12.9
Wrapper: develop using PyBind11
I am wrapping a dll develop in C++ which use Qt dlls to be used with Python. I wrote the wrapper with Visual Studio 2019 using the compiler MSVC (as my dll is compiled with MSVC). After generating the solution in VS2019 I obtain a .pyd file which can be import with python.
It works good when I use python on line command:
Start cmd.exe
$python
import MyLibName
I can use the functions/classes ...
But if I try with Spyder, I get the following error:
ImportError: DLL load failed while importing PyStack: The specified module could not be found..
So here are my questions :
Is there a way to get more information about ImportError like the name of the missing dll or something?
I don't understand why the issue only happen with spyder. I tried with IPython Qt Console and it work. Does spyder use a embeded python version or something ?
I don't fully understand how dll shall be managed, I mean shall I provide dll like libGLESV2.dll with the .pyd or just give a path where to find it ?
Thank you in advance.

My guess
I think I find out which part of Qt/python is producing this issue, but I still don't know how to solve it.
My dll use signals/slots which need an event loop to be performed. If an event loop is already running the dll will try to use it, if the loop version (ex : PyQt5==5.14.1) isn’t the same as mine (ex Qt==5.15.1) import will be impossible.
Note that the reverse is true, if I run my dll an then try to start a loop with %gui qt the command will throw an error.
How to reproduce the issue :
Compile a Qt project available here.
Copy the output dll in the folder PyMyStack/dependencies of the VS Project (available here)
Compile the VS project.
Open an IPython console (without using qt has event loop)
Import the module created with VS (Import PyMyStack)
Run the magic command %gui qt
Last command shall print : ERROR:root:DLL load failed while importing QtSvg: The specified procedure could not be found.
How to hide/solve the problem:
Disclaimer : The solutions presented here are surely not the best, if you know a better one please share it ☺
If you just want to import your lib in Spyder, you can use another event loop. Here are the steps to change this:
In Spyder menus go to Tools→Preferences
Select “IPython console”
Go to “Graphics” tab and change the backend combo box to any other values than Qt or Automatic
If you want to use Qt event loop you will have to update it. You can do this with pip command, but remember than Spyder is not compatible with some version. Here is the pip command:
Pip install PyQt5==X.Y.Z
Where X and Y are the same version use to compile your Qt project. The last digit version seems to not be important.

creating standalone exe using pyinstaller with mayavi import

I have a program that helps visualize some data in 3D by plotting a surface and a cloud of points to see how they relate to the surface. For the visualization I am using mayavi since it was easy to set up and does a better job than matplotlib in 3D. The program works great when I run it in my python environment and makes beautiful visualizations. However, I need to distribute this to a few people who don't have Python and would prefer not to have to install python and all the add-ins on each computer, so I have been using pyinstaller to create standalone .exe files that they can run after I develop a program.
For reference, this is all done on Windows 10, Python 3.6, pyqt 4.11.4, pyface 6.0.0, traits 4.6.0, pyinstaller 3.3.1, mayavi 4.5.0+vtk81. Just about every module I use was installed using pip.
The problem is that I can't seem to get a working exe if I use/import the mayavi module. I have been reading so much github documentation on different hook files and hidden-imports and each time I fix one error another pops up. It all started with scipy but I believe I have those worked out. So I have a few questions that could help me solve the problem:
1) Has anyone successfully created a standalone exe using pyinstaller with a mayavi import (specifically from mayavi import mlab)? What is your secret?!?
This seems similar but I haven't been able to figure it out yet... SO_link
I have used the following links (scipy,h5py,pandas,traits/qt4,ETS_TOOLKIT) to add hidden imports or fix other issues but I am stuck now after setting my ETS_TOOLKIT=qt4. Before setting it, I would get a pyface/traits error RuntimeError: No traitsui.toolkits plugin found for toolkit null, but now it says the same thing for qt4 instead of null. I have qt4 installed so I don't understand that... It is in the import_toolkit function in pyface/base_toolkit.py.
2) Is there a better route to go in terms of 3D visualization / exe creation? I need a 3D program that can accurately render if the points are in front of or behind the surface and be able to rotate/zoom/pan interactively, plus it needs to be intuitive. Mayavi had very simple commands similar to matplotlib but others seem very complicated working around how the UI interacts with everything.
3) How do I interpret all of these error codes I get? They are usually pretty cryptic saying a certain line threw an exception nested in 10 other function calls and it seems very difficult to back out where exactly things went wrong, especially when nothing pops up on Google that seems to be related.
EDIT
While I am still very confused, I have been able to change where the error occurs. Following the traceback, I commented out a line setting the marker color in traitsui/editors/code_editor.py (line 49), at which point the exception then started the next time the Color method was called... but I still get the same RuntimeError. So that doesn't tell me much other than I am still looking for what hidden import I need to include for this thing to work.
Also note that I get the exact same error with both PyInstaller and cx_Freeze, in case that helps...
Edit 2
I have now tried it using anaconda for python 2.7, SAME EXACT ISSUE..... I'm starting to believe the universe doesn't want this to happen. Is there somewhere else I should bring this issue up?? I have posted on the traitsui GitHub but that wasn't very helpful. This seems to be bigger than pyinstaller/cx_freeze since it happens in both....

I dealt with the same problem and finally switched to cx_freeze, which now works fine on linux and windows. The problems you are dealing with arise from statements like in the SE answer, you found, i.e. dynamic import statements, where what is imported is only determined at runtime:
be = 'pyface.ui.%s.' % tk
__import__(be + 'init')
I couldn't fix that in pyinstaller, while in cx_freeze it works, when you explicitely add the required packages in the build file. Here is the package list I used:
"packages": ["pyface.ui.qt4", "tvtk.vtk_module", "tvtk.pyface.ui.wx", "matplotlib.backends.backend_qt4",'pkg_resources._vendor','pkg_resources.extern','pygments.lexers',
'tvtk.pyface.ui.qt4','pyface.qt','pyface.qt.QtGui','pyface.qt.QtCore','numpy','matplotlib','mayavi']
Here is a full build script that works with python3.6, cx_freeze 5.0.2, mayavi 4.5.0+vtk71, traits 4.6.0, pyface 5.1.0 and traitsui 5.1.0.
import os
from cx_Freeze import setup, Executable
import cx_Freeze.hooks
def hack(finder, module):
return
cx_Freeze.hooks.load_matplotlib = hack
import scipy
import matplotlib
scipy_path = os.path.dirname(scipy.__file__) #use this if you are also using scipy in your application
build_exe_options = {"packages": ["pyface.ui.qt4", "tvtk.vtk_module", "tvtk.pyface.ui.wx", "matplotlib.backends.backend_qt4",'pygments.lexers',
'tvtk.pyface.ui.qt4','pyface.qt','pyface.qt.QtGui','pyface.qt.QtCore','numpy','matplotlib','mayavi'],
"include_files": [(str(scipy_path), "scipy"), #for scipy
(matplotlib.get_data_path(), "mpl-data"),],
"includes":['PyQt4.QtCore','PyQt4.QtGui','mayavi','PyQt4'],
'excludes':'Tkinter',
"namespace_packages": ['mayavi']
}
executables = [
Executable('main.py', targetName="main.exe",base = 'Win32GUI',)
]
setup(name='main',
version='1.0',
description='',
options = {"build_exe": build_exe_options},
executables=executables,
)
I import pyface in the following way:
os.environ['ETS_TOOLKIT'] = 'qt4'
import imp
try:
imp.find_module('PySide') # test if PySide if available
except ImportError:
os.environ['QT_API'] = 'pyqt' # signal to pyface that PyQt4 should be used
from pyface.qt import QtGui, QtCore
before importing mayavi

pandas v0.17.1 not working with py2exe

I Have a problem with python pandas v0.17.1. I upgraded from v0.16.2.
System:
Win10 x64, Python 3.4 64Bit, using PyCharm Community Edition for coding.
(numpy 1.9.3+mkl)
I'm using py2exe to create a stand-alone of a statistics program, using pandas to hold the data, matplotlib for plotting and pyqt4 for everything related to gui.
Since i upgraded pandas, the created .exe from py2exe doesn't work anymore. After doubleclick or start from commandline nothing happens. No Errors, no Errorlog file or similar, no 'window flashing' open and close again. just nothing.
I uninstalled pandas and reinstalled (fresh install) it via pip. Same problem.
I just downgraded pandas to v0.16.2 again. Everything works fine now (with v0.16.2). No other changes made.
For the sake of testing i created a simplest program as possible, only an empy pyqt mainwindow and whats needed to start the programm. works fine witout pandas. After 'import pandas' nothing happens anymore (with v0.17.1).
Somebody knows whats going on? Do i have to tweak my setup.py for the new pandas version? Because i dont get any error, i cannot check whats wrong.
main.py:
# coding=utf-8
import sys
from PyQt4 import QtCore, QtGui
import matplotlib
#import pandas
class app(QtGui.QMainWindow):
def __init__(self, *args):
QtGui.QMainWindow.__init__(self, *args)
if __name__ == "__main__":
programm = QtGui.QApplication(sys.argv)
window = app()
window.show()
eventloop = programm.exec_()
sys.exit()
setup.py:
# coding=utf-8
from distutils.core import setup
import py2exe
path_to_source = r'path to dir' # replace with your working directory
setup(
options = {"py2exe": {
'includes': ['sip'],
'excludes': [],
'optimize': 2,
'compressed' : False,
'packages': ['encodings']
#'skip_archive': True
}},
zipfile = None,
windows = [{"script": path_to_source + r"/main.py"}]
)
Just uncomment the import statement of pandas and nothing works anymore with v0.17.1.
The 'dist' directory gets created with a the same files as before.
I tried to 'include' pandas in setup.py but no effect. Dont know what to do to solve this. Are some dll's needed in the setup.py now?
Sorry for my bad english.
ps: In PyCharm, everything works fine, it's only the .exe that does not work.
ps2: Tested the same with my Win7 installation, same behavior.

I solved my problem. It was my AVAST Anti-Virus. It's 'deepscreen' feature started the programm in the background as a sandbox and analysed the .exe but never informed me about it running in the back (no info baloon etc.).
By chance, i had it deactivated while looking into Calvin's Answer.
It works on both, my PC and Laptop now, without any changes. Just deactivated AVAST 'deepscreen' feature while using the .exe created py py2exe.

I've been having the same issue as well. What I found was that a packaged called 'nbformat' caused the py2exe script to fail. I added this package into my excludes list and the script ran successfully.

No module named when using PyInstaller

I try to compile a Python project under Windows 7 using PyInstaller. The project works fine, there are no issues, however when I try to compile it the result doesn't work. Though I get no warnings during compilation there are many in the warnmain.txt file in the build directory: warnmain.txt
I don't really understand those warnings, for example "no module named numpy.pi" since numpy.pi is no module but a number. I never tried to import numpy.pi. I did import numpy and matplotlib explicitly. In addition I'm using PyQt4. I thought the error might be related to those libraries.
However I was able to compile a simple script which uses numpy succesfully:
import sys
from PyQt4 import QtGui, QtCore
import numpy as np
class MainWindow(QtGui.QMainWindow):
def __init__(self):
QtGui.QMainWindow.__init__(self)
self.pb = QtGui.QPushButton(str(np.pi), self)
app = QtGui.QApplication(sys.argv)
main = MainWindow()
main.show()
sys.exit(app.exec_())
Successfully here means that the created executable file actually showed the desired output. However there is also a warnmain.txt file created which contains exactly the same 'warnings' as the one before. So I guess the fact that compiling my actual project does not give any success is not (or at least not only) related to those warnings. But what else could be the error then? The only output during compilation are 'INFO's and none of the is a negative statement.
I did not specify an additional hook directory but the hooks where down using the default directory as far as I could read from the compile output, e.g. hook-matplotlib was executed. I could not see any hook for numpy neither could I for my small example script but this one worked. I used the following imports in my files (not all in the same but in different ones):
import numpy as np
import matplotlib.pyplot as ppl
from matplotlib.backends.backend_qt4agg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.backends.backend_qt4agg import NavigationToolbar2QTAgg as NavigationToolbar
from PyQt4 import QtGui, QtCore
import json
import sys
import numpy # added this one later
import matplotlib # added this one later
Since PyInstaller does not give any errors/warnings I could not figure out if the problem is related to the libraries or if there is something else to be considered.

Had a similar problem with no module named FileDialog. Discovered that with version 3.2, I could use
pyinstaller --hidden-import FileDialog ...
instead of modifying my main script.
See Listing Hidden Imports documentation

Pyinstaller won't see second level imports. So if you import module A, pyinstaller sees this. But any additional module that is imported in A will not be seen.
There is no need to change anything in your python scripts. You can directly add the missing imports to the spec file.
Just change the following line:
hiddenimports=[],
to
hiddenimports=["Tkinter", "FileDialog"],

If you are getting ModuleNotFoundError: No module named ... errors and you:
call PyInstaller from a directory other than your main script's directory
use relative imports in your script
then your executable can have trouble finding the relative imports.
This can be fixed by:
calling PyInstaller from the same directory as your main script
OR removing any __init__.py files (empty __init__.py files are not required in Python 3.3+)
OR using PyInstaller's paths flag to specify a path to search for imports. E.g. if you are calling PyInstaller from a parent folder to your main script, and your script lives in subfolder, then call PyInstaller as such:
pyinstaller --paths=subfolder subfolder/script.py.

The problem were some runtime dependencies of matplotlib. So the compiling was fine while running the program threw some errors. Because the terminal closed itself immediately I didn't realize that. After redirecting stdout and stderr to a file I could see that I missed the libraries Tkinter and FileDialog. Adding two imports at the top of the main solved this problem.

I was facing the same problem and the following solution worked for me:
I first removed the virtual environment in which I was working.
Reinstalled all the modules using pip (note: this time I did not create any virtual environment).
Then I called the pyinstaller.
The .exe file created thereafter executed smoothly, without any module import error.

I had the same problem with pyinstaller 3.0 and weblib. Importing it in the main didn't help.
Upgrading to 3.1 and deleting all build files helped.
pip install --upgrade pyinstaller

If the matter is that you don't need Tkinter and friends because you are using PyQt4, then it might be best to avoid loading Tkinter etc altogether. Look into /etc/matplotlibrc and change the defaults to PyQt4, see the 'modified' lines below:
#### CONFIGURATION BEGINS HERE
# The default backend; one of GTK GTKAgg GTKCairo GTK3Agg GTK3Cairo
# CocoaAgg MacOSX Qt4Agg Qt5Agg TkAgg WX WXAgg Agg Cairo GDK PS PDF SVG
# Template.
# You can also deploy your own backend outside of matplotlib by
# referring to the module name (which must be in the PYTHONPATH) as
# 'module://my_backend'.
#modified
#backend : TkAgg
backend : Qt4Agg
# If you are using the Qt4Agg backend, you can choose here
# to use the PyQt4 bindings or the newer PySide bindings to
# the underlying Qt4 toolkit.
#modified
#backend.qt4 : PyQt4 # PyQt4 | PySide
backend.qt4 : PyQt4 # PyQt4 | PySide

May not be a good practice but installing pyinstaller in the original environment used in my project (instead of a separate venv) helped resolve ModuleNotFoundError

I had similar problem with PySimpleGUI.
The problem was, pyinstaller was installed in different directory.
SOLUTION (solved for me) : just install pyinstaller in the same directory in which the file is present (that to be converted to exe)

If these solutions don't work, simply deleting and reinstalling pyinstaller can fix this for you (as it did for me just now).
Putting this here for anyone else who might come across this post.

I had the same error. Mine said "ModuleNotFoundError: No module named 'numpy'". I fixed it by typing the following in the cmd:
pip install pyinstaller numpy

PySide Unable to Freeze PyQt4 Port - PyInstaller, py2exe or cx_Freeze

I ported a PyQt4 application I wrote to PySide but I cannot create an executable file for it. I have attempted to use py2exe, cx_freeze, and PyInstaller to create the exe for PySide, the first two provide similar errors (none provide error reports during build, everything appears good until I try to run the exe) - The PyQt4 version is freezable via py2exe.
System Info
Windows 7, 64 bit - all software are 32-bit versions below packages being used:
Python 2.7.3 via Python(x,y) 2.7.3.1
Qt 4.8.2
PySide 1.1.2
NumPy 1.6.2
Pywin32 218-1
Exe Conversion Programs
PyInstaller 2.0
cx_Freeze 4.3.1-1
py2exe 0.6.9
py2exe err.Log
Traceback (most recent call last):
File "myApp.pyw", line 11, in <module>
File "PySide\QtGui.pyc", line 12, in <module>
File "PySide\QtGui.pyc", line 10, in __load
ImportError: DLL load failed: The specified procedure could not be found.
cx_freeze err.Log
Traceback (most recent call last):
File "C:\Python27\Lib\site-packages\cx_Freeze\initscript\Console.py",
line 27, in <module>
exex code in m.__dict__
File "myApp.pyw", line 11, in <module>
File "ExtensionLoader_PySide_QtGui.py", line 11, in <module>
ImportError: DLL load failed: The specified procedure could not be found.
FYI: Line11 of MyApp.pyw = from PySide import QtGui, QtCore
PyInstaller err.Log
"The application has failed to start because its side-by-side configuration is
incorrect. Please see the application event log or use the command-line sxstrace.exe
tool for more detail."
The closest I can find is here but I'm not using that library so I do not think its applicable, nor do I know how to solve that.
Snippet of SxsTrace on the PyInstaller Error
INFO: Reference: Microsoft.VC90.CRT,processorArchitecture="x86",publicKeyToken="1fc8b3b9a1e18e3b",type="win32 ",version="9.0.21022.8"
What I have tried
Placed "C:\Python27\Lib\site-packages\PySide" at start of Environment Variables Path
Placed copies of: msvcr90.dll, msvcp90.dll, msvcm90.dll into Python/dll location : I believed msvcp90.dll alone allowed py2exe make a operational PyQt4 exe
Remade all qrc resource file using pyside-rcc
Placed all pyside/dlls into dist folder (py2exe)
ran sxstrace and generated log file on PyInstaller error - not sure what I'm looking for I don't see anything that event viewer or the msgbox didn't already tell me (just more of it)
Reinstalled all python/modules, pyqt, pyside, VSE2008 C++ edition, microsoft SDK, microsoft .netframe work 2.0?, Microsoft Visual C++ 2008 Redistributable Package,
Ported all work in progress modules not needed for package but were in dir
Removed any WIP modules that were not successfully ported/were not hooked up
Copied (manually not setup.py) qt.conf (doesn't seem to exist for pyside only pyqt4?) and plugins to dist folder

I managed to get a working executable using py2exe (somewhat with cx_freeze not at all in PyInstaller). I was able to debug this by converting each module into into stand alone exe's.
This managed to reveal errors that could not be replicated otherwise.
The main issue for the above errors was from missing converting PyQt4 to PySide in one module
Note this was a data loading module and must have simply missed clicking it while testing my port prior to creating the exe. All other PySide conversions were already done to it.
NB:
from PySide import QtCore, QtGui
Is always required even if your IDE is saying that QtCore is not in use and python runs the module without issues. This is required to make an exe for that module even if you can make it work through Python without QtCore.
This is not actual needed for making the exe of the main window but being able to create an operational exe for the module was helpful.
CONVERSION PROGRAM RESULTS
py2exe - WORKS!
Nothing special just a standard setup.py file works fine.
cx_Freeze
Sort of works, but it behaves erratically and doesn't provide explanations as to why...
I have 3 near identical processes for loading data with the main difference being the files that they load. The one for the smaller file loads fine the larger ones just hang and exit out somewhere before ever running... couldn't even get them to work when putting a msgbox at the front of them.
Similarly I have a dialog that has to retrieve data and do a bit of calculations before show()... it does nothing - no idea why. The main window & most of the program works though.
PyInstaller
I can create exe's for all of the individual modules but not for the main window module. When trying to run it I received the exact same problem that Ticket#590 had except I'm using pywin32-218 & no panda.
When attemptimpting to run this occures:
ImportError: No module named pythoncom
Build process indicates:
WARNING: pythoncom is changing its name to pythoncom27
WARNING: pywintypes is changing its name to pywintypes27
The really odd part is the module that actually imports from pywin32 can be turned into an operational exe. As can widgets that call this module, but the main window which calls these other widgets give the error.
Second issue with PyInstaller
All stand alone widgets that have reimplemented the close event hang when they are closed.
def closeEvent(self, event):
self.results.addResultsToDB()
event.accept()
The above does not occur with py2exe as a stand alone apps.