On windows I have built a very simple "hello world" C extension (the file hello.c from this site https://gist.github.com/physacco/2e1b52415f3a964ad2a542a99bebed8f). Using VS2015 I successfully obtain hello.dll. The problem is that I can't figure out how to import this file/module.
In the python shell (python 3.7) I have made sure that I'm in the same folder as the hello.dll. I have also made sure that sys.path() contains the folder path. But when I write "import hello" I get an error "ModuleNotFoundError: No module named 'hello'"
Does anyone has an idea of what is wrong is this very simple setup?
Update:
When trying to import a module that does not exist the ModuleNotFoundError is reported. After renaming the hello.dll to hello.pyd an ImportError is returned. So it seems like it tries to actually load the module.
Python compiled modules on Windows have the extension .pyd, not .dll. If you'd built it using setup.py the file would be built with the correct name. However, you built it yourself and gave it a name that Python doesn't recognise as a module.
In terms of the build command: you have to link it with libpython. You don't look to be doing this. The error you report is definitely one that you can get if the module is not linked against all its dependencies.
I know you don't want to use setup.py, however I'd use it at least once just to see what it does and if it works. You'll then at least have a command that you can copy with a working set of options.
Related
I am trying to follow the tutorial for creating python packages from shared objects compiled from C++ via boost::python, but I am running into some problems I need clarification about.
Assume I have a local $install_dir into which I install the compiled shared objects in the form of a python package via CMake. Parallel to the tutorial liked above, my structure is:
$installdir/
my_package/
__init__.py
module/
__init__.py
_module.so
I have added $installdir to my $PYTHONPATH.
$installdir/my_package/__init__.py is empty.
$installdir/my_package/module/__init__.py contains:
from _module import *
When I then import my_package.module I get ModuleNotFoundError: No module named '_module' raised from $installdir/my_package/module/__init__.py.
The issue seems to be that _module.so is not found from $installdir/my_package/module/__init__.py.
Why is the approach from the tutorial not working?
If I add $installdir/my_package/module to $PYTHONPATH directly everything works fine, but it feels like that should not be neccessary, as $installdir/my_package/module/__init__.py should find _module.so locally.
I implemented the following portable workaround for now within $installdir/my_package/module/__init__.py:
import sys, pathlib
sys.path.insert(0,str(pathlib.Path(__file__).parent.absolute()))
from _module import *
Bonus Question:
Changing the file name extension from .so to .pyd breaks the import (ModuleNotFoundError) even without any packaging and .pyd being accessible directly via $PYTHONPATH. I define the extension via CMake's SUFFIX target property. This is obviously mostly cosmetic, but I would still like to understand the reason and how to fix it.
Edit:
This is Ubuntu 20.04 with python 3.8 and boost 1.71
I have a C++ application (X-Plane) for which there is a plugin which permits the use of python scripts (XPPython3 plugin). The plugin is written in C, using python CAPI, and works great, allowing me to write python scripts which get executed within the C++ application.
On Windows 10, I want to extend my python features by importing imgui. I have a python cython-built pyd file (_imgui.cp39-win_amd64.pyd).
If I place the pyd file in C\Program Files\Python39\DLLs, it works as expected: C++ application calls CAPI to python, which loads script which imports and executes imgui code.
If I place the pyd file anywhere else, embedded python either reports "module not found" -- if the pyd isn't on sys.path(), or if it is on sys.path():
ImportError: DLL load failed while importing _imgui: The parameter is incorrect.'
Changes using: os.add_dll_directory(r'D:\somewhere\else')
Does not effect whether the module is found or not, nor does it change the 'parameter incorrect' error. (see https://bugs.python.org/issue36085 for details on this change. -- my guess is add_dll_directory changes lookup for DLLs, but not for pyd?) sys.path appears to be used for locating pyd.
Yes, the pyd is compiled with python3.9: I've compiled it both with mingw and with visual studio toolchains, in case that might be a difference.
For fun, I moved python-standard _zoneinfo.pyd from Python39\DLLs and it fails in the same way in embedded python: "The parameter is incorrect". So, that would appear to rule out my specific pyd file.
The key question is/are:
Other than placing a pyd file under PythonXX\DLLs, is there a way to load a PYD in an embedded python implementation? (I want to avoid having to tell users to move my pyd file into the Python39\DLLs directory... because they'll forget.)
Note that using IDLE or python.exe, I can load pyds without error -- anywhere on sys.path -- so they don't have to be under Python39\DLLs. It's only when trying to load from embedded python that the "Parameter is incorrect" appears. And of course, this works flawlessly on Mac.
(Bonus question: what parameter? It appears to be python passing through a windows error.)
There seems to be a simple answer, though I suspect it's better characterized as a python bug.
There is nothing magical about Python39\DLLs directory.
The problem is using absolute vs relative paths in sys.path.
Python can find modules using absolute or relative paths. So if zippy.py is in folder foobar,
sys.path.append('foobar')
import zippy
# Success
Python and find, BUT NOT LOAD pyd files using relative paths. For example, move _zoneinfo.pyd from PythonXX\LDDs to foobar
sys.path.append('foobar')
import _zoneinfo
# ImportError: DLL load failed while importing _zoneinfo: The parameter is incorrect.'
Instead, use absolute path, and it will find and load PYD:
sys.path.append(r'c:\MyTest\foobar')
import _zoneinfo
# Success
So, there is actually a way to do this—that is, ship your application with the desired libraries. The solution is to use an embedded distribution and ship this with your application. You can find the correct distribution on the official Python download page corresponding to your desired version (here's the link to the lastest 3.9 release which seems to be what you're using: https://www.python.org/downloads/release/python-392/). Look for the Windows Embeddable Package.
You can then simply drop in your .pyd file alongside the standard library files (note that if your third-party library is dependent on any other libraries, you will have to include them, as well). Shipping your application with an embeddable distribution should not only solve your current issue, but will also mean that your application will work regardless of which version of Python a user has installed (or without having Python installed at all).
I am trying to run a simple python program, importing the paraview.simple module:
from paraview.simple import *
I get the following error:
Error: Could not import vtkCommonComputationalGeometry
I looked at similar posts on different fora and tried everything that was suggested there, but nothing worked for me. My Python path includes:
ParaView-5.7.0-MPI-Linux-Python2.7-64bit/bin/
ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib
ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib/python2.7/site-packages/
My LD_LIBRARY_PATH includes:
ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib/python2.7/site-packages/vtkmodules/
Does anybody know how to fix it?
Update:
I think there is an underline issue regarding the Unicode variant my python interpreter is using. I now get the following error:
Unicode error
ImportError: /home/nick/ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib/python2.7/site-packages/vtkmodules/vtkCommonCorePython.so: undefined symbol: PyUnicodeUCS2_DecodeUTF8
Does anybody know a fix?
You may want to use the pvpython program that is bundled with ParaView. It is basically a python interpreter but already setup with the correct paths.
If you want to use an external interpreter, you have to setup the PYTHONPATH environment variable to ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib/python2.7/site-packages/ and the LD_LIBRARY_PATH (on linux, PATH on windows) to ParaView-5.7.0-MPI-Linux-Python2.7-64bit/lib.
See also the ParaViewTutorial pdf from the download page (https://www.paraview.org/download/), at 3.1 Starting the Python Interpreter
I have a big C++ module with Python 3 bindings using Boost.Python, that I compile to a .so file using CMake on macOS.
When I try to import it in the REPL, everything seems to work fine:
>>>import myModule
>>>
However, as soon as I run the import statement, the famous rocket icon of Python shows up in the Dock and stays there jumping for some minutes and stops after. Obviously then, I cannot access any of the functions defined in my module, so the import looks fine but doesn't actually do anything.
I tried looking in the Console and saw that whenever I import myModule, I get two launchservicesd[83]: SecTaskLoadEntitlements failed error=22.
It brought me to this and that related questions but I can't find what the exact problem is.
The C++ module is huge so I just can't look at the code and find the problem, thus I'm asking for any hints about at least how to debug that problem.
I can suggest the following steps:
Try to import that module though local python session. So, run interactive python interpreter, and 'import myModule'.
If bad, try to check:
are python version, with which myMoudle was linked with, is similiar to used interpreter
check that build architectires are the same
check that you can load even simple boost.python example module
If ok, check that you have correctly set up module search path in your python code.
Still learning certain things about Python... I am having issues recognizing my Python script in my scripts dir. First, I checked to see that my path is set correctly:
import sys
for pythonPath in sys.path:
print pythonPath
And C:/Users/..../Documents/maya/2014-x64/scripts is listed, which is where I am placing swap.py
In Maya's script editor I am typing the following:
import swap
reload(swap)
swap.printSomething()
I get:
Error: AttributeError: file line 3: 'module' object has no attribute 'printSomething' #
If I take the same code and throw it into a package...
C:/Users/..../Documents/maya/2014-x64/scripts/swapPackage/swap.py
And then call this, it works...
import swapPackage.swap as swap
reload(swap)
swap.printSomething()
Why? I am totally confused. Mel scripts even run fine from this location as well. I just can't get a simple python script to import and run.
Also something I noticed. Even though I can get this script to run in a package, the package name must be totally different than the module name. I can't have a package named this:
C:/Users/..../Documents/maya/2014-x64/scripts/swap/swap.py
but I can have one where the package name is different:
C:/Users/..../Documents/maya/2014-x64/scripts/swapPackage/swap.py
Ok folks, I was able to solve this by executing a print of my file, only to find out that it was sourcing a totally different version someone copied elsewhere. ARGH. This solves both issues, and makes sense why changing the package name from the module worked.
import swap
reload(swap)
print swap.__file__