I'm trying to write a Rust module which can be called from Python. I'm following this page:
https://developers.redhat.com/blog/2017/11/16/speed-python-using-rust#edit_src_lib_rs
On cargo build --release I get this error:
= note: LINK : fatal error LNK1181: cannot open input file
'python39.lib'
This has come up before, and this seems the most recent and relevant answer.
I seem to have tried all the possible solutions in that answer, including locating and running vcvars64.bat, as detailed here. No joy.
I have MS Visual Studio (2019) installed, with the C++ and W10 SDK components.
Significantly, this error sometimes does not occur when I don't include the --release switch. Without it, the program (sometimes) compiles and runs OK. Given that one reason for wanting to make a Rust module is performance, however, I'd like to solve this problem. Also I have now found that this successul building of a "debug" build is a) intermittent and b) partial: when it fails, some of the desired output files are created but not others.
And...
According to the page at the first link, it seems that after build, I should then be looking for a file ending .so. In a W10 OS I'm looking for a .dll file.
If it builds completely successfully, under target\debug I have myrustlib.d, myrustlib.dll, myrustlib.dll.exp, myrustlib.dll.lib, myrustlib.pdb, pyext_myrustlib.pdb, pyext_myrustlib.d and pyext_myrustlib.exe. None of these, renamed and/or given an .dll extension as applicable, can be successfully imported as a Python module, at least as documented on that page.
I've also found this more recent page for doing the same thing. I get the same 1181 error.
python39.lib
I found out where this is located in my system, under ... Python\Python39\libs. I modified my PATH env var to specifically include this path (and rebooted). Still the same error.
Workaround:
This page, using PyO3 instead of rust-cpython (actually a fork of the latter), appears to do things successfully, on my machine + OS at least.
NB at the time of writing the second example appears to contain a typo: I believe it should be
fn hashset(_: Python, m: &PyModule) -> PyResult<()> {
rather than
fn hash_set(_: Python, m: &PyModule) -> PyResult<()> {
if you want it to compile OK (author has been notified). I have managed to create a module (hashset.cp39-win_amd64.pyd in a W10 OS) which successfully enables you to import this Rust construct into Python.
Related
I am trying to run a C function in Python. I followed examples online, and compiled the C source file into a .so shared library, and tried to pass it into the ctypes CDLL() initializer function.
import ctypes
cFile = ctypes.CDLL("libchess.so")
At this point python crashes with the message:
Could not find module 'C:\Users\user\PycharmProjects\project\libchess.so' (or one of its dependencies). Try using the full path with constructor syntax.
libchess.so is in the same directory as this Python file, so I don't see why there would be an issue finding it.
I read some stuff about how shared libraries might be hidden from later versions of python, but the suggested solutions I tried did not work. Most solutions were also referring to fixes involving linux system environment variables, but I'm on Windows.
Things I've tried that have not worked:
changing "libchess.so" to "./libchess.so" or the full path
using cdll.LoadLibrary() instead of CDLL() (apparently both do the same thing)
adding the parent directory to system PATH variable
putting os.add_dll_directory(os.getcwd()) in the code before trying to load the file
Any more suggestions are appreciated.
Solved:
Detailed explanation here: https://stackoverflow.com/a/64472088/16044321
The issue is specific to how Python performs a DLL/SO search on Windows. While the ctypes docs do not specify this, the CDLL() function requires the optional argument winmode=0 to work correctly on Windows when loading a .dll or .so. This issue is also specific to Python versions greater than 3.8.
Thus, simply changing the 2nd line to cFile = ctypes.CDLL("libchess.so", winmode=0) works as expected.
I have a C++ application (X-Plane) for which there is a plugin which permits the use of python scripts (XPPython3 plugin). The plugin is written in C, using python CAPI, and works great, allowing me to write python scripts which get executed within the C++ application.
On Windows 10, I want to extend my python features by importing imgui. I have a python cython-built pyd file (_imgui.cp39-win_amd64.pyd).
If I place the pyd file in C\Program Files\Python39\DLLs, it works as expected: C++ application calls CAPI to python, which loads script which imports and executes imgui code.
If I place the pyd file anywhere else, embedded python either reports "module not found" -- if the pyd isn't on sys.path(), or if it is on sys.path():
ImportError: DLL load failed while importing _imgui: The parameter is incorrect.'
Changes using: os.add_dll_directory(r'D:\somewhere\else')
Does not effect whether the module is found or not, nor does it change the 'parameter incorrect' error. (see https://bugs.python.org/issue36085 for details on this change. -- my guess is add_dll_directory changes lookup for DLLs, but not for pyd?) sys.path appears to be used for locating pyd.
Yes, the pyd is compiled with python3.9: I've compiled it both with mingw and with visual studio toolchains, in case that might be a difference.
For fun, I moved python-standard _zoneinfo.pyd from Python39\DLLs and it fails in the same way in embedded python: "The parameter is incorrect". So, that would appear to rule out my specific pyd file.
The key question is/are:
Other than placing a pyd file under PythonXX\DLLs, is there a way to load a PYD in an embedded python implementation? (I want to avoid having to tell users to move my pyd file into the Python39\DLLs directory... because they'll forget.)
Note that using IDLE or python.exe, I can load pyds without error -- anywhere on sys.path -- so they don't have to be under Python39\DLLs. It's only when trying to load from embedded python that the "Parameter is incorrect" appears. And of course, this works flawlessly on Mac.
(Bonus question: what parameter? It appears to be python passing through a windows error.)
There seems to be a simple answer, though I suspect it's better characterized as a python bug.
There is nothing magical about Python39\DLLs directory.
The problem is using absolute vs relative paths in sys.path.
Python can find modules using absolute or relative paths. So if zippy.py is in folder foobar,
sys.path.append('foobar')
import zippy
# Success
Python and find, BUT NOT LOAD pyd files using relative paths. For example, move _zoneinfo.pyd from PythonXX\LDDs to foobar
sys.path.append('foobar')
import _zoneinfo
# ImportError: DLL load failed while importing _zoneinfo: The parameter is incorrect.'
Instead, use absolute path, and it will find and load PYD:
sys.path.append(r'c:\MyTest\foobar')
import _zoneinfo
# Success
So, there is actually a way to do this—that is, ship your application with the desired libraries. The solution is to use an embedded distribution and ship this with your application. You can find the correct distribution on the official Python download page corresponding to your desired version (here's the link to the lastest 3.9 release which seems to be what you're using: https://www.python.org/downloads/release/python-392/). Look for the Windows Embeddable Package.
You can then simply drop in your .pyd file alongside the standard library files (note that if your third-party library is dependent on any other libraries, you will have to include them, as well). Shipping your application with an embeddable distribution should not only solve your current issue, but will also mean that your application will work regardless of which version of Python a user has installed (or without having Python installed at all).
I am new in Python and Basic. I am trying to replicate the IMPORTHTML function from Google Sheets in LibreOffice (LO) Calc. In a nutshell, I want to create a GetHtmTable( Url, Table Index) Basic function in Calc which will call a Python script to do the heavy work.
So based on Villeroy's great example, I implemented in LO 5.1.6.2. the Basic SOUNDEX function which calls the Python script sheetFunctions.py to get familiar with the process. My environment is Linux Mint 18, I use Python 3, I imported all kind of libraries such as Uno, PIP etc.. I use PycharmProjects as a Python editor.
I see clearly under the LO Calc menu tools->macro->organize macros->python the sheetFunctions.py Python script, which indeed is in the folder /usr/lib/libreoffice/share/Scripts/python.
Whenever, I run the SOUNDEX Basic function I see the following error message:
BASIC runtime error. An exception occurred Type:
com.sun.star.script.provider.ScriptFrameworkErrorException Message:
: an
error occurred during file opening
/usr/lib/libreoffice/program/pythonscript.py:429 in function
getModuleByUrl() [lastRead = self.sfa.getDateTimeModified( url )]
/usr/lib/libreoffice/program/pythonscript.py:993 in function
getScript() [mod = self.provCtx.getModuleByUrl( fileUri )]
I tried to debug the SOUNDEX basic function and found out the blocking point is when the program runs getScript("vnd.sun.star.script:sheetFunctions.py$soundex?language=Python&location=user").
I've been trying for days now to overcome this error, unsuccessfully I must confess.
I wonder if I need to bring some extra extensions in the Basic environment or a missing add-in in the Linux/Python one?
I changed the location=user by location=document and got stuck again. I added recently libreoffice-script-provider-python thanks to the command sudo apt-get install libreoffice-script-provider-python but this did not help. I also embedded in the Calc document the Python script but same this did not solve the issue.
The location name does not match. The standard place for self-written scripts is under the user directory. This is location=user, for example ~/.config/libreoffice/4/user/Scripts/python.
Then there is location=share, which refers to the path in your question. These parameters are described under Python Script in the URI Specification.
See also my answer to this question. Be sure to try the APSO extension if you haven't yet. Especially, APSO helps when using location=document, because embedding requires several steps including editing manifest.xml.
I am trying to embed Python in my C++ code to use WordNet through nltk. I am new to Python and I am using MS Visual Studio 2015 to do the same but I'm facing problems to use Python on MSVS 15.
So far I've,
installed Python 3.5.1,
included this path: ...\Python\Python35-32\include, which let me include the Python.h library,
added this path: ...\Python\Python35-32\libs to access the lib files, and python35.lib as dependency in Linker\Input setting of MSVS15,
then there an error occurred, demanding python35_d.lib, so I compiled Python's source in Debug mode & copied that file to that ...\libs folder and it solved the problem.
This all happened when I was trying to run this only-
Py_Initialize();
PyRun_SimpleString("print (\"Hello World!\\n\")");
Py_Finalize();
Now I'm trying to run a Python function from a .py file. I'm trying to run this from the Python tutorials but I'm getting this error -
While debugging, I have found that when this macro gets called: Py_DECREF(), this error occurs.
Do I only need to copy python35_d.dll from PCBuild\win32 to C:\Windows\[System|System32|SysWOW64]\ to solve the problem as I don't know if it'll have any side-effects?
Although the file is already present in ...\Python\Python35-32\DLLs\.
Is there a proper setup to be done before running Python in C/C++ using MSVS, like this for Unix-like systems? (I mean by adding other dependencies or folders to the settings)
I've been reading Python documentation continuously and even other stackoverflow questions but I'm still struggling a lot (I don't know why) even to do simple things.
Can somebody guide/enlighten me for something I may be missing because I'm continuously getting errors here & there, please?
Thanks.
copy python35.dll into the folder where .sln file is present
I recently updated from Python 2.5 to 2.7 (I tried 2.6 during my hassles) and while everything works fine from the command line or in the Django runserver, mod_wsgi cannot load any module that contains DLLs (pyd) built with MSVC.
For example, if I build my own versions of pycrypto or lxml then I will get the following error only from mod_wsgi:
ImportError at /
DLL load failed: The specified module could not be found.
Even the official PIL binaries will fail to import the _imaging C module in mod_wsgi but that may be another problem.
However, if I use a version of pycrypto built with MinGW from somewhere like http://www.voidspace.org.uk/python/modules.shtml#pycrypto then it will import fine even in mod_wsgi. I don't find this solution satisfactory though since the whole reason I updated Python was to avoid needing to hunt for prebuilt binaries and I can't build them myself because MinGW fails >50% of the time for me.
EDIT2:
I noticed this in Python27/Lib/distutils/msvc9compiler.py on lines 680-705:
try:
# Remove references to the Visual C runtime, so they will
# fall through to the Visual C dependency of Python.exe.
# This way, when installed for a restricted user (e.g.
# runtimes are not in WinSxS folder, but in Python's own
# folder), the runtimes do not need to be in every folder
# with .pyd's.
manifest_f = open(manifest_file)
try:
manifest_buf = manifest_f.read()
finally:
manifest_f.close()
pattern = re.compile(
r"""<assemblyIdentity.*?name=("|')Microsoft\."""\
r"""VC\d{2}\.CRT("|').*?(/>|</assemblyIdentity>)""",
re.DOTALL)
manifest_buf = re.sub(pattern, "", manifest_buf)
pattern = "<dependentAssembly>\s*</dependentAssembly>"
manifest_buf = re.sub(pattern, "", manifest_buf)
manifest_f = open(manifest_file, 'w')
try:
manifest_f.write(manifest_buf)
finally:
manifest_f.close()
except IOError:
pass
This probably explains why everything works from the command line but not in mod_wsgi. Commenting all this out seems to fix the problem but doesn't feel like the proper fix. The question now is where to put msvcr90.dll so that Apache can use it? I notice that Apache's bin folder contains msvcr70.dll and msvcr80.dll but putting 90 in there doesn't work.
I've had a similar issue and eventually found a solution here: download/update your apache server with one from http://www.apachelounge.com/download/.
While I don't know anything about mod_wsgi, I dare to guess that the most probable reason is missing runtime dependencies. You may want to inspect your MSVC-build with Dependency Walker that ships with MSVC (e.g., in MSVC 2005, it's located at \Common7\Tools\Bin\Depends.Exe). It will show you which DLLs are required by a binary.
As another workaround, it should be possible to build your modules with statically linked runtime (see Project Properties -> C/C++ -> Code Generation -> Runtime -- choose "Multithreaded" (not "Multithreaded DLL"); or, if building from command line, make sure /MT is used instead of /MD). However there may be problems if runtime-dependent things (e.g. FILE* objects) cross module boundary.
UPD If you have correct VC redist installed, the reason may be a problem with SxS configuration (i.e. manifest of .pyd itself is wrong or missing, or conflicts with manifest of the application that loads the .pyd). You can use sxstrace utility to see what exactly is going on. See Diagnosing SideBySide failures.
Also, did you try static linking of the runtime? Or, better yet, check what are requirements of your host process.
I was getting this error with zmq. The solution was to include the python27.dll manifest in the libzmq.pyd file (and it'll most likely work for other pyd/dll's). Make sure you use all 64-bit or all 32-bit.
"C:\Program Files (x86)\Windows Kits\8.0\bin\x64\mt.exe" -inputresource:C:\windows\system32\python27.dll;#2 -outputresource:libzmq.pyd;#2
See https://code.google.com/p/pyodbc/issues/detail?id=214