I am trying to run a C function in Python. I followed examples online, and compiled the C source file into a .so shared library, and tried to pass it into the ctypes CDLL() initializer function.
import ctypes
cFile = ctypes.CDLL("libchess.so")
At this point python crashes with the message:
Could not find module 'C:\Users\user\PycharmProjects\project\libchess.so' (or one of its dependencies). Try using the full path with constructor syntax.
libchess.so is in the same directory as this Python file, so I don't see why there would be an issue finding it.
I read some stuff about how shared libraries might be hidden from later versions of python, but the suggested solutions I tried did not work. Most solutions were also referring to fixes involving linux system environment variables, but I'm on Windows.
Things I've tried that have not worked:
changing "libchess.so" to "./libchess.so" or the full path
using cdll.LoadLibrary() instead of CDLL() (apparently both do the same thing)
adding the parent directory to system PATH variable
putting os.add_dll_directory(os.getcwd()) in the code before trying to load the file
Any more suggestions are appreciated.
Solved:
Detailed explanation here: https://stackoverflow.com/a/64472088/16044321
The issue is specific to how Python performs a DLL/SO search on Windows. While the ctypes docs do not specify this, the CDLL() function requires the optional argument winmode=0 to work correctly on Windows when loading a .dll or .so. This issue is also specific to Python versions greater than 3.8.
Thus, simply changing the 2nd line to cFile = ctypes.CDLL("libchess.so", winmode=0) works as expected.
When I import a module I built, I get this boost-python related error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: dlopen(./myMod.so, 2): Symbol not found: __ZN5boost6python7objects15function_objectERKNS1_11py_functionERKSt4pairIPKNS0_6detail7keywordES9_E
Referenced from: ./myMod.so
Expected in: flat namespace
in ./myMod.so
What does this actually mean? Why was this error raised?
Description
The problem was caused by mixing objects that compiled with libc++ and object that compiled with libstdc++.
In our case, the library myMod.so (compiled with libstdc++) need boost-python that compiled with libstdc++ (boost-python-libstdc++ from now). When boost-python is boost-python-libstdc++, it will work fine. Otherwise - on computer that its boost-python has compiled with libc++ (or another c++ library), it will have a problem loading and running it.
In our case, it happens because that libc++ developers intentionally changed the name of all of their symbols to prevent you (and save you) from mixing code from their library and code from a different one: myMod.so need a function that take an argument from the type. In libc++, this type's name is std::__1::pair. Therefore, this symbol was not found.
To understand why mixing two version of the same API is bad, consider this situation: There are two libraries: Foo and Bar. They both have a function that takes a std::string and uses it for something but they use a different c++ library. When a std::string that has been created by Foo will be passed to Bar, Bar will think that this is an instance of its c++ library's std::string and then bad things can happen (they are a completely different objects).
Note: In some cases, there would be no problem with two or more different versions of the same API in a completely different parts of a program. There will be a problem if they will pass this API's objects between them. However, checking that can be very hard, especially if they pass the API object only as a member of another object. Also, a library's initialization function can do things that should not happen twice. Another version may do these things again.
How to solve that?
You can always recompile your libraries and make them match each other.
You can link boost-python to your library as a static library. Then, it will work on almost every computer (even one that doesn't has boost-python installed). See more about that here.
Summary
myMod.so need another version of boost-python, one that compiled with a specific c++ library. Therefore, It would not work with any another version.
In my case I was receiving:
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/xmlsec.cpython-38-darwin.so, 0x0002): symbol not found in flat namespace '_xmlSecDSigNs'
BACKGROUND:
M1 MacBook Pro with Montery
I was working with a python virtualenv (using pyenv) to use an earlier version of python3.8 (3.8.2), while my system had 3.8.10 installed natively.
While I was in the activated 3.8.2 virtualenv I noticed the path in dlopen() was pointing to the package in the native python install NOT the virtualenv install.
SOLUTION:
In my case, I did not need the native 3.8 version at all so I simply removed it and this solved the problem.
I encounter the same problem.
Expected in: flat namespace
Add the linker flag fixes the problem
-lboost_python37
change the dynamic library name to the one installed on the os.
By the way, my os is macOS High Sierra and I use brew to install boost_python3.
Symbol not found means the definition of the declared function or variable was not found. When a header file of a shared object is compiled with your program, linker adds symbols of declared functions and objects to your compiled program. When your program is loaded by the OS's loader, the symbols are resolved so that their definition will be loaded. It is only at this time where if the implementation is missing, loader complains it couldn't find the definition due to may be failing to resolve the actual path to the library or the library itself wasn't compiled with the implementation/source file where the definition of the function or object resides. There is a good article on this on the linux journal http://www.linuxjournal.com/article/6463.
In my case I was just failing to import all the required sources (c++ files) when compiling with Cython.
From the string after "Symbol not found" you can understand which library you are missing.
One of the solutions I found was to uninstall and reinstall it using the no-binary flag, which forces pip to compile the module from source instead of installing from precompiled wheel.
pip install --no-binary :all: <name-of-module>
Found this solution here
Here's what I've learned (osx):
If this is supposed to work (i.e. it works on another computer), you may be experiencing clang/gcc issues. To debug this, use otool -l on the .so file which is raising the error, or a suspect library (in my example it's a boost-python dylib file) and examine the contents. Anything in the /System/ folder is built with clang, and should be installed somewhere else with the gcc compiler. Never delete anything in the /System folder.
.so files are dynamic libraries (so = shared object). On Windows they are called .dll (dynamic-link library). They contain compiled code which contains functions available for usage to any executable which links them.
What is important to notice here is that those .so are not Python files. They were probably compiled from C or C++ code and contain public functions which can be used from Python code (see documentation on Extending Python with C or C++).
On your case, well, you have a corrupt .so. Try reinstalling the affected libraries, or Python, or both.
Problem
I had this same issue when running puma as part of Rails app
LoadError:
dlopen(/Users/alucard/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/puma-5.6.4/lib/puma/puma_http11.bundle, 0x0009): symbol not found in flat namespace '_ERR_load_crypto_strings'
/Users/alucard/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/puma-5.6.4/lib/puma/puma_http11.bundle
Solution
It was solved just by installing puma gem again gem install puma
I'm making an extension for a Windows program that hooks to COM server objects. I want to use Python to do this, and from what I understand, I can use Py2exe to create a DLL from a comtypes server.
For a test, I wrote a simple class that calls a print method exposed via RPC. I created a COM interface for that class, registered it, instantiated the COM object from another Python program, and called the print method. It works.
(I've created a repository for reference here: https://github.com/jakogut/python-inproc_server)
However, when I use Py2exe to create a DLL from my COM server, and try to register that DLL using regsvr32, a popup states:
The module "RemotePrint.dll" was loaded but the call to DllRegisterServer failed with error code 0x80040201
So far, I've tried running from an administrator command prompt, clearing the comtypes gen cache and recreating the DLL, profiling the runtime library dependencies using Dependency Walker (it froze/crashed, and gave no helpful information), and running on another machine.
Based on the fact that my code works as it should when I register it using the comtypes register function, I'm assuming this doesn't work as a DLL because of improper setup on my part, or a quirk with Py2exe and Python 3.
It seems that the problem was the Python Interpreter crashing because it couldn't import RemotePrintLib from comtypes.gen. Adding 'comtypes.gen' to the includes option for distutils worked!
One thing that helped diagnose this was win32traceutil.
I just successfully installed PyEphem using pip in a pyenv. However, on import I receive:
ImportError: /python2.7/site-packages/ephem/_libastro.so: undefined symbol: PyUnicodeUCS2_AsUTF8String
In looking around I've seen it mentioned that some modules are built "against Python" in regards to Unicode and suggest recompiling. I'm quite new to Python and Ubuntu 14.04, and although I believe this is the answer to my issue, I do not know what recompiling means or how to do it.
The symbol PyUnicode_AsUTF8String(value) is used once in _libastro.c and is defined on my system in the file:
/usr/include/python2.7/unicodeobject.h
There it can be aliased one of two ways:
#ifndef Py_UNICODE_WIDE
# ...
# define PyUnicode_AsUTF8String PyUnicodeUCS2_AsUTF8String
# ...
#else
# ...
# define PyUnicode_AsUTF8String PyUnicodeUCS4_AsUTF8String
Your error message makes it sound as though your system Python is compiled to use 4-byte-wide Unicode strings (hence why the linker cannot find a UCS2 version of this function inside of it), but that the version of PyEphem that auto-compiled on your system when you ran pip install somehow got confused and unset Py_UNICODE_WIDE and thus generated C code that was expected a UCS2 symbol.
Do you have several compiled versions of Python on your system, where the Unicode setting of one version could accidentally be affecting how this compile for your system Python takes place?
I've got a PyDev project that uses protocol buffers. The protocol buffer files are located in a zip file generated by the protoc compiler. Everything works when I run the program, however PyDev reports "Undefined variable from import" for every enumeration constant. So for example:
import model_pb2
value = model_pb2.Expression(type = model_pb2.Expression.PARAMETER)
It reports the enum constant "PARAMETER" as being an undefined variable. There are several dozen similar errors in my program, and I'd like to fix them "properly" (i.e. not simply suppressing the warning.)
I found that using for builtins as can work, but only if all the proto files are in a separate located in an external library (see http://pydev.org/manual_101_project_conf2.html).
This should work:
Move (or unzip) the compiled proto files including model_pb2.py into a directory outside the pydev project.
Add an empty __init__.py file to the same directory as model_pb2.py to ensure it can be imported as a library.
In eclipse, go to Windows -> Preferences -> pydev -> Interpreter
Add the directory with model_pb2.py to the Libraries.
Add model_pb2 to the forced buildins.
If you're not addicted to autocomplete, you may using ctrl+1 to ignoring these errors instead as described in this answer. This was tested with Eclipse Kepler and pydev 2.8.
I encountered this issue with protobuf 2.6.1 and PyDev 4.5.5. I tried the suggestions above both none of them helped in my case. What ended up getting rid of the 'undefined variable' errors when using protobuf enums was simple:
Access the enum on an instantiated protobuf object rather than on the protobuf module.
I'm not sure if this could be applied to the OP's use case, but in mine it was as easy as:
from myprotobuf_module import SomeProtobufMessage
some_protobuf_object = SomeProtobufMessage()
some_enum = some_protobuf_object.SOME_ENUM
Have you tried adding "model_pb2" to your forced builtins? See: http://pydev.org/manual_101_project_conf2.html for details.