We Keep Coding

Trouble shooting using Pythonnet and setting Runtime.PythonDLL property

I try to use an assembly for .NET framework 4.8 via Pythonnet. I am using version 3.0.1 with Python 3.10. The documentation of Pythonnet is stating:
You must set Runtime.PythonDLL property or PYTHONNET_PYDLL environment variable starting with version 3.0, otherwise you will receive BadPythonDllException (internal, derived from MissingMethodException) upon calling Initialize. Typical values are python38.dll (Windows), libpython3.8.dylib (Mac), libpython3.8.so (most other Unix-like operating systems).
However, the documentation unfortunately is not stating how the property is set and I do not understand how to do this.
When I try:
import clr
from pythonnet import load
load('netfx')
clr.AddReference(r'path\to\my.dll')
unsurprisingly the following error is coming up
Failed to initialize pythonnet: System.InvalidOperationException: This property must be set before runtime is initialized
bei Python.Runtime.Runtime.set_PythonDLL(String value)
bei Python.Runtime.Loader.Initialize(IntPtr data, Int32 size)
bei Python.Runtime.Runtime.set_PythonDLL(String value)
bei Python.Runtime.Loader.Initialize(IntPtr data, Int32 size)
[...]
in load
raise RuntimeError("Failed to initialize Python.Runtime.dll")
RuntimeError: Failed to initialize Python.Runtime.dll
The question now is, where and how the Runtime.PythonDLL property or PYTHONNET_PYDLL environment variable is set
Thanks,
Jens

I believe this is because import clr internally calls pythonnet.load, and in the version of pythonnet you are using this situation does not print any warning.
E.g. the right way is to call load before you call import clr for the first time.

The way I understand your use case is for Embedding .NET in Python however, the way I understand the requirement to "set Runtime.PythonDLL property" is that is for Embedding Python in .NET which was my use case. Anyhow, maybe the following will be useful.
Hidden at the bottom of the main GitHub README.md is a link to the WiKi (and of course the tab at the top of the GitHub repo) where thankfully there is a lot more detailed information and links to useful articles.
The main README.md asserts Runtime.PythonDLL "must be set" yet their example code does not illustrate doing so. Furthermore, the documentation on the official website asserts Python.Runtime.dll must be "referenced" which further confuses things.
In my experience Python.Runtime.dll was automatically referenced when installing the pythonnet NuGet package via Visual Studio. Perhaps Python.Runtime.dll does not automatically get referenced in earlier version or maybe when Python.NET is installed in other ways other than by using NuGet?
To answer your question "how to set Runtime.PythonDLL property?". My understanding is the way that is done is by assigning a string path to that property before the other usual setup:
Runtime.PythonDLL = #"C:\Users\<username>\AppData\Local\Programs\Python\Python310\python38.dll"
In my case, I found this path by using where python on Windows (which in bash, or Get-Command in PowerShell):
C:\>where python
C:\Users\<username>\AppData\Local\Programs\Python\Python310\python.exe

ImportError: DLL load failed while importing ON WINDOWS

I fixed a super-annoying case of "ImportError: DLL load failed while importing" in a way that generally applies to Windows, so let me share it with the group. Here is the question:
I installed FINUFFT via pip install finufft. When I import finufft, I get this error:
ImportError: DLL load failed while importing _finufft: The specified module could not be found.
How do I fix it?

Read to the end before doing anything.
The error means that a DLL cannot find another DLL that it was linked with. But which other DLL?
Download Dependencies.
Locate your problematic DLL. In this specific case: Locate the folder ...\Lib\site-packages\finufft\ of the FINUFFT installation that you want to fix. ...\ is the path of your standard python installation or of your python virtual environment.
Start DependenciesGui.exe and use it to open the problematic DLL, e.g. ...\finufft\_finufft.cp38-win_amd64.pyd. (A .pyd is a regular DLL with some specific entry points for python.)
On the left, you will see a complete list of the problematic DLL's direct dependencies, whose dependencies you can in turn unfold by mouse click. Apart from typical Windows-DLLs, like kernel32.dll and MSVCRT.dll, and apart from the FFTW-DLLs, which should already be in the FINUFFT-folder, there will also be some - possibly missing - Linux-DLLs. For me, it was libgcc_s_seh-1.dll, libgomp-1.dll and libstdc++-6.dll. By checking their direct dependencies, I also discovered libwinpthread-1.dll as missing.
[See EDIT below!!!] I found those DLLs in Anaconda (...\Anaconda3\Library\mingw-w64\bin\), but you can probably also get them from cygwin (...\cygwin64\bin\), git (...\Git\mingw64\bin\) or anything else that downloads mingw64 and its packages on Windows.
To solve the problem, copy the respective DLLs into ...\Lib\site-packages\finufft\ and give them the exact filenames that the FINUFFT-DLL is expecting according to Dependencies. This works because Windows and because of the Windows DLL search order.
Now, import finufft should work in the specific python environment whose FINUFFT installation you fixed. Clearly, this method can be applied anytime DLL dependencies are missing.
EDIT - correction of my answer by #CristiFati: If possible, DLLs and similar things should always be built with the same toolchain. So if you don't compile them yourself, get them from as few different places as possible, i.e. don't mix regular python, Anaconda, cygwin, etc. - if possible. Of course, Windows DLLs will have a different origin from Linux DLLs.

How do I build numpy for unicode ucs4

I am cross compiling for an embedded device using yocto - so using pip install is not appropriate.
My build works, but keeps defaulting to ucs2 character type, which causes an error:
numpy.core.multiarray failed to import.

Caveat, I haven't really tried this...
As far as I can see, building numpy with ucs4 support means that you have to compile python with ucs4-support. Thus, you would need to add
EXTRA_OECONF += "--enable-unicode=ucs4"
in a python_xxx.bbappend, depending on which python (2 or 3) and which OE-release you're using.
If you're getting any other issues after this, is unknown...

What does "Symbol not found / Expected in: flat namespace" actually mean?

When I import a module I built, I get this boost-python related error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: dlopen(./myMod.so, 2): Symbol not found: __ZN5boost6python7objects15function_objectERKNS1_11py_functionERKSt4pairIPKNS0_6detail7keywordES9_E
Referenced from: ./myMod.so
Expected in: flat namespace
in ./myMod.so
What does this actually mean? Why was this error raised?

Description
The problem was caused by mixing objects that compiled with libc++ and object that compiled with libstdc++.
In our case, the library myMod.so (compiled with libstdc++) need boost-python that compiled with libstdc++ (boost-python-libstdc++ from now). When boost-python is boost-python-libstdc++, it will work fine. Otherwise - on computer that its boost-python has compiled with libc++ (or another c++ library), it will have a problem loading and running it.
In our case, it happens because that libc++ developers intentionally changed the name of all of their symbols to prevent you (and save you) from mixing code from their library and code from a different one: myMod.so need a function that take an argument from the type. In libc++, this type's name is std::__1::pair. Therefore, this symbol was not found.
To understand why mixing two version of the same API is bad, consider this situation: There are two libraries: Foo and Bar. They both have a function that takes a std::string and uses it for something but they use a different c++ library. When a std::string that has been created by Foo will be passed to Bar, Bar will think that this is an instance of its c++ library's std::string and then bad things can happen (they are a completely different objects).
Note: In some cases, there would be no problem with two or more different versions of the same API in a completely different parts of a program. There will be a problem if they will pass this API's objects between them. However, checking that can be very hard, especially if they pass the API object only as a member of another object. Also, a library's initialization function can do things that should not happen twice. Another version may do these things again.
How to solve that?
You can always recompile your libraries and make them match each other.
You can link boost-python to your library as a static library. Then, it will work on almost every computer (even one that doesn't has boost-python installed). See more about that here.
Summary
myMod.so need another version of boost-python, one that compiled with a specific c++ library. Therefore, It would not work with any another version.

In my case I was receiving:
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/xmlsec.cpython-38-darwin.so, 0x0002): symbol not found in flat namespace '_xmlSecDSigNs'
BACKGROUND:
M1 MacBook Pro with Montery
I was working with a python virtualenv (using pyenv) to use an earlier version of python3.8 (3.8.2), while my system had 3.8.10 installed natively.
While I was in the activated 3.8.2 virtualenv I noticed the path in dlopen() was pointing to the package in the native python install NOT the virtualenv install.
SOLUTION:
In my case, I did not need the native 3.8 version at all so I simply removed it and this solved the problem.

I encounter the same problem.
Expected in: flat namespace
Add the linker flag fixes the problem
-lboost_python37
change the dynamic library name to the one installed on the os.
By the way, my os is macOS High Sierra and I use brew to install boost_python3.

Symbol not found means the definition of the declared function or variable was not found. When a header file of a shared object is compiled with your program, linker adds symbols of declared functions and objects to your compiled program. When your program is loaded by the OS's loader, the symbols are resolved so that their definition will be loaded. It is only at this time where if the implementation is missing, loader complains it couldn't find the definition due to may be failing to resolve the actual path to the library or the library itself wasn't compiled with the implementation/source file where the definition of the function or object resides. There is a good article on this on the linux journal http://www.linuxjournal.com/article/6463.

In my case I was just failing to import all the required sources (c++ files) when compiling with Cython.
From the string after "Symbol not found" you can understand which library you are missing.

One of the solutions I found was to uninstall and reinstall it using the no-binary flag, which forces pip to compile the module from source instead of installing from precompiled wheel.
pip install --no-binary :all: <name-of-module>
Found this solution here

Here's what I've learned (osx):
If this is supposed to work (i.e. it works on another computer), you may be experiencing clang/gcc issues. To debug this, use otool -l on the .so file which is raising the error, or a suspect library (in my example it's a boost-python dylib file) and examine the contents. Anything in the /System/ folder is built with clang, and should be installed somewhere else with the gcc compiler. Never delete anything in the /System folder.

.so files are dynamic libraries (so = shared object). On Windows they are called .dll (dynamic-link library). They contain compiled code which contains functions available for usage to any executable which links them.
What is important to notice here is that those .so are not Python files. They were probably compiled from C or C++ code and contain public functions which can be used from Python code (see documentation on Extending Python with C or C++).
On your case, well, you have a corrupt .so. Try reinstalling the affected libraries, or Python, or both.

Problem
I had this same issue when running puma as part of Rails app
LoadError:
dlopen(/Users/alucard/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/puma-5.6.4/lib/puma/puma_http11.bundle, 0x0009): symbol not found in flat namespace '_ERR_load_crypto_strings'
/Users/alucard/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/puma-5.6.4/lib/puma/puma_http11.bundle
Solution
It was solved just by installing puma gem again gem install puma

Pygame and cx_freeze: segmentation fault

I'm using ubuntu and python 2.6
I found cx freeze already installed on my system (is it there a way to check if it's compatible with my Python version? )
however, i have a small pygame script (which import another module and some images) and i want to compile it;
i used this file as setup.py:
#!/usr/bin/python
from cx_Freeze import setup, Executable
setup(
name = 'Example',
version = '0.1',
description='hi',
executables = [Executable('/home/antonio/Python 26 save/opt/example.py')]
)
if i run the resulting executable, (through the terminal) i get this error:
Fatal Python error: (pygame parachute) Segmentation Fault
Aborted
what should i do? I've searched but I found very few examples and I didn't see this error on google results
ps of course the program was running perfectly before using cx freeze

I have been getting a similar problem using python 2.7. I've found two causes of this segmentation fault in my own program, but I only have a solution to one of them.
Cause 1. Initialising fonts with no path, ie calling:
pygame.font.Font(None, font_size)
In this case, valgrind reports an invalid read at the address 0x0 in ??? in pygame.font.so
I would guess that this is because None is converted to a NULL pointer which something then assumes is a valid const char* string.
The fix for this problem is to always supply a valid path to a font.
Cause 2. Rendering unicode characters in fonts
pygame.font.Font("data/DejaVuSans.ttf", 14).render(u'\u2654')
valgrind reports an invalid read in PyString_AsString in libpython2.7.so.1.0
I'm sorry to say I don't have a solution for this.
PS:
I have just found another unicode related (but not pygame related) cause of problems with cxfreeze.
print u'\u2654'
In the python interpreter will print a king (chess piece), but when the script is compiled with cxfreeze, I get the following error (not a segmentation fault):
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2654' in position 0: ordinal not in range(128)
You also get this error in the python interpreter if you call:
print str(u'\u2654')
This seems to indicate that cxfreeze is assuming strings are always ascii strings.

Do you have any optimization options set when freezing the script? I'm not too sure if it does this, but it may be that it's incorrectly changing a variable to a reference. Again, I'm not an expert at cx_freeze, but my solution would be to update. Do you have the latest version (of cx_freeze)?

Did you google for your error (http://www.google.com/search?&q=Fatal%20Python%20error%3A%20%28pygame%20parachute%29%20Segmentation%20Fault) and check the various posts reporting the same error ?
E.g.
http://www.pyweek.org/d/2976/
http://osdir.com/ml/python.pygame/2001-12/msg00055.html
...

I have been getting the similar error and I think I have found the solution.
I am using
pygame.font.SysFont(None,25)
But instead of passing None argument you should use your system's fonts.
I have using windows machine so I have replaced none with any font that my system has. So I have replaced it to:
pygame.font.SysFont("comicsansms",25)
As you can see I have replaced None with comicsansms which is preinstalled fonts on windows PC
Hope it works!

Use
pygame.font.SysFont(FONT_NAME, FONT_SIZE).