I wrote a Python 3 extension module in C but cannot seem to get Python to import it.
Is there any way to let Python print out which shared libraries (.so on Linux) it tries to load and why it fails?
Sadly all the docs I read don't really help since none describes the native import procedure in a concise way.
What I tried is:
ctypes.CDLL("libmydep1.so")
ctypes.CDLL("libmydep2.so")
try:
import my_main
print("Load python")
except:
ctypes.CDLL("libmylib.so")
print("Load shared object")
Which always prints Load shared object.
libmylib.so contains the python entry point but loading it as Python 3 extension does not seem to work, although loading as a shared library does.
EDIT:
Python does not honor linux conventions. So for a lib you do not name it libmylib.so but mylib.so.
Even worse, it only loads my_main, when the so is named my_main.so. So annoying.
Try to look at /proc/<pid>/maps directory.
Or try to use lsof -p <PID> command in shell.
Source of answer is this forum. lsof man page. See also this answer.
Related
I am trying to run a C function in Python. I followed examples online, and compiled the C source file into a .so shared library, and tried to pass it into the ctypes CDLL() initializer function.
import ctypes
cFile = ctypes.CDLL("libchess.so")
At this point python crashes with the message:
Could not find module 'C:\Users\user\PycharmProjects\project\libchess.so' (or one of its dependencies). Try using the full path with constructor syntax.
libchess.so is in the same directory as this Python file, so I don't see why there would be an issue finding it.
I read some stuff about how shared libraries might be hidden from later versions of python, but the suggested solutions I tried did not work. Most solutions were also referring to fixes involving linux system environment variables, but I'm on Windows.
Things I've tried that have not worked:
changing "libchess.so" to "./libchess.so" or the full path
using cdll.LoadLibrary() instead of CDLL() (apparently both do the same thing)
adding the parent directory to system PATH variable
putting os.add_dll_directory(os.getcwd()) in the code before trying to load the file
Any more suggestions are appreciated.
Solved:
Detailed explanation here: https://stackoverflow.com/a/64472088/16044321
The issue is specific to how Python performs a DLL/SO search on Windows. While the ctypes docs do not specify this, the CDLL() function requires the optional argument winmode=0 to work correctly on Windows when loading a .dll or .so. This issue is also specific to Python versions greater than 3.8.
Thus, simply changing the 2nd line to cFile = ctypes.CDLL("libchess.so", winmode=0) works as expected.
I am new with programming, so it is maybe harder for me to understand but I have the following issue:
I have a script that imports "eurostag.dll", which according to its manual, should work until Python 3.6. (but the manual is not updated, so it may work also with later updates, I assume).\ The issue is that I have Python 3.8. and when running the script I receive the following message:
"Failed to load EUROSTAG library (Could not find module 'D:\Eurostag\eustag_esg.dll' (or one of its dependencies). Try using the full path with constructor syntax.)"
I have tried to move the .dll library where the script is, but nothing changed. I tried also changing the directory with os.chdir, but the same message appears (with the same 'D:\Eurostag\eustag_esg.dll', so the directory was not changed.
Does anybody know if there is any workaround for this?
Thank you!
I'm new to Python and using PyCharm professional as my IDE. I have a small section of code that I want to work with from a longer file, so I created a "scratch file" with Python set as the interpreter. However, even just with importing modules I'm getting errors that the modules can't be found (even with standard modules). The file is set as a "Python" scratch file, so I'm not sure what else I need to do. The code I'm trying to run is:
from zipfile import ZipFile
import urllib
testfile = urllib.request.urlretrieve("https://ihmecovid19storage.blob.core.windows.net/latest/ihme-covid19.zip", "ihme-covid19.zip")
print("File saved at: " + (str(os.getcwd())))
with ZipFile('ihme-covid19.zip', 'r') as zipobj:
print(zipobj.printdir())
zipobj.extractall()
Everything is showing up with an error - no module urllib, no module zipfile, etc.
This may come across like a non-answer, but I do think this is a bug in Pycharm.
I even downloaded an update, restarted pycharm, and rebuilt my indexes. Same thing, here -- core Python modules are not registering as being importable (i.e. import sys or import inspect both failing type-checking).
We should probably move this to Pycharm's bug tracker. For example:
Here is a similar issue around a specific import NamedTuple
Here's an issue around the default interpreter in scratch files
There may be a better existing bug ticket, so please do look and report back here -- I'll update this comment if someone finds or creates a better bug ticket on youtrack.jetbrains.com and shares the link.
I have a big C++ module with Python 3 bindings using Boost.Python, that I compile to a .so file using CMake on macOS.
When I try to import it in the REPL, everything seems to work fine:
>>>import myModule
>>>
However, as soon as I run the import statement, the famous rocket icon of Python shows up in the Dock and stays there jumping for some minutes and stops after. Obviously then, I cannot access any of the functions defined in my module, so the import looks fine but doesn't actually do anything.
I tried looking in the Console and saw that whenever I import myModule, I get two launchservicesd[83]: SecTaskLoadEntitlements failed error=22.
It brought me to this and that related questions but I can't find what the exact problem is.
The C++ module is huge so I just can't look at the code and find the problem, thus I'm asking for any hints about at least how to debug that problem.
I can suggest the following steps:
Try to import that module though local python session. So, run interactive python interpreter, and 'import myModule'.
If bad, try to check:
are python version, with which myMoudle was linked with, is similiar to used interpreter
check that build architectires are the same
check that you can load even simple boost.python example module
If ok, check that you have correctly set up module search path in your python code.
I am having this problem that we compiled our python project into an exe with py2exe and the resulting exe does not work unless it is executed as an administrator.
I wanted to ask that is this supposed to happen ?? As in there are so many applications that we can run as not an administrator so is there any way I can convert my python code to such an application ...
Thanks a lot..
It sounds like your application is attempting to write to a directory for which the basic user doesn't have access; most likely the "Program Files" directory. I believe in Vista/Win7 this is not allowed, and the standard convention is to write to the user's appdata folder, for any user data you might wish to store.
You can reliably obtain the location of this directory using the ctypes module, here's an example:
import ctypes
from ctypes import wintypes
def get_appdata_directory():
CSIDL_APPDATA = 0x001a
dll = ctypes.windll.shell32
app_data_directory = ctypes.create_unicode_buffer(wintypes.MAX_PATH)
found = dll.SHGetFolderPathW(0, CSIDL_APPDATA, 0, 0, app_data_directory)
# FYI: if `found` is False, then it failed to locate the appdata directory
# and app_data_directory.value is empty. So you might want to add some
# code here to verify that a valid path is going to be returned.
# This would probably only happen on older versions of windows,
# but, this is just a guess as I don't have any older OSs available
# for testing. (see my note below)
return app_data_directory.value
appdata = get_appdata_directory()
print(appdata)
# outputs something such as: 'C:\Users\bob\AppData'
Note: I believe the appdata folder was introduced with WinXP/Win2k. Not sure about WinME and prior, however, I don't believe you have to worry about the administrator restrictions on these earlier OSs. If you really care to support them, you could use python's builtin platform module along with some conditionals, then simply write user data to the "Program Files" directory for the archaic versions of Windows.
I've had serious problems with getting py2exe to work. Luckily I found the excellent PyInstaller which not only works, but also creates smaller executables. I haven't ran into the problem you mention with PyInstaller so I recommend trying it.
http://www.pyinstaller.org/