I am loading a dll with ctypes under Cygwin with the following:
import ctypes
ctypes.cdll.LoadLibrary('foo.dll')
How can I get the absolute path of my dll?
The problem is that I have absolutely no clues where the dll is located. Can I relate on the following to get this information?
subprocess.Popen(["which", lib], stdout=subprocess.PIPE).stdout.read().strip()
In Unix, the path of a loaded shared library can be determined by calling dladdr with the address of a symbol in the library, such as a function.
Example:
import ctypes
import ctypes.util
libdl = ctypes.CDLL(ctypes.util.find_library('dl'))
class Dl_info(ctypes.Structure):
_fields_ = (('dli_fname', ctypes.c_char_p),
('dli_fbase', ctypes.c_void_p),
('dli_sname', ctypes.c_char_p),
('dli_saddr', ctypes.c_void_p))
libdl.dladdr.argtypes = (ctypes.c_void_p, ctypes.POINTER(Dl_info))
if __name__ == '__main__':
import sys
info = Dl_info()
result = libdl.dladdr(libdl.dladdr, ctypes.byref(info))
if result and info.dli_fname:
libdl_path = info.dli_fname.decode(sys.getfilesystemencoding())
else:
libdl_path = u'Not Found'
print(u'libdl path: %s' % libdl_path)
Output:
libdl path: /lib/x86_64-linux-gnu/libdl.so.2
If you are on Windows and do not need a programmatical solution for instance if you are debugging something, you can use Process Explorer to look which DLLs python.exe has loaded.
Related
I found this link for Magick.NET
Is there way to refer ghostscript's DLLs without installing?
It mentions function:
MagickNET.SetGhostscriptDirectory
Is there an equivalent for Python?
I tried setting path, but in python program, it fails to load gsdll64.dll when copied to other machine:
My code:
import os, time
import sys
import camelot.io as camelot
import traceback
sys.path.insert(0, r'C:\gs\gs9.56.1\bin')
sys.path.insert(0, r'C:\gs\gs9.56.1\lib')
print('path',sys.path)
#check ghostscript lib - as done in ghostscript_backend.py
import ctypes
from ctypes.util import find_library
mylib = find_library("".join(("gsdll", str(ctypes.sizeof(ctypes.c_voidp) * 8), ".dll")))
print(mylib,os.getcwd())
if mylib is None:
print('gsdll not loaded')
In site-packages\ghostscript\_gsprint.py
The code is written to get dll path from windows registry.
In function __win32_finddll, I added code to check and load dll from my folder:
LooseVrsn = "9.56.1" #from dll->Properties->details->ProductVersion
dll_path = os.path.join(os.getcwd(),'gsdll64.dll')
dlls.append((LooseVrsn, dll_path))
I want to work with the APIs of the program of structural analysis (civil engineering) Autodesk Robot Structural Analysis.
With IronPython I initialize the variables as follows:
import clr
clr.AddReferenceToFileAndPath(‘mypfad\Interop.RobotOM.dll’)
from RobotOM import *
robapp = RobotApplicationClass()
robproj = robapp.Project
robstruct = robproj.Structure
With robstruct I can call the API functions and continue working.
Now I’d like to do the same with Python 3. I have tried with ctypes and with numpy.ctypeslib without success:
import ctypes
lib_ctypes = ctypes.cdll[‘mypfad\Interop.RobotOM.dll']
print(lib_ctypes)
<CDLL 'mypfad \Interop.RobotOM.dll', handle 1a1ff900000 at 0x1a1e8e22710>
import numpy
lib_numpy = numpy.ctypeslib.load_library('Interop.RobotOM.dll', 'mypfad’)
print(lib_ numpy)
<CDLL 'mypfad\Interop.RobotOM.dll', handle 1a1ffb40000 at 0x1a1ffb194e0>
And I don’t how to continue.
My questions are: is this the right way and how shall I continue?
Edited 05.10.2018
Original code with IronPython:
import clr
# Add Robot Structural Analysis API reference
clr.AddReferenceToFileAndPath(
'C:\Program Files\Common Files\Autodesk Shared\Extensions 2018\Framework\Interop\Interop.RobotOM.dll'
)
# Add needed import to be able to use Robot Structural Analysis objects
from RobotOM import *
# Connect to the running instance of Robot Structural Analysis
robapp = RobotApplicationClass()
# Get a reference of the current project
robproj = robapp.Project
# Get a reference of the current model
robstruct = robproj.Structure
An attempt according to the comment of The Machine:
import ctypes
my_dll = ctypes.cdll.LoadLibrary(
'C:\Program Files\Common Files\Autodesk Shared\Extensions 2018\Framework\Interop\Interop.RobotOM.dll'
)
robapp = my_dll.RobotApplicationClass()
robproj = robapp.Project
robstruct = robproj.Structure
Result:
AttributeError: function 'RobotApplicationClass' not found
Edited 16.10.2018
Third attempt:
from ctypes import cdll
my_dll = cdll.LoadLibrary(
'C:\Program Files\Common Files\Autodesk Shared\Extensions 2019\Framework\Interop\Interop.RobotOM.dll'
)
my_dll.RobotApplicationClass()
Result:
AttributeError: function 'RobotApplicationClass' not found
Try this:
from ctypes import*
your_dll = cdll.LoadLibrary(‘mypfad\Interop.RobotOM.dll ')
If it is succesfully loaded than you can aces to all of classes and function in that dll file . You can call theme with your_dll.nameofclass .
I already found this question that suggests to use os.path.expanduser(path) to get the user's home directory.
I would like to achieve the same with the "Downloads" folder. I know that this is possible in C#, yet I'm new to Python and don't know if this is possible here too, preferable platform-independent (Windows, Ubuntu).
I know that I just could do download_folder = os.path.expanduser("~")+"/Downloads/", yet (at least in Windows) it is possible to change the Default download folder.
from pathlib import Path
downloads_path = str(Path.home() / "Downloads")
This fairly simple solution (expanded from this reddit post) worked for me
import os
def get_download_path():
"""Returns the default downloads path for linux or windows"""
if os.name == 'nt':
import winreg
sub_key = r'SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders'
downloads_guid = '{374DE290-123F-4565-9164-39C4925E467B}'
with winreg.OpenKey(winreg.HKEY_CURRENT_USER, sub_key) as key:
location = winreg.QueryValueEx(key, downloads_guid)[0]
return location
else:
return os.path.join(os.path.expanduser('~'), 'downloads')
The GUID can be obtained from Microsoft's KNOWNFOLDERID docs
This can be expanded to work more generically other directories
For python3+ mac or linux
from pathlib import Path
path_to_download_folder = str(os.path.join(Path.home(), "Downloads"))
Correctly locating Windows folders is somewhat of a chore in Python. According to answers covering Microsoft development technologies, such as this one, they should be obtained using the Vista Known Folder API. This API is not wrapped by the Python standard library (though there is an issue from 2008 requesting it), but one can use the ctypes module to access it anyway.
Adapting the above answer to use the folder id for downloads shown here and combining it with your existing Unix code should result in code that looks like this:
import os
if os.name == 'nt':
import ctypes
from ctypes import windll, wintypes
from uuid import UUID
# ctypes GUID copied from MSDN sample code
class GUID(ctypes.Structure):
_fields_ = [
("Data1", wintypes.DWORD),
("Data2", wintypes.WORD),
("Data3", wintypes.WORD),
("Data4", wintypes.BYTE * 8)
]
def __init__(self, uuidstr):
uuid = UUID(uuidstr)
ctypes.Structure.__init__(self)
self.Data1, self.Data2, self.Data3, \
self.Data4[0], self.Data4[1], rest = uuid.fields
for i in range(2, 8):
self.Data4[i] = rest>>(8-i-1)*8 & 0xff
SHGetKnownFolderPath = windll.shell32.SHGetKnownFolderPath
SHGetKnownFolderPath.argtypes = [
ctypes.POINTER(GUID), wintypes.DWORD,
wintypes.HANDLE, ctypes.POINTER(ctypes.c_wchar_p)
]
def _get_known_folder_path(uuidstr):
pathptr = ctypes.c_wchar_p()
guid = GUID(uuidstr)
if SHGetKnownFolderPath(ctypes.byref(guid), 0, 0, ctypes.byref(pathptr)):
raise ctypes.WinError()
return pathptr.value
FOLDERID_Download = '{374DE290-123F-4565-9164-39C4925E467B}'
def get_download_folder():
return _get_known_folder_path(FOLDERID_Download)
else:
def get_download_folder():
home = os.path.expanduser("~")
return os.path.join(home, "Downloads")
A more complete module for retrieving known folders from Python is available on github.
Some linux distributions localize the name of the Downloads folder. E.g. after changing my locale to zh_TW, the Downloads folder became /home/user/下載. The correct way on linux distributions (using xdg-utils from freedesktop.org) is to call xdg-user-dir:
import subprocess
# Copy windows part from other answers here
try:
folder = subprocess.run(["xdg-user-dir", "DOWNLOAD"],
capture_output=True, text=True).stdout.strip("\n")
except FileNotFoundError: # if the command is missing
import os.path
folder = os.path.expanduser("~/Downloads") # fallback
Note that the use of capture_output requires Python ≥3.7.
If you already use GLib or don't mind adding more dependencies, see also
these approaches using packages.
For python3 on windows try:
import os
folder = os.path.join(os.path.join(os.environ['USERPROFILE']), 'folder_name')
print(folder)
I know how to use wmi, I have used it before, however, the wmi class it seems i need to call is GetSystemPowerStatus. but i am having trouble finding and documentation on it. to be able to access it, i need to know the namespace, and the format of the data inside the class. could someone help me? Also some sample code would be nice.
Using ctypes, you can call win32 api:
from ctypes import *
class PowerClass(Structure):
_fields_ = [('ACLineStatus', c_byte),
('BatteryFlag', c_byte),
('BatteryLifePercent', c_byte),
('Reserved1',c_byte),
('BatteryLifeTime',c_ulong),
('BatteryFullLifeTime',c_ulong)]
powerclass = PowerClass()
result = windll.kernel32.GetSystemPowerStatus(byref(powerclass))
print(powerclass.BatteryLifePercent)
Above code comes from here.
Using Win32_Battery class (You need to install pywin32):
from win32com.client import GetObject
WMI = GetObject('winmgmts:')
for battery in WMI.InstancesOf('Win32_Battery'):
print(battery.EstimatedChargeRemaining)
Alternative that use wmi package:
import wmi
w = wmi.WMI()
for battery in w.query('select * from Win32_Battery'):
print battery.EstimatedChargeRemaining
```
import subprocess
wmic = subprocess.getoutput("wmic path win32_battery get EstimatedChargeRemaining")
print(wmic)
```
output:
EstimatedChargeRemaining
96
I'm using ctypes to load a DLL in Python. This works great.
Now we'd like to be able to reload that DLL at runtime.
The straightforward approach would seem to be:
1. Unload DLL
2. Load DLL
Unfortunately I'm not sure what the correct way to unload the DLL is.
_ctypes.FreeLibrary is available, but private.
Is there some other way to unload the DLL?
you should be able to do it by disposing the object
mydll = ctypes.CDLL('...')
del mydll
mydll = ctypes.CDLL('...')
EDIT: Hop's comment is right, this unbinds the name, but garbage collection doesn't happen that quickly, in fact I even doubt it even releases the loaded library.
Ctypes doesn't seem to provide a clean way to release resources, it does only provide a _handle field to the dlopen handle...
So the only way I see, a really, really non-clean way, is to system dependently dlclose the handle, but it is very very unclean, as moreover ctypes keeps internally references to this handle. So unloading takes something of the form:
mydll = ctypes.CDLL('./mylib.so')
handle = mydll._handle
del mydll
while isLoaded('./mylib.so'):
dlclose(handle)
It's so unclean that I only checked it works using:
def isLoaded(lib):
libp = os.path.abspath(lib)
ret = os.system("lsof -p %d | grep %s > /dev/null" % (os.getpid(), libp))
return (ret == 0)
def dlclose(handle)
libdl = ctypes.CDLL("libdl.so")
libdl.dlclose(handle)
It is helpful to be able to unload the DLL so that you can rebuild the DLL without having to restart the session if you are using iPython or similar work flow. Working in windows I have only attempted to work with the windows DLL related methods.
REBUILD = True
if REBUILD:
from subprocess import call
call('g++ -c -DBUILDING_EXAMPLE_DLL test.cpp')
call('g++ -shared -o test.dll test.o -Wl,--out-implib,test.a')
import ctypes
import numpy
# Simplest way to load the DLL
mydll = ctypes.cdll.LoadLibrary('test.dll')
# Call a function in the DLL
print mydll.test(10)
# Unload the DLL so that it can be rebuilt
libHandle = mydll._handle
del mydll
ctypes.windll.kernel32.FreeLibrary(libHandle)
I don't know much of the internals so I'm not really sure how clean this is. I think that deleting mydll releases the Python resources and the FreeLibrary call tells windows to free it. I had assumed that freeing with FreeLibary first would have produced problems so I saved a copy of the library handle and freed it in the order shown in the example.
I based this method on ctypes unload dll which loaded the handle explicitly up front. The loading convention however does not work as cleanly as the simple "ctypes.cdll.LoadLibrary('test.dll')" so I opted for the method shown.
windows and linux compatible minimal reproducible example from 2020
overview of similar discussion
Here an overview of similar discussions (where I constructed this answer from).
How can I unload a DLL using ctypes in Python?
ctypes unload dll
Unload shared library inside ctypes loaded shared library
forcing ctypes.cdll.LoadLibrary() to reload library from file
minimal reproducible example
This is for windows and linux, hence there are 2 scripts given for compilation.
Tested under:
Win 8.1, Python 3.8.3 (anaconda), ctypes 1.1.0, mingw-w64 x86_64-8.1.0-posix-seh-rt_v6-rev0
Linux Fedora 32, Python 3.7.6 (anaconda), ctypes 1.1.0, g++ 10.2.1
cpp_code.cpp
extern "C" int my_fct(int n)
{
int factor = 10;
return factor * n;
}
compile-linux.sh
#!/bin/bash
g++ cpp_code.cpp -shared -o myso.so
compile-windows.cmd
set gpp="C:\Program Files\mingw-w64\x86_64-8.1.0-posix-seh-rt_v6-rev0\mingw64\bin\g++.exe"
%gpp% cpp_code.cpp -shared -o mydll.dll
PAUSE
Python code
from sys import platform
import ctypes
if platform == "linux" or platform == "linux2":
# https://stackoverflow.com/a/50986803/7128154
# https://stackoverflow.com/a/52223168/7128154
dlclose_func = ctypes.cdll.LoadLibrary('').dlclose
dlclose_func.argtypes = [ctypes.c_void_p]
fn_lib = './myso.so'
ctypes_lib = ctypes.cdll.LoadLibrary(fn_lib)
handle = ctypes_lib._handle
valIn = 42
valOut = ctypes_lib.my_fct(valIn)
print(valIn, valOut)
del ctypes_lib
dlclose_func(handle)
elif platform == "win32": # Windows
# https://stackoverflow.com/a/13129176/7128154
# https://stackoverflow.com/questions/359498/how-can-i-unload-a-dll-using-ctypes-in-python
lib = ctypes.WinDLL('./mydll.dll')
libHandle = lib._handle
# do stuff with lib in the usual way
valIn = 42
valOut = lib.my_fct(valIn)
print(valIn, valOut)
del lib
ctypes.windll.kernel32.FreeLibrary(libHandle)
A more general solution (object oriented for shared libraries with dependencies)
If a shared library has dependencies, this does not necessarily work anymore (but it can - depends on the dependency ^^). I did not investigate the very details, but it looks like the mechanism is the following: library and dependency are loaded. As the dependency is not unloaded, the library can not get unloaded.
I found, that if I include OpenCv (Version 4.2) into my shared library, this messes up the unloading procedure. The following example was only tested on the linux system:
code.cpp
#include <opencv2/core/core.hpp>
#include <iostream>
extern "C" int my_fct(int n)
{
cv::Mat1b mat = cv::Mat1b(10,8,(unsigned char) 1 ); // change 1 to test unloading
return mat(0,1) * n;
}
Compile with
g++ code.cpp -shared -fPIC -Wall -std=c++17 -I/usr/include/opencv4 -lopencv_core -o so_opencv.so
Python code
from sys import platform
import ctypes
class CtypesLib:
def __init__(self, fp_lib, dependencies=[]):
self._dependencies = [CtypesLib(fp_dep) for fp_dep in dependencies]
if platform == "linux" or platform == "linux2": # Linux
self._dlclose_func = ctypes.cdll.LoadLibrary('').dlclose
self._dlclose_func.argtypes = [ctypes.c_void_p]
self._ctypes_lib = ctypes.cdll.LoadLibrary(fp_lib)
elif platform == "win32": # Windows
self._ctypes_lib = ctypes.WinDLL(fp_lib)
self._handle = self._ctypes_lib._handle
def __getattr__(self, attr):
return self._ctypes_lib.__getattr__(attr)
def __del__(self):
for dep in self._dependencies:
del dep
del self._ctypes_lib
if platform == "linux" or platform == "linux2": # Linux
self._dlclose_func(self._handle)
elif platform == "win32": # Windows
ctypes.windll.kernel32.FreeLibrary(self._handle)
fp_lib = './so_opencv.so'
ctypes_lib = CtypesLib(fp_lib, ['/usr/lib64/libopencv_core.so'])
valIn = 1
ctypes_lib.my_fct.argtypes = [ctypes.c_int]
ctypes_lib.my_fct.restype = ctypes.c_int
valOut = ctypes_lib.my_fct(valIn)
print(valIn, valOut)
del ctypes_lib
Let me know, when there are any issues with the code examples or the explanation given so far. Also if you know a better way! It would be great, if we could settle the issue once and for all.
For total cross-compatibility: I maintain a list of various dlclose() equivalents for each platform and which library to get them from. It's a bit of a long list but feel free to just copy/paste it.
import sys
import ctypes
import platform
OS = platform.system()
if OS == "Windows": # pragma: Windows
dll_close = ctypes.windll.kernel32.FreeLibrary
elif OS == "Darwin":
try:
try:
# macOS 11 (Big Sur). Possibly also later macOS 10s.
stdlib = ctypes.CDLL("libc.dylib")
except OSError:
stdlib = ctypes.CDLL("libSystem")
except OSError:
# Older macOSs. Not only is the name inconsistent but it's
# not even in PATH.
stdlib = ctypes.CDLL("/usr/lib/system/libsystem_c.dylib")
dll_close = stdlib.dlclose
elif OS == "Linux":
try:
stdlib = ctypes.CDLL("")
except OSError:
# Alpine Linux.
stdlib = ctypes.CDLL("libc.so")
dll_close = stdlib.dlclose
elif sys.platform == "msys":
# msys can also use `ctypes.CDLL("kernel32.dll").FreeLibrary()`. Not sure
# if or what the difference is.
stdlib = ctypes.CDLL("msys-2.0.dll")
dll_close = stdlib.dlclose
elif sys.platform == "cygwin":
stdlib = ctypes.CDLL("cygwin1.dll")
dll_close = stdlib.dlclose
elif OS == "FreeBSD":
# FreeBSD uses `/usr/lib/libc.so.7` where `7` is another version number.
# It is not in PATH but using its name instead of its path is somehow the
# only way to open it. The name must include the .so.7 suffix.
stdlib = ctypes.CDLL("libc.so.7")
dll_close = stdlib.close
else:
raise NotImplementedError("Unknown platform.")
dll_close.argtypes = [ctypes.c_void_p]
You can then use dll_close(dll._handle) to unload a library dll = ctypes.CDLL("your-library").
This list is taken from this file. I will update the master branch every time I encounter a new platform.
Piotr's answer helped me, but I did run into one issue on 64-bit Windows:
Traceback (most recent call last):
...
ctypes.ArgumentError: argument 1: <class 'OverflowError'>: int too long to convert
Adjusting the argument type of the FreeLibrary call as suggested in this answer solved this for me.
Thus we arrive at the following complete solution:
import ctypes, ctypes.windll
def free_library(handle):
kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)
kernel32.FreeLibrary.argtypes = [ctypes.wintypes.HMODULE]
kernel32.FreeLibrary(handle)
Usage:
lib = ctypes.CDLL("foobar.dll")
free_library(lib._handle)
If you need this functionality, you could write 2 dlls where dll_A loads/Unloads library from dll_B. Use dll_A as as python interface-loader and passthrough for functions in dll_B.