I'm trying to build a Python app to access data inside a proprietary file format (*.abf, by Axon Instruments). The fine folks who specify the format have released a DLL that contains functions you can use to open these files, read/write, etc.
So far so good. I've read a number of posts on SO and around the web about using ctypes to import DLLs, and this seems to work fine - but many of the useful functions in the DLL I'm trying to interact with take integer pointers as an argument, and ctypes doesn't have a data type for interacting with that.
Am I out of luck? Do I need to write this app in a different language, or is there some way to hack this together in Python?
Thanks all,
-Steve
Look deeper. Ctypes does have Pointers.
from ctypes import *
f = c_float()
f_pointer = pointer(f) # full pointer functionality
f_byref = byref(f) # minimalistic pointer just for passing by reference
Related
I got VS2008 project written in managed C++ that produces a utility DLL.
CLR Support: Common Language Runtime Support, Old Syntax (/clr:oldSyntax)
The resulting DLL exports a bunch of functions and complex types based on managed types like System::String, System::Byte and so on.
The DLL must be consumed (linked to) by a Python application. It seems that Python cannot use managed types/classes/structs.
Is it possible to convert the project so that it produces an "unmanaged" interface. That is, instead of working with System::String, I should somehow convert the types to std::string so that the Python app can work with?
The only "good news" is that the Python app that tries to consume the DLL needs to access only ONE exported function.
Additional question: if I only change the signature of that very function so that all the types of in/out parameters are unmanaged C++ would that work? The other exported functions will still work through managed types, but they won't be called at all.
I know Cython's purpose is to create Python extensions modules, but can the compiled libraries made with Cython be loaded by non-python programs? If not, why?
I doubt that you can load them directly on a non-python program; looking at the C code generated by the simplest Cython script it's apparent that you need all the Python scaffolding to make it work. That said, you can do it indirectly for example from C++. In C++ I use boost.python to embed the Python interpreter and load some of my modules and scripts. This may seem convoluted, but allows you to quickly use whatever extensions you already have written in Python from C++, provided that you build the appropriate gluing code (see the boost.python wiki).
The disadvantage of this approach is that you are really loading a full Python interpreter just to be able to use some extension. This wasn't an issue for me since I already had the python extension and was embedding Python in my application to provide basic scripting ability, but I would not use this approach to write new libraries.
There are two mechanisms by which you can make cdef’ed Cython structures available externally:
Use a cdef public declaration – and/or also
Use the api keyword.
Here’s one of the examples of both mechanisms, from the aforelinked pages:
cdef public struct Vehicle:
int speed
float power
cdef api void activate(Vehicle *v):
if v.speed >= 88 and v.power >= 1.21:
print "Time travel achieved"
Each of these methods will instruct the Cython compiler to generate a header (“.h”) file that you can then integrate with your orthogonal C/C++ project.
A cdef public declaration yields a file named modulename.h; using the structures in this file will require you to link with the compiled Cython extension module.
An api declaration (which may be used simultaneously with cdef public, if you so desire) yields a modulename_api.h file; code consuming an api-based header will not need to link to the extension module – but it will need to call the cdef’d function import_modulename() before any of the API code can be used (a tactic the Cython-inclined users of NumPy will find familiar).
In my personal experience, it takes little effort to expose and subsequently employ encythoned structures as an external API in this fashion, provided that the struct layouts dovetail well with the consuming code, and that you’re willing to contend with manually managing the GIL in your C/C++ code in order to make things work.
I've read in a lot of places that C-libraries can be ported to or written in python using the ctypes module from the standard library.
I've gone through the help('ctypes') page and from what I could gather I can create some of the C structures in Python, but my question is how do I use these to access the underlying system calls? For eg. when trying to port something like 'sys/if.h' to Python?
Can someone point me to good resources/documentation regarding the same?
If you want access to the system calls you could do something like this:
>>> from ctypes import CDLL
>>> libc = CDLL('libc.so.6')
>>> print libc.strlen('abcde')
5
Reference: http://blog.bstpierre.org/using-pythons-ctypes-to-make-system-calls
Or (This is the tricky part)
Wrap a system call as outlined here into your C code:
How to reimplement (or wrap) a syscall function in linux?
And, then write a compliant source code file which will be used by CTypes, as per here:
http://www.scipy.org/Cookbook/Ctypes
I hope this helps.
My end goal is to query NVAPI for gpu usage and other statistics in python. See http://developer.nvidia.com/nvapi
from ctypes import WinDLL
nvapi = WinDLL("nvapi.dll")
print nvapi# <WinDLL 'nvapi.dll', handle 718a0000 at 27c0050>
print nvapi.nvapi_QueryInterface# <_FuncPtr object at 0x026D8E40>
print nvapi.nvapi_QueryInterface()# returns 0
print nvapi.NvAPI_Initialize# AttributeError: function 'NvAPI_Initialize' not found
print nvapi.NvAPI_SYS_GetChipSetInfo# AttributeError: function 'NvAPI_SYS_GetChipSetInfo' not found
Here is a copy of the header file available for download from the link above: http://paste.pound-python.org/show/7337/
At this point, I am just trying to familiarize myself with the api... so what am I doing wrong? I can't figure out how to call any of the functions listed in the header file.
Are you sure it's a WinDLL? From the header file, it looks like a standard C calling convention to me. Have you tried CDLL instead?
EDIT:
I see now. The header you pointed to isn't actually the interface for nvapi.dll--it is a wrapper around it that must be statically linked.
From the docs downloaded from NVIDIA's developer site:
Use a Static Link with Applications
NvAPI cannot be dynamically linked to applications. You must create a static link to the library and then call NvAPI_Initialize(), which loads nvapi.dll dynamically.
If the NVIDIA drivers are not installed on the system or nvapi.dll is not present when the application calls NvAPI_Initialize(), the call just returns an error. The application will still load.
I would guess that the actual calls in nvapi.dll are completely different than the ones exposed in this wrapper library. I can't seem to find any documentation on those though. Perhaps they are internal and change between systems.
If you want to use this interface, I'm not really sure what the best solution is. It's a static library and not a dynamic one, so ctypes wouldn't handle it unless you wrapped it in another DLL. I'm not an expert at native code with Python, so maybe someone else will have an easy fix. Sorry.
Taken from your comment:
NvAPI_Initialize call still fails.
saying the function is not found.
NvAPI_Initialize is not exported from the dynamic library nvapi.dll. It is a symbol contained in nvapi.lib, the static library shipped with the NVIDIA SDK, thus it's no wonder you are unable to call it using Python.
Honestly, the easiest route here is to create a small wrapper DLL in C, statically linking to nvapi.lib and exposing a friendly interface to Python.
This may seem like a weird question, but I would like to know how I can run a function in a .dll from a memory 'signature'. I don't understand much about how it actually works, but I needed it badly. Its a way of running unexported functions from within a .dll, if you know the memory signature and adress of it.
For example, I have these:
respawn_f "_ZN9CCSPlayer12RoundRespawnEv"
respawn_sig "568BF18B06FF90B80400008B86E80D00"
respawn_mask "xxxxx?xxx??xxxx?"
And using some pretty nifty C++ code you can use this to run functions from within a .dll.
Here is a well explained article on it:
http://wiki.alliedmods.net/Signature_Scanning
So, is it possible using Ctypes or any other way to do this inside python?
If you can already run them using C++ then you can try using SWIG to generate python wrappers for the C++ code you've written making it callable from python.
http://www.swig.org/
Some caveats that I've found using SWIG:
Swig looks up types based on a string value. For example
an integer type in Python (int) will look to make sure
that the cpp type is "int" otherwise swig will complain
about type mismatches. There is no automatic conversion.
Swig copies source code verbatim therefore even objects in the same namespace
will need to be fully qualified so that the cxx file will compile properly.
Hope that helps.
You said you were trying to call a function that was not exported; as far as I know, that's not possible from Python. However, your problem seems to be merely that the name is mangled.
You can invoke an arbitrary export using ctypes. Since the name is mangled, and isn't a valid Python identifier, you can use getattr().
Another approach if you have the right information is to find the export by ordinal, which you'd have to do if there was no name exported at all. One way to get the ordinal would be using dumpbin.exe, included in many Windows compiled languages. It's actually a front-end to the linker, so if you have the MS LinK.exe, you can also use that with appropriate commandline switches.
To get the function reference (which is a "function-pointer" object bound to the address of it), you can use something like:
import ctypes
func = getattr(ctypes.windll.msvcrt, "##myfunc")
retval = func(None)
Naturally, you'd replace the 'msvcrt' with the dll you specifically want to call.
What I don't show here is how to unmangle the name to derive the calling signature, and thus the arguments necessary. Doing that would require a demangler, and those are very specific to the brand AND VERSION of C++ compiler used to create the DLL.
There is a certain amount of error checking if the function is stdcall, so you can sometimes fiddle with things till you get them right. But if the function is cdecl, then there's no way to automatically check. Likewise you have to remember to include the extra this parameter if appropriate.