I'm trying to wrap an API, that has a lot of functions where it returns a status (no issue there), but the actual output is a structure pointer array. I'm having trouble getting a useful structure back. All of the documentation I've found so far handles returns instead of pointer outputs. This is what everything looks like.
foobar.h
typedef struct
{
unsigned short serialNumber;
unsigned char ipAddr[4];
} CardInfo;
FOOBARSHARED_EXPORT bool CheckInterfaces( unsigned short someNum, // input
unsigned short *numCards, // output
CardInfo **pCardInfo ); // output
My attempt at wrapping it looks like this:
foobar.py
import ctypes as ct
FOOBAR = ct.CDLL('FOOBAR.dll')
class CardInfo(ct.Structure):
__fields__ = [
('serialNumber', ct.c_ushort),
('ipAddr', ct.c_char*4)
]
def check_interfaces(some_num):
FOOBAR.CheckInterfaces.argtypes = [
ct.c_ushort, ct.POINTER(ct.c_ushort), ct.POINTER(ct.POINTER(CardInfo))
]
FOOBAR.CheckInterfaces.restype = ct.c_bool
num_cards = ct.c_ushort(0)
card_info_pointer = ct.POINTER(CardInfo)()
status = FOOBAR.CheckInterfaces(some_num, ct.byref(num_cards), ct.byref(card_info_pointer))
return [card_info_pointer[i] for i in range(num_cards)]
This does in fact return a list of CardInfo objects, but none of the fields are there. Any ideas about what I'm missing?
Try this:
def check_interfaces(some_num):
FOOBAR.CheckInterfaces.restype = ct.c_bool
FOOBAR.CheckInterfaces.argtypes = [
ct.c_ushort,
ct.POINTER(ct.c_ushort),
ct.POINTER(ct.POINTER(CardInfo)),
]
num_cards = ct.c_ushort(0)
card_info_array = (CardInfo * 16)()
status = FOOBAR.CheckInterfaces(some_num, ct.byref(num_cards), ct.byref(card_info_array))
return [card_info_array[i] for i in range(num_cards)]
Thanks to Mark Tolonen's comment. I didn't notice that the correct structure field is _fields_ not __fields__
The correct structure should have been:
class CardInfo(ct.Structure):
_fields_ = [
('serialNumber', ct.c_ushort),
('ipAddr', ct.c_char*4)
]
Related
Why does the following not work?
import ctypes
class Test(ctypes.Structure):
_fields_ = [("my_pointer", ctypes.POINTER(ctypes.c_int16))]
t = Test()
t.my_pointer = ctypes.addressof(ctypes.c_int16(123))
Error:
TypeError: expected LP_c_short instance, got int
Is there way to generate a LP_c_short? Or any 16-bit pointer?
EDIT
Using byref instead of addressof causes:
TypeError: expected LP_c_short instance, got CArgObject
The Test structure is defined that way because it look's like that in a C-DLL.
Use byref or pointer not addressof
a = ctypes.c_int16(123)
t.my_pointer(ctypes.byref(a)
Despite having a name like the C & operator addressof returns a int and not a ctypes pointer.
ctypes.addressof(obj)
Returns the address of the memory buffer as integer. obj must be an instance of a ctypes type
EDIT
Is this not working ? :
import ctypes
class Test(ctypes.Structure):
_fields_ = [("my_pointer", ctypes.POINTER(ctypes.c_int16))]
t = Test()
cc = ctypes.c_int16(123)
t.my_pointer = ctypes.pointer(cc)
print t.my_pointer[0]
I'm not an expert with ctypes, but following the docs the following works for me:
import ctypes
PI16 = ctypes.POINTER(ctypes.c_int16)
class Test(ctypes.Structure):
_fields_ = [
("my_pointer", PI16),
]
t = Test()
t.my_pointer = PI16(ctypes.c_int16(123))
print(t.my_pointer[0])
otherwise, you could do:
t.my_pointer = ctypes.pointer(ctypes.c_int32(123))
the former will allow you to do strange things like:
PI(ctypes.c_int64(123))[0]
I.e. coerce between different width integer types, or worse…
I have a structure in C that looks like this:
typedef struct {
uint32_t id;
uint8_t *buf; // pointer to message data
} msg_t;
and some function that receives that a pointer to such structure and modifies it.
void recv_msg( msg_t *msg ) {
// Stuff happens to message here
return;
}
With cytpes I tried something like this:
from cytpes import CDLL, Structure, POINTER, c_ubyte, c_uint32
class Msg(Structure):
_fields_ = [("id", c_uint32), ("buf", POINTER(c_ubyte * 10))]
lib = CDLL("this_example.so")
get_msg = lib.recv_msg
get_msg.argtypes = [POINTER(Msg)]
get_msg.restype = None
sample_data_array = POINTER(c_ubyte * 10)()
data = sample_data_array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
sample_msg = Msg(1, data)
get_msg(sample_msg)
print sample_msg.id, sample_msg.buf[0] # Should change data
I'm getting a TypeError('expected LP_c_ubyte_Array_10 instance, got list',)
I've also tried a similar approach using Cython:
from libc.stdint cimport uint8_t, uint32_t
cdef extern from "this_example.h":
ctypedef struct msg_t:
uint32_t id;
uint8_t *buf;
void get_msg (msg_t *)
def recv_msg():
# How I would do this I don't know
print msg.id, msg.buf[0]
I should also add, I don't want use numpy (but would reluctantly if I have to). Also, the length of the data array can vary, but I also have a length variable in the msg structure, so I can initialise it to the right length for sending, and just set it to default values and max length for receiving.
Any clues? Thanks!
I'm open to accepting other answers, but for now I thought I would post my solution in the hopes it might help someone else.
I solved it using cytpes and not Cython. I would really like to see a Cython solution though.
from ctypes import CDLL, Structure, POINTER, c_uint8, c_uint32, byref
class Msg(Structure):
_fields_ = [("id", c_uint32), ("buf", POINTER(c_uint8))]
lib = CDLL('this_example.so')
get_msg = lib.recv_msg
get_msg.argtypes = [POINTER(Msg)]
get_msg.restype = None
data = (c_uint8 * 8)()
sample_msg = Msg(1, data)
get_msg(byref(sample_msg))
print sample_msg.id, sample_msg.buf[0] # Should see changed data
I'm writing framework in Python 2.7 which would provide functionality to test C written APIs. I have the DLL and the C source code itself. The problem is that some of the C API functions require as an input structure of a callback functions and I can't find the way how to create structure in Python which fields are the C function pointers. I know how to create callback function in Python using ctypes, but I need to pack those callbacks in C structure to pass to C API. All these in code looks like this:
The C function pointers and the C structure:
#define DRIVERCALLAPI
typedef void*( DRIVERCALLAPI *fn_DRV_COMM_Open )(void*);
typedef void(DRIVERCALLAPI *fn_DRV_COMM_Close)(void*);
typedef int(DRIVERCALLAPI *fn_DRV_COMM_Transfer)(void *, unsigned char *, int);
typedef struct DriverCallbacks_t
{
fn_DRV_COMM_Open drvcomm_Open;
fn_DRV_COMM_Close drvcomm_Close;
fn_DRV_COMM_Transfer drvcomm_Transfer;
} DriverCallbacks;
typedef struct InitDataEntry_t
{
char iface[64];
void* handle;
} InitDataEntry;
Where handle points to an object of DriverCallbacks.
typedef struct InitDataContainer_t
{
uint32_t size;
uint32_t id;
InitDataEntry* data;
} InitDataContainer;
The pointer of a InitDataContainer should be passed to the API function.
void* dev_Create( void* args )
The API initializes the callback functions with appropriate functions to use them later. I need to somehow create Python structures of DriverCallbacks InitDataEntry and InitDataContainer. Any hint on how it can be achieved ? Thanks in advance !
After many experiments I finally found how to create Python structures which correspond to C structures with function pointers as fields. The idea is to use void pointers instead i.e. c_void_p from ctypes. For the provided example the python code would be:
from ctypes import *
class DriverCallbacks(Structure):
_fields_ = [
("drvcomm_Open", c_void_p),
("drvcomm_Close", c_void_p),
("drvcomm_Transfer", c_void_p)
]
class InitDataEntry(Structure):
_fields_ = [
("iface", 64 * c_byte),
("handle", c_void_p)
]
class InitDataContainer(Structure):
_fields_ = [
("size", c_uint),
("id", c_uint),
("data", POINTER(InitDataEntry))
]
The creation of the objects and the library function call could be like this (it's tested and works for me):
lib = cdll.LoadLibrary(LIBNAME)
driverFuncList = DriverCallbacks()
driverFuncList.drvcomm_Open = cast(lib.ftdi_open, c_void_p)
driverFuncList.drvcomm_Close = cast(lib.ftdi_close, c_void_p)
driverFuncList.drvcomm_Transfer = cast(lib.ftdi_transfer, c_void_p)
initData = InitDataEntry()
libc = cdll.msvcrt
libc.strcpy(byref(initData.iface), c_char_p("DriverCallbacks"))
initData.handle = cast(pointer(driverFuncList), c_void_p)
initDataCont = InitDataContainer()
initDataCont.size = c_uint(3)
initDataCont.id = c_uint(0)
initDataCont.data = pointer(initData)
ret = lib.dev_Create(byref(initDataCont))
The driverFuncList object can be filled also from within the C library if there is a such function which sets the callback function pointers.
I have defined a struct :
class FILE_HANDLE(Structure):
_fields_ = [
("handle_bytes", c_uint),
("handle_type", c_int),
("f_handle", POINTER(c_char))
]
The struct is initialised :
buf = create_string_buffer(f_handle.handle_bytes)
fh = FILE_HANDLE(c_uint(8), c_int(0), buf)
I am passing it by reference to a function that populates it.
ret = libc.name_to_handle_at(dirfd, pathname, byref(fh), byref(mount_id), flags)
I can check with strace that the call works, but I have not been able to figure out how to access the value of fh.f_handle
fh.f_handle type is <ctypes.LP_c_char object at 0x7f1a7ca17560>
fh.f_handle.contents type is <ctypes.LP_c_char object at 0x7f1a7ca17560> but I get a SIGSEGV if I try to access its value.
How could I get 8 bytes from f_handle into a string or array ?
Everything actually looks right for what you've shown, but without seeing the explicit C definition of the structure and function you are calling it is difficult to see the problem.
Here's an example that works with what you have shown. I inferred what the C definitions should be from what you have declared in Python, but most likely your definition is different if you get a segfault.
C Code (Windows)
struct FILE_HANDLE
{
unsigned int handle_bytes;
int handle_type;
char* f_handle;
};
__declspec(dllexport) int name_to_handle_at(int dirfd, char* pathname, struct FILE_HANDLE* fh, int* mount_id, int flags)
{
unsigned int i;
printf("dirfd=%d pathname=%s fh->handle_bytes=%u fh->handle_type=%d flags=%d\n", dirfd, pathname, fh->handle_bytes, fh->handle_type, flags);
for(i = 0; i < fh->handle_bytes; ++i)
fh->f_handle[i] = 'A' + i;
*mount_id = 123;
return 1;
}
Python code (Works in Python 2 and 3):
from __future__ import print_function
from ctypes import *
class FILE_HANDLE(Structure):
_fields_ = [("handle_bytes", c_uint),
("handle_type", c_int),
("f_handle", POINTER(c_char))]
buf = create_string_buffer(8);
fh = FILE_HANDLE(8,0,buf)
libc = CDLL('test.dll')
mount_id = c_int(0)
ret = libc.name_to_handle_at(1,b'abc',byref(fh),byref(mount_id),7)
print('mount_id =',mount_id.value)
print('fh.f_handle =',fh.f_handle[:fh.handle_bytes])
Output
dirfd=1 pathname=abc fh->handle_bytes=8 fh->handle_type=0 flags=7
mount_id = 123
fh.f_handle = b'ABCDEFGH'
Note that since the structure is declared as a pointer to a single character, printing fh.f_handle.contents would only print b'A'. Using slicing, I've instructed Python to index the pointer up to the length allocated.
If this doesn't work for you, provide a Minimal, Complete, and Verifiable example (as I have) to reproduce your error exactly.
fh.f_handle is shown as LP_c_char because you defined the struct that way.
buf = create_string_buffer(8)
print type(buf)
fh = FILE_HANDLE(c_uint(8), c_int(0), buf)
print type(fh.f_handle)
Will output
<class 'ctypes.c_char_Array_8'>
<class 'ctypes.LP_c_char'>
You have defined your struct to accept a pointer to a c_char. So when you try to access fh.f_handle it will expect the value to be a memory address containing the address to the actual single c_char.
But by trying to input a c_char * 8 from the string buffer it will convert the first part of your buffer to a pointer.
Python tries to dereference your char[0] which means that it will look for a memory address with the value of the character you have defined in char[0]. That memory address is not valid, so your interpreter will signal a SIGSEGV.
Now to create a class which properly handles a variable length buffer is quite difficult. An easier option is to pass the buffer as an opaque handle, to access it afterwards you need to cast it back to a char array.
Example:
class FILE_HANDLE(Structure):
_fields_ = [
("handle_bytes", c_uint),
("handle_type", c_int),
("f_handle", c_void_p)
]
buf = create_string_buffer(8)
buf = cast(buf, c_void_p)
fh = FILE_HANDLE(c_uint(8), c_int(0), buf)
f_handle_value = (c_char * fh.handle_bytes).from_address(fh.f_handle)
I'm trying to use the GetTokenInformation function with ctypes.
The problem is that it will always print None.
import winappdbg
from ctypes import *
LPVOID = c_void_p
PVOID = LPVOID
PSID = PVOID
DWORD = c_uint32
class SID_AND_ATTRIBUTES(Structure):
_fields_ = [
("Sid", PSID),
("Attributes", DWORD),
]
class TOKEN_USER(Structure):
_fields_ = [
("User", SID_AND_ATTRIBUTES),]
tokenprivs = (winappdbg.win32.TOKEN_QUERY | winappdbg.win32.TOKEN_READ | winappdbg.win32.TOKEN_IMPERSONATE | winappdbg.win32.TOKEN_QUERY_SOURCE | winappdbg.win32.TOKEN_DUPLICATE | winappdbg.win32.TOKEN_ASSIGN_PRIMARY)
hProcess = winappdbg.win32.OpenProcess(winappdbg.win32.PROCESS_QUERY_INFORMATION, False, winappdbg.win32.GetCurrentProcessId())
hToken = winappdbg.win32.OpenProcessToken(hProcess, DesiredAccess = tokenprivs)
tokeninformation = TOKEN_USER()
dwLength = DWORD(0)
windll.advapi32.GetTokenInformation(hToken, winappdbg.win32.TokenUser, byref(tokeninformation), sizeof(TOKEN_USER), byref(dwLength))
print tokeninformation.User.Sid
P.S. I'm aware that win32security.GetTokenInformation exists. But I want to use ctypes because of the real process handlers.
EDIT:
Working code:
import winappdbg
from ctypes import *
LPVOID = c_void_p
PVOID = LPVOID
PSID = PVOID
DWORD = c_uint32
class SID_AND_ATTRIBUTES(Structure):
_fields_ = [
("Sid", PSID),
("Attributes", DWORD),
]
class TOKEN_USER(Structure):
_fields_ = [
("User", SID_AND_ATTRIBUTES),]
tokenprivs = (winappdbg.win32.TOKEN_QUERY | winappdbg.win32.TOKEN_READ | winappdbg.win32.TOKEN_IMPERSONATE | winappdbg.win32.TOKEN_QUERY_SOURCE | winappdbg.win32.TOKEN_DUPLICATE | winappdbg.win32.TOKEN_ASSIGN_PRIMARY)
hProcess = winappdbg.win32.OpenProcess(winappdbg.win32.PROCESS_QUERY_INFORMATION, False, winappdbg.win32.GetCurrentProcessId())
hToken = winappdbg.win32.OpenProcessToken(hProcess, DesiredAccess = tokenprivs)
dwSize = DWORD(0)
pStringSid = winappdbg.win32.LPSTR()
windll.advapi32.GetTokenInformation(hToken, winappdbg.win32.TokenUser, None, 0, byref(dwSize))
address = windll.kernel32.LocalAlloc(0x0040, dwSize)
print "Address: " + str(address)
windll.advapi32.GetTokenInformation(hToken, winappdbg.win32.TokenUser, address, dwSize, byref(dwSize))
print FormatError(GetLastError())
pToken_User = cast(address, POINTER(TOKEN_USER))
windll.advapi32.ConvertSidToStringSidA(pToken_User.contents.User.Sid, byref(pStringSid))
print "SID: " + pStringSid.value
Although a query for the token information class TokenUser stores a TOKEN_USER structure in the target buffer, it does not contain all the required information by itself. As you can see in the structure's documentation, it contains just the SID_AND_ATTRIBUTES structure, which in turn contains only a pointer to a SID and an integer with flags.
If you added some more error checking, you would see that your call to GetTokenInformation() does not succeed, the reported error code is ERROR_INSUFFICIENT_BUFFER (122) and dwLength is set to something like 36 (definitely more than sizeof(TOKEN_USER), which is just 8).
Apparently the function wants enough space in the target buffer to store even the SID itself, then point to that extra data from the documented output structure TOKEN_USER.
I don't know Ctypes much, but you will need to create the output buffer as a real buffer/array instead of just the structure, and perform some casting on it to get the data. You can either take the easy way and create the buffer large enough on the first try, or call the function twice, first to get the required buffer length, second to fill it.