We Keep Coding

Python ctypes throws TypeError: one character bytes, bytearray or integer expected for functions with pointer references

I am new to both Python and ctypes the module. Trying to call C++ a function by loading a shared library. Here is the prototype of the function I want to call.
void foo_func(const char *binary, size_t binsz, size_t memsz, void *params, size_t paramssz, void *settings);
And here is the code I have written to call this function.
import ctypes
import pathlib
class virt_buff(ctypes.Structure):
_fields_ = [("x", ctypes.c_int), ("y", ctypes.c_int)]
if __name__ == "__main__":
libname = pathlib.Path().absolute() / "build/libfoo.so"
c_lib = ctypes.CDLL(libname)
func_param = virt_buff(7, 0)
with open("build/fib.bin", mode='rb') as file: # b is important -> binary
binary = file.read()
c_lib.foo_func(ctypes.c_char(binary), file.tell(), 0x9000 + (file.tell() & ~0xFFF), func_param, 4, NULL)
But when I run this code, it gives me the following output.
Traceback (most recent call last):
File "ct_test.py", line 32, in <module>
c_lib.foo_func(ctypes.c_char(binary), file.tell(), 0x9000 + (file.tell() & ~0xFFF), virt_param, 4, NULL)
TypeError: one character bytes, bytearray or integer expected
I tried a lot of things and nothing seems to work. Can anyone help me find out what the actual problem is?

The first error message is due to trying to pass a single character (c_char) and initializing it with a multiple character data blob. You want c_char_p for C char* instead. But there are other issues after that. Declaring .argtypes and .restype for your function correctly will help ctypes catch errors.
Make the following additional commented fixes:
import ctypes as ct
import pathlib
class virt_buff(ct.Structure):
_fields_ = [("x", ct.c_int), ("y", ct.c_int)]
if __name__ == "__main__":
libname = pathlib.Path().absolute() / "build/libfoo.so"
# Declare argument types and result type that match the C call for ctypes to error check
# void foo_func(const char *binary, size_t binsz, size_t memsz, void *params, size_t paramssz, void *settings);
c_lib = ct.CDLL(str(libname)) # on windows, CDLL(libname) failed to accept WindwowsPath.
c_lib.foo_func.argtypes = ct.c_char_p, ct.c_size_t, ct.c_size_t, ct.c_void_p, ct.c_size_t, ct.c_void_p
c_lib.foo_func.restype = None
func_param = virt_buff(7, 0)
with open("build/fib.bin", mode='rb') as file:
binary = file.read()
# Indent under with, so file.tell() is in scope,
# Pass address of structure using ct.byref() and size with ct.sizeof()
# Use Python None instead of C NULL.
c_lib.foo_func(binary, file.tell(), 0x9000 + (file.tell() & ~0xFFF), ct.byref(func_param), ct.sizeof(func_param), None)

Your parameter is almost certainly supposed to a POINTER to char, and not just a single char:
c_lib.foo_func(ctypes.c_char_p(binary), ...

ctypes dll declaration of variables

I'm trying to use following function with ctypes and I do have troubles how declare all the parameters and variables right.
The Documentation of the C code is following,
/* global variables */
int main ()
char sDeviceSerialNumber[32];
FEUSB_GetScanListPara( 0, "Device-ID", sDeviceSerialNumber ) ;
sDeviceSerialNumber is supposed to be a return value of the function which I need in Python for further use.
Python code:
def FEUSB_GetScanListPara(iIndex, cPara):
libfeusb.FEUSB_GetScanListPara.argtypes = [ctypes.c_int,
ctypes.c_wchar_p,
ctypes.POINTER(ctypes.c_char_p)
]
libfeusb.FEUSB_GetScanListPara.restype = ctypes.c_int
iIndex = ctypes.c_int(iIndex)
cValue_buffer = ctypes.create_string_buffer(32)
cValue = ctypes.c_char_p(ctypes.addressof(cValue_buffer))
value = libfeusb.FEUSB_GetScanListPara(iIndex,
cPara,
ctypes.byref(cValue)
)
if __name__ == "__main__":
i = 0
RFID.FEUSB_GetScanListPara(i, "Device-ID")
When I call the function with the code above, I get an error code, FEUSB_ERR_UNKNOWN_PARAMETER, therefore I assume that I do not declare the parameters correctly.
Any input is appreciated!
EDIT 1
def FEUSB_GetScanListPara(iIndex, cPara):
libfeusb.FEUSB_GetScanListPara.argtypes = [ctypes.c_int,
ctypes.c_char_p,
ctypes.c_char_p
]
libfeusb.FEUSB_GetScanListPara.restype = ctypes.c_int
cValue = ctypes.create_string_buffer(32)
value = libfeusb.FEUSB_GetScanListPara(iIndex, cPara,
ctypes.byref(cValue))
print("1.0", cPara, "back value", " = ", value)
print("1.1", cPara, " = ", cValue.value)
print("######")
if __name__ == "__main__":
data = RFID.FEUSB_GetScanListPara(i, b"Device-ID")
Python Console:
FEUSB_ClearScanList = 0
FEUSB_Scan = 0
FEUSB_GetScanListSize = 1
Traceback (most recent call last):
File "C:\xxxxx\3.1_ObidRFID_test\OBID_RFID_06.py", line 265, in <module>
data = RFID.FEUSB_GetScanListPara(i, b"Device-ID")
File "C:\xxxxx\3.1_ObidRFID_test\OBID_RFID_06.py", line 89, in FEUSB_GetScanListPara
value = libfeusb.FEUSB_GetScanListPara(iIndex, cPara, ctypes.byref(cValue))
ArgumentError: argument 3: <class 'TypeError'>: wrong type
EDIT 2
working code
def FEUSB_GetScanListPara(iIndex, cPara):
libfeusb.FEUSB_GetScanListPara.argtypes = [ctypes.c_int,
ctypes.c_char_p,
ctypes.c_char_p
]
libfeusb.FEUSB_GetScanListPara.restype = ctypes.c_int
cValue = ctypes.create_string_buffer(32)
return_value = libfeusb.FEUSB_GetScanListPara(0, b'Device-ID',
cValue)

Your declaration of .argtypes would match the C prototype of:
int FEUSB_GetScanListPara(int, wchar_t*, char**)
You haven't provided the exact C prototype, but from your example of:
FEUSB_GetScanListPara( 0, "Device-ID", sDeviceSerialNumber ) ;
and knowing wchar_t* is not a common interface parameter, you probably actually have simple char* declarations like:
int FEUSB_GetScanListPara(int, const char*, char*);
I'm assuming the 2nd parameter is an input parameter and 3rd parameter is an output parameter. Note that c_char_p corresponds to a byte string so use b'DeviceID' for cPara. Also if you have to allocate the buffer, the 3rd parameter is unlikely to be char**. If the API itself is not returning a pointer, but filling out an already allocated buffer, char* and hence ctypes.c_char_p is appropriate. You correctly use create_string_buffer() for an output parameter.
Note you don't need to wrap iIndex in a c_int. From .argtypes, ctypes knows the 1st parameter is a c_int and converts it for you. That's also the default if no .argtypes is provided, but better to be explicit and provide .argtypes.
This code should work. I don't have the DLL to verify:
import ctypes as ct
libfeusb = CDLL('./FESUB') # assuming in same directory
libfeusb.FEUSB_GetScanListPara.argtypes = ct.c_int, ct.c_char_p, ct.c_char_p
libfeusb.FEUSB_GetScanListPara.restype = ct.c_int
cValue = ct.create_string_buffer(32)
ret = libfeusb.FEUSB_GetScanListPara(0, b'Device-ID', cValue)
if ret == 0:
print(cValue.value)
else:
print('error:',ret)
If you still have issues, edit your question with a minimal, reproducible example. Make sure to provide the real C prototype.

Accessing a 16-bit pointer with ctypes

Why does the following not work?
import ctypes
class Test(ctypes.Structure):
_fields_ = [("my_pointer", ctypes.POINTER(ctypes.c_int16))]
t = Test()
t.my_pointer = ctypes.addressof(ctypes.c_int16(123))
Error:
TypeError: expected LP_c_short instance, got int
Is there way to generate a LP_c_short? Or any 16-bit pointer?
EDIT
Using byref instead of addressof causes:
TypeError: expected LP_c_short instance, got CArgObject
The Test structure is defined that way because it look's like that in a C-DLL.

Use byref or pointer not addressof
a = ctypes.c_int16(123)
t.my_pointer(ctypes.byref(a)
Despite having a name like the C & operator addressof returns a int and not a ctypes pointer.
ctypes.addressof(obj)
Returns the address of the memory buffer as integer. obj must be an instance of a ctypes type
EDIT
Is this not working ? :
import ctypes
class Test(ctypes.Structure):
_fields_ = [("my_pointer", ctypes.POINTER(ctypes.c_int16))]
t = Test()
cc = ctypes.c_int16(123)
t.my_pointer = ctypes.pointer(cc)
print t.my_pointer[0]

I'm not an expert with ctypes, but following the docs the following works for me:
import ctypes
PI16 = ctypes.POINTER(ctypes.c_int16)
class Test(ctypes.Structure):
_fields_ = [
("my_pointer", PI16),
]
t = Test()
t.my_pointer = PI16(ctypes.c_int16(123))
print(t.my_pointer[0])
otherwise, you could do:
t.my_pointer = ctypes.pointer(ctypes.c_int32(123))
the former will allow you to do strange things like:
PI(ctypes.c_int64(123))[0]
I.e. coerce between different width integer types, or worse…

Python and WinDLL calling doesn't return from the DLL call

My DLL has the following function:
extern "C" __declspec(dllexport) bool __cdecl PcapOpen(unsigned long inSize, void * inData, unsigned long *outSize, void * outData, Function_Modes eMode)
The function will write data out through outSize and outData (meaning these need to be passed to the function with the expectation that my function inside the DLL will modify them)
In addition, Function_Modes is:
typedef enum Function_Modes { Mode_Unknown = -1, Mode_Preview = 0, Mode_Execute = 1, Mode_Query } Function_Modes;
For reference in Perl I referenced the function like so:
my %Data;
$Data{Interface} = $interface;
my ($inData, $inSize) = buildData(\%Data);
$$outSize = pack("L", 65536);
$outData = "\x00"x65536;
my $pPcapOpen = Win32::API->new($dllName, "PcapOpen", "NPPPI", "I", "_cdecl");
my $return = $pPcapOpen->Call($inSize, $inData, $$outSize, $outData, $eMode);
I created a small Python code that should run the function and show the data it returns, but the call while "appears to access the DLL", does not return (the python code just after the call is never called)
pPcapDLL = ctypes.WinDLL("Pcap Interface.dll")
LP_c_char = ctypes.POINTER(ctypes.create_string_buffer(65536).__class__)
obuf = ctypes.create_string_buffer(65536)
obuflen = ctypes.c_ulong(65536)
optr = LP_c_char(obuf)
PcapOpen = pPcapDLL["PcapOpen"]
PcapOpen.restype = ctypes.c_bool
PcapOpen.argtypes = [ctypes.c_ulong, ctypes.c_void_p, ctypes.POINTER(LP_c_char), ctypes.POINTER(ctypes.c_size_t), ctypes.c_int]
outSize = ctypes.c_ulong(65536)
LP_c_char = ctypes.POINTER(ctypes.create_string_buffer(65536).__class__)
outData = ctypes.create_string_buffer(65536)
outData_ptr = LP_c_char(outData)
print("Calling PcapClose")
returnValue = PcapClose(inSize, inData, ctypes.byref(outSize), ctypes.byref(outData_ptr), eMode)
print("returnValue: {}".format(returnValue)) # <-- This doesn't get called
There appears to be very little documentation on how to handle DLL functions that return data - just that byref should be used, but its not 100% clear how to allocate the data for these function to use them - so that could be my issue
Thank you for assisting.

As I see it there are two changes you need to make:
The C function you are attempting to call uses the cdecl calling convention, so you must access the DLL using ctypes.cdll, not ctypes.windll. ctypes.windll is used for the stdcall calling convention. See also the Python ctypes documentation.
The fourth argument to your function is a pointer, but you appear to be passing it a pointer to a pointer. Try replacing ctypes.byref(outdata_ptr) with ctypes.byref(outdata).

Calling functions with arguments from CoreFoundation using ctypes

I am trying to access the CoreMidi functions from the CoreFoundation framework in MacOSX using ctypes in Python.
When I call a function that doesn't have any parameters, everything goes fine. For example, the following code:
from ctypes import *
core_midi = cdll.LoadLibrary("/System/Library/Frameworks/CoreMIDI.framework/Versions/A/CoreMIDI")
numOfDevices = core_midi.MIDIGetNumberOfDevices()
print numOfDevices
returns
3
which is the number of MIDI devices in my computer.
However, I am not able to execute functions which require parameters. Check this example (EDIT: as eryksun pointed out, I was using a char* as client_name, and the function prototype demanded a CFString. I corrected this in the code example below but I still get the same error):
core_midi = cdll.LoadLibrary("/System/Library/Frameworks/CoreMIDI.framework/Versions/A/CoreMIDI")
client_name = core_foundation.CFStringCreateWithCString(None, "MIDI Client", 0)
midi_client = c_uint()
result = core_midi.MIDIClientCreate(client_name, None, None, byref(midi_client))
print midi_client
print result
This code doesn't print anything at all, doesn't even raise an exception. The MIDIClientCreate function's prototype is:
extern OSStatus MIDIClientCreate(
CFStringRef name,
MIDINotifyProc notifyProc,
void *notifyRefCon,
MIDIClientRef *outClient );
MIDIClientRef is defined as a UInt32, and, as I understand, it receives a pointer to a MIDIClient structure that is created, that is why I use byref() to pass it as a parameter. If I just pass the variable without byref(), the function call returns a value -50, which probably indicates some bizarre error.
EDIT: I am not sure if I am creating the CFString correctly. I tried to test the result with the following code but it doesn't print anything on screen.
client_name = core_foundation.CFStringCreateWithCString(None, "MIDI Client", 0)
cstr = ctypes.create_string_buffer(20)
core_foundation.CFStringGetCString(client_name, cstr, 20, 0)
print cstr
Thanks!
EDIT: Answered by eryksun!
I didn't know this of course, but setting the pointers is not as obvious as my naive example was trying to do.
class _CFString(Structure):
pass
cf_path = ctypes.util.find_library("CoreFoundation")
cm_path = ctypes.util.find_library("CoreMIDI")
core_foundation = ctypes.cdll.LoadLibrary(cf_path)
core_midi = ctypes.cdll.LoadLibrary(cm_path)
CFStringRef = POINTER(_CFString)
midi_client = ctypes.c_uint()
core_foundation.CFStringCreateWithCString.restype = CFStringRef
core_foundation.CFStringCreateWithCString.argtypes = [c_void_p, c_char_p, c_uint32]
client_name = core_foundation.CFStringCreateWithCString(None, "MIDI Client", 0)
core_midi.MIDIClientCreate.argtypes = [CFStringRef, c_void_p, c_void_p, POINTER(c_uint32)]
result = core_midi.MIDIClientCreate(client_name, None, None, byref(midi_client))
print midi_client
print result
Actually, I though restype and argtypes didn't affect how functions were executed or how parameters were passed to them, but it seems that they do.
The above code results in:
c_uint(4657178L)
0
That is, my MIDI client is created somewhere and the function returns without error.
Thanks again eryksun!