We Keep Coding

passing null structure pointer ctypes

I am writing a Python wrapper for cpp APIs in that for one API I am trying to pass a NULL structure pointer as a parameter. Not sure how we can achieve that in Python.
Below is my sample implementation:
cpp_header.hpp
typedef enum {
E_FLAG_ON = 0,
E_FLAG_OFF
} option;
typedef struct {
float *a;
float b;
char *file_path;
option flag;
} inputs;
// API
int op_init(const inputs*);
This is what happening inside the init API:
Implementation.cpp
int op_init(const inputs* usr_ptr) {
internal_opration_read_set(&local_struct) { // local struct variable
// read one input file and update the structure values
}
if (usr_prt != NULL) {
internal_opration_update_set(usr_ptr, &local_struct) {
// update the values only if i send NOT NULL structure
}
}
}
From cpp test application I'm passing NULL structure to initialize.
test.cpp
int main() {
inputs *usr_cfg = NULL;
op_init(usr_cfg);
}
ctypes_imple.py
From ctypes import *
class inputs(Structure):
_fields_ = [('a', POINTER(c_float)),
('b', c_float),
('file_path', c_char_p),
('flag', option)]
# loading so file
so_lib = CDLL('some so')
# how we can initialize NULL structure pointer here?
so_lib.op_init() # how to send structure pointer as argument?
NOTE: this API reads inputs from a file and updates values to a structure in C. I am clueless how we can achieve the same in a Python wrapper? I mean updating values from so file runtime to a ctypes Python class.

Use None to pass a null pointer:
so_lib.op_init(None)
To send the actual structure instantiate one and send it. Best to define .argtypes and restype as well so ctypes doesn't have to guess and can perform better error checking:
so_lib.op_init.argtypes = POINTER(inputs),
so_lib.op_init.restype = c_int
arg = inputs() # all initialized to zero/null by default.
so_lib.op_init(arg)

Python ctypes writing data to be read by C executable

I'm trying to learn how to use the Python ctypes library to write data to a file that can easily be read by C executables. In the little test case that I've put together, I'm running into some problems with reading/writing character arrays.
At the moment, I have three source files. write_struct.py creates a simple struct with two
entries, an integer value called git and a character array called command, then writes the struct to a file using ctypes.fwrite. read_struct.c and read_struct.h compile into an executable that internally defines an identical struct to the one in write_struct.py, then reads in the data written by the python script and prints it out.
At the moment, the following values are assigned in the python file (not literally in the manner shown below, scroll down to see the actual code):
git = 1
command = 'cp file1 file2'
And when run, the C executable prints the following:
git: 1
command:
I realize that the problem is almost certainly in how the command variable is being assigned in the python script. I have read that c_char_p() (the function I'm currently using to initialize the data in that variable) does not create a pointer to mutable memory, and create_string_buffer() should be used instead, however I'm not sure about how this works with either adding that data to a struct, or writing it to a file. I guess I'm also confused about how writing pointers/their data to a file works in the first place. What is the best way to go about doing this?
Thanks in advance to anyone that is able to help!!
The code of my three files is below for reference:
write_struct.py:
"""
write_struct.py
"""
from ctypes import *
libc = cdll.LoadLibrary("libc.so.6")
class DataStruct(Structure):
_fields_ = [("git", c_int),
("command", c_char_p)
]
def main():
pydata = DataStruct(1, c_char_p("cp file1 file2"))
libc.fopen.argtypes = c_char_p, c_char_p
libc.fopen.restype = c_void_p
libc.fwrite = libc.fwrite
libc.fwrite.argtypes = c_void_p, c_size_t, c_size_t, c_void_p
libc.fwrite.restype = c_size_t
libc.fclose = libc.fclose
libc.fclose.argtypes = c_void_p,
libc.fclose.restype = c_int
f = libc.fopen("stored_data", "wb")
libc.fwrite(byref(pydata), sizeof(pydata), 1, f)
libc.fclose(f)
return 0
main()
read_struct.c:
/*
* read_struct.c
*
*/
#include "read_struct.h"
int main()
{
data_struct cdata = malloc(DATASIZE);
FILE *fp;
if ((fp = fopen("stored_data", "r")) != NULL) {
fread(cdata, DATASIZE, 1, fp);
printf("git: %i\n", cdata->git);
printf("command:");
printf("%s\n", cdata->command);
fclose(fp);
} else {
printf("Could not open file\n");
exit(1);
}
return 0;
}
read_struct.h:
/*
* read_struct.h
*
*/
#include <stdio.h>
#include <stdlib.h>
typedef struct _data_struct *data_struct;
struct _data_struct {
int git;
char command[40];
};
#define DATASIZE sizeof(struct _data_struct)

You can write binary data directly with Python. ctypes can be used to create the structure and supports bit fields and unions, or for simple structures the struct module can be used.
from ctypes import *
class DataStruct(Structure):
_fields_ = [("git", c_int),
("command", c_char * 40)] # You want array here, not pointer
pydata = DataStruct(1,b'cp file1 file2') # byte string for initialization.
with open('stored_data','wb') as f: # write file in binary mode
f.write(pydata) # ctypes support conversion to bytes
import struct
# See struct docs for formatting codes
# i = int (native-endian. Use <i to force little-endian, >i for big-endian)
# 40s = char[40] (zero-padded if initializer is shorter)
pydata = struct.pack('i40s',1,b'cp file1 file2')
with open('stored_data2','wb') as f:
f.write(pydata)
Ref: https://docs.python.org/3/library/struct.html#format-strings

Creating a python callback for a C function from a DLL with a char buffer.

I am trying to create a python wrapper for a C DLL. The C DLL uses a callbacks to "send" messages via UDP from the DLL.
What I expected to happen was that python code would load the DLL library and the two functions. Then it would register the call back with the DLL using the RegisterCallbackSendMessage function. The pointer to the callback would be stored in memory in the DLL memory space. Then the python code would call the SendWhoIs DLL function. This function would create a buffer and use the SendMessage function pointer to send the message from the DLL to the python code.
The issue I am having is that the message parameter in the SendMessage function in python doesn't know how to interpreter a void * and I can not pass this buffer to sock.sendto function.
I get the following error.
sock.sendto(message, (UDP_IP, UDP_PORT))
TypeError: a bytes-like object is required, not 'int'
My question is: How do I convert a c_void_p to a byte array that sock.sendto can accept.?
I have tried to reduce my code as much as possible and still make it understandable.
This is my C code
// C DLL code
//
#define DllExport extern "C" __declspec( dllexport )
typedef uint16_t(*FPCallbackSendMessage)(const uint8_t * message, const uint32_t length);
FPCallbackSendMessage g_CallbackSendMessage;
DllExport bool RegisterCallbackSendMessage(uint16_t(*p_CallbackSendMessage)(const uint8_t * message, const uint32_t length)) {
g_CallbackSendMessage = p_CallbackSendMessage ;
return true;
}
void SendWhoIs(unsigned int dd) {
printf("dd=%d", dd);
char buffer[500];
g_CallbackSendMessage( buffer, 500 );
}
And this is my python code
# Python code
# =================
print("Start")
customDLL = cdll.LoadLibrary ("customeDLL.dll")
def SendMessage( message, length ):
print("SendMessage...")
UDP_IP = "192.168.1.1"
UDP_PORT = 47808
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(message, (UDP_IP, UDP_PORT))
return True
print("Registering Callback SendMessage...")
SendMessageFUNC = CFUNCTYPE(c_bool, c_void_p, c_uint)
SendMessage_func = SendMessageFUNC( SendMessage )
RegisterCallbackSendMessage = customDLL.RegisterCallbackSendMessage
RegisterCallbackSendMessage.argtypes = [ SendMessageFUNC ]
RegisterCallbackSendMessage( SendMessage_func )
print("Do message...")
SendWhoIs = BACnetStackDLL.SendWhoIs
SendWhoIs.argtypes = [c_uint]
SendWhoIs( 47 )
print("End")

I figured it out.
in the python SendMessage function I figured out a way to convert the c_void_p to a byte array.
# convert the pointer to a buffer.
buffer = c_char * length
messageToSend = buffer.from_address(message)

How to access the value of a ctypes.LP_c_char pointer?

I have defined a struct :
class FILE_HANDLE(Structure):
_fields_ = [
("handle_bytes", c_uint),
("handle_type", c_int),
("f_handle", POINTER(c_char))
]
The struct is initialised :
buf = create_string_buffer(f_handle.handle_bytes)
fh = FILE_HANDLE(c_uint(8), c_int(0), buf)
I am passing it by reference to a function that populates it.
ret = libc.name_to_handle_at(dirfd, pathname, byref(fh), byref(mount_id), flags)
I can check with strace that the call works, but I have not been able to figure out how to access the value of fh.f_handle
fh.f_handle type is <ctypes.LP_c_char object at 0x7f1a7ca17560>
fh.f_handle.contents type is <ctypes.LP_c_char object at 0x7f1a7ca17560> but I get a SIGSEGV if I try to access its value.
How could I get 8 bytes from f_handle into a string or array ?

Everything actually looks right for what you've shown, but without seeing the explicit C definition of the structure and function you are calling it is difficult to see the problem.
Here's an example that works with what you have shown. I inferred what the C definitions should be from what you have declared in Python, but most likely your definition is different if you get a segfault.
C Code (Windows)
struct FILE_HANDLE
{
unsigned int handle_bytes;
int handle_type;
char* f_handle;
};
__declspec(dllexport) int name_to_handle_at(int dirfd, char* pathname, struct FILE_HANDLE* fh, int* mount_id, int flags)
{
unsigned int i;
printf("dirfd=%d pathname=%s fh->handle_bytes=%u fh->handle_type=%d flags=%d\n", dirfd, pathname, fh->handle_bytes, fh->handle_type, flags);
for(i = 0; i < fh->handle_bytes; ++i)
fh->f_handle[i] = 'A' + i;
*mount_id = 123;
return 1;
}
Python code (Works in Python 2 and 3):
from __future__ import print_function
from ctypes import *
class FILE_HANDLE(Structure):
_fields_ = [("handle_bytes", c_uint),
("handle_type", c_int),
("f_handle", POINTER(c_char))]
buf = create_string_buffer(8);
fh = FILE_HANDLE(8,0,buf)
libc = CDLL('test.dll')
mount_id = c_int(0)
ret = libc.name_to_handle_at(1,b'abc',byref(fh),byref(mount_id),7)
print('mount_id =',mount_id.value)
print('fh.f_handle =',fh.f_handle[:fh.handle_bytes])
Output
dirfd=1 pathname=abc fh->handle_bytes=8 fh->handle_type=0 flags=7
mount_id = 123
fh.f_handle = b'ABCDEFGH'
Note that since the structure is declared as a pointer to a single character, printing fh.f_handle.contents would only print b'A'. Using slicing, I've instructed Python to index the pointer up to the length allocated.
If this doesn't work for you, provide a Minimal, Complete, and Verifiable example (as I have) to reproduce your error exactly.

fh.f_handle is shown as LP_c_char because you defined the struct that way.
buf = create_string_buffer(8)
print type(buf)
fh = FILE_HANDLE(c_uint(8), c_int(0), buf)
print type(fh.f_handle)
Will output
<class 'ctypes.c_char_Array_8'>
<class 'ctypes.LP_c_char'>
You have defined your struct to accept a pointer to a c_char. So when you try to access fh.f_handle it will expect the value to be a memory address containing the address to the actual single c_char.
But by trying to input a c_char * 8 from the string buffer it will convert the first part of your buffer to a pointer.
Python tries to dereference your char[0] which means that it will look for a memory address with the value of the character you have defined in char[0]. That memory address is not valid, so your interpreter will signal a SIGSEGV.
Now to create a class which properly handles a variable length buffer is quite difficult. An easier option is to pass the buffer as an opaque handle, to access it afterwards you need to cast it back to a char array.
Example:
class FILE_HANDLE(Structure):
_fields_ = [
("handle_bytes", c_uint),
("handle_type", c_int),
("f_handle", c_void_p)
]
buf = create_string_buffer(8)
buf = cast(buf, c_void_p)
fh = FILE_HANDLE(c_uint(8), c_int(0), buf)
f_handle_value = (c_char * fh.handle_bytes).from_address(fh.f_handle)

Python & Ctypes: Passing a struct to a function as a pointer to get back data

I've looked through other answers but can't seem to get this to work. I'm trying to call a function within a DLL for communicating with SMBus devices. This function takes a pointer to a struct, which has an array as one of it's fields. so...
In C:
typedef struct _SMB_REQUEST
{
unsigned char Address;
unsigned char Command;
unsigned char BlockLength;
unsigned char Data[SMB_MAX_DATA_SIZE];
} SMB_REQUEST;
I think I have to set values for the Address, Command and BlockLength while the DLL fills the Data array.
The function that requires this struct takes it as a pointer
SMBUS_API int SmBusReadByte( SMBUS_HANDLE handle, SMB_REQUEST *request );
So I've set up the struct in Python like so:
class SMB_REQUEST(ctypes.Structure):
_fields_ = [("Address", c_char),
("Command", c_char),
("BlockLength", c_char),
("Data", type(create_string_buffer(SMB_MAX_DATA_SIZE))]
*Note: I've also tried ctypes.c_char*SMB_MAX_DATA_SIZE for the data type*
To pass a pointer to a struct of this type to the function I have tried to initialise it first as follows:
data = create_string_buffer(SMB_MAX_DATA_SIZE)
smb_request = SMB_REQUEST('\x53', \x00', 1, data)
This responds with:
TypeError: expected string or Unicode object, c_char_Array_32 found
If I try leaving out the data array, like so:
smb_request = SMB_REQUEST('\x53', \x00', 1)
No, error.
However, then when I try to pass this to the function:
int_response = smbus_read_byte(smbus_handle, smb_request))
I get:
ArgumentError: argument 2: <type 'exceptions.TypeError'>: expected LP_SMB_REQUES
T instance instead of SMB_REQUEST
I've tried passing it as a pointer:
int_response = smbus_read_byte(smbus_handle, ctypes.POINTER(smb_request))
and I get:
----> 1
2
3
4
5
TypeError: must be a ctypes type
Here's how I've set up the art types:
smbus_read_byte.argtypes = (ctypes.c_void_p, ctypes.POINTER(SMB_REQUEST))
I've tried casting but still no go. Can anyone shed some light on this for me?
Update:
If I first initialise the struct like so:
smb_request = SMB_REQUEST('\xA6', '\x00', chr(1), 'a test string')
and then bass by reference:
int_response = smbus_receive_byte(smbus_handle, ctypes.byref(smb_request))
I get no error. However, the function returns -1 when it should return '0' for success and non-zero for a fail. Checking the value of smb_request.Data gives back 'a test string' so no change there.
Any suggestions as to what might be going on here would be greatly appreciated.
Thanks
UPDATE:
Since I've gotten a couple of enquiries about whether my handle is correct, here's how I'm using it. The header file for the DLL declares the following:
typedef void *SMBUS_HANDLE;
//
// This function call initializes the SMBus, opens the driver and
// allocates the resources associated with the SMBus.
// All SMBus API calls are valid
// after making this call except to re-open the SMBus.
//
SMBUS_API SMBUS_HANDLE OpenSmbus(void);
So here's how I'm doing this in python:
smbus_handle = c_void_p() # NOTE: I have also tried it without this line but same result
open_smbus = CDLL('smbus.dll').OpenSmbus
smbus_handle = open_smbus()
print 'SMBUS_API SMBUS_HANDLE OpenSmbus(void): ' + str(smbus_handle)
I call this before making the call to smbus_read_byte(). I have tried to set open_smbus.restype = c_void_p() but I get an error: TypeError: restype must be a type, a callable, or None

Here's a working example. It looks like you are passing the wrong type to the function.
Test DLL Code ("cl /W4 /LD x.c" on Windows)
#include <stdio.h>
#define SMBUS_API __declspec(dllexport)
#define SMB_MAX_DATA_SIZE 5
typedef void* SMBUS_HANDLE;
typedef struct _SMB_REQUEST
{
unsigned char Address;
unsigned char Command;
unsigned char BlockLength;
unsigned char Data[SMB_MAX_DATA_SIZE];
} SMB_REQUEST;
SMBUS_API int SmBusReadByte(SMBUS_HANDLE handle,SMB_REQUEST *request)
{
unsigned char i;
for(i = 0; i < request->BlockLength; i++)
request->Data[i] = i;
return request->BlockLength;
}
SMBUS_API SMBUS_HANDLE OpenSmbus(void)
{
return (void*)0x12345678;
}
Python code
from ctypes import *
SMB_MAX_DATA_SIZE = 5
ARRAY5 = c_ubyte * SMB_MAX_DATA_SIZE
class SMB_REQUEST(Structure):
_fields_ = [
("Address", c_ubyte),
("Command", c_ubyte),
("BlockLength", c_ubyte),
("Data", ARRAY5)]
smbus_read_byte = CDLL('x').SmBusReadByte
smbus_read_byte.argtypes = [c_void_p,POINTER(SMB_REQUEST)]
smbus_read_byte.restype = c_int
open_smbus = CDLL('x').OpenSmbus
open_smbus.argtypes = []
open_smbus.restype = c_void_p
handle = open_smbus()
print 'handle = %08Xh' % handle
smb_request = SMB_REQUEST(1,2,5)
print 'returned =',smbus_read_byte(handle,byref(smb_request))
print 'Address =',smb_request.Address
print 'Command =',smb_request.Command
print 'BlockLength =',smb_request.BlockLength
for i,b in enumerate(smb_request.Data):
print 'Data[%d] = %02Xh' % (i,b)
Output
handle = 12345678h
returned = 5
Address = 1
Command = 2
BlockLength = 5
Data[0] = 00h
Data[1] = 01h
Data[2] = 02h
Data[3] = 03h
Data[4] = 04h

You're almost there. You should use c_char * SMB_MAX_DATA_SIZE as the type for the definition of Data. This works for me on Mac OS X:
Shared library:
$ cat test.c
#include <stdio.h>
#define SMB_MAX_DATA_SIZE 16
typedef struct _SMB_REQUEST
{
unsigned char Address;
unsigned char Command;
unsigned char BlockLength;
unsigned char Data[SMB_MAX_DATA_SIZE];
} SMB_REQUEST;
int SmBusReadByte(void *handle, SMB_REQUEST *request)
{
printf("SmBusReadByte: handle=%p request=[%d %d %d %s]\n", handle,
request->Address, request->Command, request->BlockLength, request->Data);
return 13;
}
$ gcc test.c -fPIC -shared -o libtest.dylib
Python driver:
$ cat test.py
import ctypes
SMB_MAX_DATA_SIZE = 16
class SMB_REQUEST(ctypes.Structure):
_fields_ = [("Address", ctypes.c_ubyte),
("Command", ctypes.c_ubyte),
("BlockLength", ctypes.c_ubyte),
("Data", ctypes.c_char * SMB_MAX_DATA_SIZE)]
libtest = ctypes.cdll.LoadLibrary('libtest.dylib')
req = SMB_REQUEST(1, 2, 3, 'test')
result = libtest.SmBusReadByte(ctypes.c_voidp(0x12345678), ctypes.byref(req))
print 'result: %d' % result
$ python test.py
SmBusReadByte: handle=0x12345678 request=[1 2 3 test]
result: 13
UPDATE
You're having problems because you need to set the result type of open_smbus to void*. By default, ctypes assumes that functions return ints. You need to say this:
open_smbus.restype = ctypes.c_void_p
You were getting an error because you were using c_void_p() (note the extra parentheses). There's an important distinction between c_void_p and c_void_p(). The former is a type, and the latter is an instance of a type. c_void_p represents the C type void*, whereas c_void_p() represents an actual pointer instance (with a default value of 0).

Try changing
("Data", type(create_string_buffer(SMB_MAX_DATA_SIZE))
to
("Data", (c_char * SMB_MAX_DATA_SIZE)]