I have a struct:
struct some_struct_s {
int arg1;
int arg2;
};
I have a C function:
int func(some_struct_s *output);
Both are %included into my SWIG file.
I want some_struct_s *output to be treated like an output parameter. Python example:
int_val, some_struct_output = func()
"Output parameters" is covered in the manual for POD-types (sec 10.1.3), but not for non-POD types.
How do I tell SWIG I want some_struct_s *output to be an output parameter?
From the documentation:
11.5.7 "argout" typemap
The "argout" typemap is used to return values from arguments. This is most commonly used to write wrappers for C/C++ functions that need to return multiple values. The "argout" typemap is almost always combined with an "in" typemap---possibly to ignore the input value....
Here's a complete example for your code (no error checking for brevity):
%module test
// Declare an input typemap that suppresses requiring any input and
// declare a temporary stack variable to hold the return data.
%typemap(in,numinputs=0) some_struct_s* (some_struct_s tmp) %{
$1 = &tmp;
%}
// Declare an output argument typemap. In this case, we'll use
// a tuple to hold the structure data (no error checking).
%typemap(argout) some_struct_s* (PyObject* o) %{
o = PyTuple_New(2);
PyTuple_SET_ITEM(o,0,PyLong_FromLong($1->arg1));
PyTuple_SET_ITEM(o,1,PyLong_FromLong($1->arg2));
$result = SWIG_Python_AppendOutput($result,o);
%}
// Instead of a header file, we'll just declare this code inline.
// This includes the code in the wrapper, as well as telling SWIG
// to create wrappers in the target language.
%inline %{
struct some_struct_s {
int arg1;
int arg2;
};
int func(some_struct_s *output) {
output->arg1 = 1;
output->arg2 = 2;
return 0;
}
%}
Demo below. Note that the int return value of zero as well as the output parameter as a tuple are returned as a list.
>>> import test
>>> test.func()
[0, (1, 2)]
If you don't want typemaps, you can also inject code to create the object and return it to hide it from the user:
%module test
%rename(_func) func; // Give the wrapper a different name
%inline %{
struct some_struct_s {
int arg1;
int arg2;
};
int func(struct some_struct_s *output)
{
output->arg1 = 1;
output->arg2 = 2;
return 0;
}
%}
// Declare your interface
%pythoncode %{
def func():
s = some_struct_s()
r = _func(s)
return r, s
%}
Demo:
>>> import test
>>> r, s = test.func()
>>> r
0
>>> s
<test.some_struct_s; proxy of <Swig Object of type 'some_struct_s *' at 0x000001511D70A880> >
>>> s.arg1
1
>>> s.arg2
2
You can make the typemap language agnostic if you carefully select SWIG macros:
%module test
%typemap(in,numinputs=0) struct some_struct_s *output %{
$1 = malloc(sizeof(struct some_struct_s));
%}
%typemap(argout) struct some_struct_s* output {
// The last parameter passes ownership of the pointer
// to Python so it will be freed when the object's
// reference count goes to zero.
%append_output(SWIG_NewPointerObj($1, $1_descriptor, SWIG_POINTER_OWN));
}
%inline %{
struct some_struct_s {
int arg1;
int arg2;
};
int func(struct some_struct_s *output)
{
output->arg1 = 1;
output->arg2 = 2;
return 0;
}
%}
Demo:
>>> import test
>>> r, s = test.func()
>>> r
0
>>> s
<test.some_struct_s; proxy of <Swig Object of type 'some_struct_s *' at 0x000001DD0425A700> >
>>> s.arg1
1
>>> s.arg2
2
Related
I have a list of strings in python which I'm trying to pass down to a C extension for character analysis. I've gotten so far as to have the list broken up into their individual string PyObjects. Next, I'm hoping to split these strings into their individual characters so that every string PyObject is now a corresponding C-type character array. I can't seem to figure out how to do this though.
Here's what I have so far: Currently after building the .pyd file it will return a list of 1's as a filler to Python (so everything else works), I just don't know how to split a string PyObject into the C-type character array.
--- cExt.c ---
#include <Python.h>
#include <stdio.h>
static int *CitemCheck(PyObject *commandString, int commandStringLength) {
// HAALP
//char* commandChars = (char*) malloc(commandStringLength*sizeof(char*));
// char c[] = PyString_AsString("c", commandString);
// printf("%c" , c);
// printf("%s", PyString_AsString(commandString));
// for (int i=0; i<sizeof(commandChars)/sizeof(*commandChars); i++) {
// printf("%s", PyString_AsString(commandString));
// printf("%c", commandChars[i]);
// }
return 1; // TODO: RETURN PROPER RESULTANT
}
static PyObject *ClistCheck(PyObject *commandList, int commandListLength) {
PyObject *results = PyList_New(commandListLength);
for (int index = 0; index < commandListLength; index++) {
PyObject *commandString;
commandString = PyList_GetItem(commandList, index);
int commandStringLength = PyObject_Length(commandString);
// CitemCheck should take string PyObject and its length as int
int x = CitemCheck(commandString, commandStringLength);
PyObject* pyItem = Py_BuildValue("i", x);
PyList_SetItem(results, index, pyItem);
}
return results;
}
static PyObject *parseListCheck(PyObject *self, PyObject *args) {
PyObject *commandList;
int commandListLength;
if (!PyArg_ParseTuple(args, "O", &commandList)){
return NULL;
}
commandListLength = PyObject_Length(commandList);
return Py_BuildValue("O", ClistCheck(commandList, commandListLength));
}
static char listCheckDocs[] =
""; // TODO: ADD DOCSTRING
static PyMethodDef listCheck[] = {
{"listCheck", (PyCFunction) parseListCheck, METH_VARARGS, listCheckDocs},
{NULL,NULL,0,NULL}
};
static struct PyModuleDef DCE = {
PyModuleDef_HEAD_INIT,
"listCheck",
NULL,
-1,
listCheck
};
PyMODINIT_FUNC PyInit_cExt(void){
return PyModule_Create(&DCE);
}
for reference, my temporary extension build file:
--- _c_setup.py ---
(located in same folder as cExt.c)
"""
to build C files, pass:
python _c_setup.py build_ext --inplace clean --all
in command prompt which is cd'd to the file's dierctory
"""
import glob
from setuptools import setup, Extension, find_packages
from os import path
here = path.abspath(path.dirname(__file__))
files = [path.split(x)[1] for x in glob.glob(path.join(here, '**.c'))]
extensions = [Extension(
path.splitext(x)[0], [x]
) for x in files]
setup(
ext_modules = extensions,
)
You can use PyUnicode_AsEncodedString, which
Encode a Unicode object and return the result as Python bytes object. encoding and errors have the same meaning as the parameters of the same name in the Unicode encode() method. The codec to be used is looked up using the Python codec registry. Return NULL if an exception was raised by the codec.
see https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_AsEncodedString
Then with PyBytes_AsString you get a pointer to internal buffer with a terminating NUL byte. This buffer must neither be deallocated nor modified. If you need a copy you could use e.g. strdup.
see https://docs.python.org/3/c-api/bytes.html#c.PyBytes_AsString
Slightly modifying your code it could look like this:
PyObject *encodedString = PyUnicode_AsEncodedString(commandString, "UTF-8", "strict");
if (encodedString) { //returns NULL if an exception was raised
char *commandChars = PyBytes_AsString(encodedString); //pointer refers to the internal buffer of encodedString
if(commandChars) {
printf("the string '%s' consists of the following chars:\n", commandChars);
for (int i = 0; commandChars[i] != '\0'; i++) {
printf("%c ", commandChars[i]);
}
printf("\n");
}
Py_DECREF(encodedString);
}
If one would test with:
import cExt
fruits = ["apple", "pears", "cherry", "pear", "blueberry", "strawberry"]
res = cExt.listCheck(fruits)
print(res)
The output would be:
the string 'apple' consists of the following chars:
a p p l e
the string 'pears' consists of the following chars:
p e a r s
the string 'cherry' consists of the following chars:
c h e r r y
the string 'pear' consists of the following chars:
p e a r
the string 'blueberry' consists of the following chars:
b l u e b e r r y
the string 'strawberry' consists of the following chars:
s t r a w b e r r y
[1, 1, 1, 1, 1, 1]
Side note not directly related to the question:
Your CitemCheck function returns a pointer to int, but if looking at how it is called, it seems that you want to return an int value. The function signature should look more like this:
static int CitemCheck(PyObject *commandString, int commandStringLength)
(note the removed * after int).
I am trying to use ctypes to extract a structure initialized by a C library (see for example: https://tentacles666.wordpress.com/2012/01/21/python-ctypes-dereferencing-a-pointer-to-a-c).
The 'prototype' is:
mytype * createMyType();
The structure in C is:
typedef struct
{
int a;
void *b;
}mytype;
from which in python(3)! I have created a ctypes structure thus:
class mytype(ctypes.Structure):
_fields_ = [("a", ctypes.c_int),
("b", ctypes.POINTER(None))]
The C call is:
mytype*myinstance = createMyType()
The Python call is as follows:
import ctypes
f=mylib.createMyType
f.argtypes=()
f.restype=(ctypes.POINTER(mytype),)
x=f()
The problem is that x seems to be an integer; how do I interpret this as a pointer, or - as required - extract the members of x themselves?
How do I access and then modify x.a and x.b?
[See also Accessing data from a structure returned by C function in Python using ctypes, which led nowhere]
Mainly you need c_void_p for the void* and must dereference the return with .contents.
Here's a working example (Windows)...
Edit: I added an example of casting the void pointer member.
test.h
#ifdef EXPORT
#define API __declspec(dllexport)
#else
#define API __declspec(dllimport)
#endif
typedef struct
{
int a;
void* b;
} mytype;
API mytype* createMyType();
API void destroyMyType(mytype* p);
test.c
#define EXPORT
#include <stdlib.h>
#include <stdio.h>
#include "test.h"
API mytype* createMyType()
{
int* tmp;
mytype* p = malloc(sizeof(mytype));
p->a = 5;
tmp = malloc(sizeof(int));
*tmp = 123;
p->b = tmp;
printf("%d %p\n",p->a,p->b);
return p;
}
API void destroyMyType(mytype* p)
{
free(p->b);
free(p);
}
test.py
from ctypes import *
class mytype(Structure):
_fields_ = [('a',c_int),
('b',c_void_p)]
test = CDLL('test')
createMyType = test.createMyType
createMyType.argtypes = None
createMyType.restype = POINTER(mytype)
destroyMyType = test.destroyMyType
destroyMyType.argtypes = POINTER(mytype),
destroyMyType.restype = None
t = createMyType()
print('t is',t)
print('t.a is',t.contents.a)
print('t.b is',hex(t.contents.b))
b = cast(t.contents.b,POINTER(c_int))
print('*b is',b.contents)
destroyMyType(t)
Output: Note that the void* b address output in the C code matches the integer returned by t.contents.b. The cast turns that integer into a POINTER(c_int) where the contents can be extracted.
5 00000216C0E2A5D0
t is <__main__.LP_mytype object at 0x00000216C30C4A48>
t.a is 5
t.b is 0x216c0e2a5d0
*b is c_long(123)
I have a C++ myObject class that I expose via boost python using a wrapper structure:
struct myObjectWrapper{
static tuple compute(myObject& o,const Container& x0, const double& t0, Container& x){
double t;
int stat = o.evaluate(x0,t0,x,t);
return make_tuple(stat,t);
}
}
BOOST_PYTHON_MODULE(myModule)
{
// not shown here is code to expose Container class
class_<myObject>("MyObject")
.def("compute",&myObjectWrapper::compute)
;
}
Container is currently defined as:
typedef std::valarray<double> Container
and is exposed to python.
Now in python I can do.
x = Container()
(status,t) = obj.compute(Container([0.,0.,0.]),0.0,x)
print status, t, x[0]
This is not very pythonic. I would prefer to do:
(status,t,x) = obj.compute(Container([0.,0.,0.]),0.0)
print status, t, x[0]
I could write an additional wrapper in python, but I would prefer to avoid adding more wrappers.
The following code does't compile:
struct myObjectWrapper{
static tuple compute(myObject& o,const Container& x0, const double& t0){
double t;
Container x;
int stat = o.evaluate(x0,t0,x,t);
return make_tuple(stat,t,x);
}
}
Also I would prefer to steal the content of the local variable x and have python manage it rather than copy it:
return make_tuple(stat,t,std::move(x));
How do I achieve this?
In short, allocate the wrapper on the free store and use the manage_new_object result convert to transfer ownership to a Python object. This will cause Boost.Python to copy the pointer when constructing the Python object, rather than copying the pointee. For more details, see this answer.
Here is an auxiliary function that will transfer ownership to a Python object:
/// #brief Transfer ownership to a Python object. If the transfer fails,
/// then object will be destroyed and an exception is thrown.
template <typename T>
boost::python::object transfer_to_python(T* t)
{
// Transfer ownership to a smart pointer, allowing for proper cleanup
// incase Boost.Python throws.
std::unique_ptr<T> ptr(t);
// Use the manage_new_object generator to transfer ownership to Python.
namespace python = boost::python;
typename python::manage_new_object::apply<T*>::type converter;
// Transfer ownership to the Python handler and release ownership
// from C++.
python::handle<> handle(converter(*ptr));
ptr.release();
return python::object(handle);
}
And one could use it as follows:
boost::python::tuple myObjectWrapper::compute(
myObject& o, const Container& x0, const double& t0)
{
auto x1 = std::make_unique<container>();
double t1 = 0;
int stat = self.evaluate(x0, t0, *x1, t1);
return boost::python::make_tuple(stat, t1, transfer_to_python(x1.release()));
}
Here is a complete example based on the original question that demonstrates using the transfer_to_python auxiliary function.
#include <boost/python.hpp>
#include <cassert>
#include <memory> // std::unique_ptr
// Mock legacy API.
struct container
{
container() {}
container(boost::python::object) {}
container(const container&)
{
// For this example, guarantee copy is not made.
assert(false);
}
};
struct my_object
{
int evaluate(container&, double&, container&, double&) { return 42; }
};
/// #brief Transfer ownership to a Python object. If the transfer fails,
/// then object will be destroyed and an exception is thrown.
template <typename T>
boost::python::object transfer_to_python(T* t)
{
// Transfer ownership to a smart pointer, allowing for proper cleanup
// incase Boost.Python throws.
std::unique_ptr<T> ptr(t);
// Use the manage_new_object generator to transfer ownership to Python.
namespace python = boost::python;
typename python::manage_new_object::apply<T*>::type converter;
// Transfer ownership to the Python handler and release ownership
// from C++.
python::handle<> handle(converter(*ptr));
ptr.release();
return python::object(handle);
}
// API wrapper.
boost::python::tuple my_object_compute(
my_object& self, container& x0, double t0)
{
auto x1 = std::make_unique<container>();
double t1 = 21;
int stat = self.evaluate(x0, t0, *x1, t1);
return boost::python::make_tuple(stat, t1, transfer_to_python(x1.release()));
}
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<container>("Container")
.def(python::init<python::object>())
;
python::class_<my_object>("MyObject")
.def("compute", &my_object_compute)
;
}
Interactive usage:
>>> import example
>>> my_object = example.MyObject()
>>> status, t, x = my_object.compute(example.Container([1, 2, 3]), 4)
>>> assert(status == 42)
>>> assert(t == 21)
>>> assert(isinstance(x, example.Container))
I'm extending my library with Python (2.7) by wrapping interfaces with SWIG 2.0, and have a graph object in which I want to create a visitor. In C++, the interface looks like this:
struct Visitor
{
virtual void OnStateBegin() = 0;
virtual void OnNode(Node* n) = 0;
virtual void OnStateEnd() = 0;
};
I would like to define a class in Python that does the equivalent, all defined in python, that will allow for the definition of a visitor:
class GraphVisitor:
def __init__(self, label):
self._label = label
print("__init__(self,{0})".format(self._label))
def OnStateBegin(self):
print("OnStateBegin()" + self._label)
def OnNode(self, i_node):
print("OnNode()" + self._label)
def OnStateEnd(self):
print("OnStateEnd()" + self._label)
And what I'm trying to do is create an instance of a GraphVisitor in python script, and call the methods OnStateBegin(), OnNode(), and OnStateEnd() for a given instance from C++. Here's what I'd like to do in Python:
#model is a SWIG wrapped class
mvis = GraphVisitor("This is a test")
model.Visit("mvis") # I'm not sure how to pass the instance 'mvis' to C++?
And in my C++ wrapped by Swig, I'm not sure how to get at the instance 'mvis'? I can call functions defined in Python no problem, but instances has me stumped!
In order to solve this problem, I retrieved the class from the module given it's module name and class name (the code below assumes the module hasn't already been loaded):
void Model::Visit(const char* mod_name, const char* class_name)
{
PyErr_Clear();
PyObject* mod_name_obj = PyString_FromString(mod_name);
PyObject* class_name_obj = PyString_FromString(class_name);
PyObject* py_module = PyImport_Import(mod_name_obj);
PyObject* err_1 = PyErr_Occurred();
if(err_1)
PyErr_Print();
Once I had the module, I looked up the class from it's dictionary:
if(py_module)
{
PyObject* py_module_dict = PyModule_GetDict(py_module);
PyObject* py_class = PyDict_GetItem(py_module_dict, class_name_obj);
I simplified my problem a bit by instantiating the python class in C++, then created my visitor, and finally visited it:
if(py_class && PyClass_Check(py_class) && PyCallable_Check(py_class))
{
PyObject* inst = PyInstance_New(py_class, 0, 0);
if(inst && PyInstance_Check(inst))
{
IModel::IVisitorPtr py_visitor = new PyModelVisitor(inst);
_model->Visit(py_visitor);
}
}
}
}
The visitor had 3 functions OnStateBegin(), OnNode(), and OnStateEnd(). I added to my SWIG python binding generator an option to generate a header file for external access to the SWIG runtime with the -external-runtime option, so I could create a class in C++ (INode* below) and pass it to Python as the argument to the python OnNode() member function as follows (error checking removed for brevity):
VisitorCtrl OnNode(INode* node)
{
Node* node_impl = new NodeImpl(node);
PyObject* pynode = SWIG_NewPointerObj(node_impl, SWIG_TypeQuery("Node *"), 0);
PyObject* result = PyObject_CallMethodObjArgs(_inst, PyString_FromString("OnNode"), pynode, 0);
long rivis = PyInt_AsLong(result);
return(static_cast<VisitorCtrl>(rivis));
}
I don't know if that's possible with SWIG, but you can do it with SIP.
sip_vector_test.h:
class EXPORT Node {
public:
explicit Node(int n) : n_(n) {};
int getN() const { return n_; }
private:
int n_;
};
struct EXPORT NodeVisitor {
virtual void OnNode(Node* n) = 0;
};
struct EXPORT Graph {
public:
void addNode(int num);
void accept(NodeVisitor *nv);
private:
std::vector< std::shared_ptr<Node> > nodes_;
};
visitor.sip:
%Module pyvisit
%ModuleHeaderCode
#include "sip_visitor_test.h"
%End
class Node {
public:
explicit Node(int n);
int getN() const;
};
struct NodeVisitor {
virtual void OnNode(Node* n) = 0;
};
struct Graph {
public:
void addNode(int num);
void accept(NodeVisitor *nv);
};
Using it from Python:
>>> import pyvisit
>>> g = pyvisit.Graph()
>>> g.addNode(3)
>>> g.addNode(5)
>>> class PyNodeVisitor(pyvisit.NodeVisitor):
>>> def OnNode(self, node):
>>> print(node.getN())
>>> pnv = PyNodeVisitor()
>>> g.accept(pnv)
3
5
I've put a zip file containing the source code of this test project on my homepage.
What would be the C equivalent to this python code?
Thanks.
data = gather_me_some_data()
# where data = [ (metic, datapoints), ... ]
# and datapoints = [ (timestamp, value), ... ]
serialized_data = cPickle.dumps(data, protocol=-1)
length_prefix = struct.pack("!L", len(serialized_data))
message = length_prefix + serialized_data
C doesn't supports direct serialization mechanism because in C you can't get type information at run-time. You must yourself inject some type info at run-time and then construct required object by that type info. So define all your possible structs:
typedef struct {
int myInt;
float myFloat;
unsigned char myData[MY_DATA_SIZE];
} MyStruct_1;
typedef struct {
unsigned char myUnsignedChar;
double myDouble;
} MyStruct_2;
Then define enum which collects info about what structs in total you have:
typedef enum {
ST_MYSTRUCT_1,
ST_MYSTRUCT_2
} MyStructType;
Define helper function which lets to determine any struct size:
int GetStructSize(MyStructType structType) {
switch (structType) {
case ST_MYSTRUCT_1:
return sizeof(MyStruct_1);
case ST_MYSTRUCT_2:
return sizeof(MyStruct_2);
default:
// OOPS no such struct in our pocket
return 0;
}
}
Then define serialize function:
void BinarySerialize(
MyStructType structType,
void * structPointer,
unsigned char * serializedData) {
int structSize = GetStructSize(structType);
if (structSize != 0) {
// copy struct metadata to serialized bytes
memcpy(serializedData, &structType, sizeof(structType));
// copy struct itself
memcpy(serializedData+sizeof(structType), structPointer, structSize);
}
}
And de-serialization function:
void BinaryDeserialize(
MyStructType structTypeDestination,
void ** structPointer,
unsigned char * serializedData)
{
// get source struct type
MyStructType structTypeSource;
memcpy(&structTypeSource, serializedData, sizeof(structTypeSource));
// get source struct size
int structSize = GetStructSize(structTypeSource);
if (structTypeSource == structTypeDestination && structSize != 0) {
*structPointer = malloc(structSize);
memcpy(*structPointer, serializedData+sizeof(structTypeSource), structSize);
}
}
Serialization usage example:
MyStruct_2 structInput = {0x69, 0.1};
MyStruct_1 * structOutput_1 = NULL;
MyStruct_2 * structOutput_2 = NULL;
unsigned char testSerializedData[SERIALIZED_DATA_MAX_SIZE] = {0};
// serialize structInput
BinarySerialize(ST_MYSTRUCT_2, &structInput, testSerializedData);
// try to de-serialize to something
BinaryDeserialize(ST_MYSTRUCT_1, &structOutput_1, testSerializedData);
BinaryDeserialize(ST_MYSTRUCT_2, &structOutput_2, testSerializedData);
// determine which object was de-serialized
// (plus you will get code-completion support about object members from IDE)
if (structOutput_1 != NULL) {
// do something with structOutput_1
free(structOutput_1);
}
else if (structOutput_2 != NULL) {
// do something with structOutput_2
free(structOutput_2);
}
I think this is most simple serialization approach in C. But it has some problems:
struct must not have pointers, because you will never know how much memory one needs to allocate when serializing pointers and from where/how to serialize data into pointers.
this example has issues with system endianess - you need to be careful about how data is stored in memory - in big-endian or little-endian fashion and reverse bytes if needed [when casting char * to integal type such as enum] (...or refactor code to be more portable).
If you can use C++, there is the PicklingTools library