I attempting to call into existing C code from Python. The C code defines a struct B that contains an struct array of As. The C code also defines a function that puts values into the structure when called. I can access the array member variable, but it is not an list (or something that supports indexing). Instead, I am getting an object that is a proxy for B*.
I found this question, but it doesn't look like it was completely resolved. I'm also not sure how to make an instance of the Python class B to replace the PyString_FromString().
Below is the code needed to demonstrate my issue and how to execute it:
example.h
typedef struct A_s
{
unsigned int thing;
unsigned int other;
} A_t;
typedef struct B_s
{
unsigned int size;
A_t items[16];
} B_t;
unsigned int foo(B_t* b);
example.c
#include "example.h"
unsigned int
foo(B_t* b)
{
b->size = 1;
b->items[0].thing = 1;
b->items[0].other = 2;
}
example.i
%module example
%{
#define SWIG_FILE_WITH_INIT
#include "example.h"
%}
%include "example.h"
setup.py
from distutils.core import setup, Extension
module1 = Extension('example', sources=['example.c', 'example.i'])
setup(name='Example', version='0.1', ext_modules=[module1])
script.py - This uses library and demonstrates the failure.
import example
b = example.B_t()
example.foo(b)
print b.size
print b.items
for i in xrange(b.size):
print b.items[i]
How to run everything:
python setup.py build_ext --inplace
mv example.so _example.so
python script.py
you could write some C helper functions like:
int get_thing(B_t *this_b_t, int idx);
int get_other(B_t *this_b_t, int idx);
void set_thing(B_t *this_b_t, int idx, int val);
void set_other(B_t *this_b_t, int idx, int val);
wrap these and you can then use the pointer that you get from example.B_t() to access values from within your data-structure arrangement, e.g.
>>> b = example.B_t()
>>> a_thing = example.get_thing(b, 0)
>>> example.set_thing(b,0,999)
Hopefully its obvious what the implementation of these C functions should be - if not I could provide an example...
granted this isn't as seamless as being able to access C arrays as python lists - you might be able to use typemaps to achieve this but I can't remember the exact syntax you would need in this instance - you'd have to do a bit of RTFM on the SWIG documentation
also possible duplicate here: nested structure array access in python using SWIG
Related
I have a library of C++ classes that I am building a Python interface for using SWIG. Many of these classes have methods that take in a double* array or int* array parameter without inputting a size. For example, there are many methods that have a declaration like one of the following:
void func(double* array);
void func2(double* array, double unrelated_parameter, ...);
I would like to be able to use these functions in Python, with the user passing in a Python numpy array. The size of these arrays are never given as a parameter to the function. The size of the input array is given in the constructor of the objects of these C++ classes and it is assumed that every input array that is given as a parameter to these class methods will have the same size. All of the numpy examples I have seen require me to add an int array_size parameter to the C++ method/function being wrapped.
Is there a way to wrap these C++ functions without having change the API of my entire C++ library to include an int array_size parameter for every single function? Ideally, a user should pass in a Python numpy array and SWIG will automatically convert it to a double or int array on the C++ side.
I have already included numpy.i and followed the instructions here: https://numpy.org/doc/stable/reference/swig.interface-file.html but am getting errors like the following:
TypeError: in method 'func', argument 2 of type 'double *'
One way I can think of is to suppress the "no size" version of the function and extend the class to have a version with a throw-away dimension variable that uses the actual parameter in the class.
Example:
test.i
%module test
%{
#define SWIG_FILE_WITH_INIT
class Test {
public:
int _dim; // needs to be public, or have a public accessor.
Test(int dim) : _dim(dim) {}
double func(double* array) {
double sum = 0.0;
for(int i = 0; i < _dim; ++i)
sum += array[i];
return sum;
}
};
%}
%include "numpy.i"
%init %{
import_array();
%}
%apply (double* IN_ARRAY1, int DIM1) {(double* array, int /*unused*/)};
%ignore Test::func; // so the one-parameter version isn't wrapped
class Test {
public:
Test(int dim);
double func(double* array);
};
%rename("%s") Test::func; // unignore so the two-parameter version will be used.
%extend Test {
double func(double* array, int /*unused*/) {
return $self->func(array);
}
}
Demo:
>>> import test
>>> t = test.Test(5)
>>> import numpy as np
>>> a = np.array([1.5,2.0,2.5,3.75,4.25])
>>> t.func(a)
14.0
I am using SWIG to generate Python bindings for a C library. The library defines a struct with value semantics. In C++ terminology the struct would be POD - copying it with memcpy produces a semantically correct copy.
clib.h:
struct s
{
int i;
};
I compile this into a Python module using SWIG. Details on the build process are in an "appendix" of this question.
In C code, we can verify that the assignment operator for variables of struct type have value semantics:
#include <assert.h>
#include "clib.h"
int main()
{
struct s s1;
s1.i = 100;
struct s s2 = s1;
assert(s2.i == 100);
s2.i = 101;
assert(s1.i == 100);
}
In the Python wrapper, as expected, we instead have reference semantics:
import clib
s1 = clib.s()
s1.i = 100
s2 = s1
assert s2.i == 100
s2.i = 101
assert s1.i == 101
If our library were written in C++ instead, and s had a copy constructor, SWIG would generate one for the Python wrapper too (reference). We could write:
s3 = clib.s(s1)
but for a C library, this wrapper is not generated:
TypeError: __init__() takes exactly 1 argument (2 given)
We might hope that SWIG generates the appropriate magic methods for copy.deepcopy:
from copy import deepcopy
s4 = deepcopy(s1)
but it does not:
TypeError: can't pickle SwigPyObject objects
I am perplexed. This seems like it should be really simple, but I can find nothing. In the "SWIG and Python" documentation, the word "copy" only appears in the previously linked note on C++ copy constructors, and in some low level details about the generated wrapper code. The closest question on Stack Overflow has a horrifically complicated answer.
Details to reproduce
Define a trivial SWIG interface file clib.i:
%module clib
%{
#include "clib.h"
%}
%include "clib.h"
Create a Python module using setup.py:
from distutils.core import setup, Extension
clib = Extension(
"_clib",
sources=["clib_wrap.c"],
extra_compile_args=["-g"],
)
setup(name="clib", version="1.0", ext_modules=[clib])
Build the whole thing with a Makefile:
swig: setup.py clib_wrap.c
python2 setup.py build_ext --inplace
clib_wrap.c: clib.i
swig -python clib.i
Then, compile/run the Python and C test programs exactly as listed above.
While a C library can't have constructors/destructors, you can define them after-the-fact for a SWIG wrapper ref: swig docs 5.5.6. Note that constructors have to be written carefully:
There is one subtle difference to a normal C++ constructor implementation though and that is although the constructor declaration is as per a normal C++ constructor, the newly constructed object must be returned as if the constructor declaration had a return value.
test.i:
%module test
%{
#include <stdlib.h>
#include "clib.h"
%}
%include "clib.h"
%extend s { // add additional methods to the s struct
s(int i) { // constructor
struct s* t = malloc(sizeof(struct s));
t->i = i;
return t;
}
s(struct s* o) { // copy constructor
struct s* t = malloc(sizeof(struct s));
t->i = o->i;
return t;
}
~s() { // destructor
free($self);
}
}
Use case:
>>> import test
>>> s1 = test.s(5)
>>> s1.i
5
>>> s2 = test.s(s1) # copy
>>> s2.i
5
>>> s2.i = 7
>>> s1.i
5
>>> s2.i
7
i use c/python api to bind my class to python.
I make new module and bind just class type successfully.
c/python api have PyMethodDef structure to bind function to python class.
PyMethodDef pyMethodDef = { "PyFunction", PYFunction, METH_KEYWORDS |
METH_VARARGS , "PyFunction" };
PyObject *pFunc = PyCFunction_New(&pyMethodDef, NULL);
PyObject *pMethod = PyInstanceMethod_New(pFunc);
int result = PyDict_SetItemString(pPyClassDict, pyMethodDef.ml_name,
pMethod);
But, there is a terrible rule that bind function must have PyObject* return type and it must get 2 PyObject* arguments.
PyObject* PYFunction(PyObject* self, PyObject* args)
{
char *s = "Hello from C!";
return Py_BuildValue("s", s);
}
So i can not bind my class and member function like this.
class Position
{
int x;
int y;
public:
Position() {x = 0; y = 0}
~Potition() { }+
void SetZero()
{
x = 0, y = 0
}
}
It is impossible to change all member function's signature like below.
Because actually i have many class to bind.
PyObject* method(PyObject*, PyObject*)
Well the boost python will be a solution but it is too big to use.
I only need python binder and it's too difficult to use.
Is there have any way to bind c++ member function(method) to python just only use pure c / python api?
(PyMethodDef structure's second member is PyCFunction type...)
To get a simple wrapper working quickly, whilst having to write very little glue then SWIG is your best bet.
Say you want to wrap example.{h,cpp}, typically all you need to write is:
/* File: example.i */
%module example
%{
#define SWIG_FILE_WITH_INIT
#include "example.h"
%}
%include example.h
%module example tells SWIG to generate a module called "example"
#include "example.h" makes your C++ function definitions available in example.i
%include example.h tells SWIG to export every function and class definition it finds directly in example.h.
Then run:
swig -c++ -python example.i # generates example.py and example_wrap.cxx
g++ -c -fPIC example.cxx
g++ -c -fPIC example_wrap.cxx -I /usr/include/python3.5
g++ -shared example.o example_wrap.o -o _example.so
And you are done. (For reference, I am using python 3.5 and SWIG 3.0)
Function Overloading
A common pitfall is if you have overloaded functions that SWIG cannot disambiguate (eg. they overload integral types [foo(int) vs foo(short)], floating point types, or pointers vs references [Foo* vs Foo&]). You can add an ignore or rename directive in example.i to help it know which functions to ignore or how to expose them.
For example, given:
/* File: example.h */
int add(int a, int b);
short add(short a, short b);
long long add(long long a, long long b);
double add(double a, double b);
You could add the following to your example.i:
%ignore add(short, short);
%rename("add_long_long") add(long long, long long);
And then you could use it like:
>>> import example
>>> example.add(1, 2) # add(int, int)
3
>>> example.add(1.5, 2) # add(double, double)
3.5
>>> example.add_long_long(1, 2) # add(long, long)
3
>>> example.add_long_long(1 << 30, 1 << 30) # add(long, long)
2147483648
>>> example.add(1 << 30, 1 << 30) # add(int, int)
-2147483648
I a very new to swig and I am trying to create a swig wrapper in order to use a few C++ files in python. I have the following C++ class.
The following is a snippet of the code that I am trying to convert:
/*packet_buffer.h*/
class CPacketBuffer {
public:
// construct based on given buffer, data is not copied
CPacketBuffer(uint8_t* data, uint32_t length) {
mpBuffer = data;
mLength = length;
mHead = 0;
mTail = length;
}
uint8_t* GetBuffer() {
return (mpBuffer + mHead);
}
void Add(const uint8_t* data, uint32_t length) {
if ((mTail + length) > mLength) {
length = (mLength - mTail);
}
//....
}
I have been trying to write a example.i file that would accept pointers to typedefs(uint8_t *) all day today using help from swig documentation, but I have been unsuccessful.
The following is a packet_buffer.i file that I have tried which doesn't work.
%module packet_buffer
%include typemaps.i
%apply unsigned char* {uint8_t*};
%apply unit8_t *INPUT {uint8_t *data};
%{
#define SWIG_FILE_WITH_INIT
#include "../include/packet_buffer.h"
%}
%include "../include/packet_buffer.h"
How do I write a swig code for member functions that take pointers to typedefs?
Can I write a common %apply that can be used across the code or will I have to write specifics for each INPUT, OUTPUT parameter?
If I've understood this correctly the problem you're facing isn't that they're pointers, it's that they're potentially unbounded arrays.
You can warp an unbounded C array using carrays.i and the "%array_class" macro, e.g.:
%module packet
%include "stdint.i"
%{
#include "packet.h"
%}
%include "carrays.i"
%array_class(uint8_t, buffer);
%include "packet.h"
Would then allow you to in Python write something like:
a = packet.buffer(10000000)
p = packet.CPacketBuffer(a.cast(), 10000000)
Note that you'll need to ensure the life of the buffer is sufficient - if the Python object gets released without the C++ code being aware you'll end up with undefined behaviour.
You can convert uint8_t* pointers (unbounded arrays) to buffer instances in Python using the frompointer methods that the %array_class macro also creates, e.g.:
r = packet.GetBuffer()
buf = packet.buffer_frompointer(r)
You can add additional Python code to automate/hide most of the conversion between buffers if desired, or use MemoryViews to integrate tighter with Python on the C API side.
In general though since this is C++ I'd suggest using std::vector for this - it's much nicer to use on the Python side than the unbounded arrays and the cost is minimal for the safety and simplicity it gives you.
myPythonClient (below) wants to invoke a ringBell function (loaded from a DLL using ctypes). However, attempting to access ringBell via its name results in an AttributeError. Why?
RingBell.h contains
namespace MyNamespace
{
class MyClass
{
public:
static __declspec(dllexport) int ringBell ( void ) ;
} ;
}
RingBell.cpp contains
#include <iostream>
#include "RingBell.h"
namespace MyNamespace
{
int __cdecl MyClass::ringBell ( void )
{
std::cout << "\a" ;
return 0 ;
}
}
myPythonClient.py contains
from ctypes import *
cdll.RingBell[1]() # this invocation works fine
cdll.RingBell.ringBell() # however, this invocation errors out
# AttributeError: function 'ringBell' not found
Your C++ compiler is mangling the names of all externally visible objects to reflect (as well as their underlying names) their namespaces, classes, and signatures (that's how overloading becomes possible).
In order to avoid this mangling, you need an extern "C" on externally visible names that you want to be visible from non-C++ code (and therefore such names cannot be overloaded, nor in C++ standard can they be inline, within namespaces, or within classes, though some C++ compilers extend the standard in some of these directions).
All is working now :) To summarize your posts:
Write DLL in C++:
// Header
extern "C"
{ // Name in DLL will be "MyAdd" - but you won't be able to find parameters etc...
__declspec(dllexport) int MyAdd(int a, int b);
}
// Name will be with lot of prefixes but some other info is provided - IMHO better approach
__declspec(dllexport) int MyAdd2(int a, int b);
//.cpp Code
__declspec(dllexport) int MyAdd(int a, int b)
{ return a+b;
}
__declspec(dllexport) int MyAdd2(int a, int b)
{ return a+b;
}
Then you can use program link.exe to see real function name in dll.
link.exe is for example in MSVC2010 here:
c:\program files\microsoft visual studio 10.0\VC\bin\link.exe
use:
link /dump /exports yourFileName.dll
you see Something like:
ordinal hint RVA name
1 0 00001040 ?MyAdd2##YAHHH#Z = ?MyAdd2##YAHHH#Z (int __cdecl MyAdd2(int,int))
2 1 00001030 MyAdd = _MyAdd
Then in python you can import it as:
import ctypes
mc = ctypes.CDLL('C:\\testDll3.dll')
#mc.MyAdd2(1,2) # this Won't Work - name is different in dll
myAdd2 = getattr(mc,"?MyAdd2##YAHHH#Z") #to find name use: link.exe /dump /exports fileName.dll
print myAdd2(1,2)
#p1 = ctypes.c_int (1) #use rather c types
print mc[1](2,3) # use indexing - can be provided using link.exe
print mc.MyAdd(4,5)
print mc[2](6,7) # use indexing - can be provided using link.exe
Perhaps because the C++ name is mangled by the compiler and not exported from the DLL as RingBell. Have you checked that it appears in the exported names exactly like that?