i have a SWIG C-python interface. I want to make individual functions from test.c available in my python interface. I feel like i am wasting hrs on the documentation without finding the simple way to do it. The only options i know are to either:
%module test
%{
#include "test.h"
//#include "test.c"
%}
%include "stdint.i" # Add support for uint8_t, int32_t, etc...
/** Keep test immutable for now because we have const struct members in the
* header file which can not be reassigned.
*/
%immutable;
%include test.h // Test header for interface debugging
//%include test.c
//%import test.h
%mutable;
I tried either #including 'test.c' in the %{} section of my interface (This section just copies the file content into the target interface), %include 'test.c' or %import 'test.c'
But i have a whole bunch of other functions in 'test.c' that have dependencies to other files and i cannot import the generated interface in python (ImportError: Undefined symbol:)
#include "test.h"
#include "someotherfunctions.h"
#include "specialdefines.h"
int testFunc(int n, int m) // The function i want to use
{
int res = n + m;
return res;
}
int otherFunc(void)
{
otherDifferentFunc(); // Stuff from other header/c files which my interface doesnt know
}
SPECIALMACRO(X) // More stuff from other headers that i dont want in my interface
int foo = someOtherOperation(500);
The header file test.h:
/* Functions ----------------------------------------------------------------*/
int testFunc(int n, int m);
I feel like i am not having the right approach to make single functions available in my Python interface without including all the unwanted stuff from all the headers in my source file. In reality, my 'test.c' is quite large and includes a whole bunch of header for all sorts of symbols and external functions used. If i dont #include all oft those in my %{} interface section, i get the import error. But if i include them, they again require other headers (its a large project) and i am starting a messy chain of includes for just a tiny function, unnecessary bloating my interface. This can't be the SWIG way to go guys?
If someone knows a good way to solve this, please let me know.
%module test
%{
// Whatever goes here is directly copied to the generated wrapper code.
// Put what you need here for the wrapper to build, typically just
// your public headers.
#include "test.h"
%}
// Anything you declare or %import is processed to make the SWIG interface.
// You can %import "test.h" and all the functions *directly* in
// the header (not other #include in the header), are exposed to Python.
//
// If you only want one function, just put its prototype, like:
int testFunc(int n, int m);
When you build the wrapper, swig the .i file to generate test_wrap.c, then compile test.c and test_wrap.c and link them together at the link step.
An example:
test.i
%module test
%{
#include "test.h"
%}
// uncomment to bring in func1 and func2 (not func3, but it's still used by func2)
//%include "test.h"
// only exposes func1
int func1();
test.h
int func1();
int func2();
test.c
#include "test.h"
int func3() { return 3; }
int func2() { return func3() - 1; }
int func1() { return func2() - 1; }
makefile (Windows, Microsoft compiler)
all: _test.pyd
test_wrap.c test.py: test.i test.h
echo swigging...
swig -python test.i
_test.pyd: test_wrap.c test.c test.h
cl /nologo /LD /MD /W3 /Fe_test.pyd /Id:\dev\python310\include test_wrap.c test.c -link /libpath:d:\dev\python310\libs
Output (built as above, only func1() exposed):
>>> import test
>>> test.func1
<function func1 at 0x000001CF6DEF7F40>
>>> test.func1()
1
>>> test.func2()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'test' has no attribute 'func2'. Did you mean: 'func1'?
Output (built uncommenting the %include and commenting the prototype):
>>> import test
>>> test.func1()
1
>>> test.func2()
2
>>> test.func3()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'test' has no attribute 'func3'. Did you mean: 'func1'?
Related
Preface:
I have two header files: "Sample.h" and "Sample2.h". Here are the contents of the two headers:
"Sample.h" has:
#include "Sample2.h"
typedef struct {
int c;
sJustFloats sJf;
}sTest;
"Sample2.h" has:
typedef struct {
float a;
float b;
}sJustFloats;
Sample.c just has:
#include "Sample.h"
My SWIG interface file has:
%module Sample
%{
/* Put header files here or function declarations like below */
#include "Sample.h"
%}
%include "Sample.h"
%apply float {float a}; // For sJustFloats.a
%apply float {float b}; // For sJustFloats.b
%apply int {int c}; // For sTest.c
%apply sJustFloats {sJustFloats sJf}; // For sTest.sJf
%apply sTest {sTest test}; //I guess this exposes test struct itself.
Steps followed to get .so file that is imported in Python:
gcc -c -fPIC Sample.c Sample_wrap.c -I /usr/include/python3.6m
ld -shared Sample.o Sample_wrap.o -o _Sample.so
Observations:
I am able to import this as a module in Python and I can access fields of sTest as follows:
test = Sample.sTest()
test.c // To access "c" -- This works fine
Problem: I am not able to access fields of struct sJustFloats
If I do test.sJf(), it gives me TypeError: 'SwigPyObject' object is not callable. Similarly test.sJf.a gives: AttributeError: 'SwigPyObject' object has no attribute 'a'
Can someone please tell me how to access members of the struct, sJustFloats?
Additional observations:
When I do type(Sample.sTest), it gives me <class 'type'>, whereas doing type(Sample.sTest.sJf) gives me <class 'property'>.
If I have both structs in the same file "Sample.h", I am able to access all the fields by doing:
test = Sample.sTest()
sjf = test.sJf()
sjf.a # Works fine
sjf.b # Works fine
Am I missing something in the interface file? Are there are nuances of SWIG that I'm unaware of?
SWIG doesn’t recurse by default, but you need %include "Sample2.h" after %include "Sample.h" and don’t need any of the %apply:
%module Sample
%{
#include "Sample.h"
%}
%include "Sample.h"
%include "Sample2.h"
I am new to learning swig. I am interested in calling C++ from Python on an Ubuntu machine.
I just started looking at the intro tutorial here http://www.swig.org/tutorial.html
Consider the interface file on that page example.i copied as is below.
/* example.i */
%module example
%{
/* Put header files here or function declarations like below */
extern double My_variable;
extern int fact(int n);
extern int my_mod(int x, int y);
extern char *get_time();
%}
extern double My_variable;
extern int fact(int n);
extern int my_mod(int x, int y);
extern char *get_time();
Why is are the contents between the %{ %} repeated in the second half of the file? As given in the manual, http://www.swig.org/Doc3.0/SWIGDocumentation.html#Introduction_nn5
The %{ %} block provides a location for inserting additional code,
such as C header files or additional C declarations, into the
generated C wrapper code.
But it doesn't address the point of the repetition in the example. What am I missing?
The code between %{ and %} is inserted verbatim in the generated SWIG wrapper, and is used to give the wrapper code access to the headers or declarations listed.
The code outside those markers instructs SWIG to make a wrapper for each of the declarations (or entire header files) listed.
If you left out extern int fact(int n); in the first portion, the wrapper, when compiled and linked to the source or library containing the function, wouldn't be able to access the function since the extern declaration would be missing. If left out of the second portion, a wrapper wouldn't be generated to access it from the scripting language.
There is a shortcut:
%inline %{
...
%}
That instructs SWIG to both insert and wrap the declarations.
i use c/python api to bind my class to python.
I make new module and bind just class type successfully.
c/python api have PyMethodDef structure to bind function to python class.
PyMethodDef pyMethodDef = { "PyFunction", PYFunction, METH_KEYWORDS |
METH_VARARGS , "PyFunction" };
PyObject *pFunc = PyCFunction_New(&pyMethodDef, NULL);
PyObject *pMethod = PyInstanceMethod_New(pFunc);
int result = PyDict_SetItemString(pPyClassDict, pyMethodDef.ml_name,
pMethod);
But, there is a terrible rule that bind function must have PyObject* return type and it must get 2 PyObject* arguments.
PyObject* PYFunction(PyObject* self, PyObject* args)
{
char *s = "Hello from C!";
return Py_BuildValue("s", s);
}
So i can not bind my class and member function like this.
class Position
{
int x;
int y;
public:
Position() {x = 0; y = 0}
~Potition() { }+
void SetZero()
{
x = 0, y = 0
}
}
It is impossible to change all member function's signature like below.
Because actually i have many class to bind.
PyObject* method(PyObject*, PyObject*)
Well the boost python will be a solution but it is too big to use.
I only need python binder and it's too difficult to use.
Is there have any way to bind c++ member function(method) to python just only use pure c / python api?
(PyMethodDef structure's second member is PyCFunction type...)
To get a simple wrapper working quickly, whilst having to write very little glue then SWIG is your best bet.
Say you want to wrap example.{h,cpp}, typically all you need to write is:
/* File: example.i */
%module example
%{
#define SWIG_FILE_WITH_INIT
#include "example.h"
%}
%include example.h
%module example tells SWIG to generate a module called "example"
#include "example.h" makes your C++ function definitions available in example.i
%include example.h tells SWIG to export every function and class definition it finds directly in example.h.
Then run:
swig -c++ -python example.i # generates example.py and example_wrap.cxx
g++ -c -fPIC example.cxx
g++ -c -fPIC example_wrap.cxx -I /usr/include/python3.5
g++ -shared example.o example_wrap.o -o _example.so
And you are done. (For reference, I am using python 3.5 and SWIG 3.0)
Function Overloading
A common pitfall is if you have overloaded functions that SWIG cannot disambiguate (eg. they overload integral types [foo(int) vs foo(short)], floating point types, or pointers vs references [Foo* vs Foo&]). You can add an ignore or rename directive in example.i to help it know which functions to ignore or how to expose them.
For example, given:
/* File: example.h */
int add(int a, int b);
short add(short a, short b);
long long add(long long a, long long b);
double add(double a, double b);
You could add the following to your example.i:
%ignore add(short, short);
%rename("add_long_long") add(long long, long long);
And then you could use it like:
>>> import example
>>> example.add(1, 2) # add(int, int)
3
>>> example.add(1.5, 2) # add(double, double)
3.5
>>> example.add_long_long(1, 2) # add(long, long)
3
>>> example.add_long_long(1 << 30, 1 << 30) # add(long, long)
2147483648
>>> example.add(1 << 30, 1 << 30) # add(int, int)
-2147483648
I'm writing a Python C extension by SWIG but getting frustrated when passing binary buffer to C api functions. Here's my example:
In utils.c
#include "utils.h"
void my_print_hexbytes(uint32_t *bytes, uint32_t bytes_len)
{
uint32_t i;
for(i = 0; i < bytes_len; i++)
printf("%02x", bytes[i]);
printf("\n");
}
In utils.h
#include "commons.h"
#ifndef XXXX_UTILS_H
#define XXXX_UTILS_H
void my_print_hexbytes(uint32_t *bytes, uint32_t bytes_len);
#endif /* XXXX_UTILS_H */
In commons.h
#ifndef XXXX_COMMONS_H
#define XXXX_COMMONS_H
....
....
#include <stdint.h>
....
....
#endif
In xxxx_for_py.i
%module xxxx_for_py
%{
#define SWIG_FILE_WITH_INIT
#include "commons.h"
#include "utils.h"
%}
%include "stdint.i"
void my_print_hexbytes(uint32_t *bytes, uint32_t bytes_len);
1st Try
After compiling out _xxxx_for_py.so, I gave a try testing it in IPython:
In [1]: import xxxx_for_py
In [2]: from ctypes import *
In [3]: a_t = c_uint * 2
In [4]: a = a_t(0x1, 0x2)
In [5]: xxxx_for_py.my_print_hexbytes(a, 2)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-5-3c023fcf8b04> in <module>()
----> 1 xxxx_for_py.my_print_hexbytes(a, 2)
TypeError: in method 'my_print_hexbytes', argument 1 of type 'uint32_t *'
That time I had no idea how to handle this case. I tried changing the a as bytearray but it didn't work.
Not sure if anyone who could help me on this issue. Thanks!
2nd Try
Thanks for #Petesh comment. I've hit a try by casting a to POINTER(c_uint) but still not work :(
In [5]: xxxx_for_py.my_print_hexbytes(cast(a, POINTER(c_uint)), 2)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-5-2abb9dae484d> in <module>()
----> 1 xxxx_for_py.my_print_hexbytes(cast(a, POINTER(c_uint)), 2)
TypeError: in method 'my_print_hexbytes', argument 1 of type 'uint32_t *'
3rd Try
Referenced by Unbounded Array section in SWIG documentation, I added carray.i to xxxx_for_py.i
In xxxx_for_py.i
....
....stdint.i"
%include "carrays.i"
%array_class(uint32_t, uint32Array)
....
After re-compiling and loading .so, it still got same error
In [1]: import xxxx_for_py
In [2]: a = xxxx_for_py.uint32Array(2)
In [3]: a[0] = 0x0
In [4]: a[1] = 0x1
In [5]: xxxx_for_py.my_print_hexbytes(a, 2)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-5-3c023fcf8b04> in <module>()
----> 1 xxxx_for_py.my_print_hexbytes(a, 2)
TypeError: in method 'my_print_hexbytes', argument 1 of type 'uint32_t *'
4th Try
After a week later at that time, user Nethanel post an answer. Though it still did not work, it could help me find something interesting.
After brief testing, I found solution by myself. See this article for details http://au9ustine.github.io/wiki/crypto/cuda/swig.html#typeerror-in-method-xxxx-argument-y-of-type-zzzz (Update on 2017-01-13: This page is broken, I'll move that page content here)
A brief anatomy
Keeping trying on testing Python codes on passing binary buffer to a trivial C module (i.e. library compiled by gcc without extra dependencies). And internals might look like:
integer list -> array/struct -> raw string -> C function parameter
From some descriptions from create_string_buffer. By create_string_buffer, Python could provide such a feature dynamically allocating pieces of memory for current variables for use.
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
size_t
print_int_buf(void *src, size_t len)
{
size_t i = 0;
uint32_t *int_ptr = (uint32_t *)src;
for (i = 0; i < len; i++) {
printf("%p: %x\n", int_ptr + i, int_ptr[i]);
}
return i;
}
If compiled as a shared library named my_ext_lib.so, the code could be loaded in Python
import ctypes
from ctypes import *
import array
my_ext = ctypes.CDLL('./my_ext_lib.so')
buf = create_string_buffer(array.array('I', range(0x10)).tostring())
my_ext_lib.print_int_buf(buf, 0x10)
And it did work.
Though ctype and SWIG are quite different approaches for
interoperating integration between Python and C, I use ctype code
here to elaborate the basic mechanism and its working way of passing
parameters to a compiled module (both for Python module and trivial
C shared library module).
Interim solution
Since the significant difference was the function interface (or method signature) from my view, why not change it to void * instead of uint32_t *? (Though it's not the best way, it could be still an interim solution for containment)
I replaced uint32_t * with void * for the method my_print_hexbytes, and appended dependency cdata.i (I'm not sure if it works but for safety it has been added). So the changes were listed below and eventually it displayed expected result.
utils.c
#include "utils.h"
...
void
my_print_hexbytes(void *bytes, size_t bytes_len)
{
size_t i = 0;
uint32_t *int_buf_ptr = (uint32_t *)bytes;
for(i = 0; i < bytes_len; i++)
printf("%02x", int_buf_ptr[i]);
printf("\n");
}
header utils.h
#include "commons.h"
#ifndef XXXX_UTILS_H
#define XXXX_UTILS_H
...
void my_print_hexbytes(void *bytes, size_t bytes_len);
#endif /* XXXX_UTILS_H */
xxxx_for_py.i
%module xxxx_for_py
%{
#define SWIG_FILE_WITH_INIT
#include "commons.h"
#include "utils.h"
%}
%include "stdint.i"
%include "carrays.i"
%include "cdata.i"
%array_class(uint32_t, uint32Array)
...
void my_print_hexbytes(void *bytes, size_t bytes_len);
Invoking method in Python, it would be like:
In [1]: import xxxx_for_py
In [2]: a = xxxx_for_py.uint32Array(0x10)
In [3]: for i in range(0x10):
...: a[i] = i
...:
In [4]: xxxx_for_py.my_print_hexbytes(a, 0x10)
000102030405060708090a0b0c0d0e0f
I had a very simple case but still got this error. The fix from this question worked:
Swig python - c++ how to use type int8_t
I added %include "stdint.i" directly after the module def in the .i file
Found you question, because of same problem.
Unfortunately, it seems advanced types are not supported by SWIG: in http://www.swig.org/Doc3.0/Library.html#Library_carrays, after description of array_class macro, it is written: "When using this macro, type is restricted to a simple type name like int or float."
I also tried unsuccessfully to use "unsigned int".
In the end my workaround was to use C++ vectors instead of uint32_t pointers, here is relevant part of swig file:
%include "stdint.i"
%include "std_vector.i"
namespace std {
%template(vectoruint32) vector<uint32_t>;
};
part of h file:
#include <vector>
static uint32_t foo(std::vector<uint32_t> v) {return v[1] - v[0];}
python:
import example
l = [1, 2, 3, 4]
example.foo(l)
Hope this helps
I'm trying to get SWIG to recognize a simple preprocessor macro that "defines" a new function based on another definition and more complicated function. So, in the C header file I have:
#define FOO 1
#define my_macro_fun(args) my_fun(args,FOO)
SWIG sees and successfully wraps my_fun, but I want it to wrap my_macro_fun instead.
SWIG tries to spot macros that are constants and wrap them, but it won't be able to do anything smart with a macro like that. Fortunately there's an easy work around. Imagine you have the following header file:
#define FOO 1
#define my_macro_fun(args) my_fun(args,FOO)
void my_fun(int a, int b);
You can wrap it like:
%module test
%{
#include "test.h"
%}
%include "test.h"
which skips the my_macro_fun function. To get SWIG to wrap that though all you need to do is:
%module test
%{
#include "test.h"
%}
// Lie! Tell SWIG that it should be wrapped like any other function.
void my_macro_fun(int);
// This is entirely optional: it will cause my_fun to be wrapped as well
%include "test.h"
This little lie is perfectly fine in SWIG - it'll generate Wrapper code that assumes my_macro_fun(int) is callable, exactly like you would if you were using the macro. When compiling the wrapper the compiler will end up using the macro there and nobody is any the wiser.
Note that the order is important - the function that's really a macro needs to come before the %include in the interface file otherwise SWIG will try to expand the macro during the parsing of your declaration which makes for a syntax error. You can skip the %include entirely, or use a %ignore as well if you want to include it for other parts but suppress the original my_fun in the generated interface.
With some SWIG languages (e.g. Python) you can also use the typemap default:
%module test
%{
#include "test.h"
%}
%typemap(default) int b {
$1 = FOO;
}
%include "test.h"
To supply a value for an argument if none is given for it.