I have a C function which signature looks like this:
typedef double (*func_t)(double*, int)
int some_f(func_t myFunc);
I would like to pass a Python function (not necessarily explicitly) as an argument for some_f. Unfortunately, I can't afford to alter declaration of some_f, that's it: I shouldn't change C code.
One obvious thing I tried to do is to create a basic wrapping function like this:
cdef double wraping_f(double *d, int i /*?, object f */):
/*do stuff*/
return <double>f(d_t)
However, I can't come up with a way to actually "put" it inside wrapping_f's body.
There is a very bad solution to this problem: I could use a global object variable, however this forces me copy-n-paste multiple instances of essentially same wrapper function that will use different global functions (I am planning to use multiple Python functions simultaneously).
I keep my other answer for historical reasons - it shows, that there is no way to do what you want without jit-compilation and helped me to understand how great #DavidW's advise in this answer was.
For the sake of simplicity, I use a slightly simpler signature of functions and trust you to change it accordingly to your needs.
Here is a blueprint for a closure, which lets ctypes do the jit-compilation behind the scenes:
%%cython
#needs Cython > 0.28 to run because of verbatim C-code
cdef extern from *: #fill some_t with life
"""
typedef int (*func_t)(int);
static int some_f(func_t fun){
return fun(42);
}
"""
ctypedef int (*func_t)(int)
int some_f(func_t myFunc)
#works with any recent Cython version:
import ctypes
cdef class Closure:
cdef object python_fun
cdef object jitted_wrapper
def inner_fun(self, int arg):
return self.python_fun(arg)
def __cinit__(self, python_fun):
self.python_fun=python_fun
ftype = ctypes.CFUNCTYPE(ctypes.c_int,ctypes.c_int) #define signature
self.jitted_wrapper=ftype(self.inner_fun) #jit the wrapper
cdef func_t get_fun_ptr(self):
return (<func_t *><size_t>ctypes.addressof(self.jitted_wrapper))[0]
def use_closure(Closure closure):
print(some_f(closure.get_fun_ptr()))
And now using it:
>>> cl1, cl2=Closure(lambda x:2*x), Closure(lambda x:3*x)
>>> use_closure(cl1)
84
>>> use_closure(cl2)
126
This answer is more in Do-It-Yourself style and while not unintersting you should refer to my other answer for a concise recept.
This answer is a hack and a little bit over the top, it only works for Linux64 and probably should not be recommended - yet I just cannot stop myself from posting it.
There are actually four versions:
how easy the life could be, if the API would take the possibility of closures into consideration
using a global state to produce a single closure [also considered by you]
using multiple global states to produce multiple closures at the same time [also considered by you]
using jit-compiled functions to produce an arbitrary number of closures at the same time
For the sake of simplicity I chose a simpler signature of func_t - int (*func_t)(void).
I know, you cannot change the API. Yet I cannot embark on a journey full of pain, without mentioning how simple it could be... There is a quite common trick to fake closures with function pointers - just add an additional parameter to your API (normally void *), i.e:
#version 1: Life could be so easy
# needs Cython >= 0.28 because of verbatim C-code feature
%%cython
cdef extern from *: #fill some_t with life
"""
typedef int (*func_t)(void *);
static int some_f(func_t fun, void *params){
return fun(params);
}
"""
ctypedef int (*func_t)(void *)
int some_f(func_t myFunc, void *params)
cdef int fun(void *obj):
print(<object>obj)
return len(<object>obj)
def doit(s):
cdef void *params = <void*>s
print(some_f(&fun, params))
We basically use void *params to pass the inner state of the closure to fun and so the result of fun can depend on this state.
The behavior is as expected:
>>> doit('A')
A
1
But alas, the API is how it is. We could use a global pointer and a wrapper to pass the information:
#version 2: Use global variable for information exchange
# needs Cython >= 0.28 because of verbatim C-code feature
%%cython
cdef extern from *:
"""
typedef int (*func_t)();
static int some_f(func_t fun){
return fun();
}
static void *obj_a=NULL;
"""
ctypedef int (*func_t)()
int some_f(func_t myFunc)
void *obj_a
cdef int fun(void *obj):
print(<object>obj)
return len(<object>obj)
cdef int wrap_fun():
global obj_a
return fun(obj_a)
cdef func_t create_fun(obj):
global obj_a
obj_a=<void *>obj
return &wrap_fun
def doit(s):
cdef func_t fun = create_fun(s)
print(some_f(fun))
With the expected behavior:
>>> doit('A')
A
1
create_fun is just convenience, which sets the global object and return the corresponding wrapper around the original function fun.
NB: It would be safer to make obj_a a Python-object, because void * could become dangling - but to keep the code nearer to versions 1 and 4 we use void * instead of object.
But what if there are more than one closure in use at the same time, let's say 2? Obviously with the approach above we need 2 global objects and two wrapper functions to achieve our goal:
#version 3: two function pointers at the same time
%%cython
cdef extern from *:
"""
typedef int (*func_t)();
static int some_f(func_t fun){
return fun();
}
static void *obj_a=NULL;
static void *obj_b=NULL;
"""
ctypedef int (*func_t)()
int some_f(func_t myFunc)
void *obj_a
void *obj_b
cdef int fun(void *obj):
print(<object>obj)
return len(<object>obj)
cdef int wrap_fun_a():
global obj_a
return fun(obj_a)
cdef int wrap_fun_b():
global obj_b
return fun(obj_b)
cdef func_t create_fun(obj) except NULL:
global obj_a, obj_b
if obj_a == NULL:
obj_a=<void *>obj
return &wrap_fun_a
if obj_b == NULL:
obj_b=<void *>obj
return &wrap_fun_b
raise Exception("Not enough slots")
cdef void delete_fun(func_t fun):
global obj_a, obj_b
if fun == &wrap_fun_a:
obj_a=NULL
if fun == &wrap_fun_b:
obj_b=NULL
def doit(s):
ss = s+s
cdef func_t fun1 = create_fun(s)
cdef func_t fun2 = create_fun(ss)
print(some_f(fun2))
print(some_f(fun1))
delete_fun(fun1)
delete_fun(fun2)
After compiling, as expected:
>>> doit('A')
AA
2
A
1
But what if we have to provide an arbitrary number of function-pointers at the same time?
The problem is, that we need to create the wrapper-functions at the run time, because there is no way to know how many we will need while compiling, so the only thing I can think of is to jit-compile these wrapper-functions when they are needed.
The wrapper function looks quite simple, here in assembler:
wrapper_fun:
movq address_of_params, %rdi ; void *param is the parameter of fun
movq address_of_fun, %rax ; addresse of the function which should be called
jmp *%rax ;jmp instead of call because it is last operation
The addresses of params and of fun will be known at run time, so we just have to link - replace the placeholder in the resulting machine code.
In my implementation I'm following more or less this great article: https://eli.thegreenplace.net/2017/adventures-in-jit-compilation-part-4-in-python/
#4. version: jit-compiled wrapper
%%cython
from libc.string cimport memcpy
cdef extern from *:
"""
typedef int (*func_t)(void);
static int some_f(func_t fun){
return fun();
}
"""
ctypedef int (*func_t)()
int some_f(func_t myFunc)
cdef extern from "sys/mman.h":
void *mmap(void *addr, size_t length, int prot, int flags,
int fd, size_t offset);
int munmap(void *addr, size_t length);
int PROT_READ # #define PROT_READ 0x1 /* Page can be read. */
int PROT_WRITE # #define PROT_WRITE 0x2 /* Page can be written. */
int PROT_EXEC # #define PROT_EXEC 0x4 /* Page can be executed. */
int MAP_PRIVATE # #define MAP_PRIVATE 0x02 /* Changes are private. */
int MAP_ANONYMOUS # #define MAP_ANONYMOUS 0x20 /* Don't use a file. */
# |-----8-byte-placeholder ---|
blue_print = b'\x48\xbf\x00\x00\x00\x00\x00\x00\x00\x00' # movabs 8-byte-placeholder,%rdi
blue_print+= b'\x48\xb8\x00\x00\x00\x00\x00\x00\x00\x00' # movabs 8-byte-placeholder,%rax
blue_print+= b'\xff\xe0' # jmpq *%rax ; jump to address in %rax
cdef func_t link(void *obj, void *fun_ptr) except NULL:
cdef size_t N=len(blue_print)
cdef char *mem=<char *>mmap(NULL, N,
PROT_READ | PROT_WRITE | PROT_EXEC,
MAP_PRIVATE | MAP_ANONYMOUS,
-1,0)
if <long long int>mem==-1:
raise OSError("failed to allocated mmap")
#copy blueprint:
memcpy(mem, <char *>blue_print, N);
#inject object address:
memcpy(mem+2, &obj, 8);
#inject function address:
memcpy(mem+2+8+2, &fun_ptr, 8);
return <func_t>(mem)
cdef int fun(void *obj):
print(<object>obj)
return len(<object>obj)
cdef func_t create_fun(obj) except NULL:
return link(<void *>obj, <void *>&fun)
cdef void delete_fun(func_t fun):
munmap(fun, len(blue_print))
def doit(s):
ss, sss = s+s, s+s+s
cdef func_t fun1 = create_fun(s)
cdef func_t fun2 = create_fun(ss)
cdef func_t fun3 = create_fun(sss)
print(some_f(fun2))
print(some_f(fun1))
print(some_f(fun3))
delete_fun(fun1)
delete_fun(fun2)
delete_fun(fun3)
And now, the expected behavior:
>>doit('A')
AA
2
A
1
AAA
3
After looking at this, maybe there is a change the API can be changed?
Related
I'm using Cython in CPython 3.6 and I'd like to mark some pointers as "not aliased", to improve performance and be explicit semantically.
In C this is done with the restrict (__restrict, __restrict__) keyword. But how do I use restrict on my cdef variables from a Cython .pyx code?
Thanks!
Cython doesn't have syntax/support for the restrict-keyword (yet?). Your best bet is to code this function in C, most convenient is probably to use verbatim-C-code, e.g. here for a dummy example:
%%cython
cdef extern from *:
"""
void c_fun(int * CYTHON_RESTRICT a, int * CYTHON_RESTRICT b){
a[0] = b[0];
}
"""
void c_fun(int *a, int *b)
# example usage
def doit(k):
cdef int i=k;
cdef int j=k+1;
c_fun(&i,&j)
print(i)
Here I use Cython's (undocumented) define CYTHON_RESTRICT, which is defined as
// restrict
#ifndef CYTHON_RESTRICT
#if defined(__GNUC__)
#define CYTHON_RESTRICT __restrict__
#elif defined(_MSC_VER) && _MSC_VER >= 1400
#define CYTHON_RESTRICT __restrict
#elif defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L
#define CYTHON_RESTRICT restrict
#else
#define CYTHON_RESTRICT
#endif
#endif
so it works not only for C99-compliant c-compiler. In the long run however, it is probably best to define something similar and not to depend on undocumented features.
I have a function with void * parameter and I want to store it in the dict.
What I do is:
%%cython
cdef void foo(void * bar):
pass
cdef dict foobar = {'foo': foo}
But this code raises an Error: Cannot convert 'void (void *)' to Python object Any ways to overcome this problem?
The easiest solution is to create a cdef class that can wrap this function. Since the cdef class is a Python object it can be stored in a dictionary like any other Python object.
ctypedef void (*void_func_ptr)(void*)
cdef class VoidFuncWrapper:
cdef void_func_ptr func
def __cinit__(self):
self.func = NULL
#staticmethod
cdef VoidFuncWrapper make_from_ptr(void_func_ptr f):
cdef VoidFuncWrapper out = VoidFuncWrapper()
out.func = f
return out
Then you can simply do:
cdef dict foobar = {'foo': VoidFuncWrapper.make_from_ptr(foo)}
Your problem here is that void* has no defined converter from a Python object, so a Python wrapper function (that accepts Python objects, converts them, and passes them to the underlying C function) can't be defined, either explicitly with cpdef or implicitly by putting the cdef function in a Python object (the dict in this case).
You could make this work by defining the argument to be something Cython knows how to convert to, e.g.:
cpdef void foo(const char * bar):
pass
cdef dict foobar = {'foo': foo}
Try it online!
But that may not work for your scenario if the function needs to accept an arbitrary pointer. If that's the case, you may want to switch to using C++ containers that can hold your function pointer type directly:
from libcpp.unordered_map cimport unordered_map
from libcpp.string cimport string
cdef void foo(void *bar):
pass
ctypedef void (*f_type)(void *)
cdef unordered_map[string, f_type] foobar
foobar['foo'.encode('ascii')] = foo
My question is similar to the one asked here - Passing C++ vector to Numpy through Cython without copying and taking care of memory management automatically I am also getting a segmentation fault but before I can fix it I have some compilation errors to take of with the move method in Cython.
This is the simplest example I have (just an extension of the Rectangle example provided in the net
What I want to do
My C++ code returns a deque of Points. It can return a vector of points as well. Any container will do
In the cython code I want to store the returned deque of points(one for each iteration of the for loop) in a collection(such as vector or deque). This collection will be an instance variable.
Later on I want to iterate through the collection and convert that deque of deque of points into a python list of lists. Here is where I believe I am getting the segmentation fault.
Point.h
#ifndef POINT_H
#define POINT_H
class Point
{
private:
double coordinate1,coordinate2;
public:
virtual double getCoordinate1() const;
virtual double getCoordinate2() const ;
virtual void setCoordinate1(double coordinate1);
virtual void setCoordinate2(double coordinate2);
};
Rectangle.h
#include <deque>
#include "Point.h"
using std:deque;
deque<Point> getAllPoints(Point query);
Rectangle.cpp
include "Rectangle.h"
deque<Point> Rectangle::getAllPoints(Point query)
{
deque<Point> deq;
for (int i = 0;i < 10000; ++i)
{
deq.push_back(query);
}
return deq;
Note Unlike the linked question I am not returning an address but returning a reference
rect.pxd
cdef extern from "<utility>" namespace "std" nogil:
T move[T](T) #
cdef extern from "Point.h":
cdef cppclass Point:
Point() nogil except +
double getCoordinate1()
double getCoordinate2()
void setCoordinate1(double coordinate1) nogil
void setCoordinate2(double coordinate2) nogil
cdef cppclass SphericalPoint(Point):
SphericalPoint() nogil except +
double getCoordinate1()
double getCoordinate2()
void setCoordinate1(double lat) nogil
void setCoordinate2(double lon) nogil
cdef extern from "Rectangle.h" namespace "shapes":
cdef cppclass Rectangle:
Rectangle(int, int, int, int) except + nogil
deque[Point] getArea(Point p) nogil
And finally
rect.pyx
cdef class PyRectangle:
cdef Rectangle *rect
cdef deque[Point] colOfPoints
def __cinit__(self, int x0, int y0, int x1, int y1):
self.rect = new Rectangle(x0, y0, x1, y1)
self.colOfPoints = deque[Point]()
def performCalc(self,maxDistance,chunk):
cdef deque[Point] area
cdef double[:,:] gPoints
gPoints = memoryview(chunk)
for i in range(0,len(gPoints)):
with nogil:
area = self.getArea(gPoints[i])
self.colOfPoints = move(area)
cdef deque[Point] getArea(self,double[:] p) nogil:
cdef deque[Point] area
area = self.rect.getArea(point)
return area
I believe I am setting the C++ 17 in setup.pyx
setup.py
os.environ['CFLAGS'] = '-O3 -Wall -std=c++17'
ext_modules = [Extension("rect",
["rect.pyx","Rectangle.cpp"],
include_dirs=['/usr/local/include'],
extra_link_args=["-std=c++17"],
language='c++',
)]
extensions = cythonize(ext_modules, language_level = "3")
I am getting these compilation errors
rect.cpp: In function ‘PyObject* __pyx_pf_4rect_11PyRectangle_6performCalc(__pyx_obj_4rect_PyRectangle*, PyObject*, PyObject*)’:
rect.cpp:3781:81: error: no matching function for call to ‘move<std::deque<Point, std::allocator<Point> > >(std::deque<Point>&)’
__pyx_v_self->colOfPoints = std::move<std::deque<Point> >(__pyx_v_area);
^
In file included from /usr/include/c++/7 /bits/nested_exception.h:40:0,
from /usr/include/c++/7/exception:143,
from /usr/include/c++/7/ios:39,
from rect.cpp:632:
/usr/include/c++/7/bits/move.h:98:5: note: candidate: template<class _Tp> constexpr typename std::remove_reference< <template-parameter-1-1> >::type&& std::move(_Tp&&)
move(_Tp&& __t) noexcept
^~~~
/usr/include/c++/7/bits/move.h:98:5: note: template argument deduction/substitution failed:
rect.cpp:3781:81: note: cannot convert ‘__pyx_v_area’ (type ‘std::deque<Point>’) to type ‘std::deque<Point>&&’
__pyx_v_self->colOfPoints = std::move<std::deque<Point> >(__pyx_v_area);
^
In file included from /usr/include/c++/7/bits/char_traits.h:39:0,
from /usr/include/c++/7/ios:40,
from rect.cpp:632:
/usr/include/c++/7/bits/stl_algobase.h:479:5: note: candidate: template<class _II, class _OI> _OI std::move(_II, _II, _OI)
move(_II __first, _II __last, _OI __result)
^~~~
/usr/include/c++/7/bits/stl_algobase.h:479:5: note: template argument deduction/substitution failed:
rect.cpp:3781:81: note: candidate expects 3 arguments, 1 provided
__pyx_v_self->colOfPoints = std::move<std::deque<Point> >(__pyx_v_area);
^
In file included from /usr/include/c++/7/deque:66:0,
from rect.cpp:636:
/usr/include/c++/7/bits/deque.tcc:1048:5: note: candidate: template<class _Tp> std::_Deque_iterator<_Tp, _Tp&, _Tp*> std::move(std::_Deque_iterator<_Tp, const _Tp&, const _Tp*>, std::_Deque_iterator<_Tp, const _Tp&, const _Tp*>, std::_Deque_iterator<_Tp, _Tp&, _Tp*>)
move(_Deque_iterator<_Tp, const _Tp&, const _Tp*> __first,
^~~~
/usr/include/c++/7/bits/deque.tcc:1048:5: note: template argument deduction/substitution failed:
rect.cpp:3781:81: note: candidate expects 3 arguments, 1 provided
__pyx_v_self->colOfPoints = std::move<std::deque<Point> >(__pyx_v_area);
^
In file included from /usr/include/c++/7/deque:64:0,
from rect.cpp:636:
/usr/include/c++/7/bits/stl_deque.h:424:5: note: candidate: template<class _Tp> std::_Deque_iterator<_Tp, _Tp&, _Tp*> std::move(std::_Deque_iterator<_Tp, _Tp&, _Tp*>, std::_Deque_iterator<_Tp, _Tp&, _Tp*>, std::_Deque_iterator<_Tp, _Tp&, _Tp*>)
move(_Deque_iterator<_Tp, _Tp&, _Tp*> __first,
^~~~
/usr/include/c++/7/bits/stl_deque.h:424:5: note: template argument deduction/substitution failed:
rect.cpp:3781:81: note: candidate expects 3 arguments, 1 provided
__pyx_v_self->colOfPoints = std::move<std::deque<Point> >(__pyx_v_area);
Below is the slightly simplifier version that I used to reproduce your issue. I'm just including it as an illustration of how to cut your example down further - note that I don't need to use C++ files - I can just include the code directly in the pyx file by putting it in the docstring.
#distutils: language = c++
from libcpp.deque cimport deque
cdef extern from *:
"""
#include <deque>
using std::deque;
class Point{};
deque<Point> getAllPoints() {
deque<Point> deq;
for (int i=0; i<10000; ++i) {
deq.push_back(Point{});
}
return deq;
}
"""
cdef cppclass Point:
pass
deque[Point] getAllPoints()
cdef extern from "<utility>" namespace "std" nogil:
T move[T](T)
cdef class PyRectange:
cdef deque[Point] colOfPoints
def something(self):
cdef deque[Point] area = self.getArea()
self.colOfPoints = move(area)
cdef deque[Point] getArea(self):
return getAllPoints()
The basic problem is that when Cython generates C++ code for templates it writes std::move<deque<Point>>(area) rather that std::move(area) and letting C++ deduce the template type. For reasons I don't fully understand this seems to generate the wrong code a lot of the time.
I have two and a half solutions:
Don't tell Cython that move is a template function. Instead just tell it about the overloads you want:
cdef extern from "<utility>" namespace "std" nogil:
deque[Point] move(deque[Point])
This is easiest I think and probably my approach.
If you avoid creating the temporary area then the C++ code does work with the template:
self.colOfPoints = move(self.getArea())
I suspect in this case you might not even need the move - C++ will probably automatically use the move assignment operator anyway.
This answer claimed to come up with a small wrapper helps Cython call the correct move (it also slightly missed the actually problem on the question it was answering...). I haven't tested it myself, but it might be worth a go if you want a templated move.
Cython equivalent of c define
#define myfunc(Node x,...) SetNode(x.getattributeNode(),__VA_ARGS__)
I have a c api SetNode which takes first argument a node of struct type node and N variables (N is variable number from 0-N)
here is a c example to solve such problum
exampleAPI.c
#include<stdarg.h>
float sumN(int len,...){
va_list argp;
int i;
float s=0;
va_start(argp,len);
for(i=0;i<len;i++){
s+=va_arg(argp,int);
}
va_end(argp);
}
exampleAPI.h
#include<stdarg.h>
float sumN(int len,...)
examplecode.c
#include<stdarg.h>
#include"exampleAPI.h"
int len(float first,...){
va_list argp;
int i=1;
va_start(argp,first);
while(1){
if(va_arg(argp,float)==NULL){
return i
}
else{
i++;
}
}
va_end(argp);
}
#define sum(...) sumN(len(__VA_ARGS__),__VA_ARGS__)
Now calling
sum(1,2,3,4);
will return 10.000000
sum(1.5,6.5);
will return 8.00000
I need a cython alternative for bellow c definition and not above example
because I have a C-API which has SetNode function which takes variable number of arguments and I want to wrap it in cython and call from python
#define myfunc(Node x,...) SetNode(x.getattributeNode(),__VA_ARGS__)
here Node is a class defined in cython which holds a c stuct as attribute and getattributeNode() is a function of Node class which returns c struct that needs to be passed into C-API.
cdef extern "Network.h":
ctypedef struct node_bn:
pass
node_bn* SetNode(node_bn* node,...)
cdef class Node:
cdef node_bn *node
cdef getattributeNode(self):
return self.node
def setNode(self,*arg):
self.node=SetNode(self.node,*arg) # Error cannot convert python objects to c type
Alternative thing I tried
cdef extern from "stdarg.h":
ctypedef struct va_list:
pass
ctypedef struct fake_type:
pass
void va_start(va_list, void* arg)
void* va_arg(va_list, fake_type)
void va_end(va_list)
fake_type int_type "int"
cdef extern "Network.h":
ctypedef struct node_bn:
pass
node_bn* VSetNode(node_bn* node,va_list argp)
cdef class Node:
cdef node_bn *node
cdef getattributeNode(self):
return self.node
cpdef _setNode(self,node_bn *node,...):
cdef va_list agrp
va_start(va_list, node)
self.node=VSetNode(node,argp)
va_end(va_list)
def setNode(self,*arg):
self._setNode(self.node,*arg)
works fine when argument list is empty
n = Node()
n.setNode() #This works
n.SetNode("top",1) # error takes exactly one argument given 3 in self._setNode(self.node,*arg)
If anyone could suggest cython equivalent to it, it would be great.
I don't think it's easily done though Cython (the problem is telling Cython what type conversions to do for an arbitrary number of arguments). The best I can suggest is to use the standard library ctypes library for this specific case and wrap the rest in Cython.
For the sake of an example, I've used a very simple sum function. va_sum.h contains:
typedef struct { double val; } node_bn;
node_bn* sum_va(node_bn* node,int len, ...);
/* on windows this must be:
__declspec(dllexport) node_bn* sum_va(node_bn* node,int len, ...);
*/
and va_sum.c contains:
#include <stdarg.h>
#include "va_sum.h"
node_bn* sum_va(node_bn* node,int len, ...) {
int i;
va_list vl;
va_start(vl,len);
for (i=0; i<len; ++i) {
node->val += va_arg(vl,double);
}
va_end(vl);
return node;
}
I've written it so it adds everything to a field in a structure just to demonstrate that you can pass pointers to structures without too much trouble.
The Cython file is:
# definition of a structure
cdef extern from "va_sum.h":
ctypedef struct node_bn:
double val;
# obviously you'll want to wrap things in terms of Python accessible classes, but this atleast demonstrates how it works
def test_sum(*args):
cdef node_bn input_node;
cdef node_bn* output_node_p;
input_node.val = 5.0 # create a node, and set an initial value
from ctypes import CDLL, c_double,c_void_p
import os.path
# load the Cython library with ctypes to gain access to the "sum_va" function
# Assuming you've linked it in when you build the Cython module
full_path = os.path.realpath(__file__)
this_file_library = CDLL(full_path)
# I treat all my arguments as doubles - you may need to do
# something more sophisticated, but the idea is the same:
# convert them to the c-type the function is expecting
args = [ c_double(arg) for arg in args ]
sum_va = this_file_library.sum_va
sum_va.restype = c_void_p # it returns a pointer
# pass the pointers as a void pointer
# my c compiler warns me if I used int instead of long
# but which integer type you have to use is likely system dependent
# and somewhere you have to be careful
output_node_p_as_integer = sum_va(c_void_p(<long>&input_node),len(args),
*args)
# unfortunately getting the output needs a two stage typecast - first to a long, then to a pointer
output_node_p = <node_bn*>(<long>(output_node_p_as_integer))
return output_node_p.val
You need to compile your va_sum.c together with your Cython file (e.g. by adding sources = ['cython_file.pyx','va_sum.c'] in setup.py)
Ctypes is probably a bit slower than Cython (I think there's a reasonable overhead on each call), and it's odd to mix them, but this should at least let you write the main wrapper in Cython, and use ctypes to get round the specific limitation.
This is probably not the proper answer, since I am not sure I understand the question fully. I would have replied in a comment, but the code formatting is too poor.
In Python the functions sum and len are available:
def my_len(*args):
return len(args)
def my_sum(*args):
return sum(args)
print "len =", my_len("hello", 123, "there")
print "sum =", my_sum(6.5, 1.5, 2.0)
outputs:
len = 3
sum = 10.0
I am trying to write a Cython wrapper around a C library. I am very new to Cython, so my apologies in advance if the problem is obvious.
In a file wrapper.pxd, I define a struct (reduced example):
cdef extern from "thiscouldbeyourlibrary.h":
cdef struct foo:
double **output
I then have a class:
cdef class Bar:
cdef wrapper.foo __stuff
cdef do_something(self):
self.__stuff.output = NULL
This fails:
Cannot convert 'void *' to Python object.
Apparently, Cython cannot be sure that self.__stuff.output is always a pointer. But I declared its type and the class is a "cdef" class, so I don't really understand why.
The problem is the incompatibility between NULL and double **. You could assign it to a char, int or void *, for example, doing:
wrapper.pyd:
cdef extern from "thiscouldbeyourlibrary.h":
cdef struct foo:
char a
int b
void *c
double **output
thiscouldbeyourlibrary.h:
struct foo
{
char a;
int b;
void *c;
double **output;
};
main.pyx:
cimport wrapper
cdef class Bar:
cdef wrapper.foo __stuff
def __init__(self):
self.__stuff.a = <char>NULL
self.__stuff.b = <int>NULL
self.__stuff.c = NULL
def main():
bar = Bar()
print bar.__stuff.a
print bar.__stuff.b
If you had allocated the memory for output before, you could have done it:
self.__stuff.output[0] = NULL
without allocating it will crash...