I have header (fingisdk.h) file like this:
#ifndef FINGISDK_H_
#define FINGISDK_H_
#include "fingienum.h"
#ifdef __cplusplus
extern "C" {
#endif
typedef void (*fingi_event)(FINGI_EVENT_ID eventId, char* msg);
FINGI_EVENT_ID start_fingi_sdk(char* ini_file_location);
FINGI_EVENT_ID download_room_info(char** roominfo);
void register_fingi_sdk_status_event_listener(fingi_event pointer_to_listener_fnc);
#ifdef __cplusplus
}
#endif
#endif
And then i have written a python Wrapper for this one:
#include <fingisdk.h>
#include <fingienum.h>
#include <boost/python.hpp>
BOOST_PYTHON_MODULE(libpythonWrapper)
{
using namespace boost::python;
def("start_fingi_sdk", start_fingi_sdk);
}
And python file for calling it is like this:
import libpythonWrapper
print libpythonWrapper.start_fingi_sdk('file_location.ini')
So far this works fine. However, i can not find out how to expose double the double pointer
function :
FINGI_EVENT_ID download_room_info(char** roominfo);
And callback function :
void register_fingi_sdk_status_event_listener(fingi_event pointer_to_listener_fnc);
Can anyone point me to some documentation or help me solve it
Thankyou
EDIT 1:
After messing around a bit, i figured out how to do the pointer to pointer function . Python does not support pointer , so have to wrap the download_room_info(char** roominfo) to return simple string:
std::string download_room_info_python_wrapper() {
char * value;
FINGI_EVENT_ID result = download_room_info(&value);
return std::string(value);
}
And then:
def("download_room_info", download_room_info_python_wrapper);
Still looking for solution for callback function
So you want to bind this API to Python:
typedef void (*fingi_event)(FINGI_EVENT_ID eventId, char* msg);
void register_fingi_sdk_status_event_listener(fingi_event);
That's a register function which takes a function which takes an eventId and a string. OK, let's define one of our own (in extern "C" of course, as the API is):
void my_callback(FINGI_EVENT_ID eventId, char* msg)
{
// do something
}
Now we can write a wrapper for the register function:
void my_register()
{
register_fingi_sdk_status_event_listener(my_callback);
}
And bind it:
def("my_register", my_register);
That should all work. But it's useless, because we didn't actually do anything in the callback. So this leads to a sub-question, which is how can you possibly do anything in the callback? One idea I have is that you should make your own function registration mechanism which lets you register a Python callback function into something like a global PyObject which will be set to a Python function and invoked using Boost Python:
boost::python::object g_callback;
void my_callback(FINGI_EVENT_ID eventId, char* msg)
{
if (g_callback)
g_callback(eventId, msg);
}
Now it's just a matter of letting the user assign the global callback:
void set_callback(boost::python::object func)
{
g_callback = func;
register_fingi_sdk_status_event_listener(my_callback);
}
def("set_callback", set_callback);
Then in Python:
def callback(eventId, msg):
print eventId, msg
set_callback(callback)
That should be about it, I think. This whole thing would be a ton easier if your callback API supported a "void* userData" argument like many callback APIs do. We'd then use that to store a PyObject or something useful. But the API lacks that, so we're stuck with a global somewhere to remember which function to call.
Related
I have a C++ library that manipulates (among other things) a list of wrappers that I have been working on converting to Python using pybind11. The rest of the library operates on a pointer to a list of pointers: std::list<Symbol*>*. The problem is that when attempting to autocast a Python list to this C++ list and then initializing a ParamMap, an object that holds the list on the C++ side, the pointers of the list get all messed up. Inspection in GDB reveals that the "next-object pointers" of all the objects are invalid, and this leads to segfaults when traversing the list.
There is no sign of the objects being deallocated on the C++ side, as neither the destructors for the list container ParamMap nor the list objects Symbol are called. I've deduced that the issue might be Python hyperactively deleting objects C++ is still using, but I've tried object terms like py::return_value_policy::reference and py::keep_alive, and they haven't fixed the problem. What is going wrong here? Unfortunately, changing the list type on the C++ side is not an option, but I would really appreciate some help in making this work on the Python side. Thank you!
Here is some minimal reproduction code:
Symbol.hpp
#include <string>
class Symbol {
private:
std::string val1;
int val2;
public:
Symbol(std::string con1, int con2) : val1(con1), val2(con2) {}
};
ParamMap.hpp
#include <list>
#include "Symbol.hpp"
class ParamMap {
private:
std::list<Symbol*>* lst;
int otherData;
public:
ParamMap(std::list<Symbol*>* symbolList, int dat) : lst(symbolList), otherData(dat) {}
std::list<Symbol*>* getSymbols() { return lst; }
int getOtherData() { return otherData; }
};
Query.cpp
#include <iostream>
#include "ParamMap.hpp"
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
namespace py = pybind11;
void getSymbolListSize(ParamMap* map) {
std::cout << "Entering query method" << std::endl;
auto sz = map->getSymbols()->size(); // SEGFAULT OCCURS WHEN GETTING SIZE
std::cout << "Got size successfully. Size = " << sz << std::endl;
}
PYBIND11_MODULE(list_test, handle) {
handle.def("getSymbolListSize", &getSymbolListSize);
py::class_<ParamMap>(handle, "ParamMap")
.def(py::init<std::list<Symbol*>*, int>(), py::keep_alive<1, 2>())
.def("getOtherData", &ParamMap::getOtherData)
.def("getSymbols", &ParamMap::getSymbols);
py::class_<Symbol>(handle, "Symbol")
.def(py::init<std::string, int>());
}
test.py
import list_test as p
# Creating a list of some random symbols
symbol_list = []
symbol1 = p.Symbol("Hello", 1)
symbol_list.append(symbol1)
symbol2 = p.Symbol("World", 2)
symbol_list.append(symbol2)
# Creating a parammap and passing it the symbol list
pm = p.ParamMap(symbol_list, 71)
print("Symbol list and ParamMap init'd successfully")
# Here, calling Query.cpp's only method
sz = p.getSymbolListSize(pm)
print(sz)
I don't know a lot about how pybind11 works its magic and therefore I can't help you understanding what is going on. However, I have the feeling that pybind attempts to build the list even though your code only uses a pointer to the list. If I were you I'd consider this a pybind bug and post it as an issue on their github page.
As per your code, doing something like this seems to work (although it's not very clean):
#include <list>
#include "Symbol.hpp"
class ParamMap {
private:
std::list<Symbol*>* lst;
int otherData;
public:
ParamMap(std::list<Symbol*> *symbolList, int dat) : lst(symbolList), otherData(dat) {
lst = new std::list<Symbol *>;
for(auto s : *symbolList) {
lst->push_back(s);
}
}
~ParamMap() {
delete lst;
}
std::list<Symbol*>* getSymbols() { return lst; }
int getOtherData() { return otherData; }
};
I don't know who's supposed to manage the lifetime of the pointed list, so you may want to remove the destructor in case someone else is supposed to deallocate the list.
I am using pybind11 to implement binds for my c++ project.
So, my problem is basically how to define a python function in the interpreter
and call it from the C++ code.
The C++ interface passes data using a pointer (double*) and I don't know how to code the function in the interpreter and how to convert it for a std::function to perform the evaluation:
// C++
//--------
double cpp_call( const std::array<double,N> &value, const std::function<double(double*)> &func)
{
return func(value.data());
}
// python binding with pybind11
// module definition...
...
m.def("py_call", &cpp_call);
//python interpreter
//-------------------
?
Please, could someone give some tip to me ?
You're most likely missing a couple of requires headers to get this working, #include <pybind11/functional.h> (for the std::function support) and #include <pybind11/stl.h> (for the stl container support); neither header is included by default (to keep the core project smaller).
With those, your example almost works (it just needs a const added to the inner argument of the std::function, i.e. const std::function<double(const double *)> &func: the std::array is const and thus its .data() returns a const pointer).
Here's a full example showing this working:
#include <pybind11/pybind11.h>
#include <pybind11/functional.h>
#include <pybind11/stl.h>
double cpp_call(const std::array<double, 3> &values,
const std::function<double(double *)> &func) {
double ret = 0;
for (auto d : values) ret += func(&d);
return ret;
}
PYBIND11_MODULE(stack92, m) {
m.def("sum", &cpp_call);
}
Python:
>>> import stack92
>>> def f(v): return v**.5
...
>>> print("1+2+3 =", stack92.sum([1, 4, 9], f))
1+2+3 = 6.0
I want to know the names of the NetworkInterfaces from python, but it seems it's not possible from python so I'm using this C code:
#include <Python.h>
#include <windows.h>
#include <iphlpapi.h>
#include <stdio.h>
#pragma comment(lib, "iphlpapi.lib")
PyObject* GetInterfaces (PyObject* self){
ULONG buflen = sizeof(IP_ADAPTER_INFO);
IP_ADAPTER_INFO *pAdapterInfo = (IP_ADAPTER_INFO *)malloc(buflen);
if (GetAdaptersInfo(pAdapterInfo, &buflen) == ERROR_BUFFER_OVERFLOW) {
free(pAdapterInfo);
pAdapterInfo = (IP_ADAPTER_INFO *)malloc(buflen);
}
if (GetAdaptersInfo(pAdapterInfo, &buflen) == NO_ERROR) {
for (IP_ADAPTER_INFO *pAdapter = pAdapterInfo; pAdapter; pAdapter = pAdapter->Next) {
printf("%s (%s)\n", pAdapter->IpAddressList.IpAddress.String, pAdapter->Description);
}
}
if (pAdapterInfo) free(pAdapterInfo);
return 0;
}
static char interfaces_docs[] =
"GetInterfaces( ): prints the interfaces name and IP\n";
static PyMethodDef interfaces_funcs[] = {
{"GetInterfaces", (PyCFunction)GetInterfaces,
METH_NOARGS, interfaces_docs},
{NULL}
};
void initinterfaces(void)
{
Py_InitModule3("interfaces", interfaces_funcs,
"Interfaces Module");
}
Is this good? And what are the steps to importing it into Python with ctypes? How can I do it? Also is there a way to return a list of tuples instead of printing it? Do I need to compile it? If I do how can I?
Is this good?
Almost. Never return 0/null as a PyObject* unless you're signaling an exception; instead incref Py_None and return it. And you may want to add actual error checking code as well.
And what are the steps to importing it into Python with ctypes? How can I do it?
What you've written doesn't need ctypes, since it's an actual Python module written in C. Use import interfaces after compiling the code to interfaces.pyd.
Also is there a way to return a list of tuples instead of printing it? Do I need to compile it? If I do how can I?
With the normal list and tuple functions; create the object and set each element in turn as required.
I want to wrap a quite long C function published by a scientist with Cython. I am following
http://scipy-lectures.github.io/advanced/interfacing_with_c/interfacing_with_c.html#interfacing-with-c
and I am able to replicate the examples with numpy support.
I am confused about what to do (or to do anything at all) about variables declared as extern in the library.cpp. Obviously, I will have a main.py that will call this function, not a main.cpp. How should I declare ini_flag and A and pass to func? Is there any other way than making them arguments to the function? If possible I would prefer not to edit original library.cpp file.
// main.cpp:
int ini_flag=0, *A;
void main(){
func(...)
...
}
// library.cpp:
extern int ini_flag, *A;
void func(...){
if (ini_flag==0){
// malloc and initialize A
ini_flag=1;
}
// use A and do other stuff
}
You should create a corresponding library.hpp file:
int ini_flag, *A;
void func(...)
And then in Cython:
cdef extern from "library.hpp":
int ini_flag, *A
void func(...)
Then you can then use them normally in Cython.
I implemented a bunch of functions and they are dispatched from the same C function called by the Python interpreter:
PyObject *
CmdDispatch(PyObject *self, PyObject *args, PyObject *kwargs)
Unexpectedly, self is NULL, and I need to get the function name currently being called. Is there any way to get this information?
I have dozens of functions which are all going through this routine. This command processes all of the options into a C++ map and passes it along to the implementation of each command.
Update:
http://docs.python.org/extending/extending.html#a-simple-example specifically says "The self argument points to the module object for module-level functions; for a method it would point to the object instance.", but I am getting passed null when linking against python 2.6.
I've been trying to solve very similar problem.
The conclusion I've come to suggests there is no way to determine name of current function or caller(s) at the level of Python C API. The reason being, Python interpreter puts on call stack only pure Python functions (implemented in Python itself). Functions implemented in C, regardless if registered in module methods table, are not put on Python stack, thus it's not possible to find them inspecting the stack frames.
Here is a quick example in Python illustrating what I wanted to achieve (I assume Juan asks for similar behaviour):
import sys
def foo():
print('foo:', sys._getframe(0).f_code.co_name)
def bar():
print('bar:', sys._getframe(0).f_code.co_name)
foo()
bar()
Here is close equivalent of this example (based on the Python 3 docs) but implemented using Python C API:
// Based on example from Python 3.2.1 documentation, 5.4. Extending Embedded Python
// http://docs.python.org/release/3.2.1/extending/embedding.html#extending-embedded-python
//
#include <Python.h>
#include <frameobject.h>
static void foo()
{
PyThreadState * ts = PyThreadState_Get();
PyFrameObject* frame = ts->frame;
while (frame != 0)
{
char const* filename = _PyUnicode_AsString(frame->f_code->co_filename);
char const* name = _PyUnicode_AsString(frame->f_code->co_name);
printf("foo: filename=%s, name=%s\n", filename, name);
frame = frame->f_back;
}
}
static void bar()
{
PyRun_SimpleString(
"import sys\n"
"print(\"bar: filename=%s, name=%s\" % (sys._getframe(0).f_code.co_filename, sys._getframe(0).f_code.co_name))"
);
}
static PyObject* emb_numargs(PyObject *self, PyObject *args)
{
foo();
bar();
PyRun_SimpleString(
"import sys\n"
"print(\"emb_numargs: filename=%s, name=%s\" % (sys._getframe(0).f_code.co_filename, sys._getframe(0).f_code.co_name))"
);
return PyLong_FromLong(0);
}
static PyMethodDef EmbMethods[] = {
{"numargs", emb_numargs, METH_VARARGS, "Return number 0"},
{NULL, NULL, 0, NULL}
};
static PyModuleDef EmbModule = {
PyModuleDef_HEAD_INIT, "emb", NULL, -1, EmbMethods,
NULL, NULL, NULL, NULL
};
static PyObject* PyInit_emb(void)
{
return PyModule_Create(&EmbModule);
}
int main(int argc, char* argv[])
{
PyImport_AppendInittab("emb", &PyInit_emb);
Py_Initialize();
PyRun_SimpleString(
"import emb\n"
"print('This is Zero: ', emb.numargs())\n"
);
Py_Finalize();
return 0;
}
I hope this completes Ned's answer too.
The Python api isn't built to tell you what function it is calling. You've created a function, and it is calling it, the API assumes you know what function you've written. You'll need to create a small wrapper function for each of your Python functions. The best way to do this is to register your one C function as one Python function that takes a string as its first argument. Then, in your Python layer, create as many Python functions as you need, each invoking your C function with the proper string argument identifying what function you really want to call.
Another alternative is to rethink the structure of your C code, and have as many functions as you need, each of which invokes your common helper code to process options, etc.
NOTICE error checking for API is not being provided for clarity;
This example insert a new function directly on python __builtin__ module, allowing to call the method without class.method schema. Just change mymodule to any other module as you wish.
PyObject* py_cb(PyObject *self, PyObject *args)
{
const char *name = (const char *) PyCObject_AsVoidPtr(self);
printf("%s\n", name);
Py_RETURN_NONE;
}
int main(int argc, char *argv)
{
PyObject *mod, *modname, *dict, *fnc, *usrptr;
const char *mymodule = "__builtin__";
PyMethodDef *m;
const char *method = "hello_python";
Py_Initialize();
mod = PyImport_ImportModule(mymodule);
modname = PyString_FromString(mymodule);
dict = PyModule_GetDict(mod);
m = (PyMethodDef *) calloc(1, sizeof(PyMethodDef));
m->ml_name = strdup(method);
m->ml_meth = py_cb;
usrptr = PyCObject_FromVoidPtr("I'm am the hello_python!", NULL);
fnc = PyCFunction_NewEx(m, usrptr, modname);
PyDict_SetItemString(dict, method, fnc);
...
When python script execute hello_python, the py_cb extension function will show:
I'm am the hello_python!
The self is used to send a real pointer such as the library context instead of this const char * of this example, this is now a matter of changing it to something interesting though.
I don't know if can be done directly from the C-API. At worst, you could call the traceback module from C, to get the name of the caller.