We Keep Coding

C++ to communicate to Python function

I am new to c++,
I created DLL which contains Class, and functions and all the return type of each function is PyObject (Python Object), So now i want to write the C++ application which loads DLL dynamically using LoadLibrary function.
Able to execute with adding the project to same solution and adding the reference to DLL.
I am able to load the DLL, but when i am calling the function it returns PyObject data type, how to store the return type of PyObject in C++?

You should take a look at Python's documentation on Concrete Objects Layer. Basically you have to convert PyObject into a C++ type using a function of the form Py*T*_As*T*(PyObject* obj) where T is the concrete type you want to retrieve.
The API assumes you know which function you should call. But, as stated in the doc, you can check the type before use:
...if you receive an object from a Python program and you are not sure that it has the right type, you must perform a type check first; for example, to check that an object is a dictionary, use PyDict_Check().
Here is an example to convert a PyObject into long:
PyObject* some_py_object = /* ... */;
long as_long(
PyLong_AsLong(some_py_object)
);
Py_DECREF(some_py_object);
Here is another, more complicated example converting a Python list into a std::vector:
PyObject* some_py_list = /* ... */;
// assuming the list contains long
std::vector<long> as_vector(PyList_Size(some_py_list));
for(size_t i = 0; i < as_vector.size(); ++i)
{
PyObject* item = PyList_GetItem(some_py_list, i);
as_vector[i] = PyLong_AsLong(item);
Py_DECREF(item);
}
Py_DECREF(some_py_list);
A last, more complicated example, to parse a Python dict into a std::map:
PyObject* some_py_dict = /* ... */;
// assuming the dict uses long as keys, and contains string as values
std::map<long, std::string> as_map;
// first get the keys
PyObject* keys = PyDict_Keys(some_py_dict);
size_t key_count = PyList_Size(keys);
// loop on the keys and get the values
for(size_t i = 0; i < key_count; ++i)
{
PyObject* key = PyList_GetItem(keys, i);
PyObject* item = PyDict_GetItem(some_py_dict, key);
// add to the map
as_map.emplace(PyLong_AsLong(key), PyString_AsString(item));
Py_DECREF(key);
Py_DECREF(item);
}
Py_DECREF(keys);
Py_DECREF(some_py_dict);

C++ Embedded Python and create_string_buffer

I'm a newbie in Python and embedding it too. And I have one problem:
There is function in my python module that recieves buffer created with ctypes.create_string_buffer(size) and fills it by content from some memory address:
def get_mem(self, address, size, data):
self.mem.read_ram_block(address, size, data)
How should I call this method with using of (char *) buffer? I want fill my C++ buffer with recieved from python.

If you only want to call the Python function ctypes.create_string_buffer(size), you could easily mirror the Python coding on the C++ side:
static PyObject* create_string_buffer(unsigned long size) {
PyObject *ctypes = PyImport_ImportModule("ctypes");
if (!ctypes) return 0;
PyObject *buf = PyObject_CallMethod(ctypes, "create_string_buffer", "k", size);
Py_DECREF(ctypes);
return buf;
}
If you'd like to use another type than unsigned long for the size, you'd need to change the format in PyObject_CallMethod as well. For example O is used for PyObject*. For a complete list of formats see the documentation for Building values.

Python C-extension class with a dynamic size

I try to write C-extension which contains python class.
This class takes a filename as a parameter fname of constructor, then loads that file to memory and stores as a config.
Below what I have:
typedef struct {
PyObject_HEAD
XmlConfig m_cfg;
} PyAgent;
static int PyAgent_init(PyAgent *self, PyObject *args) {
const char *fname;
if (!PyArg_ParseTuple(args, "s", &fname)) {
return NULL;
}
self->m_cfg.Load(fname);
return 0;
}
static PyTypeObject PyAgentType = {
PyObject_HEAD_INIT(NULL)
0, /* ob_size */
"classify.Agent", /* tp_name */
...
}
...
I get segmentation fault when I try to load file. As I understand it happens because PyAgent struct has object size that increases because of memory allocation for file.
I've tried to use PyObject_VAR_HEAD macro but it doesn't help.
Can you give a clue or working similar example?

Maybe self->m_cfg needs to be initialized first? You are calling a C++ method on it, but didn't call its constructor first. The struct is allocated by C code, not C++, so it doesn't know it needs to construct this field.
There are ways to manually call the constructor (something like new(&self->m_cfg) XmlConfig() if I remember correctly), but the easiest is probably to have instead a field XmlConfig *cfg, and allocate and free it as needed with the usual syntax of the C++ operators new and delete (the latter to be put in the tp_dealloc).

GDAL: Get pointer/handle of underlying C object

I have the following setup:
GDAL library with Python bindings (SWIG)
Some glue code (Python)
A C library, interfaced with ctypes
I want to pass the underlying dataset pointer/handle of the SWIG Dataset object to my C library. How can I retrieve this pointer?
I do not want to interface the C library with SWIG.

It was actually quite easy, and I hope that my solution is portable. Given, that my C function definition looks somewhat like this:
int myfunc(GDALDatasetH ds);
Then my ctypes definition is like this:
_lib = C.LibraryLoader(C.CDLL).LoadLibrary(lib_path)
_myfunc = _lib.myfunc
_myfunc.argtypes = [C.c_void_p]
_myfunc.restype = C.POINTER(C.c_char)
And I can call the C function with:
ds = gdal.Open(path)
...
_myfunc(C.c_void_p(long(ds.this)))

My reservation with the ctypes approach for this problem is that the reference count of the ds object is not incremented automatically and will become a bad pointer if it were to go out of scope.
A better approach would be to define a C python extension module that would manage the data reference counter.
I'm using a static PyObject * to hold the object, obviously a real implementation would store it more intelligently.
static PyObject * ds;
PyObject* GiveDsToC(PyObject * self, PyObject * args)
{
PyObject * pThis=NULL;
unsigned long addr;
if(!PyArg_ParseTuple(args, "O", &ds))
return NULL;
/* Ensure the interpreter keeps ds around while we have it */
Py_INCREF(ds);
pThis = PyObject_GetAttrString(ds, "this"); // new reference
addr = PyLong_AsLong(pThis); // convert using __int__ method
Py_DECREF(pThis); // Release the object back
CallSomeCFunction(addr);
Py_RETURN_NONE;
}
void FinishedWithDS(void)
{
// Lock the GIL and decrement the reference counter
PyGILState_STATE state = PyGILState_Ensure();
Py_DECREF(ds);
PyGILState_Release(state);
}

Boost::Python: How do I expose a dynamic non-object array to a PyBuf?

I'm working on a Computer Vision system with OpenCV in C++. I wrote a small GUI for it by using Boost::Python and PyQT4. Since I don't want to introduce QT to the C++ project, I need a way to expose Mat::data (an unsigned char * member) to Python in order to create a QImage there.
First I tried it like this:
class_<cv::Mat>("Mat", init<>())
.add_property("data_", make_getter(&Mat::data))
but then I got this in Python: "TypeError: No to_python (by-value) converter found for C++ type: unsigned char*"
I couldn't write a converter for it because a PyBuf of course needs to know its size.
So my next approach was trying to create a proxy object like this:
struct uchar_array {
uchar *data;
size_t size;
bool copied;
static const bool debug = true;
// copy from byte array
uchar_array(uchar *ptr, size_t size, bool copy) {
this->size = size;
this->copied = copy;
if(copied) {
data = new uchar[size];
memcpy(data, ptr, size);
} else {
data = ptr;
}
if(debug) LOG_ERR("init %d bytes in #%p, mem #%p", size, this, data);
}
PyObject *py_ptr() {
if(debug) LOG_ERR("py_ptr");
return boost::python::incref(PyBuffer_FromMemory(data, size));
}
~uchar_array() {
if(debug) LOG_ERR("~uchar_array #%p", this);
if(copied) {
if(debug) LOG_ERR("free #%p, mem #%p", this, data);
delete [] data;
}
}
};
And exposing this via a non-member method:
uchar_array *getMatData(Mat &mat) {
size_t size = mat.rows * mat.cols * mat.elemSize();
uchar_array *arr = new uchar_array(mat.data, size, true);
return arr;
}
class_<cv::Mat>("Mat", init<>())
.def("data", getMatData, with_custodian_and_ward_postcall<1, 0, return_value_policy<manage_new_object> >())
class_<uchar_array, shared_ptr<uchar_array> >("uchar_array", no_init)
.def("ptr", &uchar_array::py_ptr);
This works and gets me the buffer into Python, but there are two problems with this approach:
I now have to use mat.data().ptr(), it would be nicer to just do mat.data
When doing mat.data().ptr(), it seems the temporary uchar_array gets destructed immediately after calling ptr(), thus freeing the memory while I still want to use it
I did several experiments with custodian_and_ward and other stuff but got to a point where I stopped to understand this.
So, could anyone please tell me: What's the preferred way to export an unsigned char * to a PyBuf? In two variants, if possible: allocated for Python so should be freed by Python or as internal pointer where C++ frees it.

char* buffers are not really python friendly. On my project (which is not performance sensitive) I would use a std::vector or std::string, depending on what it was intended to contain. Both of these are nicely python friendly.
If you are not able to alter the underlying data structure, you can use add_property and a couple of getter and setter functions to convert data to a more convenient structure.