We Keep Coding

Python Method Call in a C++ Code through Python C-API

I am working on my Project which implies the use of Empirical Mode Decomposition in C++ for EEG Signals. The input Data is Eigen::MatrixXd, where the rows are the Channels and the columns are the samples.
I did not found a good C++ library for EMD so I want to use a Python one (dsatools). I have downloaded the package through Pip installer from the setup.py file on Xubuntu... so it's a system package now.
the problem is that the program can't read the module.
this is the code:
std::vector <Eigen::MatrixXd> DataAquisition::EMD (Eigen::MatrixXd array, int order, int iterations, int locality) {
std::vector <Eigen::MatrixXd> IMFs;
for (int i = 0; i < array.rows(); i++) {
Eigen::MatrixXd Kanals = array.row(i);
Eigen::MatrixXd IMFs_Cpp;
Py_Initialize();
//PyRun_SimpleString("from dsatools._base._imf_decomposition import * ");
PyObject* sys_path = PySys_GetObject("path");
PyObject* ProgrammName = PyUnicode_FromString("/home/user/Schreibtisch/mne-cpp-main/applications/mne_bci/MNE-BCI-QT/dsatools-master/dsatools/_base/_imf_decomposition/_emd.py");
PyList_Append(sys_path, ProgrammName);
PyObject* pModuleString = PyUnicode_FromString ((char*)"_emd.py");
PyObject* pModule = PyImport_Import(pModuleString);
PyObject* pFunction = PyObject_GetAttrString(pModule,(char*)"emd");
//PyObject* pDict = PyModule_GetDict(pModule);
//PyObject* pFunc = PyDict_GetItemString(pDict, (char*)"emd");
if (PyCallable_Check(pFunction))
{
PyObject* Signal = Py_BuildValue("(d)",(double*)Kanals.data());
PyObject* Order = Py_BuildValue("(i)",order);
PyObject* method = Py_BuildValue("(z)",(char*)"cubic");
PyObject* max_itter = Py_BuildValue("(i)",iterations);
PyObject* args = PyTuple_Pack(4,Signal,Order,method,max_itter);
PyErr_Print();
PyObject* IMFs_Py = PyObject_CallObject(pFunction,args);
PyErr_Print();
if (PyArray_Check(IMFs_Py))
std::cout << "EMD Output is NOT Array \n";
PyArrayObject *np_ret = reinterpret_cast <PyArrayObject*> (IMFs_Py);
int Rows = PyArray_SHAPE(np_ret)[0];
int Cols = PyArray_SHAPE(np_ret)[1];
double* c_out = reinterpret_cast<double*>(PyArray_DATA(np_ret));
Eigen::MatrixXd IMFs_Cpp = Eigen::Map <Eigen::MatrixXd> (c_out,Rows,Cols);
IMFs.push_back(IMFs_Cpp);
}
else
std::cout << "Python did not call the function \n";
Py_Finalize();
}
return IMFs;}
this is how the code in Python should look like and I just want to call the emd function:

Running opencv-python script hangs in sub-interpreter

I am hosting python interpreter(2.7.13) in a C application and running a python script that has some calls to opencv-python functions.
The problem is that when the script is run in the main interpreter, it works fine. But, when the script is run in a sub-interpreter, it hangs.
Is there any known incompatibility between opencv-python package and sub-interpreter ?
The C application code is as follows:-
const char *pythonSource = "abc_module";
const char *funcName = "abc_function";
Py_Initialize();
PyEval_InitThreads();
if(run_in_subinterpreter){
PyThreadState* threadState = Py_NewInterpreter();
PyThreadState *mainState = PyThreadState_Swap(threadState);
}
pName = PyString_FromString(pythonSource);
pModule = PyImport_Import(pName);
pDict = PyModule_GetDict(pModule);
pFunc = PyDict_GetItemString(pDict, funcName);
PyObject *pyList = PyList_New(gArray.size()); //std::vector<int> gArray;
for (size_t i = 0; i < gArray.size(); ++i)
{
PyObject *elem = PyInt_FromLong(gArray[i]);
PyList_SetItem(pyList, i, elem);
}
PyObject* pArgs = PyTuple_New(1);
PyTuple_SetItem(pArgs, 0, pyList);
pValue = PyObject_CallObject(pFunc, pArgs);
Py_Finalize();
return;
The python script code is as follows:-
import sys
import cv2
import numpy as np
def abc_function(list):
array = np.asarray(list)
array = np.reshape(array, (100, 200, 4)).astype(np.uint8)
array = abc_function_core(array)
return array
def abc_function_core(array):
# array is manipulated by calling the below functions
cv2.resize(....)
cv2.cvtColor(...)
cv2.equalizeHist(...)
cv2.CascadeClassifier.detectMultiScale(...)
cv2.rectangle(...)
return array

PyObject_CallMethod returns Null

I have an image processing application that I am using OpenCV within Python and then have it embedded in Embarcadero C++ XE6 IDE for Windows.
I followed all the great exampled and was able to get the embedded Python code working in the bare application. However when I try to use numpy (loadImage) I get the following error:
<type 'exception.AttributeError'>:'NoneType' object has no attribute 'dtype'
If I simply return out of the Python function with a True (commented out now) it returns a valid PyObject. If I try to average the intensity of the pixels with numpy it returns a NULL object.
I am thinking I don't have the imports setup correctly. The stand-alone Python application works as expected but not when it's embedded into my C++ application.
Python Code:
import numpy
import cv2
class Camera:
def __init__(self):
print 'OpenCV Version:', cv2.__version__
def loadImage(self):
'''
Load image from file
'''
global img
img = cv2.imread('cameraImageBlob.png',cv2.IMREAD_GRAYSCALE)
#return True
return numpy.average(img)
if __name__ == '__main__':
dc = Camera()
print dc.loadImage()
C++ Code:
#include "Python.h"
int main() {
PyObject *pName, *pModule, *pDict, *pFunc, *pValue, *pClass, *pInstance;
double dValue;
// Initilize the Python interpreter
Py_Initialize();
// Set runtime paths
PyRun_SimpleString("import sys");
PyRun_SimpleString("import numpy");
PyRun_SimpleString("import cv2");
// Build the name object - create a new reference
pName = PyString_FromString((char*)"OpenCVTest");
// Load the module object
pModule = PyImport_Import(pName);
if(pModule != NULL) {
//pDict is a borrowed reference so no DECREF
pDict = PyModule_GetDict(pModule);
// Get the Camera Class
pClass = PyDict_GetItemString(pDict, "Camera");
if(PyCallable_Check(pClass)) {
pInstance = PyObject_CallObject(pClass, NULL);
} else {
return;
}
// Load Image and get intensity
pValue = PyObject_CallMethod(pInstance, "loadImage", NULL);
if(pValue == NULL) {
GetError();
} else {
dValue = PyFloat_AsDouble(pValue);
}
}
Py_DecRef(pName);
Py_DecRef(pModule);
Py_DecRef(pValue);
Py_Finalize();
return 0;
}

Pass Variable from Python to C

I have Python Script embedded in C which I run in a thread. I need to pass the variable 'a' from the Python-Class 'Detect Motion' to my C program continuously. (Not as a return value)
I know I could do this with a fifo or something like that, but is there a way to pass it directly to C, maybe by calling a C function?
C:
#include <Python.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <pthread.h>
pthread_t mythread;
void *ThreadProc();
PyObject *pName, *pModule, *pDict, *pFunc, *pFunc2;
int main(int argc, char *argv[])
{
py_callback = PyCFunction_New(&callback_descr, NULL);
char *script = "motion";
char *functionUse = "get_values";
Py_Initialize();
pName = PyString_FromString(script);
pModule = PyImport_Import(pName);
// pDict and pFunc are borrowed references
pDict = PyModule_GetDict(pModule);
pFunc = PyDict_GetItemString(pDict, functionUse);
// POSIX code
pthread_create( &mythread, NULL, ThreadProc, NULL);
// Random testing code
for(int i = 0; i < 10; i++)
{
printf("Printed from the main thread.\n");
sleep(1);
}
printf("Main Thread waiting for My Thread to complete...\n");
// Join and wait for the created thread to complete...
// POSIX code
pthread_join(mythread, NULL);
printf("Main thread finished gracefully.\n");
return 0;
}
void *ThreadProc()
{
if (PyCallable_Check(pFunc))
{
PyObject_CallObject(pFunc, NULL);
}
else {
PyErr_Print();
}
// Clean up
Py_DECREF(pModule);
Py_DECREF(pName);
Py_Finalize();
printf("My thread is finishing...\n");
}
Python:
import numpy as np
import picamera
import picamera.array
class DetectMotion(picamera.array.PiMotionAnalysis):
def analyse(self, a):
a = np.sqrt(
np.square(a['x'].astype(np.float)) +
np.square(a['y'].astype(np.float))
).clip(0, 255).astype(np.uint8)
# If there're more than 10 vectors with a magnitude greater
# than 60, then say we've detected motion
print a
if (a > 60).sum() > 10:
print 'Motion detected!'
def get_values():
with picamera.PiCamera() as camera:
with DetectMotion(camera) as output:
camera.resolution = (640, 480)
camera.start_preview()
camera.start_recording(
'/dev/null', format='h264', motion_output=output)
camera.wait_recording(10)
camera.stop_recording()
camera.stop_preview()

Python callback from C++

I have a C++ class
class EventHandler {
virtual long readFromDaemon(void *buf, size_t count) = 0;
};
and a Python program that uses it
class Handler(EventHandler):
def readFromDaemon(self, buf, count):
...
C++ code calls EventHandler::readFromDaemon().
SWIG converts arguments and calls Python's readFromDaemon():
long SwigDirector_EventHandler::readFromDaemon(void *buf, size_t count) {
...
swig::SwigVar_PyObject obj0;
obj0 = SWIG_NewPointerObj(SWIG_as_voidptr(buf), SWIGTYPE_p_void, 0 );
swig::SwigVar_PyObject obj1;
obj1 = SWIG_From_size_t(static_cast< size_t >(count));
...
swig::SwigVar_PyObject swig_method_name = SWIG_Python_str_FromChar((char *)"readFromDaemon");
swig::SwigVar_PyObject result = PyObject_CallMethodObjArgs(swig_get_self(), (PyObject *) swig_method_name ,(PyObject *)obj0,(PyObject *)obj1, NULL);
I want to convert (void *buf, size_t count) to PyBytesObject
def readFromDaemon(self, buf):
# buf is bytes
But I didn't found a way how to achieve it.
%typemap(in) (void *buf, size_t count) { ... } does not help.

Here's an example of reading and writing to C++ data buffers. the C++ functions in my example are:
void read(void* buf, size_t count);
void write(const void* buf, size_t count);
SWIG interface file:
%module x
%include <exception.i>
// Handle an input-only (const) buffer.
// A single Python input is translated into a buffer and a size.
%typemap(in) (const void *buf, size_t count) (Py_ssize_t tmp) %{
if(PyBytes_AsStringAndSize($input,(char**)&$1,&tmp) == -1)
return NULL;
$2 = tmp;
%}
// Handle an output-only (non-const) buffer.
// The single integer Python input is allocated in a temporary
// buffer and the pointer and its size are created as C++ parameters.
%typemap(in,numinputs=1) (void *buf, size_t count) (long tmp) %{
if(!PyLong_Check($input))
SWIG_exception(SWIG_TypeError,"expected integer");
tmp = PyLong_AsLong($input);
if(tmp < 1 || tmp > 65535)
SWIG_exception(SWIG_ValueError,"expected value 1-65535");
$2 = tmp;
$1 = new char[$2];
%}
// The pair of output arguments are translated into a single
// Python bytes object and appended to any existing return value.
%typemap(argout) (void *buf, size_t count) (PyObject* po) %{
po = PyBytes_FromStringAndSize((char*)$1,$2);
$result = SWIG_Python_AppendOutput($result,po);
%}
// Free the temporary buffer created in the "in" typemap.
%typemap(freearg) (void* buf, size_t count) %{
delete $1;
%}
// SWIG will wrap these two functions prototypes.
void read(void* buf, size_t count);
void write(const void* buf, size_t count);
// Implementation
%{
#include <iostream>
void write(const void* buf, size_t count)
{
char* tmp = (char*)buf;
for(size_t i = 0; i < count; i++)
std::cout << i << ": " << (int)tmp[i] << std::endl;
}
void read(const void* buf, size_t count)
{
char* tmp = (char*)buf;
for(size_t i = 0; i < count; i++)
tmp[i] = (char)i;
}
%}
Demo:
>>> import x
>>> x.read(10)
b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t'
>>> x.read(-1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: expected value 1-65535
>>> x.read(1000000)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: expected value 1-65535
>>> x.write(b'abcdefg')
0: 97
1: 98
2: 99
3: 100
4: 101
5: 102
6: 103
>>> x.write(12)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: expected bytes, int found

I found a decision in another topic here.
%typemap(directorin) (void *buf, size_t count) {
$input = PyBytes_FromStringAndSize(static_cast<char*>($1), $2);
}
But there is another problem. The memory buffer is copied from C++ to Python object. How to copy bytes back from python object to C++? There is memoryview since python 2.7, but what to do with 2.6 and olders versions?
This is C++ code that calls EventHandler::readFromDaemon, for clarity
std::vector<char> buf(maxlen);
long cnt = m_evt_handler->readFromDaemon(&buf[0], buf.size());
Here is wrapper code now
long SwigDirector_EventHandler::readFromDaemon(void *buf, size_t count) {
long c_result;
swig::SwigVar_PyObject obj0;
{
obj0 = PyBytes_FromStringAndSize(static_cast<char*>(buf), count);
}
...