We Keep Coding

Embedding Python into C - can't import method from python module

I'm building C application which will be using Python plugins. When trying to call the method from another Python module, the function PyImport_ImportModule() seems to imports the module properly, then i try to get the function from this module using PyObject_GetAttrString() and all that I get is null.
I already tried using PyModule_GetDict() and PyDict_GetItemString() to get the method from the module, but the effect was the same.
main.c:
#include <stdio.h>
#include <python3.6/Python.h>
int main()
{
PyObject *arg, *pModule, *ret, *pFunc, *pValue, *pMethod, *pDict;
Py_Initialize();
PyObject *sys = PyImport_ImportModule("sys");
PyObject *path = PyObject_GetAttrString(sys, "path");
PyList_Append(path, PyUnicode_FromString("."));
pModule = PyImport_ImportModule("test");
if(pModule == NULL)
{
perror("Can't open module");
}
pMethod = PyObject_GetAttrString(pModule, "myfun");
if(pMethod == NULL)
{
perror("Can't find method");
}
ret = PyEval_CallObject(pMethod, NULL);
if(ret == NULL)
{
perror("Couldn't call method");
}
PyArg_Parse(ret, "&d", pValue);
printf("&d \n", pValue);
Py_Finalize();
return 0;
}
test.py:
def myfun():
c = 123 + 123
print('the result is: ', c)
myfun()
The result i got is:
Can't find method: Success
Segmentation fault (core dumped)
When I used the gdb debugger the output was:
pModule = (PyObject *) 0x7ffff5a96f48
pMethod = (PyObject *) 0x0

Your program is not wroking because the module being imported is the test built-in module, rather than your test.py script. This is because you are appending the current directory to sys.path, so it is checked after every other already existing path in the list. You should insert it at the beginning of the list instead, so that it is checked first.
This will work:
PyObject *sys = PyImport_ImportModule("sys");
PyObject *path = PyObject_GetAttrString(sys, "path");
PyList_Insert(path, 0, PyUnicode_FromString("."));
By the way, you should #include the Python header before anything else, as stated in the documentation:
Note: Since Python may define some pre-processor definitions which affect the standard headers on some systems, you must include Python.h before any standard headers are included.

Passing an argument to Python C API function call

I am trying to call a python function from C++. However, I could not figure out how to pass an argument to python function call.
I have a python file named "arbName.py" and a c++ file named "main.cpp".
arbName.py
def someFunction(a):
return a
main.cpp
#include <Python.h>
int main()
{
Py_Initialize();
//add . to path
PyRun_SimpleString("import sys");
PyRun_SimpleString("sys.path.append(\".\")");
PyObject* myModuleString = PyUnicode_FromString((char*)"arbName");
PyObject* myModule = PyImport_Import(myModuleString);
PyObject* myFunction = PyObject_GetAttrString(myModule,(char*)"someFunction");
//couldn't figure out this part.
//How to declare a python variable and/or pass it to the python function.
PyObject* args = PyUnicode_FromString("1");
PyObject *myResult = PyObject_CallFunctionObjArgs(myFunction, args);
return 0;
}
EDIT: I was able to do it by changing the last two lines with
PyObject* args = Py_BuildValue("(d)", 2.0);
PyObject *myResult = PyObject_CallObject(myFunction, args);

Python C Extension - Error in `python3.6': free(): invalid pointer: 0x

I've been learning about extending Python with C and I finally got it to work in some way, but I'm getting this really weird error. I made a test module with a single function to test if a number is prime. Here is the code:
cmodule.c
#include <stdbool.h>
#include <math.h>
#include <Python.h>
bool isprime(int n) {
if (n == 0 || n == 1)
return false;
for (int i = 2; i < (int)sqrt(n) + 1; i++) {
if (n % i == 0)
return false;
}
return true;
}
static PyObject * isprime_wrapper(PyObject * self, PyObject * args) {
int n;
if (!PyArg_ParseTuple(args, "i", &n))
return NULL;
bool retval = isprime(n);
if (retval)
return Py_True;
else
return Py_False;
}
static PyMethodDef methods[] = {
{"isprime", isprime_wrapper, METH_VARARGS, "Tests if a number is prime."},
{NULL, NULL, 0, NULL}
};
static struct PyModuleDef cmodule = {
PyModuleDef_HEAD_INIT,
"cmodule",
"Test module.",
-1,
methods
};
PyMODINIT_FUNC PyInit_cmodule(void) {
return PyModule_Create(&cmodule);
}
setup.py
from distutils.core import setup, Extension
cmodule = Extension("cmodule", sources = ["cmodule.c"])
setup(name = "CModuleTest",
ext_modules = [cmodule])
At first glance it works fine. I run python3.6 setup.py build and it builds with no errors, and I can go into the directory with the .so file, start up a Python session, and call cmodule.isprime() fine. However, the weird part is when I use the code in a loop, where I get this error:
>>> import cmodule
>>> for i in range(1, 1000001):
... n = cmodule.isprime(i)
...
*** Error in `python3.6': free(): invalid pointer: 0x0000561cc1faf620 ***
I can add the full backtrace if anyone would find it helpful. I printed out all the values of i while doing this, and it seemed to always be getting this error when it tries to run isprime(154). However, when I just open up Python in the terminal and try running isprime(154) or any other number by itself, it works fine. The only time I've been able to replicate this is in the loop, and I have no clue where it could be coming from. I've looked at other people that have had similar problems and it seemed to stem from them using free() in their C code improperly, but I never call free() from my code. Does anyone have an idea what could be causing this? Thanks for any help.

CPython objects are referenced counted and returning Py_True and Py_False could be the issue. Use Py_RETURN_TRUE, etc. instead. They properly increment the reference count of and return their respective singleton objects.

How to add dynamically C function in embedded Python

I declare a C function as Python prototype
static PyObject* MyFunction(PyObject* self, PyObject* args)
{
return Py_None ;
}
Now I want to add it into a dynamically loaded module
PyObject *pymod = PyImport_ImportModule("mymodule");
PyObject_SetAttrString( pymod, "myfunction", ? );
How to convert C function into PyObject callable ?

You need to construct a new PyCFunctionObject object from the MyFunction. Usually this is done under the hood using the module initialization code, but as you're now doing it the opposite way, you need to construct the PyCFunctionObject yourself, using the undocumented PyCFunction_New or PyCFunction_NewEx, and a suitable PyMethodDef:
static PyMethodDef myfunction_def = {
"myfunction",
MyFunction,
METH_VARARGS,
"the doc string for myfunction"
};
...
// Use PyUnicode_FromString in Python 3.
PyObject* module_name = PyString_FromString("mymodule");
if (module_name == NULL) {
// error exit!
}
// this is adapted from code in code in
// Objects/moduleobject.c, for Python 3.3+ and perhaps 2.7
PyObject *func = PyCFunction_NewEx(&myfunction_def, pymod, module_name);
if (func == NULL) {
// error exit!
}
if (PyObject_SetAttrString(module, myfunction_def.ml_name, func) != 0) {
Py_DECREF(func);
// error exit!
}
Py_DECREF(func);
Again, this is not the preferred way to do things; usually a C extension creates concrete module objects (such as _mymodule) and mymodule.py would import _mymodule and put things into proper places.

Calling python object's method from c++

I am trying to achieve the following: passing a python object to a c++ callback chain (which are typical in many popular c++ libraries). In the c++ code, callbacks pass on objects that have necessary information for consecutive callbacks in the cascade/chain.
Here is a small test code I wrote: we pass a python object to a c routine (case 1) and call it's method. That works ok. But when I pass the python object to a c++ object and try to call it "inside" the c++ object, I get segfault.. :(
Here it goes:
c++ module ("some.cpp"):
#include <stdint.h>
#include <iostream>
#include <Python.h>
/* objective:
* create c++ objects / routines that accept python objects
* then call methods of the python objects inside c++
*
* python objects (including its variables and methods) could be passed along, for example in c++ callback chains ..
* .. and in the end we could call a python callback
*
* Compile and test as follows:
* python setup.py build_ext
* [copy/link some.so where test.py is]
* python test.py
*
*/
class testclass {
public:
testclass(int* i, PyObject* po) {
std::cerr << "testclass constructor! \n";
i=i; po=po;
}
~testclass() {}
void runpo() {
PyObject* name;
const char* mname="testmethod";
name=PyString_FromString(mname);
std::cerr << "about to run the python method .. \n";
PyObject_CallMethodObjArgs(po, name, NULL);
std::cerr << ".. you did it - i will buy you a beer!\n";
}
public:
int* i;
PyObject* po;
};
/* Docstrings */
static char module_docstring[] = "hand-made python module";
/* Available functions */
static PyObject* regi_wrapper(PyObject * self, PyObject * args);
void regi(int* i, PyObject* po);
/* Module specification */
static PyMethodDef module_methods[] = {
{"regi_wrapper",regi_wrapper, METH_VARARGS, "lets see if we can wrap this sucker"},
{NULL, NULL, 0, NULL}
};
/* Initialize the module */
PyMODINIT_FUNC initsome(void)
{
PyObject *m = Py_InitModule3("some", module_methods, module_docstring);
if (m == NULL)
return;
// import_array(); // numpy not required here ..
}
static PyObject* regi_wrapper(PyObject * self, PyObject * args)
{
int* input_i; // whatever input variable
PyObject* input_po; // python object
PyObject* ret; // return variable
// parse arguments
if (!PyArg_ParseTuple(args, "iO", &input_i, &input_po)) {
return NULL;
}
// https://stackoverflow.com/questions/16606872/calling-python-method-from-c-or-c-callback
// Py_INCREF(input_po); // need this, right? .. makes no difference
/* // seems not to make any difference ..
PyGILState_STATE gstate;
gstate = PyGILState_Ensure();
*/
regi(input_i, input_po);
// PyGILState_Release(gstate); // .. makes no difference
// Py_DECREF(input_po); // .. makes no difference
Py_RETURN_TRUE;
}
void regi(int* i, PyObject* po) {
// search variable and methods from PyObject "po" and call its methods?
PyObject* name;
const char* mname="testmethod";
testclass* testobj;
testobj=new testclass(i,po);
/* [insert // in front of this line to test case 1]
// ***** this one works! *********
name=PyString_FromString(mname);
PyObject_CallMethodObjArgs(po, name, NULL);
*/ // [insert // in front of this line to test case 1]
// *** I WOULD LIKE THIS TO WORK *** but it gives segfault.. :(
testobj->runpo(); // [uncomment to test case 2]
}
setup.py:
from distutils.core import setup, Extension
# the c++ extension module
extension_mod = Extension("some", ["some.cpp"])
setup(name = "some", ext_modules=[extension_mod])
test.py:
import some
class sentinel:
def __init__(self):
pass
def testmethod(self):
print "hello from sentinel.testmethod"
pass
se=sentinel()
some.regi_wrapper(1,se)
This question seems relevant:
Calling python method from C++ (or C) callback
.. however the answer did not help me.
What am I missing/misunderstanding here (my c++ sucks big time, so I might have missed something obvious) .. ?
Also, some bonus questions:
a) I am familiar with swig and swig "directors".. however, I would like to use swig for general wrapping of the code, but my custom wrapping for the sort of things described in this question (i.e. without directors). Is there any way to achieve this?
b) Any other suggestions to achieve what I am trying to achieve here, are highly appreciated.. is this possible or just pure insanity?

Using in the constructor
po=this->po
solves the "issue". Sorry for the spam! I will leave here this thing as an example.. maybe someone finds it useful.