We Keep Coding

Embedding Python into C - can't import method from python module

I'm building C application which will be using Python plugins. When trying to call the method from another Python module, the function PyImport_ImportModule() seems to imports the module properly, then i try to get the function from this module using PyObject_GetAttrString() and all that I get is null.
I already tried using PyModule_GetDict() and PyDict_GetItemString() to get the method from the module, but the effect was the same.
main.c:
#include <stdio.h>
#include <python3.6/Python.h>
int main()
{
PyObject *arg, *pModule, *ret, *pFunc, *pValue, *pMethod, *pDict;
Py_Initialize();
PyObject *sys = PyImport_ImportModule("sys");
PyObject *path = PyObject_GetAttrString(sys, "path");
PyList_Append(path, PyUnicode_FromString("."));
pModule = PyImport_ImportModule("test");
if(pModule == NULL)
{
perror("Can't open module");
}
pMethod = PyObject_GetAttrString(pModule, "myfun");
if(pMethod == NULL)
{
perror("Can't find method");
}
ret = PyEval_CallObject(pMethod, NULL);
if(ret == NULL)
{
perror("Couldn't call method");
}
PyArg_Parse(ret, "&d", pValue);
printf("&d \n", pValue);
Py_Finalize();
return 0;
}
test.py:
def myfun():
c = 123 + 123
print('the result is: ', c)
myfun()
The result i got is:
Can't find method: Success
Segmentation fault (core dumped)
When I used the gdb debugger the output was:
pModule = (PyObject *) 0x7ffff5a96f48
pMethod = (PyObject *) 0x0

Your program is not wroking because the module being imported is the test built-in module, rather than your test.py script. This is because you are appending the current directory to sys.path, so it is checked after every other already existing path in the list. You should insert it at the beginning of the list instead, so that it is checked first.
This will work:
PyObject *sys = PyImport_ImportModule("sys");
PyObject *path = PyObject_GetAttrString(sys, "path");
PyList_Insert(path, 0, PyUnicode_FromString("."));
By the way, you should #include the Python header before anything else, as stated in the documentation:
Note: Since Python may define some pre-processor definitions which affect the standard headers on some systems, you must include Python.h before any standard headers are included.

Python/C++ : Embedding script in Qt/C++ application Error

I am attempting to embed a python script in a Qt/C++ project. In my Qt Project file, I have added the Python Framework and the Python library location.
INCLUDEPATH += "/Library/Frameworks/Python.framework/Versions/3.6/lib"
INCLUDEPATH += "/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m"
LIBS += -F"/Library/Frameworks/Python.framework/" -framework python
After reading the documentation, I am able to run a simple python script, using PyRun_SimpleFile(). I am now trying to run a script that takes an argument as input from C++ and returns an argument as output.
Using the documentation, I could write the following function :
const char *ss = "MainFile";
PyObject* name ;
name = PyUnicode_DecodeFSDefault(ss);
PyObject* pluginModule = PyImport_ImportModule("MainFile");
Py_DECREF(name);
if (!pluginModule)
{
PyErr_Print();
return "Error importing module";
}
PyObject* filterFunc = PyObject_GetAttrString(pluginModule, "attoMainFunction");
Py_DECREF(pluginModule);
if (!filterFunc)
{
PyErr_Print();
return "Error retrieving 'attoMainFunction'";
}
PyObject* args = Py_BuildValue("(s)", s.toStdString().c_str());
if (!args)
{
PyErr_Print();
Py_DECREF(filterFunc);
return "Error building args tuple";
}
PyObject* resultObj = PyObject_CallObject(filterFunc, args);
Py_DECREF(filterFunc);
Py_DECREF(args);
if (!resultObj)
{
PyErr_Print();
return "Error invoking 'filterFunc'";
}
const char* resultStr = PyUnicode_DecodeFSDefault(resultObj);
if (!resultStr)
{
PyErr_Print();
Py_DECREF(resultObj);
return "Error converting result to C string";
}
std::string result = resultStr;
Py_DECREF(resultObj);
return result;
When I run the above function, I get the following error :
Undefined symbols for architecture x86_64:
"_PyUnicode_DecodeFSDefault", referenced from:
MainWindow::pythonTry(QString&) in mainwindow.o
Am I missing linking a Python library with the project? Or is there anything wrong with the function I'm calling?
Thanks for your time.

Why am I getting this segfault when using the Python/C API?

I am getting a segmentation fault when decrefing a PyObject* in my C++ code using the Python/C API, and I can't figure out why. I am using C++ and Python 2.7. I am using new-style classes for future Python 3 compatibility.
My goal is to create a C++ class MyClass to serve as a wrapper for a class defined in a Python module. In the MyClass constructor, I pass in the name of the Python module, import the module, locate the class (which always has a pre-defined name PyClass), and call that class to create an instance of it. I then store the resulting PyObject* in MyClass for future use. In the MyClass destructor, I decref that stored PyObject* to avoid memory leaks.
I have already verified that everything is working correctly as far as locating the class and creating an instance of it. I have even verified that I can use the stored PyObject* in other MyClass methods, for example, to access methods in the PyClass. However, when the destructor does the decref, it causes a segfault.
Here is a sample of my code. I also call Py_Initialize() and Py_Finalize() elsewhere at appropriate times, and I have left out some of my error-checking code for brevity:
MyPythonModule.py
class PyClass:
pass
MyClass.h
class MyClass {
public:
MyClass(const char* modulename);
~MyClass();
private:
void* _StoredPtr;
};
MyClass.cpp
#include <Python.h>
#include <iostream>
#include "MyClass.h"
MyClass::MyClass(const char* modulename) {
_StoredPtr = NULL;
PyObject *pName = NULL, *pModule = NULL, *pAttr = NULL;
// Import the Python module.
pName = PyString_FromString(modulename);
if (pName == NULL) {goto error;}
pModule = PyImport_Import(pName);
if (pModule == NULL) {goto error;}
// Create a PyClass instance and store a pointer to it.
pAttr = PyObject_GetAttrString(pModule, "PyClass");
if (pAttr == NULL) {goto error;}
_StoredPtr = (void*) PyObject_CallObject(pAttr, NULL);
Py_DECREF(pAttr);
if (_StoredPtr == NULL) {goto error;}
error:
if (PyErr_Occurred()) {PyErr_Print();}
Py_XDECREF(pName);
Py_XDECREF(pModule);
return;
}
MyClass::~MyClass() {
std::cout << "Starting destructor..." << std::endl;
Py_XDECREF((PyObject*)(_StoredPtr));
std::cout << "Destructor complete." << std::endl;
}
I know that I could avoid the segfault by leaving out the Py_XDECREF() in the destructor, but I am afraid of causing a memory leak because I do not understand exactly why this is happening. It seems especially strange that I can use _StoredPtr successfully in other MyClass methods, yet I can't decref it.
I have also tried storing the PyObject* of the imported module in MyClass and holding on to it until after _StoredPtr is decrefed, but the _StoredPtr decref still segfaults. I tried commenting out the Py_DECREF(pAttr); line, but that doesn't help.
As I mentioned, I can retrieve methods in the PyClass using _StoredPtr, and I have also tried storing these in MyClass and decrefing them in the destructor. When I do this, I can decref _StoredPtr, but then it segfaults when I try to decref the method's PyObject*. If I do this with several methods, it is always the last decref that causes the segfault, no matter what order I put them in.
Any insights as to what's happening here?

This works for me
#include <Python.h>
#include <iostream>
#include "MyClass.h"
MyClass::MyClass(const char* modulename) {
_StoredPtr = NULL;
PyObject *pName = NULL, *pModule = NULL, *pAttr = NULL;
// Import the Python module.
pName = PyString_FromString(modulename);
if (pName == NULL) {goto error;}
pModule = PyImport_Import(pName);
if (pModule == NULL) {goto error;}
// Create a PyClass instance and store a pointer to it.
pAttr = PyObject_GetAttrString(pModule, "PyClass");
if (pAttr == NULL) {goto error;}
_StoredPtr = (void*) PyObject_CallObject(pAttr, NULL);
Py_DECREF(pAttr);
if (_StoredPtr == NULL) {goto error;}
else{
// do something with _StoredPtr
Py_XDECREF((*PyObject)_StoredPtr)
}
error:
if (PyErr_Occurred()) {PyErr_Print();}
Py_XDECREF(pName);
Py_XDECREF(pModule);
return;
}
MyClass::~MyClass() {}
I basically moved the XDECREF outside the destructor into the function that is using the PyObject.

I can create my C++ object in Python, but can't access methods

Sorry, there's a lot here, but I really think all of this information is relevant to the question I'm about to ask.
So I wrapped my C++ class using SWIG, compiled, etc. and in a python script I create an instance of my class and then try to call a method on that class. Notice I said "try" to call. The call on the method fails. Details on implementation & "fails" follows.
First, I am building the Python3.4 interpreter with my Windows application and calling my python script as follows:
Py_Initialize();
/* Some code here to set up python paths. */
.....
PyObject *pName = PyUnicode_FromString("hello");
if(pName == NULL)
{
Py_Finalize();
return false;
}
PyObject* pModule = PyImport_Import(pName);
if(pModule == NULL)
{
Py_Finalize();
return false;
}
/* "go" is the name of the function in my python script that I want to call. */
PyObject *pFunc = PyObject_GetAttrString(pModule, "go");
if(pFunc == NULL)
{
Py_Finalize();
return false;
}
PyObject *pArgs = PyTuple_New(0);
PyObject *pValue = PyObject_CallObject(pFunc, pArgs);
if(pValue == NULL)
{
Py_Finalize();
return false;
}
Py_Finalize();
return true;
===================
Here's my class structure:
class BoganDocument
{
private:
BoganMetadataSet myMetadata;
public:
// Constructor
BoganDocument();
// Destructor
virtual ~BoganDocument();
// Useful methods.
wstring getMetadataValue(wstring metadata_name);
}
===================
Here's my Python script (named "hello.py").
import BoganDocument
def go():
print("I'm in go()")
d = BoganDocument.BoganDocument()
print("I made a document")
ts = d.getMetadataValue("CreationTimestamp");
print("The creation timestamp is " + ts)
=====================
And the output on my console screen looks like this:
trying C:\Program Files (x86)\MyApp\Python\swig_runtime_data4_d.pyd
trying C:\Program Files (x86)\MyApp\Python\swig_runtime_data4.py
trying C:\Program Files (x86)\MyApp\Python\swig_runtime_data4.pyw
trying C:\Program Files (x86)\MyApp\Python\swig_runtime_data4.pyc
trying c:\MyApp\Workplace\swig_runtime_data4_d.pyd
trying c:\MyApp\Workplace\swig_runtime_data4.py
trying c:\MyApp\Workplace\swig_runtime_data4.pyw
trying c:\MyApp\Workplace\swig_runtime_data4.pyc
import 'BoganDocument' # <_frozen_importlib.SourceFileLoader object at 0x10D2E3F0>
import 'hello' # <_frozen_importlib.SourceFileLoader object at 0x10D2E1C0>
I'm in go()
I made a document
===================
Notice that I have set PYTHONVERBOSE to 11 to get as much diagnostic information as I can. And notice that all messages stop after "print("I made a document")". No syntax error, nothing. The value of pValue after "PyObject *pValue = PyObject_CallObject(pFunc, pArgs);" is NULL.
Now, the call to the constructor at "d = BoganDocument.BoganDocument()" is known to work, as I've set a breakpoint in the BoganDocument constructor and single stepped through.
I also set a breakbpoint in BoganDocument's getMetadataValue method but never get there. And there's the mystery.
No doubt I have a bug, but I haven't a clue.

Did you step into the SWIG wrapper for getMetadataValue? The C++ wrapper gets called when you do d.getMetadataValue() frm Python, the C++ wrapper then calls Bogan::getMetadataValue on the appropriate C++ instance. Since you never get to the C++ instance's getMetadataValue, maybe SWIG wrapper is finding the call to be illegal or such (SWIG doesn't normally silently ignore problems but I don't know what else to suggest). Since you are using wide string, make sure you include std_wstring.i.

How do I find out why importing failed with PyImportModule?

I have this code in a C application that's embedding Python (2.7.1):
{
PyObject *user_dict;
PyObject *user_func;
PyObject *result;
PyObject *header_tuple;
PyObject *original_recipients;
PyObject *working_recipients;
if (!Py_IsInitialized())
{
Py_Initialize();
}
if (!expy_exim_dict)
{
PyObject *module = Py_InitModule(expy_exim_module, expy_exim_methods); /* Borrowed reference */
Py_INCREF(module); /* convert to New reference */
expy_exim_dict = PyModule_GetDict(module); /* Borrowed reference */
Py_INCREF(expy_exim_dict); /* convert to New reference */
}
if (!expy_user_module)
{
if (expy_path_add)
{
PyObject *sys_module;
PyObject *sys_dict;
PyObject *sys_path;
PyObject *add_value;
sys_module = PyImport_ImportModule("sys"); /* New Reference */
if (!sys_module)
{
PyErr_Clear();
*return_text = "Internal error, can't import Python sys module";
log_write(0, LOG_REJECT, "Couldn't import Python 'sys' module");
return PYTHON_FAILURE_RETURN;
}
sys_dict = PyModule_GetDict(sys_module); /* Borrowed Reference, never fails */
sys_path = PyMapping_GetItemString(sys_dict, "path"); /* New reference */
if (!sys_path || (!PyList_Check(sys_path)))
{
PyErr_Clear(); /* in case sys_path was NULL, harmless otherwise */
*return_text = "Internal error, sys.path doesn't exist or isn't a list";
log_write(0, LOG_REJECT, "expy: Python sys.path doesn't exist or isn't a list");
return PYTHON_FAILURE_RETURN;
}
add_value = PyString_FromString(expy_path_add); /* New reference */
if (!add_value)
{
PyErr_Clear();
log_write(0, LOG_PANIC, "expy: Failed to create Python string from [%s]", expy_path_add);
return PYTHON_FAILURE_RETURN;
}
if (PyList_Append(sys_path, add_value))
{
PyErr_Clear();
log_write(0, LOG_PANIC, "expy: Failed to append [%s] to Python sys.path", expy_path_add);
}
Py_DECREF(add_value);
Py_DECREF(sys_path);
Py_DECREF(sys_module);
}
expy_user_module = PyImport_ImportModule(expy_scan_module); /* New Reference */
if (!expy_user_module)
{
PyErr_Clear();
/* Handle error */
}
}
When PyImport_ImportModule fails, it returns NULL. How can I find out why it failed to import? (e.g. when importing the module works find outside of the embedding).
(The code is part of py-exim-localscan, and I'm wanting to add more information about failures in the rare cases when they occur).

You do this by looking at the exception that was raised. Currently you wipe the exception (that's what PyErr_Clear() does.) Don't do that, and instead print the traceback or inspect the exception object. See http://docs.python.org/c-api/exceptions.html for information on how to do that from C code, but usually the best idea is to just let the exception propagate.