I'm building C application which will be using Python plugins. When trying to call the method from another Python module, the function PyImport_ImportModule() seems to imports the module properly, then i try to get the function from this module using PyObject_GetAttrString() and all that I get is null.
I already tried using PyModule_GetDict() and PyDict_GetItemString() to get the method from the module, but the effect was the same.
main.c:
#include <stdio.h>
#include <python3.6/Python.h>
int main()
{
PyObject *arg, *pModule, *ret, *pFunc, *pValue, *pMethod, *pDict;
Py_Initialize();
PyObject *sys = PyImport_ImportModule("sys");
PyObject *path = PyObject_GetAttrString(sys, "path");
PyList_Append(path, PyUnicode_FromString("."));
pModule = PyImport_ImportModule("test");
if(pModule == NULL)
{
perror("Can't open module");
}
pMethod = PyObject_GetAttrString(pModule, "myfun");
if(pMethod == NULL)
{
perror("Can't find method");
}
ret = PyEval_CallObject(pMethod, NULL);
if(ret == NULL)
{
perror("Couldn't call method");
}
PyArg_Parse(ret, "&d", pValue);
printf("&d \n", pValue);
Py_Finalize();
return 0;
}
test.py:
def myfun():
c = 123 + 123
print('the result is: ', c)
myfun()
The result i got is:
Can't find method: Success
Segmentation fault (core dumped)
When I used the gdb debugger the output was:
pModule = (PyObject *) 0x7ffff5a96f48
pMethod = (PyObject *) 0x0
Your program is not wroking because the module being imported is the test built-in module, rather than your test.py script. This is because you are appending the current directory to sys.path, so it is checked after every other already existing path in the list. You should insert it at the beginning of the list instead, so that it is checked first.
This will work:
PyObject *sys = PyImport_ImportModule("sys");
PyObject *path = PyObject_GetAttrString(sys, "path");
PyList_Insert(path, 0, PyUnicode_FromString("."));
By the way, you should #include the Python header before anything else, as stated in the documentation:
Note: Since Python may define some pre-processor definitions which affect the standard headers on some systems, you must include Python.h before any standard headers are included.
Related
I'm trying to embed Python in a C++ application, and I need it to run as a release build, as I am only interested in debugging the C++ code. Also, I do not have the _d debug versions of all libraries I need. I am using Python 3.7.0, in MSVC 2017 with C++11, the library I don't have debug symbols for is VTK, which I installed through a wheel file supplied by my employer. If I try to build the Python wrapper myself I get another load of issues, so I am unable to build the debug files. If I run in Debug mode, I am unable to import the library, whereas if I run in Release, I get further issues:
Exception thrown at 0x00007FF90841DBC9 (python37_d.dll) in PythonEmbedding.exe: 0xC0000005: Access violation reading location 0x0000000000000025.
The C++ code:
int main(int argc, char *argv[])
{
PyObject *pIntrospector = NULL;
if (PyVtk_InitIntrospector(pIntrospector) == OK)
{
printf("Initialization succeeded\n");
}
vtkObjectBase *pConeSource = NULL;
if (PyVtk_CreateVtkObject(pIntrospector, "vtkConeSource", pConeSource) == OK)
{
printf("Object creation succeeded\n");
}
return 0;
}
int PyVtk_InitIntrospector(
PyObject *pIntrospector)
{
/* Activating virtual environment */
#ifdef _DEBUG
// For Visual Studio debug builds
const wchar_t *sPyHome = L"venv-dbg";
#else
// For release builds
const wchar_t *sPyHome = L"venv";
#endif
Py_SetPythonHome(sPyHome);
/* Initializing Python environment and setting PYTHONPATH. */
Py_Initialize();
PyRun_SimpleString("import sys\nimport os");
PyRun_SimpleString("sys.path.append( os.path.dirname(os.getcwd()) )");
PyRun_SimpleString("sys.path.append(\".\")");
PyRun_SimpleString("import importlib.machinery as m");
PyRun_SimpleString("print(m.all_suffixes())");
/* Decode module from its name. Returns error if the name is not decodable. */
PyObject *pIntrospectorModuleName = PyUnicode_DecodeFSDefault("Introspector");
if (pIntrospectorModuleName == NULL)
{
fprintf(stderr, "Fatal error: cannot decode module name\n");
return PYTHON_INTROSPECTION_STRING_DECODE_ERROR;
}
/* Imports the module previously decoded. Returns error if the module is not found. */
PyObject *pIntrospectorModule = PyImport_Import(pIntrospectorModuleName);
if (pIntrospectorModule == NULL)
{
if (PyErr_Occurred())
{
PyErr_Print();
}
fprintf(stderr, "Failed to load \"Introspector\"\n");
Py_DECREF(pIntrospectorModuleName);
return PYTHON_INTROSPECTION_MODULE_LOAD_ERROR;
}
/* Looks for the Introspector class in the module. If it does not find it, returns and error. */
PyObject* pIntrospectorClass = PyObject_GetAttrString(pIntrospectorModule, "Introspector");
if (pIntrospectorClass == NULL || !PyCallable_Check(pIntrospectorClass))
{
if (PyErr_Occurred())
{
PyErr_Print();
}
fprintf(stderr, "Cannot find class \"Introspector\"\n");
if (pIntrospectorClass != NULL)
{
Py_DECREF(pIntrospectorClass);
}
Py_DECREF(pIntrospectorModuleName);
Py_DECREF(pIntrospectorModule);
return PYTHON_INTROSPECTION_CLASS_NOT_FOUND_ERROR;
}
/* Instantiates an Introspector object. If the call returns NULL there was an error
creating the object, and thus it returns error. */
pIntrospector = PyObject_CallObject(pIntrospectorClass, NULL);
if (pIntrospector == NULL)
{
if (PyErr_Occurred())
{
PyErr_Print();
}
fprintf(stderr, "Introspector instantiation failed\n");
Py_DECREF(pIntrospectorModuleName);
Py_DECREF(pIntrospectorModule);
Py_DECREF(pIntrospectorClass);
return PYTHON_INTROSPECTION_OBJECT_CREATION_ERROR;
}
/* Decreasing reference to local data. */
Py_DECREF(pIntrospectorModuleName);
Py_DECREF(pIntrospectorModule);
Py_DECREF(pIntrospectorClass);
return OK;
}
I have not added the code to the PyVtk_CreateVtkObject function as it won't enter it, but if I do not add the calls after PyVtk_InitIntroepsctor it won't give the aforementioned error. Finally, if I import Introspector in the Python interpreter myself, it works fine.
Is there a solution to either run it in Debug or Release? I cannot wrap my head around it...
P.S.: I already tried to use Boost::Python, I have two issues open on it as it is giving me problems as well.
Update 1: In particular, the excpetion is thrown when I do this:
PyObject *pIntrospectorModule = PyImport_Import(pIntrospectorModuleName);
Update 2: I have further scoped down the issue to this: whenever I import the vtk package from within the embedded interpreter, it throws the Access Violation on this code:
// Add special attribute __vtkname__
PyObject *s = PyString_FromString(classname);
PyDict_SetItemString(pytype->tp_dict, "__vtkname__", s);
Py_DECREF(s); // <-- In particular on this Py_DECREF
If I try to import anything else, there is no issue, it seems.
I am trying to embed python script into c++ project.
Below is what I have tried so far.
#include<iostream>
#include <Python.h>
int
main()
{
Py_Initialize();
PyObject* sysPath = PySys_GetObject("path");
PyObject* modPath = PyBytes_FromString("C:\\Users\\naal\\Documents\\Visual Studio 2017\\Projects\\Project1\pyscripts");
int result = PyList_Insert(sysPath,0, modPath);
PyObject *pModule = PyImport_ImportModule("myscript2");
printf("%p\n", pModule);
return 0;
}
below is the python script "myscript2.py"
def find_me():
print("hey you found me")
The problem is, the main module is not able to find the python script i.e object pyModule is always NULL, no matter how I change python script path.
What am I doing wrong ?
I ended up implementing this in another way.
#include<iostream>
#include <Python.h>
int main() {
std::string location = "C:\\Users\\myscript.py";
const char* file_location = location.c_str(); FILE* file_pointer;
Py_Initialize();
file_pointer = _Py_fopen(file_location, "r");
PyRun_SimpleFile(file_pointer, file_location);
Py_Finalize();
return 1;
}
The above seemed to work. I still don't know why the SYSPATH idea originially mentioned in the question didn't work.
I am getting a segmentation fault when decrefing a PyObject* in my C++ code using the Python/C API, and I can't figure out why. I am using C++ and Python 2.7. I am using new-style classes for future Python 3 compatibility.
My goal is to create a C++ class MyClass to serve as a wrapper for a class defined in a Python module. In the MyClass constructor, I pass in the name of the Python module, import the module, locate the class (which always has a pre-defined name PyClass), and call that class to create an instance of it. I then store the resulting PyObject* in MyClass for future use. In the MyClass destructor, I decref that stored PyObject* to avoid memory leaks.
I have already verified that everything is working correctly as far as locating the class and creating an instance of it. I have even verified that I can use the stored PyObject* in other MyClass methods, for example, to access methods in the PyClass. However, when the destructor does the decref, it causes a segfault.
Here is a sample of my code. I also call Py_Initialize() and Py_Finalize() elsewhere at appropriate times, and I have left out some of my error-checking code for brevity:
MyPythonModule.py
class PyClass:
pass
MyClass.h
class MyClass {
public:
MyClass(const char* modulename);
~MyClass();
private:
void* _StoredPtr;
};
MyClass.cpp
#include <Python.h>
#include <iostream>
#include "MyClass.h"
MyClass::MyClass(const char* modulename) {
_StoredPtr = NULL;
PyObject *pName = NULL, *pModule = NULL, *pAttr = NULL;
// Import the Python module.
pName = PyString_FromString(modulename);
if (pName == NULL) {goto error;}
pModule = PyImport_Import(pName);
if (pModule == NULL) {goto error;}
// Create a PyClass instance and store a pointer to it.
pAttr = PyObject_GetAttrString(pModule, "PyClass");
if (pAttr == NULL) {goto error;}
_StoredPtr = (void*) PyObject_CallObject(pAttr, NULL);
Py_DECREF(pAttr);
if (_StoredPtr == NULL) {goto error;}
error:
if (PyErr_Occurred()) {PyErr_Print();}
Py_XDECREF(pName);
Py_XDECREF(pModule);
return;
}
MyClass::~MyClass() {
std::cout << "Starting destructor..." << std::endl;
Py_XDECREF((PyObject*)(_StoredPtr));
std::cout << "Destructor complete." << std::endl;
}
I know that I could avoid the segfault by leaving out the Py_XDECREF() in the destructor, but I am afraid of causing a memory leak because I do not understand exactly why this is happening. It seems especially strange that I can use _StoredPtr successfully in other MyClass methods, yet I can't decref it.
I have also tried storing the PyObject* of the imported module in MyClass and holding on to it until after _StoredPtr is decrefed, but the _StoredPtr decref still segfaults. I tried commenting out the Py_DECREF(pAttr); line, but that doesn't help.
As I mentioned, I can retrieve methods in the PyClass using _StoredPtr, and I have also tried storing these in MyClass and decrefing them in the destructor. When I do this, I can decref _StoredPtr, but then it segfaults when I try to decref the method's PyObject*. If I do this with several methods, it is always the last decref that causes the segfault, no matter what order I put them in.
Any insights as to what's happening here?
This works for me
#include <Python.h>
#include <iostream>
#include "MyClass.h"
MyClass::MyClass(const char* modulename) {
_StoredPtr = NULL;
PyObject *pName = NULL, *pModule = NULL, *pAttr = NULL;
// Import the Python module.
pName = PyString_FromString(modulename);
if (pName == NULL) {goto error;}
pModule = PyImport_Import(pName);
if (pModule == NULL) {goto error;}
// Create a PyClass instance and store a pointer to it.
pAttr = PyObject_GetAttrString(pModule, "PyClass");
if (pAttr == NULL) {goto error;}
_StoredPtr = (void*) PyObject_CallObject(pAttr, NULL);
Py_DECREF(pAttr);
if (_StoredPtr == NULL) {goto error;}
else{
// do something with _StoredPtr
Py_XDECREF((*PyObject)_StoredPtr)
}
error:
if (PyErr_Occurred()) {PyErr_Print();}
Py_XDECREF(pName);
Py_XDECREF(pModule);
return;
}
MyClass::~MyClass() {}
I basically moved the XDECREF outside the destructor into the function that is using the PyObject.
I am trying to make a call to a python module function from my cpp file.
The call i have made is as follows:
#include <iostream>
#include "Python.h"
int
main(int argc, char** argv)
{
Py_Initialize();
PyObject *pName = PyString_FromString("tmpPyth");
PyObject *pModule = PyImport_Import(pName);
std::cout<< "Works fine till here";
PyObject *pDict = PyModule_GetDict(pModule);
if (pModule != NULL) {
PyObject *pFunc = PyObject_GetAttrString(pDict, "pyFunc");
if(pFunc != NULL){
PyObject_CallObject(pFunc, NULL);
}
}
else
std::cout << "Python Module not found";
return 0;
}
My python module is defined as follows:
import numpy
import scipy
import matplotlib
from scipy import stats
def blah():
baseline = [9.74219, 10.2226, 8.7469, 8.69791, 9.96442, 9.96472, 9.37913, 9.75004]
follow_up = [9.94227,9.46763,8.53081,9.43679,9.97695,10.4285,10.159,8.86134]
paired_sample = stats.ttest_rel(baseline , follow_up )
print "The t-statistic is %.3f and the p-value is %.3f." % paired_sample
The code in the cpp file runs fine till the 1st "std::cout" but then ends up giving me a "seg fault". Running the python code separately works fine and gives the desired output.
I cant figure out what is going wrong. Any help will be appreciated.
(Note the program is compiling correctly and running correctly till the 1st "cout")
So there are a couple of things that you were not doing right. See the comments inline. Assuming that both your CPP file and Python file lives at the following path: /home/shanil/project.
test.cpp:
#include <iostream>
#include "Python.h"
int
main(int argc, char** argv)
{
Py_Initialize();
// First set in path where to find your custom python module.
// You have to tell the path otherwise the next line will try to load
// your module from the path where Python's system modules/packages are
// found.
PyObject* sysPath = PySys_GetObject("path");
PyList_Append(sysPath, PyString_FromString("/home/shanil/project"));
// Load the module
PyObject *pName = PyString_FromString("my_mod");
PyObject *pModule = PyImport_Import(pName);
// Random use-less check
std::cout<< "Works fine till here\n";
if (pModule != NULL) {
std::cout << "Python module found\n";
// Load all module level attributes as a dictionary
PyObject *pDict = PyModule_GetDict(pModule);
// Remember that you are loading the module as a dictionary, the lookup you were
// doing on pDict would fail as you were trying to find something as an attribute
// which existed as a key in the dictionary
PyObject *pFunc = PyDict_GetItem(pDict, PyString_FromString("my_func"));
if(pFunc != NULL){
PyObject_CallObject(pFunc, NULL);
} else {
std::cout << "Couldn't find func\n";
}
}
else
std::cout << "Python Module not found\n";
return 0;
}
my_mod.py:
def my_func():
print 'got called'
I was going through the Python documentation regarding Extending Python with C/C++, and I found a very interesting scenario. Consider that I have the following, trivial Python script named test_script.py in my working directory:
import time
def My_Python_Func():
print "My_Python_Func() Called: " + str(time.time())
In the same directory, I have a file called another_test_script.py, which has:
import test_script
test_script.My_Python_Func()
Which works fine, when called as python ./another_test_script.py. Now, I'm trying to call said function from a C environment linked with the Python libraries as follows:
#include <stdio.h>
#include <Python.h>
PyObject *pName, *pModule, *pDict, *pFunc, *pVal;
int main()
{
const char* script_name = "test_script";
const char* func_name = "My_Python_Func";
Py_Initialize();
pName = PyString_FromString(script_name);
PyRun_SimpleString("import os");
PyRun_SimpleString("import sys");
/* * * * IF I COMMENT THE FOLLOWING LINE OUT, pModule IS ALWAYS SET TO NULL * * * */
PyRun_SimpleString("sys.path.insert(0, os.getcwd())");
pModule = PyImport_Import(pName);
Py_DECREF(pName);
if (pModule != NULL)
{
pFunc = PyObject_GetAttrString(pModule, func_name);
}
pDict = PyModule_GetDict(pModule);
pFunc = PyDict_GetItemString(pDict, func_name);
if (PyCallable_Check(pFunc))
{
PyObject_CallObject(pFunc, NULL);
Py_DECREF(pModule);
Py_DECREF(pName);
}
else
{
PyErr_Print();
}
Py_Finalize();
return 0;
}
As noted in the comments in the above C program, commenting out the line containing PyRun_SimpleString("sys.path.insert(0, os.getcwd())") causes the call to PyImport_Import to fail (returning NULL). Furthermore, calling PyRun_SimpleString("import test_script") appears to behave the same way.
Why do I need to manually add the current working directory to Python's sys.path list of strings, when simply importing it from another Python script in the same working directory? Furthermore, why does Python not search the current working directory in the first place (as os.getcwd() returns the correct path)? Is this an appropriate solution, if I wish to import functions from scripts bundled with my C application?
Adding the script directory to sys.path is a peculiarity of the python executable; it is not done by default when the interpreter is initialized, since it is not always appropriate to do so in embedded applications.