PyRun_File with recursive functions in Python C-API - python

I am working on embedding python interpreter in a larger system and one of the features is to run a python script on the fly.
Testing with the following code snippet works great, but with recursive function only the first call is executed then crashes
//C++ code
int main()
{
Py_Initialize();
PyObject* m_pMainModule = PyImport_AddModule("__main__");
PyObject* m_pGlobalDict = PyModule_GetDict(m_pMainModule);
PyObject* m_pLocalDict = PyDict_New();
PyObject* fd = PyFile_FromString("script.py", "r");
if (fd == NULL)
{
PyErr_SetString(PyExc_IOError, "File not found");
}
PyObject * s = PyRun_File(PyFile_AsFile(fd), "script.py", Py_file_input, m_pGlobalDict, m_pLocalDict);
Py_XDECREF(fd);
Py_XDECREF(s);
if (PyErr_Occurred())
{
std::string result;
PyObject* ptype;
PyObject* pvalue;
PyObject* ptraceback;
PyErr_Fetch(&ptype, &pvalue, &ptraceback);
PyErr_NormalizeException(&ptype, &pvalue, &ptraceback); // in order to convert pvalue from tuples to real objects
//Attach exception name first
PyObject* objectStr = PyObject_GetAttrString(ptype, "__name__");
result = PyString_AS_STRING(objectStr);
result = "Exception: " + result;;
Py_XDECREF(objectStr);
objectStr = PyObject_Str(pvalue);
if (objectStr != NULL) {
result = result + " was raised with error message : " + PyString_AS_STRING(objectStr);
Py_XDECREF(objectStr);
}
std::cout << result;
}
return 0;
}
Here is the python script I use
def fac(i):
print "Call to FAC(",i,") !"
if i <= 1:
return 1
else:
return i*fac(i-1)
print "Hello world"
print fac(4)
And here is the output
Hello world
Call to FAC( 4 ) !
Exception: NameError was raised with error message : global name 'fac' is not defined
While the expected output ( when run directly by invoking the script )
Hello world
Call to FAC( 4 ) !
Call to FAC( 3 ) !
Call to FAC( 2 ) !
Call to FAC( 1 ) !
24
Any ideas on how to proceed ?
Edit:
Platform: Windows 10 x64
Python: 2.7.14

Explanation:
In the following line, PyRun_File uses two different dictionaries; one for globals and one for locals.
PyObject * s = PyRun_File(PyFile_AsFile(fd), "script.py", Py_file_input, m_pGlobalDict, m_pLocalDict);
The strange part that fac function name is added to the local one of file.
I have no idea Why, But I really wanaa know.
So the recursive call fails as there is no fac inside locals of the function or the global.
Solution:
pass the same dictionary for both local and global
PyObject * s = PyRun_File(PyFile_AsFile(fd), "script.py", Py_file_input, m_pGlobalDict, m_pGlobalDict);

Related

How to print custom Python types in boost.python

I am trying to use the boost.python library in order to get data from the Python skyfield API, but I am having trouble printing out the data. The program runs, but I get this error: "Error in Python: <class 'TypeError'>: 'NoneType' object is not callable"
The correct output should show something like this:
10h 47m 56.24s
+09deg 03' 23.1"
2.33251 au
I am using the first code example from this website as a reference: https://rhodesmill.org/skyfield/
If you have any ideas on how to get the data to correctly output, please let me know!
Here is my code so far:
#include <iostream>
#include <boost/python.hpp>
#include <Python.h>
namespace py = boost::python;
std::string parse_python_exception();
int main()
{
Py_Initialize(); //Initializes the Python interpreter
//Imports the __main__ module and extracts the namespace
//This creates a blank 'canvas' upon which we can call Python code
py::object main_module = py::import("__main__");
//Loads the dictionary object from the main module
py::object main_namespace = main_module.attr("__dict__");
try
{
//Imports
py::exec("from skyfield.api import load");
py::object skyfield_mod = py::import("skyfield.api");
py::object skyfield_load = skyfield_mod.attr("load");
py::object skyfield_load_timescale = skyfield_load.attr("timescale");
py::exec("print('This program computes the current position of Mars in the sky')", main_namespace);
py::object ts = skyfield_load_timescale();
py::object ts_now = ts.attr("now");
py::object t = ts_now();
py::object planets = skyfield_load("de421.bsp");
py::object earth = planets["earth"];
py::object mars = planets["mars"];
py::object earth_at = earth.attr("at");
py::object earth_at_observe = earth_at(t).attr("observe");
py::object m_earth_at_observe = earth_at_observe(mars);
py::object astrometric = m_earth_at_observe;
py::object ra, dec, distance = astrometric.attr("radec");
py::object m_ra = ra();
py::object m_dec = dec();
py::object m_distance = distance();
py::exec("print(m_ra)", main_namespace);
}
catch (boost::python::error_already_set const&)
{
std::string perror_str = parse_python_exception();
std::cout << "Error in Python: " << perror_str << std::endl;
}
}
// Parses the value of the active python exception
// NOTE SHOULD NOT BE CALLED IF NO EXCEPTION
std::string parse_python_exception() {
PyObject* type_ptr = NULL, * value_ptr = NULL, * traceback_ptr = NULL;
// Fetch the exception info from the Python C API
PyErr_Fetch(&type_ptr, &value_ptr, &traceback_ptr);
// Fallback error
std::string ret("Unfetchable Python error");
// If the fetch got a type pointer, parse the type into the exception string
if (type_ptr != NULL) {
py::handle<> h_type(type_ptr);
py::str type_pstr(h_type);
// Extract the string from the boost::python object
py::extract<std::string> e_type_pstr(type_pstr);
// If a valid string extraction is available, use it
// otherwise use fallback
if (e_type_pstr.check())
ret = e_type_pstr();
else
ret = "Unknown exception type";
}
// Do the same for the exception value (the stringification of the exception)
if (value_ptr != NULL) {
py::handle<> h_val(value_ptr);
py::str a(h_val);
py::extract<std::string> returned(a);
if (returned.check())
ret += ": " + returned();
else
ret += std::string(": Unparseable Python error: ");
}
// Parse lines from the traceback using the Python traceback module
if (traceback_ptr != NULL) {
py::handle<> h_tb(traceback_ptr);
// Load the traceback module and the format_tb function
py::object tb(py::import("traceback"));
py::object fmt_tb(tb.attr("format_tb"));
// Call format_tb to get a list of traceback strings
py::object tb_list(fmt_tb(h_tb));
// Join the traceback strings into a single string
py::object tb_str(py::str("\n").join(tb_list));
// Extract the string, check the extraction, and fallback in necessary
py::extract<std::string> returned(tb_str);
if (returned.check())
ret += ": " + returned();
else
ret += std::string(": Unparseable Python traceback");
}
return ret;
}

How to get the error message when PyRun_AnyFile failed

I am trying to embed python in my c++ project by using PyRun_AnyFile.
Here is my python script:
import aaa
if __name__ == "__main__":
print("Here")
Here is my c++ code:
const char pFile[] = "C:\\testing.py";
FILE* fp = _Py_fopen(pFile, "r");
int ret = PyRun_AnyFile(fp, pFile);
I get error message like this:
Traceback (most recent call last):
File "C:\testing.py", line 1, in <module>
import aaa
ModuleNotFoundError: No module named 'aaa'
This is expected. I would like to ask how I get the above error message in the code and redirect it to other error files.
I have tried:
PyObject* ptype, * pvalue, * ptraceback;
PyErr_Fetch(&ptype, &pvalue, &ptraceback);
Py_ssize_t string_len = 0;
std::wcout << std::wstring(PyUnicode_AsWideCharString(pvalue, &string_len)) << std::endl;
or:
PyErr_Print();
But that is not success.
I had similar problem some time ago. I solved reading of error that way. You have to execute this code after recognizing that there was an error on script execution.
PyRun_SimpleString("import traceback, sys");
PyRun_SimpleString("trace = ''.join(traceback.format_exception(sys.last_type, sys.last_value, sys.last_traceback))");
PyObject *mainModule = PyImport_AddModule("__main__");
if (PyObject_HasAttrString(mainModule, std::string("trace").c_str())) {
PyObject *var = PyObject_GetAttrString(mainModule, std::string("trace").c_str());
if (PyUnicode_Check(var)) {
std::string errMsg = (std::string) PyUnicode_AsUTF8(var) + "\n";
std::cout << errMsg;
}
}

How can I get the print of python function from cpp

I use cpp to call python function, I have compiled the program without errors, but why can not I see the print result in python function.
here is the cpp codes:
#include<python2.7/Python.h>
....
using namespace std;
int main()
{
Py_Initialize();
PyRun_SimpleString("import sys");
PyRun_SimpleString("import os");
PyRun_SimpleString("import string");
PyRun_SimpleString("sys.path.append('./')");
PyObject * pModule = NULL;
PyObject * pFunc = NULL;
PyObject * pClass = NULL;
PyObject * pInstance = NULL;
pModule = PyImport_ImportModule("download");
if(!pModule)
{
std::cout << "there is no this file." << std::endl;
}
pFunc= PyObject_GetAttrString(pModule, "geturl");
if(!pFunc)
{
std::cout << "there is no this func." << std::endl;
}
std::string url = "www";
PyObject* args = Py_BuildValue("ss", url.c_str());
PyEval_CallObject(pFunc, args);
Py_DECREF(pFunc);
Py_Finalize();
return 0;
}
here is the download.py file in the same dir
def geturl(url):
print(url)
print("hello")
here is result, without errors nor print:
root#cvm-172_16_20_84:~/klen/test/cpppython # g++ t.cpp -o printurl -lpython2.7
root#cvm-172_16_20_84:~/klen/test/cpppython # ./printurl
root#cvm-172_16_20_84:~/klen/test/cpppython #
How can I see the print, has the function geturl run successfully? Thanks
The PyEval_CallObject function is encountering a Python exception before it reaches your print statements. Adding error handling to this call (call PyErr_Print on NULL return value) will show the exception being raised:
TypeError: geturl() takes exactly 1 argument (2 given)
The root cause is the format string:
Py_BuildValue("ss", url.c_str());
You are creating a tuple of two values and passing that as the arguments geturl(). You need to pass a tuple with only one value. You are also invoking undefined behavior here because you haven't provided a second string pointer.
Resolve the issue by passing a tuple with only one value:
Py_BuildValue("(s)", url.c_str());

Exception TypeError: 'argument list must be a tuple'

I am trying to call a simple python function from cpp.
Below is the cpp code:
#include <iostream>
#include <Python.h>
#include <numpy/arrayobject.h>
using namespace std;
void init_numpy()
{
import_array();
}
int main()
{
char dir[500];
cout << "Enter directory: \n";
cin >> dir;
Py_Initialize();
const char *scriptDirectoryName = dir;
PyObject *sysPath = PySys_GetObject("path");
PyObject *path = PyString_FromString(scriptDirectoryName);
int result = PyList_Insert(sysPath, 0, path);
PyObject *pModule = PyImport_ImportModule("mytest");
init_numpy();
double getItem[2] ;
getItem[0] = getItem[1] = 2;
npy_intp dims = 2 ;
PyObject* arrayToPass = PyArray_SimpleNewFromData(1, &dims, NPY_DOUBLE , (void *)getItem);
PyObject* myFunction = PyObject_GetAttrString(pModule,(char*)"stuff");
//PyObject* args = PyTuple_Pack(1,PyFloat_FromDouble(getItem));
PyObject* myResult = PyObject_CallObject(myFunction, arrayToPass);
//double resultasd = PyFloat_AsDouble(myResult);
//cout << resultasd << endl;
Py_Finalize();
return 0;
}
And here is my python code:
mytest.py
def stuff(a):
x = a[1]
return x
This just testing of a bigger patch I am writing, but the methodology is the same.
I am getting this error:
Exception TypeError: 'argument list must be a tuple' in <module 'threading' from '/usr/lib/python2.7/threading.pyc'> ignored
I searched multiple threads but everyone had a unique typo or a bug.
Any suggestions where I am doing wrong?
I am compiling as:
g++ -I /usr/include/python2.7/ che.cpp -lpython2.7 -o linkArr
From documentation:
PyObject* PyObject_CallObject(PyObject *callable, PyObject *args)
Return value: New reference.
Call a callable Python object callable, with arguments given by the tuple args. If no arguments are needed, then args can be NULL.
Return the result of the call on success, or raise an exception and return NULL on failure.
This is the equivalent of the Python expression: callable(*args).
The function PyObject_CallObject(...) requires tuple as an argument.
Also checkout this and this post for more clarification.

How to use multi-threading to two c++ functions with embedding python?

I am learning embedding python in c++ code. I have trouble to use multi-threading to parallelize two c++ functions with embedding python.
My sample codes are shown below:
thread_test.py
import time
def test1():
time.sleep(5) # delays for 5 seconds
print 1935
return 'happy'
def test2():
time.sleep(10) # delays for 10 seconds
print 3000
py_thread.h
string test_func1(string file_dir){
string result_dir;
string str = "import sys; sys.path.insert(0," "\'"+file_dir+"\'"+")";
const char * c = str.c_str();
PyRun_SimpleString (c);
PyObject * pModule,* pFunc, *pName, *presult, *pArgs;
pName = PyString_FromString("thread_test");
pModule = PyImport_Import(pName);
Py_DECREF(pName);
pFunc = PyObject_GetAttrString(pModule, "test1");
if(pFunc != NULL) {
presult=PyObject_CallObject(pFunc,NULL);
result_dir = PyString_AsString(presult);
}
else {
printf("pFunc returned NULL\n");
}
Py_DECREF(pModule);
Py_DECREF(pFunc);
return result_dir;
}
void test_func2(string file_dir){
// Almost the same test_func1 except replacing "test1" with "test2" and no return value of result_dir
}
In main class, if I don't use multi-threading and just run the two functions and other normal c++ functions in serious, it works. But if I use some c++ threading techniques, such as OPENMP, it will give me SEGMENTATION FAULT. (code is shown below)
main.cpp
int main(){
Py_Initialize();
#pragma omp parallel num_threads(2)
{
int i = omp_get_thread_num();
if(i == 0)
{
test_func1("../");
}
if(i == 1 || omp_get_num_threads() != 2)
{
ANOTHER_C++_ONLY_SIMPLE_FUNCTION();
test_func2("../");
}
}
Py_Finalize();
return 0;
}
I also have tried thread in c++11 and pthread. They all give me segmentation fault. So how can I parallelize the two function???
Thank you!

Categories