if I call a SWIG-wrapped C/C++ function from Python, is it possible to obtain the current call stack? I would like something similar to the result of ''.join(traceback.format_stack()), but I don't want to pass this from Python to my C/C++ functions because I don't always need it. So I would like to obtain it on the fly and print it if something wrong happens on my C/C++ side.
I figured out a solution following this post, although I still prefer more natural ways of getting the same thing if there is any.
// This is similar to the python code:
// def GetScriptingLanguageCallStack():
// import traceback
// return ''.join(traceback.format_stack())
string GetScriptingLanguageCallStack() {
string result;
PyObject* module_name = PyString_FromString("traceback");
PyObject* pyth_module = PyImport_Import(module_name);
Py_DECREF(module_name);
if (pyth_module != nullptr) {
PyObject* pyth_func = PyObject_GetAttrString(pyth_module, "format_stack");
if (pyth_func != nullptr) {
if (PyCallable_Check(pyth_func)) {
PyObject* pyth_val = PyObject_CallFunctionObjArgs(pyth_func, 0);
if (pyth_val != nullptr) {
if (PyList_Check(pyth_val)) {
const int size = PyList_GET_SIZE(pyth_val);
for (int i = 0; i < size; ++i) {
PyObject* pyth_line = PyList_GET_ITEM(pyth_val, i);
result += PyString_AsString(pyth_line);
}
}
Py_DECREF(pyth_val);
}
}
Py_DECREF(pyth_func);
}
Py_DECREF(pyth_module);
}
return result;
}
By the way, I do not prefer the approach in the linked post which uses frame object, because the line number given is not pointing to the exact line which makes the further function calls, but only on the line containing the function name.
Related
I'm trying to get a simple C++ 20 based generator pattern work with PyBind11. This is the code:
#include <pybind11/pybind11.h>
#include <coroutine>
#include <iostream>
struct Generator2 {
Generator2(){}
struct Promise;
using promise_type=Promise;
std::coroutine_handle<Promise> coro;
Generator2(std::coroutine_handle<Promise> h): coro(h) {}
~Generator2() {
if(coro)
coro.destroy();
}
int value() {
return coro.promise().val;
}
bool next() {
std::cout<<"calling coro.resume()";
coro.resume();
std::cout<<"coro.resume() called";
return !coro.done();
}
struct Promise {
void unhandled_exception() {std::rethrow_exception(std::move(std::current_exception()));}
int val;
Generator2 get_return_object() {
return Generator2{std::coroutine_handle<Promise>::from_promise(*this)};
}
std::suspend_always initial_suspend() {
return {};
}
std::suspend_always yield_value(int x) {
val=x;
return {};
}
std::suspend_never return_void() {
return {};
}
std::suspend_always final_suspend() noexcept {
return {};
}
};
};
Generator2 myCoroutineFunction() {
for(int i = 0; i < 100; ++i) {
co_yield i;
}
}
class Gen{
private:
Generator2 myCoroutineResult;
public:
Gen(){
myCoroutineResult = myCoroutineFunction();
}
int next(){
return (myCoroutineResult.next());
}
};
PYBIND11_MODULE(cmake_example, m) {
pybind11::class_<Gen>(m, "Gen")
.def(pybind11::init())
.def("next", &Gen::next);
}
However I'm getting an error:
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
Could c++ coroutines, coroutine_handles, co_yield etc. be a low-level thing that is not supported by PyBind11 yet?
Even though PyBind11 does not support coroutines directly, your problem does not mix coroutine and pybind code since you are hiding the coroutine behind Gen anyway.
The problem is that your Generator2 type uses the compiler provided copy and move constructors.
This line:
myCoroutineResult = myCoroutineFunction();
Creates a coroutine handle when you call myCoroutineFunction, and puts it in the temporary Generator2 in the right hand side. Then, you initialize myCoroutineResult from the right hand side generator. All is well, but then the temporary gets destroyed. Your destructor checks whether the handle is valid or not:
~Generator2() {
if(coro)
coro.destroy();
}
But in your implementation, the coro member of the member generator gets copied from the temporary without resetting the temporary's coro member. So the coroutine itself gets destroyed once you initialize myCoroutineResult, and you are holding onto a dangling coroutine handle. Remember that std::coroutine_handles behave like a raw pointer.
Essentially, you have a violation of the rule of 5. You have a custom destructor, but no copy/move constructors or assignment operators. Since you cannot copy construct a coroutine, you can ignore the copy constructors but you need to provide move constructors/assigment operators:
Generator2(Generator2&& rhs) : coro{std::exchange(rhs.coro, nullptr)} {
// rhs will not delete our coroutine,
// since we put nullptr to its coro member
}
Generator2& operator=(Generator2&& rhs) {
if (&rhs == this) {
return *this;
}
if (coro) {
coro.destroy();
}
coro = std::exchange(rhs.coro, nullptr);
return *this;
}
Also, use member initialization list to initialize members instead of assigning them within the constructor body. So instead of this:
Gen(){
myCoroutineResult = myCoroutineFunction();
}
Use this:
Gen() : myCoroutineResult{myCoroutineFunction()} {}
The reasoning can be seen even in this answer. The first one calls the assignment operator, which performs a bunch of additional work, whereas the second one calls the move constructor, which is as lean as it gets.
I am trying to overload a python extension function that would take either a object or a string.
typedef struct
{
PyObject_HEAD
} CustomObject;
PyObject* customFunction(CustomObject* self, PyObject* args);
PyMethodDef methods[] =
{
{"customFunction", (PyCFunction) customFunction, METH_VARAGS, "A custom function"},
{NULL}
}
PyTypeObject TypeObj =
{
PyVarObject_HEAD_INIT(NULL, 0)
.tp_name = "customModule.CustomObject",
.tp_doc = "Custom Object",
.tp_basicsize = sizeof(CustomObject),
.tp_itemsize = 0,
.tp_flags = Py_TPFLAGS_DEFAULT,
.tp_methods = methods,
}
// Area of problem
PyObject* customFunction(CustomObject* self, PyObject* args)
{
const char* string;
PyObject* object;
if (PyArg_ParseTuple(args, "O!", &TypeObj, &object)) // TypeObj is the PyTypeObject fpr CustomObject
{
std::cout << "Object function\n"
// Do whatever and return PyObject*
}
else if (PyArg_ParseTuple(args, "s", &string))
{
std::cout << "String function\n"
// Do whatever and return PyObject*
}
return PyLong_FromLong(0); // In case nothing above works
}
In python I have a try except for the function and I get this error Error: <built-in method customFunction of CustomModule.CustomObject object at 0xmemoryadress> returned a result with an error set
Here are the Python docs for this PyArg_ParseTuple:
int PyArg_ParseTuple(PyObject *args, const char *format, ...)
Parse the parameters of a function that takes only positional parameters into local variables. Returns true on success; on failure, it returns false and raises the appropriate exception
I am guessing that PyArg_ParseTuple is setting an error, which is causing the entire function not to work (I do have customFunction in my method table for the module, I am just omitting that code). If I have the following Python:
import CustomModule
try:
CustomModule.customFunction("foo")
except Exception as e:
print("Error:", e)
String function does get outputted, so the code in the string if statement does work, but I assume the error occurs because PyArg_ParseTuple for the object failed, so it returns an error (not 100% sure if this is correct).
Is there a way I can prevent PyArg_ParseTuple() from raising an error, is there another function, or is there a better way to 'overload' my custom functions?
I'd probably just use PyArg_ParseTuple to get a generic unspecified object, and then handle the object types later with Py*_Check:
if (!PyArg_ParseTuple(args, "O", &object)) {
return NULL;
}
if (PyObject_IsInstance(object, (PyObject*)&PyType)) { // or a more specific function if one exists
std::cout << "Object function\n";
} else if (PyUnicode_Check(object)) {
std::cout << "String function\n";
} else {
// set an error, return NULL
}
The reason for this is that the Python "ask forgiveness, not permission" pattern of
try:
something()
except SomeException:
somethingElse()
doesn't translate very well into C, and involves quite a bit of code to handle the exceptions. If you really want to do it that way then you need to call PyErr_Clear before the second PyArg_ParseTuple, and ideally you should check it's the exception you think, and not something else entirely.
I am new to the Python/C API and while I got some basic functions to work, I am struggling with this one.
PyObject* sum_elements(PyObject*, PyObject *o)
{
Py_ssize_t n = PyList_Size(o);
long total = 0;
if (n < 0)
{
return PyLong_FromLong(total);
}
PyObject* item;
for (int i = 0; i < n; i++)
{
item = PyList_GetItem(o, i);
if (!PyLong_Check(item)) continue;
total += PyLong_AsLong(item);
}
return PyLong_FromLong(total);
}
Basically this is the function from the introduction on the doc page. It should receive a python list and return the sum of all elements. The function works fine if i pass a list, if I pass something else however i get the error message
SystemError: c:\_work\5\s\objects\listobject.c:187: bad argument to internal function
This situation should be handled by the if (n<0) statement, as n is -1 if the passed object is not a list.
I am binding the function the following way:
static PyMethodDef example_module_methods[] = {
{ "sum_list", (PyCFunction)sum_elements, METH_O, nullptr},
{ nullptr, nullptr, 0, nullptr }
};
Thanks.
The error
SystemError: c:\_work\5\s\objects\listobject.c:187: bad argument to internal function
is actually occurs at
Py_ssize_t n = PyList_Size(o)
Because PyList_Size has an extra check to see whether the object of list type, If not it will call PyErr_BadInternalCall API to raise the SystemError. See the implementation of PyList_Size in listobject.c
PyList_Size(PyObject *op)
{
if (!PyList_Check(op)) {
PyErr_BadInternalCall();
return -1;
}
else
return Py_SIZE(op);
}
The PyErr_BadInternalCall a shorthand for PyErr_SetString(PyExc_SystemError, message), where message indicates that an internal operation (e.g. a Python/C API function) was invoked with an illegal argument.
You should use PyList_Check API to check whether the object is of list type . As per the doc it Return true if object is a list object or an instance of a subtype of the list type.
PyObject* sum_elements(PyObject*, PyObject *o)
{
// Check if `o` is of `list` type, if not raise `TypeError`.
if (!PyList_Check(o)) {
PyErr_Format(PyExc_TypeError, "The argument must be of list or subtype of list");
return NULL;
}
// The argument is list type, perform the remaining calculations.
Py_ssize_t n = PyList_Size(o);
long total = 0;
if (n < 0)
{
return PyLong_FromLong(total);
}
PyObject* item;
for (int i = 0; i < n; i++)
{
item = PyList_GetItem(o, i);
if (!PyLong_Check(item)) continue;
total += PyLong_AsLong(item);
}
return PyLong_FromLong(total);
}
Once this extra check is added, the function call will raise
TypeError: The argument must be of list or sub type of list
when the argument other than list type is supplied.
I've chosen setuptools to use my C/C++ code from python scripts.
One of the phases when building such wrapper is to convert the C/C++ return value into python object.
So far I was able to convert simple primitive values and list of primitive values. However, I wish to extend it to multi-value struct, as shown in the example below.
My main challenge right now is how do I create the python struct representation (PyObject* ret = PyList_New(...);) and I do I set it's values properly with the different types.
I tried to create list of items from the same types (such as std::vector<float>) and manage to set the values properly using Py_BuildValue and PyList_SetItem, but I'm still struggling with the multi types...
typedef struct _fileParams
{
bool valid;
int index;
std::string key;
std::value value;
} fileParams;
FileDataBase * db;
static PyObject *searchFileInDB(PyObject *self, PyObject *args)
{
if (db == NULL)
{
PyErr_SetString(PyExc_RuntimeError, "DB could not be initialized");
return NULL;
}
char* fileName = NULL;
int fileNameSize = 0;
PyArg_ParseTuple(args, "s#", &fileName, &fileNameSize);
try
{
fileParams p;
bool res = db->lookup(fileName, fileNameSize, p);
PyObject* ret = PyList_New(...);
if (res)
{
PyObject* r1 = Py_BuildValue("b", p.valid);
PyList_SetItem(ret, 0, r1);
PyObject* r2 = Py_BuildValue("i", p.index);
PyList_SetItem(ret, 1, r2);
PyObject* r1 = Py_BuildValue("s", p.key);
PyList_SetItem(ret, 2, r3);
PyObject* r1 = Py_BuildValue("s", p.value);
PyList_SetItem(ret, 3, r4);
}
return ret;
} catch (...) {
PyErr_SetString(PyExc_RuntimeError, "failed with C exception");
return NULL;
}
}
You probably want to look into the Dictionary Object: Dictionary Objects
I'm guessing you'd want to set values with PyDict_SetItemString() as per that doc.
HTH
I have a list of base classes in C++, I want to access them in Python as a list of their derived most classes.
Is there a build in means to cater for this in Boost.Python?
I've made an example the problem I ma facing:
// ------------------------------- Code ----------------------------------//
#include<memory>
#include<iostream>
#include<vector>
namespace boost { template<class T> T* get_pointer(std::shared_ptr<T>& p){ return p.get(); }}
struct Vehicle{ virtual ~Vehicle(){} friend bool operator==(const Vehicle& lhs, const Vehicle& rhs) { return true; }};
struct Boat: public Vehicle{
virtual ~Boat(){}
friend bool operator==(const Boat& lhs, const Boat& rhs) { return true; }
char const* testBoatSpecificMethod() { return "Floating."; }
};
struct Truck: public Vehicle{
virtual ~Truck(){}
friend bool operator==(const Truck& lhs, const Truck& rhs) { return true; }
char const* testTruckSpecificMethod() { return "Trucking."; }
};
class Garage
{
public:
Garage() {};
~Garage() {};
char const* test() { std::string temp = "Vehicle List Size: " + std::to_string(m_VehicleList.size()); return temp.c_str(); }
friend bool operator==(const Garage& lhs, const Garage& rhs) { return true; }
std::vector<std::shared_ptr<Vehicle>>& vehicleList() { return m_VehicleList; }
private:
std::vector<std::shared_ptr<Vehicle>> m_VehicleList;
};
#include <boost/python.hpp>
#include <boost/python/suite/indexing/vector_indexing_suite.hpp>
BOOST_PYTHON_MODULE(garage_ext)
{
using namespace boost::python;
class_<Garage>("Garage")
.def("test", &Garage::test)
.def("vehicleList", &Garage::vehicleList, return_internal_reference<1>());
class_<Vehicle,std::shared_ptr<Vehicle>>("Vehicle");
class_<Boat,std::shared_ptr<Boat>,bases<Vehicle>>("Boat")
.def("testBoatSpecificMethod", &Boat::testBoatSpecificMethod);
class_<Truck,std::shared_ptr<Truck>,bases<Vehicle>>("Truck")
.def("testTruckSpecificMethod", &Truck::testTruckSpecificMethod);
implicitly_convertible<std::shared_ptr<Boat>,std::shared_ptr<Vehicle>>();
implicitly_convertible<std::shared_ptr<Truck>,std::shared_ptr<Vehicle>>();
class_<std::vector<std::shared_ptr<Vehicle>> >("stl_vector_Vehicle")
.def(vector_indexing_suite<std::vector<std::shared_ptr<Vehicle>> >());
}
// --------------------------- Test Script -------------------------------//
import garage_ext
g = garage_ext.Garage()
l = g.vehicleList()
l.append(garage_ext.Boat())
print "Expecting a Boat object:"
print str(l[0])
print g.vehicleList()[0].testBoatSpecificMethod()
garage_ext.f2("Done.")
// ------------------------------ Output ---------------------------------//
Expecting a Boat object
Traceback (most recent call last):
File "test_garage.py", line 7, in
print g.vehicleList()[0].testBoatSpecificMethod()
AttributeError: 'Vehicle' object has no attribute 'testBoatSpecificMethod'
'Vehicle' object has no attribute 'testBoatSpecificMethod'
Here I want Vehicle to be a Boat object.
If there is not a build-in or recommended/known Boost.Python means to handle this problem,
I'll try wrapping the list (Lots of wrapping to be done then in my library.) with a get accessor returning a boost::python::list, storing the derived most types in the python list object. Getting the derived most type possibly by calling overriden 'getAsDerivedClass' method.
I would like to avoid this. I dislike having to add python usage specific methods to the library, for our design and vision values / reasons. Another concern is that this way will introduce a lot of extramaintenance work.
EDIT:
What I want works when I am using raw pointers instead of smart pointers.
For what I feel are obvious reasons,I do not want to use raw pointers in place of smart pointers.
This does give me a relieve in that knowing what I want this concept isn't so far-fetched as I started to fear. (I am struggling still to make it work with smart pointers. The python object asks for a converter, too much work to write one by hand.)