I understand a basic C++ function wrapped for Python looks like this:
int square(int n)
{
return n*n;
}
static PyObject* square_wrapper(PyObject* self, PyObject* args)
{
int n = 0;
if(!PyArg_ParseTuple(args, "i", &n))
return Py_RETURN_NONE;
return Py_BuildValue("i", square(n));
}
Some of the C++ functions I'd like to wrap take in double arrays and modify them. This isn't a supported type by PyArg_ParseTuple. After doing some searching it is my understanding that using PyArg_ParseTuple to get a PyObject and then making that a PyArray_DOUBLE is an option. How would I go about doing this and how would I traverse such an array. I assume if I modify the array in the C++ extension that the Python version will be modified as well. Also, is NumPy the best way to make a double/float array to pass to my C++ extension?
Related
I have a boost python application that exports a class to Python, performs a calculation and returns the output back to C++:
import engine # c++ library
import glm # pyglm
class Game(engine.Application):
def __init__(self):
engine.Application.__init__(self, title, fullscreen)
self.shader = engine.Shader();
self.shader.setup("shader.vs", "shader.fs")
self.shader.setMat4("model", glm.mat4(1.0)
def update(self):
...
Amongst other classes, the shader is being wrapped and exported from C++ like this:
struct ShaderWrap : Shader, boost::python::wrapper<Shader>
{
int setup(const char* vertexPath, const char* fragmentPath)
{
return Shader::setup(vertexPath, fragmentPath);
}
int setMat4(std::string name, glm::mat4 mat)
{
return Shader::setMat4(name, mat);
}
};
...
BOOST_PYTHON_MODULE(engine)
{
namespace python = boost::python;
...
python::class_<ShaderWrap, boost::noncopyable>("Shader")
.def("setup", &Shader::setup)
.def("setMat4", &Shader::setMat4)
;
...
}
The application works fine up until the self.shader.setMat4 call and I get the error:
Boost.Python.ArgumentError: Python argument types in
StaticShader.setMat4(StaticShader, str, glm::detail::tmat4x4)
did not match C++ signature:
setMat4(StaticShader {lvalue}, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, glm::mat<4, 4, float, (glm::qualifier)0>)
I've tried to pass the mat4 value as a float* back to C++ with no success.
I've also tried calling value_ptr(mat4) which returns a c_void_p, but the compiler complains saying c_void_p and void* are not the same type (also tried various attempts of casting it with the ctypes library).
this question and this question are the basically the same problem but I got lost in the implementation.
In the PyGLM documentation, the developer says the library is written in C++. So,
What would be a possible way to pass through a glm.mat4 value to C++ to be interpreted as a glm::mat4 value?
I want to pass a glm.mat4 such as the view matrix, being updated in the Python code update function back to C++ to be further used, and was hoping there would be a fairly simple interface between the pyGLM library/GLM library through boost::python. Thanks in advance.
Ended up contacting the developer about this issue and this was there response:
This is what matrix objects look like in C++ code:
template<int C, int R, typename T>
struct mat {
PyObject ob_base;
uint8_t info;
glm::mat<C, R, T> super_type;
};
That is most likely why it's not quite compatible to glm matrices.
If you're using the latest version of PyGLM, value_ptr(mat4) should return a LP_c_float object, i.e. a float*.
I'm sure there is a way to transfer such a pointer over to boost::python, though as I said, I don't know how.
Technically you could convert the pointer to an integer, transfer that and cast it back to a float*.
You can get the address of a pointer by using:
int.from_bytes(ptr, "little")
Alternatively you could simply send over a bytes object.
as_bytes = bytes(mat4)
Or as a tuple:
as_tuple = mat4.to_tuple()
Or as a numpy array (which seems to have it's own boost::python::numpy module):
np_arr = numpy.array(mat4)
I was able to get the right output by passing the memory address as an int* back to C++ which then did some slicing to get it back into a glm::mat4, but I found this solution too convoluted and ended up wrapping the glm::mat4 class manually (see https://stackoverflow.com/a/62672081/9238288 for a basic example of how to achieve this).
According to Swig docs and the marvelous explanation at SWIG in typemap works, but argout does not by #Flexo, the argout typemap turns reference arguments into return values in Python.
I have a scenario, in which I pass a dict, which then is converted to an unordered_map in typemap(in), which then gets populated in the C++ lib. Stepping through the code, I can see the mapping changed after it returned from C++, so I wonder why there is not a possibility to just convert the unordered_map back in place in to the dict that was passed. Or is it possible by now and I'm just overlooking something?
Thanks!
I am a little confused as to what exactly you are asking, but my understanding is:
You have an "in" typemap to convert a Python dict to a C++ unordered_map for some function argument.
The function then modifies the unordered_map.
After completion of the function, you want the Python dict updated to the current unordered_map, and are somehow having trouble with this step.
Since you know how to convert a dict to an unordered_map, I assume you basically do know how to convert the unordered_map back to the dict using the Python C-API, but are somehow unsure into which SWIG typemap to put the code.
So, under these assumptions, I'll try to help:
"the argout typemap turns reference arguments into return values in Python". Not really, although it is mostly used for this purpose. An "argout" typemap simply supplies code to deal with some function argument (internally referred to as $1) that is inserted into the wrapper code after the C++ function is called. Compare this with an "in" typemap that supplies code to convert the supplied Python argument $input to a C++ argument $1, which is obviously inserted into the wrapper code before the C++ function is called.
The original passed Python dict can be referred to in the "argout" typemap as $input, and the modified unordered_map as $1 (see the SWIG docs linked above).
Therefore, all you need to do is write an "argout" typemap for the same argument signature as the "in" typemap that you already have, and insert the code (using the Python C-API) to update the contents of the Python dict ($input) to reflect the contents of the unordered_map ($1).
Note that this is different from the classical use of "argout" typemaps, which would typically convert the $1 back to a new Python dict and append this to the Python return object, which you can refer to by $result.
I hope this helps. If you are still stuck at some point, please edit your question to make clear at which point you are having trouble.
I am well aware of that the user has already solved his issue, but here goes a solution. Some validation of inputs may be introduced to avoid non-string values of the input dictionary.
Header file
// File: test.h
#pragma once
#include <iostream>
#include <string>
#include <unordered_map>
void method(std::unordered_map<std::string, std::string>* inout) {
for( const auto& n : (*inout) ) {
std::cout << "Key:[" << n.first << "] Value:[" << n.second << "]\n";
}
(*inout)["BLACK"] = "#000000";
};
Interface file
// File : dictmap.i
%module dictmap
%{
#include "test.h"
%}
%include "typemaps.i"
%typemap(in) std::unordered_map<std::string, std::string>* (std::unordered_map<std::string, std::string> temp) {
PyObject *key, *value;
Py_ssize_t pos = 0;
$1 = &temp;
temp = std::unordered_map<std::string, std::string>();
while (PyDict_Next($input, &pos, &key, &value)) {
(*$1)[PyString_AsString(key)] = std::string(PyString_AsString(value));
}
}
%typemap(argout) std::unordered_map<std::string, std::string>* {
$result = PyDict_New();
for( const auto& n : *$1) {
PyDict_SetItemString($result, n.first.c_str(),
PyString_FromString(n.second.c_str()));
}
}
%include "test.h"
Test
import dictmap
out = dictmap.method({'WHITE' : '#FFFFFF'})
Output is an updated dictionary
In[2]: out
Out[3] : {'BLACK': '#000000', 'WHITE': '#FFFFFF'}
Basically, it should look like this (in Pseudocode):
C++ file:
void do_stuff(){
printf("Callback", );
}
void main(){
call_python_function(&do_stuff);
}
Python file:
def python_function(ptr):
ptr()
Just calling Python from C++ with some parameters is relatively easy, using Python.h, but how do I do the callback to C++?
Also, I also want to pass around some data, so ideally, it should look something like this in the end:
C++ file:
vector<double> do_stuff(double a, double b){
vector<double> v;
for(int i=0; i<1000; i++) v.push_back(a*sin(b*i));
}
void main(){
call_python_function(&do_stuff);
}
Python file:
def python_function(ptr):
print ptr(1.2,3.4)
Is this possible somehow? I am considering using a message passing interface like ZeroMQ with some threading, but maybe there is a cleaner, and potentially simpler solution which looks more or less like what I have shown here?
May be you can try using swig (http://www.swig.org/) to wrap the C++ functions which can be called from python script.
I have been trying to get to grips with extending python with C, and so far, based on the documentation, I have had reasonable success in writing small C functions and extending it with Python.
However, I am now struck on a rather simple problem - to which I am not able to find a solution. So, what I'd like to do is pass a double list to my C function. For example, to pass an int, I do the following:
int squared(int n)
{
if (n > 0)
return n*n;
else
return 0;
}
static PyObject*
squaredfunc(PyObject* self, PyObject* args)
{
int n;
if (!PyArg_ParseTuple(args, "i", &n))
return NULL;
return Py_BuildValue("i", squared(n));
}
This passes the int n with no problems to my C function named squared.
But, how does one pass a list to the C function? I did try to google it and read the docs, and so far, I havent found anything useful on this.
Would really appreciate if someone could point me in the right direction.
Thanks.
PyArg_ParseTuple can only handle simple C types, complex numbers, char *, PyStringObject *, PyUnicodeObject *, and PyObject *. The only way to work with a PyListObject is by using some variant of "O" and extracting the object as a PyObject *. You can then use the List Object API to check that the object is indeed a list (PyList_Check). Then you can then use PyList_Size and PyList_GetItem to iterate over the list. Please note that when iterating, you will get PyObject * and will have to use the floating point API to access the actual values (by doing PyFloat_Check and PyFloat_AsDouble.) As an alternative to the List API, you can be more flexible and use the iterator protocol (in which case you should just use PyIter_Check). This will allow you to iterate over anything that supports the iterator protocol, like lists, tuples, sets, etc.
Finally, if you really want your function to accept double n[] and you want to avoid all of that manual conversion, then you should use something like boost::python. The learning curve and APIs are more complex, but boost::python will handle all of the conversions for you automatically.
Here is an example of looping using the iterator protocol (this is untested and you'd need to fill in the error handling code):
PyObject *obj;
if (!PyArg_ParseTuple(args, "O", &obj)) {
// error
}
PyObject *iter = PyObject_GetIter(obj);
if (!iter) {
// error not iterator
}
while (true) {
PyObject *next = PyIter_Next(iter);
if (!next) {
// nothing left in the iterator
break;
}
if (!PyFloat_Check(next)) {
// error, we were expecting a floating point value
}
double foo = PyFloat_AsDouble(next);
// do something with foo
}
The PyArg_ParseTuple function allows you to cast directly to a Python object subtype using the format string "O!" (notice-this is different than just plain "O"). If the argument does not match the specified PyObject type, it will throw a TypeError. For example:
PyObject *pList;
PyObject *pItem;
Py_ssize_t n;
int i;
if (!PyArg_ParseTuple(args, "O!", &PyList_Type, &pList)) {
PyErr_SetString(PyExc_TypeError, "parameter must be a list.");
return NULL;
}
n = PyList_Size(pList);
for (i=0; i<n; i++) {
pItem = PyList_GetItem(pList, i);
if(!PyInt_Check(pItem)) {
PyErr_SetString(PyExc_TypeError, "list items must be integers.");
return NULL;
}
}
As a side note, remember that iterating over the list using PyList_GetItem returns a borrowed reference to each item, so you do not need Py_DECREF(item) to handle the reference count. On the other hand, with the useful Iterator Protocol (see the answer by #NathanBinkert), each item returned is a new reference - so you must remember to discard it when done using Py_DECREF(item).
I'm working on embedding python into my C++ program using swig. At the moment I have a object written in C++ which I want to pass to a python function. I've created the swig interface to wrap the class.
What I'm trying to do is take this C++ object which I've created and pass it to a python function with the ability to use it like I would in C++. Is it possible for me to use code generate by swig to do this? If not how can I approach this?
You can use PyObject_CallMethod to pass a newly created object back to python. Assuming ModuleName.object is a python object with a method called methodName that you want to pass a newly created C++ object to you want to roughly (from memory, I can't test it right now) do this in C++:
int callPython() {
PyObject* module = PyImport_ImportModule("ModuleName");
if (!module)
return 0;
// Get an object to call method on from ModuleName
PyObject* python_object = PyObject_CallMethod(module, "object", "O", module);
if (!python_object) {
PyErr_Print();
Py_DecRef(module);
return 0;
}
// SWIGTYPE_p_Foo should be the SWIGTYPE for your wrapped class and
// SWIG_POINTER_NEW is a flag indicating ownership of the new object
PyObject *instance = SWIG_NewPointerObj(SWIG_as_voidptr(new Foo()), SWIGTYPE_p_Foo, SWIG_POINTER_NEW);
PyObject *result = PyObject_CallMethod(python_object, "methodName", "O", instance);
// Do something with result?
Py_DecRef(instance);
Py_DecRef(result);
Py_DecRef(module);
return 1;
}
I think I've got the reference counting right for this, but I'm not totally sure.