We Keep Coding

PyBind11: simple invoke task, standard C++ library not found

i am trying to wrap the C++ library PyrochloreAFM.hpp which itself uses the lib boost/randoom.hpp so that i can import it as a python module. The C++ part itself works fine and i can succesfully import and use all of this from my main.
#include "PyrochloreAFM.hpp"
int main (int argc, const char *argv[]) {
PyrochloreAFM pyrochloreAFM (&parameters, &statistics, &rng);
}
Now following a tutorial i set up my c++ wrapper:
// pybind11_wrapper.cpp
#include <pybind11/pybind11.h>
#include <PyrochloreAFM.hpp>
PYBIND11_MODULE(pybind11_example, m) {
m.doc() = "pybind11 example plugin"; // Optional module docstring
m.def("cpp_function", &PyrochloreAFM, "A function that multiplies two numbers");
}
and my tasks.py file
# tasks.py
import invoke
invoke.run(
"g++ -O3 -Wall -Werror -shared -std=c++11 -fPIC PyrochloreAFM.cpp "
"-o libpyro.so "
)
However now $invoke build-PyrochloreAFM or even $invoke --list seem to have lost the track of the standard C++ library.
In file included from PyrochloreAFM.cpp:1:
./Parameters.hpp:16:10: fatal error: 'boost/random.hpp' file not found
#include "boost/random.hpp" // tested with boost 1.53
^~~~~~~~~~~~~~~~~~
1 error generated.
This might be a simple PATH problem so i would be very glad for any tips!
Thank you, Andres!

"Of course" the tasks.py file which compiles the .cpp file through the python module invoke has to instruct the compiler g++ to use the libraries we desire. In my case it is the flag -I /opt/homebrew/Cellar/boost/1.80.0/include i also use in my makefile or directly in terminal.

How to wrap C++ VTK custom function for Python

Is there any place I can find a working example to wrap a C++ function (using VTK) to be used by
Python?
Searched and found this page
mix VTK and SWIG Python
but there is no compiling details, and this simple example on this page
https://public.kitware.com/pipermail/vtkusers/2002-October/013980.html
But for my system Ubuntu 16.04, vtk 8.2.0, Python 3.6.8, I could not make the example work.
I have to modify them to let the compiling done without error.
Details:
#============simple.cpp===================
// simple.c
//based on https://public.kitware.com/pipermail/vtkusers/2002-October/013980.html
#include "Python.h"
#include "vtkPythonUtil.h"
#include "vtkObject.h"
#include "vtkObjectBase.h"
#include <stdio.h>
#include "vtkRenderer.h"
extern "C"
{
static PyObject* simple_vtk_function(PyObject *self, PyObject *args);
}
static PyObject* simple_vtk_function(PyObject *self, PyObject *args)
{
PyObject *vtkpy;
if (!PyArg_ParseTuple(args, "O", &vtkpy))
return NULL;
vtkObjectBase *obj = vtkPythonUtil::GetPointerFromObject(vtkpy,"PyObject");
/* Do what you want to do here. */
//printf("%s\n", obj->GetClassName());
printf("inside module simple_vtk, function simple_vtk_function\n");
Py_RETURN_NONE;
}
static char simple_docs[] =
"simple( ): Any message you want to put here!!\n";
static PyMethodDef module_methods[] =
{
{
"simple_vtk_function", (PyCFunction) simple_vtk_function,
METH_VARARGS, simple_docs
},
{NULL}
};
static struct PyModuleDef simple_vtk =
{
PyModuleDef_HEAD_INIT,
"simple_vtk",/* name of module */
"",/* module documentation, may be NULL */
-1,
module_methods
};
PyMODINIT_FUNC PyInit_simple_vtk(void)
{
return PyModule_Create(&simple_vtk);
}
#============setup.py===================
#!/usr/bin/env python3
import distutils.core as dc
module = dc.Extension('simple_vtk',
sources = ['simple.cpp'],
include_dirs=['/usr/local/include/vtk-8.2'],
libraries_dirs = ['/usr/local/include/vtk-8.2'],
#libraries=['vtkPythonUtil']
)
dc.setup(name = 'simple_vtk',
version = '1.0',
ext_modules = [module])
compile command
CFLAGS="-std=c++11" python setup.py build
and get this printing out
/home/jun/anaconda3/lib/python3.6/distutils/extension.py:131: UserWarning: Unknown Extension options: 'libraries_dirs'
warnings.warn(msg)
running build
running build_ext
building 'simple_vtk' extension
gcc -pthread -B /home/jun/anaconda3/compiler_compat -Wl,--sysroot=/ -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c++11 -fPIC -I/usr/local/include/vtk-8.2 -I/home/jun/anaconda3/include/python3.6m -c simple.cpp -o build/temp.linux-x86_64-3.6/simple.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
simple.cpp: In function ‘PyObject* simple_vtk_function(PyObject*, PyObject*)’:
simple.cpp:22:19: warning: unused variable ‘obj’ [-Wunused-variable]
vtkObjectBase *obj = vtkPythonUtil::GetPointerFromObject(vtkpy,"PyObject");
^
g++ -pthread -shared -B /home/jun/anaconda3/compiler_compat -L/home/jun/anaconda3/lib -Wl,-rpath=/home/jun/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ -std=c++11 build/temp.linux-x86_64-3.6/simple.o -o build/lib.linux-x86_64-3.6/simple_vtk.cpython-36m-x86_64-linux-gnu.so
to test in python
>>> import simple_vtk
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /home/jun/project/play/swig/for_vtk_v2/simple_vtk.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN13vtkPythonUtil20GetPointerFromObjectEP7_objectPKc
The C++ function is so simple, it just to convert a PyObject and convert to vtkObjectBase. I didn't even remove the simple print
printf("%s\n", obj->GetClassName());
But I could never make it work. I think it is a common need for people to wrap a custom C++ vtk function for Python, a little bit surpring there is almost no woring example on internet.
Could you please help me on this? Thanks!

Creating a module for python2 and python3 with SWIG

I have a task in which I have to write the Python bindings for an existing C++ library. Since SWIG have support for not only Python but also for languages like Java and Perl, I am using SWIG. I am relatively new in SWIG so I have a doubt. I want my python library to be supportable under Python 2.7 and Python 3.x. But I don't have any idea regarding how to do this. So if anyone can suggest me. Any help would be appreciated. You can ask me for more details if you want.
What I have tried so far.
This is my code file.
/* example.c file */
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
int my_mod(int n, int m) {
return n % m;
}
int sieve(int number) {
int* arr = malloc(sizeof (int) * (number + 10));
int* prime = malloc(sizeof (int) * (number + 10));
/* printf("Size of arr: %lu", sizeof(arr));
printf("Size of int: %lu", sizeof(int)); */
memset(arr, 0, sizeof(int) * (number + 10));
int counter = 0;
prime[counter++] = 2;
arr[0] = arr[1] = 0;
for (int i = 3; i * i <= number; i += 2) {
if (!arr[i]) {
for (int j = i + i; j < number; j += i) {
arr[j] = 1;
}
}
}
for (int i = 3; i < number; i += 2)
if (!arr[i])
prime[counter++] = i;
// free(arr);
// free(prime);
return counter;
}
And my interface file is
/* example.i */
%module example
%{
#include "stdio.h"
#include "stdlib.h"
#include "string.h"
extern int my_mod(int n, int m);
extern int sieve(int number);
%}
extern int my_mod(int n, int m);
extern int sieve(int number);
My steps for compilation
swig -python example.i
gcc -fpic -c example.c example_wrap.c $(pkg-config --cflags --libs python3)
gcc -shared example.o example_wrap.o -o _example.so
In above compilation format module is working fine with python3 but failing in python2 with error log as
ImportError: ./_example.so: undefined symbol: PyUnicode_FromFormat
And if I use below compilation commands
swig -python example.i
gcc -fpic -c example.c example_wrap.c $(pkg-config --cflags --libs python2)
gcc -shared example.o example_wrap.o -o _example.so
Than module is working with python2 but when I try to import in python3 than error message is
ImportError: /home/deepanshu/env/swig/env/src/deep/_example.so: undefined symbol: PyInstance_Type
I am sure than the error is because $(pkg-config --cflags --libs pythonX) I am specifying version at place of X but how can I make sure my module works with both the Python versions?
I tried -py3 flag of swig, but I wasn't able to make module work for both of the Python version with above specified flag.

C extension don't work the same way in python3 vs python2. This link can help you identify what you need to do on the C level to have the library work for both python 2 and 3 (not using swig). http://python3porting.com/cextensions.html
Using Swig, what I would do is make use of the pip wheels naming convention :
https://packaging.python.org/tutorials/distributing-packages/#packaging-your-project
Doing so, I would package the library for python3 and make the package follow the wheel naming standard ("https://www.python.org/dev/peps/pep-0425/#id1" "py3-none-any") and then make the library for Python2 and package it using the same strategy("py2-none-any"). then you can upload it to Pypi and pip would know that it need to use the py2 for python2 and py3 for python3.

iterate returned vector<pair<int,int>> in python from SWIG bindings

I found this very helpful question and answer:
Return vector<pair<int,int>> & from c++ method to python list of tuples using swig typemap
However I have some problems iterating the return vector if it's not a reference, here the example:
myclass.h:
#include <vector>
#include <utility>
using std::vector;
using std::pair;
class MyClass {
private:
vector<pair<int,int> > _myvector;
public:
MyClass( );
const vector<pair<int,int> > & GetMyVector() const;
};
myclass.cpp:
#include "myclass.h"
MyClass::MyClass(): _myvector()
{_myvector.push_back(std::make_pair(1,2));_myvector.push_back(std::make_pair(1,2));};
const vector<pair<int,int>> & MyClass::GetMyVector() const {
return _myvector;
};
myclass.i:
%module x
%include <std_pair.i>
%include <std_vector.i>
%include <std_string.i>
%template() std::pair<int,int>;
%template(PairVector) std::vector<std::pair<int,int> >;
%{
#include "myclass.h"
%}
%include "myclass.h"
compiled with:
g++ -std=c++11 -c -fPIC myclass.cpp
swig -c++ -v -python myclass.i
g++ -std=c++11 -fPIC -c myclass.cpp myclass_wrap.cxx -I/usr/include/python2.7
g++ myclass.o myclass_wrap.o -shared -fPIC -o _x.so
but when I run something like this in python:
import x
b=x.MyClass()
print(b.GetMyVector())
for a,b in b.GetMyVector():
print(a,b)
then I get:
<Swig Object of type 'vector< std::pair< int,int >,std::allocator< std::pair< int,int > > > *' at 0x7ff06804b1b0>
Traceback (most recent call last):
File "Test.py", line 6, in <module>
for a,b in b.GetMyVector():
TypeError: 'SwigPyObject' object is not iterable
How can I iterate the returned vector properly in python? And why is a pointer to vector returned? Do I have to change something in the swig file?
In case it is relevant: (on Ubuntu)
SWIG Version 2.0.11
g++ (Ubuntu 4.9.4-2ubuntu1~14.04.1) 4.9.4
Python 2.7.6

SWIG doesn't understand the using directive correctly.
Same as in this question and answer:
SWIG argument error when using "using std::vector" in python
As to why a pointer is returned, well, if SWIG cannot convert a returned object into a python object, then it wraps a pointer to the object.

SWIG : AttributeError: 'module' object has no attribute 'fact'

I am studying how to use swig to make a C expansion for my python code.And I use the code I get from website as example.
Here is my code:
example.c
#include <time.h>
double My_variable = 3.0;
int fact(int n) {
if (n <= 1) return 1;
else return n*fact(n-1);
}
int my_mod(int x, int y) {
return (x%y);
}
example.h
#ifndef EXAMPLE_H_
#define EXAMPLE_H_
extern double My_variable;
extern int fact(int n);
extern int my_mod(int x, int y);
#endif
example.i
%module example
%{
/* Put header files here or function declarations like below */
#define SWIG_FILE_WITH_INIT
#include "example.h"
%}
%#include "example.h"
Makefile
all:
rm -f *.so *.o *_wrap.* *.pyc
swig -python example.i
gcc -c -fPIC example_wrap.c -I/usr/include/python2.7
gcc -shared example_wrap.o -o _example.so
clean:
rm -f *.so *.o *_wrap.* *.pyc
test.py
import example
print str(example.fact(2))
The test.py is used to check if the expansion works.But when I run the test.py , it output:
Traceback (most recent call last):
File "test.py", line 3, in <module>
print str(example.fact(2))
AttributeError: 'module' object has no attribute 'fact'
Here is the output when I use dir(example):
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '_example', '_newclass', '_object', '_swig_getattr', '_swig_property', '_swig_repr', '_swig_setattr', '_swig_setattr_nondynamic']
What's the reason of this output appears?
If I want to the programme run successfully,how should I do?

Please try to replace as below:
%#include "example.h"
by
%include "example.h"

Reference
Try changing the Makefile to
%module example
%{
#include "example.h"
%}
int fact(int n);
I've assumed you have only one method to export