We Keep Coding

PyBind11: simple invoke task, standard C++ library not found

i am trying to wrap the C++ library PyrochloreAFM.hpp which itself uses the lib boost/randoom.hpp so that i can import it as a python module. The C++ part itself works fine and i can succesfully import and use all of this from my main.
#include "PyrochloreAFM.hpp"
int main (int argc, const char *argv[]) {
PyrochloreAFM pyrochloreAFM (&parameters, &statistics, &rng);
}
Now following a tutorial i set up my c++ wrapper:
// pybind11_wrapper.cpp
#include <pybind11/pybind11.h>
#include <PyrochloreAFM.hpp>
PYBIND11_MODULE(pybind11_example, m) {
m.doc() = "pybind11 example plugin"; // Optional module docstring
m.def("cpp_function", &PyrochloreAFM, "A function that multiplies two numbers");
}
and my tasks.py file
# tasks.py
import invoke
invoke.run(
"g++ -O3 -Wall -Werror -shared -std=c++11 -fPIC PyrochloreAFM.cpp "
"-o libpyro.so "
)
However now $invoke build-PyrochloreAFM or even $invoke --list seem to have lost the track of the standard C++ library.
In file included from PyrochloreAFM.cpp:1:
./Parameters.hpp:16:10: fatal error: 'boost/random.hpp' file not found
#include "boost/random.hpp" // tested with boost 1.53
^~~~~~~~~~~~~~~~~~
1 error generated.
This might be a simple PATH problem so i would be very glad for any tips!
Thank you, Andres!

"Of course" the tasks.py file which compiles the .cpp file through the python module invoke has to instruct the compiler g++ to use the libraries we desire. In my case it is the flag -I /opt/homebrew/Cellar/boost/1.80.0/include i also use in my makefile or directly in terminal.

How to wrap C++ VTK custom function for Python

Is there any place I can find a working example to wrap a C++ function (using VTK) to be used by
Python?
Searched and found this page
mix VTK and SWIG Python
but there is no compiling details, and this simple example on this page
https://public.kitware.com/pipermail/vtkusers/2002-October/013980.html
But for my system Ubuntu 16.04, vtk 8.2.0, Python 3.6.8, I could not make the example work.
I have to modify them to let the compiling done without error.
Details:
#============simple.cpp===================
// simple.c
//based on https://public.kitware.com/pipermail/vtkusers/2002-October/013980.html
#include "Python.h"
#include "vtkPythonUtil.h"
#include "vtkObject.h"
#include "vtkObjectBase.h"
#include <stdio.h>
#include "vtkRenderer.h"
extern "C"
{
static PyObject* simple_vtk_function(PyObject *self, PyObject *args);
}
static PyObject* simple_vtk_function(PyObject *self, PyObject *args)
{
PyObject *vtkpy;
if (!PyArg_ParseTuple(args, "O", &vtkpy))
return NULL;
vtkObjectBase *obj = vtkPythonUtil::GetPointerFromObject(vtkpy,"PyObject");
/* Do what you want to do here. */
//printf("%s\n", obj->GetClassName());
printf("inside module simple_vtk, function simple_vtk_function\n");
Py_RETURN_NONE;
}
static char simple_docs[] =
"simple( ): Any message you want to put here!!\n";
static PyMethodDef module_methods[] =
{
{
"simple_vtk_function", (PyCFunction) simple_vtk_function,
METH_VARARGS, simple_docs
},
{NULL}
};
static struct PyModuleDef simple_vtk =
{
PyModuleDef_HEAD_INIT,
"simple_vtk",/* name of module */
"",/* module documentation, may be NULL */
-1,
module_methods
};
PyMODINIT_FUNC PyInit_simple_vtk(void)
{
return PyModule_Create(&simple_vtk);
}
#============setup.py===================
#!/usr/bin/env python3
import distutils.core as dc
module = dc.Extension('simple_vtk',
sources = ['simple.cpp'],
include_dirs=['/usr/local/include/vtk-8.2'],
libraries_dirs = ['/usr/local/include/vtk-8.2'],
#libraries=['vtkPythonUtil']
)
dc.setup(name = 'simple_vtk',
version = '1.0',
ext_modules = [module])
compile command
CFLAGS="-std=c++11" python setup.py build
and get this printing out
/home/jun/anaconda3/lib/python3.6/distutils/extension.py:131: UserWarning: Unknown Extension options: 'libraries_dirs'
warnings.warn(msg)
running build
running build_ext
building 'simple_vtk' extension
gcc -pthread -B /home/jun/anaconda3/compiler_compat -Wl,--sysroot=/ -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c++11 -fPIC -I/usr/local/include/vtk-8.2 -I/home/jun/anaconda3/include/python3.6m -c simple.cpp -o build/temp.linux-x86_64-3.6/simple.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
simple.cpp: In function ‘PyObject* simple_vtk_function(PyObject*, PyObject*)’:
simple.cpp:22:19: warning: unused variable ‘obj’ [-Wunused-variable]
vtkObjectBase *obj = vtkPythonUtil::GetPointerFromObject(vtkpy,"PyObject");
^
g++ -pthread -shared -B /home/jun/anaconda3/compiler_compat -L/home/jun/anaconda3/lib -Wl,-rpath=/home/jun/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ -std=c++11 build/temp.linux-x86_64-3.6/simple.o -o build/lib.linux-x86_64-3.6/simple_vtk.cpython-36m-x86_64-linux-gnu.so
to test in python
>>> import simple_vtk
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /home/jun/project/play/swig/for_vtk_v2/simple_vtk.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN13vtkPythonUtil20GetPointerFromObjectEP7_objectPKc
The C++ function is so simple, it just to convert a PyObject and convert to vtkObjectBase. I didn't even remove the simple print
printf("%s\n", obj->GetClassName());
But I could never make it work. I think it is a common need for people to wrap a custom C++ vtk function for Python, a little bit surpring there is almost no woring example on internet.
Could you please help me on this? Thanks!

importing opencv in python script called from Qt crashes

I'm embedding Python in Qt C++. When I import cv2 in the Python script it crashes with segmentation fault. Running the script in Python works. Running from C++ without Qt works.
Python scrip: works
print("Importing cv2")
import cv2
print("cv2 imported")
C++: works
#include <Python.h>
#include <string>
int main(int argc, char *argv[])
{
std::string module_name("PythonModule");
Py_Initialize();
PyObject* sysPath = PySys_GetObject((char*)"path");
PyObject* pModuleDirPath = PyUnicode_FromString(".");
PyList_Append(sysPath, pModuleDirPath);
Py_DECREF(pModuleDirPath);
PyObject* pModuleName = PyUnicode_FromString(module_name.c_str());
PyObject* pModule = PyImport_Import(pModuleName);
Py_DECREF(pModuleName);
Py_DECREF(pModule);
Py_Finalize();
return 0;
}
Qt: segmentation fault after printing "Importing cv2"
MainWindow::MainWindow(QWidget *parent) :
QMainWindow(parent),
ui(new Ui::MainWindow)
{
ui->setupUi(this); 
 
std::string module_name("PythonModule");
 
Py_Initialize();
 
PyObject* sysPath = PySys_GetObject((char*)"path");
PyObject* pModuleDirPath = PyUnicode_FromString(".");
PyList_Append(sysPath, pModuleDirPath);
Py_DECREF(pModuleDirPath);
 
PyObject* pModuleName = PyUnicode_FromString(module_name.c_str());
PyObject* pModule = PyImport_Import(pModuleName);
 
Py_DECREF(pModuleName);
Py_DECREF(pModule);
 
Py_Finalize();
}
.pro file:
LIBS += -L /usr/local/lib/python3.5m -lpython3.5m
 
INCLUDEPATH += /usr/include/python3.5m
Versions:
Ubuntu 16.04
Python 3.5
opencv-python 3.4.2.17
Qt Creator 4.7.0 Based on Qt 5.11.1
Do anyone recognize this problem?

The problem was that opencv uses Qt as well so a collision occurred.
The solution was to install opencv without Qt support:
pip install opencv-contrib-python-headless
https://pypi.org/project/opencv-contrib-python-headless/

SWIG Python undefined symbol error

I'm trying to create a *.so file for further use in Python using SWIG, but something isn't working.
I have two files:
DataGatherer.h
#include <iostream>
#include <stdlib.h>
#include <time.h>
#include "gnublin.h"
#include <pthread.h>
class dataGatherer
{
private:
int threshold;
int timeThreshold;
int data[4096];
bool running;
gnublin_spi* spiDevice;
pthread_t spiThread;
void *params;
public:
dataGatherer(void);
dataGatherer(int, int);
void initData();
int getThreshold(void);
int* getData(void);
int getPeak(void);
void initSPI(void);
void gatherData();
void * run(void * arg);
void stop(void);
// for use of thread we have to implement some methods from C
static void * start_static(void * params)
{
dataGatherer * thread_this = static_cast<dataGatherer*>(params);
return thread_this->run(thread_this->params);
}
void start(void * params)
{
this->params = params;
pthread_create(&spiThread, 0, &dataGatherer::start_static, this);
}
};
and spiController.h
#include "dataGatherer.h"
class spiController
{
private:
bool runGather;
dataGatherer* gatherer;
int data[4096];
public:
spiController(void);
spiController(int, int);
void initData();
bool checkStop();
void stop();
void start();
};
My spiController.i interface file looks like this:
/* spiController.i */
%module spiController
%{
#include "dataGatherer.h"
#include "spiController.h"
#include "gnublin.h"
#include <stdlib.h>
#include <time.h>
#include <pthread.h>
extern void initData();
extern bool checkStop();
extern void stop();
extern void start();
%}
extern void initData();
extern bool checkStop();
extern void stop();
extern void start();
At the end I try to create the *.so file using commands in the terminal like in the example on the SWIG page with:
swig -python -c++ spiController.i
c++ -c spiController_wrap.c -I/usr/include/python2.7
c++ -shared spiController_wrap.o -o _spiController.so
*.cxx, *.o and *.so file are created with no error, but when I import the spiController into the python code I get:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "spiController.py", line 26, in <module>
_spiController = swig_import_helper()
File "spiController.py", line 22, in swig_import_helper
_mod = imp.load_module('_spiController', fp, pathname, description)
ImportError: ./_spiController.so: undefined symbol: _Z9checkStopv
It's my first try using SWIG and I'm already stuck at this point. How can I resolve this?

I just got the same error and finally figured out why. As above people said, when it says unfound symbol like yours and gives the undefined function name '_Z9checkStopv', always check for the implementation of this function in .cpp file as well as any function declaration of the same name!!
For my case, my cpp does define my 'unfound symbol' constructor function, but in my .h file, i have an overloaded operator= (for the constructor) which is undefined in .cpp file. So swig wraps both default constructor(implemented in .cpp) and operator= (not implemented). Therefore when import, this unimplemented operator= produces the error. Hope this helps!

You must link the library that defines your C++ functions that you have declared like checkStop etc. You would add -L<path to your C++ DLL> -l<name of your C++ DLL> on 3rd line of your example's compile steps.
Like this:
c++ -L<path to DLL> -l<name of your dll> -shared spiController_wrap.o -o _spiController.so

Just as Adam's comment and my experience, you should first compile your XXX.cpp file into XXX.o, the whole command lines maybe like following:
swig -python -c++ XXX.i
g++ -c -fpic XXX.cpp* (this command will generate XXX.o file)
g++ -c -fpic XXX_wrap.cxx -I/usr/include/python2.7* (this command will generate XXX_wrap.o file)
g++ -shared XXX.o XXX_wrap.o -o XXX.so

Although there may be many causes for this problem, I got the exact same error when I compiled the shared library with the python v3.5 headers, e.g.
swig -python example.i
gcc -fPIC -c example.c example_wrap.c -I/usr/include/python3.5 # <-- ISSUE HERE
gcc -shared example.o example_wrap.o -o _example.so
But then later tried to use the example library using python test.py, which ran python v2.7 on system (so it was a python version mismatch issue).

In my case I was also getting that error, and spent some time trying out stuff without success. My problem was that although my source file was plain C I had it named with the .cpp extension, assuming it wouldn't matter. Changing the extension to .c solved automatically the issue.
Another way of solving it was to add the line #include "example.cpp" to the header section of SWIG's .i file.
So, summarizing:
example.c
int fact(int n) {
if (n <= 1) return 1;
else return n*fact(n-1);
}
example.i
%module example
%{
extern int fact(int n);
%}
extern int fact(int n);
Then the following worked for me (in Ubuntu 17.10):
swig -c++ -python example.i
gcc -fPIC -c example.c example_wrap.c -I /usr/include/python2.7
gcc -shared example.o example_wrap.o -o _example.so
python -c "import example; print example.fact(5)"
Hope this helps someone!
Cheers

I know the solution
At the end of make up the share,you should usr g++ -shared -fpic *.o *.o -o _***.so

Building a DLL with MinGW and loading it with Python's ctypes module

I am trying to build a C DLL which can be loaded within python using ctypes.windll.loadlibrary(...)
I can create a DLL and a client program all in C which work following the MinGW tutorial at http://www.mingw.org/wiki/MSVC_and_MinGW_DLLs.
When I try to load the same DLL within python I get an error:
OSError: [WinErrror 193] %1 is not a valid Win32 application
Can someone give me some idea as to what I am doing incorrectly?
Here are the files:
noise_dll.h
#ifndef NOISE_DLL_H
#define NOISE_DLL_H
// declspec will identify which functions are to be exported when
// building the dll and imported when 'including' this header for a client
#ifdef BUILDING_NOISE_DLL
#define NOISE_DLL __declspec(dllexport)
#else
#define NOISE_DLL __declspec(dllimport)
#endif
//this is a test function to see if the dll is working
// __stdcall => use ctypes.windll ...
int __stdcall NOISE_DLL hello(const char *s);
#endif // NOISE_DLL_H
noise_dll.c
#include <stdio.h>
#include "noise_dll.h"
__stdcall int hello(const char *s)
{
printf("Hello %s\n", s);
return 0;
}
I build the DLL with:
gcc -c -D BUILDING_NOISE_DLL noise_dll.c
gcc -shared -o noise_dll.dll noise_dll.o -Wl,--out-implib,libnoise_dll.a
The python code is simply:
import ctypes
my_dll = ctypes.windll.LoadLibrary("noise_dll")
and I get the error above: '%1 is not not a valid Win32 application'
I know the DLL is not completely wrong, because if i create a client file:
noise_client.c
#include <stdio.h>
#include "noise_dll.h"
int main(void)
{
hello("DLL");
return 0;
}
and build with:
gcc -c noise_client.c
gcc -o noise_client.exe noise_client.o -L. -lnoise_dll
I get a working executable. I have some understanding of all of what goes on in the code, options and preprocessor directives above, but am still a little fuzzy on how the .dll file and the .a file are used. I know if I remove the .a file I can still build the client, so I am not even sure what its purpose is. All I know is that it is some kind of archive format for multiple object files
I can ctypes.windll.loadlibrary(...) an ordinary windows DLL found in the windows/system32 without an issue.
One final point:
I am using 64 bit python 3.3. I am using the version of minGW tat comes with the recommended installer (mingw-get-inst-20120426.exe). I am not sure if it is 32 bit, or if that matters.
Thanks!

Use 32-bit python - your DLL probably wasn't compiled as 64 bit code.