We Keep Coding

PyBind11 Using Qt results in ImportError when importing library in Python

I've been struggling to get an example of pybind11 with Qt working. I can import other libraries like VTK fine, but when I include a Qt library, say QString, and create a simple QString object inside one of my functions, the built library has an import error when it's being imported in Python. I am not sure how to debug these issues, as there is no useful error anywhere that I can see. I tried to look at docs, but they don't show a way to debug these errors. There are no warnings or other issues when building the library.
>>> import pyLib
ImportError: DLL load failed while importing pyLib: The specified module could not be found.
I tried to create a minimal example below. The executable target pyLib2 builds and runs just fine, but the pyLib python library target doesn't work when imported due to this line QString x;. Without it, it works fine:
CMake
cmake_minimum_required (VERSION 3.24)
project (pybindTest)
include_directories(${PROJECT_BINARY_DIR} src)
# set C++ settings
set (CXX_VERSION 20) # sets which standard of C we are using, e.g. C++20
set (CMAKE_CXX_FLAGS "/EHsc /O2 /favor:INTEL64 /W4 /MP -std:c++${CXX_VERSION}")
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION TRUE)
add_subdirectory (external/pybind11)
add_executable(pyLib2 main.cpp)
pybind11_add_module(pyLib pyLib.cpp)
target_include_directories(pyLib PUBLIC
"C:/.../external/pybind11/include"
)
find_package (Qt5 5.15.2 EXACT COMPONENTS CONFIG REQUIRED Core Widgets SerialPort Network)
target_link_libraries(pyLib PUBLIC
Qt5::Core
Qt5::Widgets
Qt5::SerialPort
)
target_link_libraries(pyLib2 PUBLIC
Qt5::Core
Qt5::Widgets
Qt5::SerialPort
)
pyLib.cpp
#include <pybind11/pybind11.h>
#include <QString>
#include <array>
#include <iostream>
namespace py = pybind11;
float test(float a)
{
QString x; // Commenting this line out works fine
return a * 2.0;
}
void test2()
{
std::cout << "test2!" << std::endl;
}
void init_pyLib(py::module& handle)
{
std::cout << "here!" << std::endl;
}
PYBIND11_MODULE(pyLib, handle)
{
handle.doc() = "test doc";
handle.def("testpy", &test, py::arg("i"));
handle.def("testpy2", &test2);
init_pyLib(handle);
}
main.cpp
#include <QString>
#include <array>
#include <iostream>
float test(float a)
{
QString x;
return a * 2.0;
}
void test2()
{
std::cout << "test2!" << std::endl;
}
void init_pyLib()
{
std::cout << "here!" << std::endl;
}
int main()
{
std::cout << "hello!\n";
QString x;
test(5.0f);
std::cout << "goodbye!\n";
}

Boost Python issue with converters - static linking

I have a question regarding the below code.
It's an example how to pass a custom class via shared_ptr to embedded python code and it works when boost is dynamically linked.
Unfortunately the same code with statically linked boost doesn't work with the following error message:
"No to_python (by-value) converter found for C++ type: class boost::shared_ptr".
I don't understand why a different linking can affect type recognition of a registered converter. What am I missing?
Can anybody help me out?
Thanks,
Dominik
Example from here.
#include <boost/shared_ptr.hpp>
#include <boost/make_shared.hpp>
#include <boost/python.hpp>
#include <string>
#include <iostream>
namespace bp = boost::python;
struct Foo{
Foo(){}
Foo(std::string const& s) : m_string(s){}
void doSomething() {
std::cout << "Foo:" << m_string << std::endl;
}
std::string m_string;
};
typedef boost::shared_ptr<Foo> foo_ptr;
BOOST_PYTHON_MODULE(hello)
{
bp::class_<Foo, foo_ptr>("Foo")
.def("doSomething", &Foo::doSomething)
;
};
int main(int argc, char **argv)
{
Py_Initialize();
try {
PyRun_SimpleString(
"a_foo = None\n"
"\n"
"def setup(a_foo_from_cxx):\n"
" print 'setup called with', a_foo_from_cxx\n"
" global a_foo\n"
" a_foo = a_foo_from_cxx\n"
"\n"
"def run():\n"
" a_foo.doSomething()\n"
"\n"
"print 'main module loaded'\n"
);
foo_ptr a_cxx_foo = boost::make_shared<Foo>("c++");
inithello();
bp::object main = bp::object(bp::handle<>(bp::borrowed(
PyImport_AddModule("__main__")
)));
// pass the reference to a_cxx_foo into python:
bp::object setup_func = main.attr("setup");
setup_func(a_cxx_foo);
// now run the python 'main' function
bp::object run_func = main.attr("run");
run_func();
}
catch (bp::error_already_set) {
PyErr_Print();
}
Py_Finalize();
return 0;
}

I far as I understand the documentation about Boost Python linkage, it seems that the conversion registry used for automatic conversion of Python object into C++ object is not available when statically linked. I'm facing the same issue and that's a pity it is actually the case. I would have imagined at least the required converter to be bundle but I'm afraid it is not the case for some reason.

Embedding Python into C - can't import method from python module

I'm building C application which will be using Python plugins. When trying to call the method from another Python module, the function PyImport_ImportModule() seems to imports the module properly, then i try to get the function from this module using PyObject_GetAttrString() and all that I get is null.
I already tried using PyModule_GetDict() and PyDict_GetItemString() to get the method from the module, but the effect was the same.
main.c:
#include <stdio.h>
#include <python3.6/Python.h>
int main()
{
PyObject *arg, *pModule, *ret, *pFunc, *pValue, *pMethod, *pDict;
Py_Initialize();
PyObject *sys = PyImport_ImportModule("sys");
PyObject *path = PyObject_GetAttrString(sys, "path");
PyList_Append(path, PyUnicode_FromString("."));
pModule = PyImport_ImportModule("test");
if(pModule == NULL)
{
perror("Can't open module");
}
pMethod = PyObject_GetAttrString(pModule, "myfun");
if(pMethod == NULL)
{
perror("Can't find method");
}
ret = PyEval_CallObject(pMethod, NULL);
if(ret == NULL)
{
perror("Couldn't call method");
}
PyArg_Parse(ret, "&d", pValue);
printf("&d \n", pValue);
Py_Finalize();
return 0;
}
test.py:
def myfun():
c = 123 + 123
print('the result is: ', c)
myfun()
The result i got is:
Can't find method: Success
Segmentation fault (core dumped)
When I used the gdb debugger the output was:
pModule = (PyObject *) 0x7ffff5a96f48
pMethod = (PyObject *) 0x0

Your program is not wroking because the module being imported is the test built-in module, rather than your test.py script. This is because you are appending the current directory to sys.path, so it is checked after every other already existing path in the list. You should insert it at the beginning of the list instead, so that it is checked first.
This will work:
PyObject *sys = PyImport_ImportModule("sys");
PyObject *path = PyObject_GetAttrString(sys, "path");
PyList_Insert(path, 0, PyUnicode_FromString("."));
By the way, you should #include the Python header before anything else, as stated in the documentation:
Note: Since Python may define some pre-processor definitions which affect the standard headers on some systems, you must include Python.h before any standard headers are included.

Swig downcasting from Base* to Derived*

I have the following c++ classes (simplified) which I am exposing to Python using SWIG:
struct Component
{
virtual void update();
}
struct DerivedComponent : public Component
{
void update() { cout << "DerivedComponent::update()" << endl; }
void speak() { cout << "DerivedComponent::speak()" << endl; }
}
class Entity
{
public:
Component* component(const std::string& class_name)
{
return m_components[class_name];
}
private:
std::unordered_map<std::string, Component*> m_components;
}
Now, in Python I can successfully call component("DerivedComponent").update() on an Entity instance. However, I cannot call component("DerivedComponent").speak() since the type returned by component("DerivedComponent") is reported as <class 'module.Component'>.
I obviously need to downcast the result of the component() function in order to call methods defined in DerivedComponent. I had hoped that Swig would perform automatic downcasting like I believe that Boost.Python does.
Short of defining a whole bunch of typecasting functions in c++ and exposing them to Python, is there any better solution for downcasting using either Swig or Python? What are my options?

You can do exactly what you want in Python, with a little work. It works as you hope because in Python downcasting is kind of meaningless as the return types of functions (or types in general) aren't strongly typed, so we can modify your Entity::component function to always return the most derived type no matter what it is.
To make that work with your C++/Python binding you need to write an 'out' typemap for Entity::component. I've written an example of how it might work. In this case we have to bodge it slightly because the only way to know what to downcast it to comes from the argument to the function. (If for example your base class had a method that returned this as a string/enum you could simplify this further and not depend on the input arguments).
%module test
%{
#include "test.hh"
%}
%include <std_string.i>
%typemap(out) Component * Entity::component {
const std::string lookup_typename = *arg2 + " *";
swig_type_info * const outtype = SWIG_TypeQuery(lookup_typename.c_str());
$result = SWIG_NewPointerObj(SWIG_as_voidptr($1), outtype, $owner);
}
%include "test.hh"
This uses the SWIG_TypeQuery function to ask the Python runtime to lookup the type based on arg2 (which for your example is the string).
I had to make some changes to your example header (named test.hh in my example) to fix a few issues before I could make this into a fully working demo, it ended up looking like:
#include <iostream>
#include <map>
#include <string>
struct Component
{
virtual void update() = 0;
virtual ~Component() {}
};
struct DerivedComponent : public Component
{
void update() { std::cout << "DerivedComponent::update()" << std::endl; }
void speak() { std::cout << "DerivedComponent::speak()" << std::endl; }
};
class Entity
{
public:
Entity() {
m_components["DerivedComponent"] = new DerivedComponent;
}
Component* component(const std::string& class_name)
{
return m_components[class_name];
}
private:
std::map<std::string, Component*> m_components;
};
I then built it with:
swig -py3 -c++ -python -Wall test.i
g++ -Wall -Wextra test_wrap.cxx -I/usr/include/python3.4/ -lpython3.4m -shared -o _test.so
With this in place I could then run the following Python:
from test import *
e=Entity()
print(e)
c=e.component("DerivedComponent")
print(c)
print(type(c))
c.update()
c.speak()
This works as you'd hope:
<test.Entity; proxy of <Swig Object of type 'Entity *' at 0xb7230458> >
Name is: DerivedComponent *, type is: 0xb77661d8
<test.DerivedComponent; proxy of <Swig Object of type 'DerivedComponent *' at 0xb72575d8> >
<class 'test.DerivedComponent'>
DerivedComponent::update()
DerivedComponent::speak()

I was looking to do something similar and came up with a similar but different solution based on this question.
If you know the possible types ahead of time and don't mind the extra overhead, you can have the 'out' typemap loop through and dynamic_cast to each to automatically return the object with its real type. SWIG already has this implemented for pointers with the %factory feature:
%factory(Component* /* or add method name. this is just the typemap filter */,
DerivedComponent1,
DerivedComponent2);
Looking at factory.swg and boost_shared_ptr.i I got this working for shared_ptr and dynamic_pointer_cast as well:
%define %_shared_factory_dispatch(Type)
if (!dcast) {
SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<Type> dobj
= SWIG_SHARED_PTR_QNAMESPACE::dynamic_pointer_cast<Type>($1);
if (dobj) {
dcast = 1;
SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<Type> *smartresult
= dobj ? new SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<Type>(dobj) : 0;
%set_output(SWIG_NewPointerObj(%as_voidptr(smartresult),
$descriptor(SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<Type> *),
SWIG_POINTER_OWN));
}
}%enddef
%define %shared_factory(BaseType,Types...)
%typemap(out) SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<BaseType> {
int dcast = 0;
%formacro(%_shared_factory_dispatch, Types)
if (!dcast) {
SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<BaseType> *smartresult
= $1 ? new SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<BaseType>($1) : 0;
%set_output(SWIG_NewPointerObj(%as_voidptr(smartresult),
$descriptor(SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<BaseType> *),
SWIG_POINTER_OWN));
}
}%enddef
// Apply dynamic_pointer cast to all returned shared_ptrs of this type
%factory(Component /* must be a type for shared_ptr */,
DerivedComponent1,
DerivedComponent2);

Extract constant value from python by boost python

codes of python(which work)
import ldap,ldif
l = ldap.initialize('ldaps://RIO-PC:636')
l.set_option(ldap.OPT_TIMEOUT);
I want to extract the constant value of ldap.OPT_TIMEOUT, but how to?
#include <iostream>
#include <boost/python.hpp>
int main()
{
namespace bp = boost::python;
try{
Py_Initialize();
bp::object main_module = bp::import("__main__");
bp::import("ldap");
bp::import("ldif");
bp::object main_namespace = main_module.attr("__dict__");
bp::exec("import ldap,ldif\n"
"l = ldap.initialize('ldaps://RIO-PC')\n",
main_namespace);
boost::python::object ldap = boost::python::extract<boost::python::object>(main_namespace["l"]);
ldap.attr("OPT_TIMEOUT"); //this line will throw exception
}catch(boost::python::error_already_set const &){
PyErr_Print();
PyErr_Clear();
}
}
the error message is
AttributeError: SimpleLDAPObject has no attribute 'OPT_TIMEOUT'
environment
compiler : vc2008
boost version : 1.55_0
os : win7 64bits(x64)

Your C++ code is trying to extract l.OPT_TIMEOUT, not ldap.OPT_TIMEOUT. Try
bp::object ldap_module = bp::import("ldap");
ldap_module.attr("OPT_TIMEOUT");