I have a question regarding the below code.
It's an example how to pass a custom class via shared_ptr to embedded python code and it works when boost is dynamically linked.
Unfortunately the same code with statically linked boost doesn't work with the following error message:
"No to_python (by-value) converter found for C++ type: class boost::shared_ptr".
I don't understand why a different linking can affect type recognition of a registered converter. What am I missing?
Can anybody help me out?
Thanks,
Dominik
Example from here.
#include <boost/shared_ptr.hpp>
#include <boost/make_shared.hpp>
#include <boost/python.hpp>
#include <string>
#include <iostream>
namespace bp = boost::python;
struct Foo{
Foo(){}
Foo(std::string const& s) : m_string(s){}
void doSomething() {
std::cout << "Foo:" << m_string << std::endl;
}
std::string m_string;
};
typedef boost::shared_ptr<Foo> foo_ptr;
BOOST_PYTHON_MODULE(hello)
{
bp::class_<Foo, foo_ptr>("Foo")
.def("doSomething", &Foo::doSomething)
;
};
int main(int argc, char **argv)
{
Py_Initialize();
try {
PyRun_SimpleString(
"a_foo = None\n"
"\n"
"def setup(a_foo_from_cxx):\n"
" print 'setup called with', a_foo_from_cxx\n"
" global a_foo\n"
" a_foo = a_foo_from_cxx\n"
"\n"
"def run():\n"
" a_foo.doSomething()\n"
"\n"
"print 'main module loaded'\n"
);
foo_ptr a_cxx_foo = boost::make_shared<Foo>("c++");
inithello();
bp::object main = bp::object(bp::handle<>(bp::borrowed(
PyImport_AddModule("__main__")
)));
// pass the reference to a_cxx_foo into python:
bp::object setup_func = main.attr("setup");
setup_func(a_cxx_foo);
// now run the python 'main' function
bp::object run_func = main.attr("run");
run_func();
}
catch (bp::error_already_set) {
PyErr_Print();
}
Py_Finalize();
return 0;
}
I far as I understand the documentation about Boost Python linkage, it seems that the conversion registry used for automatic conversion of Python object into C++ object is not available when statically linked. I'm facing the same issue and that's a pity it is actually the case. I would have imagined at least the required converter to be bundle but I'm afraid it is not the case for some reason.
Related
I am trying to embed python script into c++ project.
Below is what I have tried so far.
#include<iostream>
#include <Python.h>
int
main()
{
Py_Initialize();
PyObject* sysPath = PySys_GetObject("path");
PyObject* modPath = PyBytes_FromString("C:\\Users\\naal\\Documents\\Visual Studio 2017\\Projects\\Project1\pyscripts");
int result = PyList_Insert(sysPath,0, modPath);
PyObject *pModule = PyImport_ImportModule("myscript2");
printf("%p\n", pModule);
return 0;
}
below is the python script "myscript2.py"
def find_me():
print("hey you found me")
The problem is, the main module is not able to find the python script i.e object pyModule is always NULL, no matter how I change python script path.
What am I doing wrong ?
I ended up implementing this in another way.
#include<iostream>
#include <Python.h>
int main() {
std::string location = "C:\\Users\\myscript.py";
const char* file_location = location.c_str(); FILE* file_pointer;
Py_Initialize();
file_pointer = _Py_fopen(file_location, "r");
PyRun_SimpleFile(file_pointer, file_location);
Py_Finalize();
return 1;
}
The above seemed to work. I still don't know why the SYSPATH idea originially mentioned in the question didn't work.
I'm trying to create an instance of Game, pass it into the main namespace of the test.py as the variable game, then call game.add(e) to run the C++ function that will add Entity e into the std::vector. However, this code produces the error:
unbound method Boost.Python.function object must be called with Game instance as first argument (got Entity instance instead)
(Some context: I'm trying to let Python create instances that will be kept in a container for C++ to run through every tick and update. I thought I had it working a few weeks ago but I came back to it and apparently it wasn't working - I know, source control.)
#include <vector>
class Entity{
public:
Entity(){}
Entity(float x, float y){}
};
class Game{
public:
Game();
void add(Entity* entity);
private:
std::vector<Entity*> objects_;
};
Game::Game(){
}
void Game::add(Entity* entity){
objects_.push_back(entity);
}
main.cpp:
#include <iostream>
#include <boost/python.hpp>
#include "Game.h"
#include "Entity.h"
using namespace boost::python;
BOOST_PYTHON_MODULE(sfgame){
class_<Game>("Game")
.def("add", &Game::add)
;
class_<Entity>("Entity", init<float, float>())
;
}
int main(){
PyImport_AppendInittab("sfgame", &initsfgame);
Py_Initialize();
object main_module = import("__main__");
object main_namespace = main_module.attr("__dict__");
import("sfgame");
Game* game = new Game();
try{
main_namespace["game"] = ptr(game);
exec_file("test.py", main_namespace);
}
catch (const boost::python::error_already_set &){
PyObject *ptype, *pvalue, *ptraceback;
PyErr_Fetch(&ptype, &pvalue, &ptraceback);
std::string error;
error = boost::python::extract<std::string>(pvalue);
std::cout << error << std::endl;
}
delete game;
system("PAUSE");
return 0;
}
test.py:
from sfgame import *
e = Entity(5,5)
game.add(e)
You would receive that error if you set the variable name in the main namespace to Game, as it would be the same as the class name.
However, the code posted is correct. You must have compiled the .pyd file using the variable Game, realised your error, then changed it to game and compiled the C++ file to test it without recompiling the .pyd file so the error remained.
I have the following c++ classes (simplified) which I am exposing to Python using SWIG:
struct Component
{
virtual void update();
}
struct DerivedComponent : public Component
{
void update() { cout << "DerivedComponent::update()" << endl; }
void speak() { cout << "DerivedComponent::speak()" << endl; }
}
class Entity
{
public:
Component* component(const std::string& class_name)
{
return m_components[class_name];
}
private:
std::unordered_map<std::string, Component*> m_components;
}
Now, in Python I can successfully call component("DerivedComponent").update() on an Entity instance. However, I cannot call component("DerivedComponent").speak() since the type returned by component("DerivedComponent") is reported as <class 'module.Component'>.
I obviously need to downcast the result of the component() function in order to call methods defined in DerivedComponent. I had hoped that Swig would perform automatic downcasting like I believe that Boost.Python does.
Short of defining a whole bunch of typecasting functions in c++ and exposing them to Python, is there any better solution for downcasting using either Swig or Python? What are my options?
You can do exactly what you want in Python, with a little work. It works as you hope because in Python downcasting is kind of meaningless as the return types of functions (or types in general) aren't strongly typed, so we can modify your Entity::component function to always return the most derived type no matter what it is.
To make that work with your C++/Python binding you need to write an 'out' typemap for Entity::component. I've written an example of how it might work. In this case we have to bodge it slightly because the only way to know what to downcast it to comes from the argument to the function. (If for example your base class had a method that returned this as a string/enum you could simplify this further and not depend on the input arguments).
%module test
%{
#include "test.hh"
%}
%include <std_string.i>
%typemap(out) Component * Entity::component {
const std::string lookup_typename = *arg2 + " *";
swig_type_info * const outtype = SWIG_TypeQuery(lookup_typename.c_str());
$result = SWIG_NewPointerObj(SWIG_as_voidptr($1), outtype, $owner);
}
%include "test.hh"
This uses the SWIG_TypeQuery function to ask the Python runtime to lookup the type based on arg2 (which for your example is the string).
I had to make some changes to your example header (named test.hh in my example) to fix a few issues before I could make this into a fully working demo, it ended up looking like:
#include <iostream>
#include <map>
#include <string>
struct Component
{
virtual void update() = 0;
virtual ~Component() {}
};
struct DerivedComponent : public Component
{
void update() { std::cout << "DerivedComponent::update()" << std::endl; }
void speak() { std::cout << "DerivedComponent::speak()" << std::endl; }
};
class Entity
{
public:
Entity() {
m_components["DerivedComponent"] = new DerivedComponent;
}
Component* component(const std::string& class_name)
{
return m_components[class_name];
}
private:
std::map<std::string, Component*> m_components;
};
I then built it with:
swig -py3 -c++ -python -Wall test.i
g++ -Wall -Wextra test_wrap.cxx -I/usr/include/python3.4/ -lpython3.4m -shared -o _test.so
With this in place I could then run the following Python:
from test import *
e=Entity()
print(e)
c=e.component("DerivedComponent")
print(c)
print(type(c))
c.update()
c.speak()
This works as you'd hope:
<test.Entity; proxy of <Swig Object of type 'Entity *' at 0xb7230458> >
Name is: DerivedComponent *, type is: 0xb77661d8
<test.DerivedComponent; proxy of <Swig Object of type 'DerivedComponent *' at 0xb72575d8> >
<class 'test.DerivedComponent'>
DerivedComponent::update()
DerivedComponent::speak()
I was looking to do something similar and came up with a similar but different solution based on this question.
If you know the possible types ahead of time and don't mind the extra overhead, you can have the 'out' typemap loop through and dynamic_cast to each to automatically return the object with its real type. SWIG already has this implemented for pointers with the %factory feature:
%factory(Component* /* or add method name. this is just the typemap filter */,
DerivedComponent1,
DerivedComponent2);
Looking at factory.swg and boost_shared_ptr.i I got this working for shared_ptr and dynamic_pointer_cast as well:
%define %_shared_factory_dispatch(Type)
if (!dcast) {
SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<Type> dobj
= SWIG_SHARED_PTR_QNAMESPACE::dynamic_pointer_cast<Type>($1);
if (dobj) {
dcast = 1;
SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<Type> *smartresult
= dobj ? new SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<Type>(dobj) : 0;
%set_output(SWIG_NewPointerObj(%as_voidptr(smartresult),
$descriptor(SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<Type> *),
SWIG_POINTER_OWN));
}
}%enddef
%define %shared_factory(BaseType,Types...)
%typemap(out) SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<BaseType> {
int dcast = 0;
%formacro(%_shared_factory_dispatch, Types)
if (!dcast) {
SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<BaseType> *smartresult
= $1 ? new SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<BaseType>($1) : 0;
%set_output(SWIG_NewPointerObj(%as_voidptr(smartresult),
$descriptor(SWIG_SHARED_PTR_QNAMESPACE::shared_ptr<BaseType> *),
SWIG_POINTER_OWN));
}
}%enddef
// Apply dynamic_pointer cast to all returned shared_ptrs of this type
%factory(Component /* must be a type for shared_ptr */,
DerivedComponent1,
DerivedComponent2);
My C++ program:
#include <iostream>
using namespace std;
struct FirstStructure
{
public:
int first_int;
int second_int;
};
struct SecondStructure
{
public:
int third_int;
FirstStructure ft;
};
int test_structure(SecondStructure ss)
{
int sum = ss.ft.first_int + ss.ft.second_int + ss.third_int;
return sum;
}
extern "C"
{
int test(SecondStructure ss)
{
return test_structure(ss);
}
}
And I compile the cpp file use this command "g++ -fPIC -shared -o array.so array.cpp".
Then I call the file array.so use python,my python program as these:
#coding=utf-8
import ctypes
from ctypes import *
class FirstStructure(Structure):
_fields_ = [
("first_int", c_int),
("second_int", c_int)
]
class SecondStructure(Structure):
_fields_ = [
("third_int", c_int),
("ft", FirstStructure)
]
if __name__ == '__main__':
fs = FirstStructure(1, 2)
ss = SecondStructure(3, fs)
print ss.ft.first_int
lib = ctypes.CDLL("./array.so")
print lib.test(ss)
When I run the python program,the console show an error, the error is "segmentation fault".I read the documentation from the url "https://docs.python.org/2/library/ctypes.html",how to fix the bug.
You have to declare a function's argument and return types in python, in order to be able to call it properly.
So, insert the following before calling the test function:
lib.test.argtypes = [SecondStructure]
lib.test.restype = ctypes.c_int
Things should work then, as far as I can see...
As long as the amount of C-to-python interfaces remains "small" (whatever that is), I think ctypes is just fine.
ok,I got it,modified code as these:
#include <iostream>
using namespace std;
extern "C"
{
struct FirstStructure
{
public:
int first_int;
int second_int;
};
struct SecondStructure
{
public:
int third_int;
FirstStructure ft;
};
int test_structure(SecondStructure *ss)
{
int sum = ss->ft.first_int + ss->ft.second_int + ss->third_int;
return sum;
}
int test(SecondStructure *ss)
{
return test_structure(ss);
}
}
and then,I fixed the bug.
Well if you are intending to design communication medium between C++ and python then I would suggest go for combination zmq and google protocol buffers.
where proto buf would serve for serialization/deserialization and zmq for communication medium.
You might want to have a look at Boost.python
https://wiki.python.org/moin/boost.python/SimpleExample
It will allow you to compile python modules from C++ and define how the python is allowed to access the c++ code in an easy to understand fashion
Hello and thanks for your help in advance !
I am writing a python wrapper (SWIG 2.0 + Python 2.7) for a C++ code. The C++ code has typedef which I need to access in python wrapper. Unfortunately, I am getting following error when executing my Python code:
tag = CNInt32(0)
NameError: global name 'CNInt32' is not defined
I looked into SWIG documentation section 5.3.5 which explains size_t as typedef but I could not get that working too.
Following is simpler code to reproduce the error:
C++ header:
#ifndef __EXAMPLE_H__
#define __EXAMPLE_H__
/* File: example.h */
#include <stdio.h>
#if defined(API_EXPORT)
#define APIEXPORT __declspec(dllexport)
#else
#define APIEXPORT __declspec(dllimport)
#endif
typedef int CNInt32;
class APIEXPORT ExampleClass {
public:
ExampleClass();
~ExampleClass();
void printFunction (int value);
void updateInt (CNInt32& var);
};
#endif //__EXAMPLE_H__
C++ Source:
/* File : example.cpp */
#include "example.h"
#include <iostream>
using namespace std;
/* I'm a file containing use of typedef variables */
ExampleClass::ExampleClass() {
}
ExampleClass::~ExampleClass() {
}
void ExampleClass::printFunction (int value) {
cout << "Value = "<< value << endl;
}
void ExampleClass::updateInt(CNInt32& var) {
var = 10;
}
Interface file:
/* File : example.i */
%module example
typedef int CNInt32;
%{
#include "example.h"
%}
%include <windows.i>
%include "example.h"
Python Code:
# file: runme.py
from example import *
# Try to set the values of some typedef variables
exampleObj = ExampleClass()
exampleObj.printFunction (20)
var = CNInt32(5)
exampleObj.updateInt (var)
Thanks again for your help.
Santosh
I got it working. I had to use typemaps in the interface file, see below:
- Thanks a lot to "David Froger" on Swig mailing lists.
- Also, thanks to doctorlove for initial hints.
%include typemaps.i
%apply CNInt32& INOUT { CNInt32& };
And then in python file:
var = 5 # Note: old code problematic line: var = CNInt32(5)
print "Python value = ",var
var = exampleObj.updateInt (var) # Note: 1. updated values returned automatically by wrapper function.
# 2. Multiple pass by reference also work.
# 3. It also works if your c++ function is returning some value.
print "Python Updated value var = ",var
Thanks again !
Santosh