I'm trying to make a basic csv parser in c++ for a particular csv schema, and I'm trying to wrap the function for Python, but I keep getting a "StdVectorTraits not found" warning after wrapper generation. The wrapper is still able to be compiled using g++, but when I try to import the underlying shared object using the object.py script, I get "ImportError: undefined symbol: _Z8myVectorRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE"
This is the swig interface file:
%module parser;
%include "/usr/share/swig4.0/std/std_vector.i";
%include "/usr/share/swig4.0/std/std_iostream.i";
%include "/usr/share/swig4.0/std/std_sstream.i";
%include "/usr/share/swig4.0/std/std_string.i";
%include "/usr/share/swig4.0/std/std_basic_string.i";
%{
#include <vector>
#include <fstream>
#include <sstream>
#include <iostream>
std::vector<std::vector<double>> myVector(std::string&);
%}
%template(doubleVector) std::vector<double>;
%template(doubleVecVector) std::vector<std::vector<double>>;
std::vector<std::vector<double>> myVector(std::string& path)
{
std::ifstream file;
std::string read;
std::vector<std::vector<double>> data;
file.open(path);
for (int i = 0; i < 21; i++)
{
std::getline(file, read);
}
for (int i = 0; i < 32; i++)
{
std::vector<double> line;
std::getline(file, read);
std::stringstream ss(read);
for (int j = 0; j < 48; j++)
{
std::getline(ss, read, ',');
line.push_back(std::stof(read));
}
data.push_back(line);
}
file.close();
return data;
}
Error:
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
/home/../test.ipynb Cell 1 in <cell line: 1>()
----> 1 import parser
File ~/../parser.py:15, in <module>
13 from . import _parser
14 else:
---> 15 import _parser
17 try:
18 import builtins as __builtin__
ImportError: /home/../_parser.so: undefined symbol: _Z8myVectorRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
The function definition should be between %{ and %}. Everything between %{/%} is included directly in the generated wrapper. The function prototype should be at the end of the file after the %template declarations to direct SWIG to generate a wrapper for that function.
Since the function body is in the wrong place it isn't defined hence the undefined symbol error.
Stripped down example:
test.i
%module test
%{
#include <vector>
std::vector<std::vector<double>> myVector()
{
return {{1.0,1.5},{2.0,2.5}};
}
%}
%include <std_vector.i>
%template(doubleVector) std::vector<double>;
%template(doubleVecVector) std::vector<std::vector<double>>;
std::vector<std::vector<double>> myVector();
Demo:
>>> import test
>>> test.myVector()
((1.0, 1.5), (2.0, 2.5))
Related
I have been trying to get this to work for a while now. I am trying to wrap a LOT of c++ classes in swig, but I can't even get the first one to work. The error is at the bottom. Here is my interface file, setup.py, and class file.
Interface
//This file is automatically generated from "build_swig_files.py
//Makes changes to build_swig_files.py to edit jcm.i
%module jcm
%{
#include "jtag/GenericJTAGDevice.h"
typedef unsigned int u32;
%}
class GenericJTAGDevice {
public:
virtual ~GenericJTAGDevice();
GenericJTAGDevice(int irLength, int idCode);
unsigned int getIrLength();
unsigned int getIdCode();
private:
unsigned int idCode;
unsigned int irLength;
};
typedef unsigned int u32;
%include <std_string.i>
using std::string;
%include "cpointer.i"
%pointer_functions(u32, u32p);
%include "carrays.i"
%array_class(u32, u32a);
%include "std_vector.i"
namespace std {
%template(IntVector) vector<int>;
}
setup.py
from distutils.core import setup, Extension
jcm_sources = [
"jcm.i",
"/root/git/jcm/jcm_source/base/src/jtag/GenericJTAGDevice.cpp"
]
jcm_module = Extension('_jcm',
sources=jcm_sources,
swig_opts=[ '-I/root/git/jcm/jcm_source/base/include',
'-I/root/git/jcm/jcm_source/base/include/jtag',
'-I/root/git/jcm/jcm_source/base/include/util',
'-I/root/git/jcm/jcm_source/base/include/xilinx',
'-c++'],
include_dirs=[ '/root/git/jcm/jcm_source/base/include',
'/root/git/jcm/jcm_source/base/include/jtag',
'/root/git/jcm/jcm_source/base/include/util',
'/root/git/jcm/jcm_source/base/include/xilinx'],
libraries=['supc++'])
setup (name = 'jcm', version = '0.3', author = 'BYUCCL', ext_modules = [jcm_module], py_modules = ["jcm"])
Class Header
#ifndef GENERIC_JTAG_DEVICE_H
#define GENERIC_JTAG_DEVICE_H
#include <string>
#include <vector>
//#include "JTAGDevice.h"
using namespace std;
/**
* #brief Basic implementation of a JTAGDevice
*
* \class GenericJTAGDevice
*
**/
class GenericJTAGDevice {
public:
virtual ~GenericJTAGDevice();
GenericJTAGDevice(int irLength, int idCode);
unsigned int getIrLength();
unsigned int getIdCode();
private:
unsigned int idCode;
unsigned int irLength;
};
#endif // GENERIC_JTAG_DEVICE_H
Here is the error:
>>> import sys
>>> sys.path.insert(0, '/root/git/JCM/jcm_source/python/swig')
>>> import jcm
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/root/git/jcm/jcm_source/python/swig/jcm.py", line 64, in <module>
class GenericJTAGDevice(object):
File "/root/git/jcm/jcm_source/python/swig/jcm.py", line 67, in GenericJTAGDevice
__swig_destroy__ = _jcm.delete_GenericJTAGDevice
AttributeError: module '_jcm' has no attribute 'delete_GenericJTAGDevice'
I have tried a couple variations in the interface file, such as not having the whole class definition and just doing %include GenericJTAGDevice.h. I have a feeling it has to do with the virtual destructor, but I don't know how to fix that because I need the destructor.
Edit: I tried with another class and it did the same thing. So perhaps I am understanding the interface file wrong.
Edit: All I am running is python3 setup.py build
So I saw the answer in the link below before but didn't understand what it was saying. Basically, my process to build swig didn't include making a new _jcm.so. So pretty much the first time I ran it was it, and after that all the changes I made to the .i or the code or setup.py didn't mean anything because the _jcm.so wasn't being rewritten. In my case, I run a "make clean" from my make file and it deletes the _jcm.so. After that I build the _jcm.so again, and then run setup.py.
Simple, but hard to find.
http://swig.10945.n7.nabble.com/Req-module-object-has-no-attribute-delete-TSP-CA-td2271.html
I'm trying to wrap an unordered map in a python dictionary using swig:
// UsingAnUnorderedMap.h
#ifndef LEARNSWIG_USINGUNORDEREDMAP_H
#define LEARNSWIG_USINGUNORDEREDMAP_H
#include <unordered_map>
#include <string>
std::unordered_map<std::string, int> makeStringToIntMap(){
return std::unordered_map<std::string, int>{
{"first", 4},
{"second", 5},
};
}
#endif //LEARNSWIG_USINGUNORDEREDMAP_H
//UsingAnUnorderedMap.i
%module UsingUnorderedMap
%{
#include "UsingUnorderedMap.h"
#include <iostream>
#include <unordered_map>
%}
%include "std_string.i"
%include "std_pair.i"
%include "std_unordered_map.i"
%template(StringToIntMap) std::unordered_map<std::string,int>;
%include "UsingUnorderedMap.h"
%typemap(out) StringToIntMap {
PyDictObject* dict = PyDict_New($input);
for (auto &item: StringToIntMap){
PyDict_SetItem(dict, PyUnicode_FromString(item.first), item.second);
}
$result = dict;
}
# testUsingAnUnorderedMap.py
import sys
sys.path += [
r"D:\LearnSwig\install-msvc\UsingUnorderedMap"
]
import UsingUnorderedMap
print(type(UsingUnorderedMap.makeStringToIntMap()))
This produces
<class 'UsingUnorderedMap.StringToIntMap'>
i.e. it just ignores the typemap. Technically the StringToIntMap behaves pretty much the same as a Python dict - at far as I can tell, but I think there's confort for Python users in Python dictionaries and so it would be better if this were a straight up dictionary. Does anybody have any pointers on how to achieve this?
For convenience, you can build this code using the following CMake code. Note that I build this using the command -DSWIG_EXECUTABLE=/path/to/swig.exe.
# CMakeLists.txt
set(Python_ROOT_DIR "C:/Miniconda3")
find_package(Python COMPONENTS Interpreter Development NumPy)
message("Python_EXECUTABLE ${Python_EXECUTABLE}")
find_package(SWIG 4.0.0 REQUIRED
COMPONENTS python
)
include(UseSWIG)
set_property(SOURCE UsingUnorderedMap.i PROPERTY CPLUSPLUS ON)
swig_add_library(UsingUnorderedMap
LANGUAGE python
TYPE MODULE
SOURCES UsingUnorderedMap.i UsingUnorderedMap)
set_target_properties(UsingUnorderedMap
PROPERTIES LANGUAGE CXX)
target_include_directories(UsingUnorderedMap PUBLIC
${Python_INCLUDE_DIRS}
${CMAKE_CURRENT_SOURCE_DIR}
)
target_link_libraries(UsingUnorderedMap PUBLIC ${Python_LIBRARIES})
install(TARGETS UsingUnorderedMap DESTINATION UsingUnorderedMap)
install(FILES
${CMAKE_CURRENT_BINARY_DIR}/UsingUnorderedMap.py
${CMAKE_CURRENT_SOURCE_DIR}/testUsingUnorderedMap.py
DESTINATION UsingUnorderedMap)
Here goes a small example showing conversion of std::unordered_map to a python dictionary
%module dictmap
%{
#include "test.h"
%}
%typemap(out) std::unordered_map<std::string, std::string> (PyObject* obj) %{
obj = PyDict_New();
for (const auto& n : $1) {
PyObject* strA = PyUnicode_FromString(n.first.c_str());
PyObject* strB = PyUnicode_FromString(n.second.c_str());
PyDict_SetItem(obj, strA, strB);
Py_XDECREF(strA);
Py_XDECREF(strB);
}
$result = SWIG_Python_AppendOutput($result, obj);
%}
%include "test.h"
Small inline function
#pragma once
#include <string>
#include <unordered_map>
std::unordered_map<std::string, std::string> makeStringMap() {
return std::unordered_map<std::string, std::string> {
{"first", "hello"},
{"second", "world"},
};
}
I hope that you can see what you have made wrong. You are returning an empty dictionary created using an empty $input. I am not sure of whether you need to defined a PyObject argument, but I do this always in case it is needed together with another typemap.
I am trying to use swig to call c++ functions from python. Its actually working (I can import the module in python), but there are some warnings that i want to solve before i get in trouble later.
I tried to just include the code, that produces the warning.
I have this Classes in my MDF4.h Header File:
// MDF4.h
.
.
.
typedef struct
{
// Block Header
enum { RID=M4ID_FH };
// enumeration of links
enum
{
fh_fh_next, // Link to next FHBLOCK (can be NIL if list finished)
fh_md_comment,
LinkMax // # of known links
};
// Data members
M_DATE fh_time;
M_UINT8 fh_reserved[3]; // Reserved
} m4FHRecord;
.
.
template<class R,class T=BYTE,int ID=R::RID> class m4BlockImpl : public m4Block,public R
{
public:
m4BlockImpl(size_t nVar=0) : m4Block(R::LinkMax),m_var(nVar)
{
m_Hdr.hdr_hdr=M4ID_ID;
m_Hdr.hdr_id=ID;
R *pThis=static_cast<R *>(this);
memset(pThis,0,sizeof(R));
}
BOOL setCommentBlk(m4Block &TXorMD,int linkNo)
{
// cannot call twice
if (!hasLink(linkNo))
{
ATLASSERT(TXorMD.hdrID()==M4ID_TX || TXorMD.hdrID()==M4ID_MD);
ATLASSERT(m_File);
M_LINK mdAt=TXorMD.Create(m_File,3);
if (mdAt)
{
setLink(linkNo,mdAt);
return TRUE;
}
}
return FALSE;
}
};
.
.
class M4FHBlock : public m4BlockImpl<m4FHRecord>
{
public:
M4FHBlock(MDF4File *File); // ctor: create and insrt current time
M4FHBlock(); // used for reading
BOOL setComment(m4Block &md); // CANNOT be a TX Block!
DECLARE_DUMP
};
And this is my MDF4.i interface file:
%module MDF4
%{
#include "mdf4_lib_v2_019\\stdafx.h"
#include "mdf4_lib_v2_019\\utf8.h"
#include "mdf4_lib_v2_019\\Resource.h"
#include "mdf4_lib_v2_019\\mdfConfig.h"
#include "mdf4_lib_v2_019\\Ptrlist.h"
#include "mdf4_lib_v2_019\\dynArray.h"
#include "mdf4_lib_v2_019\\miniz.c"
#include "mdf4_lib_v2_019\\md5.h"
#include "mdf4_lib_v2_019\\m4Dump.h"
#include "mdf4_lib_v2_019\\mdFile.h"
#include "mdf4_lib_v2_019\\mdfTypes.h"
#include "mdf4_lib_v2_019\\mdf4.h"
%}
%feature("autodoc", "1");
%feature("flatnested");
%rename(__incr__) Indent::operator++;
%rename(__incr__) ptrlist::iterator::operator++;
%rename(__eq__) ptrlist::iterator::operator=;
%rename(__decr__) ptrlist::iterator::operator--;
%rename(__eq__) ptrlist::iterator::operator=;
%rename(__invert__) ptrlist::iterator::operator!;
%rename(__incr__) DbtObjPtrList::iterator::operator++;
%rename(__eq__) DbtObjPtrList::iterator::operator=;
%rename(__decr__) DbtObjPtrList::iterator::operator--;
%rename(__eq__) DbtObjPtrList::iterator::operator=;
%rename(__invert__) DbtObjPtrList::iterator::operator!;
%include <wchar.i>
%include <cwstring.i>
%include <std_vector.i>
%include <std_map.i>
%include "mdf4_lib_v2_019\\stdafx.h"
%include "mdf4_lib_v2_019\\utf8.h"
%include "mdf4_lib_v2_019\\Resource.h"
%include "mdf4_lib_v2_019\\mdfConfig.h"
%include "mdf4_lib_v2_019\\Ptrlist.h"
%include "mdf4_lib_v2_019\\dynArray.h"
%include "mdf4_lib_v2_019\\miniz.c"
%include "mdf4_lib_v2_019\\md5.h"
%include "mdf4_lib_v2_019\\m4Dump.h"
%include "mdf4_lib_v2_019\\mdFile.h"
%include "mdf4_lib_v2_019\\mdfTypes.h"
%include "mdf4_lib_v2_019\\mdf4.h"
But i get always this warning after calling swig -c++ -python MDF4.i:
mdf4_lib_v2_019\mdf4.h(1226) : Warning 401: Nothing known about base class 'm4BlockImpl< m4FHRecord >'. Ignored.
mdf4_lib_v2_019\mdf4.h(1226) : Warning 401: Maybe you forgot to instantiate 'm4BlockImpl< m4FHRecord >' using %template.
---- EDIT -----
I created a simple example with the same warning output:
test.h:
typedef struct
{
enum { RID=1};
}myStruct;
template<class A,class B=BYTE,int ID=A::RID> class Foo : public A
{
int a;
public:
Foo()
{
int a = 1;
}
};
class Bar : public Foo<myStruct>
{
public:
Bar();
};
test.i:
%module test
%{
#include "test.h"
%}
%include "test.h"
I try to use a c++-class for socket communication in Python. Therefore, I created a class that uses nngpp. When importing the swigged file to python, I get the ImportError: undefined symbol: nng_msg_insert. The definition of the class is:
/* commclass.h */
#include <nngpp/nngpp.h>
#include <nngpp/protocol/req0.h>
#include <nngpp/protocol/rep0.h>
#include <nngpp/msg_body.h>
#include <nngpp/msg_header.h>
#include <nngpp/msg.h>
#include <nngpp/socket.h>
#include <nngpp/view.h>
#include <string>
#include <nlohmann/json.hpp>
//#include <thread>
#include <iostream>
#include <cstdio>
#include "/usr/local/include/nng/nng.h"
//#include <memory>
#include <chrono>
using json = nlohmann::json;
class CommIF
{
private:
nng::socket socket;
nng::msg message;
int msg_size;
public:
CommIF(const std::string option, std::string ipToListen, std::string ipToDial)
{
message = nng::make_msg(0);
if (option.compare("rep") == 0)
{
socket = std::move(nng::rep::v0::open());
}
else if (option.compare("req") == 0)
{
socket = std::move(nng::req::v0::open());
}
else
{
printf("EXCEPTION");
}
socket.listen(ipToListen.c_str());
bool connected = false;
while (connected == false)
{
try
{
socket.dial(ipToDial.c_str());
connected = true;
std::cout << "successfully connected\n";
}
catch (const nng::exception &e)
{
std::cerr << e.what() << "; retry in 1 s" << '\n';
//std::this_thread::sleep_for(std::chrono::seconds(1));
}
}
msg_size = 0;
}
};
The interface file for swig is:
/* commclass.i */
%module commclass
%{
#include "src/commclass.h"
%}
%include "src/commclass.h"
I then start with the command python3 build_commclass.py build_ext --inplace the build process. The file build_commclass.py is the following
from distutils.core import setup, Extension
import os
name = "commclass"
version = "0.0.1"
os.environ["CC"] = "g++"
setup(name = name, version = version, ext_modules = [Extension(
name = '_commclass',
sources = ["commclass.i"],#"src/commclass.h"],
include_dirs = ['src'],#'/home/user1/Documents/extLibs','/usr/local/include'],
swig_opts = ["-c++", "-modern"]
)])
When I now import the the class to python, I get the error mentioned above. I searched a lot on google and stackoverflow and I am quite sure this is a linker issue. I also tried a lot of different things with the compiler and linker options of distutils.core, but I didn't find a solution.
Edit 1:
I now changed the interface file as following
/* commclass.i */
/* module*/
%module commclass
%{
#include "/usr/local/include/nng/nng.h"
#include "src/nngpp/nngpp.h"
#include "src/nngpp/protocol/req0.h"
#include "src/nngpp/protocol/rep0.h"
#include "src/nngpp/socket.h"
#include "src/nngpp/msg.h"
#include "src/nngpp/aio.h"
#include "src/nngpp/aio_view.h"
#include "src/nngpp/msg_body.h"
#include "src/nngpp/msg_header.h"
#include "src/commclass.h"
%}
%include "/usr/local/include/nng/nng.h"
%include "src/commclass.h"
I now get the same error when importing in python, but the undefined symbol changed. It is now nng_aio_set_iov. Both nng_msg_insert and nng_aio_set_iov are defined in the file nng.h which I have now included. I am confused now.
SWIG only generates interfaces for definitions that are directly specified by %include by default. nng_msg_insert is not defined in src/commclass.h. You need additional %include statements to bring in the definition.
Note you can change the default to recurse into all #include statements with the -includeall SWIG flag, but you generally don't want the entire <iostream>, <cstdio>, <string>, etc. interfaces to be wrapped and it likely won't work without a lot of additional effort.
Hello and thanks for your help in advance !
I am writing a python wrapper (SWIG 2.0 + Python 2.7) for a C++ code. The C++ code has typedef which I need to access in python wrapper. Unfortunately, I am getting following error when executing my Python code:
tag = CNInt32(0)
NameError: global name 'CNInt32' is not defined
I looked into SWIG documentation section 5.3.5 which explains size_t as typedef but I could not get that working too.
Following is simpler code to reproduce the error:
C++ header:
#ifndef __EXAMPLE_H__
#define __EXAMPLE_H__
/* File: example.h */
#include <stdio.h>
#if defined(API_EXPORT)
#define APIEXPORT __declspec(dllexport)
#else
#define APIEXPORT __declspec(dllimport)
#endif
typedef int CNInt32;
class APIEXPORT ExampleClass {
public:
ExampleClass();
~ExampleClass();
void printFunction (int value);
void updateInt (CNInt32& var);
};
#endif //__EXAMPLE_H__
C++ Source:
/* File : example.cpp */
#include "example.h"
#include <iostream>
using namespace std;
/* I'm a file containing use of typedef variables */
ExampleClass::ExampleClass() {
}
ExampleClass::~ExampleClass() {
}
void ExampleClass::printFunction (int value) {
cout << "Value = "<< value << endl;
}
void ExampleClass::updateInt(CNInt32& var) {
var = 10;
}
Interface file:
/* File : example.i */
%module example
typedef int CNInt32;
%{
#include "example.h"
%}
%include <windows.i>
%include "example.h"
Python Code:
# file: runme.py
from example import *
# Try to set the values of some typedef variables
exampleObj = ExampleClass()
exampleObj.printFunction (20)
var = CNInt32(5)
exampleObj.updateInt (var)
Thanks again for your help.
Santosh
I got it working. I had to use typemaps in the interface file, see below:
- Thanks a lot to "David Froger" on Swig mailing lists.
- Also, thanks to doctorlove for initial hints.
%include typemaps.i
%apply CNInt32& INOUT { CNInt32& };
And then in python file:
var = 5 # Note: old code problematic line: var = CNInt32(5)
print "Python value = ",var
var = exampleObj.updateInt (var) # Note: 1. updated values returned automatically by wrapper function.
# 2. Multiple pass by reference also work.
# 3. It also works if your c++ function is returning some value.
print "Python Updated value var = ",var
Thanks again !
Santosh