I have a C file dtls_udp_echo.c in which I use SSL functions. I am trying to create a Python wrapper for this file using SWIG. I've done the following steps:
1) Created an interface file udp.i:
%module udp
%{
/* Put header files here or function declarations like below */
#define SWIG_FILE_WITH_INIT
#include "dtls_udp_echo.h"
%}
int THREAD_setup();
int THREAD_cleanup();
int handle_socket_error();
int generate_cookie(SSL *ssl, unsigned char *cookie, unsigned int *cookie_len);
int verify_cookie(SSL *ssl, unsigned char *cookie, unsigned int cookie_len);
int dtls_verify_callback (int ok, X509_STORE_CTX *ctx) ;
void* connection_handle(void *info);
void start_server(int port, char *local_address);
void start_client(char *remote_address, char *local_address, int port, int length, int messagenumber);
2) Run the command swig -python udp.i.
3) Run the command gcc -O2 -fPIC -c dtls_udp_echo.c -I/usr/local/ssl/include -L/usr/local/ssl/lib -lcrypto -lssl. The path to the include and library is correct, I checked it!
4) Run the command gcc -O2 -fPIC -c udp_wrap.c -I/usr/include/python2.5 -I/usr/local/ssl/include -L/usr/local/ssl/lib -lcrypto -lssl.
5) Run the command gcc -shared dtls_udp_echo.o udp_wrap.o -o _udp.so.
It seems to complete OK as no errors are reported. But, when I try to import the module, I get the following traceback:
>>> import udp
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "udp.py", line 28, in <module>
> import _udp ImportError: ./_udp.so: undefined symbol:
> SSL_get_rbio
Can anybody help me to fix this problem?
It can't find the OpenSSL library. Add it to your ld search path; see the ldconfig(8) man page for details.
Related
Is there any place I can find a working example to wrap a C++ function (using VTK) to be used by
Python?
Searched and found this page
mix VTK and SWIG Python
but there is no compiling details, and this simple example on this page
https://public.kitware.com/pipermail/vtkusers/2002-October/013980.html
But for my system Ubuntu 16.04, vtk 8.2.0, Python 3.6.8, I could not make the example work.
I have to modify them to let the compiling done without error.
Details:
#============simple.cpp===================
// simple.c
//based on https://public.kitware.com/pipermail/vtkusers/2002-October/013980.html
#include "Python.h"
#include "vtkPythonUtil.h"
#include "vtkObject.h"
#include "vtkObjectBase.h"
#include <stdio.h>
#include "vtkRenderer.h"
extern "C"
{
static PyObject* simple_vtk_function(PyObject *self, PyObject *args);
}
static PyObject* simple_vtk_function(PyObject *self, PyObject *args)
{
PyObject *vtkpy;
if (!PyArg_ParseTuple(args, "O", &vtkpy))
return NULL;
vtkObjectBase *obj = vtkPythonUtil::GetPointerFromObject(vtkpy,"PyObject");
/* Do what you want to do here. */
//printf("%s\n", obj->GetClassName());
printf("inside module simple_vtk, function simple_vtk_function\n");
Py_RETURN_NONE;
}
static char simple_docs[] =
"simple( ): Any message you want to put here!!\n";
static PyMethodDef module_methods[] =
{
{
"simple_vtk_function", (PyCFunction) simple_vtk_function,
METH_VARARGS, simple_docs
},
{NULL}
};
static struct PyModuleDef simple_vtk =
{
PyModuleDef_HEAD_INIT,
"simple_vtk",/* name of module */
"",/* module documentation, may be NULL */
-1,
module_methods
};
PyMODINIT_FUNC PyInit_simple_vtk(void)
{
return PyModule_Create(&simple_vtk);
}
#============setup.py===================
#!/usr/bin/env python3
import distutils.core as dc
module = dc.Extension('simple_vtk',
sources = ['simple.cpp'],
include_dirs=['/usr/local/include/vtk-8.2'],
libraries_dirs = ['/usr/local/include/vtk-8.2'],
#libraries=['vtkPythonUtil']
)
dc.setup(name = 'simple_vtk',
version = '1.0',
ext_modules = [module])
compile command
CFLAGS="-std=c++11" python setup.py build
and get this printing out
/home/jun/anaconda3/lib/python3.6/distutils/extension.py:131: UserWarning: Unknown Extension options: 'libraries_dirs'
warnings.warn(msg)
running build
running build_ext
building 'simple_vtk' extension
gcc -pthread -B /home/jun/anaconda3/compiler_compat -Wl,--sysroot=/ -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c++11 -fPIC -I/usr/local/include/vtk-8.2 -I/home/jun/anaconda3/include/python3.6m -c simple.cpp -o build/temp.linux-x86_64-3.6/simple.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
simple.cpp: In function ‘PyObject* simple_vtk_function(PyObject*, PyObject*)’:
simple.cpp:22:19: warning: unused variable ‘obj’ [-Wunused-variable]
vtkObjectBase *obj = vtkPythonUtil::GetPointerFromObject(vtkpy,"PyObject");
^
g++ -pthread -shared -B /home/jun/anaconda3/compiler_compat -L/home/jun/anaconda3/lib -Wl,-rpath=/home/jun/anaconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ -std=c++11 build/temp.linux-x86_64-3.6/simple.o -o build/lib.linux-x86_64-3.6/simple_vtk.cpython-36m-x86_64-linux-gnu.so
to test in python
>>> import simple_vtk
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /home/jun/project/play/swig/for_vtk_v2/simple_vtk.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN13vtkPythonUtil20GetPointerFromObjectEP7_objectPKc
The C++ function is so simple, it just to convert a PyObject and convert to vtkObjectBase. I didn't even remove the simple print
printf("%s\n", obj->GetClassName());
But I could never make it work. I think it is a common need for people to wrap a custom C++ vtk function for Python, a little bit surpring there is almost no woring example on internet.
Could you please help me on this? Thanks!
I want to use SWIG to create Python bindings for C library, but I have some troubles with it. I used the following .c and .i files from tutorial.
example.c
/* Compute factorial of n */
int fact(int n) {
if (n <= 1)
return 1;
else
return n*fact(n-1);
}
/* Compute n mod m */
int my_mod(int n, int m) {
return(n % m);
}
double My_variable;
example.i
%module example
%{
extern double My_variable;
extern int fact(int);
extern int my_mod(int n, int m);
%}
extern double My_variable;
extern int fact(int);
extern int my_mod(int n, int m);
Commands that I used:
$ swig -python -py3 example.i
$ gcc -c -fpic example.c example_wrap.c -I/usr/include/python3.6
$ gcc -shared example.o example_wrap.o -o example.so
And when I try to import it in python3 I am getting this
>>> import example
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: dynamic module does not define module export function (PyInit_example)
Any help appreciated
SWIG generates a module with underscore prepended by default, so you'll get a module called _example which needs to be in _example.so.
This is explained in the SWIG documentation 34.2.3 Hand compiling a dynamic module:
While the preferred approach to building an extension module is to use the distutils, some people like to integrate building extensions with a larger build system, and thus may wish to compile their modules without the distutils. To do this, you need to compile your program using commands like this (shown for Linux):
$ swig -python example.i
$ gcc -O2 -fPIC -c example.c
$ gcc -O2 -fPIC -c example_wrap.c -I/usr/local/include/python2.5
$ gcc -shared example.o example_wrap.o -o _example.so
The exact commands for doing this vary from platform to platform. However, SWIG tries to guess the right options when it is installed. Therefore, you may want to start with one of the examples in the SWIG/Examples/python directory. If that doesn't work, you will need to read the man-pages for your compiler and linker to get the right set of options. You might also check the SWIG Wiki for additional information.
When linking the module, the name of the output file has to match the name of the module prefixed by an underscore. If the name of your module is example, then the name of the corresponding object file should be _example.so or _examplemodule.so. The name of the module is specified using the %module directive or the -module command line option.
Compatibility Note: In SWIG-1.3.13 and earlier releases, module names did not include the leading underscore. This is because modules were normally created as C-only extensions without the extra Python support file (instead, creating Python code was supported as an optional feature). This has been changed in SWIG-1.3.14 and is consistent with other Python extension modules. For example, the socket module actually consists of two files; socket.py and _socket.so. Many other built-in Python modules follow a similar convention.
I have a task in which I have to write the Python bindings for an existing C++ library. Since SWIG have support for not only Python but also for languages like Java and Perl, I am using SWIG. I am relatively new in SWIG so I have a doubt. I want my python library to be supportable under Python 2.7 and Python 3.x. But I don't have any idea regarding how to do this. So if anyone can suggest me. Any help would be appreciated. You can ask me for more details if you want.
What I have tried so far.
This is my code file.
/* example.c file */
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
int my_mod(int n, int m) {
return n % m;
}
int sieve(int number) {
int* arr = malloc(sizeof (int) * (number + 10));
int* prime = malloc(sizeof (int) * (number + 10));
/* printf("Size of arr: %lu", sizeof(arr));
printf("Size of int: %lu", sizeof(int)); */
memset(arr, 0, sizeof(int) * (number + 10));
int counter = 0;
prime[counter++] = 2;
arr[0] = arr[1] = 0;
for (int i = 3; i * i <= number; i += 2) {
if (!arr[i]) {
for (int j = i + i; j < number; j += i) {
arr[j] = 1;
}
}
}
for (int i = 3; i < number; i += 2)
if (!arr[i])
prime[counter++] = i;
// free(arr);
// free(prime);
return counter;
}
And my interface file is
/* example.i */
%module example
%{
#include "stdio.h"
#include "stdlib.h"
#include "string.h"
extern int my_mod(int n, int m);
extern int sieve(int number);
%}
extern int my_mod(int n, int m);
extern int sieve(int number);
My steps for compilation
swig -python example.i
gcc -fpic -c example.c example_wrap.c $(pkg-config --cflags --libs python3)
gcc -shared example.o example_wrap.o -o _example.so
In above compilation format module is working fine with python3 but failing in python2 with error log as
ImportError: ./_example.so: undefined symbol: PyUnicode_FromFormat
And if I use below compilation commands
swig -python example.i
gcc -fpic -c example.c example_wrap.c $(pkg-config --cflags --libs python2)
gcc -shared example.o example_wrap.o -o _example.so
Than module is working with python2 but when I try to import in python3 than error message is
ImportError: /home/deepanshu/env/swig/env/src/deep/_example.so: undefined symbol: PyInstance_Type
I am sure than the error is because $(pkg-config --cflags --libs pythonX) I am specifying version at place of X but how can I make sure my module works with both the Python versions?
I tried -py3 flag of swig, but I wasn't able to make module work for both of the Python version with above specified flag.
C extension don't work the same way in python3 vs python2. This link can help you identify what you need to do on the C level to have the library work for both python 2 and 3 (not using swig). http://python3porting.com/cextensions.html
Using Swig, what I would do is make use of the pip wheels naming convention :
https://packaging.python.org/tutorials/distributing-packages/#packaging-your-project
Doing so, I would package the library for python3 and make the package follow the wheel naming standard ("https://www.python.org/dev/peps/pep-0425/#id1" "py3-none-any") and then make the library for Python2 and package it using the same strategy("py2-none-any"). then you can upload it to Pypi and pip would know that it need to use the py2 for python2 and py3 for python3.
I found this very helpful question and answer:
Return vector<pair<int,int>> & from c++ method to python list of tuples using swig typemap
However I have some problems iterating the return vector if it's not a reference, here the example:
myclass.h:
#include <vector>
#include <utility>
using std::vector;
using std::pair;
class MyClass {
private:
vector<pair<int,int> > _myvector;
public:
MyClass( );
const vector<pair<int,int> > & GetMyVector() const;
};
myclass.cpp:
#include "myclass.h"
MyClass::MyClass(): _myvector()
{_myvector.push_back(std::make_pair(1,2));_myvector.push_back(std::make_pair(1,2));};
const vector<pair<int,int>> & MyClass::GetMyVector() const {
return _myvector;
};
myclass.i:
%module x
%include <std_pair.i>
%include <std_vector.i>
%include <std_string.i>
%template() std::pair<int,int>;
%template(PairVector) std::vector<std::pair<int,int> >;
%{
#include "myclass.h"
%}
%include "myclass.h"
compiled with:
g++ -std=c++11 -c -fPIC myclass.cpp
swig -c++ -v -python myclass.i
g++ -std=c++11 -fPIC -c myclass.cpp myclass_wrap.cxx -I/usr/include/python2.7
g++ myclass.o myclass_wrap.o -shared -fPIC -o _x.so
but when I run something like this in python:
import x
b=x.MyClass()
print(b.GetMyVector())
for a,b in b.GetMyVector():
print(a,b)
then I get:
<Swig Object of type 'vector< std::pair< int,int >,std::allocator< std::pair< int,int > > > *' at 0x7ff06804b1b0>
Traceback (most recent call last):
File "Test.py", line 6, in <module>
for a,b in b.GetMyVector():
TypeError: 'SwigPyObject' object is not iterable
How can I iterate the returned vector properly in python? And why is a pointer to vector returned? Do I have to change something in the swig file?
In case it is relevant: (on Ubuntu)
SWIG Version 2.0.11
g++ (Ubuntu 4.9.4-2ubuntu1~14.04.1) 4.9.4
Python 2.7.6
SWIG doesn't understand the using directive correctly.
Same as in this question and answer:
SWIG argument error when using "using std::vector" in python
As to why a pointer is returned, well, if SWIG cannot convert a returned object into a python object, then it wraps a pointer to the object.
I'm trying to create a *.so file for further use in Python using SWIG, but something isn't working.
I have two files:
DataGatherer.h
#include <iostream>
#include <stdlib.h>
#include <time.h>
#include "gnublin.h"
#include <pthread.h>
class dataGatherer
{
private:
int threshold;
int timeThreshold;
int data[4096];
bool running;
gnublin_spi* spiDevice;
pthread_t spiThread;
void *params;
public:
dataGatherer(void);
dataGatherer(int, int);
void initData();
int getThreshold(void);
int* getData(void);
int getPeak(void);
void initSPI(void);
void gatherData();
void * run(void * arg);
void stop(void);
// for use of thread we have to implement some methods from C
static void * start_static(void * params)
{
dataGatherer * thread_this = static_cast<dataGatherer*>(params);
return thread_this->run(thread_this->params);
}
void start(void * params)
{
this->params = params;
pthread_create(&spiThread, 0, &dataGatherer::start_static, this);
}
};
and spiController.h
#include "dataGatherer.h"
class spiController
{
private:
bool runGather;
dataGatherer* gatherer;
int data[4096];
public:
spiController(void);
spiController(int, int);
void initData();
bool checkStop();
void stop();
void start();
};
My spiController.i interface file looks like this:
/* spiController.i */
%module spiController
%{
#include "dataGatherer.h"
#include "spiController.h"
#include "gnublin.h"
#include <stdlib.h>
#include <time.h>
#include <pthread.h>
extern void initData();
extern bool checkStop();
extern void stop();
extern void start();
%}
extern void initData();
extern bool checkStop();
extern void stop();
extern void start();
At the end I try to create the *.so file using commands in the terminal like in the example on the SWIG page with:
swig -python -c++ spiController.i
c++ -c spiController_wrap.c -I/usr/include/python2.7
c++ -shared spiController_wrap.o -o _spiController.so
*.cxx, *.o and *.so file are created with no error, but when I import the spiController into the python code I get:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "spiController.py", line 26, in <module>
_spiController = swig_import_helper()
File "spiController.py", line 22, in swig_import_helper
_mod = imp.load_module('_spiController', fp, pathname, description)
ImportError: ./_spiController.so: undefined symbol: _Z9checkStopv
It's my first try using SWIG and I'm already stuck at this point. How can I resolve this?
I just got the same error and finally figured out why. As above people said, when it says unfound symbol like yours and gives the undefined function name '_Z9checkStopv', always check for the implementation of this function in .cpp file as well as any function declaration of the same name!!
For my case, my cpp does define my 'unfound symbol' constructor function, but in my .h file, i have an overloaded operator= (for the constructor) which is undefined in .cpp file. So swig wraps both default constructor(implemented in .cpp) and operator= (not implemented). Therefore when import, this unimplemented operator= produces the error. Hope this helps!
You must link the library that defines your C++ functions that you have declared like checkStop etc. You would add -L<path to your C++ DLL> -l<name of your C++ DLL> on 3rd line of your example's compile steps.
Like this:
c++ -L<path to DLL> -l<name of your dll> -shared spiController_wrap.o -o _spiController.so
Just as Adam's comment and my experience, you should first compile your XXX.cpp file into XXX.o, the whole command lines maybe like following:
swig -python -c++ XXX.i
g++ -c -fpic XXX.cpp* (this command will generate XXX.o file)
g++ -c -fpic XXX_wrap.cxx -I/usr/include/python2.7* (this command will generate XXX_wrap.o file)
g++ -shared XXX.o XXX_wrap.o -o XXX.so
Although there may be many causes for this problem, I got the exact same error when I compiled the shared library with the python v3.5 headers, e.g.
swig -python example.i
gcc -fPIC -c example.c example_wrap.c -I/usr/include/python3.5 # <-- ISSUE HERE
gcc -shared example.o example_wrap.o -o _example.so
But then later tried to use the example library using python test.py, which ran python v2.7 on system (so it was a python version mismatch issue).
In my case I was also getting that error, and spent some time trying out stuff without success. My problem was that although my source file was plain C I had it named with the .cpp extension, assuming it wouldn't matter. Changing the extension to .c solved automatically the issue.
Another way of solving it was to add the line #include "example.cpp" to the header section of SWIG's .i file.
So, summarizing:
example.c
int fact(int n) {
if (n <= 1) return 1;
else return n*fact(n-1);
}
example.i
%module example
%{
extern int fact(int n);
%}
extern int fact(int n);
Then the following worked for me (in Ubuntu 17.10):
swig -c++ -python example.i
gcc -fPIC -c example.c example_wrap.c -I /usr/include/python2.7
gcc -shared example.o example_wrap.o -o _example.so
python -c "import example; print example.fact(5)"
Hope this helps someone!
Cheers
I know the solution
At the end of make up the share,you should usr g++ -shared -fpic *.o *.o -o _***.so