We Keep Coding

What is the correct way to import a python module wrapped with swig in c++

Creating a python module with swig is the easy part.
But what if this module has to interact with my C++ application that imports it from its embedded python instance?
For example #Flexo has made a pretty good example on how to expose the python module to the application here: How can I implement a C++ class in Python, to be called by C++?
All I want is that the application and the module are able to interact with each other like sharing variables and calling callbacks and so on so I found this answer a bit too complex for the task.
I couldn't really find much else on how to achieve this and the swig documentation doesn't explain this either ( I guess it has more to do with python and not with SWIG in particular that's why it is not mentioned in the SWIG documentation ).
I then found out that I can also just include the *_wrap.cxx file created by SWIG in my C++ application and achieve the same results (see code below)
I just started with SWIG and Python thats why my question now is, is there a better way than mine or an official way to achieve the same result? Have I overlooked something?
Can I import my swig module before initializing the python instance without having to include the Swigtest_wrap.cxx file?
That's how I made it:
1. Files
following three files contain everything needed for my python module. Containing only a class which I later want to use a as base class for a new python class.
Swigtest.h - header for python module
#ifndef SWIGTEST_H
#define SWIGTEST_H
class Callback
{
public:
Callback(){}
virtual ~Callback() {}
// we want to override this function in python
virtual void Exec() {}
static void callFunc();
static void setCallback(Callback* callback);
private:
static Callback* m_callback;
};
#endif // SWIGTEST_H
Swigtest.cpp - for python module
Callback* Callback::m_callback = nullptr;
void Callback::callFunc()
{
if(m_callback != nullptr)
{
m_callback->Exec();
return;
}
std::cout << "callback not set" << std::endl;
}
void Callback::setCallback(Callback* callback)
{
m_callback = callback;
}
Swigtest.i - interface file for SWIG
the only thing to note here is the activation of the "director" feature
%module(directors="1") mymodule
// We need to include Swigtest.h in the SWIG generated C++ file
%{
#include <iostream>
#include "Swigtest.h"
%}
// Enable cross-language polymorphism in the SWIG wrapper.
%feature("director") Callback;
// Tell swig to wrap everything in Swigtest.h
%include "Swigtest.h"
Switest.py
this python script creates a new Class derived from our Callback class and sets the callback to
this class.
import mymodule
# lets create a new class derived from callback
class MyPyCallbackFromC(mymodule.Callback):
def Exec(self):
print("this class was created in python - It worked!")
callback = mymodule.Callback()
mycallback = MyPyCallbackFromC()
# set callback to this new class
callback.setCallback(mycallback)
# now lets call it from our c++ application
main.cpp
#include <Python.h>
#include <iostream>
#include "Swigtest.h"
// we include this file to be able to append the python
// module table with our own swig wrapped module
#include "Swigtest_wrap.cxx"
int main()
{
// *_wrap.cxx has to be included for PyInit__mymodule;
// must be added before the Python instance is initialized!
PyImport_AppendInittab("_mymodule", PyInit__mymodule);
Py_Initialize();
// create Callback class
Callback* callback = new Callback();
// call Exec() function of linked class
// should return error because no class is set as Callback yet
callback->callFunc();
// execute our script which sets a new class as our callback
FILE* fp;
const char* filename;
filename = "Swigtest.py";
fp = _Py_fopen(filename, "r");
PyRun_SimpleFile(fp, filename);
// if we now call the callback from our application
// the Exec function that was defined in our python script should be executed
callback->callFunc();
delete callback;
Py_Finalize();
}
2. Building
Building the module
#swig
swig -c++ -python Swigtest.i
#compile
g++ -fpic -c Swigtest.cpp Swigtest_wrap.cxx -I/pathTo/python
#build
g++ -Wall -Wextra -shared Swigtest.o Swigtest_wrap.o -o _mymodule.so
Building the application
#compile
g++ -fpic -c Swigtest.cpp
g++ -fpic -c main.cpp -I/pathTo/python
#build
g++ main.o Swigtest.o -o libmyapp.so -I/pathTo/python -lpython3
3. Execution:
starting the application from the terminal
$ ./libmyapp.so
output
callback not set
this class was created in python - It worked!
That's it.

Wrapper DLL for Python: "fatal error LNK1127: library is corrupt"

Brief description
I have a DLL programmed in ADA with GNAT. I want to compile with MSVC another DLL in C as a wrapper to the ADA_DLL in order to use it with Python.
I have compiled the ada_DLL, then I have generated the .lib file according to gnat documentation about MSVC. And finally I tried to compile the C_DLL with Visual-Studio, getting the error:
libmath.lib : fatal error LNK1127: library is corrupt
Update: In the case of compiling with gcc as suggested by #Brian, I get the following output:
>"C:\GNAT\2015\bin\gcc.exe" -c -IC:\Python27\include -o libmath_c.o libmath_c.c
>"C:\GNAT\2015\bin\gcc.exe" -shared -LC:\Python27\libs -L./ -l libmath -o DIVISION_CPP.pyd libmath_c.o -lpython27
.//libmath.lib: error adding symbols: Malformed archive
collect2.exe: error: ld returned 1 exit status
Things I tried & more data:
I have tried importing the ADA_DLL directly with ctypes in Python and it works, so I believe that the ADA_DLL is correctly compiled. Also, forgetting about the C_DLL is not really an option.
I did a small example with a division example module. My .def file looks something like:
; dlltool -z libmath.def --export-all-symbols libmath.dll
EXPORTS
[...]
div # 259
[...]
The libmath_c.c:
#include "libmath_c.h"
PyObject* _wrap_DIVISION(PyObject *self, PyObject *args){
div(10, 2);
return Py_None;
}
__declspec(dllexport) void __cdecl initDIVISION_CPP(void){
Py_InitModule("DIVISION_CPP", LIB_METHODS_methods);
}
The libmath_c.h:
#include <windows.h>
#include <stdio.h>
#include <Python.h>
PyObject* _wrap_DIVISION(PyObject *self, PyObject *args);
static PyMethodDef LIB_METHODS_methods[] = {
{ "CPP_DIVISION", _wrap_DIVISION, METH_VARARGS },
{NULL, NULL, 0, NULL} //Added as indicated by #Brian. Thanks!
};
__declspec(dllexport) void __cdecl initDIVISION_CPP(void);
Any idea of what is happening? Any help would be really appreciated. Thanks!

Preamble: Apologies if this turns out to be a non-answer; I want to be able to come back to this and find the links again, and comments tend to rot...
First, gcc (in the version matching Gnat) may work as an alternative C compiler, and if it does, it may eliminate difficulties with incompatible library versions.
GCC can be used for building Windows DLLs so the result should be usable from other Windows executables.
Following comments; gcc does appear to allow compilation, but the result is not currently usable from Python - here, my Python knowledge is shallow, and we don't have an MCVE, so this is speculative:
This Q&A addresses the same error message between Python and pure C, with no Ada, suggesting this error may not be specific to C-wrapped Ada.
You have already bypassed the asker's specific error,
static PyMethodDef* _npfindmethods = { ... };
which was using a pointer; you are (correctly according to the answer) statically allocating an array. However, the accepted answer terminates the list of methods
static PyMethodDef _npfindmethods[] = {
{"add", py_add, METH_VARARGS, py_add_doc},
{NULL, NULL, 0, NULL}
};
with a NULL method; your example does not:
static PyMethodDef LIB_METHODS_methods[] = {
{ "CPP_DIVISION", _wrap_DIVISION, METH_VARARGS }
};
So my hypothesis is that when you run setup() on this module, it finds CPP_DIVISION successfully, then in the absence of a NULL method it runs off into the weeds, producing the same symptoms despite the difference in cause.
I could test this hypothesis using the MCVE in that question by deleting the NULL method; however I don't have a Windows system handy, only Linux.
Alternatively, I see no reason for a C layer. If there isn't one, this Q&A addresses direct interaction between Python and Ada with no C layer, though it appears to use a different method, getattr() to import the external method. Might be an alternative?

Finally I managed to compile with gcc+gnat but not with MSVC+gnat.
With gcc+gnat, I was getting .//libmath.lib: error adding symbols: Malformed archive. The solution consists on using libmath.dll instead of building the .lib from the .dll.
So, in summary:
If you have a .dll generated by gnat, use it with gcc. You don't need to build a .lib.
If you have a .lib (for example python27.lib) or a .dll not generated by gnat, convert it to a .a using a tool like "pexport" (DO NOT USE SED!).
If you really need to compile using the MSVC... I'm sorry, I could not manage to make it work. Your princess is in another castle.

Error compiling SWIG wrapper

I'm trying to wrap some C++ functions in Python using SWIG. Following the tutorial, I wrote an interface file
%module rasnetclient
%{
#include "rasnetclientcomm.hh"
%}
RasnetClientComm(std::string rasmgrHost, int rasmgrPort);
RasnetClientCommDestroy();
int connectClient(std::string userName, std::string passwordHash);
int disconnectClient();
and generated a wrapper. However, when I try to compile it using g++ -c -std=c++0x rasnetclientcomm.cc rasnetclient_wrap.c -I /usr/include/python2.7 I get the following errors:
rasnetclient_wrap.c: In function 'PyObject* _wrap_connectClient(PyObject*, PyObject*)':
rasnetclient_wrap.c:3383:40: error: 'connectClient' was not declared in this scope
rasnetclient_wrap.c: In function 'PyObject* _wrap_disconnectClient(PyObject*, PyObject*)':
rasnetclient_wrap.c:3396:34: error: 'disconnectClient' was not declared in this scope
Am I doing something wrong with the definition of the interface file? Any help would be appreciated.

Calling c++ from python using ctypes: g++ undefined reference

I've got a Camera (thorlabs dcc1645c) which comes with a "uc480.h" + "uc480.lib" to program it with c++. My problem is that I need to use the Camera with Python, so I tried to call a C++ Class with Python as told by Florian:
Calling C/C++ from python?
This is my c++ code (Cam2.cpp):
\#include "uc480.h"
class Cam{
public:
UC480_CAMERA_LIST* pucl;
int nNumberOfCameras;
int GetNumberOfCameras(){
return is_GetNumberOfCameras(&nNumberOfCameras);
}
};
extern "C" {
Cam* Cam_new(){ return new Cam(); }
int Cam_GetNumberOfCameras(Cam* cam){ cam->GetNumberOfCameras(); }
}
I tried to compile with g++:
g++ -fPIC -c Cam2.cpp -o Cam2.o
g++ -shared -Wl,-soname,libCam2.so -o libCam2.so Cam2.o
The second line gives an error:
Cam2.o:Cam2.cpp:(.text$_ZN3Cam18GetNumberOfCamerasEv[Cam::GetNumberOfCameras()]+
0x1a): undefined reference to `__imp_is_GetNumberOfCameras'
collect2.exe: error: ld returned 1 exit status
'is_GetNumberOfCameras' is defined in "uc480.lib", so in my opinion there is a problem with the linking. How do I fix it?
Also tell me please, if you now other possibilities to use the Camera with Python.

I am working from memory; it might be enough to simply add the lib file to the link command, as in
g++ -shared -Wl,-soname,libCam2.so -o libCam2.so uc480.lib Cam2.o
Or, you might need to add an option like -L. -luc480 to the link command, assuming the library is in the same directory as the code.

Thanks for your help so far. I found a solution for my problem. First of all here is my current c++ code:
#include "uc480.h"
class Cam{
public:
UC480_CAMERA_LIST* pucl = 0;
int nNumberOfCameras;
int GetNumberOfCameras(){
return is_GetNumberOfCameras(&nNumberOfCameras);
}
extern "C"{
__declspec(dllexport) Cam* Cam_New() { return new Cam(); }
__declspec(dllexport) int Cam_GetNumberOfCameras(Cam* obj){ return obj->GetNumberOfCameras(); }
}
I used Visual Studio to build a DLL. I followed the first steps of this instruction http://msdn.microsoft.com/de-de/library/ms235636.aspx :
To create a dynamic link library (DLL) project
On the menu bar, choose File, New, Project.
In the left pane of the New Project dialog box, expand Installed, Templates, Visual C++, and then select Win32.
In the center pane, select Win32 Console Application.
Specify a name for the solution — for example, MyDLL — in the Solution name box. Choose the OK button.
On the Overview page of the Win32 Application Wizard dialog box, choose the Next button.
On the Application Settings page, under Application type, select DLL.
Choose the Finish button to create the project.
After that I added my c++ code to the MyDLL.cpp and compiled to get MyDLL.dll
This can one use with python ctypes:
from ctypes import cdll
lib = cdll.LoadLibrary("PathToLib\MyDLL.dll")
class Camera(object):
def __init__(self):
self.obj = lib.Cam_New()
def GetNumberOfCameras(self):
return lib.Cam_GetNumberOfCameras(self.obj)
MyCam = Camera()
Numbers = MyCam.GetNumberOfCameras()
This works fine for me.

How to use the cl command?

All, I found a piece of information on how to call c files in python, in these examples: there is a c file, which includes many other header files, the very beginning of this c files is #include Python.h, then I found that #include Python.h actually involves many many other header files, such as pystate.h, object.h, etc, so I include all the required header files. In an cpp IDE environment, it did not show errors. What I am trying to do is call this c code in python, so from ctypes import *, then it seems that a dll should be generated by code such as: cl -LD test.c -test.dll, but how to use the cl in this case? I used the cygwin: gcc, it worked fine. Could anyone help me with this i.e.: Call the C in python? Do I make myself clear? Thank you in advance!!
Well, Now I feel it important to tell me what I did:
The ultimate goal I wanna achieve is:
I am lazy, I do not want to re-write those c codes in python, (which is very complicated for me in some cases), so I just want to generate dll
files that python could call. I followed an example given by googleing "python call c", there are two versions in this examples: linux and windows:
The example test.c:
#include <windows.h>
BOOL APIENTRY
DllMain(HANDLE hModule, DWORD dwReason, LPVOID lpReserved) {
return TRUE;
}
__declspec(dllexport) int
multiply(int num1, int num2) {
return num1 * num2;
}
Two versions:
1, Complie under linux
gcc -c -fPIC test.c
gcc -shared test.o -o test.so
I did this in cygwin on my vista system, it works fine; :)
2, Compile under windows:
cl -LD test.c -test.dll
I used the cl in windows command line prompt, it won't work!
These are the python codes:
from ctypes import *
import os
libtest = cdll.LoadLibrary(os.getcwd() + '/test.so')
print test.multiply(2, 2)
Could anyone try this and tell me what you get? thank you!

You will find the command line options of Microsoft's C++ compiler here.
Consider the following switches for cl:
/nologo /GS /fp:precise /Zc:forScope /Gd
...and link your file using
/NOLOGO /OUT:"your.dll" /DLL <your lib files> /SUBSYSTEM:WINDOWS /MACHINE:X86 /DYNAMICBASE
Please have a look at what those options mean in detail, I just listed common ones. You should be aware of their effect nonetheless, so try to avoid copy&paste and make sure it's really what you need - the documentation linked above will help you. This is just a setup I use more or less often.
Be advised that you can always open Visual Studio, configure build options, and copy the command line invokations from the project configuration dialog.
Edit:
Ok, here is some more advice, given the new information you've edited into your original question. I took the example code of your simple DLL and pasted it into a source file, and made two changes:
#include <windows.h>
BOOL APIENTRY DllMain(HANDLE hModule, DWORD dwReason, LPVOID lpReserved)
{
return TRUE;
}
extern "C" __declspec(dllexport) int __stdcall multiply(int num1, int num2)
{
return num1 * num2;
}
First of all, I usually expect functions exported from a DLL to use stdcall calling convention, just because it's a common thing in Windows and there are languages who inherently cannot cope with cdecl, seeing as they only know stdcall. So that's one change I made.
Second, to make exports more friendly, I specified extern "C" to get rid of name mangling. I then proceeded to compile the code from the command line like this:
cl /nologo /GS /Zc:forScope /Gd c.cpp /link /OUT:"foobar.dll" /DL kernel32.lib /SUBSYSTEM:WINDOWS /MACHINE:X86
If you use the DUMPBIN tool from the Visual Studio toolset, you can check your DLL for exports:
dumpbin /EXPORTS foobar.dll
Seeing something like this...
ordinal hint RVA name
1 0 00001010 ?multiply##YGHHH#Z
...you can notice the exported name got mangled. You'll usually want clear names for exports, so either use a DEF file to specify exports in more details, or the shortcut from above.
Afterwards, I end up with a DLL that I can load into Python like this:
In [1]: import ctypes
In [2]: dll = ctypes.windll.LoadLibrary("foobar.dll")
In [3]: dll.multiply
Out[3]: <_FuncPtr object at 0x0928BEF3>
In [4]: dll.multiply(5, 5)
Out[4]: 25
Note that I'm using ctypes.windll here, which implies stdcall.