Hi everyone and thanks for trying to help me !
I encounter trouble when trying to import a python module generated by swig.
I have a basic library "example" containing few methods.
Next to it I have a main program dynamically linked to python.
This program imports the generated module and calls a function in it.
If my library example is a shared one, named _example.so, everything works perfectly, and I can import it in python.
But if my library is static, _example.a, and linked to the main program, then I will have the error "no module named _example was found" unless I add a call to SWIG_init() in the main function.
What exactly does SWIG_init() , and when should I use it ? It seems quite weird to me because it's never said in the documentation to do such a call.
I know that dealing with a .so shared library is better but I try to reproduce the behavior of what I have on a big project at work, so I really have to understand what happens when the module is static.
Here is my main file :
#include "Python.h"
#include <iostream>
#if PY_VERSION_HEX >= 0x03000000
# define SWIG_init PyInit__example
#else
# define SWIG_init init_example
#endif
#ifdef __cplusplus
extern "C"
#endif
#if PY_VERSION_HEX >= 0x03000000
PyObject*
#else
void
#endif
SWIG_init(void);
int main (int arc, char** argv)
{
Py_Initialize();
SWIG_init(); // needed only using the statically linked version of example ?
PyRun_SimpleString("print \"Hello world from Python !\"");
PyRun_SimpleString("import sys");
PyRun_SimpleString("sys.path.append(\"/path/to/my/module\")");
PyRun_SimpleString("import example");
PyRun_SimpleString("a = example.Example()");
PyRun_SimpleString("print a.fact(5)");
}
Here is how things are generated :
swig -c++ -python example.i
g++ -fpic -c example.cpp example_wrap.cxx -I/include/python2.7 -lstdc++
ar rvs libexample.a example.o example_wrap.o
// to generate dynamic instead of static : g++ -shared example.o example_wrap.o -o _example.so
g++ main.cpp -I/include/python2.7 libexample.a -lstdc++ -L/lib/python -lpython2.7 -o main
What you are calling is the init function of the native python module _example that is loaded by the SWIG generated python wrapper. For python 2 this function is named init_example, and for python 3 it is named PyInit__example.
Every python extension with C or C++ needs such a function, it basically initializes everything and registers the name of the module and all the methods available for it. In your case SWIG has generated this function for you.
The reason you have to call this function yourself when you compiled the library statically is simply that the python wrapper example imports the native module _example which is by the python convention a shared object, which you did not compile, and which is thus not found.
By calling SWIG_init, you "preload" the module, so python does not try to reimport it, so it works even though there is no shared object anywhere on the python module path.
If you have the shared object for your module, python will call this function for you after loading the shared object and you don't have to worry about this.
Related
i am trying to wrap the C++ library PyrochloreAFM.hpp which itself uses the lib boost/randoom.hpp so that i can import it as a python module. The C++ part itself works fine and i can succesfully import and use all of this from my main.
#include "PyrochloreAFM.hpp"
int main (int argc, const char *argv[]) {
PyrochloreAFM pyrochloreAFM (¶meters, &statistics, &rng);
}
Now following a tutorial i set up my c++ wrapper:
// pybind11_wrapper.cpp
#include <pybind11/pybind11.h>
#include <PyrochloreAFM.hpp>
PYBIND11_MODULE(pybind11_example, m) {
m.doc() = "pybind11 example plugin"; // Optional module docstring
m.def("cpp_function", &PyrochloreAFM, "A function that multiplies two numbers");
}
and my tasks.py file
# tasks.py
import invoke
invoke.run(
"g++ -O3 -Wall -Werror -shared -std=c++11 -fPIC PyrochloreAFM.cpp "
"-o libpyro.so "
)
However now $invoke build-PyrochloreAFM or even $invoke --list seem to have lost the track of the standard C++ library.
In file included from PyrochloreAFM.cpp:1:
./Parameters.hpp:16:10: fatal error: 'boost/random.hpp' file not found
#include "boost/random.hpp" // tested with boost 1.53
^~~~~~~~~~~~~~~~~~
1 error generated.
This might be a simple PATH problem so i would be very glad for any tips!
Thank you, Andres!
"Of course" the tasks.py file which compiles the .cpp file through the python module invoke has to instruct the compiler g++ to use the libraries we desire. In my case it is the flag -I /opt/homebrew/Cellar/boost/1.80.0/include i also use in my makefile or directly in terminal.
I have a problem with python api in c. I am trying to run a python script with PyRun_SimpleFile but it fails
I get this error: d:/gcc/bin/../lib/gcc/x86_64-w64-mingw32/11.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\aggel\AppData\Local\Temp\ccRzYgwa.o:pyboot.c:(.text+0x47): undefined reference to __imp_PyRun_SimpleFileExFlags' collect2.exe: error: ld returned 1 exit status
The code:
define PY_SSIZE_T_CLEAN
#include <stdio.h>
#include <conio.h>
#include "Python.h"
#include "fileapi.h"
#include "fileobject.h"
int main(){
PyObject* pInt;
FILE *file = fopen( "test.py", "r+" );
PyRun_SimpleFile(file, "test.py");
return 0;
}
undefined reference to __imp_PyRun_SimpleFileExFlags
This means that the function PyRun_SimpleFileExFlags is declared (possibly in file Python.h), but not defined. The definition exists in the compiled python library. The name of the library can be something like libpython.so (for dynamic libraries) or libpython.a (for static libraries).
You need to link your program with the python library.
In gcc, you can use the -l flag. Something like
gcc -lpython3 prog.c
The name "python3" may vary, if the library name starts with "lib" you need not write lib in -l flag.
However, you might need to pass the location of the library explicitly if this does not work. You can pass the location with the -L flag.
gcc -lpython3 -L/location/to/libpython.a prog.c
Only after proper linking will you be able to use the functionalities of Python API.
SOLVED (at least for me) Add
-Wl,path to your python installation\Python\Pythonversionlibs\pythonversion.lib
E.G.
-Wl,C:\Users\Developer\AppData\Local\Programs\Python\Python310\libs
python310.lib
That comma after -Wl is really important
Okay so according to the answer I found here to the question titled "Calling C/C++ from Python?" here, and also on the cppyy documentation website, I made some sample classes in .h and .cpp files and tried to include them in Python. While the .h file gets included easily, when I try to use the cppyy.load_library() function, it gives me a runtime error for some reason. Can someone please help? I've tried to look for solutions online but apparently no one has got a similar problem before. This is what I'm running in Jupyter Notebook:
import cppyy
cppyy.include("foo.h")
cppyy.load_library("libfoo")
the final line is giving me the following error:
RuntimeError Traceback (most recent call last)
<ipython-input-3-eea6173ad08e> in <module>
----> 1 cppyy.load_library("libfoo")
~\anaconda3\lib\site-packages\cppyy\__init__.py in load_library(name)
219 sc = gSystem.Load(name)
220 if sc == -1:
--> 221 raise RuntimeError('Unable to load library "%s"%s' % (name, err.err))
222 return True
223
RuntimeError: Unable to load library "libfoo"
This is my .h file:
class Foo {
public:
void bar();
};
And here is my .cpp file:
#include "foo.h"
#include <iostream>
void Foo::bar() { std::cout << "Hello" << std::endl; }
I'm using the commands g++ -c -fPIC foo.cpp -o foo.o and g++ -shared -Wl,-soname,libfoo.so -o libfoo.so foo.o to compile my cpp code.
Please can someone help?
The example works just fine for me.
To debug, first make sure to start python in the same directory where libfoo.so is located, or to add the directory where libfoo.so lives to LD_LIBRARY_PATH (for any process to use), or call cppyy.add_library_path() with the path as argument to add it for use by cppyy only. As concerns the name, the .so extension is automatically added as needed, as is the lib part, so either one of foo, libfoo, foo.so, or libfoo.so is fine.
If that still fails, a reasonable way on Linux (only) of getting more information for what could be going wrong, is to use ctypes:
$ python
>>> import cypes
>>> lib = ctypes.CDLL("libfoo.so")
which will show you if there are different problems such as missing symbols or missing dependent libraries (but neither is the case here).
I want to use SWIG to create Python bindings for C library, but I have some troubles with it. I used the following .c and .i files from tutorial.
example.c
/* Compute factorial of n */
int fact(int n) {
if (n <= 1)
return 1;
else
return n*fact(n-1);
}
/* Compute n mod m */
int my_mod(int n, int m) {
return(n % m);
}
double My_variable;
example.i
%module example
%{
extern double My_variable;
extern int fact(int);
extern int my_mod(int n, int m);
%}
extern double My_variable;
extern int fact(int);
extern int my_mod(int n, int m);
Commands that I used:
$ swig -python -py3 example.i
$ gcc -c -fpic example.c example_wrap.c -I/usr/include/python3.6
$ gcc -shared example.o example_wrap.o -o example.so
And when I try to import it in python3 I am getting this
>>> import example
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: dynamic module does not define module export function (PyInit_example)
Any help appreciated
SWIG generates a module with underscore prepended by default, so you'll get a module called _example which needs to be in _example.so.
This is explained in the SWIG documentation 34.2.3 Hand compiling a dynamic module:
While the preferred approach to building an extension module is to use the distutils, some people like to integrate building extensions with a larger build system, and thus may wish to compile their modules without the distutils. To do this, you need to compile your program using commands like this (shown for Linux):
$ swig -python example.i
$ gcc -O2 -fPIC -c example.c
$ gcc -O2 -fPIC -c example_wrap.c -I/usr/local/include/python2.5
$ gcc -shared example.o example_wrap.o -o _example.so
The exact commands for doing this vary from platform to platform. However, SWIG tries to guess the right options when it is installed. Therefore, you may want to start with one of the examples in the SWIG/Examples/python directory. If that doesn't work, you will need to read the man-pages for your compiler and linker to get the right set of options. You might also check the SWIG Wiki for additional information.
When linking the module, the name of the output file has to match the name of the module prefixed by an underscore. If the name of your module is example, then the name of the corresponding object file should be _example.so or _examplemodule.so. The name of the module is specified using the %module directive or the -module command line option.
Compatibility Note: In SWIG-1.3.13 and earlier releases, module names did not include the leading underscore. This is because modules were normally created as C-only extensions without the extra Python support file (instead, creating Python code was supported as an optional feature). This has been changed in SWIG-1.3.14 and is consistent with other Python extension modules. For example, the socket module actually consists of two files; socket.py and _socket.so. Many other built-in Python modules follow a similar convention.
I am trying to build a C DLL which can be loaded within python using ctypes.windll.loadlibrary(...)
I can create a DLL and a client program all in C which work following the MinGW tutorial at http://www.mingw.org/wiki/MSVC_and_MinGW_DLLs.
When I try to load the same DLL within python I get an error:
OSError: [WinErrror 193] %1 is not a valid Win32 application
Can someone give me some idea as to what I am doing incorrectly?
Here are the files:
noise_dll.h
#ifndef NOISE_DLL_H
#define NOISE_DLL_H
// declspec will identify which functions are to be exported when
// building the dll and imported when 'including' this header for a client
#ifdef BUILDING_NOISE_DLL
#define NOISE_DLL __declspec(dllexport)
#else
#define NOISE_DLL __declspec(dllimport)
#endif
//this is a test function to see if the dll is working
// __stdcall => use ctypes.windll ...
int __stdcall NOISE_DLL hello(const char *s);
#endif // NOISE_DLL_H
noise_dll.c
#include <stdio.h>
#include "noise_dll.h"
__stdcall int hello(const char *s)
{
printf("Hello %s\n", s);
return 0;
}
I build the DLL with:
gcc -c -D BUILDING_NOISE_DLL noise_dll.c
gcc -shared -o noise_dll.dll noise_dll.o -Wl,--out-implib,libnoise_dll.a
The python code is simply:
import ctypes
my_dll = ctypes.windll.LoadLibrary("noise_dll")
and I get the error above: '%1 is not not a valid Win32 application'
I know the DLL is not completely wrong, because if i create a client file:
noise_client.c
#include <stdio.h>
#include "noise_dll.h"
int main(void)
{
hello("DLL");
return 0;
}
and build with:
gcc -c noise_client.c
gcc -o noise_client.exe noise_client.o -L. -lnoise_dll
I get a working executable. I have some understanding of all of what goes on in the code, options and preprocessor directives above, but am still a little fuzzy on how the .dll file and the .a file are used. I know if I remove the .a file I can still build the client, so I am not even sure what its purpose is. All I know is that it is some kind of archive format for multiple object files
I can ctypes.windll.loadlibrary(...) an ordinary windows DLL found in the windows/system32 without an issue.
One final point:
I am using 64 bit python 3.3. I am using the version of minGW tat comes with the recommended installer (mingw-get-inst-20120426.exe). I am not sure if it is 32 bit, or if that matters.
Thanks!
Use 32-bit python - your DLL probably wasn't compiled as 64 bit code.