I am using the library written by M. Wimmer to compute pfaffians.
I have big code in python, in which I invoke the function from the package pfaffian.
However, I need to compute these pfaffians many times, therefore I would like to increase efficiency (using profiler I checked that computation of pfaffian consumes much time).
Therefore, I would like to use f2py tool to invoke functions from Fortran library in python code. The problem is, that I need only function skpfa which depends on other functions which belong to the library.
I tried to use instructions from here to link with static library libpfapack.a, however while importing the output module (which I called pfaf.so) I got the error message:
./pfaf.so: undefined symbol: skpfa_
I read that the problem can lie in the fact that I want to link static library, therefore I created dynamical library from provided sources using command:
gfortran -O3 -fimplicit-none -c -fPIC file.f -o file.o
for all source files, and then
gfortran -shared $(OBJECTS) -o libpfapack
I created following signature file pfaf.pyf:
python MODULE pfaffian
PUBLIC
INTERFACE
SUBROUTINE SKPFA(A, PFSFF, UPLO, MTHD, INFO)
double precision, intent(in) :: A(:,:)
double precision, intent(inout) :: p
character, intent(in), optional :: UPLO, MTHD
integer, intent(out), optional :: info
end subroutine skpfa
end interface
end python module pfaffian
and I invoked:
f2py -c --lower --fcompiler=gnu95 pfaf.pyf -L{path to directory with libpfapack} -lpfapack
and I got a message
/usr/bin/ld: cannot find -lpfapack
If I invoke f2py command without -lpfapack the things compile and produce file pfaf.so, however I get an error in python :
./pfaf.so: undefined symbol: skpfa_.
Does anyone have an idea how to fix it?
Related
I need to write a python wrapper for an existing C++ Module. First I tested the procedere with this basic example (which now actually works fine): C++ - Python Binding with ctypes - Return multiple values in function
Now I tried to change the setting: I want to use the existing lib instead of my single cpp file. I tried it with this:
g++ -c -I. -fPIC projectionWrapper.cpp -o projectionWrapper.o
g++ -shared -Wl,-soname,libproj.so
-L./build/liborig_interface.a,./build/liborig_base.a
-o libproj.so projectionWrapper.o
I wanted to link against both .a files from the given library with the -L command. I don't get any errors on that, but when I try to import the module via ipython, I get this:
import myprojection # I load libproj.so in this python file
OSError: ./libproj.so: undefined symbol: _Z29calibration_loadPKcjbP14camera_typetS2_
There is a function "calibration_load", as well a "camera_type" in the original framework. But I have no clue where the cryptic things in between come from.
Sorry for my vague explanation, I tried to explain it as good as possible, but a C++ Wrapper is not one of my topics where I feel "at home".
The problem is that you're not linking against the external library that you use in your C++ code; add -l<library> to your second g++ call.
g++ -shared projectionWrapper.o
-L./build/base -L./build/interface
-loriginterface -lorigbase
-Wl,-soname,libproj.so
-o libproj.so
did the job. Thanks for the hint that I actually didn't link the libraries as I only used the -L option.
Moreover the order of the options was wrong. I had to state "projectionWrapper.o" right at the beginning, as well as "-loriginterface" before "-lorigbase". This was answered here: "undefined reference" when linking against a static library
(complete name of the libs are: liboriginterface.a and liborigbase.a)
I am having trouble building my own python extension. Building this code worked before (on Debian 7 Wheezy), but is now failing (on Ubuntu 15.04 Vivid).
The modules in question appear to link correctly, but I get an error on import. I have tried two linker lines, one with g++ (which gives me a missing symbol error for a fortran runtime function), and the other with gfortran (which gives me a missing symbol error for a runtime vtable.)
The module uses:
some polymorphic C++ code written by me,
a FORTRAN (90) routine from stripack, linked to C via a header file written by me using yolinux's guide,
a Cython file to expose some python entry points to the routines.
Thus, it needs to be linked with both the C++ standard libraries and the FORTRAN runtime, and be built into a monolithic shared object file.
I use a python build manager that I wrote myself to solve this problem: the currently published version generates the following linker line:
gfortran -fno-strict-aliasing -fPIC -pthread -shared \
-Wl,-O1 -Wl,-Bsymbolic-functions -lc -lstdc++ cpp1.o cpp2.o f90.o pyx.o -o \
module.so
(with some files omitted and paths shortened.)
This yields the following import error:
ImportError: module.so: undefined symbol: _ZTVN10__cxxabiv117__class_type_infoE
Which is apparently a reference to "vtable for __cxxabiv1::__class_type_info". This means, I think, that the -lstdc++ entry in the linker line is not doing its job correctly.
Similarly, I have tried to modify my build system to generate a g++ line linking against libgfortran like so:
c++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -lgfortran cpp1.o cpp2.o \
f90.o pyx.o -o module.so
Again, this gives me a missing symbol error:
ImportError: module.so: undefined symbol: _gfortran_st_write_done
I have also experimented with static linking, which gives an error at link time itself (like "unable to move symbol"), and initially I thought that the runtimes may be split over separate files and tried -lfoo for many foo. However, I've checked the libgfortran.so file on the linker path with scanelf, and it does contain the relevant symbol.
This method of linking used to work (on Debian Wheezy, also tested on Mac OS X 10.7 a long time ago.) I am struggling to understand how it can have broken on the latest ubuntu (with a newer GCC, 4.9).
Any ideas, theories and tests to help debug/solve this would be much appreciated.
Thank you to #Marc Glisse for your comment: the order of the arguments is indeed wrong.
If the original gfortran line is modified so that the -lc and -lstdc++ terms are last, the ImportError goes away. Thank you!
I want to create a python module which can have its functions called from a C++ class and call c++ functions from that class
i have looked at boost however it hasn't seemed to make any sense
it refers to a shared library (which i have no idea how to create) and i cant fallow the code they use in examples (it seems very confusing)
here is their hello world tutorial
(http://www.boost.org/doc/libs/1_55_0b1/libs/python/doc/tutorial/doc/html/index.html#python.quickstart)
Following C/C++ tradition, let's start with the "hello, world". A C++ Function:
char const* greet()
{
return "hello, world";
}
can be exposed to Python by writing a Boost.Python wrapper:
include <boost/python.hpp>
BOOST_PYTHON_MODULE(hello_ext)
{
using namespace boost::python;
def("greet", greet);
}
That's it. We're done. We can now build this as a shared library. The resulting DLL is now
visible to Python. Here's a sample Python session:
>>> import hello_ext
>>> print hello_ext.greet()
hello, world
Next stop... Building your Hello World module from start to finish...
could someone please help explain what is being done and most of all how python knows about the C++ file
Python does not know about the C++ file, it will only be aware of the extension module that is compiled from the C++ file. This extension module is an object file, called a shared library. This file has an interface that looks to Python as if it was a normal Python module.
This object file will only exist after you tell a compiler to compile the C++ file and link it with all the libraries it needs. Of course, the first library needed is Boost.Python itself, which must be available on the system where you are compiling.
You can tell Python to compile the C++ file for you, so that you do not need to mess with the compiler and its library flags. In order to do so, you need a file called setup.py where you use the Setuptools library or the standard Distutils to define how your other Python modules are to be installed on the system. One of the steps for installing is compiling all extension modules, called the build_ext phase.
Let us imagine you have the following directories and files:
hello-world/
├── hello_ext.cpp
└── setup.py
The content of setup.py is:
from distutils.core import setup
from distutils.extension import Extension
hello_ext = Extension(
'hello_ext',
sources=['hello_ext.cpp'],
include_dirs=['/opt/local/include'],
libraries=['boost_python-mt'],
library_dirs=['/opt/local/lib'])
setup(
name='hello-world',
version='0.1',
ext_modules=[hello_ext])
As you can see, we are telling Python there is an Extension we want to compile, where the source file is, and where the included libraries are to be found. This is system-dependent. The example shown here is for a Mac OS X system, where Boost libraries were installed via MacPorts.
The content of hello_ext.cpp is as shown in the tutorial, but take care to reorder things so that the BOOST_PYTHON_MODULE macro comes after the definitions of whatever must be exported to Python:
#include <boost/python.hpp>
char const* greet()
{
return "hello, world";
}
BOOST_PYTHON_MODULE(hello_ext)
{
using namespace boost::python;
def("greet", greet);
}
You can then tell Python to compile and link for you by executing the following on the command line:
$ python setup.py build_ext --inplace
running build_ext
building 'hello_ext' extension
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -pipe -Os -fwrapv -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/include -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c hello_ext.cpp -o build/temp.macosx-10.9-x86_64-2.7/hello_ext.o
/usr/bin/clang++ -bundle -undefined dynamic_lookup -L/opt/local/lib -Wl,-headerpad_max_install_names -L/opt/local/lib/db46 build/temp.macosx-10.9-x86_64-2.7/hello_ext.o -L/opt/local/lib -lboost_python-mt -o ./hello_ext.so
(The --inplace flag tells Python to leave the products of compilation right next to the source files. The default is to move them to a build directory, to keep the source directory clean.)
After that, you will find a new file called hello_ext.dll (or hello_ext.so on Unix) on the hello-world directory. If you start a Python interpreter in that directory, you will be able to import the module hello_ext and use the function greet, as shown in the Boost tutorial.
Python is an interpreted language. This means that it needs a virtual machine to execute the statements. For example, if it encounters a = 5, python (or rather the virtual machine that interprets your python code), will create an object in memory that holds some information and the value 5 and will make sure that any following reference to a will find the object. Same goes for more complex statements like input, on these commands, the virtual machine will trigger a hard coded routine which will do a lot of work under the hood before returning back to read the next piece of python code. So far, so good.
About modules. When issuing the import statement, python will look for the specified module name into its path. This is usually a .py file containing only pure python code to interpret. But that can also be a .pyd file, containing compiled routines that python can use like an executable would do with a shared library. This file contains symbols and entry points so that when the interpreter finds a special method name like mymodule.mymethod() it knows where to find the routine to execute and runs it.
However, these routines have to conform to a specific interface, and that's why it is not straightforward to expose C/C++ functions to python. The most obvious problem is that python int is not a C int, not a short, not even a long. It's a special structure that holds a lot more information like how often the variable is referenced (to be able to free memory for variables that are not referenced anymore), the type of the value it holds, etc. Of course, a typical C/C++ library doesn't work with these complex types, but uses vanilla int, float, char* and other nice plain types. So one has to translate the necessary python values to simple C types that can be understood by the library, and convert back the potential results delivered by the library into a format usable by python's virtual machine. This is what is called the wrapper. The wrapper also has to take care of funny things like reference counts, memory management on the heap, initialization and finalization, and other monkeys. See some examples to get an idea of how such code can look like. This is not extremely complicated, but still some work.
Now you get an idea of all the hard work done under the hood by the Python.Boost library (or other wrapping tools for that matters) when calling the ridiculously simple def("greet", greet);.
I have a working c++ code that I want to wrap into a python module on Windows XP and Python 2.7. I have never done this before, so I looked into swig and distutils.
I created an interface file and a setup.py and compiled using
python setup.py build_ext -c mingw32
The script creates a module_wrap.cpp from my module.i and module.cpp file, and then creates a module_wrap.o and a module.o. The creation of module.o creates a bunch of Warnings for unused variables and deprecated char*, but it seems to work. Because the C++-code is not mine, I don't really want to get into these right now.
The last step is executing
g++.exe -shared -s build\temp.win32-2.7\Release\module_wrap.o build\temp.win32-2.7\Release\module.o build\temp.win32-2.7\Release\_module.def -LC:\Programme\Python27\libs -LC:\Programme\Python27\PCbuild -lpython27 -o build\lib.win32-2.7\_module.pyd
I get
Cannot export init_module: symbol not defined
error: command 'g++' failed with exit status 1
I googled a lot to this now, and I just can not find a solution to this problem. The previously created _module.def seems to try to export this init since it contains
LIBRARY _module.pyd
EXPORTS
init_module
Obviously this doesn't work, but I have no idea why. Can anyone help me out here?
I figured it out. The problem was the (not posted) interface file module.i for swig. There I named the module %module usemodule, whereas in the setup.py i named the module name=module. This way swig created an init_function, that did not match the name the created module was expecting it. In the end: just a typo...
Thanks for your support nevertheless!
I am having difficulty with getting a f2py compiled module work in Python.
I have a piece of software written in Fortran that compiles well on a Linux 64bit machine.
Further on F2Py compiles a Python module that uses Fortran bits as well.
Here is how the Python module is compiled:
f2py --fcompiler=gfortran -I"path-to-dir-with-mod-files" -c -m mod_landems mod_landem.f90
But once I want to import that module I get an error (in Ipython):
----> 1 import mod_landems
ImportError: ./mod_landems.so: undefined symbol: __nesdis_landem_module_MOD_nesdis_landem
To be honest I am confused with this error. Search did not help much so I need to ask you here: how can I possibly make it work? If I put the python module code in the same directory as where the mod files are it produces same error message.
Here is a piece of my (primitive) code:
module n_landem
implicit none
! INPUT VARIABLES
real(8) Angle
real(8) Sm_Content
real(8) Veg_Frac
real(8) Soil_Temp
real(8) Land_Temp
real(8) Snow_Depth
real(8) Frequency
! OUTPUT VARIABLES
real(8) Emis_H
real(8) Emis_V
contains
subroutine landem
USE NESDIS_LANDEM_MODULE
USE TYPE_KINDS, ONLY : fp
call NESDIS_LandEM(Angle,Frequency,Sm_Content,Veg_Frac,Soil_Temp,Land_Temp,Snow_Depth,Emis_H,Emis_V)
end subroutine landem
end module n_landem
If I recall correctly some time ago this module was importable, but can't seem to make it work on either debian64 installation or debian32bit computer.
in your f2py call, you have to pass the libraries you are linking explicitly with '-l', the same you would pass it to your Fortran compiler (i.e. gfortran). Therefore, does
f2py -c --fcompiler=gfortran -I"path-to-dir-with-mod-files" --fcompiler=gfortran -I"path-to-dir-with-mod-files" -lNESDIS_LandEM_Module -m mod_landems mod_landem.f90 -m mod_landems mod_landem.f90
work for you ?
Best,
Max.
I'm working on a similar project and too am new to Fortran. Using the reference below I found that you import the fortran module just as you would import a library and then call a function similarly.
http://cens.ioc.ee/projects/f2py2e/usersguide/#the-quick-and-smart-way