I am working on a project which required a large number of dictionary lookups in cython. To try improve the speed I attempted to replace the dictionaries with unordered_maps from libcpp.
#!python
#cython: boundscheck=False, wraparound=False, infer_types=True,cdivision = True
from libcpp.unordered_map cimport unordered_map
However when I attempt to compile with gcc on the command line the compilation fails with
CAStar2.c:482:19: fatal error: utility: No such file or directory
#include <utility>
^
compilation terminated.
It seems the compiler cannot find multiple required files.
How would I point it to those files?
The cython generated C file needs to be compiled by a setuptools Extension.
If compiling manually all the include and lib directories that setuptools would otherwise use need to be also specified manually.
See cython setuptools documentation here and here for a minimal compile from cmd line example.
I turns out the issue was my compiler not activating support for c++11 which cython seems to produce. Adding the option -std=c++11 everything compiled normally.
Thanks though to everyone who helped.
Related
I want to add python functions in C++ code.
I made a GUI in gtk (on the Raspberry PI) and now I want to work with a camera module which is easy to handle in python. (I want to start a video directly when I push a button.)
So I included the file Python.h
#include <python3.4m/Python.h>
#include <python3.4m/pythonrun.h>
then I thought it should work, but when I try to compile Py_Initialize()
I get the error:
undefined reference to Py_Initialize.
I think this is strange because, when I type in, there came the selection for Py_Initialize.
In terms of headers you should be fine, since it compiled but failed at linking.
Now you need to link against the Python libraries. The way this is done largely depends on what toolchain you are using.
Maybe you can see my answer in another question:
if with python 3.x installed, maybe this command can work:
g++ hw.cpp `/usr/bin/python3-config --cflags` `/usr/python3-config --ldflags`
By the way, you should check you gcc and python version.
As I know, if gcc version is 5.4 and python version is 3.7, it doesn't work.(python 3.5 >is work)
When you run /usr/bin/python3-config --cflags, in fact, it is the compile option.
Set the python include folder and it static lib on gcc command line and put the python dynamic lib on LD_LIBRARY_PATH. Before Py_Initialize(), do not forget to set python home with Py_SetPythonHome(). These steps must be sufficient for your code compile and run.
SWIG generates wrapper code from your C/C++ in a desired target language (Python, Java, C#, etc) using an interface (.i) file that specifies the input code to be wrapped as described in the SWIG tutorial. CMake can be used to call swig in order to generate target code from the .i interface, as described in the SWIG documentation.
However, using this method CMake only generates a dependency for the interface file itself, but not for the source files it includes. One can manually add dependencies, but SWIG can generate dependencies automatically with the -MM option, and I would like for these to be utilized by CMake.
There was a commit to CMake that utilized dependencies generated by swig -MM but it was later reverted due to a problem with generated sources that didn't exist at the time of the call to swig. At this time the problem seems to remain unsolved.
So I put the issue to the brilliant StackOverflow community: Is there a way with the current CMake to utilize dependencies generated by swig -MM when the interface file (a) does not include generated code (e.g. config.h), and (b) includes generated code?
Here is a small example that can be used for experimentation (download it here).
// swig_example.h
int foo(int n);
//*** comment this declaration after compiling once to witness dependency failure ***/
int another_function();
// swig_example.cpp
#include "swig_example.h"
int another_function() {return -1;}
int foo(int n)
{
if (n <= 1) return 1;
else return another_function();
}
// swig_example: example.i
%module example
%{
#include "swig_example.h"
%}
%include "swig_example.h"
# swig_example: CMakeLists.txt
FIND_PACKAGE(SWIG REQUIRED)
INCLUDE(${SWIG_USE_FILE})
FIND_PACKAGE(PythonLibs)
INCLUDE_DIRECTORIES(${PYTHON_INCLUDE_PATH})
INCLUDE_DIRECTORIES(${CMAKE_CURRENT_SOURCE_DIR})
SET(CMAKE_SWIG_FLAGS "")
SET_SOURCE_FILES_PROPERTIES(example.i PROPERTIES CPLUSPLUS ON)
#add manual dependencies (must be called before SWIG_ADD_MODULE)
#SET(SWIG_MODULE_example_EXTRA_DEPS ${CMAKE_SOURCE_DIR}/swig_example.h)
SWIG_ADD_MODULE(example python example.i swig_example.cpp)
SWIG_LINK_LIBRARIES(example ${PYTHON_LIBRARIES})
Compile it once, then comment the declaration of another_function and try to compile again. Because the swig interface is not regenerated an error occurs trying to compile examplePYTHON_wrap.cxx.
examplePYTHON_wrap.cxx:3220:17: error: use of undeclared identifier 'another_function'
result = (int)another_function();
Uncomment the add manual dependencies line of the CMakeLists.txt and the interface will be properly regenerated. However, I want this to work using dependencies generated from swig -MM instead of needing to manually specify dependencies.
$ swig -python -MM -c++ ../example.i
../example_wrap.cxx: \
../example.i \
../swig_example.h \
Turning my comments into an answer
I don't think - if you want to do this automatically and e.g. want to utilize swig -MM - that this can be done without changing the UseSWIG.cmake code.
When I look at why the previous attempt you have linked was reverted - namely the discussion of on "0012307: regression in 2.8.5 rc2: UseSWIG.cmake broken" - I don't believe that SWIG_GET_WRAPPER_DEPENDENCIES() was actually broken it just introduced a new restriction: "this simply require all headers for swig module to be present" before calling SWIG_ADD_MODULE().
So I suggested to add the -ignoremissing SWIG option, but this would need further testing.
Update (April 2017)
With CMake version 3.8.0 there came a fix "Automatically scan dependencies of SWIG files for Makefile generators" that works for makefile generators.
Reference
The general discussion of how to fix this (including my suggestion) is discussed at "Issue #4147: [MODULES][UseSWIG] Use swig to compute dependencies". The ticket is still open (was reopened), so please feel free to add your support, suggestions or test results there.
I'm trying to integrate CUDA to an existing aplication wich uses boost::spirit.
Isolating the problem, I've found out that the following code does not copile with nvcc:
main.cu:
#include <boost/spirit/include/qi.hpp>
int main(){
exit(0);
}
Compiling with nvcc -o cudaTest main.cu I get a lot of errors that can be seen here.
But if I change the filename to main.cpp, and compile again using nvcc, it works. What is happening here and how can I fix it?
nvcc sometimes has trouble compiling complex template code such as is found in Boost, even if the code is only used in __host__ functions.
When a file's extension is .cpp, nvcc performs no parsing itself and instead forwards the code to the host compiler, which is why you observe different behavior depending on the file extension.
If possible, try to quarantine code which depends on Boost into .cpp files which needn't be parsed by nvcc.
I'd also make sure to try the nvcc which ships with the recent CUDA 4.1. nvcc's template support improves with each release.
I want to create a python module which can have its functions called from a C++ class and call c++ functions from that class
i have looked at boost however it hasn't seemed to make any sense
it refers to a shared library (which i have no idea how to create) and i cant fallow the code they use in examples (it seems very confusing)
here is their hello world tutorial
(http://www.boost.org/doc/libs/1_55_0b1/libs/python/doc/tutorial/doc/html/index.html#python.quickstart)
Following C/C++ tradition, let's start with the "hello, world". A C++ Function:
char const* greet()
{
return "hello, world";
}
can be exposed to Python by writing a Boost.Python wrapper:
include <boost/python.hpp>
BOOST_PYTHON_MODULE(hello_ext)
{
using namespace boost::python;
def("greet", greet);
}
That's it. We're done. We can now build this as a shared library. The resulting DLL is now
visible to Python. Here's a sample Python session:
>>> import hello_ext
>>> print hello_ext.greet()
hello, world
Next stop... Building your Hello World module from start to finish...
could someone please help explain what is being done and most of all how python knows about the C++ file
Python does not know about the C++ file, it will only be aware of the extension module that is compiled from the C++ file. This extension module is an object file, called a shared library. This file has an interface that looks to Python as if it was a normal Python module.
This object file will only exist after you tell a compiler to compile the C++ file and link it with all the libraries it needs. Of course, the first library needed is Boost.Python itself, which must be available on the system where you are compiling.
You can tell Python to compile the C++ file for you, so that you do not need to mess with the compiler and its library flags. In order to do so, you need a file called setup.py where you use the Setuptools library or the standard Distutils to define how your other Python modules are to be installed on the system. One of the steps for installing is compiling all extension modules, called the build_ext phase.
Let us imagine you have the following directories and files:
hello-world/
├── hello_ext.cpp
└── setup.py
The content of setup.py is:
from distutils.core import setup
from distutils.extension import Extension
hello_ext = Extension(
'hello_ext',
sources=['hello_ext.cpp'],
include_dirs=['/opt/local/include'],
libraries=['boost_python-mt'],
library_dirs=['/opt/local/lib'])
setup(
name='hello-world',
version='0.1',
ext_modules=[hello_ext])
As you can see, we are telling Python there is an Extension we want to compile, where the source file is, and where the included libraries are to be found. This is system-dependent. The example shown here is for a Mac OS X system, where Boost libraries were installed via MacPorts.
The content of hello_ext.cpp is as shown in the tutorial, but take care to reorder things so that the BOOST_PYTHON_MODULE macro comes after the definitions of whatever must be exported to Python:
#include <boost/python.hpp>
char const* greet()
{
return "hello, world";
}
BOOST_PYTHON_MODULE(hello_ext)
{
using namespace boost::python;
def("greet", greet);
}
You can then tell Python to compile and link for you by executing the following on the command line:
$ python setup.py build_ext --inplace
running build_ext
building 'hello_ext' extension
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -pipe -Os -fwrapv -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/include -I/opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c hello_ext.cpp -o build/temp.macosx-10.9-x86_64-2.7/hello_ext.o
/usr/bin/clang++ -bundle -undefined dynamic_lookup -L/opt/local/lib -Wl,-headerpad_max_install_names -L/opt/local/lib/db46 build/temp.macosx-10.9-x86_64-2.7/hello_ext.o -L/opt/local/lib -lboost_python-mt -o ./hello_ext.so
(The --inplace flag tells Python to leave the products of compilation right next to the source files. The default is to move them to a build directory, to keep the source directory clean.)
After that, you will find a new file called hello_ext.dll (or hello_ext.so on Unix) on the hello-world directory. If you start a Python interpreter in that directory, you will be able to import the module hello_ext and use the function greet, as shown in the Boost tutorial.
Python is an interpreted language. This means that it needs a virtual machine to execute the statements. For example, if it encounters a = 5, python (or rather the virtual machine that interprets your python code), will create an object in memory that holds some information and the value 5 and will make sure that any following reference to a will find the object. Same goes for more complex statements like input, on these commands, the virtual machine will trigger a hard coded routine which will do a lot of work under the hood before returning back to read the next piece of python code. So far, so good.
About modules. When issuing the import statement, python will look for the specified module name into its path. This is usually a .py file containing only pure python code to interpret. But that can also be a .pyd file, containing compiled routines that python can use like an executable would do with a shared library. This file contains symbols and entry points so that when the interpreter finds a special method name like mymodule.mymethod() it knows where to find the routine to execute and runs it.
However, these routines have to conform to a specific interface, and that's why it is not straightforward to expose C/C++ functions to python. The most obvious problem is that python int is not a C int, not a short, not even a long. It's a special structure that holds a lot more information like how often the variable is referenced (to be able to free memory for variables that are not referenced anymore), the type of the value it holds, etc. Of course, a typical C/C++ library doesn't work with these complex types, but uses vanilla int, float, char* and other nice plain types. So one has to translate the necessary python values to simple C types that can be understood by the library, and convert back the potential results delivered by the library into a format usable by python's virtual machine. This is what is called the wrapper. The wrapper also has to take care of funny things like reference counts, memory management on the heap, initialization and finalization, and other monkeys. See some examples to get an idea of how such code can look like. This is not extremely complicated, but still some work.
Now you get an idea of all the hard work done under the hood by the Python.Boost library (or other wrapping tools for that matters) when calling the ridiculously simple def("greet", greet);.
I have a working c++ code that I want to wrap into a python module on Windows XP and Python 2.7. I have never done this before, so I looked into swig and distutils.
I created an interface file and a setup.py and compiled using
python setup.py build_ext -c mingw32
The script creates a module_wrap.cpp from my module.i and module.cpp file, and then creates a module_wrap.o and a module.o. The creation of module.o creates a bunch of Warnings for unused variables and deprecated char*, but it seems to work. Because the C++-code is not mine, I don't really want to get into these right now.
The last step is executing
g++.exe -shared -s build\temp.win32-2.7\Release\module_wrap.o build\temp.win32-2.7\Release\module.o build\temp.win32-2.7\Release\_module.def -LC:\Programme\Python27\libs -LC:\Programme\Python27\PCbuild -lpython27 -o build\lib.win32-2.7\_module.pyd
I get
Cannot export init_module: symbol not defined
error: command 'g++' failed with exit status 1
I googled a lot to this now, and I just can not find a solution to this problem. The previously created _module.def seems to try to export this init since it contains
LIBRARY _module.pyd
EXPORTS
init_module
Obviously this doesn't work, but I have no idea why. Can anyone help me out here?
I figured it out. The problem was the (not posted) interface file module.i for swig. There I named the module %module usemodule, whereas in the setup.py i named the module name=module. This way swig created an init_function, that did not match the name the created module was expecting it. In the end: just a typo...
Thanks for your support nevertheless!