compiling Cython with MinGW - undefined reference PyExc - python

I'm trying to compile a simple code snippet from the book "Cython - A guide for Python programmers", and when i compile, i get the following error:
H:\Cython>python setup.py build_ext -i --compiler=mingw32
running build_ext
building 'fib' extension
C:\MinGW\bin\gcc.exe -mdll -O -Wall -IC:\Anaconda3\include -IC:\Anaconda3\include -c fib.c -o build\temp.win32-3.4\Release\fib.o
writing build\temp.win32-3.4\Release\fib.def
C:\MinGW\bin\gcc.exe -shared -s build\temp.win32-3.4\Release\fib.o build\temp.win32-3.4\Release\fib.def -LC:\Anaconda3\libs -LC:\Ana
conda3\PCbuild -lpython34 -lmsvcr100 -o H:\Cython\fib.pyd
build\temp.win32-3.4\Release\fib.o:fib.c:(.text+0xb6): undefined reference to `_imp__PyExc_TypeError'
build\temp.win32-3.4\Release\fib.o:fib.c:(.text+0xf3): undefined reference to `_imp__PyExc_TypeError'
build\temp.win32-3.4\Release\fib.o:fib.c:(.text+0x3cc): undefined reference to `_imp___PyThreadState_Current'
build\temp.win32-3.4\Release\fib.o:fib.c:(.text+0x857): undefined reference to `_imp__PyExc_NameError'
build\temp.win32-3.4\Release\fib.o:fib.c:(.text+0xa55): undefined reference to `_imp__PyExc_ImportError'
c:/mingw/bin/../lib/gcc/mingw32/4.8.1/../../../../mingw32/bin/ld.exe: build\temp.win32-3.4\Release\fib.o: bad reloc address 0x0 in s
ection `.data'
collect2.exe: error: ld returned 1 exit status
error: command 'C:\\MinGW\\bin\\gcc.exe' failed with exit status 1
H:\Cython>
setup.py:
from distutils.core import setup
from Cython.Build import cythonize
setup(ext_modules=cythonize('fib.pyx'))
fib.pyx:
def fib(int n):
cdef int i
cdef double a=0.0, b=1.0
for i in range(n):
a, b = a + b, a
return a
When I google this problem, others who have had the error had a mix between 32 and 64 bit stuff, I'm not doing that.

I sat down and looked the errors over again today, and found the issue.
The problem is that I used Anaconda rather than compiling everything from scratch myself - that means that some Cython components were compiled with MSVC. As above you can see that I'm trying to use MinGW to compile the Cython test script. Why mixing compilers like that doesn't work is outside my scope of knowledge, but it doesn't. Compiling my Cython test scripts with MSVC works.
(use Visual Studio C++ 2008/2010 for python 2.x/3.x respectively)
As for the reason that I attempted to use MinGW (against standard recommendation) was that my msiserver service had somehow broken (I'm on an old laptop, so i don't recall the reason), and was hoping to find a quick way out, rather than fixing the msiserver service.
The fix for the msiserver service is pretty unrelated to this question, but it was rather hard to find, so i figured I'd link and mirror it here:
http://www.vistax64.com/vista-installation-setup/96680-repair-windows-installer-service-vista-all-versions.html
For all those unfortunate souls searching and Googling for how to
repair the Windows Installer Service, I have some info for you. A
couple days ago I tried to uninstall one of my apps and stalled on an
error "Windows Installer Service cannot be accessed". After many
trials and errors trying to fix this issue, I stumbled upon a new fix
for this issue that has worked in all these situations where the
Windows Installer Service will not manually start and in essense, not
allow install or uninstall tasks to complete.
Here's the easy steps:
1. Go to a Windows Vista (Any Version) computer that has the Windows
Installer service running correctly and run regedit(Start-Run-Regedit)
2. Go to the location
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\msiserver
3. Right click on this key and select "Export" and save the key to a Flash
Drive or other.
4. Run sfc /scannow on the damaged Vista computer - you won't need the
install disk as it goes to backup files on your HD. Do not reboot when
complete
5. Double click saved .reg file from working machine and import registry
settings into damaged Vista computer.
6. Now reboot and try to install/uninstall
If many of you have success with this method, please post this fix
around the WWW as I went through over 1000 links with users having the
same problem and not being able to solve it. Shame on Microsoft, very
sloppy. It would have been so nice if Microsoft released Windows
Installer 4.0 as a standalone installation with Vista's release so I
could have repaired it, most users have been doing fresh installs to
fix this. Get your act together Microsoft!!!!
The CAT
Thank you, the cat.

Related

How to install C++ dependencies "from source" for a Python package on Mac OS?

There is a Github repo containing Python "bindings" for a C++ library that I am interested in playing with. The README has abundant information about how to install the C++ library on Linux like machines, but no information about how to do so with a mac OS.
I have also opened up an issue requesting the README installation instructions include mac OS-specific installs in addition to linux. There hasn't been any activity on that issue.
Here are the two repos:
(Python) https://github.com/asiffer/python3-libspot
(C++) https://github.com/asiffer/libspot
Since the C++ package isn't available for installing via Brew/pip/anaconda, I'm not sure how to get going.
What I've Tried:
I have tried ./configure, and make. There is no ./configure file.
To address the lack of ./configure, read about a tool called autoconf which supposedly generates ./configure for you. I installed it with brew, but am not sure what arguments to pass it. These docs were pretty hard to understand: https://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/Making-configure-Scripts.html
Just using make results in the error clang: error: unsupported option '-fopenmp'That sent me down a whole different rabbit hole which had me adding lines to the Makefile:
CPP = /usr/local/opt/llvm/bin/clang
CPPFLAGS = -I/usr/local/opt/llvm/include -fopenmp
LDFLAGS = -L/usr/local/opt/llvm/lib
omp_hello: omp_hello.c
$(CPP) $(CPPFLAGS) $^ -o $# $(LDFLAGS)
That felt dangerous because I have no idea what any of that stuff means. Plus it resulted in a new error: *** missing separator. Stop.
So then I read that's probably due to using "soft" tabs instead of "hard" tabs which can be identified using cat -e -t -v makefile_name. I found the one line where a "hard" tab was missing (the indented line above) and inserted it. This resulted in a new error:
make: *** No rule to make target `omp_hello.c', needed by `omp_hello'. Stop.
Next, following the advice of Yang Yushi and his follow on comments, I changed lines 39 and 40 according to his answer, plus added the locations of some additional files to the CXXFLAGS variable:
-I//opt/homebrew/Cellar/libomp/11.0.1/include
-L/opt/homebrew/Cellar/libomp/11.0.1/lib
And this got me a little further. Next, OSX didn't like where this script was trying to install, as explained by this answer. So I changed these two lines in the makefile which seemed to dictate install location:
INSTALL_HEAD_DIR = $(DESTDIR)/usr/include/libspot
INSTALL_LIB_DIR = $(DESTDIR)/usr/lib
to
INSTALL_HEAD_DIR = $(DESTDIR)/usr/local/include/libspot
INSTALL_LIB_DIR = $(DESTDIR)/usr/local/lib
And that indeed got me a little farther. Next I ran into an error complaining about the flat -t at these lines in the makefile:
#install -t $(INSTALL_LIB_DIR) $(LIB_DIR)/*.so
#install -t $(INSTALL_HEAD_DIR) $(INC_DIR)/*.h
So I deleted those flags, which then resulted in this error:
Checking the headers installation directory (/usr/local/include/libspot)
Checking the library installation directory (/usr/local/lib)
Installing the shared library (libspot.so)
install: /usr/local/lib: Inappropriate file type or format
For which I can find no reading material and have no clue how to fix. Any further assistance appreciated.
Here's a list of SO and other resources I've perused trying to answer this question:
Enable OpenMP support in clang in Mac OS X (sierra & Mojave)
makefile error: make: *** No rule to make target `omp.h' ; with OpenMP
makefile:4: *** missing separator. Stop
http://www.idryman.org/blog/2016/03/10/autoconf-tutorial-1/
https://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/Making-configure-Scripts.html
https://developer.gnome.org/anjuta-build-tutorial/stable/create-autotools.html.en
My Question
How do I proceed.
If you know how to do this, could you also include a brief explanation of the concepts behind each step? I'd be happy to learn a little instead of just copying and pasting commands in the right order.
Compile the C++ source code with Apple Clang
I downloaded the prjoect (libspot) and successfully compiled it on my Mac. I change two lines (39 and 40) in the Makefile to make it work. (Following this answer)
CC = clang++ # change from g++ to default Apple clang
CXXFLAGS = -std=c++11 -Wall -pedantic -Xpreprocessor -fopenmp -lomp # additional flags
You should get the binary file by just type make with a "correct" Makefile.
(If you see something like "cant find omp.h", add -I/usr/local/opt/libomp/include to the CXXFLAGS.)
For the Question
The error message in the updated question description
make: *** No rule to make target omp_hello.c', needed by omp_hello'. Stop.
is telling us that the file omp_hello.c is missing. The Makefile is written to compile the source code omp_hello.c to an executable binary file omp_hello. If I have the C source file (omp_hello.c), the Makefile will allow me to compile by just typing
make
instead of
/usr/local/opt/llvm/bin/clang \
-I/usr/local/opt/llvm/include -fopenmp \
-L/usr/local/opt/llvm/lib \
omp_hello.c -o omp_hello
This is just a normal compile process, it has nothing to do with Python. The error message is saying the source code to be compiled (omp_hello.c) is missing.
It looks like this is a small project with custom Makefile. Normally you compile the code with just make. The error you got seems to suggest the lack of llvm. You may want to try install llvm following this answer.
Usually it comes to running brew install <your C++ package> or downloading the source code to some directory and running a set of commands:
./configure
make
make install
While usually it works, some packages can not be installed on Mac since their maintainers did not prepare configuration for Mac.

Py_Initialize undefined reference

I want to add python functions in C++ code.
I made a GUI in gtk (on the Raspberry PI) and now I want to work with a camera module which is easy to handle in python. (I want to start a video directly when I push a button.)
So I included the file Python.h
#include <python3.4m/Python.h>
#include <python3.4m/pythonrun.h>
then I thought it should work, but when I try to compile Py_Initialize()
I get the error:
undefined reference to Py_Initialize.
I think this is strange because, when I type in, there came the selection for Py_Initialize.
In terms of headers you should be fine, since it compiled but failed at linking.
Now you need to link against the Python libraries. The way this is done largely depends on what toolchain you are using.
Maybe you can see my answer in another question:
if with python 3.x installed, maybe this command can work:
g++ hw.cpp `/usr/bin/python3-config --cflags` `/usr/python3-config --ldflags`
By the way, you should check you gcc and python version.
As I know, if gcc version is 5.4 and python version is 3.7, it doesn't work.(python 3.5 >is work)
When you run /usr/bin/python3-config --cflags, in fact, it is the compile option.
Set the python include folder and it static lib on gcc command line and put the python dynamic lib on LD_LIBRARY_PATH. Before Py_Initialize(), do not forget to set python home with Py_SetPythonHome(). These steps must be sufficient for your code compile and run.

Building VRPN server with Python 3.4 64-bit on Windows

I'm trying to build a VRPN server with Python3 flag using Python 3.4 64-bit on Windows 7 64-bit but there seems to be a problem. I need this for BlenderVR software.
This is my procedure:
1) I use CMake to create makefiles (I'm using 3.4.0 version but I've also tried different ones). I do it with this command (those flags should be there but the result seems to be the same without them anyway):
cmake -G"MinGW Makefiles" -HD:\My\BlenderVR\plugins\vrpn
-BD:\My\BlenderVR\plugins\cmake -DVRPN_BUILD_PYTHON=OFF -DVRPN_BUILD_PYTHON_HANDCODED_2X=OFF -DVRPN_BUILD_PYTHON_HANDCODED_3X=ON
I used to add those flags as well but it seems that it can find Python without them
-DPYTHON_INCLUDE_DIR=D:\My\BlenderVR\Required\Python3\include
-DPYTHON_LIBRARY=D:\My\BlenderVR\Required\Python3\libs\python34.lib
Python is correctly found and this operation doesn't throw any error.
2) Then I use mingw32-make.exe to build it and I get this error:
[ 90%] Linking CXX shared module vrpn.pyd D:/My/BlenderVR/Required/Python3/libs/python34.lib: error adding
symbols: File f ormat not recognized collect2.exe: error: ld
returned 1 exit status
python\CMakeFiles\vrpn-python.dir\build.make:505: recipe for
target 'python/vrpn .pyd' failed mingw32-make[2]: * * *
[python/vrpn.pyd] Error 1 CMakeFiles\Makefile2:3247: recipe for
target 'python/CMakeFiles/vrpn-python.dir/ all' failed
mingw32-make[1]: * * * [python/CMakeFiles/vrpn-python.dir/all]
Error 2 Makefile:159: recipe for target 'all' failed
mingw32-make: [all] Error 2
vprn.pyd is the crucial thing for my future work.
I figured out that it needs libpython34.a file (probably). When I created it and copied to Python3/libs folder it worked and finished without errors but the crated vprn.pyd didn't worked as it should.
What I need is to get import vrpn to work with this simple test in python (appending path where vrpn.pyd was build):
import sys
sys.path.append('D:/My/BlenderVR/plugins/cmake/python')
import vrpn
It lags my whole computer for a while and then pops out that Python has stop working.
I suspect that problem is in the libpython34.a file that I created doing this:
gendef python34.dll (in Windows/System32)
dlltool -D python34.dll -d python34.def -l libpython34.a
I don't how else should I get the libpython file. I've tried various versions of CMake and MinGW (like MinGWPy, TDM, w64) with many CMake flags. I was able to make it work using 32-bit Python but I need 64-bit version otherwise it is not working with BlenderVR enviroment.
I know this is very specific problem and probably kind of confusing at first but I didn't know how else to put it. I'll be glad for anything that could help. Thank you.
mingwpy should be installed with pip (until it is officially released at PYPI):
pip install -i https://pypi.anaconda.org/carlkl/simple mingwpy
all necessary import files are atomatically copied into the python\libs folder.
If python\Scripts is in the PATH it should work out of the box.
You have to make sure, that Blender Python is equiped with two import files
D:\My\BlenderVR\Required\Python3\libs\libpython\libpython34.dll.a
D:\My\BlenderVR\Required\Python3\libs\libpython\libmsvcr100.a

python c++ extension: symbol not defined error

I have a working c++ code that I want to wrap into a python module on Windows XP and Python 2.7. I have never done this before, so I looked into swig and distutils.
I created an interface file and a setup.py and compiled using
python setup.py build_ext -c mingw32
The script creates a module_wrap.cpp from my module.i and module.cpp file, and then creates a module_wrap.o and a module.o. The creation of module.o creates a bunch of Warnings for unused variables and deprecated char*, but it seems to work. Because the C++-code is not mine, I don't really want to get into these right now.
The last step is executing
g++.exe -shared -s build\temp.win32-2.7\Release\module_wrap.o build\temp.win32-2.7\Release\module.o build\temp.win32-2.7\Release\_module.def -LC:\Programme\Python27\libs -LC:\Programme\Python27\PCbuild -lpython27 -o build\lib.win32-2.7\_module.pyd
I get
Cannot export init_module: symbol not defined
error: command 'g++' failed with exit status 1
I googled a lot to this now, and I just can not find a solution to this problem. The previously created _module.def seems to try to export this init since it contains
LIBRARY _module.pyd
EXPORTS
init_module
Obviously this doesn't work, but I have no idea why. Can anyone help me out here?
I figured it out. The problem was the (not posted) interface file module.i for swig. There I named the module %module usemodule, whereas in the setup.py i named the module name=module. This way swig created an init_function, that did not match the name the created module was expecting it. In the end: just a typo...
Thanks for your support nevertheless!

How to install twisted 10.2.0 with pypy 1.4.1?

I can't install twisted 10.2.0 to my ubuntu box. After download twisted source I issue the command: pypy setup.py install. After that I receive the error:
building 'twisted.runner.portmap' extension
creating build/temp.linux-x86_64-2.5
creating build/temp.linux-x86_64-2.5/twisted
creating build/temp.linux-x86_64-2.5/twisted/runner
cc -I/builds/pypy-1.4.1-linux64/include -c twisted/runner/portmap.c -o build/temp.linux-x86_64-2.5/twisted/runner/portmap.o
cc -shared build/temp.linux-x86_64-2.5/twisted/runner/portmap.o -o build/lib.linux-x86_64-2.5/twisted/runner/portmap.pypy-14.so
/usr/bin/ld.bfd.real: build/temp.linux-x86_64-2.5/twisted/runner/portmap.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
build/temp.linux-x86_64-2.5/twisted/runner/portmap.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
error: command 'cc' failed with exit status 1
So, how can I fix it? Thanks.
That's a "bug" in twisted build. It might be or might not be a bug - twisted does not consider it's C extensions optional, although they're far from necessary for the most part. Twisted would still work if you just point PYTHONPATH to where it is situated, but without those C extensions.
You may be able to skip building extensions by commenting off the following line in setup.py (73):
conditionalExtensions = getExtensions(),
This should avoid collecting and trying to build extensions scattered in the directory tree.
As for now Twisted trunk could be compiled OK with PyPy (I used 1.6.1-dev0).
However in order to install Twisted you'd need to apply a patch to zipfile.py that I have submitted to stdlib.
zipfile.py 2.7.1+ leaks file descriptors because of CPython-oriented programming style used in zipfile.py
Therefore installation fails under PyPy that uses other GC model than CPython does.

Categories