Compiler failure when compiling SciPy with MinGW+GCC - python

I am running some python code where some code needs gcc compilation (dont ask me I dont have a clue). I was getting the error that gcc not recognised. I installed Mingw and gcc compiler, so that was sorted. But now the assembler is creating a very large file name and getting into error. see the error below:
Assembler messages:
Fatal error: can't create c:\users\kd1234\appdata\local\temp\scipy-as07487-7op0cx\python27_intermediate\compiler_6cb9c52cab22cd58c3b2a33f029b68476828f4189dc8dd305efd20ec06666d43\Release\users\as07487\appdata\local\temp\kd1234\python27_compiled\sc_ed5391b748bc47781f90305835197df10c5f33c0bbba9a3f5660ab3c277c2b50657.o: No such file or directory
I looked at it and it looks like this file name generated by the compiler is too large for windows thus cant create it and getting into error.Has anyone got some idea how to sort this one out!
Cheers

That path length is 290 characters, which does indeed exceed the limit of 260 characters imposed by most Windows APIs.
Part of the path bloat appears to be due to %TEMP%; try setting %TEMP% to C:\ before running the script and see if that makes it work.

Related

How to install C++ dependencies "from source" for a Python package on Mac OS?

There is a Github repo containing Python "bindings" for a C++ library that I am interested in playing with. The README has abundant information about how to install the C++ library on Linux like machines, but no information about how to do so with a mac OS.
I have also opened up an issue requesting the README installation instructions include mac OS-specific installs in addition to linux. There hasn't been any activity on that issue.
Here are the two repos:
(Python) https://github.com/asiffer/python3-libspot
(C++) https://github.com/asiffer/libspot
Since the C++ package isn't available for installing via Brew/pip/anaconda, I'm not sure how to get going.
What I've Tried:
I have tried ./configure, and make. There is no ./configure file.
To address the lack of ./configure, read about a tool called autoconf which supposedly generates ./configure for you. I installed it with brew, but am not sure what arguments to pass it. These docs were pretty hard to understand: https://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/Making-configure-Scripts.html
Just using make results in the error clang: error: unsupported option '-fopenmp'That sent me down a whole different rabbit hole which had me adding lines to the Makefile:
CPP = /usr/local/opt/llvm/bin/clang
CPPFLAGS = -I/usr/local/opt/llvm/include -fopenmp
LDFLAGS = -L/usr/local/opt/llvm/lib
omp_hello: omp_hello.c
$(CPP) $(CPPFLAGS) $^ -o $# $(LDFLAGS)
That felt dangerous because I have no idea what any of that stuff means. Plus it resulted in a new error: *** missing separator. Stop.
So then I read that's probably due to using "soft" tabs instead of "hard" tabs which can be identified using cat -e -t -v makefile_name. I found the one line where a "hard" tab was missing (the indented line above) and inserted it. This resulted in a new error:
make: *** No rule to make target `omp_hello.c', needed by `omp_hello'. Stop.
Next, following the advice of Yang Yushi and his follow on comments, I changed lines 39 and 40 according to his answer, plus added the locations of some additional files to the CXXFLAGS variable:
-I//opt/homebrew/Cellar/libomp/11.0.1/include
-L/opt/homebrew/Cellar/libomp/11.0.1/lib
And this got me a little further. Next, OSX didn't like where this script was trying to install, as explained by this answer. So I changed these two lines in the makefile which seemed to dictate install location:
INSTALL_HEAD_DIR = $(DESTDIR)/usr/include/libspot
INSTALL_LIB_DIR = $(DESTDIR)/usr/lib
to
INSTALL_HEAD_DIR = $(DESTDIR)/usr/local/include/libspot
INSTALL_LIB_DIR = $(DESTDIR)/usr/local/lib
And that indeed got me a little farther. Next I ran into an error complaining about the flat -t at these lines in the makefile:
#install -t $(INSTALL_LIB_DIR) $(LIB_DIR)/*.so
#install -t $(INSTALL_HEAD_DIR) $(INC_DIR)/*.h
So I deleted those flags, which then resulted in this error:
Checking the headers installation directory (/usr/local/include/libspot)
Checking the library installation directory (/usr/local/lib)
Installing the shared library (libspot.so)
install: /usr/local/lib: Inappropriate file type or format
For which I can find no reading material and have no clue how to fix. Any further assistance appreciated.
Here's a list of SO and other resources I've perused trying to answer this question:
Enable OpenMP support in clang in Mac OS X (sierra & Mojave)
makefile error: make: *** No rule to make target `omp.h' ; with OpenMP
makefile:4: *** missing separator. Stop
http://www.idryman.org/blog/2016/03/10/autoconf-tutorial-1/
https://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/Making-configure-Scripts.html
https://developer.gnome.org/anjuta-build-tutorial/stable/create-autotools.html.en
My Question
How do I proceed.
If you know how to do this, could you also include a brief explanation of the concepts behind each step? I'd be happy to learn a little instead of just copying and pasting commands in the right order.
Compile the C++ source code with Apple Clang
I downloaded the prjoect (libspot) and successfully compiled it on my Mac. I change two lines (39 and 40) in the Makefile to make it work. (Following this answer)
CC = clang++ # change from g++ to default Apple clang
CXXFLAGS = -std=c++11 -Wall -pedantic -Xpreprocessor -fopenmp -lomp # additional flags
You should get the binary file by just type make with a "correct" Makefile.
(If you see something like "cant find omp.h", add -I/usr/local/opt/libomp/include to the CXXFLAGS.)
For the Question
The error message in the updated question description
make: *** No rule to make target omp_hello.c', needed by omp_hello'. Stop.
is telling us that the file omp_hello.c is missing. The Makefile is written to compile the source code omp_hello.c to an executable binary file omp_hello. If I have the C source file (omp_hello.c), the Makefile will allow me to compile by just typing
make
instead of
/usr/local/opt/llvm/bin/clang \
-I/usr/local/opt/llvm/include -fopenmp \
-L/usr/local/opt/llvm/lib \
omp_hello.c -o omp_hello
This is just a normal compile process, it has nothing to do with Python. The error message is saying the source code to be compiled (omp_hello.c) is missing.
It looks like this is a small project with custom Makefile. Normally you compile the code with just make. The error you got seems to suggest the lack of llvm. You may want to try install llvm following this answer.
Usually it comes to running brew install <your C++ package> or downloading the source code to some directory and running a set of commands:
./configure
make
make install
While usually it works, some packages can not be installed on Mac since their maintainers did not prepare configuration for Mac.

Cstdint Missing Error When Installing Pydaedalus with PiP

I am working on an application which involves route finding (a completely different subject), but for testing I need example mazes to test on. A colleague suggested I use pydaedalus to generate large scale mazes in the format I need. I am using the following code to try and install the module:
$pip3.6 install pydaedalus
This returns the following error:
-Wno-error=format-security
In file included from daedalus/_maze.cpp:467:
In file included from daedalus/wrapper.h:8:
daedalus/src/util.h:31:10: fatal error: 'cstdint' file not found
#include <cstdint>
^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
I have done some research and have found nothing which addresses this. I have also done some (limited) C++ development using cstdint, which has always worked.
I came across this question, but it appears to address a separate issue.
I am developing in OSX 10.10.5
Any help that you can provide is much appreciated!
These compile errors are down to daedalus's requirement of the C++11 standard, which is sometimes a bit tricky to get working on Mac OS X. One idea might be to check to make sure your Xcode is completely up to date.
The page you linked also suggests to try linking against clang's standard library instead of the GCC standard library. I'm not sure if this will work, or if it will give you linking errors on build or when you import daedalus into python, but you could give it a shot anyway:
CFLAGS='-stdlib=libc++' pip3.6 install pydaedalus
Another idea would be to encourage pip to use the clang++ frontend, which your link also suggests might help. You should be able to set this with the environment variable CXX (or, just possibly, CC).
CXX=clang++ pip3.6 install pydaedalus
Try various combinations of those environment settings (e.g., CXX and CFLAGS), and hopefully something will work eventually.

Error when converting C code to Web Assembly

I have successfully installed Emscripten and have it running on an Ubuntu 16.04 virtual machine. I have also successfully converted a helloworld.c file to web assembly. Currently, I am attempting to convert python to web assembly with emscripten. The issue is that emscripten does not support python currently, so as a work around I have attempted to convert the python code to C with Cython, which I successfully did. Though I am getting an error when attempting to convert the cython c file to Web assembly. Here is the console log:
$emcc pony_gp.c -o pony_gp.html
In file included from pony_gp.c:11:
In file included from /usr/include/python2.7/Python.h:58:
/usr/include/python2.7/pyport.h:886:2: error: "LONG_BIT definition appears
wrong for platform (bad gcc/glibc config?)."
#error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."
ERROR:root:compiler frontend failed to generate LLVM bitcode, halting
According to pyport.h, this error is generated because in some 32 bit systems LONG_BIT is defined incorrectly as 64 when it should be 32. I have tried commenting out this line, but this only allowed the program to silently run, in the end without producing any web assembly code, only html and javascript.
I have read here, that the issue is because "cmake is picking up one version of the python dylib and a separate version of python for the headers". This makes sense as I recently downgraded from Python 2.7.13-1 to Python 2.7.11-1 because Python 2.7.13-1 was not compatible with python-dev packages. Though, I don't know how I would fix this.
Does anyone have an idea on what to do?
While not a full answer, you should be able to compile pony_gp.c directly to LLVM (.ll) using clang, preferably the same clang provided with Emscripten, for example:
source ~/emsdk/emsdk_env.sh
cython hello.py
clang `python2-config --cflags` -S -emit-llvm hello.c
Then, the generated .ll file could be fed directly to Emscripten.
For producing a fully working Python -> WebAssembly you should probably also need to link against the Python runtime - you could use the one distributed with emcc which is already compiled into LLVM bytecode (.bc), emsdk/emscripten/incoming/tests/python/python.bc.
Also, this may be of help: https://github.com/dgym/cpython-emscripten

compiling Cython with MinGW - undefined reference PyExc

I'm trying to compile a simple code snippet from the book "Cython - A guide for Python programmers", and when i compile, i get the following error:
H:\Cython>python setup.py build_ext -i --compiler=mingw32
running build_ext
building 'fib' extension
C:\MinGW\bin\gcc.exe -mdll -O -Wall -IC:\Anaconda3\include -IC:\Anaconda3\include -c fib.c -o build\temp.win32-3.4\Release\fib.o
writing build\temp.win32-3.4\Release\fib.def
C:\MinGW\bin\gcc.exe -shared -s build\temp.win32-3.4\Release\fib.o build\temp.win32-3.4\Release\fib.def -LC:\Anaconda3\libs -LC:\Ana
conda3\PCbuild -lpython34 -lmsvcr100 -o H:\Cython\fib.pyd
build\temp.win32-3.4\Release\fib.o:fib.c:(.text+0xb6): undefined reference to `_imp__PyExc_TypeError'
build\temp.win32-3.4\Release\fib.o:fib.c:(.text+0xf3): undefined reference to `_imp__PyExc_TypeError'
build\temp.win32-3.4\Release\fib.o:fib.c:(.text+0x3cc): undefined reference to `_imp___PyThreadState_Current'
build\temp.win32-3.4\Release\fib.o:fib.c:(.text+0x857): undefined reference to `_imp__PyExc_NameError'
build\temp.win32-3.4\Release\fib.o:fib.c:(.text+0xa55): undefined reference to `_imp__PyExc_ImportError'
c:/mingw/bin/../lib/gcc/mingw32/4.8.1/../../../../mingw32/bin/ld.exe: build\temp.win32-3.4\Release\fib.o: bad reloc address 0x0 in s
ection `.data'
collect2.exe: error: ld returned 1 exit status
error: command 'C:\\MinGW\\bin\\gcc.exe' failed with exit status 1
H:\Cython>
setup.py:
from distutils.core import setup
from Cython.Build import cythonize
setup(ext_modules=cythonize('fib.pyx'))
fib.pyx:
def fib(int n):
cdef int i
cdef double a=0.0, b=1.0
for i in range(n):
a, b = a + b, a
return a
When I google this problem, others who have had the error had a mix between 32 and 64 bit stuff, I'm not doing that.
I sat down and looked the errors over again today, and found the issue.
The problem is that I used Anaconda rather than compiling everything from scratch myself - that means that some Cython components were compiled with MSVC. As above you can see that I'm trying to use MinGW to compile the Cython test script. Why mixing compilers like that doesn't work is outside my scope of knowledge, but it doesn't. Compiling my Cython test scripts with MSVC works.
(use Visual Studio C++ 2008/2010 for python 2.x/3.x respectively)
As for the reason that I attempted to use MinGW (against standard recommendation) was that my msiserver service had somehow broken (I'm on an old laptop, so i don't recall the reason), and was hoping to find a quick way out, rather than fixing the msiserver service.
The fix for the msiserver service is pretty unrelated to this question, but it was rather hard to find, so i figured I'd link and mirror it here:
http://www.vistax64.com/vista-installation-setup/96680-repair-windows-installer-service-vista-all-versions.html
For all those unfortunate souls searching and Googling for how to
repair the Windows Installer Service, I have some info for you. A
couple days ago I tried to uninstall one of my apps and stalled on an
error "Windows Installer Service cannot be accessed". After many
trials and errors trying to fix this issue, I stumbled upon a new fix
for this issue that has worked in all these situations where the
Windows Installer Service will not manually start and in essense, not
allow install or uninstall tasks to complete.
Here's the easy steps:
1. Go to a Windows Vista (Any Version) computer that has the Windows
Installer service running correctly and run regedit(Start-Run-Regedit)
2. Go to the location
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\msiserver
3. Right click on this key and select "Export" and save the key to a Flash
Drive or other.
4. Run sfc /scannow on the damaged Vista computer - you won't need the
install disk as it goes to backup files on your HD. Do not reboot when
complete
5. Double click saved .reg file from working machine and import registry
settings into damaged Vista computer.
6. Now reboot and try to install/uninstall
If many of you have success with this method, please post this fix
around the WWW as I went through over 1000 links with users having the
same problem and not being able to solve it. Shame on Microsoft, very
sloppy. It would have been so nice if Microsoft released Windows
Installer 4.0 as a standalone installation with Vista's release so I
could have repaired it, most users have been doing fresh installs to
fix this. Get your act together Microsoft!!!!
The CAT
Thank you, the cat.

f2py does not find any compiler

I have the NAG Fortran compiler installed. I can compile Fortran code by calling nagfor -o helloworld helloworld.f90. If I run f2py with f2py -c -m helloworld helloworld.f90 --fcompiler=nagfor nothing happens. Additionally, if I just run f2py nothing happens. f2py --help-fcompiler gives no output.
I have Windows 7 installed and use the the Anaconda Python distribution. Any idea how I should address this problem?
Following Ian`s comments and this post I managed to run f2py (unfortunately only with the GNU Fortran compiler).
I had to change line 337 in C:\Loopy\Lib\site-packages\numpy\distutils\fcompiler\gnu.py to:
pass #raise NotImplementedError("Only MS compiler supported with gfortran on win64")
Additionally I use C:\Loopy\Scripts\f2py.py.
It's unusual that you aren't seeing any error output at all.
That makes it sound like you're calling something else.
Make sure Anaconda's scripts directory on your path and that you don't have some sort of script in your current directory called f2py.
Depending on how you have your computer is set up to interpret file types, you may need to run something like python f2py.py with the rest of the arguments the same.
If you're using Anaconda, you should already have a copy of gfortran intsalled too.
If you want to use that instead, make sure Anaconda's bin directory is on your path.
Unless you have a very recent (1.10, currently in development) version of numpy, to use gfortran, you'll need to go to Anaconda/Lib/site-packages/numpy/distutils/fcompiler/gnu.py and comment out the lines (somewhere around line 330) that raise an error if you're on 64 bit windows.
Once you've done that, it should work fine.
Edit: judging by the old f2py docs and the current source, the proper fcompiler flag is --fcompiler=nag.
The compiler is specified by vendor, not by executable name.

Categories