Currently, I am trying to build Ipopt linking against openblas. I downloaded the openblas source and did make in the parent directory.
The configure script of Ipopt has several options to link against blas:
I tried ./configure --with-blas="-L/home/moritz/build/CoinIpopt_test/ThirdParty/OpenBLAS-0.2.14/libopenblas.so"
but I do get the error
checking whether user supplied BLASLIB="-L/home/moritz/build/CoinIpopt_test/ThirdParty/OpenBLAS-0.2.14/libopenblas.so" works... no
configure: error: user supplied BLAS library "-L/home/moritz/build/CoinIpopt_test/ThirdParty/OpenBLAS-0.2.14/libopenblas.so" does not work
Any tips how to achieve what I want ? Finally, I would like to make a conda package. I do have installed openblas with anaconda. But I do get the same error message if I link against the installed libopenblas.so
Managed to get it work. I had to install openblas to a directory of my choice by
make install PREFIX=/home/....../
aferwards I compiled Ipopt using
./configure --with-blas-incdir="-I/home/.../openblas/include/" --with-blas-lib="-L/home/.../openblas/lib/"
Related
I'm trying to build a local version of Numpy from source against BLIS (for BLAS and CBLAS) and against OpenBLAS for LAPACK.
I started with building BLIS locally for zen3 with CBLAS enabled, like so:
./configure --enable-threading=openmp --enable-cblas --prefix=$HOME/blis zen3
then ran the tests (which all passed) and ran make install. I made sure all relevant files are in the $HOME/blis library (see attached screenshot).
I also built openBLAS locally, no special configs there.
Afterwards, I modified numpy's site.cfg to configure openBLAS and blis folders' accordingly:
[blis]
libraries = blis
library_dirs = /home/or/blis/lib/
include_dirs = /home/or/blis/include/blis
runtime_library_dirs = /home/or/blis/lib/
[openblas]
libraries = openblas
library_dirs = /opt/OpenBLAS/lib
include_dirs = /opt/OpenBLAS/include
runtime_library_dirs = /opt/OpenBLAS/lib
I continued by building and installing numpy with:
NPY_BLAS_ORDER=blis NPY_LAPACK_ORDER=openblas NPY_CBLAS_LIBS= python ./numpy/setup.py build -j 32
Note that NPY_CBLAS_LIBS is empty as numpy's build docs say to do so if CBLAS is included in the BLIS library, which it is.
Then, importing numpy resulting in:
Original error was: /home/or/.pyenv/versions/3.9.6/lib/python3.9/site-packages/numpy-1.24.0.dev0+998.g6a5086c9b-py3.9-linux-x86_64.egg/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so: undefined symbol: cblas_sgemm
I'm clueless at this point as I couldn't find anything online about this specific case.
Installing numpy from pip (which comes built with openblas) can be imported successfully.
Update 1:
While reading make install logs, I found out that it couldn't find my BLIS library files at the location, even though the files are in the specified path. I also tried to recompile and install BLIS in various paths and reconfigure numpy before compiling it, but got the same result.
When I downloaded a pre-compiled version of BLIS from AMD's website, numpy seems to get it, but this isn't the recommended way to go because I'm missing optimizations for Zen3.
I am attempting to install from a Python setup. I had this working after a month of hell-ish attempts on 18.04, and now need it on 20.04.
The setup.py while I am using is on this repository ( https://github.com/flashlight/flashlight/tree/main/bindings/python ).
I am using Ubuntu 20.04, fresh install sans a few dependencies I know the system needs.
I have Python3.10 installed locally.
During the install at steps
cd bindings/python
python3.10 setup.py install
I hit the error
CMake Error at cmake/FindMKL.cmake:400. MKL library not found. Please specify the library location by appending the root directory of the MKL installation to the environment variable CMAKE_PREFIX_PATH
Which is weird for a variety of reasons.
I did a sudo apt-get install mkl-devel which took, but also
The setup script claims it defaults to MKL off.
Right prior to this error, I do get a warning.
Cmake Warning at cmake/FindMKL.cmake:387. MKL libraries files are found, but MKL header files are not. You can get them by 'pip install mkl-devel' (I did this btw) if using pip. If build fails with header files available on the system, please make sure that CMake will search the directory containing them by setting CMAKE_INCLUDE_PATH
That last part I did not do, primarily because I'm not sure how to tell where pip install mkl-devel would have put those header files.
That said, when I do a find / "*mkl*" I notice files in two primary locations
.h files : /home/myusername/.local/include
.so files: /home/myusername/.local/lib
So I set the following environment variables in my terminal
LIBRARY_PATH=/home/myusername/.local/lib
LD_LIBRARY_PATH=/home/myusername/.local/lib
KENLM_ROOT=/home/myusername/Flashlight/kenlm
USE_MKL=0
CMAKE_INCLUDE_PATH=/home/myusername/.local/include/
CMAKE_PREFIX_PATH=/home/myusername/.local/lib
However when I try to install again, the process still fails with both of the MKL issues above (Warning and Error)
I'm baffled at what I am supposed to be pointing where to get this to succeed at this point.
For installing dlib, I followed this tutorial : http://www.pyimagesearch.com/2017/03/27/how-to-install-dlib/.
I am on Mac OS X 10.12.5 and using Python 3.5.
I run
$ brew install cmake
$ brew install boost
$ brew install boost-python --with-python3
It works without any error.
But when I try to install dlib with pip install dlib. I have an error :
The C compiler
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc"
is not able to compile a simple test program.
error: cmake configuration failed
ld: can't map file, errno=22 file '/usr/local/opt/qt/lib' for architecture x86_64
For the full error, please see on this link (doesn't want to paste the full error) :
https://gist.github.com/alexattia/3e98685310d90b65031db640d3ea716a
After retracing the error, when I tried to make dlib manually, I have this :
Linking C executable cmTC_05e45
/usr/local/Cellar/cmake/3.8.2/bin/cmake -E cmake_link_script
CMakeFiles/cmTC_05e45.dir/link.txt --verbose=1
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-Wl,-search_paths_first -Wl,-headerpad_max_install_names
/usr/local/opt/qt/lib CMakeFiles/cmTC_05e45.dir/testCCompiler.c.o -o
cmTC_05e45
For the full trace expand : https://gist.github.com/alexattia/1e54ffb87c9eb4c811033f5cadd90331
I reinstalled XCode (from Apple Store) and CMake (3.8.2 from the downloaded page), I even installed Qt Creator to have a clean version of Qt, but I still have the same error.
I tried to install it with conda but after the installation, I still don't have the module in python.
Thank you very much for any help.
You commented:
Indeed, in my .bash_profile, I have export LDFLAGS="/usr/local/opt/qt/lib",
export CPPFLAGS="/usr/local/opt/qt/include", export PATH="/usr/local/opt/qt/bin:$PATH".
But even while commenting it, I still have the same error
Neither of your assignments to LDFLAGS or CPPFLAGS makes sense, and the
first one is the cause the linker failure that concerns you.
The value of the environment variable LDFLAGS, if set, is interpreted by your build system
as linkage options. Likewise The value of the environment variable
CPPFLAGS, if set, is interpreted as preprocessor options.
/usr/local/opt/qt/lib is not a linkage option and /usr/local/opt/qt/include
is not a preprocessor option. These are simply directory names. Any argument that
you pass to the linker (or preprocessor, or compiler) that is not an option is
interpeted by the tool as an input file. Thus you have led the linker to believe
that /usr/local/opt/qt/lib is an input file to your linkage.
ld: can't map file, errno=22 file '/usr/local/opt/qt/lib' for architecture x86_64
is what the linker says when it discovers that /usr/local/opt/qt/lib is not
a file at all.
Presumably, you wish to instruct the linker that /usr/local/opt/qt/lib is
a directory in which it should search for libraries required by your linkage.
The linkage option that expresses that intent is:
-L/usr/local/opt/qt/lib
Here are the GCC options for linking
Similarly you intend to instruct that preprocessor that /usr/local/opt/qt/include
is a directory in which it should search for header files. The preprocessor
option to express that is:
-I/usr/local/opt/qt/include
Here are the GCC options for preprocessing
It is abnormal and inadvisable to specify compilation or linkage options
in your bash login profile, as you are doing. Specify such options in the
build system's input files (makefile, cmakelists file or similar), or as arguments to
the build system's configuration. But if you insist on specifying them in
your bash login profile, then you should specify:
LDFLAGS=-L/usr/local/opt/qt/lib
CPPFLAGS=-I/usr/local/opt/qt/include
And once you have made these environment settings in your bash_profile they will
only take effect in new login shells.
I had a similar issue but found out it was due to boost.
Try this.
brew uninstall boost-python
brew uninstall boost
brew install boost-python --with-python3 --without-python
pip3 install dlib
I am trying to run theano on ubuntu which requires libatlas.
I have already installed libatlas but I can find it in /usr/lib/atlas-base
I have also copied all of the files to a new folder called /atlas:
cp -a /usr/lib/atlas-base/* /usr/lib/atlas
But still, when I run the python code I see:
/usr/bin/ld: cannot find -latlas
/usr/bin/ld: cannot find -l477blas
/usr/bin/ld: cannot find -lcblas
I also tried adding to environment variables but didn't work:
set LIBPATH = [BUILD_LIB_DIR, /usr/lib/atlas]
Also I tried adding the path path to ld file:
/usr/lib/atlas
or
/usr/lib/atlas-base
None of them worked and I still see the error running the Python code.
To change how Theano link to BLAS, you need to use Theano flags[1]. They can be set with an environment variable THEANO_FLAGS or with a configuration file.
How did you told Theano to use atlas? If you just installed the atlas packages, it won't work. You need to install the libatlas-dev pacakge as per this Theano installation instruction for Ubuntu[2]
A last point, we don't recommand atlas, especially for Ubuntu. OpenBLAS is packaged for Unbuntu and is faster. See [2] for detail on how to installed them. You will need to remove atlas before installing openblas, otherwise, there will be conflict.
[1]http://www.deeplearning.net/software/theano/library/config.html#envvar-THEANO_FLAGS
[2]http://www.deeplearning.net/software/theano/install_ubuntu.html#install-ubuntu
I have looked for an easy way to install/compile Numpy with OpenBLAS but didn't find an easy answer. All the documentation I have seen takes too much knowledge as granted for someone like me who is not used to compile software.
There are two packages in Ubuntu related to OpenBLAS : libopenblas-base and libopenblas-dev.
Once they are installed, what should I do to install Numpy again with them?
Thanks!
Note that when these OpenBLAS packages are installed, Numpy doesn't work anymore: it can't be imported: ImportError: /usr/lib/liblapack.so.3gf: undefined symbol: ATL_chemv.
The problem occurs as well when installing Theano with their website instructions for Ubuntu.
This was noticed here already.
Run sudo update-alternatives --all and set liblapack.so.3gf to /usr/lib/lapack/liblapack.so.3gf
To add to the accepted answer (of using update-alternatives), the reason for this is because OpenBlas is not compatible with the Atlas version of Lapack. For each of the Blas and Lapack versions:
Default Blas + Default Lapack => OK
OpenBlas + Default Lapack => OK
Atlas-Blas + Default Lapack => OK
Atlas-Blas + Atlas-Lapack => OK
OpenBlas + Atlas-Lapack => ERROR! (The following case here.)
This is from both personal experience (with the exact same issue) and realizing why such a combination wasn't mentioned in this comparison blog.
By the way, you can just find the necessary files in /etc/alternatives/, usually with a filename starting with lib*. For each one do sudo update-alternatives --config <filename>. For example, do to following:
sudo update-alternatives --config libblas.so
sudo update-alternatives --config libblas.so.3
to change the Blas version.
Consider using EasyBuild (http://hpcugent.github.io/easybuild/), an open-source framework for building and installing software.
It allows you to (very easily) build and install (scientific) software with various compiler, and using different BLAS libraries (ATLAS, OpenBLAS, ACML, Intel MKL, ...).
Once you install EasyBuild (pro tip: use the bootstrapping procedure described at https://github.com/hpcugent/easybuild/wiki/Bootstrapping-EasyBuild), it boils down to running a single command, something like:
eb numpy-1.6.2-goolf-1.4.10-Python-2.7.3.eb -ldr
This will first build and install of full compiler toolchain (goolf: GCC+OpenBLAS+OpenMPI+LAPACK+FFTW), and subsequently build Python and numpy with that toolchain. And all that while you're getting lunch. ;-)
Disclaimer: I'm one of the EasyBuild developers.