I want to install scipy on a machine where I am not root following the instruction I found on this page but I meeting a BLAS not found error despite the fact that BLAS is install and "properly defined".
So I started installing the last python release (2.7.6) in a custom directoty ~/Tools/Python) which works fine. I compile then BLAST and LAPACK (3..5.0) following the instruction on this page. The compilation works fine and I put the respective .a files in some custom folders also (~Tools/BLAS with blas_LINUX.a and ~/TOOLS/LAPACK with liblapack.a and libtmglib.a). I then define two environment variables BLAS and LAPACK poiting to the directory above where the .a files are.
I then install numpy (1.8.1) using
python setup.py install --prefix=~/TOOLS/Python
and update my PYTHONPATH accordingly. Numpy works fine. However when shifting to the install of scipy (0.13.3), I get the following error:
numpy.distutils.system_info.BlasNotFoundError:
Blas (http://www.netlib.org/blas/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [blas]) or by setting
the BLAS environment variable.
I try various combinaisons like specifying BLAS just before the call to the setup function but still it doesn't work. However I don't understand as when I type
echo $BLAS
the BLAS path is outputed. So any idea of what went wrong?
Related
LINK : fatal error LNK1181: cannot open input file 'lapack.lib'
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.34.31933\\bin\\HostX86\\x64\\link.exe' failed with exit code 1181
I'm getting this error while installing cvxopt. I've set the directory for lapack lib in the setup.py file inside the cvxopt folder. However running the command
python setup.py install
generates this error. Can anyone help me with this?
If you use pip install cvxopt it will use a pre-built binary wheel package already.
If you want to build from source (as you are currently doing) then you need to have BLAS and LAPACK installed on your system
https://cvxopt.org/install/#building-and-installing-from-source
Required and optional software
The package requires version 3.x of Python, and building from source requires core binaries and header files and libraries for Python.
The installation requires BLAS and LAPACK. Using an architecture optimized implementation such as ATLAS, OpenBLAS, or MKL is recommended and gives a large performance improvement over reference implementations of the BLAS and LAPACK libraries.
The installation also requires SuiteSparse. We recommend linking against a shared SuiteSparse library. It is also possible to build the required components of SuiteSparse when building CVXOPT, but this requires the SuiteSparse source which is no longer included with CVXOPT and must be downloaded separately.
The following software libraries are optional.
The GNU Scientific Library GSL.
FFTW is a C library for discrete Fourier transforms.
GLPK is a linear programming package.
MOSEK version 9 is a commercial library of convex optimization solvers.
DSDP5.8 is a semidefinite programming solver.
I'm trying to build a local version of Numpy from source against BLIS (for BLAS and CBLAS) and against OpenBLAS for LAPACK.
I started with building BLIS locally for zen3 with CBLAS enabled, like so:
./configure --enable-threading=openmp --enable-cblas --prefix=$HOME/blis zen3
then ran the tests (which all passed) and ran make install. I made sure all relevant files are in the $HOME/blis library (see attached screenshot).
I also built openBLAS locally, no special configs there.
Afterwards, I modified numpy's site.cfg to configure openBLAS and blis folders' accordingly:
[blis]
libraries = blis
library_dirs = /home/or/blis/lib/
include_dirs = /home/or/blis/include/blis
runtime_library_dirs = /home/or/blis/lib/
[openblas]
libraries = openblas
library_dirs = /opt/OpenBLAS/lib
include_dirs = /opt/OpenBLAS/include
runtime_library_dirs = /opt/OpenBLAS/lib
I continued by building and installing numpy with:
NPY_BLAS_ORDER=blis NPY_LAPACK_ORDER=openblas NPY_CBLAS_LIBS= python ./numpy/setup.py build -j 32
Note that NPY_CBLAS_LIBS is empty as numpy's build docs say to do so if CBLAS is included in the BLIS library, which it is.
Then, importing numpy resulting in:
Original error was: /home/or/.pyenv/versions/3.9.6/lib/python3.9/site-packages/numpy-1.24.0.dev0+998.g6a5086c9b-py3.9-linux-x86_64.egg/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so: undefined symbol: cblas_sgemm
I'm clueless at this point as I couldn't find anything online about this specific case.
Installing numpy from pip (which comes built with openblas) can be imported successfully.
Update 1:
While reading make install logs, I found out that it couldn't find my BLIS library files at the location, even though the files are in the specified path. I also tried to recompile and install BLIS in various paths and reconfigure numpy before compiling it, but got the same result.
When I downloaded a pre-compiled version of BLIS from AMD's website, numpy seems to get it, but this isn't the recommended way to go because I'm missing optimizations for Zen3.
I like to compile python 3.8.7 from scratch and add numpy, scipy.
Python compiles fine, numpy as well (disabled LAPACK=None, BLAS=None, ATLAS=None).
If i compile scipy i run into the following error:
File "scipy/linalg/setup.py", line 20, in configuration
raise NotFoundError('no lapack/blas resources found')
numpy.distutils.system_info.NotFoundError: no lapack/blas resources found
I intentionally don't want a LAPACK, BLAS, ATLAS dependency!
Is there a way of disabling these libraries?
setenv PYHOME "/path/to/my/python"
FFI_VERSION "3.3"
PYTHON_VERSION "3.8.7"
CYTHON_VERSION "0.29.21"
NUMPY_VERSION "1.19.5"
PYBIND11_VERSION "2.6.1"
SCIPY_VERSION "1.5.4"
setenv BLAS "None"
setenv ATLAS "None"
setenv LAPACK "None"
setenv NPY_BLAS_ORDER ""
setenv NPY_LAPACK_ORDER ""
${PYHOME}/bin/python3 setup.py build
${PYHOME}/bin/python3 setup.py install --prefix=${PYHOME}
The documentation says:
Before building, you will also need to install packages that NumPy and SciPy depend on
BLAS and LAPACK libraries (optional but strongly recommended for NumPy, required for SciPy): typically ATLAS + OpenBLAS, or MKL.
Thank you!
setenv PYTHON_VERSION "3.6.4"
setenv CYTHON_VERSION "0.24.1"
setenv NUMPY_VERSION "1.11.1"
setenv SCIPY_VERSION "0.18.1"
this version/setup works without LAPACK etc. disabled by the LAPACK,BLAS,ATLAS=None switches. So this hard requirement of the Lapack dependence of SciPy was introduced later...
Currently, I am trying to build Ipopt linking against openblas. I downloaded the openblas source and did make in the parent directory.
The configure script of Ipopt has several options to link against blas:
I tried ./configure --with-blas="-L/home/moritz/build/CoinIpopt_test/ThirdParty/OpenBLAS-0.2.14/libopenblas.so"
but I do get the error
checking whether user supplied BLASLIB="-L/home/moritz/build/CoinIpopt_test/ThirdParty/OpenBLAS-0.2.14/libopenblas.so" works... no
configure: error: user supplied BLAS library "-L/home/moritz/build/CoinIpopt_test/ThirdParty/OpenBLAS-0.2.14/libopenblas.so" does not work
Any tips how to achieve what I want ? Finally, I would like to make a conda package. I do have installed openblas with anaconda. But I do get the same error message if I link against the installed libopenblas.so
Managed to get it work. I had to install openblas to a directory of my choice by
make install PREFIX=/home/....../
aferwards I compiled Ipopt using
./configure --with-blas-incdir="-I/home/.../openblas/include/" --with-blas-lib="-L/home/.../openblas/lib/"
I am trying to run theano on ubuntu which requires libatlas.
I have already installed libatlas but I can find it in /usr/lib/atlas-base
I have also copied all of the files to a new folder called /atlas:
cp -a /usr/lib/atlas-base/* /usr/lib/atlas
But still, when I run the python code I see:
/usr/bin/ld: cannot find -latlas
/usr/bin/ld: cannot find -l477blas
/usr/bin/ld: cannot find -lcblas
I also tried adding to environment variables but didn't work:
set LIBPATH = [BUILD_LIB_DIR, /usr/lib/atlas]
Also I tried adding the path path to ld file:
/usr/lib/atlas
or
/usr/lib/atlas-base
None of them worked and I still see the error running the Python code.
To change how Theano link to BLAS, you need to use Theano flags[1]. They can be set with an environment variable THEANO_FLAGS or with a configuration file.
How did you told Theano to use atlas? If you just installed the atlas packages, it won't work. You need to install the libatlas-dev pacakge as per this Theano installation instruction for Ubuntu[2]
A last point, we don't recommand atlas, especially for Ubuntu. OpenBLAS is packaged for Unbuntu and is faster. See [2] for detail on how to installed them. You will need to remove atlas before installing openblas, otherwise, there will be conflict.
[1]http://www.deeplearning.net/software/theano/library/config.html#envvar-THEANO_FLAGS
[2]http://www.deeplearning.net/software/theano/install_ubuntu.html#install-ubuntu