We Keep Coding

Importing Numpy fails after building from source against amd blis

I'm trying to build a local version of Numpy from source against BLIS (for BLAS and CBLAS) and against OpenBLAS for LAPACK.
I started with building BLIS locally for zen3 with CBLAS enabled, like so:
./configure --enable-threading=openmp --enable-cblas --prefix=$HOME/blis zen3
then ran the tests (which all passed) and ran make install. I made sure all relevant files are in the $HOME/blis library (see attached screenshot).
I also built openBLAS locally, no special configs there.
Afterwards, I modified numpy's site.cfg to configure openBLAS and blis folders' accordingly:
[blis]
libraries = blis
library_dirs = /home/or/blis/lib/
include_dirs = /home/or/blis/include/blis
runtime_library_dirs = /home/or/blis/lib/
[openblas]
libraries = openblas
library_dirs = /opt/OpenBLAS/lib
include_dirs = /opt/OpenBLAS/include
runtime_library_dirs = /opt/OpenBLAS/lib
I continued by building and installing numpy with:
NPY_BLAS_ORDER=blis NPY_LAPACK_ORDER=openblas NPY_CBLAS_LIBS= python ./numpy/setup.py build -j 32
Note that NPY_CBLAS_LIBS is empty as numpy's build docs say to do so if CBLAS is included in the BLIS library, which it is.
Then, importing numpy resulting in:
Original error was: /home/or/.pyenv/versions/3.9.6/lib/python3.9/site-packages/numpy-1.24.0.dev0+998.g6a5086c9b-py3.9-linux-x86_64.egg/numpy/core/_multiarray_umath.cpython-39-x86_64-linux-gnu.so: undefined symbol: cblas_sgemm
I'm clueless at this point as I couldn't find anything online about this specific case.
Installing numpy from pip (which comes built with openblas) can be imported successfully.
Update 1:
While reading make install logs, I found out that it couldn't find my BLIS library files at the location, even though the files are in the specified path. I also tried to recompile and install BLIS in various paths and reconfigure numpy before compiling it, but got the same result.
When I downloaded a pre-compiled version of BLIS from AMD's website, numpy seems to get it, but this isn't the recommended way to go because I'm missing optimizations for Zen3.

How to build NumPy from source linked to Apple Accelerate framework?

It is my understanding that NumPy dropped support for using the Accelerate BLAS and LAPACK at version 1.20.0. According to the release notes for NumPy 1.21.1, these bugs have been resolved and building NumPy from source using the Accelerate framework on MacOS >= 11.3 is now possible again: https://numpy.org/doc/stable/release/1.21.0-notes.html, but I cannot find any documentation on how to do so. This seems like it would be an interesting thing to try and do because the Accelerate framework is supposed to be highly-optimized for M-series processors. I imagine the process is something like this:
Download numpy source code folder and navigate to this folder.
Make a site.cfg file that looks something like:
[DEFAULT]
library_dirs = /some/directory/
include_dirs = /some/other/directory/
[accelerate]
libraries = Accelerate, vecLib
Run python setup.py build
The problem is I do not know 1. what the variables library_dirs and include_dirs should be so that NumPy knows to use Accelerate BLAS and LAPACK and 2. if there are any other additional steps that need to be taken. If anyone knows how to do this or can provide any insight, it would be greatly appreciated.

I actually attempted this earlier today and these are the steps I used:
In the site.cfg file, put
[accelerate]
libraries = Accelerate, vecLib
Build with NPY_LAPACK_ORDER=accelerate python3 setup.py build
Install with pip3 install .
Afterwards, np.show_config() returned the following
blas_mkl_info:
NOT AVAILABLE
blis_info:
NOT AVAILABLE
openblas_info:
NOT AVAILABLE
accelerate_info:
extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers']
extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
blas_opt_info:
extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers']
extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
lapack_mkl_info:
NOT AVAILABLE
openblas_lapack_info:
NOT AVAILABLE
openblas_clapack_info:
NOT AVAILABLE
flame_info:
NOT AVAILABLE
lapack_opt_info:
extra_compile_args = ['-I/System/Library/Frameworks/vecLib.framework/Headers']
extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
Supported SIMD extensions in this NumPy install:
baseline = NEON,NEON_FP16,NEON_VFPV4,ASIMD
found = ASIMDHP,ASIMDDP
not found =
and my quick test suggest significant performance boost relative to OpenBlas.

No it doesn't have to be that complicated. I used these two commands and was able to install numpy with Apple Accelerate on Mac M1.
pip install cython pybind11
pip install --no-binary :all: --no-use-pep517 numpy
Reference: How to install SciPy on Apple Silicon (ARM / M1)

Installing PyTorch on Jetson Nano Ubuntu 18

I am trying to install PyTorch on Jetson Nano Ruining Ubuntu 1804. My reference is https://dev.to/evanilukhin/guide-to-install-pytorch-with-cuda-on-ubuntu-18-04-5217
When I try the following command this is what I get:
(my_env) crigano#crigano-desktop:~$ python3.8 -m pip install numpy ninja pyyaml mkl mkl-include setuptools cmake cffi typing
Collecting numpy
Using cached numpy-1.20.2-cp38-cp38-manylinux2014_aarch64.whl (12.7 MB)
Collecting ninja
Using cached ninja-1.10.0.post2.tar.gz (25 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting pyyaml
Using cached PyYAML-5.4.1-cp38-cp38-manylinux2014_aarch64.whl (818 kB)
ERROR: Could not find a version that satisfies the requirement mkl
ERROR: No matching distribution found for mkl

If you just want to use PyTorch on the bare-metal Jetson Nano, simply install it with NVIDIA's pre-compiled binary wheel. Other packages can be found in the Jetson Zoo.
MKL is developed by Intel "to optimize code for current and future generations of Intel® CPUs and GPUs." [PyPI]. Apparently it does run on other x86-based chips like AMD's (although Intel has historically intentionally crippled the library for non-Intel chips [Wikipedia]), but unsurprisingly Intel is not interested in supporting ARM devices and has not ported MKL to ARM architectures.
If your goal is to use MKL for math optimization in numpy, openblas is a working alternative for ARM. libopenblas-base:arm64 and libopenblas-dev:arm64 come pre-installed on NVIDIA's "L4T PyTorch" Docker images. You can confirm that numpy detects them with numpy.__config__.show(). This is what I get using numpy 1.12 in python 3.69 on the l4t-pytorch:r32.5.0-pth1.6-py3 image:
blas_mkl_info:
NOT AVAILABLE
blis_info:
NOT AVAILABLE
openblas_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/lib/aarch64-linux-gnu']
language = c
define_macros = [('HAVE_CBLAS', None)]
blas_opt_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/lib/aarch64-linux-gnu']
language = c
define_macros = [('HAVE_CBLAS', None)]
lapack_mkl_info:
NOT AVAILABLE
openblas_lapack_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/lib/aarch64-linux-gnu']
language = c
define_macros = [('HAVE_CBLAS', None)]
lapack_opt_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/lib/aarch64-linux-gnu']
language = c
define_macros = [('HAVE_CBLAS', None)]
So presumably it will use openblas in place of MKL for math optimization. If your use case is also for numpy optimization, you can likewise use openblas and shouldn't need MKL... which is fortunate, since it isn't available anyway. 😅

What is the easiest way to install numpy with LAPACK/BLAS?

I'm on an Ubuntu 14.04.
I would have expected doing:
sudo apt-get install python-numpy
would've worked but it isn't the case...
The way I tried to check is by doing locate blas and found nothing that seemed relevant.
I would like a solution that doesn't involve me compiling from source.
The ideal solution is something that uses the ubuntu repos.

In Ubuntu 14.04 and later, blas and lapack are installed as part of python-scipy and python3-scipy, so you need to install python-scipy. In Ubuntu 18.04 and earlier open the terminal and type:
sudo apt install python-scipy
This command will also install libblas3 (Basic Linear Algebra Subprograms reference implementations, shared library) and liblapack3 (Library of linear algebra routines 3 - shared version) as dependencies, and it will also install python-numpy as a dependency if you don't already have it installed.

numpy.show_config() revealed that I had no BLAS support even though python3-scipy was already installed. Uninstalling and re-installing python3-scipy and python3-numpy fixed it:
sudo apt-get remove python3-scipy python3-numpy
sudo apt-get install python3-scipy python3-numpy
Now I have LAPACK/BLAS support:
>>> numpy.show_config()
openblas_lapack_info:
NOT AVAILABLE
mkl_info:
NOT AVAILABLE
atlas_3_10_blas_threads_info:
NOT AVAILABLE
lapack_info:
libraries = ['lapack', 'lapack']
language = f77
library_dirs = ['/usr/lib']
atlas_3_10_blas_info:
NOT AVAILABLE
openblas_info:
NOT AVAILABLE
blas_opt_info:
libraries = ['blas', 'blas']
define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
library_dirs = ['/usr/lib']
language = c
blas_info:
libraries = ['blas', 'blas']
language = c
library_dirs = ['/usr/lib']
define_macros = [('HAVE_CBLAS', None)]
blas_mkl_info:
NOT AVAILABLE
atlas_info:
NOT AVAILABLE
atlas_3_10_threads_info:
NOT AVAILABLE
lapack_mkl_info:
NOT AVAILABLE
atlas_blas_threads_info:
NOT AVAILABLE
atlas_3_10_info:
NOT AVAILABLE
atlas_threads_info:
NOT AVAILABLE
atlas_blas_info:
NOT AVAILABLE
lapack_opt_info:
libraries = ['lapack', 'lapack', 'blas', 'blas']
define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
library_dirs = ['/usr/lib']
language = c

Have your tried to use pip?
sudo pip install numpy
If you dont have pip install pip with instructions here

I have very good experience with the anaconda package manager (learning took me 1-2h). In my opinion it is easier to use than venv and more felixble than pip/env alone. After download and setup you have most of the package like numpy ready to go. So no problems like this anymore!

Does Python SciPy need BLAS?

numpy.distutils.system_info.BlasNotFoundError:
Blas (http://www.netlib.org/blas/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [blas]) or by setting
the BLAS environment variable.
Which tar do I need to download off this site?
I've tried the fortrans, but I keep getting this error (after setting the environment variable obviously).

If you need to use the latest versions of SciPy rather than the packaged version, without going through the hassle of building BLAS and LAPACK, you can follow the below procedure.
Install linear algebra libraries from repository (for Ubuntu),
sudo apt-get install gfortran libopenblas-dev liblapack-dev
Then install SciPy, (after downloading the SciPy source): python setup.py install or
pip install scipy
As the case may be.

The SciPy webpage used to provide build and installation instructions, but the instructions there now rely on OS binary distributions. To build SciPy (and NumPy) on operating systems without precompiled packages of the required libraries, you must build and then statically link to the Fortran libraries BLAS and LAPACK:
mkdir -p ~/src/
cd ~/src/
wget http://www.netlib.org/blas/blas.tgz
tar xzf blas.tgz
cd BLAS-*
## NOTE: The selected Fortran compiler must be consistent for BLAS, LAPACK, NumPy, and SciPy.
## For GNU compiler on 32-bit systems:
#g77 -O2 -fno-second-underscore -c *.f # with g77
#gfortran -O2 -std=legacy -fno-second-underscore -c *.f # with gfortran
## OR for GNU compiler on 64-bit systems:
#g77 -O3 -m64 -fno-second-underscore -fPIC -c *.f # with g77
gfortran -O3 -std=legacy -m64 -fno-second-underscore -fPIC -c *.f # with gfortran
## OR for Intel compiler:
#ifort -FI -w90 -w95 -cm -O3 -unroll -c *.f
# Continue below irrespective of compiler:
ar r libfblas.a *.o
ranlib libfblas.a
rm -rf *.o
export BLAS=~/src/BLAS-*/libfblas.a
Execute only one of the five g77/gfortran/ifort commands. I have commented out all, but the gfortran which I use. The subsequent LAPACK installation requires a Fortran 90 compiler, and since both installs should use the same Fortran compiler, g77 should not be used for BLAS.
Next, you'll need to install the LAPACK stuff. The SciPy webpage's instructions helped me here as well, but I had to modify them to suit my environment:
mkdir -p ~/src
cd ~/src/
wget http://www.netlib.org/lapack/lapack.tgz
tar xzf lapack.tgz
cd lapack-*/
cp INSTALL/make.inc.gfortran make.inc # On Linux with lapack-3.2.1 or newer
make lapacklib
make clean
export LAPACK=~/src/lapack-*/liblapack.a
Update on 3-Sep-2015:
Verified some comments today (thanks to all): Before running make lapacklib edit the make.inc file and add -fPIC option to OPTS and NOOPT settings. If you are on a 64bit architecture or want to compile for one, also add -m64. It is important that BLAS and LAPACK are compiled with these options set to the same values. If you forget the -fPIC SciPy will actually give you an error about missing symbols and will recommend this switch. The specific section of make.inc looks like this in my setup:
FORTRAN = gfortran
OPTS = -O2 -frecursive -fPIC -m64
DRVOPTS = $(OPTS)
NOOPT = -O0 -frecursive -fPIC -m64
LOADER = gfortran
On old machines (e.g. RedHat 5), gfortran might be installed in an older version (e.g. 4.1.2) and does not understand option -frecursive. Simply remove it from the make.inc file in such cases.
The lapack test target of the Makefile fails in my setup because it cannot find the blas libraries. If you are thorough you can temporarily move the blas library to the specified location to test the lapack. I'm a lazy person, so I trust the devs to have it working and verify only in SciPy.

On Fedora, this works:
yum install lapack lapack-devel blas blas-devel
pip install numpy
pip install scipy
Remember to install 'lapack-devel' and 'blas-devel' in addition to 'blas' and 'lapack' otherwise you'll get the error you mentioned or the "numpy.distutils.system_info.LapackNotFoundError" error.

I guess you are talking about installation in Ubuntu. Just use:
apt-get install python-numpy python-scipy
That should take care of the BLAS libraries compiling as well. Else, compiling the BLAS libraries is very difficult.

For Windows users there is a nice binary package by Chris (warning: it's a pretty large download, 191 MB):
http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy-stack

Following the instructions given by 'cfi' works for me, although there are a few pieces they left out that you might need:
1) Your lapack directory, after unzipping, may be called lapack-X-Y (some version number), so you can just rename that to LAPACK.
cd ~/src
mv lapack-[tab] LAPACK
2) In that directory, you may need to do:
cd ~/src/LAPACK
cp lapack_LINUX.a libflapack.a

Try using
sudo apt-get install python3-scipy