How to install SciPy on Apple Silicon (ARM / M1) - python

I have successfully installed python 3.9.1 with Numpy and Matplotlib on a new Mac mini with Apple Silicon. However, I cannot install SciPy : I get compilation errors when using
python3 -m pip install scipy
I also tried installing everything from brew, and import scipy works, but using it gives a seg fault. I have installed ARM versions of lapack and openblas, but this does not fix the problem.
Has anyone succeeded? (I am interested in running it natively, not through Rosetta).

It's possible to install on regular arm64 brew python, you need to compile it yourself.
If numpy is already installed (from wheels) you'll need to uninstall it:
pip3 uninstall -y numpy pythran
I had to compile numpy, which requires cython and pybind11:
pip3 install cython pybind11
Then numpy can be compiled:
pip3 install --no-binary :all: --no-use-pep517 numpy
Scipy needs pythran (this should happen after installing numpy):
pip3 install pythran
Then we need to compile scipy itself, it depends on fortran and BLAS/LACK:
brew install openblas gfortran
Tell scipy where it can find this library:
export OPENBLAS=/opt/homebrew/opt/openblas/lib/
Then finally compilescipy:
pip3 install --no-binary :all: --no-use-pep517 scipy

This one worked for me after wasting hours:
pip install --pre -i https://pypi.anaconda.org/scipy-wheels-nightly/simple scipy

This solution worked on my M1 machine with pyenv:
brew install openblas
OPENBLAS="$(brew --prefix openblas)" pip install numpy scipy

You can install miniforge from https://github.com/conda-forge/miniforge#miniforge3
and then install those packages with,
conda install numpy scipy matplotlib

For me the easiest solutions:
brew install scipy
Probably good idea to edit the PATH, so the homebrew version will be the default.

I managed to get scipy installed on Apple Silicon. I mostly followed the instructions by lutzroeder here: https://github.com/scipy/scipy/issues/13409
Those instructions weren't successful for me, but running 'pip3 install scipy' worked afterwards. I think this fixed the problem for me:
/opt/homebrew/bin/brew install openblas
export OPENBLAS=$(/opt/homebrew/bin/brew --prefix openblas)
export CFLAGS="-falign-functions=8 ${CFLAGS}"

For those who need it for short-term purposes and don't want too much hustle - it seems to work with python 3.6.4 and scipy 1.5.4 out of the box (Big Sur 11.5.2, M1 chip).

In addition, if someone has this error message>
########### CLIB COMPILER OPTIMIZATION ###########
Platform :
Architecture: aarch64
Compiler : clang
CPU baseline :
Requested : 'min'
Enabled : none
Flags : none
Extra checks: none
CPU dispatch :
Requested : 'max -xop -fma4'
Enabled : none
Generated : none
CCompilerOpt.cache_flush[809] : write cache to path
I found this solution before compile numpy and scipy
Analysis of reasons:
From the above error message, you can see that the last error shows that clang has an error, so it is speculated that it should be an error caused by the compiler, because the new version of the xcode command tool uses the arm version of the compilation method by default, and if we want to use For x86 architecture, we need to manually set the specific architecture through environment variables.
export ARCHFLAGS="-arch x86_64"
example:
3c790c45799ec8c598753ebb22/build/temp.macosx-10.14.6-arm64-3.8/ccompiler_opt_cache_clib.py
----------------------------------------
ERROR: Command errored out with exit status 1: /Users/daniel_edu/Projects/PERSONAL/great_expectation_demo/.env/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/zb/c_b9kh2x1px7vl5683rwz8fr0000gn/T/pip-install-y8alaej_/numpy_3d813a3c790c45799ec8c598753ebb22/setup.py'"'"'; __file__='"'"'/private/var/folders/zb/c_b9kh2x1px7vl5683rwz8fr0000gn/T/pip-install-y8alaej_/numpy_3d813a3c790c45799ec8c598753ebb22/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/zb/c_b9kh2x1px7vl5683rwz8fr0000gn/T/pip-record-q9vraevr/install-record.txt --single-version-externally-managed --compile --install-headers /Users/daniel_edu/Projects/PERSONAL/great_expectation_demo/.env/include/site/python3.8/numpy Check the logs for full command output.
(.env) ➜ great_expectation_demo git:(master) ✗ export ARCHFLAGS="-arch x86_64"
(.env) ➜ great_expectation_demo git:(master) ✗ pip install --no-binary :all: --no-use-pep517 numpy
Collecting numpy
Using cached numpy-1.21.5.zip (10.7 MB)
Preparing metadata (setup.py) ... done
Skipping wheel build for numpy, due to binaries being disabled for it.
Installing collected packages: numpy
Running setup.py install for numpy ... done
Successfully installed numpy-1.21.5

What version of scipy you're trying to install?
To me running on Macbook air M1, I needed to increase the version from scipy==1.5.1 to scipy==1.7.3, so I guess you should use 1.7.3 version or above it and all will be fine...
pip install -Iv scipy==1.7.3
Or just add in your file requirements.txt this line:
scipy==1.7.3

The following worked for me.
I'm currently using Python 3.10.8, installed using brew.
And currently, when installing numpy==1.23.4, setuptools < 60.0.0 is required.
I'm using (brew --prefix)/bin/python3 -m pip for explicitly calling the pip from python 3.10 installed by brew.
Here are the versions I've just installed.
# python 3.10.8
# pip 22.3
# setuptools 59.8.0
# wheel 0.37.1
# numpy 1.23.4
# scipy 1.9.3
# pandas 1.5.1
# scikit-learn 1.1.3
# seaborn 0.12.1
# statsmodels 0.13.2
# gcc 12.2.0
# openblas 0.3.21
# gfortran 12
# pybind11 2.10.0
# Cython 0.29.32
# pythran 0.12.0
Here are the steps I followed:
# setuptools < 60.0.0 is required for numpy==1.23.4 in Python 3.10.8
$(brew --prefix)/bin/python3 -m pip install --upgrade pip==22.3 setuptools==59.8.0 wheel==0.37.1
# uninstall numpy and pythran first
$(brew --prefix)/bin/python3 -m pip uninstall -y numpy pythran
# uninstall scipy
$(brew --prefix)/bin/python3 -m pip uninstall -y scipy
# install prerequisites (with brew)
brew install gcc
brew install openblas
brew install gfortran
# set environment variables for compilers to find openblas
export LDFLAGS="-L/opt/homebrew/opt/openblas/lib"
export CPPFLAGS="-I/opt/homebrew/opt/openblas/include"
# install the prerequisites (with pip)
$(brew --prefix)/bin/python3 -m pip install pybind11
$(brew --prefix)/bin/python3 -m pip install Cython
# install numpy
$(brew --prefix)/bin/python3 -m pip install --no-binary :all: numpy
# install pythran after installing numpy, before installing scipy
$(brew --prefix)/bin/python3 -m pip install pythran
# install scipy
export OPENBLAS="$(brew --prefix)/opt/openblas/lib/"
$(brew --prefix)/bin/python3 -m pip install scipy
# install pandas
$(brew --prefix)/bin/python3 -m pip install pandas
# install scikit-learn
$(brew --prefix)/bin/python3 -m pip install scikit-learn
# install seaborn
$(brew --prefix)/bin/python3 -m pip install seaborn
# install statsmodels
$(brew --prefix)/bin/python3 -m pip install statsmodels

I use conda install scipy to resolve this problem. Conda have a custom version of scipy for Apple M1. Update macOS to 12 if you don't want to use Conda.

According to this Github issue, Scipy doesn't work on MacOS 11 (Big Sur). If none of these solutions are working for you I'd suggest updating your OS.

SciPy maintainer here.
SciPy is now distributed for M1, on both conda and pip. You need at least macOS 12.0 (Monterey) and Python >= 3.8. Our support for macOS is good, meaning you should not need to do anything special or compile yourself unless you want to use a development version or older macOS versions.
If you need to compile, only use our guides (e.g. https://scipy.github.io/devdocs/dev/contributor/building.html#building-from-sources). There are tons of moving pieces and doing something else might work today, but not tomorrow.
Avoid using pip to compile SciPy. Use our developer script python dev.py build after having followed the guides I linked to setup system dependencies for your platform.
Do NOT use brew to install a Python package, use an environment. brew is great to install system dependencies or other dev tools, but should not be used to install Python packages. Down the road this is a recipe for trouble.

Related

Error installing glpk package "Could not build wheels for glpk which use PEP 517 and cannot be installed directly" on Macbook

I'm practicing for a linear programming/optimization course I'll be taking in the fall.
Most of the course uses jupyter and part of the course requires me to install a package called 'glpk.' I'd like to install it on my mac since anaconda normally makes things like this a breeze, but so far everything I've tried has not worked.
I've tried:
sudo apt install libglpk-dev python3.8-dev libgmp3-dev
sudo pip3 install glpk --no-binary :all:
pip install --upgrade pip setuptools wheel
pip install glpk --no-cache-dir
pip --verbose install glpk
All of these have run into their own troubleshooting problems or have had no effect.
I'll happily listen to any advice or suggestions anyone has to offer, Thank you.
This answer assumes you have homebrew installed.
brew install glpk
# set C linker and include flags so that the pip wheel build can
# find the headers and library
LDFLAGS="-L$(brew --prefix glpk)/lib" CFLAGS="-I$(brew --prefix glpk)/include" pip install glpk
This should allow the glpk module to build.

error: command '/usr/bin/clang' failed with exit code 1

I download a not commonly-used software package in github in Mac M1. I am trying to compile and install myself according to the instruction.
I have encountered the following problem saying "command/usr/bin/clang with exits error 1". I did install xcode in my mac. Because the built-in gcc version is 4.2, I upgrade the version using brew install gcc#7 command and the link to this gcc version. But I still face the same compiling problem.
Does anyone have instructions for me how to solve this? The author did not maintain the source code anymore and I have struggled for one day and still can not fix the problem.
The follow steps worked!
upgrade pip and related components by:
python -m pip install --upgrade pip
pip install –upgrade wheel
pip install –upgrade setuptools
install openssl
brew install openssl re2
reinstall your package with some environment to be set
LDFLAGS="-L$(/opt/homebrew/bin/brew --prefix openssl)/lib -L$(/opt/homebrew/bin/brew --prefix re2)/lib" CPPFLAGS="-I$(/opt/homebrew/bin/brew --prefix openssl)/include -I$(/opt/homebrew/bin/brew --prefix re2)/include" GRPC_BUILD_WITH_BORING_SSL_ASM="" GRPC_PYTHON_BUILD_SYSTEM_RE2=true GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true GRPC_PYTHON_BUILD_SYSTEM_ZLIB=true pip install <your package name>
Source: https://candid.technology/error-command-usr-bin-clang-failed-with-exit-code-1/

How to install dlib for python on mac?

I'm getting an error when trying to install dlib on Python 3.7 on macOS with
pip3 install dlib
I have installed CMake, so that is not the problem.
I'm getting these error messages:
Failed building wheel for dlib
and
Command "/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/j2/nvk5521j2vn9s1w95_0vlwkm0000gn/T/pip-install-ls2e_3mr/dlib/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/j2/nvk5521j2vn9s1w95_0vlwkm0000gn/T/pip-record-hy3hu262/install-record.txt --single-version-externally-managed --compile"
failed with error code 1 in /private/var/folders/j2/nvk5521j2vn9s1w95_0vlwkm0000gn/T/pip-install-ls2e_3mr/dlib/
at the end in red. The reason I need dlib is to install face_recognition.
I think there's more to the error message and it's probably related to CMake, either it's not installed properly or it's not compatible with the pip3 install dlib.
I suggest using Homebrew to install Python3 (which includes pip3) and CMake. Homebrew manages the installation of packages, tools, libraries that might depend on system-related tools, paths, etc. It also prevents you from having to use sudo to install stuff on your system.
Install Homebrew
See the "Install Homebrew" section of https://brew.sh/
Basically:
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
Install Python3 (and pip3) using Homebrew
See https://docs.brew.sh/Homebrew-and-Python
See Homebrew's formula for the latest Python3 (python3.7, python3.8, python3.9)
Basically do one of the following:
$ brew install python#3.7 # or python#3.8 or python#3.9
$ brew install python#3 # get whichever is the latest version
Check Python installation
$ python3 -V
$ python3 -m pip -V
Install CMake using Homebrew
See https://formulae.brew.sh/formula/cmake
Basically:
$ brew install cmake
Check CMake installation
$ brew info cmake
$ cmake --version
Finally, install dlib with pip
$ python3 -m pip install dlib
If you don't want to use Homebrew (for some reason), you can try installing CMake for Mac directly using the installers (dmg or tar.gz) from here: https://cmake.org/download/.
Install Homebrew from here
A bunch of dependencies are needed for dlib:
brew install cmake
brew install boost
brew install boost-python
brew install dlib
pip3 install numpy
pip3 install scipy
pip3 install scikit-image
pip3 install dlib
If this does not work, try:
python3 -m pip install dlib
dlib is dependent on cmake so you need to install that first.
You just need to run following commands on your terminal:
$ brew install cmake
$ pip install cmake
$ brew install dlib
$ pip install dlib
You can install the conda-forge Dlib package by running
conda install -c conda-forge dlib
Conda-forge has made sure to resolve all the dependency conflicts, so I had no issues with the installation. They have pre-built binaries for Windows, Linux, and Mac OS, and Python 2.7 through 3.8. The current package (at the time of this writing) is based on Dlib v19.19.
$ pip3 install --upgrade pip
$ pip3 install global
$ pip3 install cmake
$ pip3 install dlib
$ brew install dlib
$ sudo pip install dlib
The only thing you need to do, I guess, is run pip3 install dlib with sudo.
sudo pip3 install dlib

"failed with error code 1" while installing scipy

I have Python 2.7.9 on windows 7 64-bits. I'm trying to install scipy using pip. I used pip install scipy but I get the following error :
Command "C:\Python27\python.exe -c "import setuptools, tokenize;__file__='c:\\us
ers\\admin\\appdata\\local\\temp\\pip-build-xpl5cw\\scipy\\setup.py';exec(compil
e(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file
__, 'exec'))" install --record c:\users\admin\appdata\local\temp\pip-b68pfc-reco
rd\install-record.txt --single-version-externally-managed --compile" failed with
error code 1 in c:\users\admin\appdata\local\temp\pip-build-xpl5cw\scipy
I searched on google and stackoverflow for solutions but nothing worked. I tried upgrading pip and it didn't work too.
Here are my installed packages and their versions :
httplib2 (0.9.1)
nltk (3.0.3)
nose (1.3.7)
numpy (1.9.2)
pip (7.0.3)
pytz (2015.4)
scikit-learn (0.16.1)
setuptools (17.1.1)
six (1.9.0)
twilio (4.4.0)
wheel (0.24.0)
I'm on Ubuntu 14.04. I tried installing through pip which kept on failing with error code 1 or took forever to finish (never finished actually!).
However, the following command worked without a hitch:
sudo apt-get install python-numpy python-scipy python-matplotlib ipython ipython-notebook python-pandas python-sympy python-nose
Reference: http://www.scipy.org/install.html
On Windows, you'll need to use the unofficial precompiled binaries:
http://www.lfd.uci.edu/~gohlke/pythonlibs/
Select the version matching your architecture and Python version.
Installing setuptools made it work for me (Mac OS)
pip install -U pip setuptools
I also had the same error when installing scipy.
Then I downloaded and installed MiniConda. And then I used the below command to install pytables.
conda install -c conda-forge scipy
Please refer the below screenshot.
On windows you can check below command:
python -m pip install psycopg2

Install line 'pip install scipy' fails [duplicate]

It is possible to install NumPy with pip using pip install numpy.
Is there a similar possibility with SciPy? (Doing pip install scipy does not work.)
Update
The package SciPy is now available to be installed with pip!
Prerequisite:
sudo apt-get install build-essential gfortran libatlas-base-dev python-pip python-dev
sudo pip install --upgrade pip
Actual packages:
sudo pip install numpy
sudo pip install scipy
Optional packages:
sudo pip install matplotlib OR sudo apt-get install python-matplotlib
sudo pip install -U scikit-learn
sudo pip install pandas
src
An attempt to easy_install indicates a problem with their listing in the Python Package Index, which pip searches.
easy_install scipy
Searching for scipy
Reading http://pypi.python.org/simple/scipy/
Reading http://www.scipy.org
Reading http://sourceforge.net/project/showfiles.php?group_id=27747&package_id=19531
Reading http://new.scipy.org/Wiki/Download
All is not lost, however; pip can install from Subversion (SVN), Git, Mercurial, and Bazaar repositories. SciPy uses SVN:
pip install svn+http://svn.scipy.org/svn/scipy/trunk/#egg=scipy
Update (12-2012):
pip install git+https://github.com/scipy/scipy.git
Since NumPy is a dependency, it should be installed as well.
In Ubuntu 10.04 (Lucid), I could successfully pip install scipy (within a virtualenv) after installing some of its dependencies, in particular:
$ sudo apt-get install libamd2.2.0 libblas3gf libc6 libgcc1 libgfortran3 liblapack3gf libumfpack5.4.0 libstdc++6 build-essential gfortran libatlas-sse2-dev python-all-dev
To install scipy on windows follow these instructions:-
Step-1 : Press this link http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy to download a scipy .whl file (e.g. scipy-0.17.0-cp34-none-win_amd64.whl).
Step-2: Go to the directory where that download file is there from the command prompt (cd folder-name ).
Step-3: Run this command:
pip install scipy-0.17.0-cp27-none-win_amd64.whl
I tried all the above and nothing worked for me. This solved all my problems:
pip install -U numpy
pip install -U scipy
Note that the -U option to pip install requests that the package be upgraded. Without it, if the package is already installed pip will inform you of this and exit without doing anything.
If I first install BLAS, LAPACK and GCC Fortran as system packages (I'm using Arch Linux), I can get SciPy installed with:
pip install scipy
On Fedora, this works:
sudo yum install -y python-pip
sudo yum install -y lapack lapack-devel blas blas-devel
sudo yum install -y blas-static lapack-static
sudo pip install numpy
sudo pip install scipy
If you get any public key errors while downloading, add --nogpgcheck as parameter to yum, for example:
yum --nogpgcheck install blas-devel
On Fedora 23 onwards, use dnf instead of yum.
For the Arch Linux users:
pip install --user scipy prerequisites the following Arch packages to be installed:
gcc-fortran
blas
lapack
Addon for Ubuntu (Ubuntu 10.04 LTS (Lucid Lynx)):
The repository moved, but a
pip install -e git+http://github.com/scipy/scipy/#egg=scipy
failed for me... With the following steps, it finally worked out (as root in a virtual environment, where python3 is a link to Python 3.2.2):
install the Ubuntu dependencies (see elaichi), clone NumPy and SciPy:
git clone git://github.com/scipy/scipy.git scipy
git clone git://github.com/numpy/numpy.git numpy
Build NumPy (within the numpy folder):
python3 setup.py build --fcompiler=gnu95
Install SciPy (within the scipy folder):
python3 setup.py install
In my case, it wasn't working until I also installed the following package : libatlas-base-dev, gfortran
sudo apt-get install libatlas-base-dev gfortran
Then run pip install scipy
install python-3.4.4
scipy-0.15.1-win32-superpack-python3.4
apply the following commend doc
py -m pip install --upgrade pip
py -m pip install numpy
py -m pip install matplotlib
py -m pip install scipy
py -m pip install scikit-learn
The answer is yes, there is.
First you can easily install numpy use commands:
pip install numpy
Then you should install mkl, which is required by Scipy, and you can download it here
After download the file_name.whl you install it
C:\Users\****\Desktop\a> pip install mkl_service-1.1.2-cp35-cp35m-win32.whl
Processing c:\users\****\desktop\a\mkl_service-1.1.2-cp35-cp35m-win32.whl
Installing collected packages: mkl-service
Successfully installed mkl-service-1.1.2
Then at the same website you can download scipy-0.18.1-cp35-cp35m-win32.whl
Note:You should download the file_name.whl according to you python version, if you python version is 32bit python3.5 you should download this one, and the "win32" is about your python version, not your operating system version.
Then install file_name.whl like this:
C:\Users\****\Desktop\a>pip install scipy-0.18.1-cp35-cp35m-win32.whl
Processing c:\users\****\desktop\a\scipy-0.18.1-cp35-cp35m-win32.whl
Installing collected packages: scipy
Successfully installed scipy-0.18.1
Then there is only one more thing to do: comment out a specfic line or there will be error messages when you imput command "import scipy".
So comment out this line
from numpy._distributor_init import NUMPY_MKL # requires numpy+mkl
in this file: your_own_path\lib\site-packages\scipy__init__.py
Then you can use SciPy :)
Here tells you more about the last step.
Here is a similar anwser to a similar question.
Besides all of these answers,
If you install python of 32bit on your 64bit machine, you have to download scipy of 32-bit irrespective of your machine.
http://www.lfd.uci.edu/~gohlke/pythonlibs/
In the above URL you can download the packages and command is: pip install
For gentoo, it's in the main repository:
emerge --ask scipy
You can also use this in windows with python 3.6 python -m pip install scipy

Categories