We Keep Coding

How do I import Pandas on Apple M1 chip

When I call:
import pandas as pd
on tensorflow_macos (for M1 chip)
I get:
ImportError Traceback (most recent call last)
~/tensorflow_macos_venv/lib/python3.8/site-packages/pandas/__init__.py in <module>
28 try:
---> 29 from pandas._libs import hashtable as _hashtable, lib as _lib, tslib as _tslib
30 except ImportError as e: # pragma: no cover
~/tensorflow_macos_venv/lib/python3.8/site-packages/pandas/_libs/__init__.py in <module>
12
---> 13 from pandas._libs.interval import Interval
14 from pandas._libs.tslibs import (
ImportError: dlopen(/Users/ashkan/tensorflow_macos_venv/lib/python3.8/site-packages/pandas/_libs/interval.cpython-38-darwin.so, 2): no suitable image found. Did find:
/Users/ashkan/tensorflow_macos_venv/lib/python3.8/site-packages/pandas/_libs/interval.cpython-38-darwin.so: mach-o, but wrong architecture
/Users/ashkan/tensorflow_macos_venv/lib/python3.8/site-packages/pandas/_libs/interval.cpython-38-darwin.so: mach-o, but wrong architecture
The above exception was the direct cause of the following exception:
ImportError Traceback (most recent call last)
<ipython-input-3-94f55571b0d6> in <module>
1 import numpy as np
----> 2 import pandas as pd
3
4 import matplotlib.pyplot as plt
5
~/tensorflow_macos_venv/lib/python3.8/site-packages/pandas/__init__.py in <module>
31 # hack but overkill to use re
32 module = str(e).replace("cannot import name ", "")
---> 33 raise ImportError(
34 f"C extension: {module} not built. If you want to import "
35 "pandas from the source directory, you may need to run "
ImportError: C extension: dlopen(/Users/ashkan/tensorflow_macos_venv/lib/python3.8/site-packages/pandas/_libs/interval.cpython-38-darwin.so, 2): no suitable image found. Did find:
/Users/ashkan/tensorflow_macos_venv/lib/python3.8/site-packages/pandas/_libs/interval.cpython-38-darwin.so: mach-o, but wrong architecture
/Users/ashkan/tensorflow_macos_venv/lib/python3.8/site-packages/pandas/_libs/interval.cpython-38-darwin.so: mach-o, but wrong architecture not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --force' to build the C extensions first.

I hope Xcode Command Line Tools is already installed. If not, please install it then follow the steps.
Step 1:miniforge
Install miniforge for arm64 (Apple Silicon) from miniforge GitHub. Miniforge enables installing python packages natively compiled for Apple Silicon.
Step 2: create Conda environment
Don’t forget to open a new session or to source your .zshrc after miniforge install and before going through this step.
Create an empty Conda environment, then activate it and install python 3.8. and all the needed packages. Please note numpy is unnecessary here as pandas already install it, but it will be overwritten in the last step with the version provided by Apple.
conda create --name mytf
conda activate mytf
conda install -y python==3.8.6
conda install -y pandas matplotlib scikit-learn jupyterlab
I have tested it. It will work after these steps.

As shown in the next post(https://github.com/apple/tensorflow_macos/issues/6), it seems that the solution is to install pandas from conda until they fix this problem.
To install pandas:
conda install pandas
Or installing directly from the source:
https://github.com/pandas-dev/pandas#installation-from-sources

Follow these steps -
Important With Tensorflow Installation
With miniforge installation do this -
conda install numpy scipy scikit-learn pandas matplotlib seaborn

Use Anaconda visual UI to install into the conda environment you created. You just select your environment and search for pandas.

Can not get pytorch working with tensorboard

I"m going through this tutorial to set up pytorch (v1.3.0 through conda) with tensorboard https://pytorch.org/tutorials/intermediate/tensorboard_tutorial.html#
but on the step
from torch.utils.tensorboard import SummaryWriter
# default `log_dir` is "runs" - we'll be more specific here
writer = SummaryWriter('runs/fashion_mnist_experiment_1')
I keep getting the error
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
C:\ProgramData\Anaconda3\envs\fastai_v1\lib\site-packages\torch\utils\tensorboard\__init__.py in
1 try:
----> 2 from tensorboard.summary.writer.record_writer import RecordWriter # noqa F401
3 except ImportError:
ModuleNotFoundError: No module named 'tensorboard.summary'; 'tensorboard' is not a package
During handling of the above exception, another exception occurred:
ImportError Traceback (most recent call last)
c:\Users\matt\Documents\code\playground\tensorboard.py in
----> 1 from torch.utils.tensorboard import SummaryWriter
2
3 # default `log_dir` is "runs" - we'll be more specific here
4 writer = SummaryWriter('runs/fashion_mnist_experiment_1')
C:\ProgramData\Anaconda3\envs\fastai_v1\lib\site-packages\torch\utils\tensorboard\__init__.py in
2 from tensorboard.summary.writer.record_writer import RecordWriter # noqa F401
3 except ImportError:
----> 4 raise ImportError('TensorBoard logging requires TensorBoard with Python summary writer installed. '
5 'This should be available in 1.14 or above.')
6 from .writer import FileWriter, SummaryWriter # noqa F401
ImportError: TensorBoard logging requires TensorBoard with Python summary writer installed. This should be available in 1.14 or above.
Does anyone have any suggestions?

The error log says, among other things,
ImportError: TensorBoard logging requires TensorBoard with Python summary writer installed. This should be available in 1.14 or above.
So, when it tries to import TensorBoard, it's unable to do so because it's missing it in the search path. You can install the latest version (without specifying any version number), as in:
$ conda install -c conda-forge tensorboard
Apart from that, you might also need to install protobuf:
$ conda install -c conda-forge protobuf
These installations should fix the ImportErrors.

I came across the same error, I solved by taking the following steps:
Removed all installation of Tensorflow or Tensorboard from the conda environment.
Then by activating the same conda environment, type "pip install -U tb-nightly"
Then type, "pip install -U future"
done

I think it's a version problem.
just run this:
pip install tensorboard==1.14.0
(not pip install tensorboard==1.14)
or just install the tensoflow 1.14.0, which contains the tensorboard 1.14.0:
pip install tensorflow==1.14.0
This version of tensorflow worked for me in pytorch 1.2.

I've done:
conda install -y tensorboard
before with no problems, so Im not sure why that wouldn't work. It's the simplest.

I met the same error, and my conda also didn't work that time, so I chose to use tensorboardX, it is almost absolutely the same as tensorboard(also its operations).
Just install it with pip install tensorboardX
And you can import it with from tensorboardX import SummaryWriter

Cannot import category_encoders module

I am not able to import category_encoders module in jupyter notebook in python 3 virtual environment.
Error
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-15-86725efc8d1e> in <module>()
9 from plotly import graph_objs
10 from datetime import datetime
---> 11 import category_encoders as ce
12
13 import sklearn
ModuleNotFoundError: No module named 'category_encoders'
Output of "which pip"
/opt/virtual_env/py3/bin/pip
Output of "pip show category_encoders" is
Name: category-encoders
Version: 1.3.0
Summary: A collection sklearn transformers to encode categorical variables as numeric
Home-page: https://github.com/wdm0006/categorical_encoding
Author: Will McGinnis
Author-email: will#pedalwrencher.com
License: BSD
Location: /opt/virtual_env/py3/lib/python3.6/site-packages
Requires: numpy, pandas, statsmodels, scikit-learn, patsy, scipy
Required-by:
Output of "pip list"
Package Version
------------------- -------
absl-py 0.7.0
astor 0.7.1
backcall 0.1.0
category-encoders 1.3.0
cycler 0.10.0

pip install --upgrade category_encoders
Now,
Restart your kernel
import category_encoders as... so on and so forth
Also,
you can checkout the sk-learn's preprocessing class CategoricalEncoder...
from sklearn.preprocessing import CategoricalEncoder
get sk-learn from here,
pip install git+git://github.com/scikit-learn/scikit-learn.git
or Simply
pip install sklearn

I was using anaconda environment.
Initially, I installed category_encoders using pip (within the conda enviornment) and it failed to resolve. Then I uninstalled ``category_encoders` from pip
pip uninstall category-encoders
and installed it using conda:
conda install -c conda-forge category_encoders
That solved the issue.

If you have given your virtual environment ('py3' for your case) the permission for accessing the system site-packages directory you can try installing the package to your base (root) environment.
Search 'pyvenv.cfg' file in your system and check if it includes:
include-system-site-packages = true
If yes, install the package to your root environment as stated here and try importing it again in your Jupyter Notebook.

You can add a cell above and type ! pip install --upgrade category_encoders
It will install the package without opening a terminal window.

Geopandas Installation

I am trying to install geopandas on my Mac OS X right now.
What I have done:
• Downloaded and installed Anaconda for Python 2.7 here. Graphical installer.
• python --version returns the following result: Python 2.7.13 :: Anaconda 4.3.0 (x86_64)
• Installed geopandas using this line : conda install -c conda-forge geopandas. This command also, as I understand, installed all other required packages. Here's the list of all that command installed:
click-plugins: 1.0.3-py27_0 conda-forge
cligj: 0.4.0-py27_0 conda-forge
descartes: 1.1.0-py27_0 conda-forge
expat: 2.1.0-2 conda-forge
fiona: 1.7.3-np111py27_0 conda-forge
freexl: 1.0.2-1 conda-forge
gdal: 2.1.3-np111py27_0 conda-forge
geopandas: 0.2.1-py27_3 conda-forge
geos: 3.5.1-1 conda-forge
giflib: 5.1.4-0 conda-forge
hdf4: 4.2.12-0 conda-forge
json-c: 0.12-0 conda-forge
kealib: 1.4.6-3 conda-forge
libdap4: 3.18.3-2 conda-forge
libnetcdf: 4.4.1.1-2 conda-forge
libpq: 9.5.4-3 conda-forge
libspatialindex: 1.8.5-1 conda-forge
libspatialite: 4.3.0a-14 conda-forge
munch: 2.1.0-py27_0 conda-forge
openjpeg: 2.1.2-2 conda-forge
pcre: 8.39-0 conda-forge
proj4: 4.9.3-2 conda-forge
psycopg2: 2.6.2-py27_1 conda-forge
pyproj: 1.9.5.1-py27_0 conda-forge
pysal: 1.13.0-py27_0 conda-forge
rtree: 0.8.3-py27_0 conda-forge
shapely: 1.5.17-np111py27_2 conda-forge
xerces-c: 3.1.4-0
• geopandas appears when I run conda list
• I wrote some code in test.py:
import geopandas as gpd
gpd.read_file("osm_mexico_city/mexico-city_mexico_roads.geojson")
The problem(s):
• When I run execute python test.py in the terminal I get the following error:
Traceback (most recent call last):
File "test.py", line 1, in <module>
import geopandas as gpd
File "//anaconda/lib/python2.7/site-packages/geopandas/__init__.py", line 4, in <module>
from geopandas.io.file import read_file
File "//anaconda/lib/python2.7/site-packages/geopandas/io/file.py", line 3, in <module>
import fiona
File "//anaconda/lib/python2.7/site-packages/fiona/__init__.py", line 69, in <module>
from fiona.collection import Collection, BytesCollection, vsi_path
File "//anaconda/lib/python2.7/site-packages/fiona/collection.py", line 9, in <module>
from fiona.ogrext import Iterator, ItemsIterator, KeysIterator
ImportError: dlopen(//anaconda/lib/python2.7/site-packages/fiona/ogrext.so, 2): Library not loaded: #rpath/libpng16.16.dylib
Referenced from: //anaconda/lib/libgdal.20.dylib
Reason: Incompatible library version: libgdal.20.dylib requires version 45.0.0 or later, but libpng16.16.dylib provides version 44.0.0
• Second way I tried is through PyCharm. I have set the project interpreter to python 2.7. There geopandas does not appear in the package list, therefore PyCharm blocks even before that:
import geopandas as gpd
ImportError: No module named geopandas
Hope anyone has an idea of how to fix both.
UPDATE 1
After installing Anaconda brew doctor returns this now:
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry and just ignore them. Thanks!
Warning: Anaconda is known to frequently break Homebrew builds, including Vim and
MacVim, due to bundling many duplicates of system and Homebrew-available
tools.
If you encounter a build failure please temporarily remove Anaconda
from your $PATH and attempt the build again prior to reporting the
failure to us. Thanks!
Warning: "config" scripts exist outside your system or Homebrew directories.
`./configure` scripts often look for *-config scripts to determine if
software packages are installed, and what additional flags to use when
compiling and linking.
Having additional scripts in your path can confuse software installed via
Homebrew if the config script overrides a system or Homebrew provided
script of the same name. We found the following "config" scripts:
//anaconda/bin/curl-config
//anaconda/bin/dap-config
//anaconda/bin/freetype-config
//anaconda/bin/gdal-config
//anaconda/bin/geos-config
//anaconda/bin/icu-config
//anaconda/bin/kea-config
//anaconda/bin/libpng-config
//anaconda/bin/libpng16-config
//anaconda/bin/nc-config
//anaconda/bin/pcre-config
//anaconda/bin/python-config
//anaconda/bin/python2-config
//anaconda/bin/python2.7-config
//anaconda/bin/xml2-config
//anaconda/bin/xslt-config
Seems related to this issue. How do I fix these to make my stuff work?
UPDATE 2
We can forget about the PyCharm problem, it turns out somehow I had another python 2.7 and this is the one he was suggesting. I've changed it to the Anaconda python 2.7 and now it gives the same error as the terminal output.

Faced the same problem, and finally solved this through this method:
First install geopandas using conda (This will install all dependencies)
conda install -c conda-forge geopandas
Uninstall Fiona
conda uninstall fiona
Download fiona wheel and install
(I downloaded from here https://pypi.python.org/packages/71/ea/908bf078499b30d1ec374eb5baba016a568fc8142ee6ccf72e356d20871c/Fiona-1.7.4-cp27-cp27m-macosx_10_6_intel.whl#md5=971393c23ffc552664b7c694b992fb3e)
pip install Fiona-1.7.4-cp27-cp27m-macosx_10_6_intel.whl
Reinstall geopandas
pip install git+git://github.com/geopandas/geopandas.git

Not sure what happened with Anaconda, but it turns out the dependencies were not compatible.
conda install fiona=1.6
Downgrading Fiona to 1.6 solved everything for me.

Jupyter gets Import Error when importing matplotlib

I'm just now trying to integrate to Jupyter and I have been using Anaconda for a long while now.
When trying to import the matplotlib in the IPython Notebook,
import matplotlib.pyplot as plt
%matplotlib inline
I get this error:
ImportError Traceback (most recent call last)
<ipython-input-2-385145dcc870> in <module>()
----> 1 import matplotlib.pyplot as plt
2 get_ipython().magic(u'matplotlib inline')
ImportError: No module named matplotlib.pyplot

I faced the same problem. Then I used folllowing command to install matplotlib library to my virtual environment.
pip install matplotlib
Then I could import matplotlib to jupyter notebook.

I think you should install matplotlib library if using conda then in terminal type:
# create a new environment with the required packages
conda create -n "matplotlib_build" python=3.5 numpy python-dateutil pyparsing pytz tornado cycler tk libpng zlib freetype
activate matplotlib_build
# if you want a qt backend, you also have to install pyqt (be aware that pyqt doesn't mix well if
# you have created the environment with conda-forge already activated...)
conda install pyqt
# this package is only available in the conda-forge channel
conda install -c conda-forge msinttypes
# copy the libs which have "wrong" names
set LIBRARY_LIB=%CONDA_PREFIX%\Library\lib
mkdir lib || cmd /c "exit /b 0"
copy %LIBRARY_LIB%\zlibstatic.lib lib\z.lib
copy %LIBRARY_LIB%\libpng_static.lib lib\png.lib
# Make the header files and the rest of the static libs available during the build
# CONDA_DEFAULT_ENV is a env variable which is set to the currently active environment path
set MPLBASEDIRLIST=%CONDA_PREFIX%\Library\;.
# build the wheel
python setup.py bdist_wheel
This code was taken from https://matplotlib.org/users/installing.html#conda-packages. Hope this helps.
Thanks
Michael