We Keep Coding

Confusion about conda distributions/channels on arm processor

I've been using Anaconda for a few years now, but since I started using a Mac with a M1 processor I had to deal with a bunch of problem with the installation of some packages, which left me a little confused about some basic concepts.
For example, I was trying to install Tensorflow, and it turns out the proper way is to install miniforge, and get Tensorflow from the conda-forge channel (which is the default for miniforge), as explained here.
Then, I was wondering whether I could do the same using Anaconda/Miniconda...set up the conda-forge channel as default, and install Tensorflow (or any other arm-compatible package), but I've been told it's not possible
So, now I'm trying to understand how this all works.
If a Tensorflow version compatible with M1 processors exists in the conda-forge channel (and it does exist), why can't I install it by using Anaconda/Miniconda, after configuring it to use said channel? To phrase it in another way, what is the difference between Anaconda/Miniconda and Miniforge, other than the channels they look into for packages (and, as I understand, some licenses)?
Here there is a similar question, but the answers don't seem to address my main concern (why Anaconda/Miniconda with conda-forge as default channel is different than miniforge).

It's not impossible, but you'll have to jump through hoops to get it done.
First, if you have an Anaconda installation, you can't install conda-forge packages into the base environment consistently, because the anaconda package in the base environment of Anaconda will conflict with packages from conda-forge.
Second, since Anaconda is only x86_64 at the moment, you can only install it via Rossetta emulation. After that, you need to tell conda that you need arm64 compatible packages by setting the env variable CONDA_SUBDIR.
CONDA_SUBDIR=osx-arm64 conda create -n native numpy -c conda-forge
will get you a new env with native arm64 packages. However if you want to update this env, you have to prefix all your conda commands with CONDA_SUBDIR=osx-arm64.
To fix this permanently, you can do the following
conda activate native
conda config --env --set subdir osx-arm64
which will make conda use osx-arm64 for this environment.

Conflict resolution in pip vs. conda

I assume both pip and conda, despite differences, are package managers and check for consistency of packages installed in an environment! In my case though, I have a list of requirements.txt, on top of python=3.6. In my conda virtual environment, I installed them one-by-one. The strange thing is that when locating some packages in anaconda.org channels and installing them with conda install, conda complains! An example is when I tried to install statistics=1.0.3.5, and I got this message on terminal:
UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:
Specifications:
- statistics=1.0.3.5 -> python[version='2.7.*|<3|>=2.7,<2.8.0a0']
Your python: python=3.6
However, when I did it with pip, it worked!
Why is that?
Am I going to bump into a problem down the road with this package?
I read this Stackoverflow post about the difference between pip and conda and tried to understand it from the doc (Although not that successful).

When working with conda virtual environments, installing packages with pip should be the last resort. If a package isn't available through the default channel, try installing from conda-forge first.
The difference between conda and pip is huge (not to mention virtual environments): Conda aims to install a consistent set of packages - which results in an optimization problem - while pip just installs dependencies, no matter if that is in conflict with any previously installed package.
However, since you are writing unit tests with your code you'll immediatly realize if you bump into a problem.

Cannot find package on Anaconda navigator after installing it using pip

I followed the instructions here: Can't find package on Anaconda Navigator. What to do next?
I clicked Open terminal from environment on Anaconda navigator, and then used "pip3 install lmfit" in the terminal. But after installing the lmfit package using pip3, I still cannot find it in the conda list. What should I do?

The Problem
At the time of this question, Conda builds of pip had only just started including a pip3 entrypoint,1 therefore pip3 is very likely referring to a non-Conda version of Python and that is where the package was installed. Try checking which pip3 to find out where it went.
Recommendation
Conda First
Generally, it is preferable to use Conda to install packages in Conda environments, and in this case the package is available via the Conda Forge channel:
conda install -c conda-forge lmfit
Contrary to M. Newville's answer, this recommendation to prefer Conda packages is not about benefiting Conda developers, but instead a rule of thumb to help users avoid creating unstable or unreproducible environments. More info about the risks of mixing pip install and conda install can be found in the post "Using Pip in a Conda Environment".
Nevertheless, the comment that not all packages (and specifically lmfit) are found in the default repository and this complicates installation by requiring resorting to third-party channels is a good point. In fact, because third-parties are free to use different build stacks there are known problems with mixing packages built by Anaconda and those from Conda Forge. However, these issues tend to be rare and limited to compiled code. Additionally, adding trusted channels to a configuration and setting channel priorities heuristically solves the issue.
As for risks in using third-party channels, arbitrary Anaconda Cloud user channels are risky: one should only source packages from channels you trust (just like anything else one installs). Conda Forge in particular is well-reputed and all feedstocks are freely available on GitHub. Moreover, many Python package builds on Conda Forge are simply wrappers around the PyPI build of the package.
PyPI Last
Sometimes it isn't possible to avoid using PyPI. When one must resort to installing from PyPI, it is better practice to use the pip entrypoint from an activate environment, rather than pip3, since only some Conda builds of pip include pip3. For example,
conda activate my_env
pip install lmfit
Again, following the recommendations in "Using Pip in a Conda Environment", one should operate under the assumption that any subsequent calls to conda (install|upgrade|remove) in the environment could have undefined behavior.
PyPI Only
For the sake of completeness, I will note that a stable way of using Conda that is consistent with the recommendations is to limit Conda to the role of environment creation and use pip for all package installation.
This strategy is perhaps the least burden on the Python-only user, who doesn't want to deal with things like finding the Conda-equivalent package name or searching non-default channels. However, its applicability seems limited to Python-only environments, since other libraries may still need to resort to conda install.
[1]: Conda Forge and Anaconda started consistently including pip3 entrypoints for the pip module after version 20.2.

Installing a pure Python package, such as lmfit with the correct version of pip install lmfit should be fine.
Conda first is recommended to make the life of the conda maintainers and packagers easier, not the user's life. FWIW, I maintain both kinds of packages,
and there is no reason to recommend conda install lmfit over pip install lmfit.
In fact, lmfit is not in the default anaconda repository so that installing it requires going to a third-party conda channel such as conda-forge. That adds complexity and risk to your conda environment.
Really, pip install lmfit should be fine.

python cannot import graph-tool after installing it on a mac using brew (homebrew)

I used homebrew to install graph-tool on a macOS 10.14 using the command brew install graph-tool, but I cannot import it in python (that is installed through anaconda in /opt/anaconda3/bin/python). How can I make Python able to import graph-tool?

Prefer Conda Packages
If you want to use something in a Conda environment, then the best practice is to install through Conda. In this case, the graph-tool package is available through Conda Forge and can be installed with
conda install -c conda-forge graph-tool
Personally, since I started using Conda in 2016, I've found no reason to use Homebrew or MacPorts. There might be some exceptions, but this package isn't one.
conda install not working
Sometimes the packages you already have installed can conflict with the ones you want to add. This is especially the case when it comes to massive envs like the base in Anaconda. Best practice for using Conda is to avoid installing in base and instead create envs for each project or project type. This also helps you avoid breaking code you've already written because sometimes installs can trigger a version change in a package you were using.
In this case, you could make a new env, say foo, with
conda create --name foo -c conda-forge graph-tool
You can list any additional packages you know you'll need after graph-tool.

Trouble importing GDAL and ogr using Jyupter Notebook

I am working on a school project doing some spatial analysis on of counties in California; using Jyupter Notebook to run Python(2.7) scripts and create visualizations.
I was successful in importing Basemap into my notebook after I downloaded it using conda. I am very new to Python and can't really remember how I pulled that off. I believe I used the instructions from this website.
But now, when trying to impose my shapefile over the basemap I run into this error:
ValueError: shapefile must have lat/lon vertices - it looks like this one has vertices in map projection coordinates. You can convert the shapefile to geographic
coordinates using the shpproj utility from the shapelib tools
(http://shapelib.maptools.org/shapelib-tools.html)
For the past hour I have been searching for ways to convert the coordinates and learned about GDAL and some command ogr2ogr. I have tried following the instructions posted on gdal.org that tell me to run this command: conda install -c conda-forge gdal. Is there a difference between conda and conda-forge? I have tried many variations of this command like: conda install -c anaconda2 gdal and install conda gdal. Nothing has worked because when I go back to my notebook and try import gdal or import ogr I get an error like so:
ImportError: dlopen(/anaconda2/lib/python2.7/site-packages/osgeo/_gdal.so, 2): Library not loaded: #rpath/libgif.7.dylib
Referenced from: /anaconda2/lib/libgdal.20.dylib
Reason: image not found
I'm really confused as to how conda works, the difference between conda and conda-forge, and where all this stuff I have been downloading has been going. Can anyone help me make sense of what is going on and what I am doing wrong?

First, when you ask a question, please specify your environment: OS, conda version (conda info), Python version, etc.
What is a channel
conda is a Python environment manager and a package manager. To install packages, conda has to download the packages from somewhere. The place where conda get the list of packages is called a channel.
By defaults, two channels named main and r are enable, both of which are maintained by Anaconda, the company itself. conda-force is a channel by a third-party community.
Quote from conda-force documentation
What is conda-forge?
conda-forge is a community effort that provides conda packages for a wide range of software.
Specify a channel in command line with -c channelname, shortcut for --channel.
How to fix your problem
Your problem was caused by a mixed channel usage for one package. gdal was specified to be installed from conda-forge, but not its dependencies. Since channel of its dependencies are not specified, the default channel main and r will be used.
To fix the mixed dependency use, enable "strict" channel priority.
conda config --set channel_priority strict
which makes channel specified by --channel prioritize over other channels.
channel_priority (ChannelPriority)
With strict channel priority, packages in lower
priority channels are not considered if a package with the same name
appears in a higher priority channel.
Then recreate an environment and install gdal.
conda create --name test4gdal python=2.7
conda activate test4gdal
conda install -c conda-forge gdal
This time, all gdal related packages are installed from conda-forge.
References
Get started with conda
conda-forge documentation
import error from geopandas, mac os, anaconda