I recently installed Anaconda in my Windows. I did that to use some packages from some specific channels required by an application that is using Python 3.5 as its scripting language.
I adjusted my PATH variable to use Conda, pointing to the Python environment of the particular program, but now I would like to use Conda as well for a different Python installation that I have on my Windows.
When installing Anaconda then it isn't asking for a Python version to be related to. So, how can I use Conda to install into the other Python installation. Both Python installations are 'physical' installations - not virtual in any way.
Uninstall the other python installation and create different conda environments, that is what conda is great at.
Using conda from your anaconda installation to manage packages from another, independent python installation is not possible and not very feasible.
Something like this could serve your needs:
Create one env for python 3.5 conda create -n py35 python=3.5
Create one env for some other python version you would like to use, e.g. 3.6: conda create -n py36 python=3.6
Use conda activate py35, conda deactivate, conda activate py36 to switch between your virtual environments.
Related
I have an Anaconda installation on Windows 10 primarily to run Jupyter Notebook and Spyder.
I find the Python runtime (python.exe) in 3 places after installation, viz:
D:\ProgramData\Anaconda3
D:\ProgramData\Anaconda3\pkgs\python-3.7.6-h60c2a47_2
D:\ProgramData\Anaconda3\pkgs\python-3.8.2-he1778fa_13
The python.exe under 1. and 2. are identical and run Python 3.7. 3. runs Python 3.8.
Questions:
What is the rationale of having two versions under pkgs (as in 2. and 3. above) but just one default version (as in 1.)?
The contents under the pkgs directory - are they complete Python installations?
What is the best way to make Jupyter Notebook pickup Python 3.8? It currently picks up Python 3.7 because the location 1. is in PATH. (That is, are the pkgs directories full installations)?
If I want to work with Python 3.9, is there a way to upgrade the current Ananconda to that extent? Or, do I have to delete the current Ananconda3 and install the latest Anaconda provided, of course it supports Python 3.9?
What is the rationale of having two versions under pkgs (as in II and III above) but just one default version (as in I)?
The contents under the pkgs directory - are they complete Python installations?
The pkgs folder is only a type of cache where packages that conda downloads and decompresses are kept so that they can be installed more quickly into new environments, so no they are not complete python installations ready to be used. There can only be one python version in one environment, in your case D:\ProgramData\Anaconda3\python.exe is the one that belongs to the base environment
If I want to work with Python 3.9, is there a way to upgrade the current Ananconda to that extent? Or, do I have to delete the current Ananconda3 and install the latest Anaconda provided, of course it supports Python 3.9?
To install a different python version into the current environment, simply do conda install python=<version>. You can use conda search python to check the available versions, or see on the website that the default channel has 3.9 as a newest version. However upgrading your base will most likely fail. Anaconda comes with a huge list of preinstalled packages and python 3.9 is too new, so that conda will not be able to resolve dependencies with newer python versions. The newest anaconda installer comes with python 3.8. only
What is the best way to make Jupyter Notebook pickup Python 3.8?
To have multiple python installations, use virtual environments (as there can only be one python version in one environment) which is very easy to use:
conda create -n py39 python=3.9
conda create -n py38 python=3.8
conda create -n py37 python=3.7
would create three environments that you can selectively activate with
conda activate py37 #or
conda activate py38 #or
conda activate py39
To use an environment, you need to activate it and then you can also install packages for that environment, e.g. to set up jupyter for one of them, simply do
conda activate py37
conda install ipykernel jupyter
python -m ipykernel install --user --name py37 --display-name "Python 3.7"
then you can start jupyter as you are used to and select Python 3.6 as the kernel.
Note that for each environment you will need to install all packages again, there is no cross-talk between them, so doing
conda activate py37
conda install numpy
will install numpy only to the py37 env, not to base, py38 or py39
Conda allows me to list all virtual environments as shown here. The commands are:
conda info --envs OR conda env list
I want to do that using pip. Does pip have any option to list all virtual environments created by me ? I have created a virtual environment on my desktop but I cannot figure out a way to list it along with the base environment.
No, not in an easy way.
Python virtual environments created with venv, virtualenv and most other python-only virtual environments can be stored anywhere on the disk. And AFAIK they are not indexed, they are truly isolated (after all you can just remove venv directory and be done with it, you don't need to do anything special). They are also unmanaged by an environment manager. So that would require entire disk scan. Which potentially can be done (you can search for all Python executables for example) but is rather painful.
It works with miniconda because miniconda manages other packages and files that it installs, so it places venvs in concrete path, e.g. /home/username/miniconda/envs/.
One indirect way is to run below command and see the directories where the python executable resides. Some of the paths will be python virtual environments and you can verify it by listing the files in those paths.
$ whereis python
You can use conda to create and manage venv and virtualenv environments and other packages installed using pip.
First create a conda environment with CONDA AND PIP installed into it, e.g.,
conda create --name core --channel conda-forge python=3.9 conda pip
Here I created the conda environment named "core" and installed Python 3.9, conda, and pip into it. So now the 'core' conda environment functions like an administrative environment shell. By installing conda into the conda environment, conda will track packages installed by pip into that environment. You must use "pip install" INSIDE this new conda environment, so conda will index and track those pip package installations. However, conda will still not index and centrally manage the venv environments, like it does for its own conda environments.
Here is a very good detailed guide that explains how and why to use conda and pip for virtual environments. It covers the important aspect of using conda and pip together.
Why You Need Python Environments and How to Manage Them with Conda
Create virtual environments for python with conda
I want to install Anaconda with Python Version 3.6.5. If I install Anaconda3-5.2.0, It install Python 3.5.1. Where to download the Anaconda with Python 3.6.5. The Big Data Scripts work only with Anaconda Python 3.6.5.
Anaconda Downloads
The Anaconda distribution with Python 3.6.5 was version 5.2.0.1 You can download this from the Anaconda distribution archive. If you do install from this, then make sure to update Conda immediately after installation:
conda update conda
However, I strongly recommend the following alternate solution as better practice.
Miniconda + Anaconda environment
Reasoning
What is installed in the base environment is relatively fixed once installed. Ultimately, you don't want to mess with your base environment, so best practice is to have the latest version there. Fortunately, you don't have to install a full Anaconda distribution, but rather can use a lightweight Miniconda (or Miniforge) distribution and create a secondary environment for the purpose of having an Anaconda Python 3.6.5 distribution. In the long run this will give you better stability.
Steps
Download and install Miniconda or a Miniforge variant. Once that is working...
Create your Anaconda env:
conda create --name my_env -c anaconda python=3.6.5 anaconda=5.2.0
Use your new isolated env:
conda activate my_env
[1] I determined this by running conda create -n foo --dry-run -c anaconda python=3.6.5 anaconda and then examining the version of the anaconda package that Conda ended up with in the solve.
Also try
conda install python=3.6.5
but you may encounter some incompatibility issues with other packages.
Alternatively, you may want to try creating a new environment. From the anaconda prompt, create a custom environment and specify the repository channel to find the version
conda create --name py365 python=3.6.5 --channel conda-forge
Activate the new environment
conda activate py365
However, the activation will not be permanent, and you will need to activate each time you start the anaconda prompt.
In your anaconda prompt, you can manually update your python to the latest version with :
conda update python
In case you are not familiar with it, anaconda prompt is installed to your computer when you install anaconda. Just make a search for it on your computer.
You can refer to this post : How do I upgrade to Python 3.6 with conda?
I was investigating the use of Anaconda environments for CI/CD (since, to my knowledge, it is the only platform that supports Linux, MacOS, and Windows). I tried to use Miniconda which is supposed to only install the bare minimum. However, I realised that, by default, Miniconda is not "mini" after all. For example, if I attempt to create a new Python environment (conda create -n py36 python=3.6 anaconda), it will install a bunch of not needed stuff like JupyterLab and others. So, before moving to pyenv (for Linux and MacOS) and pyenv-win (for Windows), I would like to ask:
Is there a way to setup different python environments with anaconda/miniconda without having to install a bunch of extra packages every time I create a new environment?
Is there any other tool for managing python environments that supports Linux, MacOS, and Windows?
Thank you.
Only install python and its dependencies by
conda create -n py36 python=3.6
without the anaconda package.
Detailed Explanation
conda create -n py36 python=3.6
conda create -n py36, create an environment, actually an empty folder
python=3.6, installed python 3.6 into this env
conda is a package manager, both python and anaconda are packages could be installed by it.
Unlike package python, anaconda is a meta package, which does not contain actual software and simply depends on other packages to be installed.
Download an anaconda package here and extract content from it. The actual packages to be installed is listed in info/recipe/meta.yaml.
package:
name: anaconda
version: '2019.07'
build:
ignore_run_exports:
- '*'
number: '0'
pin_depends: strict
string: py37_0
requirements:
build:
- python 3.7.3 h8c8aaf0_1
is_meta_pkg:
- true
run:
- alabaster 0.7.12 py37_0
- anaconda-client 1.7.2 py37_0
- anaconda-project 0.8.3 py_0
# ...
# about 260 packages in total
You want virtualenv: https://virtualenv.pypa.io/en/latest/
$ virtualenv env --python "[path to python version]"
This will create an environment from the python base you chose in the previous command, in a folder called 'env'. There will be no additional packages installed save pip and a few other core ones.
You then need to 'activate' the environment - this changes based on operating system. For windows;
$ env\Scripts\activate
You will then have the command prompt;
(env) $
Showing it's activated. You can then use pip install as normal to install whatever requirements you need into that environment (they will live inside the env folder). To leave the environment;
(env) $ deactivate
You can have as many as you need, and define different python versions and requirements. Just remember to activate the environment before installing packages.
I want to work with the python package holopy. Apparently you have to use conda to install it, so I first installed Anaconda 4.2.0 (since I'm using Python 3.5). I opened the virtual environment I normally use and installed holopy as they recommend on the official site:
conda install -c conda-forge holopy
Afterwards, when in the virtual environment I type conda list, holopy shows up. But when I type python3 and then import holopy, it says package not found. It does however work when I leave the virtual environment. I need it in the virtual environment though, how can I do that?
I'm not sure how well anaconda and virtual environments i.e.venv work together. If you're using anaconda anyway then I highly recommend using anaconda environments. Please go through this short tutorial about anaconda environments - you won't regret it.
Why it didn't work for you?
The conda command is available only in the base anaconda environment. So when you run the command - conda insall -c conda-forge holopy, it installed holopy in the base anaconda environment and it won't be available to you in your venv.
After looking at the documentation of holopy it seems probable that when they said virtual environment they actually meant anaconda virtual environment. Therefore the solution is to first create an anaconda virtual environment called holopy-env and then run the command conda install -n holopy-env -c conda-forge holopy.
A better way of doing things with Anaconda
I will also give you a quick and clean example of how to create an environment using anaconda. If you're using Anaconda then it would be wise to use it's environment management tools. Create an environment.yml file with the following contents:
environment.yml using conda-forge/holopy & python 3.6
name: holopy-env # any name for the environment
channels:
- conda-forge
dependencies: # everything under this, installed by conda
- python=3.6
- holopy
- pip: # everything under this, installed by pip
- future
How to install the environment?
conda create --force -f environment.yml
How to activate the environment?
source activate opencv-env
After activating the environment
You should be able to import holopy
Install pip packages using pip install <package>
Install conda packages using conda install -n holopy-env -c CHANNEL <package>
conda is a packaging tool and installer that aims to do more than what pip can do; handle library dependencies outside of the Python packages as well as the Python packages themselves. Conda also creates a virtual environment, like virtualenv does. For creating virtualenv with conda, use the following command:-
conda create -n yourenvname python=x.x anaconda
Use the following to activate the virtualenv in conda
source activate yourenvname
Then, you can install the packages in virtualenv using conda as:-
conda install -n yourenvname [package]
To Deactivate use:-
source deactivate
And to delete a no longer needed virtualenv, use :-
conda remove -n yourenvname -all
I know this is a bit late, but you don't need to use conda to install HoloPy. This is just the least technical option. Otherwise, you need to be able to compile HoloPy's fortran components yourself, which is fairly straightforward on Unix-based systems but complicated on Windows. Instructions can be found in HoloPy's documentation at https://holopy.readthedocs.io/en/latest/users/dev_tutorial.html.
We are also working on putting together a singularity container distribution of HoloPy. Let me know if this is of interest to you and I will make it a priority.