I would like to use Python for scientific applications and after some research decided that I will use Anaconda as it comes bundled with loads of packages and add new modules using conda install through the cmd is easy.
I prefer to use the 64 bit version for better RAM use and efficiency but
32bit version is needed as well because some libraries are 32bit. Similarly, I prefer to use Python 3.5 as that is the future and the way things go. But loads of libraries are still 2.7 which means I need both.
I have to install 4 versions of Anaconda (64bit 2.7, 64bit 3.5, 32bit 2.7, 64bit 3.5). Each version is about 380MB. I am aiming to use Jupyter notebook and Spyder as the IDE. I had to switch between versions when required. I had conflicting libraries, path issues and all sorts of weird problems.
So, I am planning to do a clean install from scratch. I would like to know if there is a more sensible way to handle this. I use Windows 7 64 bit for now if that matters.
Make sure to set the right environmental variables (https://github.com/conda/conda/issues/1744)
Create a new environment for 32bit Python 2.7:
set CONDA_FORCE_32BIT=1
conda create -n py27_32 python=2.7
Activate it:
set CONDA_FORCE_32BIT=1
activate py27_32
Deactivate it:
deactivate py27_32
Create one for 64 bit Python 3.5:
set CONDA_FORCE_32BIT=
conda create -n py35_64 python=3.5
Activate it:
set CONDA_FORCE_32BIT=
activate py35_64
The best would be to write the activation commands in a batch file so that you have to type only one command and cannot forget to set the right 32/64 bit flag.
UPDATE
You don't need to install a full Anaconda distribution for this. Miniconda is enough:
These Miniconda installers contain the conda package manager and Python. Once Miniconda is installed, you can use the conda command to install any other packages and create environments, etc. ...
There are two variants of the installer: Miniconda is Python 2 based and Miniconda3 is Python 3 based. Note that the choice of which Miniconda is installed only affects the root environment. Regardless of which version of Miniconda you install, you can still install both Python 2.x and Python 3.x environments.
I would recommend you to use Miniconda3 64-bit as your root environment.
You can always install a full Anaconda later with:
conda install anaconda
Note that it might downgrade some of your previously install packages in your active environment.
Setting the Subdirectory Constraint
Conda has a configuration variable subdir that can be used to constrain package searching to platforms (e.g., win-32). I think the most reliable procedure is to create the empty environment, set its subdir, then proceed with the (constrained) installations. For example,
win-32, Python 2.7
conda create -n py27_32
conda activate py27_32
conda config --env --set subdir win-32
conda install python=2.7
win-64, Python 3.7
conda create -n py37_64
conda activate py37_64
conda config --env --set subdir win-64
conda install python=3.7
Alternatively, if you need to, for example, create an environment from a YAML file, but want a win-32 platform, one can use the CONDA_SUBDIR environment variable:
set CONDA_SUBDIR=win-32
conda env create -f env.yaml -n my_env_32
set CONDA_SUBDIR=
conda activate my_env_32
conda config --env --set subdir win-32
The nice thing about this procedure is the variable will now always be set whenever activating the environment, so future changes to the environment will remain within the specified subdirectory.
Ad Hoc Constraints
It is also possible to specify the platform in the --channel|-c argument:
conda install -c defaults/win-32 --override-channels python=3.7
Here the --override-channels is required to ensure that only the provided channel(s) and subdirectory (win-32) is used.
However, setting the subdir on the whole env is likely a more reliable practice.
YAML Constraints
It is also possible to use subdir specifications in a YAML environment definition. However, this is less reliable (see below and comments). For example,
py37_win32.yaml
name: py37_win32
channels:
- defaults/win-32
dependencies:
- python=3.7
#Bicudo has tried this and confirms it works, but notes that it does not set any environment-specific constraints on future updates to the environment. Additionally, #Geeocode pointed out that the default subdir can still leak in, which is likely due to conda env create still having access to the global channels configuration during solving (there is no equivalent --override-channels flag for conda env create). Hence, it would be good practice to still set the subdir before and after environment creation, e.g.,
set CONDA_SUBDIR=win-32
conda env create -f py37_win32.yaml
set CONDA_SUBDIR=
conda activate py37_win32
conda config --env --set subdir win-32
Alternatively, beginning with Conda v4.9, one can also specify environment variables as part of the YAML. That is, one can effectively define an environment's CONDA_SUBDIR value at environment creation:
py37_win32.yaml
name: py37_win32
channels:
- defaults/win-32
dependencies:
- python=3.7
variables:
CONDA_SUBDIR: win-32
I just wanted to add to Mike Mullers answer, as I also wanted my IPython to switch between 32 bit and 64 bit.
After setting up the 32bit or 64bit environment. Use the following commands
pip install ipykernel
to install ipykernel on this env. Then assign it with:
python -m ipykernel install --user --name myenv --display-name "Python (myenv)"
here myenv is the name of your new environment. See this page here for further details on switching kernels - http://ipython.readthedocs.io/en/stable/install/kernel_install.html
(now in conda win64 - python64 activate env)
set CONDA_SUBDIR=win-32
conda install python
you will see
The following packages will be SUPERSEDED by a higher-priority
channel:
ca-certificates anaconda/pkgs/main/win-64::ca-certifi~ -->
anaconda/pkgs/main/win-32::ca-certificates-2021.7.5-h9f7ea03_1
openssl anaconda/pkgs/main/win-64::openssl-1.~ -->
anaconda/pkgs/main/win-32::openssl-1.1.1k-hc431981_0 python
anaconda/pkgs/main/win-64::python-3.9~ -->
anaconda/pkgs/main/win-32::python-3.9.5-h53c7b84_3
Proceed ([y]/n)?
just type "y"
this setting is saved in file "\anaconda\envs\ you env \ .condarc"
Related
How can I create a exclusively 32 bits conda environment? I tried:
set CONDA_FORCE_32BIT=1
conda create -n py310_32 python=3.10.5
But it didn't work.
Follwing commands will successfully get a 32-bit python. I suppose the main problem is the environment variable. You know windows is this. :(
conda create -n py27_32
conda activate py27_32
conda config --env --set subdir win-32
conda install python=2.7
useful links:
Using multiple Python engines (32Bit/64bit and 2.7/3.5)
How can I have two different environments within Anaconda? (both Python 3.7, one 32bit and one 64bit)
https://github.com/conda/conda/issues/1744
I recently installed Anaconda in my Windows. I did that to use some packages from some specific channels required by an application that is using Python 3.5 as its scripting language.
I adjusted my PATH variable to use Conda, pointing to the Python environment of the particular program, but now I would like to use Conda as well for a different Python installation that I have on my Windows.
When installing Anaconda then it isn't asking for a Python version to be related to. So, how can I use Conda to install into the other Python installation. Both Python installations are 'physical' installations - not virtual in any way.
Uninstall the other python installation and create different conda environments, that is what conda is great at.
Using conda from your anaconda installation to manage packages from another, independent python installation is not possible and not very feasible.
Something like this could serve your needs:
Create one env for python 3.5 conda create -n py35 python=3.5
Create one env for some other python version you would like to use, e.g. 3.6: conda create -n py36 python=3.6
Use conda activate py35, conda deactivate, conda activate py36 to switch between your virtual environments.
I was investigating the use of Anaconda environments for CI/CD (since, to my knowledge, it is the only platform that supports Linux, MacOS, and Windows). I tried to use Miniconda which is supposed to only install the bare minimum. However, I realised that, by default, Miniconda is not "mini" after all. For example, if I attempt to create a new Python environment (conda create -n py36 python=3.6 anaconda), it will install a bunch of not needed stuff like JupyterLab and others. So, before moving to pyenv (for Linux and MacOS) and pyenv-win (for Windows), I would like to ask:
Is there a way to setup different python environments with anaconda/miniconda without having to install a bunch of extra packages every time I create a new environment?
Is there any other tool for managing python environments that supports Linux, MacOS, and Windows?
Thank you.
Only install python and its dependencies by
conda create -n py36 python=3.6
without the anaconda package.
Detailed Explanation
conda create -n py36 python=3.6
conda create -n py36, create an environment, actually an empty folder
python=3.6, installed python 3.6 into this env
conda is a package manager, both python and anaconda are packages could be installed by it.
Unlike package python, anaconda is a meta package, which does not contain actual software and simply depends on other packages to be installed.
Download an anaconda package here and extract content from it. The actual packages to be installed is listed in info/recipe/meta.yaml.
package:
name: anaconda
version: '2019.07'
build:
ignore_run_exports:
- '*'
number: '0'
pin_depends: strict
string: py37_0
requirements:
build:
- python 3.7.3 h8c8aaf0_1
is_meta_pkg:
- true
run:
- alabaster 0.7.12 py37_0
- anaconda-client 1.7.2 py37_0
- anaconda-project 0.8.3 py_0
# ...
# about 260 packages in total
You want virtualenv: https://virtualenv.pypa.io/en/latest/
$ virtualenv env --python "[path to python version]"
This will create an environment from the python base you chose in the previous command, in a folder called 'env'. There will be no additional packages installed save pip and a few other core ones.
You then need to 'activate' the environment - this changes based on operating system. For windows;
$ env\Scripts\activate
You will then have the command prompt;
(env) $
Showing it's activated. You can then use pip install as normal to install whatever requirements you need into that environment (they will live inside the env folder). To leave the environment;
(env) $ deactivate
You can have as many as you need, and define different python versions and requirements. Just remember to activate the environment before installing packages.
I installed brew, python3 (default and latest version) and pip3, pyenv.
TensorFlow does not support python3.7 now, so I heard that I should make a virtualenv that runs 3.6 or lower version independently.
I installed python 3.6.7 by pyenv install 3.6.7 but can't make virtualenv -p 3.6.7 (mydir) because 3.6.7 is not in the PATH (usr/local/bin).
How can I update my PATH?
You don't need the executable to be on the PATH. Assuming you want /usr/local/bin/python3.6.7 to be used in the virtual environment,
virtualenv -p /usr/local/bin/python3.6.7 mydir
Updating your PATH is easy:
PATH=/usr/local/bin:$PATH
This will only update it in your current session; you might want to add this to your shell's startup files to make it permanent. This is a common FAQ but depends on a number of factors (your shell, etc) so google for details. Here is one question with several popular variants in the answers: Setting PATH environment variable in OSX permanently
I know that this doesn't answer the question exactly, but for completeness I'd like to add an Anaconda solution. Provided that an Anaconda environment is present on the system, a new Python environment can be created using conda create -n py36 python=3.6 pip. The name py36 can be arbitrarily chosen (could also be e.g. myenv or tensorflow), the desired Python version (in this example 3.6) is specified by python=3.6.
This environment can then be activated using conda activate py36 (or whatever name you assigned in the previous step). Once the environment is active, you can install tensorflow via pip: pip install tensorflow-gpu. To deactivate the current environment and return to the default environment, use conda deactivate. In this way, you don't have to modify PATH variables.
See also this documentation page for more details on the Anaconda environment.
I want to work with the python package holopy. Apparently you have to use conda to install it, so I first installed Anaconda 4.2.0 (since I'm using Python 3.5). I opened the virtual environment I normally use and installed holopy as they recommend on the official site:
conda install -c conda-forge holopy
Afterwards, when in the virtual environment I type conda list, holopy shows up. But when I type python3 and then import holopy, it says package not found. It does however work when I leave the virtual environment. I need it in the virtual environment though, how can I do that?
I'm not sure how well anaconda and virtual environments i.e.venv work together. If you're using anaconda anyway then I highly recommend using anaconda environments. Please go through this short tutorial about anaconda environments - you won't regret it.
Why it didn't work for you?
The conda command is available only in the base anaconda environment. So when you run the command - conda insall -c conda-forge holopy, it installed holopy in the base anaconda environment and it won't be available to you in your venv.
After looking at the documentation of holopy it seems probable that when they said virtual environment they actually meant anaconda virtual environment. Therefore the solution is to first create an anaconda virtual environment called holopy-env and then run the command conda install -n holopy-env -c conda-forge holopy.
A better way of doing things with Anaconda
I will also give you a quick and clean example of how to create an environment using anaconda. If you're using Anaconda then it would be wise to use it's environment management tools. Create an environment.yml file with the following contents:
environment.yml using conda-forge/holopy & python 3.6
name: holopy-env # any name for the environment
channels:
- conda-forge
dependencies: # everything under this, installed by conda
- python=3.6
- holopy
- pip: # everything under this, installed by pip
- future
How to install the environment?
conda create --force -f environment.yml
How to activate the environment?
source activate opencv-env
After activating the environment
You should be able to import holopy
Install pip packages using pip install <package>
Install conda packages using conda install -n holopy-env -c CHANNEL <package>
conda is a packaging tool and installer that aims to do more than what pip can do; handle library dependencies outside of the Python packages as well as the Python packages themselves. Conda also creates a virtual environment, like virtualenv does. For creating virtualenv with conda, use the following command:-
conda create -n yourenvname python=x.x anaconda
Use the following to activate the virtualenv in conda
source activate yourenvname
Then, you can install the packages in virtualenv using conda as:-
conda install -n yourenvname [package]
To Deactivate use:-
source deactivate
And to delete a no longer needed virtualenv, use :-
conda remove -n yourenvname -all
I know this is a bit late, but you don't need to use conda to install HoloPy. This is just the least technical option. Otherwise, you need to be able to compile HoloPy's fortran components yourself, which is fairly straightforward on Unix-based systems but complicated on Windows. Instructions can be found in HoloPy's documentation at https://holopy.readthedocs.io/en/latest/users/dev_tutorial.html.
We are also working on putting together a singularity container distribution of HoloPy. Let me know if this is of interest to you and I will make it a priority.