I want to export an environment AND its packages, using conda 4.10.
Reading the conda docs, it suggests exporting environments using conda env export > environment.yml. However, I am not sure if it is my problem (and if so, what the solution is), but there is no package information.
name: guest
channels:
- defaults
prefix: C:\Anaconda3\envs\guest
After some googling, I learnt to export packages using conda list --export > requirements.txt. This time, there is no environment information.
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: win-64
ca-certificates=2021.5.30=h5b45459_0
certifi=2021.5.30=py38haa244fe_0
...
How do I export both into one file and use it? Or should I just export two files, create an environment first, and install the packages?
On a side note, how do I make my packages match requirements.txt, that is, to remove extra packages, install missing ones, and update/ downgrade to the specific version? Is there a command for this, or should I delete the whole environment and start from scratch?
Related
I have an environment.yml file, which contains the necessary packages required to build a Docker container. Though, one of the packages I am using is installed locally (--offline flag is passed).
Shell file, which installs necessary packages looks like this right now:
conda env update -n base -f ./environment.yml
conda install --offline <LOCAL_PACKAGE.tar.bz2>
Is it possible to somehow add this package inside my environment.yml file?
I am trying to update the package flopy, within a virtual environment called flopyenv using the Anaconda Prompt command line. First, I activate the virtual environment using conda activate flopyenv. Then to update flopy, I've tried conda update flopy. I get the following error:
PackageNotInstalledError: Package is not installed in prefix.
prefix: C:\Users\person\Anaconda3\envs\flopyenv
package name: flopy
which makes sense since the flopy directory was installed in a different directory (C:\Users\person\Anaconda3\envs\flopyenv\lib\site-packages\flopy). Also, I have checked using conda list and flopy is listed in the environment. How do I point conda update to the proper directory to update flopy within the virtual environment?
Edit: Per merv's comment I've included the output below.
(flopyenv) C:\Users\person>conda list -n flopyenv flopy
# packages in environment at C:\Users\person\Anaconda3\envs\flopyenv:
#
# Name Version Build Channel
flopy 3.3.1 pypi_0 pypi
Looks like I used pip to install flopy not conda which I guess is why the directories weren't lining up when I tried updating using conda. I was able to successfully update the flopy package using pip.
Seems like OP figured it out, but it may be worth mentioning that in addition to using pip to update, it might also work to enable the pip_interop_enabled configuration option. I would only do this on a per-environment basis:
conda activate flopyenv
conda config --env --set pip_interop_enabled true
conda update flopy
However, this is still (as of Conda v 4.9) considered an experimental feature, AFAIK.
I have a python 3.4 environment with very specific packages installed. Is there a straight forward way to create an anaconda environment will all the same packages without creating a python environment then manually installing the packages. e.g..
conda create -n myenv python=3.4
conda activate myenv
conda install beautifulsoup4==4.6.1
I'd like to do this as i want to share a python script and make it as easy as possible for others to run it.
These are all the packages installed in my python3.4 environment
'altgraph==0.16.1',
'asn1crypto==0.24.0',
'beautifulsoup4==4.6.1',
'certifi==2018.4.16',
'cffi==1.11.5',
'chardet==3.0.4',
'configargparse==0.13.0',
'cryptography==2.3',
'defusedxml==0.5.0',
'dis3==0.1.3',
'future==0.17.1',
'h5py==2.8.0',
'idna==2.7',
'jira==2.0.0',
'macholib==1.11',
'numpy==1.15.3',
'oauthlib==2.1.0',
'pbr==4.2.0',
'pefile==2018.8.8',
'pycparser==2.18',
'pyinstaller==3.4',
'pyjwt==1.6.4',
'pyqt4==4.11.4',
'pywin32-ctypes==0.2.0',
'pywin32==220',
'regex==2018.07.11',
'requests-oauthlib==1.0.0',
'requests-toolbelt==0.8.0',
'requests==2.19.1',
'six==1.11.0',
'urllib3==1.23',
'xmltodict==0.11.0'
conda-env is your friend:
conda-env export -n orig > orig.yml
where orig is the name of the environment you want to clone, followed by
conda-env create -n new -f=orig.yml
where new is the new name of the copy. The file orig.yml contains everything about the environment you want to copy: dependencies, installed packages incl. version, and such.
The conda docs at http://conda.pydata.org/docs/using/envs.html explain how to share environments with other people.
However, the docs tell us this is not cross platform:
NOTE: These explicit spec files are not usually cross platform, and
therefore have a comment at the top such as # platform: osx-64 showing the
platform where they were created. This platform is the one where this spec
file is known to work. On other platforms, the packages specified might not
be available or dependencies might be missing for some of the key packages
already in the spec.
NOTE: Conda does not check architecture or dependencies when installing
from an explicit specification file. To ensure the packages work correctly,
be sure that the file was created from a working environment and that it is
used on the same architecture, operating system and platform, such as linux-
64 or osx-64.
Is there a good method to share and recreate a conda environment in one platform (e.g. CentOS) in another platform (e.g. Windows)?
This answer is given with the assumption that you would like to make sure that
the same versions of the packages that you generally care about are on
different platforms and that you don't care about the exact same versions of
all packages in the entire dependency tree. If you are trying to install the
exact same version of all packages in your entire dependency tree that has a
high likelihood of failure since some conda packages have different
dependencies for osx/win/linux. For example, the recipe for
otrobopt
will install different packages on Win vs. osx/linux, so the environment list
would be different.
Recommendation: manually create an environment.yaml file and specify or pin
only the dependencies that you care about. Let the conda solver do the rest.
Probably worth noting is that conda-env (the tool that you use to manage conda
environments) explicitly recommends that you "Always create your
environment.yml file by hand."
Then you would just do conda env create --file environment.yml
Have a look at the readme for
conda-env.
They can be quite simple:
name: basic_analysis
dependencies:
- numpy
- pandas
Or more complex where you pin dependencies and specify anaconda.org channels to
install from:
name: stats-web
channels:
- javascript
dependencies:
- python=3.4 # or 2.7 if you are feeling nostalgic
- bokeh=0.9.2
- numpy=1.9
- nodejs=0.10
- flask
- pip:
- Flask-Testing
In either case, you can create an environment with conda env create --file environment.yaml.
NOTE: You may need to use .* as a version suffix if you're using an older version of conda.
Whilst it is possible to create your environment.yml file by hand, you can ensure that your environment works across platforms by using the conda env export --from-history flag.
This will only include packages that you’ve explicitly asked for, as opposed to including every package in your environment.
For example, if you create an environment and install a package conda install python=3.8 numpy, it will install numerous other dependencies as well as python and numpy.
If you then run conda env export > environment.yml, your environment.yml file will include all the additional dependencies conda automatically installed for you.
On the other hand, running conda env export --from-history will just create environment.yml with python=3.8 and numpy and thus will work across platforms.
Answer adapted from the docs.
For those interested in a solution to maintain a single environment file that can be used in Linux, macOS, and Windows, please check the conda-devenv tool at https://github.com/ESSS/conda-devenv.
conda-env export should be used used to export your complete environment to file named my_env.yml.
Check the working solution on getting only prefix on OS X instead of complete dependency including pip.
Step 1: deactivate from the environment if activated. else it will create yml file with only prefix.
Step 2: run below command to export
conda-env export -n my_env > my_env.yml
it will export every required dependency, channel and pip install in a yml file which is importable to share with others.
Step 3: run below command to import
conda-env create -n my_env -f= my_env.yml
it will create the exact environment as is on sharing fellow machine.
An aspect missing from the other answers is that the question asker mentions "spec files" and not "environment.yml" file. These are different.
Spec file
A spec file specifies the exact package URL's and is used to recreate identical environments (on the same platform).
It looks like this:
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: osx-64
#EXPLICIT
https://repo.anaconda.com/pkgs/free/osx-64/mkl-11.3.3-0.tar.bz2
https://repo.anaconda.com/pkgs/free/osx-64/numpy-1.11.1-py35_0.tar.bz2
https://repo.anaconda.com/pkgs/free/osx-64/openssl-1.0.2h-1.tar.bz2
https://repo.anaconda.com/pkgs/free/osx-64/pip-8.1.2-py35_0.tar.bz2
https://repo.anaconda.com/pkgs/free/osx-64/python-3.5.2-0.tar.bz2
https://repo.anaconda.com/pkgs/free/osx-64/readline-6.2-2.tar.bz2
https://repo.anaconda.com/pkgs/free/osx-64/setuptools-25.1.6-py35_0.tar.bz2
It can be obtained with conda list --explicit from the conda environment of interest.
To create a new environment with it one would use the conda create command:
conda create --name <env_name> --file <spec file path>
environment.yml
The environment.yml file is described well in this answer.
It can be obtained with the following commands from the conda environment of interest:
conda env export to get all packages in the current environment
conda env export --from-history to only get the packages explicitly installed (i.e. not automatically added depenedencies)
This question is quite old and conda has developed in the meantime. Perhaps the original meaning of spec file was equal to environment.yml files, but for completeness I am adding this answer.
None of these solutions worked out for me, the problem the OP has raised is about platform dependent suffixes added in the dependency making it impossible to be used it in a cross-platform way.
Turns out the solution for this is to export the environment with an additional option called no-builds
Suppose you want to export your environment from MacOS to Debian.
1.) Invoke conda env export --no-builds > env_macos.yml
2.) Invoke cp env_macos.yml env_debian.yml
3.) Move env_debian.yml to your Debian host
4.) conda env create -f env_debian.yml
Soon after you do 4, again there may be same package resolving related issues for certain packages, just remove those entries alone and invoke 4 again. Things will work.
Reference
The conda docs at http://conda.pydata.org/docs/using/envs.html explain how to share environments with other people.
However, the docs tell us this is not cross platform:
NOTE: These explicit spec files are not usually cross platform, and
therefore have a comment at the top such as # platform: osx-64 showing the
platform where they were created. This platform is the one where this spec
file is known to work. On other platforms, the packages specified might not
be available or dependencies might be missing for some of the key packages
already in the spec.
NOTE: Conda does not check architecture or dependencies when installing
from an explicit specification file. To ensure the packages work correctly,
be sure that the file was created from a working environment and that it is
used on the same architecture, operating system and platform, such as linux-
64 or osx-64.
Is there a good method to share and recreate a conda environment in one platform (e.g. CentOS) in another platform (e.g. Windows)?
This answer is given with the assumption that you would like to make sure that
the same versions of the packages that you generally care about are on
different platforms and that you don't care about the exact same versions of
all packages in the entire dependency tree. If you are trying to install the
exact same version of all packages in your entire dependency tree that has a
high likelihood of failure since some conda packages have different
dependencies for osx/win/linux. For example, the recipe for
otrobopt
will install different packages on Win vs. osx/linux, so the environment list
would be different.
Recommendation: manually create an environment.yaml file and specify or pin
only the dependencies that you care about. Let the conda solver do the rest.
Probably worth noting is that conda-env (the tool that you use to manage conda
environments) explicitly recommends that you "Always create your
environment.yml file by hand."
Then you would just do conda env create --file environment.yml
Have a look at the readme for
conda-env.
They can be quite simple:
name: basic_analysis
dependencies:
- numpy
- pandas
Or more complex where you pin dependencies and specify anaconda.org channels to
install from:
name: stats-web
channels:
- javascript
dependencies:
- python=3.4 # or 2.7 if you are feeling nostalgic
- bokeh=0.9.2
- numpy=1.9
- nodejs=0.10
- flask
- pip:
- Flask-Testing
In either case, you can create an environment with conda env create --file environment.yaml.
NOTE: You may need to use .* as a version suffix if you're using an older version of conda.
Whilst it is possible to create your environment.yml file by hand, you can ensure that your environment works across platforms by using the conda env export --from-history flag.
This will only include packages that you’ve explicitly asked for, as opposed to including every package in your environment.
For example, if you create an environment and install a package conda install python=3.8 numpy, it will install numerous other dependencies as well as python and numpy.
If you then run conda env export > environment.yml, your environment.yml file will include all the additional dependencies conda automatically installed for you.
On the other hand, running conda env export --from-history will just create environment.yml with python=3.8 and numpy and thus will work across platforms.
Answer adapted from the docs.
For those interested in a solution to maintain a single environment file that can be used in Linux, macOS, and Windows, please check the conda-devenv tool at https://github.com/ESSS/conda-devenv.
conda-env export should be used used to export your complete environment to file named my_env.yml.
Check the working solution on getting only prefix on OS X instead of complete dependency including pip.
Step 1: deactivate from the environment if activated. else it will create yml file with only prefix.
Step 2: run below command to export
conda-env export -n my_env > my_env.yml
it will export every required dependency, channel and pip install in a yml file which is importable to share with others.
Step 3: run below command to import
conda-env create -n my_env -f= my_env.yml
it will create the exact environment as is on sharing fellow machine.
An aspect missing from the other answers is that the question asker mentions "spec files" and not "environment.yml" file. These are different.
Spec file
A spec file specifies the exact package URL's and is used to recreate identical environments (on the same platform).
It looks like this:
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: osx-64
#EXPLICIT
https://repo.anaconda.com/pkgs/free/osx-64/mkl-11.3.3-0.tar.bz2
https://repo.anaconda.com/pkgs/free/osx-64/numpy-1.11.1-py35_0.tar.bz2
https://repo.anaconda.com/pkgs/free/osx-64/openssl-1.0.2h-1.tar.bz2
https://repo.anaconda.com/pkgs/free/osx-64/pip-8.1.2-py35_0.tar.bz2
https://repo.anaconda.com/pkgs/free/osx-64/python-3.5.2-0.tar.bz2
https://repo.anaconda.com/pkgs/free/osx-64/readline-6.2-2.tar.bz2
https://repo.anaconda.com/pkgs/free/osx-64/setuptools-25.1.6-py35_0.tar.bz2
It can be obtained with conda list --explicit from the conda environment of interest.
To create a new environment with it one would use the conda create command:
conda create --name <env_name> --file <spec file path>
environment.yml
The environment.yml file is described well in this answer.
It can be obtained with the following commands from the conda environment of interest:
conda env export to get all packages in the current environment
conda env export --from-history to only get the packages explicitly installed (i.e. not automatically added depenedencies)
This question is quite old and conda has developed in the meantime. Perhaps the original meaning of spec file was equal to environment.yml files, but for completeness I am adding this answer.
None of these solutions worked out for me, the problem the OP has raised is about platform dependent suffixes added in the dependency making it impossible to be used it in a cross-platform way.
Turns out the solution for this is to export the environment with an additional option called no-builds
Suppose you want to export your environment from MacOS to Debian.
1.) Invoke conda env export --no-builds > env_macos.yml
2.) Invoke cp env_macos.yml env_debian.yml
3.) Move env_debian.yml to your Debian host
4.) conda env create -f env_debian.yml
Soon after you do 4, again there may be same package resolving related issues for certain packages, just remove those entries alone and invoke 4 again. Things will work.
Reference