I have installed anaconda and the base environment is using the anaconda python:
Command type python returns /home/ya/anaconda3/bin/python.
However, when I create a new environment, the default python under the new environment is the /usr/bin/python. How can I change it?
The information below is also covered in the documentation on creating environments.
New Environments are Empty
Conda is a general package and environment manager, not a Python-specific one. Hence, when creating a new environment it does not assume that one wants Python. Instead, if one wants Python installed, it must be specified, e.g.,
conda create -n foo python
Note that the create command can also take version constraints, as well as additional packages. It is best to specify all packages one expects at the time of creation as this simplifies the solving.
Default Packages
The create_default_packages configuration option is available to specify a default set of packages that will always be installed at creation to time. For example, if you know that you will always want Python installed, one could use:
conda config --add create_default_packages python
Note that version can be specified here as well. All subsequent environment creations will then install Python.
One can also subsequently override such defaults with the flag --no-default-packages.
Related
I have several Python projects using different environments. These environments are managed using Conda and this works well, allowing the same environment to be used in production and dev/test for each project.
Conda yml files are used to define each environment.
There are a number of packages that I would like to use during development, such as autopep8. These don't need to be in the production environment so are not included in the yml file.
How can I install autopep8 and others so that they will work across any Python environment that I load in VS Code? So far I have had to manually install these packages as I switch environments.
Default Packages
One way of managing this without violating environment isolation1 would be to use Conda's default packages functionality. The idea would be to define default packages (such as autopep8) in a .condarc on only the development systems. The conda env create will respect these and add them to every env you create, so you can still keep a single YAML that describes only the essentials for the production version.
Note that there are multiple options for where to store this .condarc, and Conda can load settings in a nested fashion. If all environments for your user are categorized as "development", then a sensible place to define the default packages would be ~/.condarc. There is additionally a --no-default-packages flag, which can be used to disable such default package installation when you don't need it.
[1] While there are ways to include packages from outside a Conda environment (e.g., through PYTHONPATH), this should be regarded as substandard and only be used as a last resort. Conda is designed with an assumption of full isolation of environments - violating that can lead to undefined behavior.
The typical command to export a Anaconda environment to a YAML file is:
conda env export --name my_env > myenv.yml
However, one huge issue is the readbility of this file as it includes hard specifications for all of the libraries and all of their dependencies. Is there a way for Anaconda to export a list of the optimally smallest subset of commands that would subsume these dependencies to make the YAML more readable? For example, if all you installed in a conda environment was pip and scipy, is there a way for Anaconda to realize that the file should just read:
name: my_env
channels:
- defaults
dependencies:
- scipy=1.3.1
- pip=19.2.3
That way, the anaconda environment will still have the exact same specification, if not an improved on (if an upstream bug is fixed) and anyone who looks at the yml file will understand what is "required" to run the code, in the sense that if they did want to/couldn't use the conda environment they would know what packages they needed to install?
Options from the Conda CLI
This is sort of what the --from-history flag is for, but not exactly. Instead of including exact build info for each package, it will include only what are called explicit specifications, i.e., the specifications that a user has explicitly requested via the CLI (e.g., conda install scipy=1.3.1). Have a try:
conda env export --from-history --name my_env > myenv.yml
This will only include versions if the user originally included versions during installation. Hence, creating a new environment is very likely not going to use the exact same versions and builds. On the other hand, if the user originally included additional constraints beyond version and build they will also be included (e.g., a channel specification conda install conda-forge::numpy will lead to conda-forge::numpy).
Another option worth noting is the --no-builds flag, which will export every package in the YAML, but leave out the build specifiers. These flags work in a mutually exclusive manner.
conda-minify
If this is not sufficient, then there is an external utility called conda-minify that offers some functionality to export an environment that is minimized based on a dependency tree rather than through the user's explicit specifications.
Have a look at pipreqs. It creates a requirements.txt file only based on the imports that you are explicitely doing inside your project (and you even have a --no-pin option to ignore the version numbers). You can later use this file to create a conda environemnt via conda install --file requirements.txt.
However, if you're aiming for an evironments.yml file you have to create it manually. But that's just copy and paste from the clean requirements.txt. You only have to separate conda from "pip-only" installs.
I want to start using Anaconda the correct way, by making a new environment for each project instead of always using base. I completely reinstalled Anaconda. As far as making my first environment, do I need to specify every module that I want or are the basic ones that come with python (like OS and MATH) included?
The standard library should be included with all python installs (which also applies to conda environments when you specify python or a python version). Here you can find what is included in the standard library.
Creating a new environment in conda the following way:
conda create --name <name> python=3.x.y
or
conda create --name <name> python
will install python in your environment.
You can reference this nice cheatsheet for more conda commands.
I have Anaconda and I've been playing around with setting up virtual environments since I have scripts that I need to run that have been written in Python 2 and Python 3. I want to be able to activate my Python 3 virtual environment in a specific directory - ie Python 3 will only run in that directory and all other directories will remain at the default Python 2 that I have set in Anaconda. My problem is that every time I try to activate a new environment, it changes the version of python used everywhere on my machine instead of just in the one directory that I want. Is it possible to create a virtual environment that is limited to a specific location?
I've tried the following:
conda create --prefix=testEV1 python=3.5
source activate testEV1
and this changes my version of python everywhere in my workspace to 3.5.
No. You only have a single default Python installation in effect at any one time.
Once you're done using one virtualenv you can use deactivate to go "back" to the standard, physical default Python installation.
Or you can use different command sessions with different virtual environments activated in each session.
Or you can explicitly invoke one version or another of Python from the command line each time, rather than just using the currently active-by-default one.
It is possible to autoactivate a conda environment when entering a specific directory.
https://github.com/conda/conda/issues/5179
BUT, it doesn't change the fact that source activate xxx affect your shell/prompt instead of your directory structure. You can still manually activate an environment and it will still affect your whole prompt.
I am a new user to conda environment and was setting up to use TensorFlow , on Windows.
I came across a command -
source activate IntroToTensorFlow.
I understood IntroToTensorFlow is an environment we are creating, but does it mean we need to create this environment every time?. I am using jupyter notebook, so if I shutdown the kernel will the environment get deactivated?
And if I restart my PC, should I activate the environment everytime ?
Conda is a package manager that installs and manages (usually) Python libraries and (sometimes) non-Python packages. A conda environment is a sort of virtualenv virtual environment; its typical use case is to have a Python interpreter (any version) along with your choice of compatible Python libraries (any version).
The following example might most likely pertain to you. Suppose you have downloaded the implementation of a very nice paper implemented in TF and you want to try it out. But the authors implemented that when Tensorflow was just growing. The APIs have changed now, and so is the required CUDA version. You want to work ideally on the latest TF. Now, what do you do? An easy way to just try out this implementation is to create a different conda environment with the libraries needed for that implementation, run that in this environment, and perhaps if you like it, you might consider upgrading the TF APIs and use it in your code.
The conda environments are also pretty simple in its construction. If you installed conda using Anaconda and default options, you will have your environments in ~/anaconda3/envs. The environments are nothing but directories here, each having various configurations of Python interpreter and libraries of your choice. (So when you shutdown your PC/Jupyter, the environments will of course persist.) At the time of usage, you just switch between the environments to suit your needs. That is, when you source activate an environment, you will be allowed to use the Python interpreter and installed libraries from that environment. Note if you source deactivate or start a new terminal session, you will still be using the root environment.
Besides, Jupyter notebook, if setup with this plugin, will allow you to have nice integration with conda environments and you wouldn't even need to source activate everytime you want to switch. You can choose between the various settings (or conda environments), which are interpreted as different kernels in the notebook. So it would as simple as choosing some environment using a drop-down.
source activate IntroToTensorFlow does not create an environment, it simply activates an environment that has already been created. To create that environment (with tensorflow installed), use conda create -n IntroToTensorFlow tensorflow.
You do not need to create the environment every time, but you do need to activate it every time in order to use the packages installed in it. This is done using source activate IntroToTensorFlow
If you shutdown a kernel, the environment does not get deactivated automatically. To do so, you have to explicitly say source deactivate, or activate a separate environment using source activate xxx, replacing xxx with whatever environment name you want (that you have created previously).
When restarting your PC, (or starting a new session at the command line), you have to manually activate your desired environment to use it. Otherwise, by default, it will be running in your root environment. So, if you've only installed tensorflow in IntroToTensorFlow environment, you have to use source activate IntroToTensorFlow every time in order to use it.
Take a look here for more info