I am a new user to conda environment and was setting up to use TensorFlow , on Windows.
I came across a command -
source activate IntroToTensorFlow.
I understood IntroToTensorFlow is an environment we are creating, but does it mean we need to create this environment every time?. I am using jupyter notebook, so if I shutdown the kernel will the environment get deactivated?
And if I restart my PC, should I activate the environment everytime ?
Conda is a package manager that installs and manages (usually) Python libraries and (sometimes) non-Python packages. A conda environment is a sort of virtualenv virtual environment; its typical use case is to have a Python interpreter (any version) along with your choice of compatible Python libraries (any version).
The following example might most likely pertain to you. Suppose you have downloaded the implementation of a very nice paper implemented in TF and you want to try it out. But the authors implemented that when Tensorflow was just growing. The APIs have changed now, and so is the required CUDA version. You want to work ideally on the latest TF. Now, what do you do? An easy way to just try out this implementation is to create a different conda environment with the libraries needed for that implementation, run that in this environment, and perhaps if you like it, you might consider upgrading the TF APIs and use it in your code.
The conda environments are also pretty simple in its construction. If you installed conda using Anaconda and default options, you will have your environments in ~/anaconda3/envs. The environments are nothing but directories here, each having various configurations of Python interpreter and libraries of your choice. (So when you shutdown your PC/Jupyter, the environments will of course persist.) At the time of usage, you just switch between the environments to suit your needs. That is, when you source activate an environment, you will be allowed to use the Python interpreter and installed libraries from that environment. Note if you source deactivate or start a new terminal session, you will still be using the root environment.
Besides, Jupyter notebook, if setup with this plugin, will allow you to have nice integration with conda environments and you wouldn't even need to source activate everytime you want to switch. You can choose between the various settings (or conda environments), which are interpreted as different kernels in the notebook. So it would as simple as choosing some environment using a drop-down.
source activate IntroToTensorFlow does not create an environment, it simply activates an environment that has already been created. To create that environment (with tensorflow installed), use conda create -n IntroToTensorFlow tensorflow.
You do not need to create the environment every time, but you do need to activate it every time in order to use the packages installed in it. This is done using source activate IntroToTensorFlow
If you shutdown a kernel, the environment does not get deactivated automatically. To do so, you have to explicitly say source deactivate, or activate a separate environment using source activate xxx, replacing xxx with whatever environment name you want (that you have created previously).
When restarting your PC, (or starting a new session at the command line), you have to manually activate your desired environment to use it. Otherwise, by default, it will be running in your root environment. So, if you've only installed tensorflow in IntroToTensorFlow environment, you have to use source activate IntroToTensorFlow every time in order to use it.
Take a look here for more info
Related
I downloaded Anaconda and started using it on my Mac but now I am switching laptops. I will be using a Windows laptop now and I need to transfer my environments to my new laptop. How best can I do this?
I am using Python version 3.8 and was using Jupyter notebooks to run my code. But if I simply try to run the notebook on my Windows laptop I am getting one error after another (because I don't have the packages installed). Installing them one by one will take time and I don't even remember most of what I installed.
If you are working across platforms (osx-64 -> win-64) you'll need to be minimal about what packages you export from the existing environment. While Conda does have a recommended intra-platform procedure for exactly recreating environments, it does not directly translate to the cross-platform situation. Instead, try using:
conda env export --from-history > environment.yml
and then, on the new computer,
conda env create -f environment.yml
This will only export the packages that you have explicitly specified to be in the environment at some point (e.g., using conda install foo). Dependencies will be resolved automatically on the new system. This does not guarantee there still won't be packages that aren't available on Windows, but they should be less frequent and easier to resolve manually (typically by removing them from the YAML or adjusting versions).
I have two pieces of code that require different versions of python and versions of packages. I have two conda environments that allow each piece of code to work separately. It seems impossible to create an environment that will support both of them. Is there a way to switch conda environment during the run (in python code), so that I could execute one part using one environment and the second part using the second environment in the same script? The form and format of the result of the first part is definitely supported by the second part, so I don't see a reason why it can't work.
I have had success using conda within a shell script that calls the python process. e.g. something along these lines
conda activate my_env_1
python some_process.py
conda deactivate
conda activate my_env_2
python some_process_2.py
conda deactivate
You will have to enable your shell to use conda. See this Python - Activate conda env through shell script
If you want to change the environments in the same code at the same time, unfortunately as far as I know that will be painful, or just impossible with one word. In order to fix this issue, just add the libraries that your other environment has into the one that you use mainly.
You can achieve this by opening either your cmd (commnand line prompt), or just anaconda prompt:
activate yourenvironment name
pip/pip3 install modulename
I was suggested to conda create a new environment for installing tensorflow
First question, in general:
Why do environment exist in conda or in Python ? (Why) is it preferable to install a new library in a new environment ?
Here, in practice:
After install conda shell says $conda activate test will activate the test environment. Does it mean i can't access the lib in Spyder unless i activate test in conda shell ? Do i need to restart python shell to see the lib ? I can't access the lib (no module named tensorflow) and I assume it has to do with python not finding the path.
After install conda shell says $conda activate test will activate the
test environment. Does it mean i can't access the lib in Spyder unless
i activate test in conda shell ? Do i need to restart python shell to
see the lib ? I can't access the lib (no module named tensorflow) and
I assume it has to do with python not finding the path.
Have you installed TF within the environment?
I haven't used Spyder in a while, but what usually happens is that you can start a program (like Spyder or Jupyter) from an environment if you have installed the application within it and the environment is active. (Some editors/IDE like VS Code lets you choose the environment for a specific project, once it is able to discover all the environments.)
And, also usually, though perhaps not always, you will not need to restart the shell to import a library, after installing it. It's best to refer to the specific library's installation instructions for details like this.
Virtual Environment is used to manage Python packages for different projects. Using virtual environment allows you to avoid installing Python packages globally which could break system tools or other projects. You can install virtual environment using pip.
For example, say you have two projects, and each requires a different version of Tensorflow. This is a real problem for Python since it can’t differentiate between versions in the “site-packages” directory. So both say V1.1 and V2.1 would reside in the same directory with the same name.
This also allows easy clean up, once you are done with the project just delete the virtual environment.
Checkout more, https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/
I have Anaconda and I've been playing around with setting up virtual environments since I have scripts that I need to run that have been written in Python 2 and Python 3. I want to be able to activate my Python 3 virtual environment in a specific directory - ie Python 3 will only run in that directory and all other directories will remain at the default Python 2 that I have set in Anaconda. My problem is that every time I try to activate a new environment, it changes the version of python used everywhere on my machine instead of just in the one directory that I want. Is it possible to create a virtual environment that is limited to a specific location?
I've tried the following:
conda create --prefix=testEV1 python=3.5
source activate testEV1
and this changes my version of python everywhere in my workspace to 3.5.
No. You only have a single default Python installation in effect at any one time.
Once you're done using one virtualenv you can use deactivate to go "back" to the standard, physical default Python installation.
Or you can use different command sessions with different virtual environments activated in each session.
Or you can explicitly invoke one version or another of Python from the command line each time, rather than just using the currently active-by-default one.
It is possible to autoactivate a conda environment when entering a specific directory.
https://github.com/conda/conda/issues/5179
BUT, it doesn't change the fact that source activate xxx affect your shell/prompt instead of your directory structure. You can still manually activate an environment and it will still affect your whole prompt.
I am going through the painful process of learning how to manage packages/ different (virtual) environments in Python/Anaconda. I was told that Anaconda is basically a python installation with all the packages I need (e.g. numpy, scipy, sci-kit learn etc).
However, when I create a new environment, none of these packages is readily available. I cannot import them when using PyCharm with the newly created environment. When I check the Pycharm project interpreter, or the anaconda navigator environments tab, It seems that indeed none of these packages are installed in my new environments. Why is this? It doesn't make sense to me to provide all these packages, but then not make them ready for use when creating new environments. Do I have to install all these packages manually in new env's or am I missing something?
Kindest regards, and thanks in advance.
The reason the default python environment doesn't come with numpy is because maybe you don't want numpy in the environment. Imagine writing an API (or general software package) where your users may or may not have access to numpy. You might want to run tests to make sure your software fails gracefully or has a pure python fallback if numpy is not installed on your user's machine. Conda environments provide this (insanely useful) benefit. Of course, the package in question doesn't have to be numpy. There are some more esoteric packages where this type of testing is useful.
Furthermore, you can create a conda environment with numpy pre-installed, or any other package you want pre-installed (just add them to the end of the conda create command):
conda create --name my-env-name numpy
Anaconda comes with available packages such as numpy, scipy, and sci-kit learn, but if you want to use them within your environment, you must:
1) Create the environment:
conda create --name new_env
2) Activate the environment:
source activate new_env
3) Install the desired package using conda install
conda install numpy
If you'd like to create a new environment that includes installations of all available Anaconda packages, see create anaconda python environment with all packages. You can include anaconda in the list of packages to install in the environment, which is a 'meta-package' meaning 'all the packages that come with the Anaconda installation'.
I don't know about "conda" environments but in general virtual environments are used to provide you a "unique" environment. This might include different packages, different environment variables etc.
The whole point of making a new virtual environment is to have a separate place where you can install all the binaries ( and other resources ) required for your project. If you have some pre-installed binaries in the environment, doesn't it defeat the purpose of creating one in the first place?
The fact that you can create multiple environments helps you to separate binaries that might be needed by one and not by the other.
For instance, if you are creating a project which requires numpy:1.1 but you have numpy:2.1 installed , then you have to change it. So basically, by not installing any other packages, they are not making assumptions about your project's requirements.
You can check the packages you have in your environment with the command:
conda list
If packages are not listed you just have to add it, with the command:
conda install numpy