I am going through the painful process of learning how to manage packages/ different (virtual) environments in Python/Anaconda. I was told that Anaconda is basically a python installation with all the packages I need (e.g. numpy, scipy, sci-kit learn etc).
However, when I create a new environment, none of these packages is readily available. I cannot import them when using PyCharm with the newly created environment. When I check the Pycharm project interpreter, or the anaconda navigator environments tab, It seems that indeed none of these packages are installed in my new environments. Why is this? It doesn't make sense to me to provide all these packages, but then not make them ready for use when creating new environments. Do I have to install all these packages manually in new env's or am I missing something?
Kindest regards, and thanks in advance.
The reason the default python environment doesn't come with numpy is because maybe you don't want numpy in the environment. Imagine writing an API (or general software package) where your users may or may not have access to numpy. You might want to run tests to make sure your software fails gracefully or has a pure python fallback if numpy is not installed on your user's machine. Conda environments provide this (insanely useful) benefit. Of course, the package in question doesn't have to be numpy. There are some more esoteric packages where this type of testing is useful.
Furthermore, you can create a conda environment with numpy pre-installed, or any other package you want pre-installed (just add them to the end of the conda create command):
conda create --name my-env-name numpy
Anaconda comes with available packages such as numpy, scipy, and sci-kit learn, but if you want to use them within your environment, you must:
1) Create the environment:
conda create --name new_env
2) Activate the environment:
source activate new_env
3) Install the desired package using conda install
conda install numpy
If you'd like to create a new environment that includes installations of all available Anaconda packages, see create anaconda python environment with all packages. You can include anaconda in the list of packages to install in the environment, which is a 'meta-package' meaning 'all the packages that come with the Anaconda installation'.
I don't know about "conda" environments but in general virtual environments are used to provide you a "unique" environment. This might include different packages, different environment variables etc.
The whole point of making a new virtual environment is to have a separate place where you can install all the binaries ( and other resources ) required for your project. If you have some pre-installed binaries in the environment, doesn't it defeat the purpose of creating one in the first place?
The fact that you can create multiple environments helps you to separate binaries that might be needed by one and not by the other.
For instance, if you are creating a project which requires numpy:1.1 but you have numpy:2.1 installed , then you have to change it. So basically, by not installing any other packages, they are not making assumptions about your project's requirements.
You can check the packages you have in your environment with the command:
conda list
If packages are not listed you just have to add it, with the command:
conda install numpy
Related
Is a cloned Conda environment similar to a Python Virtual Environment?
conda create --clone arcgispro-py3 --name arcgispro-py3_clone
Or are there any benefits to create a Visual Environment for this cloned environment?
I think I understand what you're asking. "virtual environment" when it comes to python usually refers to python environments created using virtualenv specifically. You could consider conda environments "virtual environments" as well, but that just gets confusing to refer to them that way and people don't do that. Say "conda environment".
Conda is its own environment and package manager, it can be used in place of virtualenv in a lot of cases. As with most things with overlapping (and largely incompatible) use, there are pros and cons to using one over the other. Since I don't know your use case, I don't know which might be right for you.
You shouldn't create a virtual environment from your conda environment. That would probably just cause a mess down the road. Either use a system python to create a virtualenv or use conda to create a conda environment, I don't recommend mixing them.
I've been using Python on my system for about a year as a new programmer. Until recently, the topic of virtual environments hasn't come up until got to the end point of the Django course on codecademy. I'm now expected to make a Django project on my own system.
I have been just installing packages to Python without making virtual environments in the past as I wasn't aware that it was recommended to create an environment for each project.
Should I have a clean install of Python before I start using virtual environments?
If so, is there a pip command to uninstall all non-python native packages and essentially reset the install?
Should I have a clean install of Python before I start using virtual environments?
No, it's not needed. Indeed, doing so would defeat the main purpose of using a virtual environment: they are used in order to "isolate" the packages of a project, without having to load the ones you have "globally" installed.
You can just create a new virtual environment and use it, everytime you make a brand new project!
You don't necessarily need a clean install for Python to have a 'clean' environment using virtualenv.
It used to be that you would specify the --no-site-packages flag to remove visibility of the globally installed packages like the below:
virtualenv --no-site-packages venv_name
However, this is now the default option for virtualenv, and you don't need to do it explicitly unless you are running a very old version.
I have been working with base environment in anaconda till now and had installed all the required python modules/libraries. When I tried to install tensorflow-cpu, I came to know that I have to create another environment to install it from official documentation of anaconda. Now I have created the new environment named 'tf' to install tensorflow-cpu. Since I can activate only one environment at a time, I don't want want to install all the python modules/libraries again to this new tf environment as it will consume space in memory.
Is there any way, I can use all the modules of base environment to tf environment or vice versa?
Please help in this context!
It's definitely not a good idea too mix envs. They were invented to separate dependencies for independent projects. Trying to mix them is a "wrong" way of using envs.
Also, you don't have to create a new env when installing tf as the docs say, just install it where you want it to be, everything will work just fine. The docs recommend you to create one because it's not a good idea to install it into base env.
You have an env named tf now. Activate it and install all dependencies you need, then use it in your project.
Do not pollute your base env with packages, this one is not supposed to be used for development, rather for conda itself. Create a new env when you start a new project
I was suggested to conda create a new environment for installing tensorflow
First question, in general:
Why do environment exist in conda or in Python ? (Why) is it preferable to install a new library in a new environment ?
Here, in practice:
After install conda shell says $conda activate test will activate the test environment. Does it mean i can't access the lib in Spyder unless i activate test in conda shell ? Do i need to restart python shell to see the lib ? I can't access the lib (no module named tensorflow) and I assume it has to do with python not finding the path.
After install conda shell says $conda activate test will activate the
test environment. Does it mean i can't access the lib in Spyder unless
i activate test in conda shell ? Do i need to restart python shell to
see the lib ? I can't access the lib (no module named tensorflow) and
I assume it has to do with python not finding the path.
Have you installed TF within the environment?
I haven't used Spyder in a while, but what usually happens is that you can start a program (like Spyder or Jupyter) from an environment if you have installed the application within it and the environment is active. (Some editors/IDE like VS Code lets you choose the environment for a specific project, once it is able to discover all the environments.)
And, also usually, though perhaps not always, you will not need to restart the shell to import a library, after installing it. It's best to refer to the specific library's installation instructions for details like this.
Virtual Environment is used to manage Python packages for different projects. Using virtual environment allows you to avoid installing Python packages globally which could break system tools or other projects. You can install virtual environment using pip.
For example, say you have two projects, and each requires a different version of Tensorflow. This is a real problem for Python since it can’t differentiate between versions in the “site-packages” directory. So both say V1.1 and V2.1 would reside in the same directory with the same name.
This also allows easy clean up, once you are done with the project just delete the virtual environment.
Checkout more, https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/
I wish to set up a virtual environment that I can use to develop web applications using the Flask framework for Python (3.4.2, Mac OS). I was given the instructions on how to do that here, using the virtualenv. However, trying to follow these instructions I ran into a problem: I have Python installed via Anaconda, and upon trying:
sudo easy_install virtualenv
I am warned that I should be doing this with the already-installed conda package instead. I can't imagine that the conda way of doing things is much harder, but I also want to get bogged down with reading the documentation, because then I might not emerge back out of it again... So my question is, what's a quick way of setting up a virtual environment with Flask using Conda? And how can I then add more dependencies into this mix?
Your mileage may vary, but the docs tends to be where the answers are.
conda create -n my_flask_env
source activate my_flask_env
conda install condastuff
pip install otherstuff