Creating new Conda environment messes with old environments? - python

I have 2 different projects with different dependencies. One requires Tensorflow1.15 and another needs 1.14. I first created an environment env1 and pip installed Tf1.14, ran my code, all went well. Then I created a new environment env2 and pip installed Tf1.15, during which I could see it was uninstalling Tf1.14, I assumed it knew what it was doing. However now when I run my code in env1 it throws errors because tf1.14 is removed and env1 also tries to use tf1.15!
What am I doing wrong? I thought we use Conda to create completely separate environments for specifically this kind of situation but I'm confused.

You should not have used pip to install the packages. If you have Anaconda you should use conda to install them. conda install numpy will install the numpy module.
You need
conda install your-module

Related

Issue with installing python modules

I am pretty new to python. Just been working through some online tutorials on udemy. I seem to have an issue with pip installing modules.
I've tried reinstalling them.
Upgrading my python version.
In VS I always just get module not found.
If I do it in the cmd prompt this is what I get below.
You are currently working on the base environment of your computer. For safety, you can first create a new virtual environment with
python3 -m venv -n new_env
So that you won't corrupt any default installations. Then, activate it with
source new_env/bin/activate
And update the pip and setuptools with
pip3 install --upgrade pip
pip3 install --upgrade setuptools
Finally, install numpy via
pip3 install numpy
However, I would recommend using Anaconda to build your virtual environment. When you install Anaconda and make sure it is included in the path of your terminal, all you need to type is
conda create -n new_env python=3.7 numpy
and it will automatically build the wheel for numpy. Here, "new_env" is just an example for a virtual environment name, and Python version 3.7 is also an example.
You can then, activate this conda environment by
conda activate new_env
To use this virtual environment, which you built either with "venv" or "conda", you should locate and activate this environment from the project interpreter settings in VS .
Finally, I would also recommend considering Pycharm IDE which can also help you with creating a virtual environment and installing packages in it.
It seems that you already have the packages installed. Using VS, please, be sure that you selected the correct Python interpreter (https://code.visualstudio.com/docs/python/environments)

Python Conda Environment confusion(as example: problem with gym)

Trying to use gym open-ai package (and somen other) I ran into some problems,
which structure I don't really understand.
As an example:
I tried to install gym in three different conda environments.
One way to do this is
pip install gym
Another is:
git clone https://github.com/openai/gym.git
cd gym
pip install -e .
A third would be:
pip3 install gym
In some environments I would use Python2, in other env. maybe Python 3.7
Even more possibilities for installation would be:
sudo pip install gym
(and even more permutations would be possible, if we would take into account,
if we activate an environment or don't activate any environment).
To me things get even more complicated, because I tried to install conda with
a not-administrator-user-account in Ubuntu, so that conda (or rather the user itself could not install any files in the /usr directory).
I began to test some of this possibilities and cases, because installation of some libaries
(e.g. keras-rl) seemed to need access to common ressources (/usr/ dir.), even if
installed in an local conda environment. But if so: would the installations in
different conda-environments interact?
And what, if one would install a package as local user in a conda environment and
afterward install a pip or pip3 as administrator. Would the admin-installation
overwrite (or overrule or interact) the environmental installation (or parts of it)?
While experimenting with the different possibilities (or more: while trying to
find a installations, which did not produce any errors like "gym not found" or
"attribute error ... " ) there did occur errors like:
Found existing installation: gym 0.15.4
Can't uninstall 'gym'. No files were found to uninstall.
after executing:
sudo pip3 install gym --force
So on this basis my questions specifically would be:
(1) Is there a best practice for establish good conda environments
(which don't tend to interact, especially if some packages
need sudo priviledges)?
And (2) if some environments interact with
general (sudo) ressources, how can they be resolved in a way,
that distinct environments can be tested and established beneath each other?
Annotation:
there was a similiar question:
conda environment pip is trying to install dependencies globally
some time ago, but the advice, not to use sudo, seems to be difficult to follow,
if some packages require access to global ressources.
So I would like to ask for a solution to interactions at bit more specifically.
you should not use sudo to install something in a conda environment. Most likely the used pip command is not stemming from the actual (activated?) environment, but the actual system-wide pip is used. Therefore you would need to use to use sudo to install to a system owned prefix.
You can check whether you are using the desired pip by invoking "which pip". The path should point to your environment. If it does not, you shall install pip inside your conda env.
I had the same problem before. I activated conda envirement and installed with pip3 locally since conda does not have support for it. Warning: Possible of wreckig some packs.
The conda envirement should ALLWAYS be activated before installing anything orelse it ends up as a global installation.
install a new conda envirement without using sudo. If it ask for sudo you need to remove the whole thing and clean up a bit. Its very easy to forget and NEVER use sudo !
You can try installing a newer version of python3.x (python 2 is getting history very soon anyways they said. Pip = python2, pip3 = python3. And to answer one of your new question if by installing globally will mess things up, not outside conda.
google pycharm and conda. there you can just use it to install 3 different types of envirements with python. Actually a darn good editor for python coding. The rest is more linux related when we talk about cleaning up PATHS etc.
I have no better to add! Hope you get it right.

Do these commands do the same action?

Basically I would like to know if those 2 snippets do the same thing :
conda install -n myEnv myPackage
VS
conda activate myEnv
pip install myPackage
Or in a different way, does a pip install when a conda environment is activated equal doing a conda install on myEnv ?
EDIT : I thought it was obvious but => more precisely, does the second snippet only install the package on the environment or on the overall system ?
PS : Asking because there's a package available with pip but not with conda and I want it to only be installed on myEnv
The Anaconda docs make it clear that if you use conda as your virtual environment manager, you should stick to conda install to install new packages as far as possible:
Unfortunately, issues can arise when conda and pip are used together
to create an environment, especially when the tools are used
back-to-back multiple times, establishing a state that can be hard to
reproduce. … Running conda after pip has the potential to
overwrite and potentially break packages installed via pip. Similarly,
pip may upgrade or remove a package which a conda-installed package
requires.
If you can't get all the packages you need from a conda channel, they say this, which is good advice even if you don't use pip:
If there is an expectation to install software using pip along-side
conda packages it is a good practice to do this installation into a
purpose-built conda environment to protect other environments from any
modifications that pip might make.
Finally the same document notes:
Use conda environments for isolation
create a conda environment to isolate any changes pip makes
environments take up little space thanks to hard links
care should be taken to avoid running pip in the “root” environment
Provided you activate the correct conda environment first, the pip install command(s) should use that environment's pip and install only into that environment.
Yes and no.
pip downloads and installs the package from PyPI whereas conda does the same from Anaconda repositories.
There are packages in PyPI not present in Anaconda, and the other way around.
For managing the environment I would choose one way or the other, since with pip you can freeze into a requirements.txt (pip freeze > requirements.txt) and conda you can either export the whole environment (conda env export) or the list of packages (conda list --export > requirements.txt). However if you try to use a conda-generated file from pip, it will most probably fail.

Confusion and conflicts with multiple Python bins and package locations

*I hope this isn't a repeat. I've tried looking for clarity, but I'm having trouble.
I'm fairly new to Python and used Homebrew to avoid using the system Python on Mac. I guess I also installed anaconda at some point. (probably following some tutorial)
Now when I do which for the following, I get different bins:
which python: /Users/ryangoree/anaconda3/bin/python
which python2: /usr/local/bin/python2
which python3: /Users/ryangoree/anaconda3/bin/python3
which pip: /Users/ryangoree/anaconda3/bin/pip
So my issue is that when I'm using python2, there are modules that that I can't use. If I try to pip install them, It just tells me they're already installed since they are in the anaconda directory.
I don't know what I don't know right now, but I'm sure there is a better way to handle this. Can someone enlighten me or send me on the right path to developing with Python and managing the packages.
Thank you!
This stumped me for a while until I figured out one pain when using anaconda: just because you’re in a conda environment doesn’t mean that pip belongs to that environment. Instead you must run conda install pip for pip to be associated with that environment. Then every pip install will be tied to that environment.
You can check your PYTHONPATH to see the order in which the various python installations are interrogated.
Better still you should create each conda environment with its own python using:
conda create -n <envname> python=2.7 # python 2.7
conda create -n <envname> python=3.6 # python 3.6
which will automatically include pip for that environment.

"No module named tensorflow" after pip install in Anaconda Environment

I just started to learn how to use Anaconda to manage packages. I am trying to install tensorflow in conda environment. So first of all, I create an environment by:
conda create -n tensorflow
Then, I source it by:
source activate tensorflow
I can see my prompt changed so I think it is going right.
I notice that it seems that the tensorflow environment is copying from ~/anaconda2/lib/ where I do have my root version python2.7 and tensorflow0.12.0
I installed a new version Python in tensorflow environment by:
conda install python=3.5
Then, I follow the steps to install tensorflow by:
pip install --ignore-installed --upgrade TF_PYTHON_URL
However, when I do conda list, I can only see Python3.5 but not tensorflow1.0. I also failed to import tensorflow when I am in Python.
So I have two questions that really confuse me.
Why does the pip installed tensorflow not show up when I do conda list?
Although I conda install python=3.5 and I can see it from conda list, I am not using python 3.5 when I enter Python directly. It seems still using Python2.7, which comes from my root environment.
I appreciate any tutorial on how anaconda works.
I think your pip install is installing into the global environment instead of
tensorflow. Why don't you try installing by specifying the path? For example pip install --target $HOME/anaconda3/tensorflow tensorflow(Where the first tensorflow is your environment and the second is the actual package).
I just saw the last two questions. So you actually see the tensorflow you installed with pip? I am confused now. Type which pip to see if it is running from the tensorflow environment or the global. You could also try source deactivate before source activate tensorflow just to make sure that you are not using a different environment, then run which python. It should show your new environment.
If you want to create an environment using a specific version of Python (rather than the system default), you can do for example:
conda create --name myCoolEnv python=3.5
and then activate with
source activate myCoolEnv
You can read more about Anaconda environments here.

Categories