We Keep Coding

ML and Django: using Conda and Pip depending on what I am doing... no? [duplicate]

conda 4.2.13
MacOSX 10.12.1
I am trying to install packages from pip to a fresh environment (virtual) created using anaconda. In the Anaconda docs it says this is perfectly fine. It is done the same way as for virtualenv.
Activate the environment where you want to put the program, then pip install a program...
I created an empty environment in Ananconda like this:
conda create -n shrink_venv
Activate it:
source activate shrink_venv
I then can see in the terminal that I am working in my env (shrink_venv). Problem is coming up, when I try to install a package using pip:
(shrink_venv): pip install Pillow
Requirement already satisfied (use --upgrade to upgrade): Pillow in /Library/Python/2.7/site-packages
So I can see it thinks the requirement is satisfied from the system-wide package. So it seems the environment is not working correctly, definitely not like it said in the docs. Am I doing something wrong here?
Just a note, I know you can use conda install for the packages, but I have had an issue with Pillow from anaconda, so I wanted to get it from pip, and since the docs say that is fine.
Output of which -a pip:
/usr/local/bin/pip
/Users/my_user/anaconda/bin/pip
** UPDATE **
I see this is pretty common issue. What I have found is that the conda env doesn't play well with the PYTHONPATH. The system seems to always look in the PYTHONPATH locations even when you're using a conda environment. Now, I always run unset PYTHONPATH when using a conda environment, and it works much better. I'm on a mac.

For others who run into this situation, I found this to be the most straightforward solution:
Run conda create -n venv_name and conda activate venv_name, where venv_name is the name of your virtual environment.
Run conda install pip. This will install pip to your venv directory.
Find your anaconda directory, and find the actual venv folder. It should be somewhere like /anaconda/envs/venv_name/.
Install new packages by doing /anaconda/envs/venv_name/bin/pip install package_name.
This should now successfully install packages using that virtual environment's pip!

All you have to do is open Anaconda Prompt and type
pip install package-name
It will automatically install to the anaconda environment without having to use
conda install package-name
Since some of the conda packages may lack support overtime it is required to install using pip and this is one way to do it
If you have pip installed in anaconda you can run the following in jupyter notebook or in your python shell that is linked to anaconda
pip.main(['install', 'package-name'])
Check your version of pip with pip.__version__. If it is version 10.x.x or above, then install your python package with this line of code
subprocess.check_call([sys.executable, '-m', 'pip', 'install', '--upgrade', 'package-name'])
In your jupyter notebook, you can install python packages through pip in a cell this way;
!pip install package-name
or you could use your python version associated with anaconda
!python3.6 -m pip install package-name

I solved this problem the following way:
If you have a non-conda pip as your default pip but conda python is your default python (as below)
>which -a pip
/home/<user>/.local/bin/pip
/home/<user>/.conda/envs/newenv/bin/pip
/usr/bin/pip
>which -a python
/home/<user>/.conda/envs/newenv/bin/python
/usr/bin/python
Then instead of just calling
pip install <package>, you can use the module flag -m with python so that it uses the anaconda python for the installation
python -m pip install <package>
This installs the package to the anaconda library directory rather than to the library directory associated with (the non-anaconda) pip
EDIT:
The reason this works is as follows:
the command pip references a specific pip file/shortcut (which -a pip tells you which one). Similarly, the command python references a specific python file (which -a python tells you which one). For one reason or another these two commands can become unsynchronized, so that your 'default' pip is in a different folder than your default python, and therefore is associated with a different version of python.
In contrast, the python -m pip construction does not use the shortcut that the pip command points to. Instead, it asks python to find its version of pip and use that version to install a package.

This is what worked for me (Refer to image linked)
Open Anaconda
Select Environments in the left hand pane below home
Just to the right of where you selected and below the "search environments" bar, you should see base(root). Click on it
A triangle pointing right should appear, click on it an select "open terminal"
Use the regular pip install command here. There is no need to point to an environment/ path
For future reference, you can find the folder your packages are downloading to if you happen to have a requirement already satisfied. You can see it if you scroll up in the terminal. It should read something like: requirement already satisfied and then the path
[]

If you didn't add pip when creating conda environment
conda create -n env_name pip
and also didn't install pip inside the environment
source activate env_name
conda install pip
then the only pip you got is the system pip, which will install packages globally.
Bus as you can see in this issue, even if you did either of the procedure mentioned above, the behavior of pip inside conda environment is still kind of undefined.
To ensure using the pip installed inside conda environment without having to type the lengthy /home/username/anaconda/envs/env_name/bin/pip, I wrote a shell function:
# Using pip to install packages inside conda environments.
cpip() {
ERROR_MSG="Not in a conda environment."
ERROR_MSG="$ERROR_MSG\nUse \`source activate ENV\`"
ERROR_MSG="$ERROR_MSG to enter a conda environment."
[ -z "$CONDA_DEFAULT_ENV" ] && echo "$ERROR_MSG" && return 1
ERROR_MSG='Pip not installed in current conda environment.'
ERROR_MSG="$ERROR_MSG\nUse \`conda install pip\`"
ERROR_MSG="$ERROR_MSG to install pip in current conda environment."
[ -e "$CONDA_PREFIX/bin/pip" ] || (echo "$ERROR_MSG" && return 2)
PIP="$CONDA_PREFIX/bin/pip"
"$PIP" "$#"
}
Hope this is helpful to you.

python -m pip install Pillow
Will use pip of current Python activated with
source activate shrink_venv

For those wishing to install a small number of packages in conda with pip then using,
sudo $(which pip) install <instert_package_name>
worked for me.
Explainaton
It seems, for me anyway, that which pip is very reliable for finding the conda env pip path to where you are. However, when using sudo, this seems to redirect paths or otherwise break this.
Using the $(which pip) executes this independently of the sudo or any of the commands and is akin to running /home/<username>/(mini)conda(3)/envs/<env_name>/pip in Linux. This is because $() is run separately and the text output added to the outer command.

All above answers are mainly based on use of virtualenv. I just have fresh installation of anaconda3 and don't have any virtualenv installed in it. So, I have found a better alternative to it without wondering about creating virtualenv.
If you have many pip and python version installed in linux, then first run below command to list all installed pip paths.
whereis pip
You will get something like this as output.
pip: /usr/bin/pip /home/prabhakar/anaconda3/bin/pip /usr/share/man/man1/pip.1.gz
Copy the path of pip which you want to use to install your package and paste it after sudo replacing /home/prabhakar/anaconda3/bin/pip in below command.
sudo /home/prabhakar/anaconda3/bin/pip install <package-name>
This worked pretty well for me. If you have any problem installing, please comment.

if you're using windows OS open Anaconda Prompt and type activate yourenvname
And if you're using mac or Linux OS open Terminal and type source activate yourenvname
yourenvname here is your desired environment in which you want to install pip package
after typing above command you must see that your environment name is changed from base to your typed environment yourenvname in console output (which means you're now in your desired environment context)
Then all you need to do is normal pip install command e.g pip install yourpackage
By doing so, the pip package will be installed in your Conda environment

I see a lot of good answers here but still wanted to share mine that worked for me especially if you are switching from pip-era to conda-era. By following this, you can install any packages using both conda and pip.
Background
PIP - Python package manager only
Conda - Both package and environment manager for many languages including Python
Install Pip by default every time you create a new conda environment
# this installs pip for your newly created environment
conda create -n my_new_env pip
# activate your new conda environment
conda activate my_new_env
# now you can install any packages using both conda and pip
conda install package_name
#or
pip install package_name
This gives you the flexibility to install any packages in conda environment even if they are not available in conda (e.g. wordcloud)
conda activate my_new_env
# will not work as wordcloud is not available in conda
conda install wordcloud
# works fine
pip install wordcloud

I was facing a problem in installing a non conda package on anaconda, I followed the most liked answer here and it didn't go well (maybe because my anaconda is in F directory and env created was in C and bin folder was not created, I have no idea but it didn't work).
According to anaconda pip is already installed ( which is found using the command "conda list" on anaconda prompt), but pip packages were not getting installed so here is what I did, I installed pip again and then pip installed the package.
conda install pip
pip install see
see is a non-conda package.

Depends on how did you configure your PATH environmental variable.
When your shell resolves the call to pip, which is the first bin it will find?
(test)$ whereis pip
pip: /home/borja/anaconda3/envs/test/bin/pip /home/borja/anaconda3/bin/pip
Make sure the bin folder from your anaconda installation is before /usr/lib (depending on how you did install pip). So an example:
(test) borja#xxxx:~$ pip install djangorestframework
....
Successfully installed asgiref-3.2.3 django-3.0.3 djangorestframework-3.11.0 pytz-2019.3 sqlparse-0.3.1
(test) borja#xxxx:~$ conda list | grep django
django 3.0.3 pypi_0 pypi
djangorestframework 3.11.0 pypi_0 pypi
We can see the djangorestframework was installed in my test environment but if I check my base:
(base) borja#xxxx:~$ conda list | grep django
It is empty.
Personally I like to handle all my PATH configuration using .pam_environment, here an example:
(base) borja#xxxx:~$ cat .pam_environment
PATH DEFAULT=/home/#{PAM_USER}/anaconda3/bin:${PATH}
One extra commet. The way how you install pip might create issues:
You should use: conda install pip --> new packages installed with pip will be added to conda list.
You shodul NOT use: sudo apt install python3-pip --> new packages will not be added to conda list (so are not managed by conda) but you will still be able to use them (chance of conflict).

Well I tried all the above methods. None worked for me because of an issue with the proxy settings within the corporate environment. Luckily I could open the pypi website from the browser. In the end, the following worked for me:
Activate your environment
Download the .whl package manually from
https://pypi.org/simple/<package_name>/
Navigate to the folder where you have downloaded the .whl from the command line with your environment activated
perform:
pip install package_name_whatever.whl

If you ONLY want to have a conda installation. Just remove all of the other python paths from your PATH variable.
Leaving only:
C:\ProgramData\Anaconda3
C:\ProgramData\Anaconda3\Scripts
C:\ProgramData\Anaconda3\Library\bin
This allows you to just use pip install * and it will install straight into your conda installation.

I know the original question was about conda under MacOS. But I would like to share the experience I've had on Ubuntu 20.04.
In my case, the issue was due to an alias defined in ~/.bashrc: alias pip='/usr/bin/pip3'. That alias was taking precedence on everything else.
So for testing purposes I've removed the alias running unalias pip command. Then the corresponding pip of the active conda environment has been executed properly.
The same issue was applicable to python command.

Given the information described in this Anaconda blog post, I think the best practice would be to create an environment file so that your conda environments can be created predictably.
I tried a few of the answers posted here without success and I didn't feel like messing around with python paths etc. Instead, I added an environment.yml file similar to this:
name: your-environment-name
channels:
- defaults
dependencies:
- python=3.9.12
- requests=2.28.1
- pandas=1.4.4
- pip=21.2.4
- pip:
- python-dotenv==0.19.2
This guarantees that you install all conda dependencies first, then install pip in the conda environment and use it to install dependencies that are unavailable through conda. This is predictable, reusable, and follows the advice described in the blog post.
You then create a new conda environment using the file with this command:
conda env create -f environment.yml

Uninstall the duplicated python installation. Just keep anaconda and create an env with the desired python version as specified here: https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-python.html. Then your python and pip versions will change as you switch between envs.

I've looked at this answer and many other answers for hours today and couldn't figure this out with 30 years programming experience.
I ran:
conda create -n myenv python=3.9
conda activate myenv
and could not use pip. However, in another environment such as myenv2, myenv3, myenv4 it worked.
I was obtaining the dreaded urllib3 httpsconnection error.
So thought it has to be a missing urllib3 error or something else. It turns out that it was much more sinister than that. Unfortunately it works in other environments and for me I thought that it was related to the fact I'm using Debian on Windows 10 with WSL2. The fix was simple:
rm -rf $HOME/.cache
The pip cache was mangled from a previous install of the same environment. Probably due to the fact I had run an update on conda base and done a distribution upgrade. Because I'm wanting to run a production system with apache2 using a WSGI environment with flask, I want to always have the same conda instance name. So this was a must fix!

Spyder IDE plugins installation (spyder-unittest)

I am trying to install a plugin for the Spyder ide called spyder-unittest (description here).
I am using:
MacOS X Version 10.14.6
Anaconda Navigator 1.9.7
Spyder 3.3.6
Python 3.7
After a first attempt using the command
conda install -c spyder-ide spyder-unittest
the plugin did not work (i.e. the additional command Run unit tests was not available under the Run menu).
I also tried, without success:
conda install -c conda-forge spyder-unittest
I then uninstalled and installed once again Anaconda navigator, and tried conda install -c spyder-ide spyder-unittest. This time, I got a very lengthy output, indicating conflicts (please see image):
Now, I do not know what to do. Could someone please offer any help?
I also tried to install after going, through the Terminal, to the directory where I thought the Spyder plugins were installed. Same output as above.
Thank you very much in advance.
Cheers,
Orlando

I managed to install spyder-unittest without any errors on my computer.
I suggest creating a new environment to avoid any conflicts with other packages when installing.
First create a new environment using:
conda create --name env python=3.7
After creating the environment, activate it using conda activate env
Then install spyder-unittest using conda install -c conda-forge spyder-unittest

Jupyter command `jupyter-lab` not found

I have tried to install jupyter lab on my Kubuntu machine.
If I install jupyter lab with 'pip3 install jupyter jupyterlab' the command 'jupyter notebook' works completly fine. But if I try to run 'jupyter lab' every time I get the message:
Traceback (most recent call last):
File "/usr/local/bin/jupyter", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.6/dist-packages/jupyter_core/command.py", line 230, in main
command = _jupyter_abspath(subcommand)
File "/usr/local/lib/python3.6/dist-packages/jupyter_core/command.py", line 133, in _jupyter_abspath
'Jupyter command `{}` not found.'.format(jupyter_subcommand)
Exception: Jupyter command `jupyter-lab` not found.
What is wrong?
I tried to reinstall jupyter and jupyterlab multiple times with the same issue.

Its the space. Its always the space. Never ever use spaces within package name. Its always either namepart1-namepart2 or namepart1namepart2. This is because arguments are separated by space. So if you put space in between, it makes pip think that you want to install two different packages named jupyter and lab. Just use:
python -m pip install jupyterlab
Or simply:
pip install jupyterlab
No need to uninstall or reinstall anything. However to run jupyter lab server you might want to add spaces as follows:
jupyter lab

In my case, the only way to fix this was to add the following directory to the PATH in Linux:
/home/ubuntu/.local/bin

When installing jupyterlab, we may get warning like this:
Installing collected packages: jupyterlab
WARNING: The scripts jlpm, jupyter-lab, jupyter-labextension and jupyter-labhub are installed in '/home/tln/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed jupyterlab-3.0.14
So as per this warning, scripts like jupyter-lab will be unavailable unless added to the PATH.
Use below command to add these scripts to be able to use from command line:
tln#tln-X550LD:~$ export PATH="$HOME/.local/bin:$PATH"
That's it. This worked fine for me.

I had the same error on Windows 10. It was with pip install jupyterlab. Then after the error I uninstalled it with pip and reinstalled with "pip install jupyterlab". Everything worked flawlessly thereafter.
In your case you're using pip3. Try it as above or see if pip3 needs an update.

I had same issue. I solved it running pip install jupyterlab in prompt with admin privilegies.

Had the same issue and resolved it by installing with pip3
pip3 install jupyterlab;
jupyter lab

If you face -bash: jupyter-lab: command not found or -bash: jupyter: command not found, etc., you can look for "jupyter-lab" and enter that full path instead.
Mac:
$ /Users/mark/venv/bin/jupyter-lab
Windows:
C:\mark\venv\Scripts\jupyter-lab.exe

I got this same error every time I forgot to activate the virtualenv jupyterlab was installed into. After activating the virtualenv, all's well.
With pip
$ source [path_to_venv]/bin/activate
With pipenv
$ pipenv shell
Then, with a prompt indicating an activated shell, you can enter your command
(venv) $ jupyter lab
With conda or other more holistic python environments, you probably use their gui to activate a virtualenv with jupyter and jupyterlab installed.

Install with Anaconda
conda install -c conda-forge notebook
conda install -c conda-forge jupyter
conda install -c conda-forge jupyter_contrib_nbextensions

Once installed you just have to run
pip install jupyterlab
~/.local/bin/jupyter-lab
and ready and open browser http://localhost:8888/lab
Ubuntu 22
Lubuntu 22

well the problem is like this:
the jupyterlab module has not been packaged for debian, but the jupyterlab_server package has, named python3-jupyterlab-server.
please sudo apt install python3-jupyterlab-server.
then, as your user, run pip3 install jupyterlab, that will install it in your ~/.local/bin a few programs, the missing jupyter-lab among them.
last, but not least, run jupyter-serverextension enable --py jupyterlab.
to be able to run jupyter lab, you first need to run export PATH="/home/$(whoami)/.local/bin:"$PATH. this command will run automatically if added to your ~/.bash_profile.

ubuntu can not find the jupyter-lab because it is not in path.
in order to check the place of installment. run below command
find ~ -name jupyter-lab
possible result: /home/soshiant/.local/bin/jupyter-lab
for adding a directory to the Linux path you can follow below link:
How to add a directory to the PATH

if anyone still struggling ...
try:
find ~ -name jupyter-lab
and then:
export PATH=[path]
then try again: jupyter lab
note:
don't add any extra spaces and PATH is case sensitive and if u still struggling just try to install any missing pkgs after hosting to the main path by opining another tab or changing the path to main.
and then host to localhost:8888/lab
that should works fine...

I was using conda to install
conda install -c conda-forge jupyterlab
and this error came about.
I simply used this command to get it to run.
pip install jupyterlab

In my ubuntu installation this was qused by not using sudo before the install.
sudo pip install jupyterlab
If you already installed jupyterlab, and it dit not work, you can install again using sudo, you dont need to remove the old version first for it to work

Anaconda As Python in Debian Linux Terminal

I am working on a project in python on a Debian 8 VM. To work on the project further I need to install matplotlib 1.5.1. When I attempt to upgrade the current version (obtained through apt-get) or install I am told that I need freetype and png. When I go to install freetype using this link:
http://www.linuxfromscratch.org/blfs/view/svn/general/freetype2.html
After installing and entering the proper commands, I go to try to install matplotlib again and receive the same error.
I tried to install Anaconda3 because it comes with freetype and basically every package that I need for my project. But after running the .sh file I was unable to change my python to use anaconda as the interpreter. How can I do this?
Thanks!
[UPDATE]
I am having to go into my anaconda3 file, then run source bin/activate ~/anaconda3/ Is there anyway to create an alias that would do all this?

You have to first create a conda python environment:
/path/to/conda/bin/conda create --name myenv python=3
(see http://conda.pydata.org/docs/using/envs.html)
When the environment has been created you simply activate it as follows:
/path/to/conda/bin/source activate myenv
Thereafter the system will run python from the conda environment you specified and not from the standard location.

Installing Python3 on Mac OS X Yosemite

I'm trying to install Python3 on a Mac OS X Yosemite, and did that by running the following command:
$ brew install python3
When I tried that, I got the following error:
clang: error: unable to find utility "clang", not a developer tool or in PATH
otool: error: unable to find utility "otool", not a developer tool or in PATH
Error: Failure while executing: /usr/bin/otool -L /usr/bin/install_name_tool
How can I fix this issue?
Thanks.

Try brew update and then brew doctor first. The doctor diagnoses common issues.
Seems something's wrong with your Xcode/Command Line Tools. This answer might be helpful.

I would highly recommend using the anaconda distribution, in particular miniconda.
For OSX Yosemite, this is a link to the Python 3.4 64-bit installer.
After downloading the application, open a terminal window, navigate to where you downloaded the app (e.g. cd ~/Downloads) and type:
bash Miniconda-latest-MacOSX-x86_64.sh
Now close and re-open your terminal window for the changes to take effect.
To test your installation, enter the command conda list from the terminal. If installed correctly, you will see a list of packages that were installed.
From here, you should be able to follow the on-screen instructions. If you get lost, you can refer to their installation guide.
After installing conda, you need to create an environment. To install an new environment named py3 with Python 3:
conda create --name py3 python=3
To activate this environment:
source activate py3
Here, I normally install iPython, iPython notebook and pyqt:
conda install ipython, ipython-notebook, pyqt
Now, to activate an ipython shell from within your environment:
ipython qtconsole
Although this reply does not attempt fix the homebrew issue, it answers the question of how to install Python3 on Mac OS X Yosemite.

Open Terminal and
[Try]: brew update
[or Homebrew Install]: /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
(code source found at https://brew.sh/)
Make sure processes finish, and then [Re-Try]: brew install python3

For me, I ran brew doctor which said
Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
pandoc
heroku
numpy
unbound
python#3.8
So I ran brew link python#3.8
And immediately after, python3 suddenly works!