I am confused on how to install all the packages from requirements.txt shared by another person for a python project strictly using Anaconda only in Windows os.
I have installed Anaconda navigator. Should I do it in navigator or in conda prompt ?
Do I need to create an environment first and then activate it and then run command pip install requirements.txt in that environment ?
Please, could you suggest a better way to install the packages from anaconda using requirements.txt and run the python project?
conda uses an environment.yaml file instead of requirements.txt, but you can include one in the other:
# environment.yaml
name: test-env
dependencies:
- python>=3.5
- anaconda
- pip
- pip:
- -r file:requirements.txt
Then use conda to create the environment via
conda env create -f environment.yaml
While installing packages in requirements.txt using Conda through the following command
conda install --yes --file requirements.txt
In a terminal window you can enter:
pip install -r requirements.txt
You will need to enter the full path of the requirements.txt
C:\Users[UserName]\Desktop\requirements.txt
You can also see this described here:
https://note.nkmk.me/en/python-pip-install-requirements/
Kate, your question and terminology should be more precise.
First, I will answer your question... Then I will go into more detail with more precise terminology for others who have the same questions.
CONVERT REQUIREMENTS.TXT TO ENVIRONMENT.YML FILE
The best way to use a conda and a requirements.txt (the pip package manager installation specification) is to convert the requirements.txt file into an environment.yml file. To do this, copy the names of all packages from the requirements.txt file into an environment.yml file. Make sure the environment.yml file is properly formatted.
See this link for examples of environment.yml file formatting:
[Creating an environment file][1]
Then use conda from the command line and specify the environment.yml file in your conda command at the console.
Try this automated script to read and use requirements.txt in conda "on the fly". [Install only available packages using "conda install --yes --file requirements.txt" without error][2]
BUILDING THE ENVIRONMENT GRAPHICALLY USING ANACONDA NAVIGATOR:
A purely graphical (and manual) alternative is to use the Anaconda Navigator Package Manager GUI. Individually select each file specified in the requirements.txt file using the Package Manager interface (see image).
Select each green check mark for the desired package in the right-hand column. Then click the "Apply" button. If the package exist in Anaconda Navigator, then this graphical approach will work.
[![Graphical Environment and Package Manager in Anaconda Navigator][3]][3]
##############################
CLARIFYING TERMINOLOGY about 5 snakes and 1 pip ;^)
There is a body of distinct terminology and semantics in this area.
the "anaconda package",
the "Anaconda desktop" (Navigator)
the "Anaconda distribution",
the "conda utility",
the "conda package",
the "pip utility"
the "pip package",
related topics (read on...)
When you say "anaconda", you probably mean the "Anaconda Navigator Desktop" graphical user interface, not the python package "anaconda" that gets installed at a terminal command line using conda or pip.
The "anaconda" (lower case) package is for supporting automated installation of the "Anaconda Distribution" and the "Anaconda Navigator GUI Desktop".
The "Anaconda Navigator" is the desktop program shipped with the Anaconda.com distribution. It provides GUI functions for managing environments and packages within conda environments.
Under the covers the Anaconda Navigator "environment manager" executes the commands using the "conda" command line utility. Navigator effectively creates and manages environments (and packages) using the conda utility. It formulates and executes commands via conda, in a similar to how you execute conda commands on the Windows Console or Mac Terminal command line interface.
The Python package named "conda" provides a programmatic interface for calling conda functions from within Python programs.
The "conda" command line utility is an environment manager; it is also a full package manager that does everything that the pip utility does.
The "pip" command line utility is a package manager for Python-only packages in pip format.
The conda utility and conda package work in a language-agnostic way. They can manage environments and packages for many other programming languages (e.g., R, JavaScript, for starters) and many others.
CONDA AT THE COMMAND LINE OR THROUGH THE NAVIGATOR?
You can use "conda" at the command line, OR you use the Navigator Environment and Package Manager GUI. I prefer the command line, because it is much faster and more precise.
BE MINDFUL WHEN USING PIP AND CONDA TOGETHER:
Be VERY careful about using pip and conda "side-by-side". They each build and manage their own package indexes. Interoperability is still an "experimental feature". The best guide to pip and conda together is here: https://www.anaconda.com/blog/using-pip-in-a-conda-environment
One thing that I usually do is install the conda and pip packages into any new conda environment first (using "conda install -c conda-forge conda pip" at the command line). After that conda then is informed about the pip package installations. I find that pip corrupts my conda environments less frequently when I do this.
See the following document for more information about using pip inside conda environments. https://docs.conda.io/projects/conda/en/stable/user-guide/configuration/pip-interoperability.html?highlight=pip
BOTTOM LINE: Take care using pip inside conda environments!
ALWAYS LEAD WITH CONDA!
USE PIP LAST, after exhausting and using all available conda packages.
Do not keep interleaving conda and pip installation commands.
If you interleave conda and pip commands repeatedly in your Anaconda root or "base" environment, you will be eventually break your Anaconda installation, and must completely remove and re-install Anaconda.
The Anaconda un-installation can be a dirty process on Windows and Mac, but luckily there are scripts to completely remove all traces of it, and then reinstall it. [Uninstall Anaconda][4]
[Conda User Manual][5]
[1]: https://carpentries-incubator.github.io/introduction-to-conda-for-data-scientists/04-sharing-environments/index.html#:~:text=Conda%20uses%20YAML%20(%E2%80%9CYAML%20Ain,style%20indentation%20to%20indicate%20nesting.
[2]: Install only available packages using "conda install --yes --file requirements.txt" without error
[3]: https://i.stack.imgur.com/5dHFL.png
[4]: https://docs.anaconda.com/anaconda/install/uninstall/
[5]: https://docs.conda.io/projects/conda/en/stable/user-guide/index.html
conda 4.2.13
MacOSX 10.12.1
I am trying to install packages from pip to a fresh environment (virtual) created using anaconda. In the Anaconda docs it says this is perfectly fine. It is done the same way as for virtualenv.
Activate the environment where you want to put the program, then pip install a program...
I created an empty environment in Ananconda like this:
conda create -n shrink_venv
Activate it:
source activate shrink_venv
I then can see in the terminal that I am working in my env (shrink_venv). Problem is coming up, when I try to install a package using pip:
(shrink_venv): pip install Pillow
Requirement already satisfied (use --upgrade to upgrade): Pillow in /Library/Python/2.7/site-packages
So I can see it thinks the requirement is satisfied from the system-wide package. So it seems the environment is not working correctly, definitely not like it said in the docs. Am I doing something wrong here?
Just a note, I know you can use conda install for the packages, but I have had an issue with Pillow from anaconda, so I wanted to get it from pip, and since the docs say that is fine.
Output of which -a pip:
/usr/local/bin/pip
/Users/my_user/anaconda/bin/pip
** UPDATE **
I see this is pretty common issue. What I have found is that the conda env doesn't play well with the PYTHONPATH. The system seems to always look in the PYTHONPATH locations even when you're using a conda environment. Now, I always run unset PYTHONPATH when using a conda environment, and it works much better. I'm on a mac.
For others who run into this situation, I found this to be the most straightforward solution:
Run conda create -n venv_name and conda activate venv_name, where venv_name is the name of your virtual environment.
Run conda install pip. This will install pip to your venv directory.
Find your anaconda directory, and find the actual venv folder. It should be somewhere like /anaconda/envs/venv_name/.
Install new packages by doing /anaconda/envs/venv_name/bin/pip install package_name.
This should now successfully install packages using that virtual environment's pip!
All you have to do is open Anaconda Prompt and type
pip install package-name
It will automatically install to the anaconda environment without having to use
conda install package-name
Since some of the conda packages may lack support overtime it is required to install using pip and this is one way to do it
If you have pip installed in anaconda you can run the following in jupyter notebook or in your python shell that is linked to anaconda
pip.main(['install', 'package-name'])
Check your version of pip with pip.__version__. If it is version 10.x.x or above, then install your python package with this line of code
subprocess.check_call([sys.executable, '-m', 'pip', 'install', '--upgrade', 'package-name'])
In your jupyter notebook, you can install python packages through pip in a cell this way;
!pip install package-name
or you could use your python version associated with anaconda
!python3.6 -m pip install package-name
I solved this problem the following way:
If you have a non-conda pip as your default pip but conda python is your default python (as below)
>which -a pip
/home/<user>/.local/bin/pip
/home/<user>/.conda/envs/newenv/bin/pip
/usr/bin/pip
>which -a python
/home/<user>/.conda/envs/newenv/bin/python
/usr/bin/python
Then instead of just calling
pip install <package>, you can use the module flag -m with python so that it uses the anaconda python for the installation
python -m pip install <package>
This installs the package to the anaconda library directory rather than to the library directory associated with (the non-anaconda) pip
EDIT:
The reason this works is as follows:
the command pip references a specific pip file/shortcut (which -a pip tells you which one). Similarly, the command python references a specific python file (which -a python tells you which one). For one reason or another these two commands can become unsynchronized, so that your 'default' pip is in a different folder than your default python, and therefore is associated with a different version of python.
In contrast, the python -m pip construction does not use the shortcut that the pip command points to. Instead, it asks python to find its version of pip and use that version to install a package.
This is what worked for me (Refer to image linked)
Open Anaconda
Select Environments in the left hand pane below home
Just to the right of where you selected and below the "search environments" bar, you should see base(root). Click on it
A triangle pointing right should appear, click on it an select "open terminal"
Use the regular pip install command here. There is no need to point to an environment/ path
For future reference, you can find the folder your packages are downloading to if you happen to have a requirement already satisfied. You can see it if you scroll up in the terminal. It should read something like: requirement already satisfied and then the path
[]
If you didn't add pip when creating conda environment
conda create -n env_name pip
and also didn't install pip inside the environment
source activate env_name
conda install pip
then the only pip you got is the system pip, which will install packages globally.
Bus as you can see in this issue, even if you did either of the procedure mentioned above, the behavior of pip inside conda environment is still kind of undefined.
To ensure using the pip installed inside conda environment without having to type the lengthy /home/username/anaconda/envs/env_name/bin/pip, I wrote a shell function:
# Using pip to install packages inside conda environments.
cpip() {
ERROR_MSG="Not in a conda environment."
ERROR_MSG="$ERROR_MSG\nUse \`source activate ENV\`"
ERROR_MSG="$ERROR_MSG to enter a conda environment."
[ -z "$CONDA_DEFAULT_ENV" ] && echo "$ERROR_MSG" && return 1
ERROR_MSG='Pip not installed in current conda environment.'
ERROR_MSG="$ERROR_MSG\nUse \`conda install pip\`"
ERROR_MSG="$ERROR_MSG to install pip in current conda environment."
[ -e "$CONDA_PREFIX/bin/pip" ] || (echo "$ERROR_MSG" && return 2)
PIP="$CONDA_PREFIX/bin/pip"
"$PIP" "$#"
}
Hope this is helpful to you.
python -m pip install Pillow
Will use pip of current Python activated with
source activate shrink_venv
For those wishing to install a small number of packages in conda with pip then using,
sudo $(which pip) install <instert_package_name>
worked for me.
Explainaton
It seems, for me anyway, that which pip is very reliable for finding the conda env pip path to where you are. However, when using sudo, this seems to redirect paths or otherwise break this.
Using the $(which pip) executes this independently of the sudo or any of the commands and is akin to running /home/<username>/(mini)conda(3)/envs/<env_name>/pip in Linux. This is because $() is run separately and the text output added to the outer command.
All above answers are mainly based on use of virtualenv. I just have fresh installation of anaconda3 and don't have any virtualenv installed in it. So, I have found a better alternative to it without wondering about creating virtualenv.
If you have many pip and python version installed in linux, then first run below command to list all installed pip paths.
whereis pip
You will get something like this as output.
pip: /usr/bin/pip /home/prabhakar/anaconda3/bin/pip /usr/share/man/man1/pip.1.gz
Copy the path of pip which you want to use to install your package and paste it after sudo replacing /home/prabhakar/anaconda3/bin/pip in below command.
sudo /home/prabhakar/anaconda3/bin/pip install <package-name>
This worked pretty well for me. If you have any problem installing, please comment.
if you're using windows OS open Anaconda Prompt and type activate yourenvname
And if you're using mac or Linux OS open Terminal and type source activate yourenvname
yourenvname here is your desired environment in which you want to install pip package
after typing above command you must see that your environment name is changed from base to your typed environment yourenvname in console output (which means you're now in your desired environment context)
Then all you need to do is normal pip install command e.g pip install yourpackage
By doing so, the pip package will be installed in your Conda environment
I see a lot of good answers here but still wanted to share mine that worked for me especially if you are switching from pip-era to conda-era. By following this, you can install any packages using both conda and pip.
Background
PIP - Python package manager only
Conda - Both package and environment manager for many languages including Python
Install Pip by default every time you create a new conda environment
# this installs pip for your newly created environment
conda create -n my_new_env pip
# activate your new conda environment
conda activate my_new_env
# now you can install any packages using both conda and pip
conda install package_name
#or
pip install package_name
This gives you the flexibility to install any packages in conda environment even if they are not available in conda (e.g. wordcloud)
conda activate my_new_env
# will not work as wordcloud is not available in conda
conda install wordcloud
# works fine
pip install wordcloud
I was facing a problem in installing a non conda package on anaconda, I followed the most liked answer here and it didn't go well (maybe because my anaconda is in F directory and env created was in C and bin folder was not created, I have no idea but it didn't work).
According to anaconda pip is already installed ( which is found using the command "conda list" on anaconda prompt), but pip packages were not getting installed so here is what I did, I installed pip again and then pip installed the package.
conda install pip
pip install see
see is a non-conda package.
Depends on how did you configure your PATH environmental variable.
When your shell resolves the call to pip, which is the first bin it will find?
(test)$ whereis pip
pip: /home/borja/anaconda3/envs/test/bin/pip /home/borja/anaconda3/bin/pip
Make sure the bin folder from your anaconda installation is before /usr/lib (depending on how you did install pip). So an example:
(test) borja#xxxx:~$ pip install djangorestframework
....
Successfully installed asgiref-3.2.3 django-3.0.3 djangorestframework-3.11.0 pytz-2019.3 sqlparse-0.3.1
(test) borja#xxxx:~$ conda list | grep django
django 3.0.3 pypi_0 pypi
djangorestframework 3.11.0 pypi_0 pypi
We can see the djangorestframework was installed in my test environment but if I check my base:
(base) borja#xxxx:~$ conda list | grep django
It is empty.
Personally I like to handle all my PATH configuration using .pam_environment, here an example:
(base) borja#xxxx:~$ cat .pam_environment
PATH DEFAULT=/home/#{PAM_USER}/anaconda3/bin:${PATH}
One extra commet. The way how you install pip might create issues:
You should use: conda install pip --> new packages installed with pip will be added to conda list.
You shodul NOT use: sudo apt install python3-pip --> new packages will not be added to conda list (so are not managed by conda) but you will still be able to use them (chance of conflict).
Well I tried all the above methods. None worked for me because of an issue with the proxy settings within the corporate environment. Luckily I could open the pypi website from the browser. In the end, the following worked for me:
Activate your environment
Download the .whl package manually from
https://pypi.org/simple/<package_name>/
Navigate to the folder where you have downloaded the .whl from the command line with your environment activated
perform:
pip install package_name_whatever.whl
If you ONLY want to have a conda installation. Just remove all of the other python paths from your PATH variable.
Leaving only:
C:\ProgramData\Anaconda3
C:\ProgramData\Anaconda3\Scripts
C:\ProgramData\Anaconda3\Library\bin
This allows you to just use pip install * and it will install straight into your conda installation.
I know the original question was about conda under MacOS. But I would like to share the experience I've had on Ubuntu 20.04.
In my case, the issue was due to an alias defined in ~/.bashrc: alias pip='/usr/bin/pip3'. That alias was taking precedence on everything else.
So for testing purposes I've removed the alias running unalias pip command. Then the corresponding pip of the active conda environment has been executed properly.
The same issue was applicable to python command.
Given the information described in this Anaconda blog post, I think the best practice would be to create an environment file so that your conda environments can be created predictably.
I tried a few of the answers posted here without success and I didn't feel like messing around with python paths etc. Instead, I added an environment.yml file similar to this:
name: your-environment-name
channels:
- defaults
dependencies:
- python=3.9.12
- requests=2.28.1
- pandas=1.4.4
- pip=21.2.4
- pip:
- python-dotenv==0.19.2
This guarantees that you install all conda dependencies first, then install pip in the conda environment and use it to install dependencies that are unavailable through conda. This is predictable, reusable, and follows the advice described in the blog post.
You then create a new conda environment using the file with this command:
conda env create -f environment.yml
Uninstall the duplicated python installation. Just keep anaconda and create an env with the desired python version as specified here: https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-python.html. Then your python and pip versions will change as you switch between envs.
I've looked at this answer and many other answers for hours today and couldn't figure this out with 30 years programming experience.
I ran:
conda create -n myenv python=3.9
conda activate myenv
and could not use pip. However, in another environment such as myenv2, myenv3, myenv4 it worked.
I was obtaining the dreaded urllib3 httpsconnection error.
So thought it has to be a missing urllib3 error or something else. It turns out that it was much more sinister than that. Unfortunately it works in other environments and for me I thought that it was related to the fact I'm using Debian on Windows 10 with WSL2. The fix was simple:
rm -rf $HOME/.cache
The pip cache was mangled from a previous install of the same environment. Probably due to the fact I had run an update on conda base and done a distribution upgrade. Because I'm wanting to run a production system with apache2 using a WSGI environment with flask, I want to always have the same conda instance name. So this was a must fix!
I am pretty new to python. Just been working through some online tutorials on udemy. I seem to have an issue with pip installing modules.
I've tried reinstalling them.
Upgrading my python version.
In VS I always just get module not found.
If I do it in the cmd prompt this is what I get below.
You are currently working on the base environment of your computer. For safety, you can first create a new virtual environment with
python3 -m venv -n new_env
So that you won't corrupt any default installations. Then, activate it with
source new_env/bin/activate
And update the pip and setuptools with
pip3 install --upgrade pip
pip3 install --upgrade setuptools
Finally, install numpy via
pip3 install numpy
However, I would recommend using Anaconda to build your virtual environment. When you install Anaconda and make sure it is included in the path of your terminal, all you need to type is
conda create -n new_env python=3.7 numpy
and it will automatically build the wheel for numpy. Here, "new_env" is just an example for a virtual environment name, and Python version 3.7 is also an example.
You can then, activate this conda environment by
conda activate new_env
To use this virtual environment, which you built either with "venv" or "conda", you should locate and activate this environment from the project interpreter settings in VS .
Finally, I would also recommend considering Pycharm IDE which can also help you with creating a virtual environment and installing packages in it.
It seems that you already have the packages installed. Using VS, please, be sure that you selected the correct Python interpreter (https://code.visualstudio.com/docs/python/environments)
I want to work with the python package holopy. Apparently you have to use conda to install it, so I first installed Anaconda 4.2.0 (since I'm using Python 3.5). I opened the virtual environment I normally use and installed holopy as they recommend on the official site:
conda install -c conda-forge holopy
Afterwards, when in the virtual environment I type conda list, holopy shows up. But when I type python3 and then import holopy, it says package not found. It does however work when I leave the virtual environment. I need it in the virtual environment though, how can I do that?
I'm not sure how well anaconda and virtual environments i.e.venv work together. If you're using anaconda anyway then I highly recommend using anaconda environments. Please go through this short tutorial about anaconda environments - you won't regret it.
Why it didn't work for you?
The conda command is available only in the base anaconda environment. So when you run the command - conda insall -c conda-forge holopy, it installed holopy in the base anaconda environment and it won't be available to you in your venv.
After looking at the documentation of holopy it seems probable that when they said virtual environment they actually meant anaconda virtual environment. Therefore the solution is to first create an anaconda virtual environment called holopy-env and then run the command conda install -n holopy-env -c conda-forge holopy.
A better way of doing things with Anaconda
I will also give you a quick and clean example of how to create an environment using anaconda. If you're using Anaconda then it would be wise to use it's environment management tools. Create an environment.yml file with the following contents:
environment.yml using conda-forge/holopy & python 3.6
name: holopy-env # any name for the environment
channels:
- conda-forge
dependencies: # everything under this, installed by conda
- python=3.6
- holopy
- pip: # everything under this, installed by pip
- future
How to install the environment?
conda create --force -f environment.yml
How to activate the environment?
source activate opencv-env
After activating the environment
You should be able to import holopy
Install pip packages using pip install <package>
Install conda packages using conda install -n holopy-env -c CHANNEL <package>
conda is a packaging tool and installer that aims to do more than what pip can do; handle library dependencies outside of the Python packages as well as the Python packages themselves. Conda also creates a virtual environment, like virtualenv does. For creating virtualenv with conda, use the following command:-
conda create -n yourenvname python=x.x anaconda
Use the following to activate the virtualenv in conda
source activate yourenvname
Then, you can install the packages in virtualenv using conda as:-
conda install -n yourenvname [package]
To Deactivate use:-
source deactivate
And to delete a no longer needed virtualenv, use :-
conda remove -n yourenvname -all
I know this is a bit late, but you don't need to use conda to install HoloPy. This is just the least technical option. Otherwise, you need to be able to compile HoloPy's fortran components yourself, which is fairly straightforward on Unix-based systems but complicated on Windows. Instructions can be found in HoloPy's documentation at https://holopy.readthedocs.io/en/latest/users/dev_tutorial.html.
We are also working on putting together a singularity container distribution of HoloPy. Let me know if this is of interest to you and I will make it a priority.
I have a Ipython Notebook that I'd like to share with others, and it uses a lot of packages.
I'm wondering if there is any tool for installing packages with ease? So others won't need to run pip install for each packages that I listed.
In Ruby on Rails, there is a gemfile, I can just run bundle install and then all gem are installed, which saves a lot of time.
I'm wondering if there is a gemfile and bundle install for ipython notebook? So we can install packages with ease.
One way to do it is to use pip:
pip freeze > requirements.txt
You could distribute that with the notebook. Then to use it:
pip install -r requirements.txt
Here is the how to with requirements.txt method. But the basic workflow is above.
As a note the requirements file will look something like:
requests==2.8.1
SQLAlchemy==0.9.9
stripe==1.27.1
Werkzeug==0.10.4
wheel==0.26.0
WTForms==1.0.5
Where you can see there are versions for each package. This method handles dependencies as well. So if one package depends on another, pip installs in such a way that there shouldn't be any errors. Though it might not always be the case.
This method should work in a Windows Powershell, definitely does in Mac and Linux.
Another is with conda:
conda create -n myenv python==3.5.0
This creates a conda environment. Which can be activated or deactivated. If activated you can install from a dependency file as:
conda env create -f requirements.yml
Similarly the requirements.txt can be created as:
conda env export > requirements.yml
For scientific applications conda is probably the best option. It allows to install from a file of package names:
conda install --file file_with_package_names.txt
Furthermore, it offers virtual environments that are more powerful than the standard virtual env:
conda create -n my_new_env python=3.5
Activate this environment:
source conda activate my_new_env
Get help with:
conda -h
List all installed packages:
conda list
List all conda environments:
conda info -e
It offers much more and works on all major operating systems. All installs are binary. So no compilation of extensions. Makes Windows users very happy. But is great for Linux/Mac folks too.
This is a nice comparison of conda, pip, and virtualenv.