I have python 2.7 installed through homebrew and running pip install -r requirements.txt on a project's requirements file. The packages download, everything goes fine until it's time to link the binaries - then Pip tries to put the binaries for f2py (a dependency of a package in the requirements.txt file) into /bin and I'm left with this error:
IOError: [Errno 1] Operation not permitted: '/bin/f2py'
I don't have root access so I'd like for pip to put all binaries in /usr/local/bin instead. How do I tell pip to install binaries into that directory?
I'd create a virtualenv (install it with pip first), then use the virtualenv to install all your requirements. That way you both have a writable path and keep your global Python installation clean for other projects.
Alternatively, you could use the --user option to install in the site.USER_SITE location:
pip install --local virtualenv
or
pip install --local -r requirements.txt
See the User Installs section in the documentation.
If you downloaded the package you can do
python setup.py install --user
pip now supports this behaviour by passing user to setup.py
pip install --user somepackage
Related
It seems to be common practice to set up a Python virtual environment using some variant of the following:
python -m venv venv && source ./venv/bin/activate
python -m pip install -U pip -r requirements.txt
What I hope the above command does is:
Upgrade pip first
Run the installation of the packages in requirements.txt
However, what actually seems to happen is:
Collects all packages, including newest version of pip
Installs them all together
The original/outdated version of pip is what actually runs the installs
And the new version of pip is not used until after this command
Question(s)
Is it possible to have pip upgrade itself and then install a requirements file, in one command?
Would this infer any particular benefits?
Should I switch to the following?
python -m venv venv && source ./venv/bin/activate
python -m pip install -U pip
python -m pip install -r requirements.txt
What is the optimal method to install requirements files?
I see people sometimes installing/upgrading wheel and setuptools as well
The answers to your questions are:
No. pip doesn't currently treat itself as a special dependency, so it doesn't know to install then execute itself, which is what it would need to do to overcome the problems you observed.
Updating pip in a separate step is indeed the recommended way to proceed.
You may from time to time see pip issue a message advising that a newer version is available. This happens a lot if you create them from a python with an outdated pip.
I had a situation similar to yours, I needed to first upgrade pip and then install a bunch of libraries in a lab that had 20 PCs. What I did was writing all the librarie's name in a requirements.txt file, then create a .bat file with two commands:
`python -m pip install --upgrade pip`
`pip install -r requirements.txt`
The first command for upgrading pip and the second one for installing all the libraries listed in the requirements.txt file.
First, I run
pip install virtualenv
and later, I run
pip install --user virtualenv
So, this is what I have now
$ which -a virtualenv
/Users/admin/.local/bin/virtualenv
/usr/local/bin/virtualenv
This is my default virtualenv
/Users/admin/.local/bin/virtualenv
Now I want to uninstall, this,
/usr/local/bin/virtualenv
What should I do?
Thanks!
I tried
pip uninstall virtualenv
It removed /usr/local/bin/virtualenv, the system wide package.
The package I installed using --user flag, which cannot be uninstalled.
I just manually removed the folder.
See this thread.
How to uninstall a package installed with pip install --user
I am mac user, used to run pip install with --user, but recently after brew update, I found there are some strange things, maybe related.
Whatever I tries, the packages are always installed to ~/Library/Python/2.7/lib/python/site-packages
Here are the commands I run.
$ python -m site --user-site
~/Library/Python/2.7/lib/python/site-packages
$ pip install --user -r requirements.txt
$ PYTHONUSERBASE=. pip install --user -r requirements.txt
So what should be the problem?
I used for lambda zip packaging
Updates:
If using Mac OS X and you have Python installed using Homebrew (see Homebrew), the accepted command will not work. A simple workaround is to add a setup.cfg file in your /path/to/project-dir with the following content.
[install]
prefix=
https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html
You can use the target (t) flag of pip install to specify a target location for installation.
In use:
pip install -r requirements.txt -t /path/to/directory
to the current directory:
pip install -r requirements.txt -t .
I have a requirement.txt file with the list of python package to install. One of the packages is psycopg2==2.6.2 I need to update this package to psycopg2==2.7. I tried to install by pip3 install psycopg2 But it doesn't affect requirement.txt file. Can you please point me in the right direction?
Notice that running pip3 install psycopg2 doesn't respect the requirements.txt file. To upgrade this package you need to use -U option:
pip3 install -U psycopg2
which is a shorthand for:
pip3 install --upgrade psycopg2
After that, you can update your requirements.txt with the following command:
pip freeze > requirements.txt
If you're looking for a solution to automatically update the requirements.txt file after you upgrade package/packages, you can use pip-upgrader.
Installation:
pip install pip-upgrader
Usage:
pip-upgrade
The above command auto-discovers the requirements file and prompts for selecting upgrades. You can also specify a path to the requirements file or/and specify a package to upgrade:
pip-upgrade /path/to/requirements.txt -p psycopg2
As you've discovered, pip doesn't update the requirements file. So the workflow you'd likely want to use is:
Update the version of psycopg2 in your requirements file from 2.6.2 to 2.7
Run pip install with the upgrade flag
pip3 install -U -r requirements.txt
If you're familiar with tools like npm that do update the version in the catalog file, you may be interested in using pipenv, which manages your dependencies and the virtual environment for you, much like npm does.
If you don't know the latest version of your package, then use pip to figure it out:
$ pip list --outdated | grep psycopg2
psycopg2 (2.7.3.2) - Latest: 2.7.4 [wheel]
you can try:
pip install --upgrade --force-reinstall -r requirements.txt
You can also ignore installed package and install the new one :
pip install --ignore-installed -r requirements.txt
I wanted to deploy my Python app on Amazon Linux AMI 2015.09.1, which has Python2.7 (default) and pip (6.1.1). Then, I upgraded the pip using the command:
sudo pip install -U pip
However, it seemed broken, and showed the message when I tried to install packages:
pkg_resources.DistributionNotFound: pip==6.1.1
I found out that pip remove the previous files located in /usr/bin/, and installed the new one in /usr/local/bin. Thus, I tried to specify the location by using the command:
sudo pip install -U --install-option="--prefix='/usr/bin'" pip
Nevertheless, it still installed the new one in /usr/local/bin. In addition to that, pip could not work well with sudo although it successfully installed. The error message :
sudo: pip2.7: command not found
Is there a way to properly manage pip?
Try:
sudo which pip
This may reveal something like 'no pip in ($PATH)'.
If that is the case, you can then do:
which pip
Which will give you a path like /usr/local/bin/pip.
Copy + paste the path to pip to the sbin folder by running: sudo cp /usr/local/bin/pip /usr/sbin/
This will allow you to run sudo pip without any errors.
Struggled with this for a while. Here's what I've found:
ec2_user finds the pip executable, but has a module import error due to other having no read/execute permissions on the pip folders in the /usr/local/lib/python2.7/site-packages folder. This is actually okay, since in most cases, pip installs fail when not run as root anyway.
sudo cannot find pip.
Entering root with sudo su - allows pip to be run without issue.
The reason sudo pip stops working after the upgrade, is because the executable (or symbolic link) is removed from /usr/bin. However, what remains is a file called pip-27, which contains the following:
#!/usr/bin/python2.7
# EASY-INSTALL-ENTRY-SCRIPT: 'pip==6.1.1','console_scripts','pip2.7'
__requires__ = 'pip==6.1.1'
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
sys.exit(
load_entry_point('pip==6.1.1', 'console_scripts', 'pip2.7')()
)
So, that's where our error comes from, as the upgrade clearly doesn't clean this file up. Not entirely clear on where the name translation from pip to pip-2.7 occurs.
As mentioned in another answer, pip now exists in /usr/local/bin after the upgrade, which is no longer in the sudo secure path. You can add this path to the secure_path variable by running sudo visudo. Another alternative, if you'd prefer to not add that path to your secure_path is to make a symbolic link to the new pip executable in /usr/bin.
The problem is partly answered by your question. The Amazon AMI doesn't consider /usr/local/bin to be part of the root account's PATH. You can fix this by updating the root account's ~/.bashrc to include it.
Something like this...
export PATH=$PATH:/usr/local/bin
After struggling with this for hours and reading comments
which pip gave /usr/bin/pip , but the actual pip was located at /usr/local/bin/pip due to pip upgrade and clean up was not complete
So removing the pip in /usr/bin/
sudo rm /usr/bin/pip
and also adding the new pip to your export path
vim ~/.bash_profile
PATH=$PATH:$HOME/bin:/usr/local/bin
exit terminal, and login back
which pip should give /usr/local/bin/pip
pip install --upgrade pip
This works for me
sudo /usr/local/bin/pip install --upgrade pip
To add to angelokh
sudo `which pip` install --upgrade pip
I think the best strategy in this case is to manage pip is as part of a virtual environment using virtualenv rather than messing with the system-level version.
If you're OK with that, here's the basic idea:
Install the version of virtualenv packaged with the version of pip you are looking to upgrade to to the system-level pip (e.g. virtualenv==15.1.0 comes with pip==9.0.1):
$ sudo pip install -U virtualenv==15.1.0
You are using pip version 6.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting virtualenv==15.1.0
Downloading virtualenv-15.1.0-py2.py3-none-any.whl (1.8MB)
100% |████████████████████████████████| 1.8MB 135kB/s
Installing collected packages: virtualenv
Found existing installation: virtualenv 12.0.7
Uninstalling virtualenv-12.0.7:
Successfully uninstalled virtualenv-12.0.7
Successfully installed virtualenv-15.1.0
I used the virtualenv release notes to find out which version of pip corresponds to which version of virtualenv.
Create the virtual environment:
$ virtualenv myenv
New python executable in /home/ec2-user/myenv/bin/python2.7
Also creating executable in /home/ec2-user/myenv/bin/python
Installing setuptools, pip, wheel...done.
Activate the virtual environment and confirm the version and location of the upgraded pip:
$ source myenv/bin/activate
(myenv) $ pip -V
pip 9.0.1 from /home/ec2-user/myenv/local/lib/python2.7/dist-packages (python 2.7)
(myenv) $ which pip
~/myenv/bin/pip
This should allow you to install packages to this virtualenv using the pip version of your choice, without the need for sudo.
I think you've didn't installed the pythonXX-pip package.
I've upgraded mine straight to Python3.4, these commands works for me.
sudo su
yum install python34
yum install python34-pip