We Keep Coding

Should I use pip or pip3? [duplicate]

This question already has answers here:
pip or pip3 to install packages for Python 3?
(10 answers)
Closed 2 years ago.
Eventually, every single time I install a new Linux distribution I do sudo apt-get install python3.
However, once installed I always get confused. python is Python 2.7 and python3 is Python 3.x. But also it appears that pip is for Python 2 and pip3 for Python 3. That said most tutorials I see on Internet always use the traditional pip install even though it is about Python 3.
How should I deal with this? Should I simply continue to put this annoying 3 every time I use Python (pip3, ipython3, python3...)? In most of my lectures I read that creating a symlink python->python3 is a bad practice. Is that correct?

Use python3 -m pip or python -m pip. That will use the correct pip for the python version you want. This method is mentioned in the pip documentation:
python -m pip executes pip using the Python interpreter you specified as python. So /usr/bin/python3.7 -m pip means you are executing pip for your interpreter located at /usr/bin/python3.7.
Symlinking python->python3 is a bad idea because some programs might rely on python being python 2. Though, I have seen some Dockerfiles symlink python->python3, like TensorFlow's CPU dockerfile (it's less of an issue in a Docker image). Coincidentally, that same Dockerfile uses the python3 -m pip install syntax that I recommend.

creating a symlink python->python3 is a bad practice. Is that correct?
Sometimes. Some OSs (looking at you, macOS) deeply rely on python pointing to a Python 2 interpreter for internal tools and tasks. Deleting the shipped Python 2 interpreter (or aliasing python to a Python 3 interpreter) will break stuff. How to uninstall Python 2.7 on a Mac OS X 10.6.4?

Whether the correct command for Python 3 is pip or pip3 or (say) gaschplutzga depends on a number of factors.
If you only have Python 3, and you have a command named pip, that's probably safe to use. Going forward, this will be the simple, obvious, safe answer in more and more places.
If you have both, and there is a command called pip3 installed on your system, probably that's the correct one to use.
More generally, you can go through your PATH and look for commands with suitable names. On Unix-like systems with a POSIX-compatible shell, try the commands command -V pip3 and command -V pip. (On Windows systems, maybe try where pip3 and where pip, or pray to whatever dark deity informed your choice of operating system.)
If you receive output like
/opt/random/nonstandard/whoa/pip
/usr/local/bin/pip
/usr/bin/pip
you can try each of these in turn with the full path and adding the --version option to have them identify themselves. When you specify the full path, you are bypassing the system's PATH mechanism entirely. For example,
/opt/random/nonstandard/whoa/pip --version
might identify itself as belonging to Python version 3.2.1. If that's the one you want, and it's at the top of your PATH, you can simply rely on the PATH to give you this version when you type just pip. If not, perhaps you can shuffle your PATH (but understand that this changes the resolution order for all commands in the directory whose position you change) or create a simple alias or wrapper which bypasses the PATH for this particular command in your personal account. On Unix-like systems with a POSIX-compatible shell, this might look like
alias pip=/opt/random/nonstandard/whoa/pip
(to persist this across sessions, you'd add this to your .profile or similar - for Bash, try .bash_profile if it exists; for Zsh, try .zshrc. The full scoop for each shell is more complicated than I can squeeze into these narrow parentheses); on Windows, you might be able to control this by setting the environment variable PY_PYTHON, but there's a huge can of worms behind that "might".
Some sites and OSes / distros have additional wrappers or conventions which introduce additional options; if you use a specific package manager, perhaps also study its documentation. (One common example is Anaconda, though I don't believe it affects the naming or location of pip specifically.)

Use virtual environments, then pip would be associated with the python used to create that virtual environment. Whether you use pip or pip3, it will be equivalent to python3 -m pip as mentioned in jakub's answer. Also, given that Python 2.7 is already EOL (which means you will most likely work with Python 3) and that pip install-ing things onto the system packages should be avoided, then a virtual environment would be helpful here.
For example, using pipenv:
$ pipenv --python=/usr/local/opt/python#3.8/bin/python3
$ pipenv shell
Launching subshell in virtual environment...
(TEMP) $ pip --version
pip 20.2.3 from /Users/me/.venvs/temp2-SbXvZiFd/lib/python3.8/site-packages/pip (python 3.8)
(TEMP) $ pip3 --version
pip 20.2.3 from /Users/me/.venvs/temp2-SbXvZiFd/lib/python3.8/site-packages/pip (python 3.8)
For example, using venv:
$ python3.8 -m venv .venv
$ source .venv/bin/activate
(.venv) $ pip --version
pip 20.2.3 from /Users/me/temp2/.venv/lib/python3.8/site-packages/pip (python 3.8)
(.venv) $ pip3 --version
pip 20.2.3 from /Users/me/temp2/.venv/lib/python3.8/site-packages/pip (python 3.8)
The virtual environment takes care of making sure pip or pip3 in this env refers to the pip from the correct Python version. You can then happily follow tutorials that still use pip install something (unless of course that tutorial refers to a Python 2.7 or a system-wide installation).

You can install pip through pip3 and this should resolve this issue.
$ pip --version
pip 19.0.3 from /usr/local/lib/python2.7/dist-packages/pip (python 2.7)
Notice that pip here is of Python 2.7 (in this example).
You can then force pip3 of Python 3.X to install pip under itself.
$ sudo pip3 install pip --upgrade
Installing collected packages: pip
Found existing installation: pip 8.1.1
Not uninstalling pip at /usr/lib/python3/dist-packages, outside environment /usr
Successfully installed pip-19.0.3
Once you check this again, it should reference Python 3.X so you don't have to deal with
what is what.
$ pip --version
pip 19.0.3 from /usr/local/lib/python3.5/dist-packages/pip (python 3.5)
I doubt you'll want to use Python 2 after this, but if you do happen to work with Python 2 code, you can create a virtual environment to access those commands again. Otherwise, you won't have to worry about the pip or pip3 distinction after this.
Not really a duplicate of this question, but this helped me suggest this answer: Can pip (python2) and pip3 (python3) coexist?

Pip is for python version less than 3. and pip3 is used when you want to install packages for python version 3 or higher.

pip/python: normal site-packages is not writeable

I have a new Macbook - a user installed it, and then I installed a new user (mine), granted admin privileges and deleted the old one. I am on OS Catalina.
Since the installation I've been having several permission problems.
VSCode can't find Jupyter Notebook, pip installs packages at ~/Library/Python/3.7/site-packages.
When I do which python3 I get usr/bin/python3.
When I do pip3 install <package> I get: Defaulting to user installation because normal site-packages is not writeable And then it says it has already been installed, even though I can't access it when I do import <package>.
It's seems clear that this is a permission problem, pip can't install to the "base" python, and them python can't find what I've installed into ~/Library/Python/3.7/site-packages.
I've tried reinstalling the OS, but since I haven't done a clean install, it didn't change anything.
What am I missing?
How exactly can I fix permissions? Where do I want packages to be installed (venv sure, but some packages I want global (like jupyter).

As #TomdeGeus mentioned in the comments, this command works for me:
Python 3:
python3 -m pip install [package_name]
Python 2:
python -m pip install [package_name]

It's best to not use the system-provided Python directly. Leave that one alone since the OS can change it in undesired ways, as you experienced.
The best practice is to configure your own Python version(s) and manage them on a per-project basis using virtualenv (for Python 2) or venv, possibly via poetry, (for Python 3). This eliminates all dependency on the system-provided Python version, and also isolates each project from other projects on the machine.
Each project can have a different Python point version if needed, and gets its own site_packages directory so pip-installed libraries can also have different versions by project. This approach is a major problem-avoider.

python3.7 -m pip install [package_name]
(you should use the version that you have, of course)
solved it for me.
The most voted answer python3 -m pip install [package_name] does not help me here.
In my case, this was caused by a conflict with the dominating 3.6 version that was also installed as a default. You might ask yourself why you have 3.6 on your system, you will most probably not use that version now. The reason is that 3.6 is used as an independent default python version for many package installers. Those installers do not want to check which individual version you use and whether that fits, they just use 3.6 as a default, if you like it or not.
Here is a proof by example --upgrade pip:
pip3 install --upgrade pip
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pip in
/home/USERNAME/.local/lib/python3.6/site-packages (20.3.1)
python3 -m pip install --upgrade pip
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pip in
/home/USERNAME/.local/lib/python3.6/site-packages (20.3.1)
python3.7 -m pip install --upgrade pip
Collecting pip
Cache entry deserialization failed, entry ignored
Using cached https://files.pythonhosted.org/packages/ab/11/2dc62c5263d9eb322f2f028f7b56cd9d096bb8988fcf82d65fa2e4057afe/pip-20.3.1-py2.py3-none-any.whl
Installing collected packages: pip Successfully installed pip-20.3.1

I'm using Anaconda on Ubuntu and had the same problem.I fixed it by the following steps:
deactivating current environment
conda deactivate
Then, the base environment activates. I deactivated the base conda environment too. To do so, I used conda deactivate again.
Finally, I activate my project environment directly (instead of activating from the base environment) by the following command. Afterward, I installed the intended package successfully and worked perfectly.
conda activate myenv
pip install somepackage

sudo pip install
Worked for me. But pip install is not recommended to be run with sudo. The issue I was facing on BIGSUR was, it was using system python. Once I Installed python 3.9 using
brew install python#3.9
Then pip worked fine

For readers who thought themselves accidentally update system pip:
If you saw this info in your terminal output:
Defaulting to user installation because normal site-packages is not writeable
then you will be fine. Use the pip3 you just updated to run:
pyenv global system # since I use pyenv
pip3 uninstall pip # this one does the trick
Then you can check again pip3 --version will point to the original old (XCode/System-)pip. E.g. (2022/2/28):
pip 20.2.3 from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip (python 3.8)

It occurs with me when I the virtual enviroment folder name was : venv.
in this case, It gives errors like :
No module pip
Default folder is unwritable
renaming the folder solve the proplem.

Check on the command line "which python" to see if it is the value you expect.
If you have a virtual environment activated, check /venv/bin/activate to see the value of VIRTUAL_ENV= and make sure it is the correct path . The path may be wrong if you renamed or moved the project. If the path is wrong, you can delete the venv and make a new one.

For me, none of the suggestions worked so I had to delete the current virtual environment folder venv and recreate it using one of the following commands:
python -m venv venv
python3 -m venv venv
Check the source of pip on Ubuntu 20.04
which pip
returns the correct path
/home/myname/fullstack/person_api/venv/bin/pip
UPDATE
I presume that some might encounter this problem because they set python path as environmental variable like this in ~/.bashrc:
python=/path/to/python
which you should not be doing! Instead we could do:
py=python
PATH=/path/to/python:$PATH
I bumped into this issue specifically because of this!

Had this same issue on a fresh install of Debian 9.12.
Rebooting my server solved the issue.

In my case on Linux, the ownership of the conda env directory had changed to another Linux user (long story), and so the the normal site-packages was not writeable due to a permissions issue.
The solution was to change ownership back to the user doing pip install.

I met exactly the same issue.
I just type sudo python3.8 -m pip install .... and the error disappeared.
If I remove the sudo, issue remains.

For those running on a Pi, that accidentally installed pip as root. Just chown the lib folder to the pi user:
sudo chown -R pi:pi /usr/local/lib/python3.9/

in my case python3 -m pip install [package_name] did not solve that.
in my case, it was a problem related to other processes occupying the directory.
I restart Pycharm and close any other program that might occupy this folder, and reinstalled the package in site-packages directory successfully.

When this problem occurred to me I have tried all the mentioned approaches but they don't seem to work.
Instead, restarting Python language server in my VSCode did the job - my SimPy package is now found. On Mac it is Cmd+Shift+P and select "Python: Restart Language Server".

Had similar issue on Ubuntu 20.04.4 LTS in VirtualBox, but none of the suggestions here worked for me.
I was trying to install open3d in a venv and every time I was getting
"Defaulting to user installation because normal site-packages is not writeable"
which at first I didn't even noticed. open3d was always being installed in /usr/bin/python3 environment. I've restarted the VM but without luck, so I guess the problem was not just missing write access.
So in VS Code, which was using the venv, importing open3d was not possible. But testing from terminal from the activated venv with python3 -c "import open3d as o3d; print(o3d.__version__)" was working fine and that confused me totally. I even broke my system pip installation using sudo, see further below if you want to know how to fix it.
Anyhow, the solution to my problem was to explicitly point to the python3 file in the venv where I wanted to install the package:
venv/bin/python3 -m pip install open3d
So I was testing out everything and eventually installed with sudo: sudo pip3 install open3d. This of course didn't solved the problem and open3d was still missing in the venv. Even worse, I got the message:
"WARNING: You are using pip version 21.3.1; however, version 22.0.4 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command."
So I did it but with sudo, updating the system pip and then found out here that this is not good:
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Following an advice here, I tried to revert to original version, only then pip3 broke:
sudo pip3 uninstall pip
sudo pip3 --version
sudo: pip3: command not found
The apt package was still there:
sudo apt install python3-pip python3-pip is already the newest version
(20.0.2-5ubuntu1.6).
So I had to reinstalled to fix the problem:
sudo apt-get remove python3-pip
sudo apt install python3-pip

Maybe you have python, python3, pip or pip3 aliased. In that case pip might not work well anymore, as the alias isn't always available and so pip/pip3 might resolve python/python3 differently compared to in your terminal.
That could give rise to pip/pip3 trying to install in the system python, and that could give rise to your error.

I tried ever single recommendation described here. In every instance, I get the exact same result: SyntaxError: invalid syntax (<stdin>, line 1)
I'm not sure who designed the system like this, but it seems basically useless, based on my experience so far. Either create a system that works, or don't create anything at all.

Using pip to upgrade packages originally installed via apt

I would like to install the newest version of docutils via pip, but it can't figure out how to upgrade the system version installed via apt.
$ sudo --set-home python2 -m pip install --upgrade docutils
Collecting docutils
Using cached https://files.pythonhosted.org/packages/3a/dc/bf2b15d1fa15a6f7a9e77a61b74ecbbae7258558fcda8ffc9a6638a6b327/docutils-0.15.2-py2-none-any.whl
Installing collected packages: docutils
Found existing installation: docutils 0.14
ERROR: Cannot uninstall 'docutils'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
$ apt-cache show python-docutils | head -n 3
Package: python-docutils
Architecture: all
Version: 0.14+dfsg-3
None of the solutions I've thought of or found on the web appeal:
Delete the apt version with rm -rf /usr/lib/python2.7/dist-packages/docutils*. This silences pip but means:
Installed files on the system no longer match what the Debian packaging system thinks
I might break dependencies of system software on docutils 0.14
Any updates to the Debian package will cause apt to reinstall
Other problems discussed in this answer
pip install --force-reinstall. (Same problems.)
pip install --ignore-install. (Same problems.)
Is there a way to get a default environment that works for me with the newest versions of stuff from pip but has no chance of breaking any system software? The same answer above suggests using one of virtualenv, venv, pyenv, pipenv. I tried pipenv and it doesn't want to install individual packages listed on the commandline using --system and I don't know whether creating a Pipfile will actually solve this problem.
I would rather not have to manually switch environments somehow to use the apt-installed packages versus the pip-installed packages. Is there a way to get only apt-installed software to use one environment and otherwise use the environment with the pip-installed stuff?

I would rather not have to manually switch environments somehow to use the apt-installed packages versus the pip-installed packages. Is there a way to get only apt-installed software to use one environment and otherwise use the environment with the pip-installed stuff?
Ideally, one should use either the system version or the pip version.
Per the Debian Python Policy,
As long as you don't install other versions of Python in your path, Debian's Python versions won't be affected by a new version.
If you install a different micro version of the version of Python you have got installed, you will need to be careful to install all the modules you use for that version of Python too.

So far adding the following to ~/.bashrc seems work well:
if [ ! -d ~/venv/python3 ]; then
python3 -m venv --system-site-packages ~/venv/python3
fi
if [ -d ~/venv/python3 ]; then
VIRTUAL_ENV_DISABLE_PROMPT=1 . ~/venv/python3/bin/activate
fi
Most of the system-installed scripts have one of the Pythons in /usr/bin hard-coded instead of using /usr/bin/env python so they are unaffected by this.

Tox installs the wrong version of pip to it's virtual env

I am using tox to manage some testing environments. I have a dependency (backports.ssl-match-hostname) that I cannot download using the latest version of pip, so I need to revert back to pip 8.0.3 to allow the install to work.
I have included the 8.0.3 version of pip inside my tox.ini file for dependencies.
deps=
pip==8.0.3
However, when I run
source .tox/py27/bin/activate
and enter the virtual testing environment, and then run
pip --version
I end up with
8.1.2
However, outside of my tox environment, when I run the same command, I get
8.0.3
Is there anything special that tox does when grabbing pip? Why am I not able to specify the version of pip that I want to use as a dependency?
EDIT : to add to this, it seems as though I am able to grab the dependency pip==8.0.3, but for the other dependencies, they are still running from the command launched with pip==8.1.2
So, I need to be able to grab pip==8.0.3 first, and then once installed, grab everything else. Still unsure why tox is starting with pip==8.1.2

This was apparently the result of the "virtualenvs" python package containing a pre-selected group of python packages that it refers to, one of which was the latest and greatest pip.
I don't know if this is the preferred way of doing this, but I found success by running
pip uninstall virtualenv
And then reinstalling with the version that worked
pip install virtualenv==15.0.1
With the "correct" version of virtualenv in place, I was able to run my tox command
source .tox/py27/bin/activate
and see the desired version of pip
pip --version
pip 8.0.3

A workaround for this is here: https://github.com/pypa/pip/issues/3666
Although to make it work I had to write "pip install pip==8.1.1" in my script. So to recap:
Add a pip.sh script to your project:
#!/bin/bash
pip install pip==8.1.1
pip install "$#"
Add to your tox.ini:
install_command = {toxinidir}/pip.sh {opts} {packages}

I've recently hit this problem. I've had it for a while but it just didn't register because I had such occasional failures with Python 2/3 code. Another way that this can happen is, if like me, you change the virtualenv between different Python versions and don't clean up.
Check /bin or /Scripts to see whether python2 points to python. If the virtualenv is Python 3 then this will mean that python2 actually calls Python 3. Vice versa, of course, if you the virtualenv is Python 2 and you want to test Python 3 code.

New versions of virtualenv reach out to download the latest pip, setuptools, and wheel -- you can disable this behavior when running through tox with the tox-virtualenv-no-download package See: https://github.com/asottile/tox-virtualenv-no-download#wait-why

In a virtualenv, pip can see all of my site-packages when I use list, is this normal?

I thought that virtualenv was supposed to encapsulate and hide all of your packages that were already installed. But when I type
$sudo virtualenv -p /usr/bin/python3 testenv
$source ~/testenv/bin/activate
$sudo pip list
I get:
apt-xapian-index (0.45)
argparse (1.2.1)
chardet (2.0.1)
cmsplugin-filer (0.10)
colorama (0.2.5)
command-not-found (0.3)
debtagshw (0.1)
defer (1.0.6)
dirspec (13.10)
...and many more
Even with the --local parameter. Is virtualenv broken?
Also when I type: $ which pip I still get: /home/jelikraftuser/testenv/bin/pip Which seems correct.
Reading the answers in this post: pip installing in global site-packages instead of virtualenv
I found the suggestion to run pip directly with $sudo ~/testenv/bin/pip list and it actually worked, it only listed 2 packages. However when I run which pip it lists the pip in the virtualenv as being the one which would be run.
So I'm sort of lost at this point. Calling the pip list with the full path gives me the correct (small) list of two packages, and calling pip list without the full path gives me a giant list of packages, which is incorrect. So, where do I go from here?
How can I make it not recognise globally installed packages as being installed in the virtualenv when I run pip without the full path?
OKAY UPDATE! This is kinda interesting:
(testenv)$ pip --version
pip 1.5.6 from /home/jelikraftuser/testenv/lib/python3.4/site-packages (python 3.4)
(testenv)$ sudo pip --version
pip 1.5.4 from /usr/lib/python2.7/dist-packages (python 2.7)
When I run pip as sudo it runs one, and when i run it as non-sudo it runs a different pip. Why would it do that? And if I'm going to be installing a package, i'll be running it as sudo, so I need sudo to use the correct pip.
Second update:
Reading this stackoverflow:sudo changes PATH - why?
I found that on ubuntu you cannot change the path variable for sudo, but this still confuses me since it was working before... So I'm still confused. Insight anyone? Previously I could type sudo pip list in a virtualenv and get a near-empty list. Does it do the same for you?
EDIT 3: What else it does:
When I run sudo pip install --download-cache=~/.pip-cache -r piprequirements.txt
it says that everything is already installed but when I enter python I cannot import them, but when I run python as sudo I can import them. So superuser can see packages that are globally installed > but I need to use sudo to install packages > so I can't install packages that are already globally installed. Also when I try to run pip3 as sudo, it says sudo: pip3: command not found. So this is definately an issue with ubuntu and how the environment path changes when you run sudo. Is not everyone else running into this issue then? I'm sure lots of people are using ubuntu, no?