Is there any difference running package installation for a python project using these two commands?
python -m pip install <package>
pip install <package>
NOTE I'm using venv in my project.
They are nearly equivalent but they might point to different python installations (and versions). python -m pip gives you more explicit control since you can specify python3.8 -m pip and know that you are installing for the Python 3.8 interpreter. This is not explicitly obvious with pip without more investigation. Brett Cannon wrote a nice blog post: https://snarky.ca/why-you-should-use-python-m-pip/
Short answer
They are probably equivalent.
Longer answer
When you run python -m pip, you are referencing a module of Python called pip. The python command will use your installed version of Python (based on your PATH variable) - So the corresponding pip version will be used.
When you run pip install, a pip module is located by searching the PATH variable, and not by using python. This could be a different pip module than in python -m pip, but it usually isn't.
Edit: In the case of running under a virtual environment, the PATH variable should contain the virtual environment path, so both will be the same - the python that is used will use the pip from the same virtual environment.
Related
This question already has answers here:
pip or pip3 to install packages for Python 3?
(10 answers)
Closed 2 years ago.
Eventually, every single time I install a new Linux distribution I do sudo apt-get install python3.
However, once installed I always get confused. python is Python 2.7 and python3 is Python 3.x. But also it appears that pip is for Python 2 and pip3 for Python 3. That said most tutorials I see on Internet always use the traditional pip install even though it is about Python 3.
How should I deal with this? Should I simply continue to put this annoying 3 every time I use Python (pip3, ipython3, python3...)? In most of my lectures I read that creating a symlink python->python3 is a bad practice. Is that correct?
Use python3 -m pip or python -m pip. That will use the correct pip for the python version you want. This method is mentioned in the pip documentation:
python -m pip executes pip using the Python interpreter you specified as python. So /usr/bin/python3.7 -m pip means you are executing pip for your interpreter located at /usr/bin/python3.7.
Symlinking python->python3 is a bad idea because some programs might rely on python being python 2. Though, I have seen some Dockerfiles symlink python->python3, like TensorFlow's CPU dockerfile (it's less of an issue in a Docker image). Coincidentally, that same Dockerfile uses the python3 -m pip install syntax that I recommend.
creating a symlink python->python3 is a bad practice. Is that correct?
Sometimes. Some OSs (looking at you, macOS) deeply rely on python pointing to a Python 2 interpreter for internal tools and tasks. Deleting the shipped Python 2 interpreter (or aliasing python to a Python 3 interpreter) will break stuff. How to uninstall Python 2.7 on a Mac OS X 10.6.4?
Whether the correct command for Python 3 is pip or pip3 or (say) gaschplutzga depends on a number of factors.
If you only have Python 3, and you have a command named pip, that's probably safe to use. Going forward, this will be the simple, obvious, safe answer in more and more places.
If you have both, and there is a command called pip3 installed on your system, probably that's the correct one to use.
More generally, you can go through your PATH and look for commands with suitable names. On Unix-like systems with a POSIX-compatible shell, try the commands command -V pip3 and command -V pip. (On Windows systems, maybe try where pip3 and where pip, or pray to whatever dark deity informed your choice of operating system.)
If you receive output like
/opt/random/nonstandard/whoa/pip
/usr/local/bin/pip
/usr/bin/pip
you can try each of these in turn with the full path and adding the --version option to have them identify themselves. When you specify the full path, you are bypassing the system's PATH mechanism entirely. For example,
/opt/random/nonstandard/whoa/pip --version
might identify itself as belonging to Python version 3.2.1. If that's the one you want, and it's at the top of your PATH, you can simply rely on the PATH to give you this version when you type just pip. If not, perhaps you can shuffle your PATH (but understand that this changes the resolution order for all commands in the directory whose position you change) or create a simple alias or wrapper which bypasses the PATH for this particular command in your personal account. On Unix-like systems with a POSIX-compatible shell, this might look like
alias pip=/opt/random/nonstandard/whoa/pip
(to persist this across sessions, you'd add this to your .profile or similar - for Bash, try .bash_profile if it exists; for Zsh, try .zshrc. The full scoop for each shell is more complicated than I can squeeze into these narrow parentheses); on Windows, you might be able to control this by setting the environment variable PY_PYTHON, but there's a huge can of worms behind that "might".
Some sites and OSes / distros have additional wrappers or conventions which introduce additional options; if you use a specific package manager, perhaps also study its documentation. (One common example is Anaconda, though I don't believe it affects the naming or location of pip specifically.)
Use virtual environments, then pip would be associated with the python used to create that virtual environment. Whether you use pip or pip3, it will be equivalent to python3 -m pip as mentioned in jakub's answer. Also, given that Python 2.7 is already EOL (which means you will most likely work with Python 3) and that pip install-ing things onto the system packages should be avoided, then a virtual environment would be helpful here.
For example, using pipenv:
$ pipenv --python=/usr/local/opt/python#3.8/bin/python3
$ pipenv shell
Launching subshell in virtual environment...
(TEMP) $ pip --version
pip 20.2.3 from /Users/me/.venvs/temp2-SbXvZiFd/lib/python3.8/site-packages/pip (python 3.8)
(TEMP) $ pip3 --version
pip 20.2.3 from /Users/me/.venvs/temp2-SbXvZiFd/lib/python3.8/site-packages/pip (python 3.8)
For example, using venv:
$ python3.8 -m venv .venv
$ source .venv/bin/activate
(.venv) $ pip --version
pip 20.2.3 from /Users/me/temp2/.venv/lib/python3.8/site-packages/pip (python 3.8)
(.venv) $ pip3 --version
pip 20.2.3 from /Users/me/temp2/.venv/lib/python3.8/site-packages/pip (python 3.8)
The virtual environment takes care of making sure pip or pip3 in this env refers to the pip from the correct Python version. You can then happily follow tutorials that still use pip install something (unless of course that tutorial refers to a Python 2.7 or a system-wide installation).
You can install pip through pip3 and this should resolve this issue.
$ pip --version
pip 19.0.3 from /usr/local/lib/python2.7/dist-packages/pip (python 2.7)
Notice that pip here is of Python 2.7 (in this example).
You can then force pip3 of Python 3.X to install pip under itself.
$ sudo pip3 install pip --upgrade
Installing collected packages: pip
Found existing installation: pip 8.1.1
Not uninstalling pip at /usr/lib/python3/dist-packages, outside environment /usr
Successfully installed pip-19.0.3
Once you check this again, it should reference Python 3.X so you don't have to deal with
what is what.
$ pip --version
pip 19.0.3 from /usr/local/lib/python3.5/dist-packages/pip (python 3.5)
I doubt you'll want to use Python 2 after this, but if you do happen to work with Python 2 code, you can create a virtual environment to access those commands again. Otherwise, you won't have to worry about the pip or pip3 distinction after this.
Not really a duplicate of this question, but this helped me suggest this answer: Can pip (python2) and pip3 (python3) coexist?
Pip is for python version less than 3. and pip3 is used when you want to install packages for python version 3 or higher.
Edit:
I'm going to close this question as the reason its happening is different from my original assumption, and it's clearer to ask the question anew:
Pip installs packages in the wrong directory with virtualenv
The accepted answer doesn't directly answer the original question, but is a very useful overview.
Based on discussion below the issue is that even after
$ source ~/PycharmProjects/Practice/venv/bin/activate
$ pip install numpy
numpy is installed in /usr/local/lib/python2.7/site-packages
What could the reason for this be?
Original:
Using Python on OS X via Homebrew:
I've been trying much of the day to sort this out but either I get a must supply either home or prefix/exec-prefix -- not both error, or the package I try to install goes into totally the wrong place:
$ pip3 --version
pip 18.1 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
$ cd venv
$ pip3 install numpy
..... [snip with following error:]
"must supply either home or prefix/exec-prefix -- not both")
Using this hint
$ pip3 install numpy -t .
Then I get a new error,
`Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/.../pip-install-0fvveq3v/package/'
Searching around SO gives various possibilities involving pip install setuptools. but pip install throws the above error or installs in the wrong place. i.e. the solution involves something that's causing the error in the first place.
I tried to use the Python.org installer but it didn't install pip at all. (The custom installer showed the option checked but with size zero).
An introductory overview is available in this nice tutorial. Here is a good summary with more detail. But, if you renamed or moved the virtual env dir after its creation, it could break it. Create a new one from scratch: $ cd ~/PycharmProjects; python3 -mvenv newenv ; Activate: $ source newenv/bin/activate ; Install something: $ pip install colorama (same as pip3 install only if venv activated); Check: ls ~/PycharmProjects/newenv/lib/python3*/site-packages ; Deactivate: $ deactivate
Then you could try this solution for Pycharm: how to associate a virtual environment with a python project in pycharm. PyCharm indeed comes bundled with virtualenv which could have been customized, please look into Pycharm-specific resources: creating virtual environments and installing packages in Pycharm.
If you have installed PyPI's mainstream virtualenv, by default it will create new environments with the python interpreter that virtualenv was installed with. But it's possible to specify an alternate Python Interpreter upon a new env creation: $ virtualenv -p python3.7 newenvname
Regarding the error DistutilsOptionError: must supply either home or prefix - please check this and this for solutions. Homebrew'ed mappings between python and pip are described here. The normal pip install --user is disabled in homebrewed Python, but there are workarounds. MacOS system Python doesn't provide pip, but it can installed, reinstalled or upgraded for any specific python version manually. Original non-brewed installers are also available for all Python versions: https://www.python.org/downloads/mac-osx/
By default there's no pip.conf, but it can be created by hand to customize things. All possible pip.conf locations (per-user, per-venv, and global/system-wide, and how they override each other) are listed here. If someone faces an issue, they could use pip config list command to see their active configuration, or locate pip.conf and find it.
Finally, you may want to ensure you aren't using pip against macOS' system python. Shell commands such as $ brew info python, which pip, which pip3, pip3 -V, which python3 can help you see what you actually use. Since the macOS default $PATH used to be /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin, stock macOS binaries (including python) may take precedence over some homebrew'ed installations (including python). If so, a custom PATH could be exported via the ~/.bashrc if required.
I have the following version of python
import sys
print(sys.version)
3.6.5 | packaged by conda-forge | (default, Apr 6 2018, 13:44:09)
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)]
I installed a package with the following command
pip install wfdb
It is succesfully installed because when I then write the command:
pip show wfdb
The following information appears
Location:
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages
However, when I type the command import wfdb in Python notebook or the version of python in terminal, I get the following message
No module named 'wfdb'
Does it have to do with the path on which python is checking where the packages are? How to check this and how to change it?
You have (at least) 2 Python installations, one managed by Anaconda, the other what appears to be an official Python.org Mac build installed system-wide. The pip command on the command-line is the one tied to the Python.org Mac build.
pip is a script that is tied to a specific Python installation, and there can be multiple versions of the script installed in different locations, and is usually also installed with pipX and pipX.Y to match the X.Y version indicator of the Python version it is tied to. For Python 3.6, that means the same script would also be available as pip3 and pip3.6. (This also means that pip can be connected to Python 2 or Python 3, depending on your exact OS setup. It is not a given that pip, without a version number, installs into Python 2.x as some answers may claim).
Note that when you run a command without a path in your shell, (such as pip as opposed to /usr/bin/pip), you are asking your shell to find the command for you in a number of locations, listed in the PATH environment variable. The first location in the PATH list with that command is then fixed. which -a <command> would tell you all possible PATH-registered locations that the command can be found in. You can always use the full path to a command to bypass the PATH search path.
You can always verify what Python version the pip command is connected to with:
pip -V
which will output the version of pip and the location it is installed with. It'll print something like
pip pipX.pipY path/to/pythonX.Y/site-packages/pip (python X.Y)
where pipX.pipY is the pip version number and path/to/pythonX.Y tells you what Python installation this is for.
You can try to match this with the Python version by running
python -m site
which outputs the Python module search path for that Python version. Python can be run with python, pythonX and pythonX.Y too, and is subject to the same PATH search.
Note the -m switch there, that instructs Python to find a module in it's module search path and execute it as a script. Loads of modules support being run that way, including pip. This is important as that helps avoid having to search for a better pip command if you already can start the right Python version.
You have several good options here:
Since you are using Anaconda, you could look for a conda package for the same project. There is such a package for wfdb. Install it with
conda install wfdb
Anaconda aims to give you a wider software management experience that includes a broader set of software options than just the Python PyPI ecosystem, and conda packages usually manage more things than just the Python package.
Conda packages are usually maintained by a different set of developers from the package itself, so there may be a newer version available on PyPI (requiring pip install) than there is on Conda.
This is not an option for all Python packages, if there is no conda package you have to use pip. See Installing non-conda packages.
you can use the conda command to create a conda environment. Once you have an environment created, you can activate it with
source activate <name_of_cenv>
to alter your PATH settings. With the envirnoment 'active' the first directory listed on your PATH is the one for the conda environment and the pip command will be the one tied to that environment.
Note that a conda environment gives you an isolated environment for a specific project, keeping the library installation separate from the rest of your Python packages in the central site-packages location. If you want to install a package for all of your Anaconda Python projects, don't use a conda environment.
Use the Anaconda Python binary to run pip as a module; when you can run /path/to/python or pythoncommand to open the right Python version, you can use that same path to run /path/to/python -m pip ... instead of pip ... to be absolutely certain you are installing into the correct Python version.
Look for a better pip command, with which -a pip or which -a pip3.6, etc. But if you already know the Python binary, look in the same bin location for pip. If you have anaconda/bin/python, then there probably is a anaconda/bin/pip too.
As you can read here:
pip3 and pip would make a difference only when you are not using any
environment managers like virualenv (or) conda. Now as you are
creating a conda environment which has python==3.x, pip would be
equivalent to pip3.
For this reason it could be you did not activate your Conda environment before installing required packages and running your code.
Activate the new environment:
On Windows:
activate myenv
On macOS (this should be your option) and Linux:
source activate myenv
NOTE: Replace myenv with the name of the environment.
which python
gives the you the PATH to python
and then /path/to/python -m pip install thepackagetobeinstalled
Many thanks #MartijnPieters
You have installed python2.x package and you're using python3.x. Try:
pip3 install wfdb
If you don't have pip3 run:
[apt-get/yum] install python3-pip
You can see what packages you have currently installed by running:
pip freeze
and for python 3.x packages
pip3 freeze
Please remember each time you install a Python package, it will be placed in the directory for one particular Python version. Hence your error.
I have python2.7 and python3.6 installed side by side in my computer. Now when I install a package using "pip install", how can I know in which python's site-packages is my package going to be installed?
Thank you.
When you have both version 2 and 3 installations pip and pip3 differentiate the target installtion.
For installing anything on Python 3(versions 3.5 and above) use pip3
for Python 2.7 use pip
Make sure python path is set in environment variables too.
also you can use where pip or which pip as #mshsayem mentioned.
Additional Reference
if you use virtualenv, the modules are located in:
{path_to_your_virtualenv}/lib/python{your_python_version}/site-packages/
and if you don't use virtualenv, normally are installed in:
/usr/local/lib/python{your_python_version}
You have to use pip3 for install python3 modules.
Check where a specific package is installed by:
pip3 show <package_name>
List all installed packages with install locations by:
pip3 list -v
Check the install location used by default when installed without sudo:
pip3 --version
and the location for packages installed with sudo, meaning system-wide installation:
sudo pip3 --version
You can find the location of pip by which pip. Then you view the pip executable header using head `which pip` or using your preferred editor. You can find the python interpreter location on the first line. You may have a pip2 and a pip3 executable.
By the way, you can run pip as a python module by python -m pip <command>. In this way, you can specify your python interpreter.
The answer to you question is divided to two parts:
1. Which python version the native terminal selects for me?
2. How do I specify which python version to use?
Which python version the native terminal selects for me?
In windows, the default pip that will be used is the one associated with the default python version you use. You can edit it in the PATH environmental variable (Start->find-type "Environmental" and click "Edit system variables"). Look the PATH variable and see which version of python is listed. If both versions are listed, windows will select the first.
See more information on system environmental variables here.
In Ubuntu/Linux, usually pip is associated with the native legacy version (2.7), pip3 is associated with Python3.5.x and pip3.6 is associated with Python3.6.x.
However, if you are using Unix OS (such as Ubuntu) or Mac, it is highly recommended to use virtualenv and activate it. See Official documentation to see how to use it. It's true for both Python2.7 and
Python3.6. In short, you will create a lightweight copy of you python installation without any packages, and, your installed packages will be installed within this virtual environment. Once you activate a virtual environment, the pip is associated with this environment.
How do I specify which python version to use?
You have multiple choices to specify in which environment you want to install the package. It depends if you are on Windows/Linux/MAC.
Shortly, you have the following options:
Use an IDE and let it help you manage your packages (e.g. Pycharm). Using PyCharm, you will find it very easy to use its package manager. You can also open the IDE's terminal and when you use pip, it will use the package manager of the selected interpreter. See official documentation.
Use OS native terminal and specify the version. In windows, the easiest way is to go to a command line or powershell, and type "c:\path\to\python.exe -m pip install ". On Ubuntu, use pip/pip3/pip3.6. Again, on Ubuntu it is highly recommended to use venv (virtual environment) since installing wrong package on the wrong version can interrupt the native python (Ubuntu uses python for multiple reasons such as the GNOME GUI).
Use virtual environments. You can look it up, there are plenty of threads explaining on that, as well as the Official documentation.
I need to work with Both Python 2.7.12 and python 3.5.2 simultaneously on my Ubuntu 16.04.1 LTS. Python 3 came pre-installed so I've no idea where it sits, in terms of path to the directory, while python 2 sits in /usr/local/lib/python2.7/.
I found lots of questions on SO and on askubuntu about how to install but nothing about how to use them separately, installing different libraries, and what should I avoid or be careful of, if I maintain this dual python thing for the long term? For example, I usually run pip install to install a library and I can check that its installed in my python2 directory but how do I install the same package for my python3 without conflicts? Something like: python3 pip install <package> ?? Where is the default python3 installed? And how do I call python3 for paths where python is not part of the command for example: pip freeze, sudo-apt get, etc.?
PS: I've not officially worked with Virtualenv but I've been informed that is usually good for isolating projects within a python language version, rather than isolating two different language versions from each other.
Please let me know.
Thanks
This is absolutely no problem, as Python does that for you. You don't need a virtualenv at all.
If you use Ubuntu packages, make sure you use the python3- versions for Python 3, and the normal python- versions for Python 2.
For example, python3-numpy and python-numpy.
If you use pip to install extra packages, you an either use the pip script with the version number appended: pip2.7 or pip3.5, or, my preferred method, call pip as a module for the respective Python executable:
python2.7 -m pip install <whatever>
and
python3.5 -m pip install <whatever>
Other than that, there should not be any issue: Python stores the packages in completely separate directories, and each Python executable only uses its respective directive.
Do not fiddle around with PYTHONPATH, unless you really know what you're doing. This has the danger of setting your PYTHONPATH to a directory with Python 2.7 modules and then using Python 3.5 to run things.
If you start from scratch, you may need to install pip first.
For the system Python(s), use the relevant package:
sudo apt install python-pip
sudo apt install python3-pip
For your locally installed Python(s), use the built-in bootstrapper module:
pythonx.y -m ensurepip
Note on the OS-installed Python executables:
Python 3.5 lives at /usr/bin/python3.5, Python 2.7 (the OS one) at /usr/bin/python2.7.
You could even use the OS 2.7 one next to your locally installed /usr/local/bin/python2.7 (and confuse yourself when a package can't be found because you used the wrong one).
Or install Python 3.6 next to Python 3.5 (provided you've used make altinstall, so python3 doesn't get overwritten).
This is also why you don't really want to run pip (or even pip2.7) as is: pip2.7 may get you the system one, instead of the one in /usr/local/bin/pip2.7, depending on your PATH.
(The same goes for the python2.7 executable, so if you need to specify the full path /usr/local/bin/python2.7 to run that one (or have an alias), the same holds for pip2.7. If, on the other hand, /usr/local/bin is first on your PATH, you should in principle never run into the same pip and python executables.)