What is the official "preferred" way to install pip and virtualenv systemwide? - python

Is it this, which people seem to recommend most often:
$ sudo apt-get install python-setuptools
$ sudo easy_install pip
$ sudo pip install virtualenv
Or this, which I got from http://www.pip-installer.org/en/latest/installing.html:
$ curl -O https://github.com/pypa/virtualenv/raw/master/virtualenv.py
$ python virtualenv.py my_new_env
$ . my_new_env/bin/activate
(my_new_env)$ pip install ...
Or something entirely different?

If you can install the latest Python (2.7.9 and up) Pip is now bundled with it.
See: https://docs.python.org/2.7//installing/index.html
If not :
Update (from the release notes):
Beginning with v1.5.1, pip does not require setuptools prior to running get-pip.py. Additionally, if setuptools (or distribute) is not already installed, get-pip.py will install setuptools for you.
I now run the regular:
curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | sudo python
Here are the official installation instructions:
http://pip.readthedocs.org/en/latest/installing.html#install-pip
EDIT 25-Jul-2013:
Changed URL for setuptools install.
EDIT 10-Feb-2014:
Removed setuptools install (thanks #Ciantic)
EDIT 26-Jun-2014:
Updated URL again (thanks #LarsH)
EDIT 1-Mar-2015:
Pip is now bundled with Python

http://www.pip-installer.org/en/latest/installing.html is really the canonical answer to this question.
Specifically, the systemwide instructions are:
$ curl -O http://python-distribute.org/distribute_setup.py
$ python distribute_setup.py
$ curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
$ python get-pip.py
The section quoted in the question is the virtualenv instructions rather than the systemwide ones. The easy_install instructions have been around for longer, but it isn't necessary to do it that way any more.

This answer comes from #webology on Twitter:
$ sudo apt-get install python-setuptools
$ sudo easy_install pip
$ sudo pip install --upgrade pip virtualenv virtualenvwrapper
My added notes:
On Mac/Windows (and Linux if the apt repo is outdated) you'd replace the first step with downloading setuptools from http://pypi.python.org/pypi/setuptools
On Windows you'd have to omit virtualenvwrapper from the last step and install it manually somehow. I don't know if there's a way to do this without Cygwin, but I hope so.

On Ubuntu 12.04 I've had good luck just using the package manager:
sudo apt-get install python-pip virtualenvwrapper

There is no preferred method - everything depends on your needs. Often you need to have different Python interpreters on the system for whatever reason. In this case you need to install the stuff individually for each interpreter. Apart from that: I prefer installing stuff myself instead of depending of pre-packaged stuff sometimes causing issues - but that's only one possible opionion.

There really isn't a single "answer" to this question, but there are definitely some helpful concepts that can help you to come to a decision.
The first question that needs to be answered in your use case is "Do I want to use the system Python?" If you want to use the Python distributed with your operating system, then using the apt-get install method may be just fine. Depending on the operating system distribution method though, you still have to ask some more questions, such as "Do I want to install multiple versions of this package?" If the answer is yes, then it is probably not a good idea to use something like apt. Dpkg pretty much will just untar an archive at the root of the filesystem, so it is up to the package maintainer to make sure the package installs safely under very little assumptions. In the case of most debian packages, I would assume (someone can feel free to correct me here) that they simply untar and provide a top level package.
For example, say the package is "virtualenv", you'd end up with /usr/lib/python2.x/site-packages/virtualenv. If you install it with easy_install you'd get something like /usr/lib/python2.x/site-packages/virtualenv.egg-link that might point to /usr/lib/python2.x/site-packages/virtualenv-1.2-2.x.egg which may be a directory or zipped egg. Pip does something similar although it doesn't use eggs and instead will place the top level package directly in the lib directory.
I might be off on the paths, but the point is that each method takes into account different needs. This is why tools like virtualenv are helpful as they allow you to sandbox your Python libraries such that you can have any combination you need of libraries and versions.
Setuptools also allows installing packages as multiversion which means there is not a singular module_name.egg-link created. To import those packages you need to use pkg_resources and the __import__ function.
Going back to your original question, if you are happy with the system python and plan on using virtualenv and pip to build environments for different applications, then installing virtualenv and / or pip at the system level using apt-get seems totally appropriate. One word of caution though is that if you plan on upgrading your distributions Python, that may have a ripple effect through your virtualenvs if you linked back to your system site packages.
I should also mention that none of these options is inherently better than the others. They simply take different approaches. Using the system version is an excellent way to install Python applications, yet it can be a very difficult way to develop with Python. Easy install and setuptools is very convenient in a world without virtualenv, but if you need to use different versions of the same library, then it also become rather unwieldy. Pip and virtualenv really act more like a virtual machine. Instead of taking care to install things side by side, you just create an whole new environment. The downside here is that 30+ virtualenvs later you might have used up quite bit of diskspace and cluttered up your filesystem.
As you can see, with the many options it is difficult to say which method to use, but with a little investigation into your use cases, you should be able to find a method that works.

Do this:
curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
python get-pip.py
pip install virtualenv
See
https://pip.pypa.io/en/latest/installing.html
http://docs.python-guide.org/en/latest/dev/virtualenvs/

Since virtualenvs contain pip by default, I almost never install pip globally. What I do ends up looking more like:
$ sudo apt-get install python-setuptools
$ curl -O http://python-distribute.org/distribute_setup.py
$ sudo python distribute_setup.py
$ sudo easy_install virtualenv
I then proceed to install and set up virtualenvwrapper to my liking and off I go. it might also be worthwhile to take a look at Jeremy Avnet's virtualenv-burrito:
https://github.com/brainsik/virtualenv-burrito

#ericholscher says on Twitter, "The one in the official docs.."
It's a great point, you should do what the docs say.
Quoted from the official pip installation instructions at http://www.pip-installer.org/en/latest/installing.html:
$ curl -O https://github.com/pypa/virtualenv/raw/master/virtualenv.py
$ python virtualenv.py my_new_env
$ . my_new_env/bin/activate
(my_new_env)$ pip install ...

Starting from distro packages, you can either use:
sudo apt-get install python-virtualenv
which lets you create virtualenvs, or
sudo apt-get install python{,3}-pip
which lets you install arbitrary packages to your home directory.
If you're used to virtualenv, the first command gives you everything you need (remember, pip is bundled and will be installed in any virtualenv you create).
If you just want to install packages, the second command gives you what you need. Use pip like this:
pip install --user something
and put something like
PATH=~/.local/bin:$PATH
in your ~/.bashrc.
If your distro is ancient and you don't want to use its packages at all (except for Python itself, probably), you can download virtualenv, either as a tarball or as a standalone script:
wget -O ~/bin/virtualenv https://raw.github.com/pypa/virtualenv/master/virtualenv.py
chmod +x ~/bin/virtualenv
If your distro is more of the bleeding edge kind, Python3.3 has built-in virtualenv-like abilities:
python3 -m venv ./venv
This runs way faster, but setuptools and pip aren't included.

To install pip on a mac (osx), the following one liner worked great for me:
sudo easy_install pip

On Debian the best way to do it would be
sudo apt-get install python-pip

In Raspbian, there is even no need to mention python2.7. Indeed this is best way to install pip if python version in less then 2.7.9.
curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | sudo python
Thanks to #tal-weiss

https://github.com/pypa/pip/raw/master/contrib/get-pip.py is probably the right way now.

I use get-pip and virtualenv-burrito to install all this. Not sure if python-setuptools is required.
# might be optional. I install as part of my standard ubuntu setup script
sudo apt-get -y install python-setuptools
# install pip (using get-pip.py from pip contrib)
curl -O https://raw.github.com/pypa/pip/develop/contrib/get-pip.py && sudo python get-pip.py
# one-line virtualenv and virtualenvwrapper using virtualenv-burrito
curl -s https://raw.github.com/brainsik/virtualenv-burrito/master/virtualenv-burrito.sh | bash

The former method is fine. The only problem I can see is that you might end up with an old version of setuptools (if the apt repository hasn't been kept up-to-date..

Related

What is the suggested way to install multiple Python interpreters?

With Ubuntu, only certain Python versions are available from the repositories. They're easy to get, but with the rest, it's not so obvious. While I can certainly build them from source, I'm thinking there must be tools to automate the process but I can't find them.
Windows and OS X users can simply use executable installers available on the Python website. This is not true with the other OS options. If your Linux distribution's repositories do not contain all the versions you want, building from source is the recommended way, at least as far as the website is concerned.
Automating this process is possible. pythonbrew is a now-deprecated installation manager that recommends shell script-based pyenv instead. With this, installing a new version is as simple as
pyenv install <version number, e.g. 2.7.3>
Another alternative is the more Python-based (read: cross platform) pythonz, a fork of the original pythonbrew. It is just as easy:
pythonz install <version number, e.g. 2.7.3>
One way of installing multiple versions of Python in Ubuntu is to use Felix Krull's deadsnakes ppa, which includes all the major releases from 2.3 on (not point releases) if they are not already in the Ubuntu repositories. It only supports currently supported Ubuntu versions. There is no guarantee of updates, but it does make getting different versions easy.
To install the necessary repositories:
$ sudo add-apt-repository ppa:fkrull/deadsnakes
$ sudo apt-get update
If you want to install 2.7, it's as easy as:
$ sudo apt-get install python2.7
Note this only works for Ubuntu, not e.g. Debian.
$ sudo add-apt-repository ppa:deadsnakes/ppa
$ sudo apt-get update
To install Python2.7, just enter:
$ sudo apt-get install python2.7

apt-get installing older version of packages (Ubuntu)

I'm trying to install pip and virtualenv on a server (running Ubuntu 12.04.4 LTS) on which I have access, but I can only do it with sudo apt-get install (school politics). The problem is that althought I have run the sudo apt-get update command to update the packages list, I think it keeps installing old ones. After doing sudo apt-get install python-pip python-virtualenv, I do pip --version on which I get the 1.0, and virtualenv --version on which I get 1.7.1.2. These two version are quite old (pip is already in 1.5.5 and virtualenv in 1.11.5). I read that the problem is that the packages list is not up-to-date, but the command sudo apt-get update should solve this, but I guess no. How can I solve this? Thanks a lot!
apt-get update updates packages from Ubuntu package catalog, which has nothing to do with mainstream versions.
LTS in Ubuntu stands for Long Term Support. Which means that after a certain period in time they will only release security-related bugfixes to the packages. In general, major version of packages will not change inside of a major Ubuntu release, to make sure backwards-compatibility is kept.
So if then only thing you can do is apt-get update, you have 2 options:
find a PPA that provides fresher versions of packages that you need, add it and repeat the update/install exercise
find those packages elsewhere, download them in .deb format and install.
If you really need to use the latest stable versions of Python packages, then do not use apt-get for installing Python packages and use pip instead. If you would use apt-get and later install the same packages by means of pip or (better not) easy_install or setup.py, you are likely to run into version conflicts wondering, why your python based commands are of unexpected versions, or even worse, why they do not work at all.
I try to follow this pattern:
1. system wide pip installation first
Using instructions from here: http://pip.readthedocs.org/en/latest/installing.html find get-pip.py script, download it and run as python script.
$ wget https://bootstrap.pypa.io/get-pip.py
$ sudo python get-pip.py
$ rm get-pip.py
2. use pip to install virtualenv system wide
this shall be as easy as:
$ sudo pip install virtualenv
3. (optional) install virtualenvwrapper - system wide or to user profile
$ sudo pip install virtualenvwrapper
and follow instructions for configuring it.
4. Since now, install inside your virtualenv environments
This shall prevent conflicts between various versions of packages.
You are free to update particular virtualenvs as you need one by one independently.
5. (optional) Configure installation cache directories for installation speed
There are method how to speed up repeated installation of packages, what comes handy if you get used using virtualenv often. For details see my answer: https://stackoverflow.com/a/18520729/346478

Instructing Pip to Use Python 3

I'm using Ubuntu, how do I instruct pip to use the Python3 installation and not Python2.6? 2.6 is the default installation on Ubuntu. I can't upgrade that as it will break Ubuntu.
Any single pip installation is (roughly) specific to one Python installation. You can, however, have multiple parallel pip installations. Your package manager probably has a package called pip-3.3 or similar. If not, you can manually install it (run the get-pip.py script using Python 3.3), though you'll have to be careful that it ends up in the right place in PATH. You can also use a virtualenv.
Install pip by downloading the source and doing:
python3 setup.py install
Then, you should have a pip3 or a pip-3.3 command that you can use, to instruct pip to install whatever package using python3
Conversely, if you want to migrate to python3 entirely, do which python and follow the symlinks to /usr/local/bin (I think). There, you should be able to sudo ln -s to change what python points to

How to use pip with Python 3.x alongside Python 2.x

I installed Python 3.x (besides Python 2.x on Ubuntu) and slowly started to pair modules I use in Python 2.x.
So I wonder, what approach should I take to make my life easy by using pip for both Python 2.x and Python 3.x?
The approach you should take is to install pip for Python 3.2.
You do this in the following way:
$ curl -O https://bootstrap.pypa.io/get-pip.py
$ sudo python3.2 get-pip.py
Then, you can install things for Python 3.2 with pip-3.2, and install things for Python 2-7 with pip-2.7. The pip command will end up pointing to one of these, but I'm not sure which, so you will have to check.
What you can also do is to use apt-get:
apt-get install python3-pip
In my experience this works pretty fluent too, plus you get all the benefits from apt-get.
First, install Python 3 pip using:
sudo apt-get install python3-pip
Then, to use Python 3 pip use:
pip3 install <module-name>
For Python 2 pip use:
pip install <module-name>
The shortest way:
python3 -m pip install package
python -m pip install package
If you don't want to have to specify the version every time you use pip:
Install pip:
$ curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python3
and export the path:
$ export PATH=/Library/Frameworks/Python.framework/Versions/<version number>/bin:$PATH
In Windows, first installed Python 3.7 and then Python 2.7. Then, use command prompt:
pip install python2-module-name
pip3 install python3-module-name
That's all
This worked for me on OS X: (I say this because sometimes is a pain that mac has "its own" version of every open source tool, and you cannot remove it because "its improvements" make it unique for other apple stuff to work, and if you remove it things start falling appart)
I followed the steps provided by #Lennart Regebro to get pip for python 3, nevertheless pip for python 2 was still first on the path, so... what I did is to create a symbolic link to python 3 inside /usr/bin (in deed I did the same to have my 2 pythons running in peace):
ln -s /Library/Frameworks/Python.framework/Versions/3.4/bin/pip /usr/bin/pip3
Notice that I added a 3 at the end, so basically what you have to do is to use pip3 instead of just pip.
The post is old but I hope this helps someone someday. this should theoretically work for any LINUX system.
On Suse Linux 13.2, pip calls python3, but pip2 is available to use the older python version.
To use pip for a Python 2.x environment, use this command:
py -2 -m pip install -r requirements.txt
To use pip for Python 3.x environment, use this command:
py -3 -m pip install -r requirements.txt
Please note that on msys2 I've found these commands to be helpful:
$ pacman -S python3-pip
$ pip3 install --upgrade pip
$ pip3 install --user package_name
In the general case, you will need to find out the precise location of your Python 2 and Python 3 interpreters. On Windows, the whereis command will reveal where a program is installed; on Unix and related platforms, try type or command -v (don't use the popular but nonstandard tools whereis or which unless you know you can trust them).
Windows example:
C:\loser> whereis python
C:\Python37\Python.exe
Linux example:
bash$ type python
python is /usr/local/bin/python
For instance, you might have Python 3 in /usr/local/bin/python3 and Python 2 in /usr/bin/python.
(This is by no means universal; where and how you installed the different Python versions can vary enormously. You could have Python in /opt/anaconda/forest/trees/shirley for all I know. Probably also look in /opt and in your home directory, or just use the search facilities of your OS to see what's where.)
Generally, if you find a Python binary, /path/to/this/python --version will print its version number.
Once you know this, using the full path to the version you want to use will always get you precisely that version, with its specific configuration for where to install packages.
So, in this example,
/usr/local/bin/python3 -m pip install requests
will install requests for Python 3, and
/usr/bin/python -m pip install requests
for Python 2. But again, these are just examples; you need to know the correct paths for your specific system.
On Windows (and now, going forward, other platforms) there is a wrapper called py which lets you conveniently specify which version of Python you want to run with an option. py -2 runs Python 2 (provided you have installed it, of course), and you can probably guess how to get Python 3.
By the way; on Debian and derived Linux distros, including Ubuntu, Mint, etc, pip is not necessarily installed when you apt install python3 (or python etc); you will need to separately install python3-pip (or python-pip etc).
For convenience, you might want to explore using virtual environments to control your Python environment. With Python 3, you want /path/to/your/python -m venv envname to create envname which, when activated, will always run that particular Python version when you simply run python, and the correct corresponding pip when you type that command. (Again, on Debian-based distros, you will separately need to install python3-venv.) With Python 2, you need to separately install the third-party module virtualenv. (There are other tools with similar facilities but slightly different features; but these are de facto standard, pretty much.)
If there are only a couple of versions which you want to keep on using in parallel, you can create a wrapper or alias so you don't have to type the full path every time. For example, if $HOME/bin is on your PATH, you could create a file $HOME/bin/py3 with the contents
#!/bin/sh
exec /path/to/python3 "$#"
and chmod +x this file to be able to type just py3 going forward. (The syntax for a similar wrapper on Windows will be different; in cmd you want %* instead of "$#" and no exec or #!/bin/sh shebang, and probably a # in front. There is no chmod on Windows, but the file needs to have the .cmd extension.)
Of course, if the versions you want to use already have different names, you just need to make sure their directories are on your PATH, and make sure no other programs with the same name are in a directory which is earlier on your PATH.
If you want to be able to use multiple Python versions in parallel (for example, a specific version for each project you are working on) there are third-party tools like pyenv which let you maintain a number of parallel Python installations and choose between them easily. But this is just a convenience; if you know what you are doing, you can always run a specific Python version with the instructions in this answer.

ImportError: No module named bz2 for Python 2.7.2

I'm using Python 2.7.2 on Ubuntu 11.10. I got this error when importing the bz2 module:
ImportError: No module named bz2
I thought the bz2 module is supposed to come with Python 2.7. How can I fix this problem?
EDIT: I think I previously installed Python 2.7.2 by compiling from source. Probably at that point I didn't have libbz2-dev and so the bz2 module is not installed. Now, I'm hoping to install Python2.7 through
sudo apt-get install python2.7
But it will say it's already installed. Is there a way to uninstall the previous Python2.7 installation and reinstall?
I meet the same problem, here's my solution.
The reason of import error is while you are building python, system couldn't find the bz2 headers and skipped building bz2 module.
Install them on Ubuntu/Debian:
sudo apt-get install libbz2-dev
Fedora:
sudo yum install bzip2-devel
and then rebuild python
comes from another answer
#birryree's answer helps to back to the system's original python.
Okay, this is much easier to understand in answer form, so I'll move what I would write in my comment to this answer.
Luckily for you, you didn't overwrite the system version of python, as Ubuntu 11.10 comes with 2.7.2 preinstalled.
Your python binaries (python and python2.7) are located in /usr/local/bin, which is a directory where user-specific stuff is usually installed. This is fine, it means your system python is still there.
First, just try to run the system python. Type this from the command line:
/usr/bin/python -c "import bz2; print bz2.__doc__"
This should print out something like this:
λ > /usr/bin/python -c "import bz2; print bz2.__doc__"
The python bz2 module provides a comprehensive interface for
the bz2 compression library. It implements a complete file
interface, one shot (de)compression functions, and types for
sequential (de)compression.
If so, means you're fine.
So you just have to fix your PATH, which tells the shell where to find commands. /usr/local/bin is going to have priority over /usr/local, so there are some ways to fix this, in order of difficulty/annoyance/altering your system:
Remove the symlink python from /usr/local/bin
This will make it so that when you type python, it should go back to executing /usr/bin/python, which is an alias for the system's python 2.7.2.
sudo rm /usr/local/bin/python
Move /usr/bin to have higher precedence in the PATH
Might not be desirable if you already have stuff in /usr/local/bin that should have precedence over /usr/bin, but I'm adding this for completeness.
In your shell profile (not sure what Ubuntu's default is, but I'm using ~/.bash_profile, you can do this:
export PATH=/usr/bin:$PATH
Remove your python install
This is extreme and the first option I presented should be your first option.
Do you really need your own version of Python? If you want isolated python environments you probably really want virtualenv. You can probably remove yours unless there's a reason not to.
It's going to be a little annoying though, but basically:
Remove the python and python2.7 and pythonw and pythonw2.7 commands from /usr/local/bin.
Remove /usr/local/lib/python/2.7.2
This part is not complete because I forget what else there is.
In case, you must be used python2.7, you should run: (Centos 6.4)
sudo cp /usr/lib64/python2.6/lib-dynload/bz2.so /usr/local/lib/python2.7/
Maybe it will helps someone:
apt-get install libbz2-dev # for bz2
apt-get install libssl-dev # for _ssl
apt-get install libsqlite3-dev # for sqlite
apt-get install libreadline6-dev # for readline, _curses, _curses_panel
For Ubuntu/Debian:
sudo apt-get install libbz2-dev
For Fedora:
sudo yum install bzip2-devel
And then recompile the python and install it.
matocnhoi's answer works for me in centOS
sudo cp /usr/lib64/python2.6/lib-dynload/bz2.so /usr/local/lib/python2.7/
and I used virtualenv, so the command is
sudo cp /usr/lib64/python2.6/lib-dynload/bz2.so ../../../env/lib/python2.7/
I used a symlink between /usr/lib64/python2.6/lib-dynload/bz2.so /usr/local/lib/python2.7/lib-dynload/
Worked fine for me...
Make sure you bz2 installed, run sudo yum install bzip2-devel.
Centos 6
sudo cp /usr/lib64/python2.6/lib-dynload/bz2.so /python_install_path/lib/python2.7
Centos 7
sudo cp /usr/lib64/python2.7/lib-dynload/bz2.so /python_install_path/lib/python2.7
python_install_path usually is /usr/local/lib/python2.7/, you need replace that if you install python in a another path.
If your bz2 in /usr/lib64/python2.7/lib-dynload/ is named as: "bz2.x86_64-linux-gnu.so", remember to rename it to bz2.so when copying it to your path or it may not be correctly sourced:
cp /usr/lib64/python2.6/lib-dynload/bz2.x86_64-linux-gnu.so /python_install_path/lib/python2.7/bz2.so
I had the same problem with Python 2.17.15 and pyenv on Ubuntu. System python from /usr/bin/python worked fine. In my case it helped to install libbz2-dev and then to reinstall python 2.7.15:
sudo apt-get install libbz2-dev
pyenv uninstall 2.7.15
pyenv install 2.7.15

Categories