We Keep Coding

How do I check all pip packages for packages installed globally using sudo? [duplicate]

Is there a way in Python to list all installed packages and their versions?
I know I can go inside python/Lib/site-packages and see what files and directories exist, but I find this very awkward. What I'm looking for something that is similar to npm list i.e. npm-ls.

If you have pip install and you want to see what packages have been installed with your installer tools you can simply call this:
pip freeze
It will also include version numbers for the installed packages.
Update
pip has been updated to also produce the same output as pip freeze by calling:
pip list
Note
The output from pip list is formatted differently, so if you have some shell script that parses the output (maybe to grab the version number) of freeze and want to change your script to call list, you'll need to change your parsing code.

help('modules') should do it for you.
in IPython :
In [1]: import #import press-TAB
Display all 631 possibilities? (y or n)
ANSI audiodev markupbase
AptUrl audioop markupsafe
ArgImagePlugin avahi marshal
BaseHTTPServer axi math
Bastion base64 md5
BdfFontFile bdb mhlib
BmpImagePlugin binascii mimetools
BufrStubImagePlugin binhex mimetypes
CDDB bisect mimify
CDROM bonobo mmap
CGIHTTPServer brlapi mmkeys
Canvas bsddb modulefinder
CommandNotFound butterfly multifile
ConfigParser bz2 multiprocessing
ContainerIO cPickle musicbrainz2
Cookie cProfile mutagen
Crypto cStringIO mutex
CurImagePlugin cairo mx
DLFCN calendar netrc
DcxImagePlugin cdrom new
Dialog cgi nis
DiscID cgitb nntplib
DistUpgrade checkbox ntpath

If you want to get information about your installed python distributions and don't want to use your cmd console or terminal for it, but rather through python code, you can use the following code (tested with python 3.4):
import pip #needed to use the pip functions
for i in pip.get_installed_distributions(local_only=True):
print(i)
The pip.get_installed_distributions(local_only=True) function-call returns an iterable and because of the for-loop and the print function the elements contained in the iterable are printed out separated by new line characters (\n).
The result will (depending on your installed distributions) look something like this:
cycler 0.9.0
decorator 4.0.4
ipykernel 4.1.0
ipython 4.0.0
ipython-genutils 0.1.0
ipywidgets 4.0.3
Jinja2 2.8
jsonschema 2.5.1
jupyter 1.0.0
jupyter-client 4.1.1
#... and so on...

To run this in later versions of pip (tested on pip==10.0.1) use the following:
from pip._internal.operations.freeze import freeze
for requirement in freeze(local_only=True):
print(requirement)

My take:
#!/usr/bin/env python3
import pkg_resources
dists = [str(d).replace(" ","==") for d in pkg_resources.working_set]
for i in dists:
print(i)

from command line
python -c help('modules')
can be used to view all modules, and for specific modules
python -c help('os')
For Linux below will work
python -c "help('os')"

For easy_install (deprecated, Python <= v2.7, do not use this, use pip instead; use this only in old projects that still use easy_install)
You can try : Yolk
For install yolk, try:
easy_install yolk
Yolk is a Python tool for obtaining information about installed Python
packages and querying packages avilable on PyPI (Python Package
Index).
You can see which packages are active, non-active or in development
mode and show you which have newer versions available by querying
PyPI.

yes! you should be using pip as your python package manager ( http://pypi.python.org/pypi/pip )
with pip installed packages, you can do a
pip freeze
and it will list all installed packages. You should probably also be using virtualenv and virtualenvwrapper. When you start a new project, you can do
mkvirtualenv my_new_project
and then (inside that virtualenv), do
pip install all_your_stuff
This way, you can workon my_new_project and then pip freeze to see which packages are installed for that virtualenv/project.
for example:
➜ ~ mkvirtualenv yo_dude
New python executable in yo_dude/bin/python
Installing setuptools............done.
Installing pip...............done.
virtualenvwrapper.user_scripts creating /Users/aaylward/dev/virtualenvs/yo_dude/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/aaylward/dev/virtualenvs/yo_dude/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/aaylward/dev/virtualenvs/yo_dude/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/aaylward/dev/virtualenvs/yo_dude/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/aaylward/dev/virtualenvs/yo_dude/bin/get_env_details
(yo_dude)➜ ~ pip install django
Downloading/unpacking django
Downloading Django-1.4.1.tar.gz (7.7Mb): 7.7Mb downloaded
Running setup.py egg_info for package django
Installing collected packages: django
Running setup.py install for django
changing mode of build/scripts-2.7/django-admin.py from 644 to 755
changing mode of /Users/aaylward/dev/virtualenvs/yo_dude/bin/django-admin.py to 755
Successfully installed django
Cleaning up...
(yo_dude)➜ ~ pip freeze
Django==1.4.1
wsgiref==0.1.2
(yo_dude)➜ ~
or if you have a python package with a requirements.pip file,
mkvirtualenv my_awesome_project
pip install -r requirements.pip
pip freeze
will do the trick

If you're using anaconda:
conda list
will do it! See: https://conda.io/docs/_downloads/conda-cheatsheet.pdf

Here's a way to do it using PYTHONPATH instead of the absolute path of your python libs dir:
for d in `echo "${PYTHONPATH}" | tr ':' '\n'`; do ls "${d}"; done
[ 10:43 Jonathan#MacBookPro-2 ~/xCode/Projects/Python for iOS/trunk/Python for iOS/Python for iOS ]$ for d in `echo "$PYTHONPATH" | tr ':' '\n'`; do ls "${d}"; done
libpython2.7.dylib pkgconfig python2.7
BaseHTTPServer.py _pyio.pyc cgitb.pyo doctest.pyo htmlentitydefs.pyc mimetools.pyc plat-mac runpy.py stringold.pyc traceback.pyo
BaseHTTPServer.pyc _pyio.pyo chunk.py dumbdbm.py htmlentitydefs.pyo mimetools.pyo platform.py runpy.pyc stringold.pyo tty.py
BaseHTTPServer.pyo _strptime.py chunk.pyc dumbdbm.pyc htmllib.py mimetypes.py platform.pyc runpy.pyo stringprep.py tty.pyc
Bastion.py _strptime.pyc chunk.pyo dumbdbm.pyo htmllib.pyc mimetypes.pyc platform.pyo sched.py stringprep.pyc tty.pyo
Bastion.pyc _strptime.pyo cmd.py
....

for using code, for example to check what modules in Hackerrank etc :
import os
os.system("pip list")

If this is needed to run from within python you can just invoke subprocess
from subprocess import PIPE, Popen
pip_process = Popen(["pip freeze"], stdout=PIPE,
stderr=PIPE, shell=True)
stdout, stderr = pip_process.communicate()
print(stdout.decode("utf-8"))

For Windows 10, I think this is what you are looking for a list of available installed Pythons. This is different from a list of packages as you can see below. Also, on Ubuntu 20.04, I think the command is Python3 -0 list.
Yes, this works similar to node version manager.
c:\Users\user\AppData\Local\Programs\Python>py -0 list
Python 0 not found!
Installed Pythons found by py Launcher for Windows
-3.10-64 *
-3.9-64
-3.7-64
-3.6-64
-2.7-64
Requested Python version (0) not installed, use -0 for available pythons
c:\Users\user\AppData\Local\Programs\Python>py -0p
Installed Pythons found by py Launcher for Windows
-3.10-64 C:\Python310\python.exe *
-3.9-64 C:\Users\user\AppData\Local\Programs\Python\Python39\python.exe
-3.7-64 C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\python.exe
-3.6-64 C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\python.exe
-2.7-64 C:\Python27amd64\python.exe
See: https://www.infoworld.com/article/3617292/how-to-use-pythons-py-launcher-for-windows.html
See Also: https://www.freecodecamp.org/news/manage-multiple-python-versions-and-virtual-environments-venv-pyenv-pyvenv-a29fb00c296f/
From the above link, "If you wish to use multiple versions of Python on a single machine, then pyenv is a commonly used tool to install and switch between versions. This is not to be confused with the previously mentioned depreciated pyvenv script. It does not come bundled with Python and must be installed separately." -- Note: This acts similar to Node Version Manager with versions of Node.js and NPM.
See Also: https://github.com/pyenv-win/pyenv-win#installation
Action: Open PowerShell and type the following web request. The link above offers other approaches as well, but this appears to be the easiest approach. The name of the runtime output file is not a name variant like 'pyenv-win' but actually 'pyenv', as originally expected.
PS C:\Users\user> Invoke-WebRequest -UseBasicParsing -Uri "https://raw.githubusercontent.com/pyenv-win/pyenv-win/master/pyenv-win/install-pyenv-win.ps1" -OutFile "./install-pyenv-win.ps1"; &"./install-pyenv-win.ps1"
pyenv-win 2.64.11 installed.
No updates available.
PS C:\Users\user>
Example Output for working with 'pyenv', Python's Version Manager.
C:\Users\user>pyenv --version
pyenv 2.64.11
C:\Users\name>pyenv
pyenv 2.64.11
Usage: pyenv <command> [<args>]
Some useful pyenv commands are:
commands List all available pyenv commands
duplicate Creates a duplicate python environment
local Set or show the local application-specific Python version
global Set or show the global Python version
shell Set or show the shell-specific Python version
install Install a Python version using python-build
uninstall Uninstall a specific Python version
update Update the cached version DB
rehash Rehash pyenv shims (run this after installing executables)
vname Show the current Python version
version Show the current Python version and its origin
version-name Show the current Python version
versions List all Python versions available to pyenv
exec Runs an executable by first preparing PATH so that the selected Python
which Display the full path to an executable
whence List all Python versions that contain the given executable
See `pyenv help <command>' for information on a specific command.
For full documentation, see: https://github.com/pyenv-win/pyenv-win#readme
C:\Users\name>pyenv commands
--version
commands
duplicate
exec
export
global
help
install
local
rehash
shell
shims
uninstall
update
version-name
version
versions
vname
whence
which
C:\Users\name>pyenv version
No global python version has been set yet. Please set the global version by typing:
pyenv global 3.7.2
C:\Users\user>pyenv local
no local version configured for this directory
C:\Users\user>pyenv global
no global version configured
C:\Users\user>pyenv local 3.9-64
pyenv specific python requisite didn't meet. Project is using different version of python.
Install python '3.9-64' by typing: 'pyenv install 3.9-64'
My Note: Version name from 'https://www.python.org/downloads/' is different to those provided by 'pyenv'. This version was already installed locally, but it is outside the control of this Python version manager, so it is not visible to the manager.
C:\Users\user>pyenv install 3.8.10-64
:: [Info] :: Mirror: https://www.python.org/ftp/python
pyenv-install: definition not found: local
My Note(s): This Python version is not part of the managed list although this version exists at 'https://www.python.org/downloads/'. So you must see the list provided by the manager. See all available versions with `pyenv install --list'.
C:\Users\user>pyenv install --list
Note: Review the list from this call and make your selection.
C:\Users\user>pyenv install 3.8.10
:: [Info] :: Mirror: https://www.python.org/ftp/python
:: [Downloading] :: 3.8.10 ...
:: [Downloading] :: From https://www.python.org/ftp/python/3.8.10/python-3.8.10-amd64-webinstall.exe
:: [Downloading] :: To C:\Users\user\.pyenv\pyenv-win\install_cache\python-3.8.10-amd64-webinstall.exe
:: [Installing] :: 3.8.10 ...
:: [Info] :: completed! 3.8.10
My Note(s): With this Python version manager, 'pyenv', following installation, it appears that one must designate the version as 'local' or 'global' after the installation which would follow the same paradigm as the Node.js Version Manager (NVM). Again, from what I can see, the Python version manager can only see what versions of Python the manager installs; and it can only uninstall a version it has installed with the Python version manager.
C:\Users\user>pyenv local 3.8.10
C:\Users\user>pyenv local
3.8.10
C:\Users\user>pyenv version
3.8.10 (set by C:\Users\user\.python-version)
C:\Users\user>pyenv versions
* 3.8.10 (set by C:\Users\user\.python-version)
C:\Users\user>pyenv vname
3.8.10
C:\Users\user>pyenv global
no global version configured
The following below is for working with packages.
See Also: https://www.freecodecamp.org/news/manage-multiple-python-versions-and-virtual-environments-venv-pyenv-pyvenv-a29fb00c296f/
From the above link, "When the environment is active, any packages can be installed to it via pip as normal. By default, the newly created environment will not include any packages already installed on the machine. As pip itself will not necessarily be installed on the machine. It is recommended to first upgrade pip to the latest version, using 'pip install --upgrade pip'." -- I performed the pip upgrade just before making these two calls to list the packages and their versions below.
c:\Users\user\AppData\Local\Programs\Python>pip list
Package Version
---------- -------
pip 22.1
setuptools 62.2.0
wheel 0.37.1
c:\Users\user\AppData\Local\Programs\Python>pip list --local
Package Version
---------- -------
pip 22.1
setuptools 62.2.0
wheel 0.37.1

Tox not finding python3.6 even with the shim present. What is wrong with my pyenv setup?

Trying to breathe some life back into a django package that has fallen into a state of disrepair. They use tox for testing so I've setup pyenv on my MacBook. I've installed 3 versions of python as you can see below, and everything looks like it should work, but if it was I wouldn't be asking why it is not.
I've replaced my home directory with ~ to make it a bit easier to read.
pyenv was installed with brew install pyenv and the various versions of python were installed with pyenv install #.#.#
The shims exist:
$ echo $PATH
~/.pyenv/shims:~/.platformsh/bin:/usr/local/sbin:...
$ which python3.6
~/.pyenv/shims/python3.6
$ which python3.4
~/.pyenv/shims/python3.4
$ which python3.5
~/.pyenv/shims/python3.5
But executing them does not work as expected:
$ pyenv local 3.4.9 3.5.6 3.6.8
$ python3.4
Python 3.4.9 (default, Feb 12 2019, 10:33:47)
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
$ python3.5
pyenv: python3.5: command not found
The `python3.5' command exists in these Python versions:
3.5.6
$ python3.6
pyenv: python3.6: command not found
The `python3.6' command exists in these Python versions:
3.6.8
And tox fails like this:
py34-1.11: commands succeeded
ERROR: py36-1.11: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError("Failed to get version_info for python3.6: pyenv: python3.6: command not found\n\nThe `python3.6' command exists in these Python versions:\n 3.6.8\n\n", None)
ERROR: py36-2.0: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError("Failed to get version_info for python3.6: pyenv: python3.6: command not found\n\nThe `python3.6' command exists in these Python versions:\n 3.6.8\n\n", None)
py36-latest: commands succeeded
docs: commands succeeded
But in the .tox folder you will find these VirtualEnvs that can be activated manually.
$ ls .tox
dist docs flake8 log py34-1.11 py36-1.11 py36-2.0 py36-latest
Because at some point it was working....
I do understand the mechanics of why it isn't working, what I don't understand is why pyenv is not setting up the environment correctly (Or maybe this is exactly how it is supposed to behave). Everything I read seems to indicate that python3.6 should launch a python3.6.8 interpreter
$ bash -x python3.6
+ set -e
+ '[' -n '' ']'
+ program=python3.6
+ [[ python3.6 = \p\y\t\h\o\n* ]]
+ export PYENV_ROOT=~/.pyenv
+ PYENV_ROOT=~/.pyenv
+ exec /usr/local/Cellar/pyenv/1.2.9/libexec/pyenv exec python3.6
pyenv: python3.6: command not found
The `python3.6' command exists in these Python versions:
3.6.8

pyenv by default picks the python "locally", that is it looks for the PYTHON_VERSION environment variable or a .python-version file.
Personally I find this setup a little cumbersome (needing to have these files littered around all projects, especially in projects which need multiple versions). Fortunately, you can make these "shims" function anywhere with a default version of python by using pyenv global #.#.#
In your case, to make the python3.6 shim execute 3.6.8 without needing to set up the .python-version files, you'd run pyenv global 3.6.8 -- you can run this multiple times for different python versions as well: pyenv global 3.6.8 3.5.6 ...
The reason that you're likely not having these resolve inside tox is tox clears the environment when executing, so the PYTHON_VERSION environment variable will not carry through. You can turn that off by setting passenv= in your tox.ini. For example:
[testenv]
passenv = PYTHON_VERSION

Install ROS Kinetic on MacOS under a separate Anaconda environment

I'm installing ROS Kinetic on MacOS Sierra 10.12.6 under a seperate environment under Python Anaconda distribution.
Here is my python version(my seperate python environment in Anaconda is called "ros-kinetic"):
(ros-kinetic) jizhe#jizhedeMacBook-Pro  ~/ros_catkin_ws  python --version
Python 3.6.4 :: Anaconda, Inc.
I followed the guide: http://wiki.ros.org/kinetic/Installation/OSX/Homebrew/Source.
When I execute the "rosdep install" command, I got the following error:
(ros-kinetic) jizhe#jizhedeMacBook-Pro ~/ros_catkin_ws  rosdep install --from-paths src --ignore-src --rosdistro kinetic -y
ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
actionlib: No definition of [python-wxtools] for OS [osx]
catkin: No definition of [google-mock] for OS [osx]
roslisp: No definition of [google-mock] for OS [osx]
Obviously, there is something wrong with the package "python-wxtools" and "google-mock".
I've already install the wxpython via "brew install wxpython", and here is the wxpython info:
(ros-kinetic) jizhe#jizhedeMacBook-Pro  ~/ros_catkin_ws  brew info wxpython
wxpython: stable 3.0.2.0 (bottled)
Python bindings for wxWidgets
https://www.wxwidgets.org/
/usr/local/Cellar/wxpython/3.0.2.0_1 (1,108 files, 37.9MB) *
Poured from bottle on 2018-02-24 at 03:56:21
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/wxpython.rb
==> Dependencies
Required: wxmac ✔
==> Caveats
Python modules have been installed and Homebrew's site-packages is not
in your Python sys.path, so you will not be able to import the modules
this formula installed. If you plan to develop with these modules,
please run:
mkdir -p /Users/jizhe/Library/Python/2.7/lib/python/site-packages
echo 'import site; site.addsitedir("/usr/local/lib/python2.7/site-packages")' >> /Users/jizhe/Library/Python/2.7/lib/python/site-packages/homebrew.pth
And I've already run the command:
$ mkdir -p ~/Library/Python/2.7/lib/python/site-packages
$ echo "$(brew --prefix)/lib/python2.7/site-packages" >> ~/Library/Python/2.7/lib/python/site-packages/homebrew.pth
So, what's wrong with the package "google-mock" and "python-wxtools", how can I solve this problem?

Try installing with --skip-keys python-wxtools --skip-keys google-mock. It worked for me.

There is a ticket for this issue. Basically the workaround is to install googletest manually. Not sure what's wrong with python-wxtools.
edit: If you're using ros-install-osx, you can add the following lines to your rosdeps.yaml file:
google-mock:
osx:
homebrew:
packages: []

I found this question from a Google search, just for your information in 2021 installing ROS in conda also on macOS should be much straightforward, see the following links:
https://medium.com/robostack/cross-platform-conda-packages-for-ros-fa1974fd1de3
https://github.com/RoboStack/ros-noetic

Installation issues for Virtualenv and VirtualenvWrapper in Ubuntu 13.04

Ubuntu Server in VirtualBox. I am trying to install VirtualEnv to start learning Flask and bottle.
Some details of my setup.
vks#UbSrVb:~$ cat /etc/os-release
NAME="Ubuntu"
VERSION="12.04.2 LTS, Precise Pangolin"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu precise (12.04.2 LTS)"
VERSION_ID="12.04"
vks#UbSrVb:~$ python --version
Python 2.7.3
vks#UbSrVb:~$ echo $VIRTUALENVWRAPPER_PYTHON
/usr/bin/python
vks#UbSrVb:~$ echo $VIRTUALENV_PYTHON
vks#UbSrVb:~$
When I boot my Virtual Machine, I get the following error on my console
/usr/bin/python: No module named virtualenvwrapper
virtualenvwrapper.sh: There was a problem running the initialization hooks.
If Python could not import the module virtualenvwrapper.hook_loader,
check that virtualenv has been installed for
VIRTUALENVWRAPPER_PYTHON=/usr/bin/python and that PATH is
set properly.
When i try to initialize a virtualenv I get the following errors
vks#UbSrVb:~/dropbox/venv$ virtualenv try1
New python executable in try1/bin/python3.2
Also creating executable in try1/bin/python
Traceback (most recent call last):
File "/usr/local/bin/virtualenv", line 9, in <module>
load_entry_point('virtualenv==1.9.1', 'console_scripts', 'virtualenv')()
File "/usr/local/lib/python3.2/dist-packages/virtualenv.py", line 979, in main
no_pip=options.no_pip)
File "/usr/local/lib/python3.2/dist-packages/virtualenv.py", line 1081, in create_environment
site_packages=site_packages, clear=clear))
File "/usr/local/lib/python3.2/dist-packages/virtualenv.py", line 1499, in install_python
os.symlink(py_executable_base, full_pth)
OSError: [Errno 30] Read-only file system
vks#UbSrVb:~/dropbox/venv$ ls
try1
vks#UbSrVb:~/dropbox/venv$ ls try1/
bin include lib
vks#UbSrVb:~/dropbox/venv$
My .bashrc entries
export WORKON_HOME='~/dropbox/venv/'
source '/usr/local/bin/virtualenvwrapper.sh'
Q1 - As per the error at bootup, How do I ensure virtualenv is installed for VIRTUALENVWRAPPER_PYTHON=/usr/bin/python and that PATH is
set properly ?
Q2 - Even with sudo I get the same "Read-only file system" Error ?
I have tried installing virtualenv using pip and then apt-get, just to hit and try.

Try setting your WORKON_HOME global to another path (~/.virtualenvs) for example a see if that works, maybe the problem is with that shared directory, are you using windows? If you are, try installing ntfs-3g, see https://askubuntu.com/questions/70281/why-does-my-ntfs-partition-mount-as-read-only
Also in my profile configuration file I like to detect first if virtualenvwrapper is installed:
if which virtualenvwrapper.sh &> /dev/null; then
WORKON_HOME=$HOME/.virtualenvs
# path to virtualenvwrapper, in my case
source /usr/local/share/python/virtualenvwrapper.sh
fi

I had the problem where my pip was for a different version of python than the one I wanted to use.
$ python -V
Python 2.7.5+
$ pip -V
pip 1.5.4 from /usr/local/lib/python3.3/dist-packages (python 3.3)
So when I used pip to install virtualenv and virtualenvwrapper, the new python packages were put in python3.3's dist-packages, so of course my python2.7 couldn't find them!
To fix this, I had to use the appropriate version of pip, in my case it was pip2.
$ pip2 -V
pip 1.5.4 from /usr/local/lib/python2.7/dist-packages (python 2.7)
So make sure you are using the appropriate version of pip.

Migrating default python installation from 2.6 to 2.7: problems with easy_install

I'm using debian with a default python installation of 2.6
I wanted to migrate to python 2.7, including installing easy_install.
I followed someone else's instruction to delete my /usr/bin/python, then link
ln -s /usr/bin/python2.7 /usr/bin/python
I downloaded the most recent version of setuptools
and cd'ed to the file. The install help asked me to run it as a shell program, I did, with the following error:
sh setuptools-0.6c11-py2.7.egg
Traceback (most recent call last):
File "<string>", line 1 in <module>
ImportError: No Module named setuptools.command.easy_install
I have a feeling that my installation of easy_install is related to the version of python I'm running, but I can't quite figure it out. I tried also downloading the .tar.gz file, cd-ing into the directory, and running
python setup.py build; setup.py install
After I run that, I can use easy_install, with the following error:
Traceback (most recent call last):
File "/usr/local/bin/easy_install", line 5, in <module>
from pkg_resources import load_entry_point
ImportError: No Module named pkg_resources
Can anyone suggest to me a solution? Thanks for the help.

Replacing the system Python isn't a great idea. Moving up by one point release
probably won't break your system, but who wants to take the risk? Installing
libraries to your system Python using easy_install can also lead to problems
if they conflict with each other.
An alternative is to build Python from source, install it to your home
directory, and use virtualenv to create isolated environments into which you can
install whatever libraries you need for a given project using pip (which is
the more modern equivalent to easy_install).
For Python 2.7, if you want some of the 'optional' parts of the standard
library, that means building a couple of other things, too. Here's a script
(largely cobbled together from blog posts scattered across the interweb) that
works for me on Debian "Squeeze" (stable, at the time of writing):
#!/bin/bash -e
# Setup
sudo aptitude install build-essential
mkdir -p ${HOME}/.local
mkdir build-python
cd build-python
# Get sources
### Tcl/Tk <http://www.tcl.tk/software/tcltk/download.html>
wget "http://downloads.sourceforge.net/project/tcl/Tcl/8.5.13/tcl8.5.13-src.tar.gz"
wget "http://downloads.sourceforge.net/project/tcl/Tcl/8.5.13/tk8.5.13-src.tar.gz"
### Berkeley DB <http://www.oracle.com/technetwork/products/berkeleydb/downloads/index-082944.html>
wget "http://download.oracle.com/berkeley-db/db-4.8.30.tar.gz"
### Python <http://www.python.org/download/>
wget "http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz"
# Build Tcl
tar xzf tcl8.5.13-src.tar.gz
cd tcl8.5.13/unix
./configure --prefix=${HOME}/.local
make
make install
cd ../..
# Build Tk
tar xzf tk8.5.13-src.tar.gz
cd tk8.5.13/unix
./configure --prefix=${HOME}/.local
make
make install
cd ../..
# Build Berkeley DB 4.8
tar xzf db-4.8.30.tar.gz
cd db-4.8.30/build_unix
../dist/configure --prefix=${HOME}/.local/opt/BerkeleyDB.4.8 --enable-tcl --with-tcl=${HOME}/.local/lib
make
make install
cd ../..
# Set compile flags
export LDFLAGS="-L${HOME}/.local/lib -L${HOME}/.local/opt/BerkeleyDB.4.8/lib"
export CPPFLAGS="-I${HOME}/.local/include -I${HOME}/.local/opt/BerkeleyDB.4.8/include"
export CXXFLAGS=${CPPFLAGS}
export CFLAGS=${CPPFLAGS}
export LD_LIBRARY_PATH=${HOME}/.local/lib:${HOME}/.local/opt/BerkeleyDB.4.8/lib
export LD_RUN_PATH=${LD_LIBRARY_PATH}
# Build Python 2.7
tar xzf Python-2.7.3.tgz
cd Python-2.7.3
./configure --prefix=${HOME}/.local
make
make altinstall
cd ..
# Install virtualenv, pip and virtualenvwrapper
curl http://python-distribute.org/distribute_setup.py | ${HOME}/.local/bin/python2.7
curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | ${HOME}/.local/bin/python2.7
${HOME}/.local/bin/pip install virtualenvwrapper
# Update ~/.bashrc
echo 'export PATH="${HOME}/.local/bin:${PATH}"' >> ${HOME}/.bashrc
echo 'export WORKON_HOME="${HOME}/.local/virtualenv"' >> ${HOME}/.bashrc
echo 'export VIRTUALENVWRAPPER_PYTHON="${HOME}/.local/bin/python2.7"' >> ${HOME}/.bashrc
echo 'export VIRTUALENVWRAPPER_VIRTUALENV="${HOME}/.local/bin/virtualenv"' >> ${HOME}/.bashrc
echo 'export VIRTUALENVWRAPPER_VIRTUALENV_ARGS="--python=python2.7"' >> ${HOME}/.bashrc
echo 'source ${HOME}/.local/bin/virtualenvwrapper.sh' >> ${HOME}/.bashrc
# Finish ...
cd ..
echo -e "\n\n ... Done!"
This script will ask for your password in order to install GCC etc. if it needs to, then take a while to build everything - about 25 minutes on my (ancient)
machine and (terrible) internet connection. If you're paying attention, at some
point you'll see the following message scroll past:
Python build finished, but the necessary bits to build these modules were not found:
bsddb185 dl imageop
sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
Those four modules are archaic and/or deprecated, so you don't need to worry
about them. If the message mentions any other modules, that means some
necessary library isn't installed on your system. You can still run Python if that's the case, but won't be able to import those modules. Shout in the comments if you're affected by this, and
I'll update the script accordingly.
Once that's finished successfully, you need to source your .bashrc:
$ source ~/.bashrc
... and you'll then be able to run your newly compiled Python ...
$ python2.7
Python 2.7.3 (default, Nov 17 2012, 02:00:26)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
... create a virtualenv to work in ...
$ mkvirtualenv my_env
$ python
Python 2.7.3 (default, Nov 17 2012, 02:00:26)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
... install libraries into it ...
$ pip install beautifulsoup4
... and so on.
To exit a virtualenv:
$ deactivate
To re-enter it at a later date:
$ workon my_env
For more, check out the documentation for pip and virtualenvwrapper.