Related
With my virtual environment activated, I see with conda list that my pandas version is 0.24.0. When I do the same with pip list, I see the version is 0.22.0 (probably an older version that I installed before using conda). When I import pandas in python (3.6), the pandas version is 0.22.0.
Why and how to force the loading of the conda package?
EDIT: MacOS High Sierra 10.13.1
TL;DR is in Possible Fix at the bottom
A few notes, and these may or may not answer the question, but I think this is a bit better than dumping everything into comments. These assume that your environment is activated, for these examples, my environment is called new36. I am also on MacOS with High Sierra 10.13.6.
Checking conda vs pip locations
First, let's check to make sure conda and pip are both looking in the same environment. To find information surrounding conda, check:
conda info
I get the following:
active environment : new36
active env location : /Users/mm92400/anaconda3/envs/new36
shell level : 1
user config file : /Users/mm92400/.condarc
populated config files : /Users/mm92400/.condarc
conda version : 4.6.8
conda-build version : 3.0.27
python version : 3.6.3.final.0
# extra info excluded
The active env location is what we're concerned with. This should be a directory that contains the directory of pip:
which pip | head -n 1
/Users/mm92400/anaconda3/envs/new36/bin/pip
If pip does not sit in a directory under where conda lives, this could be part of the issue.
Verifying the import path of python
You should be able to check where python is sourcing files from via sys.path:
import sys
sys.path
['', '/Users/mm92400/anaconda3/envs/new36/lib/python36.zip', '/Users/mm92400/anaconda3/envs/new36/lib/python3.6', '/Users/mm92400/anaconda3/envs/new36/lib/python3.6/lib-dynload', '/Users/mm92400/anaconda3/envs/new36/lib/python3.6/site-packages']
This is a list, and that's important to note. Note how my sys.path does not have any directories that source from a file/folder based on a base install of conda, nor any of the Framework installs of python on my Mac. import will search these directories ('' is cwd) in order, pulling down the first instance of a package that it finds. If your sys.path has an element earlier than your conda env that contains pandas, this is your problem.
Verbose python
You can also verify where the pandas package is being sourced from using the verbose mode of python, python -v:
# you have gotten here by running python -v in the terminal
# there's a whole bunch of comments that pop out that I'm going to omit here
# Now run
import pandas
~snip~
# code object from '/Users/mm92400/anaconda3/envs/new36/lib/python3.6/site-packages/pandas/__pycache__/_version.cpython-36.pyc'
import 'pandas._version' # <_frozen_importlib_external.SourceFileLoader object at 0x107952b00>
import 'pandas' # <_frozen_importlib_external.SourceFileLoader object at 0x104572b38>
Note how the code object path matches where I expect python to source that package from
Possible Fix
You can hack on sys.path, though I'm not sure how recommended that is. You can prioritize where directories are in sys.path without modifying sys.path in your script like:
env PYTHONPATH=$(find $CONDA_PREFIX -type d -name "site-packages" | head -n 1) python
which will take you into an interpreter and sys.path will look like:
import sys
sys.path
['', '/Users/mm92400/anaconda3/envs/new36/lib/python3.6/site-packages', ...]
Where now the first directory it will check is the conda env site-packages. Because sys.path is a list, it will be traversed in order. The way to prioritize which one you want to use is by injecting that particular directory into the sys.path first. If I were to write a script like:
import sys
print(f"I prioritized {sys.path[1]}")
And ran it using env PYTHONPATH=$(find $CONDA_PREFIX -type d -name "site-packages" | head -n 1) python somefile.py I would get:
env PYTHONPATH=$(find $CONDA_PREFIX -type d -name "site-packages" | head -n 1) python somefile.py
I prioritized /Users/mm92400/anaconda3/envs/new36/lib/python3.6/site-packages
Alternatively, you can insert into sys.path, but I can say definitively that this is not recommended and quite fragile:
import os, sys
try:
conda_env = os.environ['CONDA_PREFIX']
except KeyError:
raise KeyError("The env var $CONDA_PREFIX was not found. Please check that your conda environment was activated")
for root, dirs, files in os.walk(conda_env):
if 'site-packages' in dirs:
syspath_add = os.path.join(root, 'site-packages')
break
else:
raise FileNotFoundError("Couldn't find site-packages!")
sys.path.insert(0, syspath_add)
sys.path
# ['/Users/mm92400/anaconda3/envs/new36/lib/python3.6/site-packages', '', ...]
I want to activate a virtualenv instance from a Python script.
I know it's quite easy to do, but all the examples I've seen use it to run commands within the env and then close the subprocess.
I simply want to activate the virtualenv and return to the shell, the same way that bin/activate does.
Something like this:
$me: my-script.py -d env-name
$(env-name)me:
Is this possible?
Relevant:
virtualenv › Invoking an env from a script
If you want to run a Python subprocess under the virtualenv, you can do that by running the script using the Python interpreter that lives inside virtualenv's /bin/ directory:
import subprocess
# Path to a Python interpreter that runs any Python script
# under the virtualenv /path/to/virtualenv/
python_bin = "/path/to/virtualenv/bin/python"
# Path to the script that must run under the virtualenv
script_file = "must/run/under/virtualenv/script.py"
subprocess.Popen([python_bin, script_file])
However, if you want to activate the virtualenv under the current Python interpreter instead of a subprocess, you can use the activate_this.py script:
# Doing execfile() on this file will alter the current interpreter's
# environment so you can import libraries in the virtualenv
activate_this_file = "/path/to/virtualenv/bin/activate_this.py"
execfile(activate_this_file, dict(__file__=activate_this_file))
The simplest solution to run your script under virtualenv's interpreter is to replace the default shebang line with path to your virtualenv's interpreter like so at the beginning of the script:
#!/path/to/project/venv/bin/python
Make the script executable:
chmod u+x script.py
Run the script:
./script.py
Voila!
It turns out that, yes, the problem is not simple, but the solution is.
First I had to create a shell script to wrap the "source" command. That said I used the "." instead, because I've read that it's better to use it than source for Bash scripts.
#!/bin/bash
. /path/to/env/bin/activate
Then from my Python script I can simply do this:
import os
os.system('/bin/bash --rcfile /path/to/myscript.sh')
The whole trick lies within the --rcfile argument.
When the Python interpreter exits it leaves the current shell in the activated environment.
Win!
To run another Python environment according to the official Virtualenv documentation, in the command line you can specify the full path to the executable Python binary, just that (no need to active the virtualenv before):
/path/to/virtualenv/bin/python
The same applies if you want to invoke a script from the command line with your virtualenv. You don't need to activate it before:
me$ /path/to/virtualenv/bin/python myscript.py
The same for a Windows environment (whether it is from the command line or from a script):
> \path\to\env\Scripts\python.exe myscript.py
Just a simple solution that works for me. I don't know why you need the Bash script which basically does a useless step (am I wrong ?)
import os
os.system('/bin/bash --rcfile flask/bin/activate')
Which basically does what you need:
[hellsing#silence Foundation]$ python2.7 pythonvenv.py
(flask)[hellsing#silence Foundation]$
Then instead of deactivating the virtual environment, just Ctrl + D or exit. Is that a possible solution or isn't that what you wanted?
The top answer only works for Python 2.x
For Python 3.x, use this:
activate_this_file = "/path/to/virtualenv/bin/activate_this.py"
exec(compile(open(activate_this_file, "rb").read(), activate_this_file, 'exec'), dict(__file__=activate_this_file))
Reference: What is an alternative to execfile in Python 3?
The child process environment is lost in the moment it ceases to exist, and moving the environment content from there to the parent is somewhat tricky.
You probably need to spawn a shell script (you can generate one dynamically to /tmp) which will output the virtualenv environment variables to a file, which you then read in the parent Python process and put in os.environ.
Or you simply parse the activate script in using for the line in open("bin/activate"), manually extract stuff, and put in os.environ. It is tricky, but not impossible.
For python2/3, Using below code snippet we can activate virtual env.
activate_this = "/home/<--path-->/<--virtual env name -->/bin/activate_this.py" #for ubuntu
activate_this = "D:\<-- path -->\<--virtual env name -->\Scripts\\activate_this.py" #for windows
with open(activate_this) as f:
code = compile(f.read(), activate_this, 'exec')
exec(code, dict(__file__=activate_this))
I had the same issue and there was no activate_this.py in the Scripts directory of my environment.
activate_this.py
"""By using execfile(this_file, dict(__file__=this_file)) you will
activate this virtualenv environment.
This can be used when you must use an existing Python interpreter, not
the virtualenv bin/python
"""
try:
__file__
except NameError:
raise AssertionError(
"You must run this like execfile('path/to/active_this.py', dict(__file__='path/to/activate_this.py'))")
import sys
import os
base = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
if(sys.platform=='win32'):
site_packages = os.path.join(base, 'Lib', 'site-packages')
else:
site_packages = os.path.join(base, 'lib', 'python%s' % sys.version[:3], 'site-packages')
prev_sys_path = list(sys.path)
import site
site.addsitedir(site_packages)
sys.real_prefix = sys.prefix
sys.prefix = base
# Move the added items to the front of the path:
new_sys_path = []
for item in list(sys.path):
if item not in prev_sys_path:
new_sys_path.append(item)
sys.path.remove(item)
sys.path[:0] = new_sys_path
Copy the file to the Scripts directory of your environment and use it like this:
def activate_virtual_environment(environment_root):
"""Configures the virtual environment starting at ``environment_root``."""
activate_script = os.path.join(
environment_root, 'Scripts', 'activate_this.py')
execfile(activate_script, {'__file__': activate_script})
activate_virtual_environment('path/to/your/venv')
Refrence: https://github.com/dcreager/virtualenv/blob/master/virtualenv_support/activate_this.py
You should create all your virtualenvs in one folder, such as virt.
Assuming your virtualenv folder name is virt, if not change it
cd
mkdir custom
Copy the below lines...
#!/usr/bin/env bash
ENV_PATH="$HOME/virt/$1/bin/activate"
bash --rcfile $ENV_PATH -i
Create a shell script file and paste the above lines...
touch custom/vhelper
nano custom/vhelper
Grant executable permission to your file:
sudo chmod +x custom/vhelper
Now export that custom folder path so that you can find it on the command-line by clicking tab...
export PATH=$PATH:"$HOME/custom"
Now you can use it from anywhere by just typing the below command...
vhelper YOUR_VIRTUAL_ENV_FOLDER_NAME
Suppose it is abc then...
vhelper abc
In Python, I'm getting an error because it's loading a module from /usr/lib/python2.6/site-packages but I'd like it to use my version in $HOME/python-modules/lib/python2.6/site-packages, which I installed using pip-python --install-option="--prefix=$HOME/python-modules --ignore-installed
How can I tell Python to use my version of the library? Setting PYTHONPATH to $HOME/python-modules/lib/python2.6/site-packages doesn't help, since /usr/lib/... apparently has precedence.
Take a look at the site module for ways to customize your environment.
One way to accomplish this is to add a file to a location currently on sys.path called usercustomize.py, when Python is starting up it will automatically import this file, and you can use it to modify sys.path.
First, set $PYTHONPATH to $HOME (or add $HOME if $PYTHONPATH has a value), then create the file $HOME/usercustomize.py with the following contents:
import sys, os
my_site = os.path.join(os.environ['HOME'],
'python-modules/lib/python2.6/site-packages')
sys.path.insert(0, my_site)
Now when you start Python you should see your custom site-packages directory before the system default on sys.path.
Newer Python versions now have built-in support to search the opendesktop location:
$HOME/.local/lib/pythonX.Y/site-packages
If you put your local modules there you don't have to any sys.path manipulations.
If one has multiple versions of a package installed, say e.g. SciPy:
>>> import scipy; print(scipy.__version__); print(scipy.__file__)
0.17.0
/usr/lib/python3/dist-packages/scipy/__init__.py
and one would like the user installed version (installed e.g. using pip install --user --upgrade scipy) to be prefered, one needs a usercustomize.py file in ~/.local/lib/python3.5/site-packages/ with e.g. this content:
import sys, os
my_site = os.path.join(
os.environ['HOME'], '.local/lib/python%d.%d/site-packages' % (
sys.version_info[0], sys.version_info[1]))
for idx, pth in enumerate(sys.path):
if pth.startswith('/usr'):
sys.path.insert(idx, my_site)
break
else:
raise ValueError("No path starting with /usr in sys.path")
(the for loop selecting index ensures that packages installed in "develop mode" takes precedence) now we get our user specific version of SciPy:
>>> import scipy; print(scipy.__version__); print(scipy.__file__)
0.18.1
/home/user/.local/lib/python3.5/site-packages/scipy/__init__.py
to prefer packages installed to userbase (e.g. pip install --user --upgrade cool_thing )
in ~/.bashrc,~/.profile, or whatever the init file for your shell is, add
export PYTHONUSERBASE="$HOME/python-modules"
in $PYTHONUSERBASE/usercustomize.py
#!/usr/bin/env python
import sys, site
sys.path.insert(0, site.getusersitepackages())
For example, I have a Python script using the Google App Engine SDK:
from google.appengine.ext import db
from google.appengine.ext import webapp
from google.appengine.ext.webapp import template
from google.appengine.ext.webapp.util import run_wsgi_app
The module db has a submodule Key, so I try to use autocomplete on it:
db.KTab
But at the bottom of the Vim window, I get the following:
-- Omni completion (^O^N^P) Pattern not found
How do I include the path to non-standard Python libraries so that Vim autocompletion can find them? And also display their docstrings?
You need to add your library files to your tags file. For instance, if you have installed the Google App Engine via pip in a virtual environment located in env/:
virtualenv --no-site-package env/
source env/bin/activate
pip install google_appengine
... then you should execute:
ctags -R --python-kinds=-i -o tags env/
If you did not install google_appengine through pip, then you should locate the path to your python libraries (hint: it should be indicated by $PYTHONPATH. And according to this reference page: "on Unix, this is usually .:/usr/local/lib/python.") and replace env/ by the path you found.
Finally, your .vimrc file should parse your tags file. For instance, in my .vimrc, I have:
set tags+=/path/to/my/tags
I grabbed this from natw's vimrc (I think...maybe sontek), but it should do the trick, so long as your packages are findable by your current install of Python. This lets you use gf, but also sets up searching these files for autocompletion. Note the py <<EOF part, which starts a section interpreted in Python. This means you'd have to have the python interpreter installed in vim to use it.
function! LoadPythonPath()
py <<EOF
# load PYTHONPATH into vim, this lets you hover over a module name
# and type 'gf' (for goto file) and open that file in vim. Useful
# and easier than rope for simple tasks
import os.path
import sys
import vim
for p in sys.path:
if os.path.isdir(p):
vim.command(r"set path+=%s" % (p.replace(" ", r"\ ")))
EOF
endfunction
Btw, I don't like to have this load automatically, so I set it to a function that intelligently loads/unloads when I call it/first enter a Python doc. And I add a let g:PythonPathLoaded=1 to the previous function.
function! GetPythonPath()
if !exists("g:PythonPathLoaded")
call LoadPythonPath()
return
elseif g:PythonPathLoaded
return
else
call LoadPythonPath()
endif
endfunction
And I have an unload function too...though I'm not sure whether this makes a huge difference.
function! UnloadPythonPath()
py <<EOF
# load PYTHONPATH into vim, this lets you hover over a module name
# and type 'gf' (for goto file) and open that file in vim. Useful
# and easier than rope for simple tasks
for p in sys.path:
if os.path.isdir(p):
vim.command(r"set path-=%s" % (p.replace(" ", r"\ ")))
EOF
let g:PythonPathLoaded = 0
endfunction
Hope this helps! Plus, an added bonus is that this will load your packages regardless of whether you are using virtualenv (since it, I believe, runs whatever is set as 'python' at the moment).
How do I find the location of my site-packages directory?
There are two types of site-packages directories, global and per user.
Global site-packages ("dist-packages") directories are listed in sys.path when you run:
python -m site
For a more concise list run getsitepackages from the site module in Python code:
python -c 'import site; print(site.getsitepackages())'
Caution: In virtual environments getsitepackages is not available with older versions of virtualenv, sys.path from above will list the virtualenv's site-packages directory correctly, though. In Python 3, you may use the sysconfig module instead:
python3 -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])'
The per user site-packages directory (PEP 370) is where Python installs your local packages:
python -m site --user-site
If this points to a non-existing directory check the exit status of Python and see python -m site --help for explanations.
Hint: Running pip list --user or pip freeze --user gives you a list of all installed per user site-packages.
Practical Tips
<package>.__path__ lets you identify the location(s) of a specific package: (details)
$ python -c "import setuptools as _; print(_.__path__)"
['/usr/lib/python2.7/dist-packages/setuptools']
<module>.__file__ lets you identify the location of a specific module: (difference)
$ python3 -c "import os as _; print(_.__file__)"
/usr/lib/python3.6/os.py
Run pip show <package> to show Debian-style package information:
$ pip show pytest
Name: pytest
Version: 3.8.2
Summary: pytest: simple powerful testing with Python
Home-page: https://docs.pytest.org/en/latest/
Author: Holger Krekel, Bruno Oliveira, Ronny Pfannschmidt, Floris Bruynooghe, Brianna Laugher, Florian Bruhin and others
Author-email: None
License: MIT license
Location: /home/peter/.local/lib/python3.4/site-packages
Requires: more-itertools, atomicwrites, setuptools, attrs, pathlib2, six, py, pluggy
>>> import site; site.getsitepackages()
['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']
(or just first item with site.getsitepackages()[0])
A solution that:
outside of virtualenv - provides the path of global site-packages,
insidue a virtualenv - provides the virtualenv's site-packages
...is this one-liner:
python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"
Formatted for readability (rather than use as a one-liner), that looks like the following:
from distutils.sysconfig import get_python_lib
print(get_python_lib())
Source: an very old version of "How to Install Django" documentation (though this is useful to more than just Django installation)
For Ubuntu,
python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"
...is not correct.
It will point you to /usr/lib/pythonX.X/dist-packages
This folder only contains packages your operating system has automatically installed for programs to run.
On ubuntu, the site-packages folder that contains packages installed via setup_tools\easy_install\pip will be in /usr/local/lib/pythonX.X/dist-packages
The second folder is probably the more useful one if the use case is related to installation or reading source code.
If you do not use Ubuntu, you are probably safe copy-pasting the first code box into the terminal.
This is what worked for me:
python -m site --user-site
A modern stdlib way is using sysconfig module, available in version 2.7 and 3.2+. Unlike the current accepted answer, this method still works regardless of whether or not you have a virtual environment active.
Note: sysconfig (source) is not to be confused with the distutils.sysconfig submodule (source) mentioned in several other answers here. The latter is an entirely different module and it's lacking the get_paths function discussed below. Additionally, distutils is deprecated in 3.10 and will be unavailable soon.
Python currently uses eight paths (docs):
stdlib: directory containing the standard Python library files that are not platform-specific.
platstdlib: directory containing the standard Python library files that are platform-specific.
platlib: directory for site-specific, platform-specific files.
purelib: directory for site-specific, non-platform-specific files.
include: directory for non-platform-specific header files.
platinclude: directory for platform-specific header files.
scripts: directory for script files.
data: directory for data files.
In most cases, users finding this question would be interested in the 'purelib' path (in some cases, you might be interested in 'platlib' too). The purelib path is where ordinary Python packages will be installed by tools like pip.
At system level, you'll see something like this:
# Linux
$ python3 -c "import sysconfig; print(sysconfig.get_path('purelib'))"
/usr/local/lib/python3.8/site-packages
# macOS (brew installed python3.8)
$ python3 -c "import sysconfig; print(sysconfig.get_path('purelib'))"
/usr/local/Cellar/python#3.8/3.8.3/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages
# Windows
C:\> py -c "import sysconfig; print(sysconfig.get_path('purelib'))"
C:\Users\wim\AppData\Local\Programs\Python\Python38\Lib\site-packages
With a venv, you'll get something like this
# Linux
/tmp/.venv/lib/python3.8/site-packages
# macOS
/private/tmp/.venv/lib/python3.8/site-packages
# Windows
C:\Users\wim\AppData\Local\Temp\.venv\Lib\site-packages
The function sysconfig.get_paths() returns a dict of all of the relevant installation paths, example on Linux:
>>> import sysconfig
>>> sysconfig.get_paths()
{'stdlib': '/usr/local/lib/python3.8',
'platstdlib': '/usr/local/lib/python3.8',
'purelib': '/usr/local/lib/python3.8/site-packages',
'platlib': '/usr/local/lib/python3.8/site-packages',
'include': '/usr/local/include/python3.8',
'platinclude': '/usr/local/include/python3.8',
'scripts': '/usr/local/bin',
'data': '/usr/local'}
A shell script is also available to display these details, which you can invoke by executing sysconfig as a module:
python -m sysconfig
Addendum: What about Debian / Ubuntu?
As some commenters point out, the sysconfig results for Debian systems (and Ubuntu, as a derivative) are not accurate. When a user pip installs a package it will go into dist-packages not site-packages, as per Debian policies on Python packaging.
The root cause of the discrepancy is because Debian patch the distutils install layout, to correctly reflect their changes to the site, but they fail to patch the sysconfig module.
For example, on Ubuntu 20.04.4 LTS (Focal Fossa):
root#cb5e85f17c7f:/# python3 -m sysconfig | grep packages
platlib = "/usr/lib/python3.8/site-packages"
purelib = "/usr/lib/python3.8/site-packages"
root#cb5e85f17c7f:/# python3 -m site | grep packages
'/usr/local/lib/python3.8/dist-packages',
'/usr/lib/python3/dist-packages',
USER_SITE: '/root/.local/lib/python3.8/site-packages' (doesn't exist)
It looks like the patched Python installation that Debian/Ubuntu are distributing is a bit hacked up, and they will need to figure out a new plan for 3.12+ when distutils is completely unavailable. Probably, they will have to start patching sysconfig as well, since this is what pip will be using for install locations.
Let's say you have installed the package 'django'. import it and type in dir(django). It will show you, all the functions and attributes with that module. Type in the python interpreter -
>>> import django
>>> dir(django)
['VERSION', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', 'get_version']
>>> print django.__path__
['/Library/Python/2.6/site-packages/django']
You can do the same thing if you have installed mercurial.
This is for Snow Leopard. But I think it should work in general as well.
As others have noted, distutils.sysconfig has the relevant settings:
import distutils.sysconfig
print distutils.sysconfig.get_python_lib()
...though the default site.py does something a bit more crude, paraphrased below:
import sys, os
print os.sep.join([sys.prefix, 'lib', 'python' + sys.version[:3], 'site-packages'])
(it also adds ${sys.prefix}/lib/site-python and adds both paths for sys.exec_prefix as well, should that constant be different).
That said, what's the context? You shouldn't be messing with your site-packages directly; setuptools/distutils will work for installation, and your program may be running in a virtualenv where your pythonpath is completely user-local, so it shouldn't assume use of the system site-packages directly either.
The native system packages installed with python installation in Debian based systems can be found at :
/usr/lib/python2.7/dist-packages/
In OSX - /Library/Python/2.7/site-packages
by using this small code :
from distutils.sysconfig import get_python_lib
print get_python_lib()
However, the list of packages installed via pip can be found at :
/usr/local/bin/
Or one can simply write the following command to list all paths where python packages are.
>>> import site; site.getsitepackages()
['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']
Note: the location might vary based on your OS, like in OSX
>>> import site; site.getsitepackages()
['/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/site-python', '/Library/Python/2.7/site-packages']
pip show will give all the details about a package:
https://pip.pypa.io/en/stable/reference/pip_show/ [pip show][1]
To get the location:
pip show <package_name>| grep Location
In Linux, you can go to site-packages folder by:
cd $(python -c "import site; print(site.getsitepackages()[0])")
All the answers (or: the same answer repeated over and over) are inadequate. What you want to do is this:
from setuptools.command.easy_install import easy_install
class easy_install_default(easy_install):
""" class easy_install had problems with the fist parameter not being
an instance of Distribution, even though it was. This is due to
some import-related mess.
"""
def __init__(self):
from distutils.dist import Distribution
dist = Distribution()
self.distribution = dist
self.initialize_options()
self._dry_run = None
self.verbose = dist.verbose
self.force = None
self.help = 0
self.finalized = 0
e = easy_install_default()
import distutils.errors
try:
e.finalize_options()
except distutils.errors.DistutilsError:
pass
print e.install_dir
The final line shows you the installation dir. Works on Ubuntu, whereas the above ones don't. Don't ask me about windows or other dists, but since it's the exact same dir that easy_install uses by default, it's probably correct everywhere where easy_install works (so, everywhere, even macs). Have fun. Note: original code has many swearwords in it.
An additional note to the get_python_lib function mentioned already: on some platforms different directories are used for platform specific modules (eg: modules that require compilation). If you pass plat_specific=True to the function you get the site packages for platform specific packages.
This works for me.
It will get you both dist-packages and site-packages folders.
If the folder is not on Python's path, it won't be
doing you much good anyway.
import sys;
print [f for f in sys.path if f.endswith('packages')]
Output (Ubuntu installation):
['/home/username/.local/lib/python2.7/site-packages',
'/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages']
This should work on all distributions in and out of virtual environment due to it's "low-tech" nature. The os module always resides in the parent directory of 'site-packages'
import os; print(os.path.dirname(os.__file__) + '/site-packages')
To change dir to the site-packages dir I use the following alias (on *nix systems):
alias cdsp='cd $(python -c "import os; print(os.path.dirname(os.__file__))"); cd site-packages'
A side-note: The proposed solution (distutils.sysconfig.get_python_lib()) does not work when there is more than one site-packages directory (as recommended by this article). It will only return the main site-packages directory.
Alas, I have no better solution either. Python doesn't seem to keep track of site-packages directories, just the packages within them.
from distutils.sysconfig import get_python_lib
print get_python_lib()
You should try this command to determine pip's install location
Python 2
pip show six | grep "Location:" | cut -d " " -f2
Python 3
pip3 show six | grep "Location:" | cut -d " " -f2
Answer to old question. But use ipython for this.
pip install ipython
ipython
import imaplib
imaplib?
This will give the following output about imaplib package -
Type: module
String form: <module 'imaplib' from '/usr/lib/python2.7/imaplib.py'>
File: /usr/lib/python2.7/imaplib.py
Docstring:
IMAP4 client.
Based on RFC 2060.
Public class: IMAP4
Public variable: Debug
Public functions: Internaldate2tuple
Int2AP
ParseFlags
Time2Internaldate
For those who are using poetry, you can find your virtual environment path with poetry debug:
$ poetry debug
Poetry
Version: 1.1.4
Python: 3.8.2
Virtualenv
Python: 3.8.2
Implementation: CPython
Path: /Users/cglacet/.pyenv/versions/3.8.2/envs/my-virtualenv
Valid: True
System
Platform: darwin
OS: posix
Python: /Users/cglacet/.pyenv/versions/3.8.2
Using this information you can list site packages:
ls /Users/cglacet/.pyenv/versions/3.8.2/envs/my-virtualenv/lib/python3.8/site-packages/
I made a really simple function that gets the job done
import site
def get_site_packages_dir():
return [p for p in site.getsitepackages()
if p.endswith(("site-packages", "dist-packages"))][0]
get_site_packages_dir()
# '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages'
If you want to retrieve the results using the terminal:
python3 -c "import site;print([p for p in site.getsitepackages() if p.endswith(('site-packages', 'dist-packages')) ][0])"
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages
I had to do something slightly different for a project I was working on: find the relative site-packages directory relative to the base install prefix. If the site-packages folder was in /usr/lib/python2.7/site-packages, I wanted the /lib/python2.7/site-packages part. I have, in fact, encountered systems where site-packages was in /usr/lib64, and the accepted answer did NOT work on those systems.
Similar to cheater's answer, my solution peeks deep into the guts of Distutils, to find the path that actually gets passed around inside setup.py. It was such a pain to figure out that I don't want anyone to ever have to figure this out again.
import sys
import os
from distutils.command.install import INSTALL_SCHEMES
if os.name == 'nt':
scheme_key = 'nt'
else:
scheme_key = 'unix_prefix'
print(INSTALL_SCHEMES[scheme_key]['purelib'].replace('$py_version_short', (str.split(sys.version))[0][0:3]).replace('$base', ''))
That should print something like /Lib/site-packages or /lib/python3.6/site-packages.
Something that has not been mentioned which I believe is useful, if you have two versions of Python installed e.g. both 3.8 and 3.5 there might be two folders called site-packages on your machine. In that case you can specify the python version by using the following:
py -3.5 -c "import site; print(site.getsitepackages()[1])