Apache can't find mysql.connector when running Python cgi - python

I have a cgi that ran fine on python 2.7.
When I converted it to python3, I had to install some modules again. I installed requests by simply copying it from a 2.7 site-packages directory to /usr/lib/python3.7/site-packages.
I used pip to install mysql.connector, during which the installer said that since the global site-packages directory was not writable, it was installing it in my user directory instead.
The cgi ran fine when I ran it from the command line on the host, but when it is called as a cgi, I get:
import mysql.connector
ModuleNotFoundError: No module named 'mysql.connector'
So, apache can't find mysql.connector, even though it finds requests and everything else. I tried copying all the mysql* stuff from my local site-packages like I did with requests, but no joy.
Here's where python3 is looking for things when I'm running it as me:
(myusername)lab:/var/www/cgi-bin$ python3 -m site
sys.path = [
'/var/www/cgi-bin',
'/usr/lib64/python37.zip',
'/usr/lib64/python3.7',
'/usr/lib64/python3.7/lib-dynload',
'/home/myusername/.local/lib/python3.7/site-packages',
'/usr/lib64/python3.7/site-packages',
'/usr/lib/python3.7/site-packages',
]
USER_BASE: '/home/myusername/.local' (exists)
USER_SITE: '/home/myusername/.local/lib/python3.7/site-packages' (exists)
ENABLE_USER_SITE: True
Here's where I think apache is looking for things:
(myusername)lab:/var/www/cgi-bin$ sudo -u apache python3 -m site
sys.path = [
'/var/www/cgi-bin',
'/usr/lib64/python37.zip',
'/usr/lib64/python3.7',
'/usr/lib64/python3.7/lib-dynload',
'/usr/lib64/python3.7/site-packages',
'/usr/lib/python3.7/site-packages',
]
USER_BASE: '/usr/share/httpd/.local' (doesn't exist)
USER_SITE: '/usr/share/httpd/.local/lib/python3.7/site-packages' (doesn't exist)
ENABLE_USER_SITE: True
Any guidance would be greatly appreciated.
Update:
I changed permissions for all the mysql* directories and their contents to be readable and executable by everybody, and that got rid of the error. But I don't know if that was the best solution. Seems like it would not be.

Related

"Django deprecated" error in Django-Admin

When I am using django-admin startproject api then it is opening file name django-admin.py and the code looks like this :
#!C:\Users\Administrator\Desktop\api\venv\Scripts\python.exe
## When the django-admin.py deprecation ends, remove this script.
import warnings
from django.core import management
try:
from django.utils.deprecation import RemovedInDjango40Warning
except ImportError:
raise ImportError(
'django-admin.py was deprecated in Django 3.1 and removed in Django '
'4.0. Please manually remove this script from your virtual environment '
'and use django-admin instead.'
)
if __name__ == "__main__":
warnings.warn(
'django-admin.py is deprecated in favor of django-admin.',
RemovedInDjango40Warning,
)
management.execute_from_command_line()
I am using virtual environment. I am not getting how to get rid of it.
Short answer:
Use django-admin.exe instead of django-admin because django-admin.py is deprecated.
Example:
django-admin.exe startproject NEW_PROJECT_NAME
django-admin.exe startapp NEW_APP_NAME
Long answer:
If you use the command "where django-admin" (Windows) / "whereis django-admin" (Linux), you will see that there are probably two shortcuts.
In my case:
C:\Program Files\Python39\Scripts\django-admin.exe
C:\Program Files\Python39\Scripts\django-admin.py
I suppose that using an .exe extension file is faster than using the .py python extension file.
There may be another solution, I haven't tried to delete the file with "py" extension (django-admin.py), I just found that the fastest solution was to use "django-admin.exe startproject PROJECT_NAME", and so on.
The most straightforward way is probably to uninstall and reinstall Django in your virtual environment, if you're okay using the latest version of 3.2:
pip uninstall Django
pip install 'Django<4'

How to prevent anaconda environment from reading libraries installed in local

Python tries to read a library installed under ~/.local, even though I am working on an anaconda environment.
> conda create -n testproj python=3.6
> conda activate testproj
> conda install pandas
> python
>>> import pandas as pd
Then I got an ImportError
ImportError: C extension: /home/myname/.local/lib/python3.6/site-packages/pandas/ ...
But if I change the permission of site-packages
> chmod 000 ~/.local/lib/python3.6/site-packages
Then I can import pandas without any error. Namely Python is looking at outside of the anaconda environment.
Question: How can I prevent Python from reading libraries outside the anaconda environment?
Environment: openSUSE Leap 15.0
EDIT: I found that sys.path contains site-packages under ~/.local. I do not think that the lines should be there.
['',
'/home/myname/anaconda3/envs/myproj/bin',
'/home/myname/anaconda3/envs/myproj/lib/python36.zip',
'/home/myname/anaconda3/envs/myproj/lib/python3.6',
'/home/myname/anaconda3/envs/myproj/lib/python3.6/lib-dynload',
'/home/myname/.local/lib/python3.6/site-packages',
'/home/myname/anaconda3/envs/myproj/lib/python3.6/site-packages',
'/home/myname/.local/lib/python3.6/site-packages/IPython/extensions',
'/home/myname/.ipython']
I get the same behavior on windows, clean environments include your user local packages. This is an open issue: https://github.com/conda/conda/issues/7173. conda doesn't support doing what you're asking directly (yet).
You can always just set the environment variable PYTHONNOUSERSITE (to any value), or invoke your interpreter with the -s switch, and you wont get your local packages (~/.local on windows is C:\Users\<username>\AppData\Roaming\Python\Python36\site-packages):
(test-env) C:\Users\matt>python -m site
sys.path = [
'C:\\Users\\matt',
'C:\\Anaconda440\\envs\\test-env\\python36.zip',
'C:\\Anaconda440\\envs\\test-env\\DLLs',
'C:\\Anaconda440\\envs\\test-env\\lib',
'C:\\Anaconda440\\envs\\test-env',
'C:\\Users\\matt\\AppData\\Roaming\\Python\\Python36\\site-packages',
'C:\\Users\\matt\\AppData\\Roaming\\Python\\Python36\\site-packages\\some_lib-1.0-py3.6.egg',
'C:\\Anaconda440\\envs\\test-env\\lib\\site-packages',
]
USER_BASE: 'C:\\Users\\matt\\AppData\\Roaming\\Python' (exists)
USER_SITE: 'C:\\Users\\matt\\AppData\\Roaming\\Python\\Python36\\site-packages' (exists)
ENABLE_USER_SITE: True
versus (note the -s switch, and now my local packages are no longer on my sys.path):
(test-env) C:\Users\matt>python -s -m site
sys.path = [
'C:\\Users\\matt',
'C:\\Anaconda440\\envs\\test-env\\python36.zip',
'C:\\Anaconda440\\envs\\test-env\\DLLs',
'C:\\Anaconda440\\envs\\test-env\\lib',
'C:\\Anaconda440\\envs\\test-env',
'C:\\Anaconda440\\envs\\test-env\\lib\\site-packages',
]
USER_BASE: 'C:\\Users\\matt\\AppData\\Roaming\\Python' (exists)
USER_SITE: 'C:\\Users\\matt\\AppData\\Roaming\\Python\\Python36\\site-packages' (exists)
ENABLE_USER_SITE: False
HTH.

How to resolve cannot import name pages (django/wagtail)

Recently I have been trying to get wagtail to work with my existing Django application. I was experiencing an error: -
ImportError: No module named wagtail
unable to load app 0 (mountpoint='') (callable not found or import error)
After much troubleshooting I managed to fix this, by copying the folder wagtail from:
/usr/local/lib/python2.7/dist-packages/
into here
/opt/django/src/
Having resolved this error, I received another about a different module, and another... each time I copied the folder from /usr/local/lib/python2.7/dist-packages/ into /opt/django/src/ and it eventually resolved the issues I was having and uWSGI started.
Now when I access the homepage of my app, I receive this error
ImportError at /
cannot import name pages
Request Method: GET
Request URL: http://example.com
Django Version: 1.9
Exception Type: ImportError
Exception Value:
cannot import name pages
Exception Location: ./wagtail/wagtailadmin/urls/__init__.py in <module>, line 4
Python Executable: /usr/local/bin/uwsgi
Python Version: 2.7.3
Python Path:
['.',
'',
'/opt/django/src',
'/root/.python',
'/opt/django/env/lib/python2.7',
'/opt/django/env/lib/python2.7/plat-linux2',
'/opt/django/env/lib/python2.7/lib-tk',
'/opt/django/env/lib/python2.7/lib-old',
'/opt/django/env/lib/python2.7/lib-dynload',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-linux2',
'/usr/lib/python2.7/lib-tk',
'/opt/django/env/local/lib/python2.7/site-packages',
'/opt/django/env/lib/python2.7/site-packages']
I have checked the file referenced (./wagtail/wagtailadmin/urls/__init__.py) and it looks like this:
from django.conf.urls import url, include
from django.views.decorators.cache import cache_control
from wagtail.wagtailadmin.urls import pages as wagtailadmin_pages_urls
from wagtail.wagtailadmin.urls import collections as wagtailadmin_collections_urls
from wagtail.wagtailadmin.urls import password_reset as wagtailadmin_password_reset_urls
from wagtail.wagtailadmin.views import account, chooser, home, pages, tags, userbar
from wagtail.wagtailadmin.api import urls as api_urls
from wagtail.wagtailcore import hooks
from wagtail.utils.urlpatterns import decorate_urlpatterns
from wagtail.wagtailadmin.decorators import require_admin_access
The offending line is the first wagtail 'from' ...
from wagtail.wagtailadmin.urls import pages as wagtailadmin_pages_urls
How can I resolve this error?
I've ran a few tests on wagtail setup. I made each test in a fresh Ubuntu 14 install (each test in a new workspace on Cloud9 IDE).
Test 1: Straight python 2.7
sudo pip install wagtail
ERRO: error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Comment: Lots of x86_64-linux-gnu-gcc: error: build/temp.linux-x86_64-2.7/libImaging/codec_fd.o: No such file or directory errors were output.
Didn't work.
Test 2 - Python 2.7 with virtualenv
sudo pip install virtualenv
virtualenv env
source env/bin/activate
Now from their github https://github.com/wagtail/wagtail
pip install wagtail
out: Successfully installed Django-1.10.7 Pillow-4.1.1 Unidecode-0.4.20 Willow-0.4 beautifulsoup4-4.5.3 django-modelcluster-3.1 django-taggit-0.22.1 django-treebeard-4.1.0 djangorestframework-3.6.2 html5lib-0.999999999 olefile-0.44 pytz-2017.2 requests-2.13.0 wagtail-1.9.1 webencodings-0.5.1
wagtail start mysite
cd mysite
python manage.py migrate
python manage.py createsuperuser
python manage.py runserver
I had to edit the settings/dev.py and include ALLOWED_HOSTS = ['*'] because I'm on cloud 9. Also needed the $IP:$PORT after runserver. WORKED!
Test3 - Python 3.4 with virtualenv
sudo pip3 install virtualenv
virtualenv -p python3 env
source env/bin/activate
pip install wagtail
Successfully installed Django-1.10.7 Pillow-4.1.1 Unidecode-0.4.20 Willow-0.4 beautifulsoup4-4.5.3 django-modelcluster-3.1 django-taggit-0.22.1 django-treebeard-4.1.0 djangorestframework-3.6.2 html5lib-0.999999999 olefile-0.44 pytz-2017.2 requests-2.13.0 wagtail-1.9.1 webencodings-0.5.1
Rest of the command are the same. And it WORKED!
Test 4 - Python 3.4 no virtualenv
sudo pip3 install wagtail
out: Successfully installed wagtail django-taggit requests Unidecode Django djangorestframework Pillow beautifulsoup4 Willow django-modelcluster django-treebeard olefile pytz
wagtail start mysite
python3 manage.py migrate
python3 manage.py createsuperuser
python3 manage.py runserver
EDIT: Made a mistake. Should've run with python3, not python in this case. Did it again and worked!
Conclusion: Looks that you need a virtualenv for this to work on Ubuntu 14 and python2. With python3 it works with and without virtualenv. I could not reproduce your error, but the idea is the same: it does not work because it looks for stuff in the wrong place.
Comments: There are two things you mention that I could not test:
I have been trying to get wagtail to work with my existing Django
application
With this big amount of heavy dependencies, (Pillow, Django rest, beatifullsoup and others I've never heard of) it would be strange if it worked out of the box with a previous setup. I mean, Django is a dependency for wagtail. So I would try the other way around. Start fresh with wagtail and port your code there (with python3 for the sake of the community ;) ). Also this manual copy of files is like programmers worst nightmare (at least to me). This looks like a version problem. Try to post the versions of everything involved, and the exact steps to reproduce the issue. I mean, did you pip install wagtail in the operating system that django was running?
I was having and uWSGI started.
So you have your app deployed already? This complicates things. If versions and virtualenv do not solve your problem, community needs setup details of deploy. Also you haven't mentioned your operational system.
I hope this helps! Good Luck!
I ran into the
cannot import name pages
error at:
from wagtail.wagtailadmin.urls import pages as wagtailadmin_pages_urls
while trying to deploy a wagtail app to Google App Engine and could not duplicate the problem on my local dev server.
A seemingly unrelated error that I saw only intermittently in GAE involved an import error in the PIL library. I was able to resolve BOTH errors by deleting PIL and pillow related folders from my vendored packages in my /lib directory. I had to import PIL through the app.yaml file instead due to the platform specific binary, but running pip install -t lib/ -r requirements-vendor.txt caused PIL to be installed into the lib folder as well (as a dependency of Wagtail)

ImportError when using web.py with new environment

I just recreated all my python environment, reinstalled python and setuptools, and installed virtualenv.
I started a test enviroment with virtualenv --no-site-packages test, activated it with Scripts\activate.bat and then easy_install web.py.
Then I create a code.py file:
import web
urls = (
'/.*', 'index',
)
app = web.application(urls, globals())
class index:
def GET(self):
return 'ok'
if __name__ == "__main__": app.run()
And I get the following error:
File "...\code.py", line 1, in <module>
import web
ImportError: No module named web
But if I use the interactive shell it works:
>>> import web
>>>
Everything done in the same cmd with the enviroment activated.
Does anyone know what is going on?
Edit:
It happens for every package installed within the environment. First it was web.py, now BeautifulSoup (same issue, cant find module, but import works in python shell)
Edit2:
The activate script is not setting the new python executable and the pythonpath print sys.executable gives C:\Python27\python.exe.
Solved.
Windows was configured to open .py files with C:\Python27\python.exe. I can even remember setting this mannualy some time ago so I wouldn't have to use python to run files (oh lazyness, what have you done to me?).
That's why it was working with the interactive shell, but not by executing the code.py file.
Running the file using python code.py works perfectly.

How do I find the location of my Python site-packages directory?

How do I find the location of my site-packages directory?
There are two types of site-packages directories, global and per user.
Global site-packages ("dist-packages") directories are listed in sys.path when you run:
python -m site
For a more concise list run getsitepackages from the site module in Python code:
python -c 'import site; print(site.getsitepackages())'
Caution: In virtual environments getsitepackages is not available with older versions of virtualenv, sys.path from above will list the virtualenv's site-packages directory correctly, though. In Python 3, you may use the sysconfig module instead:
python3 -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])'
The per user site-packages directory (PEP 370) is where Python installs your local packages:
python -m site --user-site
If this points to a non-existing directory check the exit status of Python and see python -m site --help for explanations.
Hint: Running pip list --user or pip freeze --user gives you a list of all installed per user site-packages.
Practical Tips
<package>.__path__ lets you identify the location(s) of a specific package: (details)
$ python -c "import setuptools as _; print(_.__path__)"
['/usr/lib/python2.7/dist-packages/setuptools']
<module>.__file__ lets you identify the location of a specific module: (difference)
$ python3 -c "import os as _; print(_.__file__)"
/usr/lib/python3.6/os.py
Run pip show <package> to show Debian-style package information:
$ pip show pytest
Name: pytest
Version: 3.8.2
Summary: pytest: simple powerful testing with Python
Home-page: https://docs.pytest.org/en/latest/
Author: Holger Krekel, Bruno Oliveira, Ronny Pfannschmidt, Floris Bruynooghe, Brianna Laugher, Florian Bruhin and others
Author-email: None
License: MIT license
Location: /home/peter/.local/lib/python3.4/site-packages
Requires: more-itertools, atomicwrites, setuptools, attrs, pathlib2, six, py, pluggy
>>> import site; site.getsitepackages()
['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']
(or just first item with site.getsitepackages()[0])
A solution that:
outside of virtualenv - provides the path of global site-packages,
insidue a virtualenv - provides the virtualenv's site-packages
...is this one-liner:
python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"
Formatted for readability (rather than use as a one-liner), that looks like the following:
from distutils.sysconfig import get_python_lib
print(get_python_lib())
Source: an very old version of "How to Install Django" documentation (though this is useful to more than just Django installation)
For Ubuntu,
python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"
...is not correct.
It will point you to /usr/lib/pythonX.X/dist-packages
This folder only contains packages your operating system has automatically installed for programs to run.
On ubuntu, the site-packages folder that contains packages installed via setup_tools\easy_install\pip will be in /usr/local/lib/pythonX.X/dist-packages
The second folder is probably the more useful one if the use case is related to installation or reading source code.
If you do not use Ubuntu, you are probably safe copy-pasting the first code box into the terminal.
This is what worked for me:
python -m site --user-site
A modern stdlib way is using sysconfig module, available in version 2.7 and 3.2+. Unlike the current accepted answer, this method still works regardless of whether or not you have a virtual environment active.
Note: sysconfig (source) is not to be confused with the distutils.sysconfig submodule (source) mentioned in several other answers here. The latter is an entirely different module and it's lacking the get_paths function discussed below. Additionally, distutils is deprecated in 3.10 and will be unavailable soon.
Python currently uses eight paths (docs):
stdlib: directory containing the standard Python library files that are not platform-specific.
platstdlib: directory containing the standard Python library files that are platform-specific.
platlib: directory for site-specific, platform-specific files.
purelib: directory for site-specific, non-platform-specific files.
include: directory for non-platform-specific header files.
platinclude: directory for platform-specific header files.
scripts: directory for script files.
data: directory for data files.
In most cases, users finding this question would be interested in the 'purelib' path (in some cases, you might be interested in 'platlib' too). The purelib path is where ordinary Python packages will be installed by tools like pip.
At system level, you'll see something like this:
# Linux
$ python3 -c "import sysconfig; print(sysconfig.get_path('purelib'))"
/usr/local/lib/python3.8/site-packages
# macOS (brew installed python3.8)
$ python3 -c "import sysconfig; print(sysconfig.get_path('purelib'))"
/usr/local/Cellar/python#3.8/3.8.3/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages
# Windows
C:\> py -c "import sysconfig; print(sysconfig.get_path('purelib'))"
C:\Users\wim\AppData\Local\Programs\Python\Python38\Lib\site-packages
With a venv, you'll get something like this
# Linux
/tmp/.venv/lib/python3.8/site-packages
# macOS
/private/tmp/.venv/lib/python3.8/site-packages
# Windows
C:\Users\wim\AppData\Local\Temp\.venv\Lib\site-packages
The function sysconfig.get_paths() returns a dict of all of the relevant installation paths, example on Linux:
>>> import sysconfig
>>> sysconfig.get_paths()
{'stdlib': '/usr/local/lib/python3.8',
'platstdlib': '/usr/local/lib/python3.8',
'purelib': '/usr/local/lib/python3.8/site-packages',
'platlib': '/usr/local/lib/python3.8/site-packages',
'include': '/usr/local/include/python3.8',
'platinclude': '/usr/local/include/python3.8',
'scripts': '/usr/local/bin',
'data': '/usr/local'}
A shell script is also available to display these details, which you can invoke by executing sysconfig as a module:
python -m sysconfig
Addendum: What about Debian / Ubuntu?
As some commenters point out, the sysconfig results for Debian systems (and Ubuntu, as a derivative) are not accurate. When a user pip installs a package it will go into dist-packages not site-packages, as per Debian policies on Python packaging.
The root cause of the discrepancy is because Debian patch the distutils install layout, to correctly reflect their changes to the site, but they fail to patch the sysconfig module.
For example, on Ubuntu 20.04.4 LTS (Focal Fossa):
root#cb5e85f17c7f:/# python3 -m sysconfig | grep packages
platlib = "/usr/lib/python3.8/site-packages"
purelib = "/usr/lib/python3.8/site-packages"
root#cb5e85f17c7f:/# python3 -m site | grep packages
'/usr/local/lib/python3.8/dist-packages',
'/usr/lib/python3/dist-packages',
USER_SITE: '/root/.local/lib/python3.8/site-packages' (doesn't exist)
It looks like the patched Python installation that Debian/Ubuntu are distributing is a bit hacked up, and they will need to figure out a new plan for 3.12+ when distutils is completely unavailable. Probably, they will have to start patching sysconfig as well, since this is what pip will be using for install locations.
Let's say you have installed the package 'django'. import it and type in dir(django). It will show you, all the functions and attributes with that module. Type in the python interpreter -
>>> import django
>>> dir(django)
['VERSION', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', 'get_version']
>>> print django.__path__
['/Library/Python/2.6/site-packages/django']
You can do the same thing if you have installed mercurial.
This is for Snow Leopard. But I think it should work in general as well.
As others have noted, distutils.sysconfig has the relevant settings:
import distutils.sysconfig
print distutils.sysconfig.get_python_lib()
...though the default site.py does something a bit more crude, paraphrased below:
import sys, os
print os.sep.join([sys.prefix, 'lib', 'python' + sys.version[:3], 'site-packages'])
(it also adds ${sys.prefix}/lib/site-python and adds both paths for sys.exec_prefix as well, should that constant be different).
That said, what's the context? You shouldn't be messing with your site-packages directly; setuptools/distutils will work for installation, and your program may be running in a virtualenv where your pythonpath is completely user-local, so it shouldn't assume use of the system site-packages directly either.
The native system packages installed with python installation in Debian based systems can be found at :
/usr/lib/python2.7/dist-packages/
In OSX - /Library/Python/2.7/site-packages
by using this small code :
from distutils.sysconfig import get_python_lib
print get_python_lib()
However, the list of packages installed via pip can be found at :
/usr/local/bin/
Or one can simply write the following command to list all paths where python packages are.
>>> import site; site.getsitepackages()
['/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages']
Note: the location might vary based on your OS, like in OSX
>>> import site; site.getsitepackages()
['/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/site-python', '/Library/Python/2.7/site-packages']
pip show will give all the details about a package:
https://pip.pypa.io/en/stable/reference/pip_show/ [pip show][1]
To get the location:
pip show <package_name>| grep Location
In Linux, you can go to site-packages folder by:
cd $(python -c "import site; print(site.getsitepackages()[0])")
All the answers (or: the same answer repeated over and over) are inadequate. What you want to do is this:
from setuptools.command.easy_install import easy_install
class easy_install_default(easy_install):
""" class easy_install had problems with the fist parameter not being
an instance of Distribution, even though it was. This is due to
some import-related mess.
"""
def __init__(self):
from distutils.dist import Distribution
dist = Distribution()
self.distribution = dist
self.initialize_options()
self._dry_run = None
self.verbose = dist.verbose
self.force = None
self.help = 0
self.finalized = 0
e = easy_install_default()
import distutils.errors
try:
e.finalize_options()
except distutils.errors.DistutilsError:
pass
print e.install_dir
The final line shows you the installation dir. Works on Ubuntu, whereas the above ones don't. Don't ask me about windows or other dists, but since it's the exact same dir that easy_install uses by default, it's probably correct everywhere where easy_install works (so, everywhere, even macs). Have fun. Note: original code has many swearwords in it.
An additional note to the get_python_lib function mentioned already: on some platforms different directories are used for platform specific modules (eg: modules that require compilation). If you pass plat_specific=True to the function you get the site packages for platform specific packages.
This works for me.
It will get you both dist-packages and site-packages folders.
If the folder is not on Python's path, it won't be
doing you much good anyway.
import sys;
print [f for f in sys.path if f.endswith('packages')]
Output (Ubuntu installation):
['/home/username/.local/lib/python2.7/site-packages',
'/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages']
This should work on all distributions in and out of virtual environment due to it's "low-tech" nature. The os module always resides in the parent directory of 'site-packages'
import os; print(os.path.dirname(os.__file__) + '/site-packages')
To change dir to the site-packages dir I use the following alias (on *nix systems):
alias cdsp='cd $(python -c "import os; print(os.path.dirname(os.__file__))"); cd site-packages'
A side-note: The proposed solution (distutils.sysconfig.get_python_lib()) does not work when there is more than one site-packages directory (as recommended by this article). It will only return the main site-packages directory.
Alas, I have no better solution either. Python doesn't seem to keep track of site-packages directories, just the packages within them.
from distutils.sysconfig import get_python_lib
print get_python_lib()
You should try this command to determine pip's install location
Python 2
pip show six | grep "Location:" | cut -d " " -f2
Python 3
pip3 show six | grep "Location:" | cut -d " " -f2
Answer to old question. But use ipython for this.
pip install ipython
ipython
import imaplib
imaplib?
This will give the following output about imaplib package -
Type: module
String form: <module 'imaplib' from '/usr/lib/python2.7/imaplib.py'>
File: /usr/lib/python2.7/imaplib.py
Docstring:
IMAP4 client.
Based on RFC 2060.
Public class: IMAP4
Public variable: Debug
Public functions: Internaldate2tuple
Int2AP
ParseFlags
Time2Internaldate
For those who are using poetry, you can find your virtual environment path with poetry debug:
$ poetry debug
Poetry
Version: 1.1.4
Python: 3.8.2
Virtualenv
Python: 3.8.2
Implementation: CPython
Path: /Users/cglacet/.pyenv/versions/3.8.2/envs/my-virtualenv
Valid: True
System
Platform: darwin
OS: posix
Python: /Users/cglacet/.pyenv/versions/3.8.2
Using this information you can list site packages:
ls /Users/cglacet/.pyenv/versions/3.8.2/envs/my-virtualenv/lib/python3.8/site-packages/
I made a really simple function that gets the job done
import site
def get_site_packages_dir():
return [p for p in site.getsitepackages()
if p.endswith(("site-packages", "dist-packages"))][0]
get_site_packages_dir()
# '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages'
If you want to retrieve the results using the terminal:
python3 -c "import site;print([p for p in site.getsitepackages() if p.endswith(('site-packages', 'dist-packages')) ][0])"
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages
I had to do something slightly different for a project I was working on: find the relative site-packages directory relative to the base install prefix. If the site-packages folder was in /usr/lib/python2.7/site-packages, I wanted the /lib/python2.7/site-packages part. I have, in fact, encountered systems where site-packages was in /usr/lib64, and the accepted answer did NOT work on those systems.
Similar to cheater's answer, my solution peeks deep into the guts of Distutils, to find the path that actually gets passed around inside setup.py. It was such a pain to figure out that I don't want anyone to ever have to figure this out again.
import sys
import os
from distutils.command.install import INSTALL_SCHEMES
if os.name == 'nt':
scheme_key = 'nt'
else:
scheme_key = 'unix_prefix'
print(INSTALL_SCHEMES[scheme_key]['purelib'].replace('$py_version_short', (str.split(sys.version))[0][0:3]).replace('$base', ''))
That should print something like /Lib/site-packages or /lib/python3.6/site-packages.
Something that has not been mentioned which I believe is useful, if you have two versions of Python installed e.g. both 3.8 and 3.5 there might be two folders called site-packages on your machine. In that case you can specify the python version by using the following:
py -3.5 -c "import site; print(site.getsitepackages()[1])

Categories