For various reasons, I need to install a python module somewhere outside the default location. pip install --prefix would seem like the right thing, but I'm having problems:
$ pip install --prefix /tmp/foo protobuf-3.2.0-cp27-cp27mu-manylinux1_x86_64.whl
$ export PYTHONPATH=/tmp/foo/lib64/python2.7/site-packages
$ python
Python 2.7.5 (default, Nov 6 2016, 00:28:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import google.protobuf
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named google.protobuf
>>> import sys
>>> sys.path.append('/tmp/foo/lib64/python2.7/site-packages')
>>> sys.path
['', '/tmp/foo/lib64/python2.7/site-packages', '/usr/lib64/python27.zip',
'/usr/lib64/python2.7', '/usr/lib64/python2.7/plat-linux2',
'/usr/lib64/python2.7/lib-tk', '/usr/lib64/python2.7/lib-old',
'/usr/lib64/python2.7/lib-dynload', '/usr/lib64/python2.7/site-packages',
'/usr/lib/python2.7/site-packages', '/tmp/foo/lib64/python2.7/site-packages']
>>> import google.protobuf
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named google.protobuf
But if I install it in the default location (which I cannot do in production):
$ sudo pip install protobuf-3.2.0-cp27-cp27mu-manylinux1_x86_64.whl
$ python
Python 2.7.5 (default, Nov 6 2016, 00:28:07)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import google.protobuf
>>> google.protobuf.__version__
'3.2.0'
I assume my invocation of the pip command is wrong, or there is something else I need to do to get at that module at runtime. Any advice would be much appreciated.
The best possible solution for you would be the use of "virtualenv". You can install required packages whose scope will only be limited to the virtual environment. Also it will ignore all the previously installed packages that are already in your main repository. For a quick start, follow this link: http://docs.python-guide.org/en/latest/dev/virtualenvs/
Use:
pip install --ignore-installed --install-option="--prefix=$PREFIX_PATH" package_name
--ignore-installed is to force all dependencies to be reinstalled using this new prefix. This looks to be the problem you're having. You can use --install-option multiple times to add any of the options you can use with python setup.py install with --prefix.
I am not sure why you want this path to be different. You should probably initialize a virtual environment to keep your modules project specific. However, if this is necessary for your use case, do the above.
Related
I am trying to get a module slackclient installed in a virtual Python environment.
python3 -m venv slackenv
. slackenv/activate
pip3 install slackclient
I get
python3.7
Python 3.7.6 (default, Jan 12 2022, 23:27:58)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import slackclient
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'slackclient'
This is with slackclient installed both in the virtualenv and in the main system pip list. (It shows up in both lists).
What could be the issue?
Despite the name of the package is slackclient, the actual module is slack, as can you see in the official page of the package. So, try this one:
import slack
it should work.
This used to work but I don't know what changed; nothing has changed in this repo for a while.
I have a local python package that I want to install via pip. My source tree looks like:
./local_packages/my-package-0.0.3.tar.gz
and my requirements.txt has only
my-package
I run
pip install --find-links="./local_packages" --target=./python_modules/ -r requirements.txt
This finds my package and installs it in python_modules but I can't import it:
Python 3.6.7 (default, Nov 30 2020, 14:15:49)
[GCC Apple LLVM 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from my_package import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'my_package'
When I look in python_modules I see all of the dependencies of my-package as well as my_package-0.0.3-py3.6.egg-info/ but there is no module for just my_package.
If I downgrade pip from 20.3 to 19.3.1, then I see what I was expecting, with both a folder for .egg-info as well as for the module itself. My import works correctly etc.
Is this expected behavior? Am I missing something?
I downloaded the source code of python 2.7.14 and built it and installed it on linux ( Red Hat 4.8.5-16 ). I have earlier installed python-magic and requests libraries. Now when I try to import modules installed using pip, I get this:
$ python2
Python 2.7.14 (default, Nov 9 2017, 09:05:45)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import magic
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named magic
>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named requests
while similar thing works perfectly fine in python 2.7.5 (default with the RHEL system)
$ python
Python 2.7.5 (default, May 3 2017, 07:55:04)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-14)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import magic
>>> import requests
>>>
Am I missing any configuration step here?
The modules installed via pip are only available to the standard python version. You need to install your desired packages for the non-standard python versions as well (see also Installing Python Modules)
python2 -m pip install python-magic
python2 -m pip install requests
your pip is default set for version
$ python
Python 2.7.5
SO if you want to install module for
$ python2
Python 2.7.14
use python2 -m pip install module_name
The sys.path must be different for the two installations. That's a list of folders that Python checks for imports.
It's probably simplest to just symlink the one for wherever pip installs to into the modules folder for 2.7.14.
I wrote a command line tool with cliff 2.3.0, tested on my laptop (Mac, Python 2.7.12). When I was tried to install it (python setup.py install) on a server (Linux, Python 2.7.2), I encountered this error:
Installed /private/tmp/easy_install-EGMO15/cliff-2.3.0/pbr-1.10.0-py2.7.egg
ERROR:root:Error parsing
Traceback (most recent call last): File "/private/tmp/easy_install-EGMO15/cliff-2.3.0/pbr-1.10.0-> py2.7.egg/pbr/core.py", line 111, in pbr
attrs = util.cfg_to_args(path, dist.script_args) File "/private/tmp/easy_install-EGMO15/cliff-2.3.0/pbr-1.10.0-py2.7.egg/pbr/util.py", line 248, in cfg_to_args
kwargs = setup_cfg_to_setup_kwargs(config, script_args) File "/private/tmp/easy_install-EGMO15/cliff-2.3.0/pbr-1.10.0-py2.7.egg/pbr/util.py", line 431, in setup_cfg_to_setup_kwargs
if pkg_resources.evaluate_marker('(%s)' % env_marker):
AttributeError: 'module' object has no attribute 'evaluate_marker' error: Setup script exited with error in setup command: Error parsing /private/tmp/easy_install-EGMO15/cliff-2.3.0/setup.cfg: AttributeError: 'module' object has no attribute 'evaluate_marker'
Any suggestions?
It looks like your server may have a (much) older version of the setuptools package installed (which provides the pkg_resources module). The evaluate_marker method looks as if it first showed up at the end of 2014, so if you're using an older system it is possible that method is not available.
Depending on your environment, you may be able to simply pip install -U setuptools, or you may need to see if your distribution has a newer isntallable package available.
If you can update your question to include details about your server's operating environment (what distribution and version are you running? What version of Python? What version of setuptools?), we can probably provide a more complete answer.
Update
For example, Ubuntu 12.04 only has setuptools 0.6, and the pkg_resources module (which is packaged in the python-pkg-resources package) does not have the evaluate_marker method:
# python
Python 2.7.3 (default, Jun 22 2015, 19:33:41)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
>>> pkg_resources.evaluate_marker
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'evaluate_marker'
In this environment, I can install pip:
# apt-get install python-pip
And then upgrade the installed version of setuptools:
# pip install -U setuptools
And now:
# python
Python 2.7.3 (default, Jun 22 2015, 19:33:41)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
>>> pkg_resources.evaluate_marker
<function evaluate_marker at 0x1535050>
>>>
NB Upgrading distribution packages (e.g., things installed by apt-get in this example) using pip can often lead to sadness and heartache, and you are much better off if you are able to upgrade the underlying environment to one where such workarounds are not necessary. Alternatively, running your code from a Python virtual environment (so that your upgraded packages do not override system packages) is also a technically better solution.
I have tried dulwich, and GitPython - neither of which seem mature. Now I am trying to install libgit2/pygit2. I have successfully installed them into the host packages environment, but now I need to get them installed in the virtualenv of the app I am building.
Python 2.7.3 (default, Sep 26 2013, 20:03:06)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pygit2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/pygit2/__init__.py", line 32, in <module>
import _pygit2
ImportError: libgit2.so.0: cannot open shared object file: No such file or directory
>>>
As you can see the current issue seems to be that the globally installed libgit2.so.0 cannot be found from within the virtualenv. Fair enough, that is what virtualenv is about after all,, sandboxing. So how do I go about linking/symlinking/building a version/copy of the libgit2.so for the virtualenv ?
I found the following script in a Gist that - when run while logged into a virtualenv -- will install the current libgit2/pygit2 together there in the venv.
https://gist.github.com/olivier-m/5755638
One note of caution, update the version numbers for both libraries to the same most recent version ( 0.20.0 at this time of writing ).