Related
Is it possible? When installing pip, install the python packages inside my $HOME folder. (for example, I want to install mercurial, using pip, but inside $HOME instead of /usr/local)
I'm with a mac machine and just thought about this possibility, instead of "polluting" my /usr/local, I would use my $HOME instead.
PEP370 is exactly about this. Is just creating a ˜/.local and do a pip install package enough to make these packages to be installed only at my $HOME folder?
While you can use a virtualenv, you don't need to. The trick is passing the PEP370 --user argument to the setup.py script. With the latest version of pip, one way to do it is:
pip install --user mercurial
This should result in the hg script being installed in $HOME/.local/bin/hg and the rest of the hg package in $HOME/.local/lib/pythonx.y/site-packages/.
Note, that the above is true for Python 2.6. There has been a bit of controversy among the Python core developers about what is the appropriate directory location on Mac OS X for PEP370-style user installations. In Python 2.7 and 3.2, the location on Mac OS X was changed from $HOME/.local to $HOME/Library/Python. This might change in a future release. But, for now, on 2.7 (and 3.2, if hg were supported on Python 3), the above locations will be $HOME/Library/Python/x.y/bin/hg and $HOME/Library/Python/x.y/lib/python/site-packages.
You can specify the -t option (--target) to specify the destination directory. See pip install --help for detailed information. This is the command you need:
pip install -t path_to_your_home package-name
for example, for installing say mxnet, in my $HOME directory, I type:
pip install -t /home/foivos/ mxnet
I would use virtualenv at your HOME directory.
$ sudo easy_install -U virtualenv
$ cd ~
$ virtualenv .
$ bin/pip ...
You could then also alter ~/.(login|profile|bash_profile), whichever is right for your shell to add ~/bin to your PATH and then that pip|python|easy_install would be the one used by default.
Short answer to your two questions grabbed from the other answers
One
Yes
it is possible installing pip packages to $HOME instead of /usr/local/lib/, but
Two
mkdir ˜/.local # then
pip install package
is not enough.
You need
pip install package --user
and the packages get installed to
/home/user/.local/lib/python3.x/site-packages
Exception when you do not need --user
if you are not root user
On Debian together with Wsl - Windows Subsystem Linux no notice
On Ubuntu 20.04 notice default to user installation because normal site-packages not writeable - default config is /usr/lib not writeable for other users except root
I was wondering how the above "yum install package" & "python setup.py install" are used differently in CentOS? I used yum install ... all the time. However, when I try to do python setup.py install, I always get: this setup.py file couldn't be found even though its path shows up under echo $PATH, unless I try to use it in its current directory or use the absolute path.
When you type python setup.py install, your shell will check your $PATH for the python command, and run that. Then, python will be examining its arguments, which are setup.py install. It knows that it can be given the name of a script, so it looks for the file called setup.py so it can be run. Python doesn't use your $PATH to find scripts, though, so it should be a real path to a file. If you just give it the name setup.py it will only look in your current directory.
The source directory for a python module should not, ideally, be in your $PATH.
yum install is a command that will go to a package repository, download all the files needed to install something, and then put them in the right place. yum (and equivalents on other distributions, like apt for Debian systems) will also fetch and install any other packages you need, including any that aren't python modules.
Python has a package manager, too. You may also find using pip install modulename or pip install --user modulename (if you don't have administrative rights) easier than downloading and installing the module by hand. You can often get more recent versions of modules this way, as the ones provided by an operating system (through yum) tend to be older, more stable versions. Sometimes the module is not available through yum at all. pip can't install any extra packages that aren't python modules, though.
If you don't have pip already (it comes with Python3, but might need installing separately for Python2, depending on how it was set up), then you can install it by following the instructions here: https://pip.pypa.io/en/stable/installing/
Is it possible? When installing pip, install the python packages inside my $HOME folder. (for example, I want to install mercurial, using pip, but inside $HOME instead of /usr/local)
I'm with a mac machine and just thought about this possibility, instead of "polluting" my /usr/local, I would use my $HOME instead.
PEP370 is exactly about this. Is just creating a ˜/.local and do a pip install package enough to make these packages to be installed only at my $HOME folder?
While you can use a virtualenv, you don't need to. The trick is passing the PEP370 --user argument to the setup.py script. With the latest version of pip, one way to do it is:
pip install --user mercurial
This should result in the hg script being installed in $HOME/.local/bin/hg and the rest of the hg package in $HOME/.local/lib/pythonx.y/site-packages/.
Note, that the above is true for Python 2.6. There has been a bit of controversy among the Python core developers about what is the appropriate directory location on Mac OS X for PEP370-style user installations. In Python 2.7 and 3.2, the location on Mac OS X was changed from $HOME/.local to $HOME/Library/Python. This might change in a future release. But, for now, on 2.7 (and 3.2, if hg were supported on Python 3), the above locations will be $HOME/Library/Python/x.y/bin/hg and $HOME/Library/Python/x.y/lib/python/site-packages.
You can specify the -t option (--target) to specify the destination directory. See pip install --help for detailed information. This is the command you need:
pip install -t path_to_your_home package-name
for example, for installing say mxnet, in my $HOME directory, I type:
pip install -t /home/foivos/ mxnet
I would use virtualenv at your HOME directory.
$ sudo easy_install -U virtualenv
$ cd ~
$ virtualenv .
$ bin/pip ...
You could then also alter ~/.(login|profile|bash_profile), whichever is right for your shell to add ~/bin to your PATH and then that pip|python|easy_install would be the one used by default.
Short answer to your two questions grabbed from the other answers
One
Yes
it is possible installing pip packages to $HOME instead of /usr/local/lib/, but
Two
mkdir ˜/.local # then
pip install package
is not enough.
You need
pip install package --user
and the packages get installed to
/home/user/.local/lib/python3.x/site-packages
Exception when you do not need --user
if you are not root user
On Debian together with Wsl - Windows Subsystem Linux no notice
On Ubuntu 20.04 notice default to user installation because normal site-packages not writeable - default config is /usr/lib not writeable for other users except root
i have installed many python modules plugins and libraries in my centos system.
Now i don't want to install each thing again on separate computers.
Is there any way i can make the package like rpm or any other thing so that when i install in in new location all the modules, dependencies also gets installed and everytime i install new thing i can update the install package
If you have installed the packages with pip (an improved easy_install), you can just do pip freeze > my-reqs.txt to get a list and versions of the installed packages. There is also some option to install using the reqs file, which I can not remember right now.
Pip is meant to companion virtualenv, which can be used to handle per project requirements of dependent packages.
If you are not using Pip, then you should check what packages you have in your global and/or user site-packages directories.
The solutions below are from:
How do I find the location of my Python site-packages directory?
Global site packages:
python -m site
or more concisely:
python -c "import site; print(site.getsitepackages())"
User site packages:
python -m site --user-site
The latter do not however show the site-packages of the current virtual environment – use distutils.sysconfig.get_python_lib() for that:
python -c "from distutils.sysconfig import get_python_lib;
print get_python_lib()"
Note!
You can not just copy over the packages that have C++-extensions or other compiled binaries in them. These have to be reinstalled on other machines.
I know the obvious answer is to use virtualenv and virtualenvwrapper, but for various reasons I can't/don't want to do that.
So how do I modify the command
pip install package_name
to make pip install the package somewhere other than the default site-packages?
The --target switch is the thing you're looking for:
pip install --target=d:\somewhere\other\than\the\default package_name
But you still need to add d:\somewhere\other\than\the\default to PYTHONPATH to actually use them from that location.
-t, --target <dir>
Install packages into <dir>. By default this will not replace existing files/folders in <dir>.
Use --upgrade to replace existing packages in <dir> with new versions.
Upgrade pip if target switch is not available:
On Linux or OS X:
pip install -U pip
On Windows (this works around an issue):
python -m pip install -U pip
Use:
pip install --install-option="--prefix=$PREFIX_PATH" package_name
You might also want to use --ignore-installed to force all dependencies to be reinstalled using this new prefix. You can use --install-option to multiple times to add any of the options you can use with python setup.py install (--prefix is probably what you want, but there are a bunch more options you could use).
Instead of the --target or --install-options options, I have found that setting the PYTHONUSERBASE environment variable works well (from discussion on a bug regarding this very thing):
PYTHONUSERBASE=/path/to/install/to pip install --user
(Or set the PYTHONUSERBASE directory in your environment before running the command, using export PYTHONUSERBASE=/path/to/install/to)
This uses the very useful --user option but tells it to make the bin, lib, share and other directories you'd expect under a custom prefix rather than $HOME/.local.
Then you can add this to your PATH, PYTHONPATH and other variables as you would a normal installation directory.
Note that you may also need to specify the --upgrade and --ignore-installed options if any packages upon which this depends require newer versions to be installed in the PYTHONUSERBASE directory, to override the system-provided versions.
A full example
PYTHONUSERBASE=/opt/mysterypackage-1.0/python-deps pip install --user --upgrade numpy scipy
..to install the scipy and numpy package most recent versions into a directory which you can then include in your PYTHONPATH like so (using bash and for python 2.6 on CentOS 6 for this example):
export PYTHONPATH=/opt/mysterypackage-1.0/python-deps/lib64/python2.6/site-packages:$PYTHONPATH
export PATH=/opt/mysterypackage-1.0/python-deps/bin:$PATH
Using virtualenv is still a better and neater solution!
To pip install a library exactly where I wanted it, I navigated to the location I wanted the directory with the terminal then used
pip install mylibraryName -t .
the logic of which I took from this page: https://cloud.google.com/appengine/docs/python/googlecloudstorageclient/download
Installing a Python package often only includes some pure Python files. If the package includes data, scripts and or executables, these are installed in different directories from the pure Python files.
Assuming your package has no data/scripts/executables, and that you want your Python files to go into /python/packages/package_name (and not some subdirectory a few levels below /python/packages as when using --prefix), you can use the one time command:
pip install --install-option="--install-purelib=/python/packages" package_name
If you want all (or most) of your packages to go there, you can edit your ~/.pip/pip.conf to include:
[install]
install-option=--install-purelib=/python/packages
That way you can't forget about having to specify it again and again.
Any excecutables/data/scripts included in the package will still go to their default places unless you specify addition install options (--prefix/--install-data/--install-scripts, etc., for details look at the custom installation options).
Tested these options with python3.5 and pip 9.0.3:
pip install --target /myfolder [packages]
Installs ALL packages including dependencies under /myfolder. Does not take into account that dependent packages are already installed elsewhere in Python. You will find packages from /myfolder/[package_name]. In case you have multiple Python versions, this doesn't take that into account (no Python version in package folder name).
pip install --prefix /myfolder [packages]
Checks if dependencies are already installed. Will install packages into /myfolder/lib/python3.5/site-packages/[packages]
pip install --root /myfolder [packages]
Checks dependencies like --prefix but install location will be /myfolder/usr/local/lib/python3.5/site-packages/[package_name].
pip install --user [packages]
Will install packages into $HOME:
/home/[USER]/.local/lib/python3.5/site-packages
Python searches automatically from this .local path so you don't need to put it to your PYTHONPATH.
=> In most of the cases --user is the best option to use.
In case home folder can't be used because of some reason then --prefix.
pip3 install "package_name" -t "target_dir"
source - https://pip.pypa.io/en/stable/reference/pip_install/
-t switch = target
Nobody seems to have mentioned the -t option but that the easiest:
pip install -t <direct directory> <package>
pip install packageName -t pathOfDirectory
or
pip install packageName --target pathOfDirectorty
Just add one point to #Ian Bicking's answer:
Using the --user option to specify the installed directory also work if one wants to install some Python package into one's home directory (without sudo user right) on remote server.
E.g.,
pip install --user python-memcached
The command will install the package into one of the directories that listed in your PYTHONPATH.
Newer versions of pip (8 or later) can directly use the --prefix option:
pip install --prefix=$PREFIX_PATH package_name
where $PREFIX_PATH is the installation prefix where lib, bin and other top-level folders are placed.
To add to the already good advice, as I had an issue installing IPython when I didn't have write permissions to /usr/local.
pip uses distutils to do its install and this thread discusses how that can cause a problem as it relies on the sys.prefix setting.
My issue happened when the IPython install tried to write to '/usr/local/share/man/man1' with Permission denied. As the install failed it didn't seem to write the IPython files in the bin directory.
Using "--user" worked and the files were written to ~/.local. Adding ~/.local/bin to the $PATH meant I could use "ipython" from there.
However I'm trying to install this for a number of users and had been given write permission to the /usr/local/lib/python2.7 directory. I created a "bin" directory under there and set directives for distutils:
vim ~/.pydistutils.cfg
[install]
install-data=/usr/local/lib/python2.7
install-scripts=/usr/local/lib/python2.7/bin
then (-I is used to force the install despite previous failures/.local install):
pip install -I ipython
Then I added /usr/local/lib/python2.7/bin to $PATH.
I thought I'd include this in case anyone else has similar issues on a machine they don't have sudo access to.
If you are using brew with python, unfortunately, pip/pip3 ships with very limited options. You do not have --install-option, --target, --user options as mentioned above.
Note on pip install --user
The normal pip install --user is disabled for brewed Python. This is because of a bug in distutils, because Homebrew writes a distutils.cfg which sets the package prefix.
A possible workaround (which puts executable scripts in ~/Library/Python/./bin) is:
python -m pip install --user --install-option="--prefix=" <package-name>
You might find this line very cumbersome. I suggest use pyenv for management.
If you are using
brew upgrade python python3
Ironically you are actually downgrade pip functionality.
(I post this answer, simply because pip in my mac osx does not have --target option, and I have spent hours fixing it)
With pip v1.5.6 on Python v2.7.3 (GNU/Linux), option --root allows to specify a global installation prefix, (apparently) irrespective of specific package's options. Try f.i.,
$ pip install --root=/alternative/prefix/path package_name
I suggest to follow the documentation and create ~/.pip/pip.conf file. Note in the documentation there are missing specified header directory, which leads to following error:
error: install-base or install-platbase supplied, but installation scheme is incomplete
The full working content of conf file is:
[install]
install-base=$HOME
install-purelib=python/lib
install-platlib=python/lib.$PLAT
install-scripts=python/scripts
install-headers=python/include
install-data=python/data
Unfortunatelly I can install, but when try to uninstall pip tells me there is no such package for uninstallation process.... so something is still wrong but the package goes to its predefined location.
pip install /path/to/package/
is now possible.
The difference with this and using the -e or --editable flag is that -e links to where the package is saved (i.e. your downloads folder), rather than installing it into your python path.
This means if you delete/move the package to another folder, you won't be able to use it.
system` option, that will install pip package-bins to /usr/local/bin thats accessible to all users. Installing without this option may not work for all users as things go to user specific dir like $HOME/.local/bin and then it is user specific install which has to be repeated for all users, also there can be path issues if not set for users, then bins won't work. So if you are looking for all users - yu need to have sudo access:
sudo su -
python3 -m pip install --system <module>
logout
log back in
which <module-bin> --> it should be installed on /usr/local/bin/
Sometimes it works only works with Cache argument
-m pip install -U pip --target=C:\xxx\python\lib\site-packages Pillow --cache-dir C:\tmp