I am working on a Windows system with limited download access. My workaround is to download an app into a local folder. Then in the virtualenv I use pip install -e download folder to install. For instance, pip install -e c:\django\test_virtualenv\South-0.8.4 and South is installed in the virtual environment.
In the virtualenv I am trying to install django-user-accounts, using the same method. Unfortunately, this app has two dependencies listed in setup.py, namely, "django-appconf>=0.6", "pytz>=2013.9". You can see these in the install_requires section in the setup.py file.
I was unable to redirect to a local folder such as c:\download\django-appconf
This lead to the question whether arguments were available when using install_required (part of setuptools). install_requires in setup.py depending on installed Python version points out install_requires takes just a list.
In http://pythonhosted.org/setuptools/setuptools.html#declaring-dependencies the text talks about "limited network access" and "manually download all the eggs to a single directory". This doc also suggests using easy_install with the -f option.
I have a neat system operational at the moment using pip and I am asking if anyone has found a way to redirect the downloading of dependencies from a local folder, no access to network resources for download.
Tommy.
Related
I'm on windows and want to use python and a few packages, however, due to our policies it is not possible to have admin rights and any fetch requests done by pip (or any other package manager I'd assume) are being blocked.
So far I've downloaded Python (3.9.6) and "installed" it by manually unzipping and setting the PATH env variable.
I know that packages can be manually downloaded and installed as well using setup.py, however I also do not have setuptools, so is there any way to manually install that?
pip doesn't require admin rights just use it with --user.If you want to avoid using pip here the steps
if the folder has setup.py in it
Download the packages and uncompress if needed open folder
holding shift click open with cmd or powershell
run python setup.py install --user or python3 setup.py install --user
if not present setup.py check the install documentation provided by module.
I would like to figure out a "fool-proof" installation instruction to put in the README of a Python project, call it footools, such that other people in our group can install the newest SVN version of it on their laptops and their server accounts.
The problem is getting the user-installed libs to be used by Python when they call the scripts installed by pip. E.g., we're using a server that has an old version of footools in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/.
If I do python2.7 setup.py install --user and run the main entry script, it uses the files in /Users/unhammer/Library/Python/2.7/lib/python/site-packages/. This is what I want, but setup.py alone doesn't install dependencies.
If I (revert the installation and) instead do pip-2.7 install --user . and run the main entry script, it uses the old files in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/ – that's not what I want.
If I (revert the installation and) instead do pip-2.7 install --user -e . and run the main entry script, it uses the files in . – that's not what I want, the user should be able to remove the source dir (and be able to svn up without that affecting their install).
I could use (and recommend other people to use) python2.7 setup.py install --user – but then they have to first do
pip-2.7 install -U --user -r requirements.txt -e .
pip-2.7 uninstall -y footools
in order to get the dependencies installed (since pip has no install --only-deps option). That's rather verbose though.
What is setup.py doing that pip is not doing here?
(Edited to make it clear I'm looking for simpler+safer installation instructions.)
Install virtualenvwrapper. I allows setting up separate python environments to alleviate any conflicts you might be having. Here is a tutorial for installing and using virtualenv.
Related:
https://virtualenvwrapper.readthedocs.org/en/latest/
Console scripts generated by pip in the process of installation should use user installed versions of libraries as according to PEP 370:
The user site directory is added before the system site directories
but after Python's search paths and PYTHONPATH. This setup allows the
user to install a different version of a package than the system
administrator but it prevents the user from accidently overwriting a
stdlib module. Stdlib modules can still be overwritten with
PYTHONPATH.
Sidenote
Setuptools use hack by inserting code in easy_install.pth file which is placed in site-packages directory. This code makes packages installed with setuptools come before other packages in sys.path so they shadow other packages with the same name. This is referred to as sys.path modification in the table comparing setuptools and pip. This is the reason console scripts use user installed libraries when you install with setup.py install instead of using pip.
Taking all of the above into account the reason for what you observe might be caused by:
PYTHONPATH pointing to directories with system-wide installed libraries
Having system-wide libraries installed using sudo python.py install (...)
Having OS influence sys.path construction in some way
In the first case either clearing PYTHONPATH or adding path to user installed library to the beginning of PYTHONPATH should help.
In the second case uninstalling system-wide libraries and installing them with distro package manager instead might help (please note that you never should use sudo with pip or setup.py to install Python packages).
In the third case it's necessary to find out how does OS influence sys.path construction and if there's some way of placing user installed libraries before system ones.
You might be interested in reading issue pip list reports wrong version of package installed both in system site and user site where I asked basically the same question as you:
Does it mean that having system wide Python packages installed with easy_install thus having them use sys.path manipulation breaks scripts from user bin directory? If so is there any workaround?
Last resort solution would be to manually place directory/directories with user installed libraries in the beginning of sys.path from your scripts before importing these libraries.
Having said that if your users do not need direct access to source code I would propose packaging your app together with all dependencies using tool like pex or Platter into self-contained bundle.
I'm trying to use a python app on a server for the first time. I started by adding setuptools as root:
[root#server mydirectory]# yum install python-setuptools
Cool. Then I try setup.py:
[user#server mydirectory]$ python setup.py install
running install
error: can't create or remove files in install directory
The following error occurred while trying to add or remove files in the
installation directory:
[Errno 13] Permission denied: '/usr/lib/python2.4/site-packages/test-easy-install-25752.write-test'
This directory /usr/lib/python2.4/site-packages is owned by root, so that makes sense.
My question is, should I chmod the site-packages directory, or should I be running setup.py as root?
The traditional way to install stuff system-wide as a non-root user is to use sudo. Which is why you see things like this all over the Python docs:
sudo python setup.py install
Some people prefer to instead make the site-packages group-writable by some "dev" group so you don't need to do this. (This is effectively what the Mac package manager Homebrew does.)
Alternatively, you can install into per-user site packages. Not every project can do this, but anything based on modern setuptools should be able to do so.
And, while we're at it, if you're installing stuff based on modern setuptools, it's probably better to pip install . instead of python setup.py install anyway. That will, among other benefits, create egg-info files so the package can be detected as a dependency, uninstalled, etc.
See the Python Packaging User Guide for more information.
Finally, you may want to consider using a virtual environment. With Python 3.3+, this is built in as venv, although it doesn't have its own pip until 3.4. With earlier versions of Python, you can install virtualenv off PyPI.
Many hosted server environments for Python (2.x or 3.x) come with virtualenv pre-installed. If not, installing it system-wide will of course require you to be root… but after that, you will be able to install (most) other packages into per-project virtual environments instead of system-wide.
Installing packages with pip/easy_install and running directly setup.py files require root privileges because they read/write in those restricted folders.
Usually hosts like www.openshift.com support a virtualenv for you so you just activate it and you have your own per-user environment. Affecting the global site-packages is usually forbidden since it may be a shared host.
In my experience, in a local ubuntu-installed laptop, I have two options:
Run installs as sudo
Run installs in a virtualenv
perhaps your host, if shared, supports virtualenv. Try asking them if it doesn't support it.
I am running a local pypi server. I can install packages from this server by either specifying it with the -i option of the pip command or by setting the PIP_INDEX_URL environment variable. When I install a package that has prerequisites, setup.py has historically honored the PIP_INDEX_URL environment variable, pulling the additional packages from my local server.
However, on a couple of systems that have been recently installed, it is behaving differently. Running, for instance, python setup.py develop fails because it tries to install prerequisites packages from pypi.python.org.
I have updated all of the related python packages (python, distribute, virtualenv, pip, etc...) on all the systems I'm testing on and continue to see this discrepancy. On my "original" system, setup.py downloads prerequisites from the pypi server specified in my PIP_INDEX_URL environment variable. On the newer systems, I can't seem to make it honor this variable.
What am I missing?
Create setup.cfg in the same folder as your setup.py with following content:
[easy_install]
allow_hosts = *.myintranet.example.com
From: http://pythonhosted.org/setuptools/easy_install.html#restricting-downloads-with-allow-hosts
You can use the --allow-hosts (-H) option to restrict what domains EasyInstall will look for links and downloads on.
--allow-hosts=None prevents downloading altogether.
I ran into the same issue. Fundamentally, setup.py is using setuptools which leverages easy_install, not pip. Thus, it ignores any pip-related environment variables you set.
Rather than use python setup.py develop you can run pip (from the top of the package) pip install -e . to produce the same effect.
In order to install matplotlib to a non default location, I change the file setup.cfg, setting the variable basedirlist.
I do
python setup.py build
and then
python setup.py install
but the last fail because:
copying build/lib.linux-x86_64-2.6/mpl_toolkits/axes_grid1/colorbar.py -> /opt/python/2.6.4/lib/python2.6/site-packages/mpl_toolkits/axes_grid1
error: could not delete '/opt/python/2.6.4/lib/python2.6/site-packages/mpl_toolkits/axes_grid1/colorbar.py': Read-only file system
I am not root, so how can I install matplotlib? is there any other variable I have to set?
Try with an unmodified version of setup.cfg and run python setup.py install --help There are several options for controlling where the files are installed, the important part of the help message being:
Options for 'install' command:
--prefix installation prefix
--exec-prefix (Unix only) prefix for platform-specific files
--home (Unix only) home directory to install under
--user install in user site-package
'/home/yannpaul/.local/lib/python2.6/site-packages'
--install-base base installation directory (instead of --prefix or --
home)
Read over those options and choose which one suits you best.
I recommend, however, to use vertualenv. This sets up, in a directory of your choice, a custom library location and copy of python. All the other libraries (installed by a system admin for example) are available until you install your own copy of the library in this virtualenv.
Virtualenv is also a good option if you want to play around with the development version of a library, matplotlib for example. Setup a virtualenv for these development libraries, then use the python "executable" associated with that virtualenv to get access to the development version of the library.
Check out What's the proper way to install pip, virtualenv, and distribute for Python? to get setup with virtualenv.