I have a long requirements.txt and my network environment is not stable. Using pip install -r pip_requirements.txt will never make a successful install. Because if the connection is lost during the install and after I restart the install process, pip will download these packages from the beginning again. It would not use these packages that have been downloaded.
How can I make pip install package one by one rather than install after it have successfully downloaded all of the packages?
Try the option
pip install --download-cache="/folder/"
It will allow you to save the files to a local folder for later use.
You can also use
pip install --download="/folder/"
to just download and not install.
Related
I uploaded my package to testpypi, and installed it via:
pip install -i https://test.pypi.org/simple/ myporj==0.1.6
However it refuse to install it by saying:
Requirement already satisfied: myproj==0.1.6 in ./projs/myproj (0.1.6)
I guess I may add the project in editable mode:
pip install --editable .
However, I know want to disable it. I tried:
python setup.py develop --uninstall
But it has no effect.
It may be worth creating a separate env (Virtual Environments) for the installation.
Here are some articles on this subject:
https://docs.python.org/3/tutorial/venv.html
https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment
Or does it need to be installed in the same place?
You can try to find your package pip search myporj or pip list for show all packages.
And uninstall it later pip uninstall myporj (it may require the right of sudo in linux) then install again.
Maybe you may need --no-cache-dir option to ignore the cache during installation. Here is more details: https://pip.pypa.io/en/stable/reference/pip_install/#caching
Is there a way to install packages with pip to avoid the need to repeatedly delete files like:
pip can't proceed with requirement 'Flask-Restless==0.13.1 (from -r requirements.txt (line 2))' due to a pre-existing build directory.
location: /private/var/folders/0k/t9lwmd2j1212pxydpr6l596h0000gq/T/pip_build_jacob/Flask-Restless
This is likely due to a previous installation that failed.
pip is being responsible and not assuming it can delete this.
I'm on round 4 of doing this and have no idea how long it may take to get through.
Looking at pip --help isn't helpful and man pip returns nothing.
As it has already been mentioned it's better to use virtualenv in order to avoid python package chaos on your system and install the python packages only for particular projects.
However, in your particular case you can try the following in the terminal:
pip uninstall flask-restless
Then try to run the installation again:
pip install -r requirements.txt
The options to consider during installation:
--force-reinstall
--ignore-installed
--no-deps
Add these options to the end of pip install -r requirements.txt to play with them and see if they can help.
Using
--force-reinstall
may solve your issue.
I would also recommend considering using a virtualenv for each project you are working on.
https://virtualenv.pypa.io/en/stable/
You can then activate the virtual environment for that project and pip
pip install -r requrements.txt
will install dependencies for that project in the virtual environment instead of globally. This will reduce the odds of having weird conflicts like you are having and if you do have an issue you can blow away the virtualenv and reinstall just the dependencies for that project without borking your global packages.
I'm installing the openbabel package, and it can automatically generate the necessary Python libraries during compilation. This saves a good chunk of time, since installing from source via pip takes a few minutes, and that time can be rolled into the initial compilation.
I've listed it as a requirement in my requirements.txt file, but when I go to install (pip install -r requirements.txt), it attempts to reinstall the openbabel Python library. When I run pip show or pip list, openbabel doesn't show up.
Is there a way to manually mark a package as installed so pip thinks it's installed, even if it can't find the package? Or is there a file I can create that pip will use that will tell it openbabel is installed?
Create an empty .egg-info file in your site-packages directory.
For example, on my machine I did touch /usr/lib64/python3.6/site-packages/GLWindow-1.8.0-py3.6.egg-info to trick pip3 into thinking that I've installed GLWindow.
I'm trying to install Django package in a virtualenv. I'm on a new computer (OSX 10.8.2). I installed virtualenv via easy_install. With the virtualenv activated, I ran:
(pyenv)$ pip install Django
Downloading/unpacking Django
Downloading Django-1.5.1.tar.gz (8.0MB): 2.0MB downloaded
Hash of the package https://pypi.python.org/packages/source/D/Django/Django-1.5.1.tar.gz#md5=7465f6383264ba167a9a031d6b058bff (from https://pypi.python.org/simple/Django/) (<md5 HASH object # 0x108453df0>) doesn't match the expected hash 7465f6383264ba167a9a031d6b058bff!
Bad md5 hash for package https://pypi.python.org/packages/source/D/Django/Django-1.5.1.tar.gz#md5=7465f6383264ba167a9a031d6b058bff (from https://pypi.python.org/simple/Django/)
This happens even if I delete virtualenv and start over. I've tried again repeatedly over the past few hours, it always happens. Any suggestions?
I have the same problem when I try sudo pip install Pillow, and I try sudo pip install --no-cache-dir Pillow, it works for me.
If it's just this package that you can't get to install, you could download the tarball manually, and then use pip to install it from that file. The Django download site has checksums that you can validate manually as well. I don't use osx, but probably something like this would help:
cd /tmp
wget https://pypi.python.org/packages/source/D/Django/Django-1.5.1.tar.gz
md5sum Django-1.5.1.tar.gz
pip install Django-1.5.1.tar.gz
For me below command works
pip install django --no-cache-dir
I now had this issue several times.
Like others mentioned before me, pip install [module] --no-cache-dir
helps most of the time.
But sometimes, you got some dependencies to install first and it fails installing one of these (md5 validation failed).
Just had this problem myself. In this case, installing this dependency alone like pip install dependency
worked and after that I was able to install the first module.
Also pip install -vvv is nice for more info gathering on general problem solving
If I want to use the pip command to download a package (and its dependencies), but keep all of the zipped files that get downloaded (say, django-socialregistration.tar.gz) - is there a way to do that?
I've tried various command-line options, but it always seems to unpack and delete the zipfile - or it gets the zipfile, but only for the original package, not the dependencies.
pip install --download is deprecated. Starting from version 8.0.0 you should use pip download command:
pip download <package-name>
The --download-cache option should do what you want:
pip install --download-cache="/pth/to/downloaded/files" package
However, when I tested this, the main package downloaded, saved and installed ok, but the the dependencies were saved with their full url path as the name - a bit annoying, but all the tar.gz files were there.
The --download option downloads the main package and its dependencies and does not install any of them. (Note that prior to version 1.1 the --download option did not download dependencies.)
pip install package --download="/pth/to/downloaded/files"
The pip documentation outlines using --download for fast & local installs.
I always do this to download the packages:
pip install --download /path/to/download/to_packagename
OR
pip install --download=/path/to/packages/downloaded -r requirements.txt
And when I want to install all of those libraries I just downloaded, I do this:
pip install --no-index --find-links="/path/to/downloaded/dependencies" packagename
OR
pip install --no-index --find-links="/path/to/downloaded/packages" -r requirements.txt
Update
Also, to get all the packages installed on one system, you can export them all to requirement.txt that will be used to intall them on another system, we do this:
pip freeze > requirement.txt
Then, the requirement.txt can be used as above for download, or do this to install them from requirement.txt:
pip install -r requirement.txt
REFERENCE: pip installer
pip wheel is another option you should consider:
pip wheel mypackage -w .\outputdir
It will download packages and their dependencies to a directory (current working directory by default), but it performs the additional step of converting any source packages to wheels.
It conveniently supports requirements files:
pip wheel -r requirements.txt -w .\outputdir
Add the --no-deps argument if you only want the specifically requested packages:
pip wheel mypackage -w .\outputdir --no-deps
Use pip download <package1 package2 package n> to download all the packages including dependencies
Use pip install --no-index --find-links . <package1 package2 package n> to install all the packages including dependencies.
It gets all the files from CWD.
It will not download anything
In version 7.1.2 pip downloads the wheel of a package (if available) with the following:
pip install package -d /path/to/downloaded/file
The following downloads a source distribution:
pip install package -d /path/to/downloaded/file --no-binary :all:
These download the dependencies as well, if pip is aware of them (e.g., if pip show package lists them).
Update
As noted by Anton Khodak, pip download command is preferred since version 8. In the above examples this means that /path/to/downloaded/file needs to be given with option -d, so replacing install with download works.
installing python packages offline
For windows users:
To download into a file
open your cmd and folow this:
cd <*the file-path where you want to save it*>
pip download <*package name*>
the package and the dependencies will be downloaded in the current working directory.
To install from the current working directory:
set your folder where you downloaded as the cwd then follow these:
pip install <*the package name which is downloded as .whl*> --no-index --find-links <*the file locaation where the files are downloaded*>
this will search for dependencies in that location.
All the answers mentioned in this thread assume that the packages will be downloaded on the same OS configuration as the target OS where it has to be installed.
In my personal experience i was using windows as my work machine and had to download packages for linux environment and have seen people doing vice versa as well. I had done some extensive googling, and found sodim.dev.
All i had to do was upload requirements.txt file and select the environment configuration like OS and python version and it gives out a csv with download url, source code url etc
I guess in the backend this app spins up the OS VM as requested and installs that particular python version and then generates the report, because it does take about 15-20 minutes for 30-50 packages.
P.S.: I work in an offline environment, where security is of very high concern, and downloading packages are not that frequent. We whitelist source code and download urls for each individual requests and then after running some appsec tools, we approve/reject the source code to be downloaded.
I would prefer (RHEL) - pip download package==version --no-deps --no-binary=:all: